@samp{/etc/passwd}:
@c If you change this example, change the regression test in
-@c tests/language/data-io/get-data.at to match.
+@c tests/language/commands/get-data.at to match.
@example
GET DATA /TYPE=TXT /FILE='/etc/passwd' /DELIMITERS=':'
/VARIABLES=username A20
The following syntax can be used to read the used car data:
@c If you change this example, change the regression test in
-@c tests/language/data-io/get-data.at to match.
+@c tests/language/commands/get-data.at to match.
@example
GET DATA /TYPE=TXT /FILE='cars.data' /DELIMITERS=' ' /FIRSTCASE=2
/VARIABLES=model A8
The following syntax can be used to read the pet store data:
@c If you change this example, change the regression test in
-@c tests/language/data-io/get-data.at to match.
+@c tests/language/commands/get-data.at to match.
@example
GET DATA /TYPE=TXT /FILE='pets.data' /DELIMITERS=', ' /QUALIFIER='''"' /ESCAPE
/FIRSTCASE=3
The following syntax can be used to read the used car data:
@c If you change this example, change the regression test in
-@c tests/language/data-io/get-data.at to match.
+@c tests/language/commands/get-data.at to match.
@example
GET DATA /TYPE=TXT /FILE='cars.data' /ARRANGEMENT=FIXED /FIRSTCASE=2
/VARIABLES=model 0-7 A
results visible.
@c Keep these examples in sync with the test for !BLANKS in
-@c tests/language/control/define.at:
+@c tests/language/commands/define.at:
@example
!BLANKS(0) @expansion{} @r{empty}
!BLANKS(1) @expansion{} _
combining two (or more) tokens into a single one:
@c Keep these examples in sync with the test for !CONCAT in
-@c tests/language/control/define.at:
+@c tests/language/commands/define.at:
@example
!CONCAT(x, y) @expansion{} xy
!CONCAT('x', 'y') @expansion{} xy
suffix. For example:
@c Keep these examples in sync with the test for !CONCAT in
-@c tests/language/control/define.at:
+@c tests/language/commands/define.at:
@example
!CONCAT(x, 0) @expansion{} x0
!CONCAT(x, 0, y) @expansion{} x0y
than a single one. For example:
@c Keep these examples in sync with the test for !CONCAT in
-@c tests/language/control/define.at:
+@c tests/language/commands/define.at:
@example
!CONCAT(0, x) @expansion{} 0 x
!CONCAT(0, x, y) @expansion{} 0 xy
Expands to an empty character sequence.
@c Keep these examples in sync with the test for !NULL in
-@c tests/language/control/define.at:
+@c tests/language/commands/define.at:
@example
!NULL @expansion{} @r{empty}
!QUOTE(!NULL) @expansion{} ''
string, @code{!UNQUOTE} expands to the argument unchanged.
@c Keep these examples in sync with the test for !QUOTE and !UNQUOTE in
-@c tests/language/control/define.at:
+@c tests/language/commands/define.at:
@example
!QUOTE(123.0) @expansion{} '123.0'
!QUOTE( 123 ) @expansion{} '123'
#
## Process this file with automake to produce Makefile.in -*- makefile -*-
+include $(top_srcdir)/src/language/commands/automake.mk
+include $(top_srcdir)/src/language/expressions/automake.mk
include $(top_srcdir)/src/language/lexer/automake.mk
-include $(top_srcdir)/src/language/xforms/automake.mk
-include $(top_srcdir)/src/language/control/automake.mk
-include $(top_srcdir)/src/language/dictionary/automake.mk
include $(top_srcdir)/src/language/tests/automake.mk
-include $(top_srcdir)/src/language/utilities/automake.mk
-include $(top_srcdir)/src/language/stats/automake.mk
-include $(top_srcdir)/src/language/data-io/automake.mk
-include $(top_srcdir)/src/language/expressions/automake.mk
noinst_LTLIBRARIES += src/language/liblanguage.la
src/language/command.h \
src/language/command.def \
$(language_lexer_sources) \
- $(language_xforms_sources) \
- $(language_control_sources) \
- $(language_dictionary_sources) \
+ $(language_commands_sources) \
$(language_tests_sources) \
- $(language_utilities_sources) \
- $(language_stats_sources) \
- $(language_data_io_sources) \
$(language_expressions_sources)
-
nodist_src_language_liblanguage_la_SOURCES = \
- $(src_language_data_io_built_sources) \
- $(src_language_utilities_built_sources) \
- $(src_language_stats_built_sources) \
- $(language_tests_built_sources) \
$(expressions_built_sources)
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2008, 2009, 2010, 2011, 2012, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/aggregate.h"
+
+#include <stdlib.h>
+
+#include "data/any-writer.h"
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/file-handle-def.h"
+#include "data/format.h"
+#include "data/settings.h"
+#include "data/subcase.h"
+#include "data/sys-file-writer.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/sort-criteria.h"
+#include "libpspp/assertion.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+#include "math/moments.h"
+#include "math/percentiles.h"
+#include "math/sort.h"
+#include "math/statistic.h"
+
+#include "gl/c-strcase.h"
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+/* Argument for AGGREGATE function.
+
+ Only one of the members is used, so this could be a union, but it's simpler
+ to just have both. */
+struct agr_argument
+ {
+ double f; /* Numeric. */
+ struct substring s; /* String. */
+ };
+
+/* Specifies how to make an aggregate variable. */
+struct agr_var
+ {
+ /* Collected during parsing. */
+ const struct variable *src; /* Source variable. */
+ struct variable *dest; /* Target variable. */
+ enum agr_function function; /* Function. */
+ enum mv_class exclude; /* Classes of missing values to exclude. */
+ struct agr_argument arg[2]; /* Arguments. */
+
+ /* Accumulated during AGGREGATE execution. */
+ double dbl;
+ double W; /* Total non-missing weight. */
+ int int1;
+ char *string;
+ bool saw_missing;
+ struct moments1 *moments;
+
+ struct variable *subject;
+ struct variable *weight;
+ struct casewriter *writer;
+ };
+
+/* Attributes of aggregation functions. */
+const struct agr_func agr_func_tab[] =
+ {
+#define AGRF(ENUM, NAME, DESCRIPTION, SRC_VARS, N_ARGS, ALPHA_TYPE, W, D) \
+ [ENUM] = { NAME, DESCRIPTION, SRC_VARS, N_ARGS, ALPHA_TYPE, \
+ { .type = (W) > 0 ? FMT_F : -1, .w = W, .d = D } },
+AGGREGATE_FUNCTIONS
+#undef AGRF
+ {NULL, NULL, AGR_SV_NO, 0, -1, {-1, -1, -1}},
+ };
+
+/* Missing value types. */
+enum missing_treatment
+ {
+ ITEMWISE, /* Missing values item by item. */
+ COLUMNWISE /* Missing values column by column. */
+ };
+
+/* An entire AGGREGATE procedure. */
+struct agr_proc
+ {
+ /* Break variables. */
+ struct subcase sort; /* Sort criteria (break variables). */
+ const struct variable **break_vars; /* Break variables. */
+ size_t break_n_vars; /* Number of break variables. */
+
+ enum missing_treatment missing; /* How to treat missing values. */
+ struct agr_var *agr_vars; /* Aggregate variables. */
+ size_t n_agr_vars;
+ struct dictionary *dict; /* Aggregate dictionary. */
+ const struct dictionary *src_dict; /* Dict of the source */
+ int n_cases; /* Counts aggregated cases. */
+
+ bool add_variables; /* True iff the aggregated variables should
+ be appended to the existing dictionary */
+ };
+
+static void initialize_aggregate_info (struct agr_proc *);
+
+static void accumulate_aggregate_info (struct agr_proc *,
+ const struct ccase *);
+/* Prototypes. */
+static bool parse_aggregate_functions (struct lexer *, const struct dictionary *,
+ struct agr_proc *);
+static void agr_destroy (struct agr_proc *);
+static void dump_aggregate_info (const struct agr_proc *agr,
+ struct casewriter *output,
+ const struct ccase *break_case);
+\f
+/* Parsing. */
+
+/* Parses and executes the AGGREGATE procedure. */
+int
+cmd_aggregate (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct agr_proc agr = {
+ .missing = ITEMWISE,
+ .src_dict = dict,
+ };
+ struct file_handle *out_file = NULL;
+ struct casereader *input = NULL;
+ struct casewriter *output = NULL;
+
+ bool copy_documents = false;
+ bool presorted = false;
+ int addvariables_ofs = 0;
+
+ /* OUTFILE subcommand must be first. */
+ if (lex_match_phrase (lexer, "/OUTFILE") || lex_match_id (lexer, "OUTFILE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_match (lexer, T_ASTERISK))
+ {
+ out_file = fh_parse (lexer, FH_REF_FILE, dataset_session (ds));
+ if (out_file == NULL)
+ goto error;
+ }
+
+ if (!out_file && lex_match_id (lexer, "MODE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "ADDVARIABLES"))
+ {
+ addvariables_ofs = lex_ofs (lexer) - 1;
+ agr.add_variables = true;
+ presorted = true;
+ }
+ else if (lex_match_id (lexer, "REPLACE"))
+ agr.add_variables = false;
+ else
+ {
+ lex_error_expecting (lexer, "ADDVARIABLES", "REPLACE");
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ agr.add_variables = true;
+ presorted = true;
+ }
+
+ if (lex_match_phrase (lexer, "/MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_match_id (lexer, "COLUMNWISE"))
+ {
+ lex_error_expecting (lexer, "COLUMNWISE");
+ goto error;
+ }
+ agr.missing = COLUMNWISE;
+ }
+
+ int presorted_ofs = 0;
+ for (;;)
+ if (lex_match_phrase (lexer, "/DOCUMENT"))
+ copy_documents = true;
+ else if (lex_match_phrase (lexer, "/PRESORTED"))
+ {
+ presorted = true;
+ presorted_ofs = lex_ofs (lexer) - 1;
+ }
+ else
+ break;
+
+ if (agr.add_variables)
+ agr.dict = dict_clone (dict);
+ else
+ agr.dict = dict_create (dict_get_encoding (dict));
+
+ dict_set_label (agr.dict, dict_get_label (dict));
+ dict_set_documents (agr.dict, dict_get_documents (dict));
+
+ if (lex_match_phrase (lexer, "/BREAK"))
+ {
+ lex_match (lexer, T_EQUALS);
+ bool saw_direction;
+ int break_start = lex_ofs (lexer);
+ if (!parse_sort_criteria (lexer, dict, &agr.sort, &agr.break_vars,
+ &saw_direction))
+ goto error;
+ int break_end = lex_ofs (lexer) - 1;
+ agr.break_n_vars = subcase_get_n_fields (&agr.sort);
+
+ if (! agr.add_variables)
+ for (size_t i = 0; i < agr.break_n_vars; i++)
+ dict_clone_var_assert (agr.dict, agr.break_vars[i]);
+
+ if (presorted && saw_direction)
+ {
+ lex_ofs_msg (lexer, SW, break_start, break_end,
+ _("When the input data is presorted, specifying "
+ "sorting directions with (A) or (D) has no effect. "
+ "Output data will be sorted the same way as the "
+ "input data."));
+ if (presorted_ofs)
+ lex_ofs_msg (lexer, SN, presorted_ofs, presorted_ofs,
+ _("The PRESORTED subcommand state that the "
+ "input data is presorted."));
+ else if (addvariables_ofs)
+ lex_ofs_msg (lexer, SN, addvariables_ofs, addvariables_ofs,
+ _("ADDVARIABLES implies that the input data "
+ "is presorted."));
+ else
+ msg (SN, _("The input data must be presorted because the "
+ "OUTFILE subcommand is not specified."));
+ }
+ }
+
+ /* Read in the aggregate functions. */
+ if (!parse_aggregate_functions (lexer, dict, &agr))
+ goto error;
+
+ /* Delete documents. */
+ if (!copy_documents)
+ dict_clear_documents (agr.dict);
+
+ /* Cancel SPLIT FILE. */
+ dict_clear_split_vars (agr.dict);
+
+ /* Initialize. */
+ agr.n_cases = 0;
+
+ if (out_file == NULL)
+ {
+ /* The active dataset will be replaced by the aggregated data,
+ so TEMPORARY is moot. */
+ proc_cancel_temporary_transformations (ds);
+ proc_discard_output (ds);
+ output = autopaging_writer_create (dict_get_proto (agr.dict));
+ }
+ else
+ {
+ output = any_writer_open (out_file, agr.dict);
+ if (output == NULL)
+ goto error;
+ }
+
+ input = proc_open (ds);
+ if (!subcase_is_empty (&agr.sort) && !presorted)
+ {
+ input = sort_execute (input, &agr.sort);
+ subcase_clear (&agr.sort);
+ }
+
+ struct casegrouper *grouper;
+ struct casereader *group;
+ for (grouper = casegrouper_create_vars (input, agr.break_vars,
+ agr.break_n_vars);
+ casegrouper_get_next_group (grouper, &group);
+ casereader_destroy (group))
+ {
+ struct casereader *placeholder = NULL;
+ struct ccase *c = casereader_peek (group, 0);
+
+ if (c == NULL)
+ {
+ casereader_destroy (group);
+ continue;
+ }
+
+ initialize_aggregate_info (&agr);
+
+ if (agr.add_variables)
+ placeholder = casereader_clone (group);
+
+ {
+ struct ccase *cg;
+ for (; (cg = casereader_read (group)) != NULL; case_unref (cg))
+ accumulate_aggregate_info (&agr, cg);
+ }
+
+
+ if (agr.add_variables)
+ {
+ struct ccase *cg;
+ for (; (cg = casereader_read (placeholder)) != NULL; case_unref (cg))
+ dump_aggregate_info (&agr, output, cg);
+
+ casereader_destroy (placeholder);
+ }
+ else
+ {
+ dump_aggregate_info (&agr, output, c);
+ }
+ case_unref (c);
+ }
+ if (!casegrouper_destroy (grouper))
+ goto error;
+
+ bool ok = proc_commit (ds);
+ input = NULL;
+ if (!ok)
+ goto error;
+
+ if (out_file == NULL)
+ {
+ struct casereader *next_input = casewriter_make_reader (output);
+ if (next_input == NULL)
+ goto error;
+
+ dataset_set_dict (ds, agr.dict);
+ dataset_set_source (ds, next_input);
+ agr.dict = NULL;
+ }
+ else
+ {
+ ok = casewriter_destroy (output);
+ output = NULL;
+ if (!ok)
+ goto error;
+ }
+
+ agr_destroy (&agr);
+ fh_unref (out_file);
+ return CMD_SUCCESS;
+
+error:
+ if (input != NULL)
+ proc_commit (ds);
+ casewriter_destroy (output);
+ agr_destroy (&agr);
+ fh_unref (out_file);
+ return CMD_CASCADING_FAILURE;
+}
+
+static bool
+parse_agr_func_name (struct lexer *lexer, int *func_index,
+ enum mv_class *exclude)
+{
+ if (lex_token (lexer) != T_ID)
+ {
+ lex_error (lexer, _("Syntax error expecting aggregation function."));
+ return false;
+ }
+
+ struct substring name = lex_tokss (lexer);
+ *exclude = ss_chomp_byte (&name, '.') ? MV_SYSTEM : MV_ANY;
+
+ for (const struct agr_func *f = agr_func_tab; f->name; f++)
+ if (ss_equals_case (ss_cstr (f->name), name))
+ {
+ *func_index = f - agr_func_tab;
+ lex_get (lexer);
+ return true;
+ }
+ lex_error (lexer, _("Unknown aggregation function %s."), lex_tokcstr (lexer));
+ return false;
+}
+
+/* Parse all the aggregate functions. */
+static bool
+parse_aggregate_functions (struct lexer *lexer, const struct dictionary *dict,
+ struct agr_proc *agr)
+{
+ if (!lex_force_match (lexer, T_SLASH))
+ return false;
+
+ size_t starting_n_vars = dict_get_n_vars (dict);
+ size_t allocated_agr_vars = 0;
+
+ /* Parse everything. */
+ for (;;)
+ {
+ char **dest = NULL;
+ char **dest_label = NULL;
+ size_t n_vars = 0;
+
+ struct agr_argument arg[2] = { { .f = 0 }, { .f = 0 } };
+
+ const struct variable **src = NULL;
+
+ /* Parse the list of target variables. */
+ int dst_start_ofs = lex_ofs (lexer);
+ while (!lex_match (lexer, T_EQUALS))
+ {
+ size_t n_vars_prev = n_vars;
+
+ if (!parse_DATA_LIST_vars (lexer, dict, &dest, &n_vars,
+ (PV_APPEND | PV_SINGLE | PV_NO_SCRATCH
+ | PV_NO_DUPLICATE)))
+ goto error;
+
+ /* Assign empty labels. */
+ dest_label = xnrealloc (dest_label, n_vars, sizeof *dest_label);
+ for (size_t j = n_vars_prev; j < n_vars; j++)
+ dest_label[j] = NULL;
+
+ if (lex_is_string (lexer))
+ {
+ dest_label[n_vars - 1] = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ }
+ int dst_end_ofs = lex_ofs (lexer) - 2;
+
+ /* Get the name of the aggregation function. */
+ int func_index;
+ enum mv_class exclude;
+ if (!parse_agr_func_name (lexer, &func_index, &exclude))
+ goto error;
+ const struct agr_func *function = &agr_func_tab[func_index];
+
+ /* Check for leading lparen. */
+ if (!lex_match (lexer, T_LPAREN))
+ {
+ if (function->src_vars == AGR_SV_YES)
+ {
+ bool ok UNUSED = lex_force_match (lexer, T_LPAREN);
+ goto error;
+ }
+ }
+ else
+ {
+ /* Parse list of source variables. */
+ int pv_opts = PV_NO_SCRATCH;
+ if (func_index == AGRF_SUM || func_index == AGRF_MEAN
+ || func_index == AGRF_MEDIAN || func_index == AGRF_SD)
+ pv_opts |= PV_NUMERIC;
+ else if (function->n_args)
+ pv_opts |= PV_SAME_TYPE;
+
+ int src_start_ofs = lex_ofs (lexer);
+ size_t n_src;
+ if (!parse_variables_const (lexer, dict, &src, &n_src, pv_opts))
+ goto error;
+ int src_end_ofs = lex_ofs (lexer) - 1;
+
+ /* Parse function arguments, for those functions that
+ require arguments. */
+ int args_start_ofs = 0;
+ if (function->n_args != 0)
+ for (size_t i = 0; i < function->n_args; i++)
+ {
+ lex_match (lexer, T_COMMA);
+
+ enum val_type type;
+ if (lex_is_string (lexer))
+ type = VAL_STRING;
+ else if (lex_is_number (lexer))
+ type = VAL_NUMERIC;
+ else
+ {
+ lex_error (lexer, _("Missing argument %zu to %s."),
+ i + 1, function->name);
+ goto error;
+ }
+
+ if (type != var_get_type (src[0]))
+ {
+ msg (SE, _("Arguments to %s must be of same type as "
+ "source variables."),
+ function->name);
+ if (type == VAL_NUMERIC)
+ {
+ lex_next_msg (lexer, SN, 0, 0,
+ _("The argument is numeric."));
+ lex_ofs_msg (lexer, SN, src_start_ofs, src_end_ofs,
+ _("The variables have string type."));
+ }
+ else
+ {
+ lex_next_msg (lexer, SN, 0, 0,
+ _("The argument is a string."));
+ lex_ofs_msg (lexer, SN, src_start_ofs, src_end_ofs,
+ _("The variables are numeric."));
+ }
+ goto error;
+ }
+
+ if (i == 0)
+ args_start_ofs = lex_ofs (lexer);
+ if (type == VAL_NUMERIC)
+ arg[i].f = lex_tokval (lexer);
+ else
+ arg[i].s = recode_substring_pool (dict_get_encoding (agr->dict),
+ "UTF-8", lex_tokss (lexer),
+ NULL);
+ lex_get (lexer);
+ }
+ int args_end_ofs = lex_ofs (lexer) - 1;
+
+ /* Trailing rparen. */
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ /* Now check that the number of source variables match
+ the number of target variables. If we check earlier
+ than this, the user can get very misleading error
+ message, i.e. `AGGREGATE x=SUM(y t).' will get this
+ error message when a proper message would be more
+ like `unknown variable t'. */
+ if (n_src != n_vars)
+ {
+ msg (SE, _("Number of source variables (%zu) does not match "
+ "number of target variables (%zu)."),
+ n_src, n_vars);
+ lex_ofs_msg (lexer, SN, src_start_ofs, src_end_ofs,
+ _("These are the source variables."));
+ lex_ofs_msg (lexer, SN, dst_start_ofs, dst_end_ofs,
+ _("These are the target variables."));
+ goto error;
+ }
+
+ if ((func_index == AGRF_PIN || func_index == AGRF_POUT
+ || func_index == AGRF_FIN || func_index == AGRF_FOUT)
+ && (var_is_numeric (src[0])
+ ? arg[0].f > arg[1].f
+ : buf_compare_rpad (arg[0].s.string, arg[0].s.length,
+ arg[1].s.string, arg[1].s.length) > 0))
+ {
+ struct agr_argument tmp = arg[0];
+ arg[0] = arg[1];
+ arg[1] = tmp;
+
+ lex_ofs_msg (lexer, SW, args_start_ofs, args_end_ofs,
+ _("The value arguments passed to the %s function "
+ "are out of order. They will be treated as if "
+ "they had been specified in the correct order."),
+ function->name);
+ }
+ }
+
+ /* Finally add these to the aggregation variables. */
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ const struct variable *existing_var = dict_lookup_var (agr->dict,
+ dest[i]);
+ if (existing_var)
+ {
+ if (var_get_dict_index (existing_var) >= starting_n_vars)
+ lex_ofs_error (lexer, dst_start_ofs, dst_end_ofs,
+ _("Duplicate target variable name %s."),
+ dest[i]);
+ else if (agr->add_variables)
+ lex_ofs_error (lexer, dst_start_ofs, dst_end_ofs,
+ _("Variable name %s duplicates the name of a "
+ "variable in the active file dictionary."),
+ dest[i]);
+ else
+ lex_ofs_error (lexer, dst_start_ofs, dst_end_ofs,
+ _("Variable name %s duplicates the name of a "
+ "break variable."), dest[i]);
+ goto error;
+ }
+
+ /* Add variable. */
+ if (agr->n_agr_vars >= allocated_agr_vars)
+ agr->agr_vars = x2nrealloc (agr->agr_vars, &allocated_agr_vars,
+ sizeof *agr->agr_vars);
+ struct agr_var *v = &agr->agr_vars[agr->n_agr_vars++];
+ *v = (struct agr_var) {
+ .exclude = exclude,
+ .moments = NULL,
+ .function = func_index,
+ .src = src ? src[i] : NULL,
+ };
+
+ /* Create the target variable in the aggregate dictionary. */
+ if (v->src && var_is_alpha (v->src))
+ v->string = xmalloc (var_get_width (v->src));
+
+ if (v->src && function->alpha_type == VAL_STRING)
+ v->dest = dict_clone_var_as_assert (agr->dict, v->src, dest[i]);
+ else
+ {
+ v->dest = dict_create_var_assert (agr->dict, dest[i], 0);
+
+ struct fmt_spec f;
+ if ((func_index == AGRF_N || func_index == AGRF_NMISS)
+ && dict_get_weight (dict) != NULL)
+ f = fmt_for_output (FMT_F, 8, 2);
+ else
+ f = function->format;
+ var_set_both_formats (v->dest, &f);
+ }
+ if (dest_label[i])
+ var_set_label (v->dest, dest_label[i]);
+
+ if (v->src != NULL)
+ for (size_t j = 0; j < function->n_args; j++)
+ v->arg[j] = (struct agr_argument) {
+ .f = arg[j].f,
+ .s = arg[j].s.string ? ss_clone (arg[j].s) : ss_empty (),
+ };
+ }
+
+ ss_dealloc (&arg[0].s);
+ ss_dealloc (&arg[1].s);
+
+ free (src);
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ free (dest[i]);
+ free (dest_label[i]);
+ }
+ free (dest);
+ free (dest_label);
+
+ if (!lex_match (lexer, T_SLASH))
+ {
+ if (lex_token (lexer) == T_ENDCMD)
+ return true;
+
+ lex_error (lexer, "Syntax error expecting end of command.");
+ return false;
+ }
+ continue;
+
+ error:
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ free (dest[i]);
+ free (dest_label[i]);
+ }
+ free (dest);
+ free (dest_label);
+ ss_dealloc (&arg[0].s);
+ ss_dealloc (&arg[1].s);
+ free (src);
+
+ return false;
+ }
+}
+
+/* Destroys AGR. */
+static void
+agr_destroy (struct agr_proc *agr)
+{
+ subcase_uninit (&agr->sort);
+ free (agr->break_vars);
+ for (size_t i = 0; i < agr->n_agr_vars; i++)
+ {
+ struct agr_var *av = &agr->agr_vars[i];
+
+ ss_dealloc (&av->arg[0].s);
+ ss_dealloc (&av->arg[1].s);
+ free (av->string);
+
+ if (av->function == AGRF_SD)
+ moments1_destroy (av->moments);
+
+ dict_destroy_internal_var (av->subject);
+ dict_destroy_internal_var (av->weight);
+ }
+ free (agr->agr_vars);
+ if (agr->dict != NULL)
+ dict_unref (agr->dict);
+}
+\f
+/* Execution. */
+
+/* Accumulates aggregation data from the case INPUT. */
+static void
+accumulate_aggregate_info (struct agr_proc *agr, const struct ccase *input)
+{
+ bool bad_warn = true;
+ double weight = dict_get_case_weight (agr->src_dict, input, &bad_warn);
+ for (size_t i = 0; i < agr->n_agr_vars; i++)
+ {
+ struct agr_var *av = &agr->agr_vars[i];
+ if (av->src)
+ {
+ bool is_string = var_is_alpha (av->src);
+ const union value *v = case_data (input, av->src);
+ int src_width = var_get_width (av->src);
+ const struct substring vs = (src_width > 0
+ ? value_ss (v, src_width)
+ : ss_empty ());
+
+ if (var_is_value_missing (av->src, v) & av->exclude)
+ {
+ switch (av->function)
+ {
+ case AGRF_NMISS:
+ av->dbl += weight;
+ break;
+
+ case AGRF_NUMISS:
+ av->int1++;
+ break;
+
+ case AGRF_SUM:
+ case AGRF_MEAN:
+ case AGRF_MEDIAN:
+ case AGRF_SD:
+ case AGRF_MAX:
+ case AGRF_MIN:
+ case AGRF_PGT:
+ case AGRF_PLT:
+ case AGRF_PIN:
+ case AGRF_POUT:
+ case AGRF_FGT:
+ case AGRF_FLT:
+ case AGRF_FIN:
+ case AGRF_FOUT:
+ case AGRF_CGT:
+ case AGRF_CLT:
+ case AGRF_CIN:
+ case AGRF_COUT:
+ case AGRF_N:
+ case AGRF_NU:
+ case AGRF_FIRST:
+ case AGRF_LAST:
+ break;
+ }
+ av->saw_missing = true;
+ continue;
+ }
+
+ /* This is horrible. There are too many possibilities. */
+ av->W += weight;
+ switch (av->function)
+ {
+ case AGRF_SUM:
+ av->dbl += v->f * weight;
+ av->int1 = 1;
+ break;
+
+ case AGRF_MEAN:
+ av->dbl += v->f * weight;
+ break;
+
+ case AGRF_MEDIAN:
+ {
+ struct ccase *cout = case_create (casewriter_get_proto (av->writer));
+ *case_num_rw (cout, av->subject) = case_num (input, av->src);
+ *case_num_rw (cout, av->weight) = weight;
+ casewriter_write (av->writer, cout);
+ }
+ break;
+
+ case AGRF_SD:
+ moments1_add (av->moments, v->f, weight);
+ break;
+
+ case AGRF_MAX:
+ if (!is_string)
+ av->dbl = MAX (av->dbl, v->f);
+ else if (memcmp (av->string, v->s, src_width) < 0)
+ memcpy (av->string, v->s, src_width);
+ av->int1 = 1;
+ break;
+
+ case AGRF_MIN:
+ if (!is_string)
+ av->dbl = MIN (av->dbl, v->f);
+ else if (memcmp (av->string, v->s, src_width) > 0)
+ memcpy (av->string, v->s, src_width);
+ av->dbl = MIN (av->dbl, v->f);
+ av->int1 = 1;
+ break;
+
+ case AGRF_FGT:
+ case AGRF_PGT:
+ case AGRF_CGT:
+ if (is_string
+ ? ss_compare_rpad (av->arg[0].s, vs) < 0
+ : v->f > av->arg[0].f)
+ av->dbl += weight;
+ break;
+
+ case AGRF_FLT:
+ case AGRF_PLT:
+ case AGRF_CLT:
+ if (is_string
+ ? ss_compare_rpad (av->arg[0].s, vs) > 0
+ : v->f < av->arg[0].f)
+ av->dbl += weight;
+ break;
+
+ case AGRF_FIN:
+ case AGRF_PIN:
+ case AGRF_CIN:
+ if (is_string
+ ? (ss_compare_rpad (av->arg[0].s, vs) <= 0
+ && ss_compare_rpad (av->arg[1].s, vs) >= 0)
+ : av->arg[0].f <= v->f && v->f <= av->arg[1].f)
+ av->dbl += weight;
+ break;
+
+ case AGRF_FOUT:
+ case AGRF_POUT:
+ case AGRF_COUT:
+ if (is_string
+ ? (ss_compare_rpad (av->arg[0].s, vs) > 0
+ || ss_compare_rpad (av->arg[1].s, vs) < 0)
+ : av->arg[0].f > v->f || v->f > av->arg[1].f)
+ av->dbl += weight;
+ break;
+
+ case AGRF_N:
+ av->dbl += weight;
+ break;
+
+ case AGRF_NU:
+ av->int1++;
+ break;
+
+ case AGRF_FIRST:
+ if (av->int1 == 0)
+ {
+ if (is_string)
+ memcpy (av->string, v->s, src_width);
+ else
+ av->dbl = v->f;
+ av->int1 = 1;
+ }
+ break;
+
+ case AGRF_LAST:
+ if (is_string)
+ memcpy (av->string, v->s, src_width);
+ else
+ av->dbl = v->f;
+ av->int1 = 1;
+ break;
+
+ case AGRF_NMISS:
+ case AGRF_NUMISS:
+ /* Our value is not missing or it would have been
+ caught earlier. Nothing to do. */
+ break;
+ }
+ }
+ else
+ {
+ av->W += weight;
+ switch (av->function)
+ {
+ case AGRF_N:
+ break;
+
+ case AGRF_NU:
+ av->int1++;
+ break;
+
+ case AGRF_SUM:
+ case AGRF_MEAN:
+ case AGRF_MEDIAN:
+ case AGRF_SD:
+ case AGRF_MAX:
+ case AGRF_MIN:
+ case AGRF_PGT:
+ case AGRF_PLT:
+ case AGRF_PIN:
+ case AGRF_POUT:
+ case AGRF_FGT:
+ case AGRF_FLT:
+ case AGRF_FIN:
+ case AGRF_FOUT:
+ case AGRF_CGT:
+ case AGRF_CLT:
+ case AGRF_CIN:
+ case AGRF_COUT:
+ case AGRF_NMISS:
+ case AGRF_NUMISS:
+ case AGRF_FIRST:
+ case AGRF_LAST:
+ NOT_REACHED ();
+ }
+ }
+ }
+}
+
+/* Writes an aggregated record to OUTPUT. */
+static void
+dump_aggregate_info (const struct agr_proc *agr, struct casewriter *output, const struct ccase *break_case)
+{
+ struct ccase *c = case_create (dict_get_proto (agr->dict));
+
+ if (agr->add_variables)
+ {
+ case_copy (c, 0, break_case, 0, dict_get_n_vars (agr->src_dict));
+ }
+ else
+ {
+ int value_idx = 0;
+
+ for (size_t i = 0; i < agr->break_n_vars; i++)
+ {
+ const struct variable *v = agr->break_vars[i];
+ value_copy (case_data_rw_idx (c, value_idx),
+ case_data (break_case, v),
+ var_get_width (v));
+ value_idx++;
+ }
+ }
+
+ for (size_t i = 0; i < agr->n_agr_vars; i++)
+ {
+ struct agr_var *av = &agr->agr_vars[i];
+ union value *v = case_data_rw (c, av->dest);
+ int width = var_get_width (av->dest);
+
+ if (agr->missing == COLUMNWISE && av->saw_missing
+ && av->function != AGRF_N
+ && av->function != AGRF_NU
+ && av->function != AGRF_NMISS
+ && av->function != AGRF_NUMISS)
+ {
+ value_set_missing (v, width);
+ casewriter_destroy (av->writer);
+ continue;
+ }
+
+ switch (av->function)
+ {
+ case AGRF_SUM:
+ v->f = av->int1 ? av->dbl : SYSMIS;
+ break;
+
+ case AGRF_MEAN:
+ v->f = av->W != 0.0 ? av->dbl / av->W : SYSMIS;
+ break;
+
+ case AGRF_MEDIAN:
+ {
+ if (av->writer)
+ {
+ struct percentile *median = percentile_create (0.5, av->W);
+ struct order_stats *os = &median->parent;
+ struct casereader *sorted_reader = casewriter_make_reader (av->writer);
+ av->writer = NULL;
+
+ order_stats_accumulate (&os, 1,
+ sorted_reader,
+ av->weight,
+ av->subject,
+ av->exclude);
+ av->dbl = percentile_calculate (median, PC_HAVERAGE);
+ statistic_destroy (&median->parent.parent);
+ }
+ v->f = av->dbl;
+ }
+ break;
+
+ case AGRF_SD:
+ {
+ double variance;
+
+ moments1_calculate (av->moments, NULL, NULL, &variance,
+ NULL, NULL);
+ v->f = variance != SYSMIS ? sqrt (variance) : SYSMIS;
+ }
+ break;
+
+ case AGRF_MAX:
+ case AGRF_MIN:
+ case AGRF_FIRST:
+ case AGRF_LAST:
+ if (!width)
+ v->f = av->int1 ? av->dbl : SYSMIS;
+ else
+ {
+ if (av->int1)
+ memcpy (v->s, av->string, width);
+ else
+ value_set_missing (v, width);
+ }
+ break;
+
+ case AGRF_FGT:
+ case AGRF_FLT:
+ case AGRF_FIN:
+ case AGRF_FOUT:
+ v->f = av->W ? av->dbl / av->W : SYSMIS;
+ break;
+
+ case AGRF_PGT:
+ case AGRF_PLT:
+ case AGRF_PIN:
+ case AGRF_POUT:
+ v->f = av->W ? av->dbl / av->W * 100.0 : SYSMIS;
+ break;
+
+ case AGRF_CGT:
+ case AGRF_CLT:
+ case AGRF_CIN:
+ case AGRF_COUT:
+ v->f = av->dbl;
+ break;
+
+ case AGRF_N:
+ v->f = av->W;
+ break;
+
+ case AGRF_NU:
+ case AGRF_NUMISS:
+ v->f = av->int1;
+ break;
+
+ case AGRF_NMISS:
+ v->f = av->dbl;
+ break;
+ }
+ }
+
+ casewriter_write (output, c);
+}
+
+/* Resets the state for all the aggregate functions. */
+static void
+initialize_aggregate_info (struct agr_proc *agr)
+{
+ for (size_t i = 0; i < agr->n_agr_vars; i++)
+ {
+ struct agr_var *av = &agr->agr_vars[i];
+ av->saw_missing = false;
+ av->dbl = av->W = 0.0;
+ av->int1 = 0;
+
+ int width = av->src ? var_get_width (av->src) : 0;
+ switch (av->function)
+ {
+ case AGRF_MIN:
+ if (!width)
+ av->dbl = DBL_MAX;
+ else
+ memset (av->string, 255, width);
+ break;
+
+ case AGRF_MAX:
+ if (!width)
+ av->dbl = -DBL_MAX;
+ else
+ memset (av->string, 0, width);
+ break;
+
+ case AGRF_MEDIAN:
+ {
+ struct caseproto *proto = caseproto_create ();
+ proto = caseproto_add_width (proto, 0);
+ proto = caseproto_add_width (proto, 0);
+
+ if (! av->subject)
+ av->subject = dict_create_internal_var (0, 0);
+
+ if (! av->weight)
+ av->weight = dict_create_internal_var (1, 0);
+
+ struct subcase ordering;
+ subcase_init_var (&ordering, av->subject, SC_ASCEND);
+ av->writer = sort_create_writer (&ordering, proto);
+ subcase_uninit (&ordering);
+ caseproto_unref (proto);
+ }
+ break;
+
+ case AGRF_SD:
+ if (av->moments == NULL)
+ av->moments = moments1_create (MOMENT_VARIANCE);
+ else
+ moments1_clear (av->moments);
+ break;
+
+ case AGRF_SUM:
+ case AGRF_MEAN:
+ case AGRF_PGT:
+ case AGRF_PLT:
+ case AGRF_PIN:
+ case AGRF_POUT:
+ case AGRF_FGT:
+ case AGRF_FLT:
+ case AGRF_FIN:
+ case AGRF_FOUT:
+ case AGRF_CGT:
+ case AGRF_CLT:
+ case AGRF_CIN:
+ case AGRF_COUT:
+ case AGRF_N:
+ case AGRF_NU:
+ case AGRF_NMISS:
+ case AGRF_NUMISS:
+ case AGRF_FIRST:
+ case AGRF_LAST:
+ break;
+ }
+ }
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+
+#ifndef AGGREGATE_H
+#define AGGREGATE_H
+
+#include <stddef.h>
+
+#include "data/format.h"
+#include "data/val-type.h"
+
+enum agr_src_vars
+ {
+ AGR_SV_NO,
+ AGR_SV_YES,
+ AGR_SV_OPT
+ };
+
+#define AGGREGATE_FUNCTIONS \
+ AGRF(AGRF_SUM, "SUM", N_("Sum of values"), AGR_SV_YES, 0, -1, 8, 2) \
+ AGRF(AGRF_MEAN, "MEAN", N_("Mean average"), AGR_SV_YES, 0, -1, 8, 2) \
+ AGRF(AGRF_MEDIAN, "MEDIAN", N_("Median"), AGR_SV_YES, 0, -1, 8, 2) \
+ AGRF(AGRF_SD, "SD", N_("Standard deviation"), AGR_SV_YES, 0, -1, 8, 2) \
+ AGRF(AGRF_MAX, "MAX", N_("Maximum value"), AGR_SV_YES, 0, VAL_STRING, -1, -1) \
+ AGRF(AGRF_MIN, "MIN", N_("Minimum value"), AGR_SV_YES, 0, VAL_STRING, -1, -1) \
+ AGRF(AGRF_PGT, "PGT", N_("Percentage greater than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_PLT, "PLT", N_("Percentage less than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_PIN, "PIN", N_("Percentage included in range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_POUT, "POUT", N_("Percentage excluded from range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_FGT, "FGT", N_("Fraction greater than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 3) \
+ AGRF(AGRF_FLT, "FLT", N_("Fraction less than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 3) \
+ AGRF(AGRF_FIN, "FIN", N_("Fraction included in range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 3) \
+ AGRF(AGRF_FOUT, "FOUT", N_("Fraction excluded from range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 3) \
+ AGRF(AGRF_CGT, "CGT", N_("Count greater than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_CLT, "CLT", N_("Count less than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_CIN, "CIN", N_("Count included in range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_COUT, "COUT", N_("Count excluded from range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
+ AGRF(AGRF_N, "N", N_("Number of cases"), AGR_SV_NO, 0, VAL_NUMERIC, 7, 0) \
+ AGRF(AGRF_NU, "NU", N_("Number of cases (unweighted)"), AGR_SV_OPT, 0, VAL_NUMERIC, 7, 0) \
+ AGRF(AGRF_NMISS, "NMISS", N_("Number of missing values"), AGR_SV_YES, 0, VAL_NUMERIC, 7, 0) \
+ AGRF(AGRF_NUMISS, "NUMISS", N_("Number of missing values (unweighted)"), AGR_SV_YES, 0, VAL_NUMERIC, 7, 0) \
+ AGRF(AGRF_FIRST, "FIRST", N_("First non-missing value"), AGR_SV_YES, 0, VAL_STRING, -1, -1) \
+ AGRF(AGRF_LAST, "LAST", N_("Last non-missing value"), AGR_SV_YES, 0, VAL_STRING, -1, -1)
+
+/* Aggregation functions. */
+enum agr_function
+ {
+#define AGRF(ENUM, NAME, DESCRIPTION, SRC_VARS, N_ARGS, ALPHA_TYPE, W, D) \
+ ENUM,
+AGGREGATE_FUNCTIONS
+#undef AGRF
+ };
+
+/* Attributes of an aggregation function. */
+struct agr_func
+ {
+ const char *name; /* Aggregation function name. */
+ const char *description; /* Translatable string describing the function. */
+ enum agr_src_vars src_vars; /* Whether source variables are a parameter of the function */
+ size_t n_args; /* Number of arguments (not including src vars). */
+ enum val_type alpha_type; /* When given ALPHA arguments, output type. */
+ struct fmt_spec format; /* Format spec if alpha_type != ALPHA. */
+ };
+
+extern const struct agr_func agr_func_tab[];
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2012, 2014, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/any-reader.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/file-handle-def.h"
+#include "data/missing-values.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Parses and executes APPLY DICTIONARY. */
+int
+cmd_apply_dictionary (struct lexer *lexer, struct dataset *ds)
+{
+ lex_match_id (lexer, "FROM");
+ lex_match (lexer, T_EQUALS);
+
+ struct file_handle *handle = fh_parse (lexer, FH_REF_FILE,
+ dataset_session (ds));
+ if (!handle)
+ return CMD_FAILURE;
+
+ struct dictionary *dict;
+ struct casereader *reader = any_reader_open_and_decode (handle, NULL, &dict,
+ NULL);
+ fh_unref (handle);
+ if (!reader)
+ return CMD_FAILURE;
+
+ casereader_destroy (reader);
+
+ size_t n_matched = 0;
+ for (size_t i = 0; i < dict_get_n_vars (dict); i++)
+ {
+ const struct variable *s = dict_get_var (dict, i);
+ struct variable *t = dict_lookup_var (dataset_dict (ds),
+ var_get_name (s));
+ if (t == NULL)
+ continue;
+
+ n_matched++;
+ if (var_get_type (s) != var_get_type (t))
+ {
+ msg (SW, _("Variable %s is %s in target file, but %s in "
+ "source file."),
+ var_get_name (s),
+ var_is_alpha (t) ? _("string") : _("numeric"),
+ var_is_alpha (s) ? _("string") : _("numeric"));
+ continue;
+ }
+
+ if (var_has_label (s))
+ var_set_label (t, var_get_label (s));
+
+ if (var_has_value_labels (s))
+ {
+ const struct val_labs *value_labels = var_get_value_labels (s);
+ if (val_labs_can_set_width (value_labels, var_get_width (t)))
+ var_set_value_labels (t, value_labels);
+ }
+
+ if (var_has_missing_values (s))
+ {
+ const struct missing_values *miss = var_get_missing_values (s);
+ if (mv_is_resizable (miss, var_get_width (t)))
+ var_set_missing_values (t, miss);
+ }
+
+ if (var_is_numeric (s))
+ {
+ var_set_print_format (t, var_get_print_format (s));
+ var_set_write_format (t, var_get_write_format (s));
+ }
+
+ if (var_has_attributes (s))
+ var_set_attributes (t, var_get_attributes (s));
+ }
+
+ if (!n_matched)
+ msg (SW, _("No matching variables found between the source "
+ "and target files."));
+
+ /* Data file attributes. */
+ if (dict_has_attributes (dict))
+ dict_set_attributes (dataset_dict (ds), dict_get_attributes (dict));
+
+ /* Weighting. */
+ if (dict_get_weight (dict) != NULL)
+ {
+ struct variable *new_weight
+ = dict_lookup_var (dataset_dict (ds),
+ var_get_name (dict_get_weight (dict)));
+
+ if (new_weight != NULL)
+ dict_set_weight (dataset_dict (ds), new_weight);
+ }
+
+ dict_unref (dict);
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2008, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/attributes.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+static enum cmd_result parse_attributes (struct lexer *,
+ const char *dict_encoding,
+ struct attrset **, size_t n);
+
+/* Parses the DATAFILE ATTRIBUTE command. */
+int
+cmd_datafile_attribute (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct attrset *set = dict_get_attributes (dict);
+ return parse_attributes (lexer, dict_get_encoding (dict), &set, 1);
+}
+
+/* Parses the VARIABLE ATTRIBUTE command. */
+int
+cmd_variable_attribute (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ const char *dict_encoding = dict_get_encoding (dict);
+
+ do
+ {
+ struct variable **vars;
+ struct attrset **sets;
+ size_t n_vars, i;
+ bool ok;
+
+ if (!lex_force_match_phrase (lexer, "VARIABLES=")
+ || !parse_variables (lexer, dict, &vars, &n_vars, PV_NONE))
+ return CMD_FAILURE;
+
+ sets = xmalloc (n_vars * sizeof *sets);
+ for (i = 0; i < n_vars; i++)
+ sets[i] = var_get_attributes (vars[i]);
+
+ ok = parse_attributes (lexer, dict_encoding, sets, n_vars);
+ free (vars);
+ free (sets);
+ if (!ok)
+ return CMD_FAILURE;
+ }
+ while (lex_match (lexer, T_SLASH));
+
+ return CMD_SUCCESS;
+}
+
+/* Parses an attribute name and verifies that it is valid in DICT_ENCODING,
+ optionally followed by an index inside square brackets. Returns the
+ attribute name or NULL if there was a parse error. Stores the index into
+ *INDEX. */
+static char *
+parse_attribute_name (struct lexer *lexer, const char *dict_encoding,
+ size_t *index)
+{
+ if (!lex_force_id (lexer))
+ return NULL;
+ char *error = id_is_valid__ (lex_tokcstr (lexer), dict_encoding);
+ if (error)
+ {
+ lex_error (lexer, "%s", error);
+ free (error);
+ return NULL;
+ }
+ char *name = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ if (lex_match (lexer, T_LBRACK))
+ {
+ if (!lex_force_int_range (lexer, NULL, 1, 65535))
+ goto error;
+ *index = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RBRACK))
+ goto error;
+ }
+ else
+ *index = 0;
+ return name;
+
+error:
+ free (name);
+ return NULL;
+}
+
+static bool
+add_attribute (struct lexer *lexer, const char *dict_encoding,
+ struct attrset **sets, size_t n)
+{
+ const char *value;
+ size_t index, i;
+ char *name;
+
+ name = parse_attribute_name (lexer, dict_encoding, &index);
+ if (name == NULL)
+ return false;
+ if (!lex_force_match (lexer, T_LPAREN) || !lex_force_string (lexer))
+ {
+ free (name);
+ return false;
+ }
+ value = lex_tokcstr (lexer);
+
+ for (i = 0; i < n; i++)
+ {
+ struct attribute *attr = attrset_lookup (sets[i], name);
+ if (attr == NULL)
+ {
+ attr = attribute_create (name);
+ attrset_add (sets[i], attr);
+ }
+ attribute_set_value (attr, index ? index - 1 : 0, value);
+ }
+
+ lex_get (lexer);
+ free (name);
+ return lex_force_match (lexer, T_RPAREN);
+}
+
+static bool
+delete_attribute (struct lexer *lexer, const char *dict_encoding,
+ struct attrset **sets, size_t n)
+{
+ size_t index, i;
+ char *name;
+
+ name = parse_attribute_name (lexer, dict_encoding, &index);
+ if (name == NULL)
+ return false;
+
+ for (i = 0; i < n; i++)
+ {
+ struct attrset *set = sets[i];
+ if (index == 0)
+ attrset_delete (set, name);
+ else
+ {
+ struct attribute *attr = attrset_lookup (set, name);
+ if (attr != NULL)
+ {
+ attribute_del_value (attr, index - 1);
+ if (attribute_get_n_values (attr) == 0)
+ attrset_delete (set, name);
+ }
+ }
+ }
+
+ free (name);
+ return true;
+}
+
+static enum cmd_result
+parse_attributes (struct lexer *lexer, const char *dict_encoding,
+ struct attrset **sets, size_t n)
+{
+ enum { UNKNOWN, ADD, DELETE } command = UNKNOWN;
+ do
+ {
+ if (lex_match_phrase (lexer, "ATTRIBUTE="))
+ command = ADD;
+ else if (lex_match_phrase (lexer, "DELETE="))
+ command = DELETE;
+ else if (command == UNKNOWN)
+ {
+ lex_error_expecting (lexer, "ATTRIBUTE=", "DELETE=");
+ return CMD_FAILURE;
+ }
+
+ if (!(command == ADD
+ ? add_attribute (lexer, dict_encoding, sets, n)
+ : delete_attribute (lexer, dict_encoding, sets, n)))
+ return CMD_FAILURE;
+ }
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
+ return CMD_SUCCESS;
+}
--- /dev/null
+## Process this file with automake to produce Makefile.in -*- makefile -*-
+
+# PSPP - a program for statistical analysis.
+# Copyright (C) 2017-2022 Free Software Foundation, Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+AM_CPPFLAGS += -I"$(top_srcdir)/src/language/commands"
+
+language_commands_sources = \
+ src/language/commands/aggregate.c \
+ src/language/commands/aggregate.h \
+ src/language/commands/apply-dictionary.c \
+ src/language/commands/attributes.c \
+ src/language/commands/autorecode.c \
+ src/language/commands/binomial.c \
+ src/language/commands/binomial.h \
+ src/language/commands/cache.c \
+ src/language/commands/cd.c \
+ src/language/commands/chart-category.h \
+ src/language/commands/chisquare.c \
+ src/language/commands/chisquare.h \
+ src/language/commands/cochran.c \
+ src/language/commands/cochran.h \
+ src/language/commands/combine-files.c \
+ src/language/commands/compute.c \
+ src/language/commands/correlations.c \
+ src/language/commands/count.c \
+ src/language/commands/crosstabs.c \
+ src/language/commands/ctables.c \
+ src/language/commands/ctables.inc \
+ src/language/commands/data-list.c \
+ src/language/commands/data-parser.c \
+ src/language/commands/data-parser.h \
+ src/language/commands/data-reader.c \
+ src/language/commands/data-reader.h \
+ src/language/commands/data-writer.c \
+ src/language/commands/data-writer.h \
+ src/language/commands/dataset.c \
+ src/language/commands/date.c \
+ src/language/commands/define.c \
+ src/language/commands/delete-variables.c \
+ src/language/commands/descriptives.c \
+ src/language/commands/do-if.c \
+ src/language/commands/echo.c \
+ src/language/commands/examine.c \
+ src/language/commands/factor.c \
+ src/language/commands/fail.c \
+ src/language/commands/file-handle.c \
+ src/language/commands/file-handle.h \
+ src/language/commands/flip.c \
+ src/language/commands/formats.c \
+ src/language/commands/freq.c \
+ src/language/commands/freq.h \
+ src/language/commands/frequencies.c \
+ src/language/commands/friedman.c \
+ src/language/commands/friedman.h \
+ src/language/commands/get-data.c \
+ src/language/commands/get.c \
+ src/language/commands/glm.c \
+ src/language/commands/graph.c \
+ src/language/commands/host.c \
+ src/language/commands/include.c \
+ src/language/commands/inpt-pgm.c \
+ src/language/commands/inpt-pgm.h \
+ src/language/commands/jonckheere-terpstra.c \
+ src/language/commands/jonckheere-terpstra.h \
+ src/language/commands/kruskal-wallis.c \
+ src/language/commands/kruskal-wallis.h \
+ src/language/commands/ks-one-sample.c \
+ src/language/commands/ks-one-sample.h \
+ src/language/commands/list.c \
+ src/language/commands/logistic.c \
+ src/language/commands/loop.c \
+ src/language/commands/mann-whitney.c \
+ src/language/commands/mann-whitney.h \
+ src/language/commands/matrix-data.c \
+ src/language/commands/matrix-reader.c \
+ src/language/commands/matrix-reader.h \
+ src/language/commands/matrix.c \
+ src/language/commands/mcnemar.c \
+ src/language/commands/mcnemar.h \
+ src/language/commands/mconvert.c \
+ src/language/commands/means-calc.c \
+ src/language/commands/means-parser.c \
+ src/language/commands/means.c \
+ src/language/commands/means.h \
+ src/language/commands/median.c \
+ src/language/commands/median.h \
+ src/language/commands/missing-values.c \
+ src/language/commands/mrsets.c \
+ src/language/commands/npar-summary.c \
+ src/language/commands/npar-summary.h \
+ src/language/commands/npar.c \
+ src/language/commands/npar.h \
+ src/language/commands/numeric.c \
+ src/language/commands/oneway.c \
+ src/language/commands/output.c \
+ src/language/commands/permissions.c \
+ src/language/commands/placement-parser.c \
+ src/language/commands/placement-parser.h \
+ src/language/commands/print-space.c \
+ src/language/commands/print.c \
+ src/language/commands/quick-cluster.c \
+ src/language/commands/rank.c \
+ src/language/commands/recode.c \
+ src/language/commands/regression.c \
+ src/language/commands/reliability.c \
+ src/language/commands/rename-variables.c \
+ src/language/commands/repeat.c \
+ src/language/commands/roc.c \
+ src/language/commands/roc.h \
+ src/language/commands/runs.c \
+ src/language/commands/runs.h \
+ src/language/commands/sample.c \
+ src/language/commands/save-translate.c \
+ src/language/commands/save.c \
+ src/language/commands/select-if.c \
+ src/language/commands/set.c \
+ src/language/commands/sign.c \
+ src/language/commands/sign.h \
+ src/language/commands/sort-cases.c \
+ src/language/commands/sort-criteria.c \
+ src/language/commands/sort-criteria.h \
+ src/language/commands/sort-variables.c \
+ src/language/commands/split-file.c \
+ src/language/commands/split-file.h \
+ src/language/commands/sys-file-info.c \
+ src/language/commands/t-test-indep.c \
+ src/language/commands/t-test-one-sample.c \
+ src/language/commands/t-test-paired.c \
+ src/language/commands/t-test-parser.c \
+ src/language/commands/t-test.h \
+ src/language/commands/temporary.c \
+ src/language/commands/title.c \
+ src/language/commands/trim.c \
+ src/language/commands/trim.h \
+ src/language/commands/value-labels.c \
+ src/language/commands/variable-display.c \
+ src/language/commands/variable-label.c \
+ src/language/commands/vector.c \
+ src/language/commands/weight.c \
+ src/language/commands/wilcoxon.c \
+ src/language/commands/wilcoxon.h
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2012, 2013, 2014
+ 2021, Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/compiler.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmap.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+#include "gl/c-xvasprintf.h"
+#include "gl/mbiter.h"
+#include "gl/size_max.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+/* Explains how to recode one value. */
+struct arc_item
+ {
+ struct hmap_node hmap_node; /* Element in "struct arc_spec" hash table. */
+ union value from; /* Original value. */
+ int width; /* Width of the original value */
+ bool missing; /* Is 'from' missing in its source varible? */
+ char *value_label; /* Value label in source variable, if any. */
+
+ double to; /* Recoded value. */
+ };
+
+/* Explains how to recode an AUTORECODE variable. */
+struct arc_spec
+ {
+ int width; /* Variable width. */
+ int src_idx; /* Case index of source variable. */
+ char *src_name; /* Name of source variable. */
+ struct fmt_spec format; /* Print format in source variable. */
+ struct variable *dst; /* Target variable. */
+ struct missing_values mv; /* Missing values of source variable. */
+ char *label; /* Variable label of source variable. */
+ struct rec_items *items;
+ };
+
+/* Descending or ascending sort order. */
+enum arc_direction
+ {
+ ASCENDING,
+ DESCENDING
+ };
+
+struct rec_items
+{
+ struct hmap ht; /* Hash table of "struct arc_item"s. */
+};
+
+
+
+/* AUTORECODE data. */
+struct autorecode_pgm
+{
+ struct arc_spec *specs;
+ size_t n_specs;
+
+ bool blank_valid;
+};
+
+static const struct trns_class autorecode_trns_class;
+
+static int compare_arc_items (const void *, const void *, const void *aux);
+static void arc_free (struct autorecode_pgm *);
+static struct arc_item *find_arc_item (
+ const struct rec_items *, const union value *, int width,
+ size_t hash);
+
+/* Returns WIDTH with any trailing spaces in VALUE trimmed off (except that a
+ minimum width of 1 is always returned because otherwise the width would
+ indicate a numeric type). */
+static int
+value_trim_spaces (const union value *value, int width)
+{
+ while (width > 1 && value->s[width - 1] == ' ')
+ width--;
+ return width;
+}
+
+/* Performs the AUTORECODE procedure. */
+int
+cmd_autorecode (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+
+ const struct variable **src_vars = NULL;
+ size_t n_srcs = 0;
+
+ char **dst_names = NULL;
+ size_t n_dsts = 0;
+
+ enum arc_direction direction = ASCENDING;
+ bool print = false;
+
+ /* Create procedure. */
+ struct autorecode_pgm *arc = XZALLOC (struct autorecode_pgm);
+ arc->blank_valid = true;
+
+ /* Parse variable lists. */
+ lex_match_id (lexer, "VARIABLES");
+ lex_match (lexer, T_EQUALS);
+ if (!parse_variables_const (lexer, dict, &src_vars, &n_srcs,
+ PV_NO_DUPLICATE | PV_NO_SCRATCH))
+ goto error;
+ lex_match (lexer, T_SLASH);
+ if (!lex_force_match_id (lexer, "INTO"))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+ if (!parse_DATA_LIST_vars (lexer, dict, &dst_names, &n_dsts,
+ PV_NO_DUPLICATE))
+ goto error;
+ if (n_dsts != n_srcs)
+ {
+ msg (SE, _("Source variable count (%zu) does not match "
+ "target variable count (%zu)."),
+ n_srcs, n_dsts);
+
+ goto error;
+ }
+ for (size_t i = 0; i < n_dsts; i++)
+ {
+ const char *name = dst_names[i];
+
+ if (dict_lookup_var (dict, name) != NULL)
+ {
+ msg (SE, _("Target variable %s duplicates existing variable %s."),
+ name, name);
+ goto error;
+ }
+ }
+
+ /* Parse options. */
+ bool group = false;
+ while (lex_match (lexer, T_SLASH))
+ {
+ if (lex_match_id (lexer, "DESCENDING"))
+ direction = DESCENDING;
+ else if (lex_match_id (lexer, "PRINT"))
+ print = true;
+ else if (lex_match_id (lexer, "GROUP"))
+ group = true;
+ else if (lex_match_id (lexer, "BLANK"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "VALID"))
+ {
+ arc->blank_valid = true;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ arc->blank_valid = false;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "VALID", "MISSING");
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "DESCENDING", "PRINT", "GROUP", "BLANK");
+ goto error;
+ }
+ }
+
+ if (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_error (lexer, _("Syntax error expecting end of command."));
+ goto error;
+ }
+
+ /* If GROUP is specified, verify that the variables are all string or all
+ numeric. */
+ if (group)
+ {
+ enum val_type type = var_get_type (src_vars[0]);
+ for (size_t i = 1; i < n_dsts; i++)
+ {
+ if (var_get_type (src_vars[i]) != type)
+ {
+ size_t string_idx = type == VAL_STRING ? 0 : i;
+ size_t numeric_idx = type == VAL_STRING ? i : 0;
+ lex_error (lexer, _("With GROUP, variables may not mix string "
+ "variables (such as %s) and numeric "
+ "variables (such as %s)."),
+ var_get_name (src_vars[string_idx]),
+ var_get_name (src_vars[numeric_idx]));
+ goto error;
+ }
+ }
+ }
+
+ /* Allocate all the specs and the rec_items that they point to.
+
+ If GROUP is specified, there is only a single global rec_items, with the
+ maximum width 'width', and all of the specs point to it; otherwise each
+ spec has its own rec_items. */
+ arc->specs = xmalloc (n_dsts * sizeof *arc->specs);
+ arc->n_specs = n_dsts;
+ for (size_t i = 0; i < n_dsts; i++)
+ {
+ struct arc_spec *spec = &arc->specs[i];
+
+ spec->width = var_get_width (src_vars[i]);
+ spec->src_idx = var_get_case_index (src_vars[i]);
+ spec->src_name = xstrdup (var_get_name (src_vars[i]));
+ spec->format = *var_get_print_format (src_vars[i]);
+
+ const char *label = var_get_label (src_vars[i]);
+ spec->label = xstrdup_if_nonnull (label);
+
+ if (group && i > 0)
+ spec->items = arc->specs[0].items;
+ else
+ {
+ spec->items = xzalloc (sizeof (*spec->items));
+ hmap_init (&spec->items->ht);
+ }
+ }
+
+ /* Initialize specs[*]->mv to the user-missing values for each
+ source variable. */
+ if (group)
+ {
+ /* Use the first source variable that has any user-missing values. */
+ size_t mv_idx = 0;
+ for (size_t i = 0; i < n_dsts; i++)
+ if (var_has_missing_values (src_vars[i]))
+ {
+ mv_idx = i;
+ break;
+ }
+
+ for (size_t i = 0; i < n_dsts; i++)
+ mv_copy (&arc->specs[i].mv, var_get_missing_values (src_vars[mv_idx]));
+ }
+ else
+ {
+ /* Each variable uses its own user-missing values. */
+ for (size_t i = 0; i < n_dsts; i++)
+ mv_copy (&arc->specs[i].mv, var_get_missing_values (src_vars[i]));
+ }
+
+ /* Execute procedure. */
+ struct casereader *input = proc_open (ds);
+ struct ccase *c;
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ for (size_t i = 0; i < arc->n_specs; i++)
+ {
+ struct arc_spec *spec = &arc->specs[i];
+ const union value *value = case_data_idx (c, spec->src_idx);
+ if (spec->width == 0 && value->f == SYSMIS)
+ {
+ /* AUTORECODE never changes the system-missing value.
+ (Leaving it out of the translation table has this
+ effect automatically because values not found in the
+ translation table get translated to system-missing.) */
+ continue;
+ }
+
+ int width = value_trim_spaces (value, spec->width);
+ if (width == 1 && value->s[0] == ' ' && !arc->blank_valid)
+ continue;
+
+ size_t hash = value_hash (value, width, 0);
+ if (find_arc_item (spec->items, value, width, hash))
+ continue;
+
+ struct string value_label = DS_EMPTY_INITIALIZER;
+ var_append_value_name__ (src_vars[i], value,
+ SETTINGS_VALUE_SHOW_LABEL, &value_label);
+
+ struct arc_item *item = xmalloc (sizeof *item);
+ item->width = width;
+ value_clone (&item->from, value, width);
+ item->missing = mv_is_value_missing_varwidth (&spec->mv, value,
+ spec->width);
+ item->value_label = ds_steal_cstr (&value_label);
+ hmap_insert (&spec->items->ht, &item->hmap_node, hash);
+
+ ds_destroy (&value_label);
+ }
+ bool ok = casereader_destroy (input);
+ ok = proc_commit (ds) && ok;
+
+ /* Re-fetch dictionary because it might have changed (if TEMPORARY was in
+ use). */
+ dict = dataset_dict (ds);
+
+ /* Create transformation. */
+ for (size_t i = 0; i < arc->n_specs; i++)
+ {
+ struct arc_spec *spec = &arc->specs[i];
+
+ /* Create destination variable. */
+ spec->dst = dict_create_var_assert (dict, dst_names[i], 0);
+ var_set_label (spec->dst, spec->label);
+
+ /* Set print format. */
+ size_t n_items = hmap_count (&spec->items->ht);
+ char *longest_value = xasprintf ("%zu", n_items);
+ struct fmt_spec format = { .type = FMT_F, .w = strlen (longest_value) };
+ var_set_both_formats (spec->dst, &format);
+ free (longest_value);
+
+ /* Create array of pointers to items. */
+ struct arc_item **items = xmalloc (n_items * sizeof *items);
+ struct arc_item *item;
+ size_t j = 0;
+ HMAP_FOR_EACH (item, struct arc_item, hmap_node, &spec->items->ht)
+ items[j++] = item;
+ assert (j == n_items);
+
+ /* Sort array by value. */
+ sort (items, n_items, sizeof *items, compare_arc_items, &direction);
+
+ /* Assign recoded values in sorted order. */
+ for (j = 0; j < n_items; j++)
+ items[j]->to = j + 1;
+
+ if (print && (!group || i == 0))
+ {
+ struct pivot_value *title
+ = (group
+ ? pivot_value_new_text (N_("Recoding grouped variables."))
+ : spec->label && spec->label[0]
+ ? pivot_value_new_text_format (N_("Recoding %s into %s (%s)."),
+ spec->src_name,
+ var_get_name (spec->dst),
+ spec->label)
+ : pivot_value_new_text_format (N_("Recoding %s into %s."),
+ spec->src_name,
+ var_get_name (spec->dst)));
+ struct pivot_table *table = pivot_table_create__ (title, "Recoding");
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Attributes"),
+ N_("New Value"), N_("Value Label"));
+
+ struct pivot_dimension *old_values = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Old Value"));
+ old_values->root->show_label = true;
+
+ for (size_t k = 0; k < n_items; k++)
+ {
+ const struct arc_item *item = items[k];
+ int old_value_idx = pivot_category_create_leaf (
+ old_values->root, pivot_value_new_value (
+ &item->from, item->width,
+ (item->width
+ ? &(struct fmt_spec) { .type = FMT_F, .w = item->width }
+ : &spec->format),
+ dict_get_encoding (dict)));
+ pivot_table_put2 (table, 0, old_value_idx,
+ pivot_value_new_integer (item->to));
+
+ const char *value_label = item->value_label;
+ if (value_label && value_label[0])
+ pivot_table_put2 (table, 1, old_value_idx,
+ pivot_value_new_user_text (value_label, -1));
+ }
+
+ pivot_table_submit (table);
+ }
+
+ /* Assign user-missing values.
+
+ User-missing values in the source variable(s) must be marked
+ as user-missing values in the destination variable. There
+ might be an arbitrary number of missing values, since the
+ source variable might have a range. Our sort function always
+ puts missing values together at the top of the range, so that
+ means that we can use a missing value range to cover all of
+ the user-missing values in any case (but we avoid it unless
+ necessary because user-missing value ranges are an obscure
+ feature). */
+ size_t n_missing = n_items;
+ for (size_t k = 0; k < n_items; k++)
+ if (!items[n_items - k - 1]->missing)
+ {
+ n_missing = k;
+ break;
+ }
+ if (n_missing > 0)
+ {
+ size_t lo = n_items - (n_missing - 1);
+ size_t hi = n_items;
+
+ struct missing_values mv;
+ mv_init (&mv, 0);
+ if (n_missing > 3)
+ mv_add_range (&mv, lo, hi);
+ else
+ for (size_t k = 0; k < n_missing; k++)
+ mv_add_num (&mv, lo + k);
+ var_set_missing_values (spec->dst, &mv);
+ mv_destroy (&mv);
+ }
+
+ /* Add value labels to the destination variable. */
+ for (j = 0; j < n_items; j++)
+ {
+ const char *value_label = items[j]->value_label;
+ if (value_label && value_label[0])
+ {
+ union value to_val = { .f = items[j]->to };
+ var_add_value_label (spec->dst, &to_val, value_label);
+ }
+ }
+
+ /* Free array. */
+ free (items);
+ }
+ add_transformation (ds, &autorecode_trns_class, arc);
+
+ for (size_t i = 0; i < n_dsts; i++)
+ free (dst_names[i]);
+ free (dst_names);
+ free (src_vars);
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+
+error:
+ for (size_t i = 0; i < n_dsts; i++)
+ free (dst_names[i]);
+ free (dst_names);
+ free (src_vars);
+ arc_free (arc);
+ return CMD_CASCADING_FAILURE;
+}
+
+static void
+arc_free (struct autorecode_pgm *arc)
+{
+ if (arc != NULL)
+ {
+ for (size_t i = 0; i < arc->n_specs; i++)
+ {
+ struct arc_spec *spec = &arc->specs[i];
+ struct arc_item *item, *next;
+
+ HMAP_FOR_EACH_SAFE (item, next, struct arc_item, hmap_node,
+ &spec->items->ht)
+ {
+ value_destroy (&item->from, item->width);
+ free (item->value_label);
+ hmap_delete (&spec->items->ht, &item->hmap_node);
+ free (item);
+ }
+ free (spec->label);
+ free (spec->src_name);
+ mv_destroy (&spec->mv);
+ }
+
+ size_t n_rec_items =
+ (arc->n_specs >= 2 && arc->specs[0].items == arc->specs[1].items
+ ? 1
+ : arc->n_specs);
+
+ for (size_t i = 0; i < n_rec_items; i++)
+ {
+ struct arc_spec *spec = &arc->specs[i];
+ hmap_destroy (&spec->items->ht);
+ free (spec->items);
+ }
+
+ free (arc->specs);
+ free (arc);
+ }
+}
+
+static struct arc_item *
+find_arc_item (const struct rec_items *items,
+ const union value *value, int width,
+ size_t hash)
+{
+ struct arc_item *item;
+
+ HMAP_FOR_EACH_WITH_HASH (item, struct arc_item, hmap_node, hash, &items->ht)
+ if (item->width == width && value_equal (value, &item->from, width))
+ return item;
+ return NULL;
+}
+
+static int
+compare_arc_items (const void *a_, const void *b_, const void *direction_)
+{
+ const struct arc_item *const *ap = a_;
+ const struct arc_item *const *bp = b_;
+ const struct arc_item *a = *ap;
+ const struct arc_item *b = *bp;
+
+ /* User-missing values always sort to the highest target values
+ (regardless of sort direction). */
+ if (a->missing != b->missing)
+ return a->missing < b->missing ? -1 : 1;
+
+ /* Otherwise, compare the data. */
+ int aw = a->width;
+ int bw = b->width;
+ int cmp;
+ if (aw == bw)
+ cmp = value_compare_3way (&a->from, &b->from, aw);
+ else
+ {
+ assert (aw && bw);
+ cmp = buf_compare_rpad (CHAR_CAST_BUG (const char *, a->from.s), aw,
+ CHAR_CAST_BUG (const char *, b->from.s), bw);
+ }
+
+ /* Then apply sort direction. */
+ const enum arc_direction *directionp = direction_;
+ enum arc_direction direction = *directionp;
+ return direction == ASCENDING ? cmp : -cmp;
+}
+
+static enum trns_result
+autorecode_trns_proc (void *arc_, struct ccase **c,
+ casenumber case_idx UNUSED)
+{
+ struct autorecode_pgm *arc = arc_;
+
+ *c = case_unshare (*c);
+ for (size_t i = 0; i < arc->n_specs; i++)
+ {
+ const struct arc_spec *spec = &arc->specs[i];
+ const union value *value = case_data_idx (*c, spec->src_idx);
+ int width = value_trim_spaces (value, spec->width);
+ size_t hash = value_hash (value, width, 0);
+ const struct arc_item *item = find_arc_item (spec->items, value, width,
+ hash);
+ *case_num_rw (*c, spec->dst) = item ? item->to : SYSMIS;
+ }
+
+ return TRNS_CONTINUE;
+}
+
+static bool
+autorecode_trns_free (void *arc_)
+{
+ struct autorecode_pgm *arc = arc_;
+
+ arc_free (arc);
+ return true;
+}
+
+static const struct trns_class autorecode_trns_class = {
+ .name = "AUTORECODE",
+ .execute = autorecode_trns_proc,
+ .destroy = autorecode_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2009, 2010, 2011, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/binomial.h"
+
+#include <float.h>
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_randist.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value-labels.h"
+#include "data/value.h"
+#include "data/variable.h"
+#include "language/commands/freq.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+#include "gl/minmax.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+static double calculate_binomial_internal (double n1, double n2,
+ double p);
+
+
+static void
+swap (double *i1, double *i2)
+{
+ double temp = *i1;
+ *i1 = *i2;
+ *i2 = temp;
+}
+
+static double
+calculate_binomial (double n1, double n2, double p)
+{
+ const double n = n1 + n2;
+ const bool test_reversed = (n1 / n > p) ;
+ if (test_reversed)
+ {
+ p = 1 - p ;
+ swap (&n1, &n2);
+ }
+
+ return calculate_binomial_internal (n1, n2, p);
+}
+
+static double
+calculate_binomial_internal (double n1, double n2, double p)
+{
+ /* SPSS Statistical Algorithms has completely different and WRONG
+ advice here. */
+
+ double sig1tailed = gsl_cdf_binomial_P (n1, p, n1 + n2);
+
+ if (p == 0.5)
+ return sig1tailed > 0.5 ? 1.0 :sig1tailed * 2.0;
+
+ return sig1tailed ;
+}
+
+static bool
+do_binomial (const struct dictionary *dict,
+ struct casereader *input,
+ const struct one_sample_test *ost,
+ struct freq *cat1,
+ struct freq *cat2,
+ enum mv_class exclude
+ )
+{
+ const struct binomial_test *bst = UP_CAST (ost, const struct binomial_test, parent);
+ bool warn = true;
+
+ struct ccase *c;
+
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ {
+ int v;
+ double w = dict_get_case_weight (dict, c, &warn);
+
+ for (v = 0 ; v < ost->n_vars ; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+ double value = case_num (c, var);
+
+ if (var_is_num_missing (var, value) & exclude)
+ continue;
+
+ if (bst->cutpoint != SYSMIS)
+ {
+ if (cat1[v].values[0].f >= value)
+ cat1[v].count += w;
+ else
+ cat2[v].count += w;
+ }
+ else
+ {
+ if (SYSMIS == cat1[v].values[0].f)
+ {
+ cat1[v].values[0].f = value;
+ cat1[v].count = w;
+ }
+ else if (cat1[v].values[0].f == value)
+ cat1[v].count += w;
+ else if (SYSMIS == cat2[v].values[0].f)
+ {
+ cat2[v].values[0].f = value;
+ cat2[v].count = w;
+ }
+ else if (cat2[v].values[0].f == value)
+ cat2[v].count += w;
+ else if (bst->category1 == SYSMIS)
+ msg (ME, _("Variable %s is not dichotomous"), var_get_name (var));
+ }
+ }
+ }
+ return casereader_destroy (input);
+}
+
+
+
+void
+binomial_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct one_sample_test *ost = UP_CAST (test, const struct one_sample_test, parent);
+ const struct binomial_test *bst = UP_CAST (ost, const struct binomial_test, parent);
+
+ struct freq *cat[2];
+ int i;
+
+ assert ((bst->category1 == SYSMIS) == (bst->category2 == SYSMIS) || bst->cutpoint != SYSMIS);
+
+ for (i = 0; i < 2; i++)
+ {
+ double value;
+ if (i == 0)
+ value = bst->cutpoint != SYSMIS ? bst->cutpoint : bst->category1;
+ else
+ value = bst->category2;
+
+ cat[i] = xnmalloc (ost->n_vars, sizeof *cat[i]);
+ for (size_t v = 0; v < ost->n_vars; v++)
+ {
+ cat[i][v].values[0].f = value;
+ cat[i][v].count = 0;
+ }
+ }
+
+ if (do_binomial (dataset_dict (ds), input, ost, cat[0], cat[1], exclude))
+ {
+ struct pivot_table *table = pivot_table_create (N_("Binomial Test"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Category"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Observed Prop."), PIVOT_RC_OTHER,
+ N_("Test Prop."), PIVOT_RC_OTHER,
+ (bst->p == 0.5
+ ? N_("Exact Sig. (2-tailed)")
+ : N_("Exact Sig. (1-tailed)")), PIVOT_RC_SIGNIFICANCE);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Groups"),
+ N_("Group 1"), N_("Group 2"), N_("Total"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t v = 0; v < ost->n_vars; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (var));
+
+ /* Category. */
+ if (bst->cutpoint != SYSMIS)
+ pivot_table_put3 (
+ table, 0, 0, var_idx,
+ pivot_value_new_user_text_nocopy (
+ xasprintf ("<= %.*g", DBL_DIG + 1, bst->cutpoint)));
+ else
+ for (int i = 0; i < 2; i++)
+ pivot_table_put3 (
+ table, 0, i, var_idx,
+ pivot_value_new_var_value (var, cat[i][v].values));
+
+ double n_total = cat[0][v].count + cat[1][v].count;
+ double sig = calculate_binomial (cat[0][v].count, cat[1][v].count,
+ bst->p);
+ struct entry
+ {
+ int stat_idx;
+ int group_idx;
+ double x;
+ }
+ entries[] = {
+ /* N. */
+ { 1, 0, cat[0][v].count },
+ { 1, 1, cat[1][v].count },
+ { 1, 2, n_total },
+ /* Observed Prop. */
+ { 2, 0, cat[0][v].count / n_total },
+ { 2, 1, cat[1][v].count / n_total },
+ { 2, 2, 1.0 },
+ /* Test Prop. */
+ { 3, 0, bst->p },
+ /* Significance. */
+ { 4, 0, sig }
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put3 (table, e->stat_idx, e->group_idx,
+ var_idx, pivot_value_new_number (e->x));
+ }
+ }
+
+ pivot_table_submit (table);
+ }
+
+ for (i = 0; i < 2; i++)
+ free (cat[i]);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !binomial_h
+#define binomial_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+
+#include "npar.h"
+
+
+struct binomial_test
+{
+ struct one_sample_test parent;
+ double p;
+ double category1;
+ double category2;
+ double cutpoint;
+};
+
+
+struct casereader;
+struct dataset;
+
+
+void binomial_execute (const struct dataset *,
+ struct casereader *,
+ enum mv_class,
+ const struct npar_test *,
+ bool, double);
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <errno.h>
+#include <unistd.h>
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Parses the CACHE command. */
+int
+cmd_cache (struct lexer *lexer UNUSED, struct dataset *ds UNUSED)
+{
+ return CMD_SUCCESS;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2008, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/command.h"
+
+#include <errno.h>
+#include <unistd.h>
+
+#include "language/lexer/lexer.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Parses the CD command. */
+int
+cmd_cd (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ char *path = 0;
+
+ if (! lex_force_string (lexer))
+ goto error;
+
+ path = utf8_to_filename (lex_tokcstr (lexer));
+
+ if (-1 == chdir (path))
+ {
+ int err = errno;
+ lex_error (lexer, _("Cannot change directory to %s: %s"), path,
+ strerror (err));
+ goto error;
+ }
+
+ free (path);
+ lex_get (lexer);
+
+ return CMD_SUCCESS;
+
+ error:
+
+ free(path);
+
+ return CMD_FAILURE;
+}
+
--- /dev/null
+/*
+PSPP - a program for statistical analysis.
+Copyright (C) 2017 Free Software Foundation, Inc.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#ifndef BARCHART_DEF_H
+#define BARCHART_DEF_H 1
+
+#include <stdbool.h>
+
+struct ag_func
+{
+ const char *name;
+ const char *description;
+
+ int arity;
+ bool cumulative;
+ double (*pre) (void);
+ double (*calc) (double acc, double x, double w);
+ double (*post) (double acc, double cc);
+ double (*ppost) (double acc, double ccc);
+};
+
+extern const struct ag_func ag_func[];
+
+extern const int N_AG_FUNCS;
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2007, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/chisquare.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/commands/freq.h"
+#include "language/commands/npar.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/compiler.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/message.h"
+#include "libpspp/taint.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+/* Adds frequency counts of each value of VAR in INPUT between LO and HI to
+ FREQ_HASH. LO and HI and each input value is truncated to an integer.
+ Returns true if successful, false on input error. It is the caller's
+ responsibility to initialize FREQ_HASH and to free it when no longer
+ required, even on failure. */
+static bool
+create_freq_hash_with_range (const struct dictionary *dict,
+ struct casereader *input,
+ const struct variable *var,
+ double lo_, double hi_,
+ struct hmap *freq_hash)
+{
+ struct freq **entries;
+ bool warn = true;
+ struct ccase *c;
+ double lo, hi;
+ double i_d;
+
+ assert (var_is_numeric (var));
+ lo = trunc (lo_);
+ hi = trunc (hi_);
+
+ /* Populate the hash with zero entries */
+ entries = xnmalloc (hi - lo + 1, sizeof *entries);
+ for (i_d = lo; i_d <= hi; i_d += 1.0)
+ {
+ size_t ofs = i_d - lo;
+ union value value = { i_d };
+ entries[ofs] = freq_hmap_insert (freq_hash, &value, 0,
+ value_hash (&value, 0, 0));
+ }
+
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ {
+ double x = trunc (case_num (c, var));
+ if (x >= lo && x <= hi)
+ {
+ size_t ofs = x - lo;
+ struct freq *fr = entries[ofs];
+ fr->count += dict_get_case_weight (dict, c, &warn);
+ }
+ }
+
+ free (entries);
+
+ return casereader_destroy (input);
+}
+
+/* Adds frequency counts of each value of VAR in INPUT to FREQ_HASH. LO and HI
+ and each input value is truncated to an integer. Returns true if
+ successful, false on input error. It is the caller's responsibility to
+ initialize FREQ_HASH and to free it when no longer required, even on
+ failure. */
+static bool
+create_freq_hash (const struct dictionary *dict,
+ struct casereader *input,
+ const struct variable *var,
+ struct hmap *freq_hash)
+{
+ int width = var_get_width (var);
+ bool warn = true;
+ struct ccase *c;
+
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ {
+ const union value *value = case_data (c, var);
+ size_t hash = value_hash (value, width, 0);
+ double weight = dict_get_case_weight (dict, c, &warn);
+ struct freq *f;
+
+ f = freq_hmap_search (freq_hash, value, width, hash);
+ if (f == NULL)
+ f = freq_hmap_insert (freq_hash, value, width, hash);
+
+ f->count += weight;
+ }
+
+ return casereader_destroy (input);
+}
+
+void
+chisquare_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ int v, i;
+ struct chisquare_test *cst = UP_CAST (test, struct chisquare_test,
+ parent.parent);
+ struct one_sample_test *ost = &cst->parent;
+ double total_expected = 0.0;
+
+ double *df = XCALLOC (ost->n_vars, double);
+ double *xsq = XCALLOC (ost->n_vars, double);
+ bool ok;
+
+ for (i = 0 ; i < cst->n_expected ; ++i)
+ total_expected += cst->expected[i];
+
+ if (cst->ranged == false)
+ {
+ for (v = 0 ; v < ost->n_vars ; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+
+ struct hmap freq_hash = HMAP_INITIALIZER (freq_hash);
+ struct casereader *reader =
+ casereader_create_filter_missing (casereader_clone (input),
+ &var, 1, exclude,
+ NULL, NULL);
+ if (!create_freq_hash (dict, reader, var, &freq_hash))
+ {
+ freq_hmap_destroy (&freq_hash, var_get_width (var));
+ return;
+ }
+
+ size_t n_cells = hmap_count (&freq_hash);
+ if (cst->n_expected > 0 && n_cells != cst->n_expected)
+ {
+ msg (ME, _("CHISQUARE test specified %d expected values, but "
+ "variable %s has %zu distinct values."),
+ cst->n_expected, var_get_name (var), n_cells);
+ freq_hmap_destroy (&freq_hash, var_get_width (var));
+ continue;
+ }
+
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_variable (var), "Chisquare");
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Observed N"), PIVOT_RC_COUNT,
+ N_("Expected N"), PIVOT_RC_OTHER,
+ N_("Residual"), PIVOT_RC_RESIDUAL);
+
+ struct freq **ff = freq_hmap_sort (&freq_hash, var_get_width (var));
+
+ double total_obs = 0.0;
+ for (size_t i = 0; i < n_cells; i++)
+ total_obs += ff[i]->count;
+
+ struct pivot_dimension *values = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Value"));
+ values->root->show_label = true;
+
+ xsq[v] = 0.0;
+ for (size_t i = 0; i < n_cells; i++)
+ {
+ int row = pivot_category_create_leaf (
+ values->root, pivot_value_new_var_value (
+ var, &ff[i]->values[0]));
+
+ double exp = (cst->n_expected > 0
+ ? cst->expected[i] * total_obs / total_expected
+ : total_obs / (double) n_cells);
+ double entries[] = {
+ ff[i]->count,
+ exp,
+ ff[i]->count - exp,
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (
+ table, j, row, pivot_value_new_number (entries[j]));
+
+ xsq[v] += (ff[i]->count - exp) * (ff[i]->count - exp) / exp;
+ }
+
+ df[v] = n_cells - 1.0;
+
+ int row = pivot_category_create_leaf (
+ values->root, pivot_value_new_text (N_("Total")));
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_number (total_obs));
+
+ pivot_table_submit (table);
+
+ freq_hmap_destroy (&freq_hash, var_get_width (var));
+ free (ff);
+ }
+ }
+ else /* ranged == true */
+ {
+ struct pivot_table *table = pivot_table_create (N_("Frequencies"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Category"),
+ N_("Observed N"), PIVOT_RC_COUNT,
+ N_("Expected N"), PIVOT_RC_OTHER,
+ N_("Residual"), PIVOT_RC_RESIDUAL);
+
+ struct pivot_dimension *var_dim
+ = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Variable"));
+ for (size_t i = 0 ; i < ost->n_vars ; ++i)
+ pivot_category_create_leaf (var_dim->root,
+ pivot_value_new_variable (ost->vars[i]));
+
+ struct pivot_dimension *category_dim
+ = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Category"));
+ size_t n_cells = cst->hi - cst->lo + 1;
+ for (size_t i = 0 ; i < n_cells; ++i)
+ pivot_category_create_leaf (category_dim->root,
+ pivot_value_new_integer (i + 1));
+ pivot_category_create_leaves (category_dim->root, N_("Total"));
+
+ for (size_t v = 0 ; v < ost->n_vars ; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+ struct casereader *reader =
+ casereader_create_filter_missing (casereader_clone (input),
+ &var, 1, exclude,
+ NULL, NULL);
+ struct hmap freq_hash = HMAP_INITIALIZER (freq_hash);
+ if (!create_freq_hash_with_range (dict, reader, var,
+ cst->lo, cst->hi, &freq_hash))
+ {
+ freq_hmap_destroy (&freq_hash, var_get_width (var));
+ continue;
+ }
+
+ struct freq **ff = freq_hmap_sort (&freq_hash, var_get_width (var));
+
+ double total_obs = 0.0;
+ for (size_t i = 0 ; i < hmap_count (&freq_hash) ; ++i)
+ total_obs += ff[i]->count;
+
+ xsq[v] = 0.0;
+ for (size_t i = 0 ; i < hmap_count (&freq_hash) ; ++i)
+ {
+ /* Category. */
+ pivot_table_put3 (table, 0, v, i,
+ pivot_value_new_var_value (
+ var, &ff[i]->values[0]));
+
+ double exp = (cst->n_expected > 0
+ ? cst->expected[i] * total_obs / total_expected
+ : total_obs / (double) hmap_count (&freq_hash));
+ double entries[] = {
+ ff[i]->count,
+ exp,
+ ff[i]->count - exp,
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put3 (table, j + 1, v, i,
+ pivot_value_new_number (entries[j]));
+
+
+ xsq[v] += (ff[i]->count - exp) * (ff[i]->count - exp) / exp;
+ }
+
+ df[v] = n_cells - 1.0;
+
+ freq_hmap_destroy (&freq_hash, var_get_width (var));
+ free (ff);
+
+ pivot_table_put3 (table, 1, v, n_cells,
+ pivot_value_new_number (total_obs));
+ }
+
+ pivot_table_submit (table);
+ }
+ ok = !taint_has_tainted_successor (casereader_get_taint (input));
+ casereader_destroy (input);
+
+ if (ok)
+ {
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Chi-square"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t v = 0 ; v < ost->n_vars ; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (var));
+
+ double sig = gsl_cdf_chisq_Q (xsq[v], df[v]);
+ double entries[] = { xsq[v], df[v], sig };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, row,
+ pivot_value_new_number (entries[i]));
+ }
+ pivot_table_submit (table);
+ }
+
+ free (xsq);
+ free (df);
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !chisquare_h
+#define chisquare_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+struct chisquare_test
+{
+ struct one_sample_test parent;
+
+ bool ranged ; /* True if this test has a range specified */
+
+ int lo; /* Lower bound of range (undefined if RANGED is false) */
+ int hi; /* Upper bound of range (undefined if RANGED is false) */
+
+ double *expected;
+ int n_expected;
+};
+
+struct casereader;
+struct dataset;
+
+
+void chisquare_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool,
+ double);
+
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/cochran.h"
+
+#include <float.h>
+#include <gsl/gsl_cdf.h>
+#include <stdbool.h>
+
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/val-type.h"
+#include "data/variable.h"
+#include "language/commands/npar.h"
+#include "libpspp/cast.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+struct cochran
+{
+ double success;
+ double failure;
+
+ double *hits;
+ double *misses;
+
+ const struct dictionary *dict;
+ double cc;
+ double df;
+ double q;
+};
+
+static void show_freqs_box (const struct one_sample_test *ost, const struct cochran *ch);
+static void show_sig_box (const struct cochran *ch);
+
+void
+cochran_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED, double timer UNUSED)
+{
+ struct one_sample_test *ct = UP_CAST (test, struct one_sample_test, parent);
+ int v;
+ struct cochran ch;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct variable *weight = dict_get_weight (dict);
+
+ struct ccase *c;
+ double rowsq = 0;
+ ch.cc = 0.0;
+ ch.dict = dict;
+ ch.success = SYSMIS;
+ ch.failure = SYSMIS;
+ ch.hits = xcalloc (ct->n_vars, sizeof *ch.hits);
+ ch.misses = xcalloc (ct->n_vars, sizeof *ch.misses);
+
+ for (; (c = casereader_read (input)); case_unref (c))
+ {
+ double case_hits = 0.0;
+ const double w = weight ? case_num (c, weight) : 1.0;
+ for (v = 0; v < ct->n_vars; ++v)
+ {
+ const struct variable *var = ct->vars[v];
+ const union value *val = case_data (c, var);
+
+ if (var_is_value_missing (var, val) & exclude)
+ continue;
+
+ if (ch.success == SYSMIS)
+ {
+ ch.success = val->f;
+ }
+ else if (ch.failure == SYSMIS && val->f != ch.success)
+ {
+ ch.failure = val->f;
+ }
+ if (ch.success == val->f)
+ {
+ ch.hits[v] += w;
+ case_hits += w;
+ }
+ else if (ch.failure == val->f)
+ {
+ ch.misses[v] += w;
+ }
+ else
+ {
+ msg (MW, _("More than two values encountered. Cochran Q test will not be run."));
+ goto finish;
+ }
+ }
+ ch.cc += w;
+ rowsq += pow2 (case_hits);
+ }
+ casereader_destroy (input);
+
+ {
+ double c_l = 0;
+ double c_l2 = 0;
+ for (v = 0; v < ct->n_vars; ++v)
+ {
+ c_l += ch.hits[v];
+ c_l2 += pow2 (ch.hits[v]);
+ }
+
+ ch.q = ct->n_vars * c_l2;
+ ch.q -= pow2 (c_l);
+ ch.q *= ct->n_vars - 1;
+
+ ch.q /= ct->n_vars * c_l - rowsq;
+
+ ch.df = ct->n_vars - 1;
+ }
+
+ show_freqs_box (ct, &ch);
+ show_sig_box (&ch);
+
+ finish:
+
+ free (ch.hits);
+ free (ch.misses);
+}
+
+static void
+show_freqs_box (const struct one_sample_test *ost, const struct cochran *ct)
+{
+ struct pivot_table *table = pivot_table_create (N_("Frequencies"));
+ pivot_table_set_weight_var (table, dict_get_weight (ct->dict));
+
+ char *success = xasprintf (_("Success (%.*g)"), DBL_DIG + 1, ct->success);
+ char *failure = xasprintf (_("Failure (%.*g)"), DBL_DIG + 1, ct->failure);
+ struct pivot_dimension *values = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Value"),
+ success, PIVOT_RC_COUNT,
+ failure, PIVOT_RC_COUNT);
+ values->root->show_label = true;
+ free (failure);
+ free (success);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t i = 0 ; i < ost->n_vars ; ++i)
+ {
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (ost->vars[i]));
+
+ pivot_table_put2 (table, 0, row, pivot_value_new_number (ct->hits[i]));
+ pivot_table_put2 (table, 1, row, pivot_value_new_number (ct->misses[i]));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_sig_box (const struct cochran *ch)
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+
+ pivot_table_set_weight_format (table, dict_get_weight_format (ch->dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Value"), N_("Value"));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Cochran's Q"), PIVOT_RC_SIGNIFICANCE,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ double sig = gsl_cdf_chisq_Q (ch->q, ch->df);
+ double entries[] = { ch->cc, ch->q, ch->df, sig };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, 0, i, pivot_value_new_number (entries[i]));
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !cochran_h
+#define cochran_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+
+
+void cochran_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool,
+ double);
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/any-reader.h"
+#include "data/case-matcher.h"
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/trim.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/sort-criteria.h"
+#include "libpspp/assertion.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/string-array.h"
+#include "libpspp/taint.h"
+#include "math/sort.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+enum comb_command_type
+ {
+ COMB_ADD,
+ COMB_MATCH,
+ COMB_UPDATE
+ };
+
+/* File types. */
+enum comb_file_type
+ {
+ COMB_FILE, /* Specified on FILE= subcommand. */
+ COMB_TABLE /* Specified on TABLE= subcommand. */
+ };
+
+/* One FILE or TABLE subcommand. */
+struct comb_file
+ {
+ /* Basics. */
+ enum comb_file_type type; /* COMB_FILE or COMB_TABLE. */
+ int start_ofs, end_ofs; /* Lexer offsets. */
+
+ /* Variables. */
+ struct subcase by_vars; /* BY variables in this input file. */
+ struct subcase src, dst; /* Data to copy to output; where to put it. */
+ const struct missing_values **mv; /* Each variable's missing values. */
+
+ /* Input files. */
+ struct file_handle *handle; /* Input file handle. */
+ struct dictionary *dict; /* Input file dictionary. */
+ struct casereader *reader; /* Input data source. */
+ struct ccase *data; /* The current input case. */
+ bool is_minimal; /* Does 'data' have minimum BY values across
+ all input files? */
+ bool is_sorted; /* Is file presorted on the BY variables? */
+
+ /* IN subcommand. */
+ char *in_name;
+ int in_ofs;
+ struct variable *in_var;
+ };
+
+struct comb_proc
+ {
+ struct comb_file *files; /* All the files being merged. */
+ size_t n_files; /* Number of files. */
+
+ struct dictionary *dict; /* Dictionary of output file. */
+ struct subcase by_vars; /* BY variables in the output. */
+ struct casewriter *output; /* Destination for output. */
+
+ size_t *var_sources;
+ size_t n_var_sources, allocated_var_sources;
+
+ struct case_matcher *matcher;
+
+ /* FIRST, LAST.
+ Only if "first" or "last" is nonnull are the remaining
+ members used. */
+ struct variable *first; /* Variable specified on FIRST (if any). */
+ struct variable *last; /* Variable specified on LAST (if any). */
+ struct ccase *buffered_case; /* Case ready for output except that we don't
+ know the value for the LAST var yet. */
+ union value *prev_BY; /* Values of BY vars in buffered_case. */
+ };
+
+static int combine_files (enum comb_command_type, struct lexer *,
+ struct dataset *);
+static void free_comb_proc (struct comb_proc *);
+
+static void close_all_comb_files (struct comb_proc *);
+static bool merge_dictionary (struct comb_proc *, struct lexer *,
+ struct comb_file *);
+
+static void execute_update (struct comb_proc *);
+static void execute_match_files (struct comb_proc *);
+static void execute_add_files (struct comb_proc *);
+
+static bool create_flag_var (struct lexer *lexer, const char *subcommand_name,
+ const char *var_name, int var_ofs,
+ struct dictionary *, struct variable **);
+static void output_case (struct comb_proc *, struct ccase *, union value *by);
+static void output_buffered_case (struct comb_proc *);
+
+int
+cmd_add_files (struct lexer *lexer, struct dataset *ds)
+{
+ return combine_files (COMB_ADD, lexer, ds);
+}
+
+int
+cmd_match_files (struct lexer *lexer, struct dataset *ds)
+{
+ return combine_files (COMB_MATCH, lexer, ds);
+}
+
+int
+cmd_update (struct lexer *lexer, struct dataset *ds)
+{
+ return combine_files (COMB_UPDATE, lexer, ds);
+}
+
+static int
+combine_files (enum comb_command_type command,
+ struct lexer *lexer, struct dataset *ds)
+{
+ struct comb_proc proc = {
+ .dict = dict_create (get_default_encoding ()),
+ };
+
+ bool saw_by = false;
+ bool saw_sort = false;
+ struct casereader *active_file = NULL;
+
+ char *first_name = NULL;
+ int first_ofs = 0;
+ char *last_name = NULL;
+ int last_ofs = 0;
+
+ struct taint *taint = NULL;
+
+ size_t table_idx = SIZE_MAX;
+ int sort_ofs = INT_MAX;
+ size_t allocated_files = 0;
+
+ dict_set_case_limit (proc.dict, dict_get_case_limit (dataset_dict (ds)));
+
+ lex_match (lexer, T_SLASH);
+ for (;;)
+ {
+ int start_ofs = lex_ofs (lexer);
+ enum comb_file_type type;
+ if (lex_match_id (lexer, "FILE"))
+ type = COMB_FILE;
+ else if (command == COMB_MATCH && lex_match_id (lexer, "TABLE"))
+ {
+ type = COMB_TABLE;
+ table_idx = MIN (table_idx, proc.n_files);
+ }
+ else
+ break;
+ lex_match (lexer, T_EQUALS);
+
+ if (proc.n_files >= allocated_files)
+ proc.files = x2nrealloc (proc.files, &allocated_files,
+ sizeof *proc.files);
+ struct comb_file *file = &proc.files[proc.n_files++];
+ *file = (struct comb_file) {
+ .type = type,
+ .start_ofs = start_ofs,
+ .is_sorted = true,
+ };
+
+ if (lex_match (lexer, T_ASTERISK))
+ {
+ if (!dataset_has_source (ds))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Cannot specify the active dataset since none "
+ "has been defined."));
+ goto error;
+ }
+
+ if (proc_make_temporary_transformations_permanent (ds))
+ lex_next_error (lexer, -1, -1,
+ _("This command may not be used after TEMPORARY "
+ "when the active dataset is an input source. "
+ "Temporary transformations will be made "
+ "permanent."));
+
+ file->dict = dict_clone (dataset_dict (ds));
+ }
+ else
+ {
+ file->handle = fh_parse (lexer, FH_REF_FILE, dataset_session (ds));
+ if (file->handle == NULL)
+ goto error;
+
+ file->reader = any_reader_open_and_decode (file->handle, NULL,
+ &file->dict, NULL);
+ if (file->reader == NULL)
+ goto error;
+ }
+ file->end_ofs = lex_ofs (lexer) - 1;
+
+ while (lex_match (lexer, T_SLASH))
+ if (lex_match_id (lexer, "RENAME"))
+ {
+ if (!parse_dict_rename (lexer, file->dict))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "IN"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_id (lexer))
+ goto error;
+
+ if (file->in_name)
+ {
+ lex_error (lexer, _("Multiple IN subcommands for a single FILE "
+ "or TABLE."));
+ goto error;
+ }
+ file->in_name = xstrdup (lex_tokcstr (lexer));
+ file->in_ofs = lex_ofs (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "SORT"))
+ {
+ file->is_sorted = false;
+ saw_sort = true;
+ sort_ofs = MIN (sort_ofs, lex_ofs (lexer) - 1);
+ }
+
+ if (!merge_dictionary (&proc, lexer, file))
+ goto error;
+ }
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match (lexer, T_BY))
+ {
+ if (saw_by)
+ {
+ lex_sbc_only_once (lexer, "BY");
+ goto error;
+ }
+ saw_by = true;
+
+ lex_match (lexer, T_EQUALS);
+
+ const struct variable **by_vars;
+ if (!parse_sort_criteria (lexer, proc.dict, &proc.by_vars,
+ &by_vars, NULL))
+ goto error;
+
+ bool ok = true;
+ for (size_t i = 0; i < proc.n_files; i++)
+ {
+ struct comb_file *file = &proc.files[i];
+ for (size_t j = 0; j < subcase_get_n_fields (&proc.by_vars); j++)
+ {
+ const char *name = var_get_name (by_vars[j]);
+ struct variable *var = dict_lookup_var (file->dict, name);
+ if (var != NULL)
+ subcase_add_var (&file->by_vars, var,
+ subcase_get_direction (&proc.by_vars, j));
+ else
+ {
+ const char *fn
+ = file->handle ? fh_get_name (file->handle) : "*";
+ lex_ofs_error (lexer, file->start_ofs, file->end_ofs,
+ _("File %s lacks BY variable %s."),
+ fn, name);
+ ok = false;
+ }
+ }
+ assert (!ok || subcase_conformable (&file->by_vars,
+ &proc.files[0].by_vars));
+ }
+ free (by_vars);
+
+ if (!ok)
+ goto error;
+ }
+ else if (command != COMB_UPDATE && lex_match_id (lexer, "FIRST"))
+ {
+ if (first_name != NULL)
+ {
+ lex_sbc_only_once (lexer, "FIRST");
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_id (lexer))
+ goto error;
+ first_name = xstrdup (lex_tokcstr (lexer));
+ first_ofs = lex_ofs (lexer);
+ lex_get (lexer);
+ }
+ else if (command != COMB_UPDATE && lex_match_id (lexer, "LAST"))
+ {
+ if (last_name != NULL)
+ {
+ lex_sbc_only_once (lexer, "LAST");
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_id (lexer))
+ goto error;
+ last_name = xstrdup (lex_tokcstr (lexer));
+ last_ofs = lex_ofs (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "MAP"))
+ {
+ /* FIXME. */
+ }
+ else if (lex_match_id (lexer, "DROP"))
+ {
+ if (!parse_dict_drop (lexer, proc.dict))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "KEEP"))
+ {
+ if (!parse_dict_keep (lexer, proc.dict))
+ goto error;
+ }
+ else
+ {
+ if (command == COMB_UPDATE)
+ lex_error_expecting (lexer, "BY", "MAP", "DROP", "KEEP");
+ else
+ lex_error_expecting (lexer, "BY", "FIRST", "LAST",
+ "MAP", "DROP", "KEEP");
+ goto error;
+ }
+
+ if (!lex_match (lexer, T_SLASH) && lex_token (lexer) != T_ENDCMD)
+ {
+ lex_end_of_command (lexer);
+ goto error;
+ }
+ }
+
+ if (!saw_by)
+ {
+ if (command == COMB_UPDATE)
+ {
+ lex_sbc_missing (lexer, "BY");
+ goto error;
+ }
+ if (table_idx != SIZE_MAX)
+ {
+ const struct comb_file *table = &proc.files[table_idx];
+ lex_ofs_error (lexer, table->start_ofs, table->end_ofs,
+ _("BY is required when %s is specified."), "TABLE");
+ goto error;
+ }
+ if (saw_sort)
+ {
+ lex_ofs_error (lexer, sort_ofs, sort_ofs,
+ _("BY is required when %s is specified."), "SORT");
+ goto error;
+ }
+ }
+
+ /* Add IN, FIRST, and LAST variables to master dictionary. */
+ for (size_t i = 0; i < proc.n_files; i++)
+ {
+ struct comb_file *file = &proc.files[i];
+ if (!create_flag_var (lexer, "IN", file->in_name, file->in_ofs,
+ proc.dict, &file->in_var))
+ goto error;
+ }
+ if (!create_flag_var (lexer, "FIRST", first_name, first_ofs, proc.dict, &proc.first)
+ || !create_flag_var (lexer, "LAST", last_name, last_ofs, proc.dict, &proc.last))
+ goto error;
+
+ dict_delete_scratch_vars (proc.dict);
+ dict_compact_values (proc.dict);
+
+ /* Set up mapping from each file's variables to master
+ variables. */
+ for (size_t i = 0; i < proc.n_files; i++)
+ {
+ struct comb_file *file = &proc.files[i];
+ size_t src_n_vars = dict_get_n_vars (file->dict);
+
+ file->mv = xnmalloc (src_n_vars, sizeof *file->mv);
+ for (size_t j = 0; j < src_n_vars; j++)
+ {
+ struct variable *src_var = dict_get_var (file->dict, j);
+ struct variable *dst_var = dict_lookup_var (proc.dict,
+ var_get_name (src_var));
+ if (dst_var != NULL)
+ {
+ size_t n = subcase_get_n_fields (&file->src);
+ file->mv[n] = var_get_missing_values (src_var);
+ subcase_add_var (&file->src, src_var, SC_ASCEND);
+ subcase_add_var (&file->dst, dst_var, SC_ASCEND);
+ }
+ }
+ }
+
+ proc.output = autopaging_writer_create (dict_get_proto (proc.dict));
+ taint = taint_clone (casewriter_get_taint (proc.output));
+
+ /* Set up case matcher. */
+ proc.matcher = case_matcher_create ();
+ for (size_t i = 0; i < proc.n_files; i++)
+ {
+ struct comb_file *file = &proc.files[i];
+ if (file->reader == NULL)
+ {
+ if (active_file == NULL)
+ {
+ proc_discard_output (ds);
+ file->reader = active_file = proc_open_filtering (ds, false);
+ }
+ else
+ file->reader = casereader_clone (active_file);
+ }
+ if (!file->is_sorted)
+ file->reader = sort_execute (file->reader, &file->by_vars);
+ taint_propagate (casereader_get_taint (file->reader), taint);
+ file->data = casereader_read (file->reader);
+ if (file->type == COMB_FILE)
+ case_matcher_add_input (proc.matcher, &file->by_vars,
+ &file->data, &file->is_minimal);
+ }
+
+ if (command == COMB_ADD)
+ execute_add_files (&proc);
+ else if (command == COMB_MATCH)
+ execute_match_files (&proc);
+ else if (command == COMB_UPDATE)
+ execute_update (&proc);
+ else
+ NOT_REACHED ();
+
+ case_matcher_destroy (proc.matcher);
+ proc.matcher = NULL;
+ close_all_comb_files (&proc);
+ if (active_file != NULL)
+ proc_commit (ds);
+
+ dataset_set_dict (ds, proc.dict);
+ dataset_set_source (ds, casewriter_make_reader (proc.output));
+ proc.dict = NULL;
+ proc.output = NULL;
+
+ free_comb_proc (&proc);
+
+ free (first_name);
+ free (last_name);
+
+ return taint_destroy (taint) ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+
+ error:
+ if (active_file != NULL)
+ proc_commit (ds);
+ free_comb_proc (&proc);
+ taint_destroy (taint);
+ free (first_name);
+ free (last_name);
+ return CMD_CASCADING_FAILURE;
+}
+
+/* Merge the dictionary for file F into master dictionary for PROC. */
+static bool
+merge_dictionary (struct comb_proc *proc, struct lexer *lexer,
+ struct comb_file *f)
+{
+ struct dictionary *m = proc->dict;
+ struct dictionary *d = f->dict;
+
+ if (dict_get_label (m) == NULL)
+ dict_set_label (m, dict_get_label (d));
+
+ /* FIXME: If the input files have different encodings, then
+ the result is undefined.
+ The correct thing to do would be to convert to an encoding
+ which can cope with all the input files (eg UTF-8).
+ */
+ if (strcmp (dict_get_encoding (f->dict), dict_get_encoding (m)))
+ msg (MW, _("Combining files with incompatible encodings. String data may "
+ "not be represented correctly."));
+
+ const struct string_array *d_docs = dict_get_documents (d);
+ const struct string_array *m_docs = dict_get_documents (m);
+ if (d_docs)
+ {
+ if (!m_docs)
+ dict_set_documents (m, d_docs);
+ else
+ {
+ size_t n = m_docs->n + d_docs->n;
+ struct string_array new_docs = {
+ .strings = xmalloc (n * sizeof *new_docs.strings),
+ };
+ for (size_t i = 0; i < m_docs->n; i++)
+ new_docs.strings[new_docs.n++] = m_docs->strings[i];
+ for (size_t i = 0; i < d_docs->n; i++)
+ new_docs.strings[new_docs.n++] = d_docs->strings[i];
+
+ dict_set_documents (m, &new_docs);
+
+ free (new_docs.strings);
+ }
+ }
+
+ for (size_t i = 0; i < dict_get_n_vars (d); i++)
+ {
+ struct variable *dv = dict_get_var (d, i);
+ struct variable *mv = dict_lookup_var (m, var_get_name (dv));
+
+ if (dict_class_from_id (var_get_name (dv)) == DC_SCRATCH)
+ continue;
+
+ if (!mv)
+ {
+ mv = dict_clone_var_assert (m, dv);
+ if (proc->n_var_sources >= proc->allocated_var_sources)
+ proc->var_sources = x2nrealloc (proc->var_sources,
+ &proc->allocated_var_sources,
+ sizeof *proc->var_sources);
+ proc->var_sources[proc->n_var_sources++] = f - proc->files;
+ }
+ else
+ {
+ if (var_get_width (mv) != var_get_width (dv))
+ {
+ const char *var_name = var_get_name (dv);
+ msg (SE, _("Variable %s has different type or width in different "
+ "files."), var_name);
+
+ for (size_t j = 0; j < 2; j++)
+ {
+ const struct variable *ev = !j ? mv : dv;
+ const struct comb_file *ef
+ = !j ? &proc->files[proc->var_sources[var_get_dict_index (mv)]] : f;
+ const char *fn = ef->handle ? fh_get_name (ef->handle) : "*";
+
+ if (var_is_numeric (ev))
+ lex_ofs_msg (lexer, SN, ef->start_ofs, ef->end_ofs,
+ _("In file %s, %s is numeric."),
+ fn, var_name);
+ else
+ lex_ofs_msg (lexer, SN, ef->start_ofs, ef->end_ofs,
+ _("In file %s, %s is a string with width %d."),
+ fn, var_name, var_get_width (ev));
+ }
+
+ return false;
+ }
+
+ if (var_has_value_labels (dv) && !var_has_value_labels (mv))
+ var_set_value_labels (mv, var_get_value_labels (dv));
+ if (var_has_missing_values (dv) && !var_has_missing_values (mv))
+ var_set_missing_values (mv, var_get_missing_values (dv));
+ if (var_get_label (dv) && !var_get_label (mv))
+ var_set_label (mv, var_get_label (dv));
+ }
+ }
+
+ return true;
+}
+
+/* If VAR_NAME is non-NULL, attempts to create a
+ variable named VAR_NAME, with format F1.0, in DICT, and stores
+ a pointer to the variable in *VAR. Returns true if
+ successful, false if the variable name is a duplicate (in
+ which case a message saying that the variable specified on the
+ given SUBCOMMAND is a duplicate is emitted).
+
+ Does nothing and returns true if VAR_NAME is null. */
+static bool
+create_flag_var (struct lexer *lexer, const char *subcommand,
+ const char *var_name, int var_ofs,
+ struct dictionary *dict, struct variable **var)
+{
+ if (var_name != NULL)
+ {
+ struct fmt_spec format = fmt_for_output (FMT_F, 1, 0);
+ *var = dict_create_var (dict, var_name, 0);
+ if (*var == NULL)
+ {
+ lex_ofs_error (lexer, var_ofs, var_ofs,
+ _("Variable name %s specified on %s subcommand "
+ "duplicates an existing variable name."),
+ var_name, subcommand);
+ return false;
+ }
+ var_set_both_formats (*var, &format);
+ }
+ else
+ *var = NULL;
+ return true;
+}
+
+/* Closes all the files in PROC and frees their associated data. */
+static void
+close_all_comb_files (struct comb_proc *proc)
+{
+ for (size_t i = 0; i < proc->n_files; i++)
+ {
+ struct comb_file *file = &proc->files[i];
+ subcase_uninit (&file->by_vars);
+ subcase_uninit (&file->src);
+ subcase_uninit (&file->dst);
+ free (file->mv);
+ fh_unref (file->handle);
+ dict_unref (file->dict);
+ casereader_destroy (file->reader);
+ case_unref (file->data);
+ free (file->in_name);
+ }
+ free (proc->files);
+ proc->files = NULL;
+ proc->n_files = 0;
+}
+
+/* Frees all the data for the procedure. */
+static void
+free_comb_proc (struct comb_proc *proc)
+{
+ close_all_comb_files (proc);
+ dict_unref (proc->dict);
+ casewriter_destroy (proc->output);
+ case_matcher_destroy (proc->matcher);
+ if (proc->prev_BY)
+ {
+ caseproto_destroy_values (subcase_get_proto (&proc->by_vars),
+ proc->prev_BY);
+ free (proc->prev_BY);
+ }
+ subcase_uninit (&proc->by_vars);
+ case_unref (proc->buffered_case);
+ free (proc->var_sources);
+}
+\f
+static bool scan_table (struct comb_file *, union value by[]);
+static struct ccase *create_output_case (const struct comb_proc *);
+static void apply_case (const struct comb_file *, struct ccase *);
+static void apply_nonmissing_case (const struct comb_file *, struct ccase *);
+static void advance_file (struct comb_file *, union value by[]);
+static void output_case (struct comb_proc *, struct ccase *, union value by[]);
+static void output_buffered_case (struct comb_proc *);
+
+/* Executes the ADD FILES command. */
+static void
+execute_add_files (struct comb_proc *proc)
+{
+ union value *by;
+
+ while (case_matcher_match (proc->matcher, &by))
+ for (size_t i = 0; i < proc->n_files; i++)
+ {
+ struct comb_file *file = &proc->files[i];
+ while (file->is_minimal)
+ {
+ struct ccase *output = create_output_case (proc);
+ apply_case (file, output);
+ advance_file (file, by);
+ output_case (proc, output, by);
+ }
+ }
+ output_buffered_case (proc);
+}
+
+/* Executes the MATCH FILES command. */
+static void
+execute_match_files (struct comb_proc *proc)
+{
+ union value *by;
+
+ while (case_matcher_match (proc->matcher, &by))
+ {
+ struct ccase *output = create_output_case (proc);
+ for (size_t i = proc->n_files; i-- > 0;)
+ {
+ struct comb_file *file = &proc->files[i];
+ if (file->type == COMB_FILE)
+ {
+ if (file->is_minimal)
+ {
+ apply_case (file, output);
+ advance_file (file, NULL);
+ }
+ }
+ else
+ {
+ if (scan_table (file, by))
+ apply_case (file, output);
+ }
+ }
+ output_case (proc, output, by);
+ }
+ output_buffered_case (proc);
+}
+
+/* Executes the UPDATE command. */
+static void
+execute_update (struct comb_proc *proc)
+{
+ union value *by;
+ size_t n_duplicates = 0;
+
+ while (case_matcher_match (proc->matcher, &by))
+ {
+ struct comb_file *first, *file;
+ struct ccase *output;
+
+ /* Find first nonnull case in array and make an output case
+ from it. */
+ output = create_output_case (proc);
+ for (first = &proc->files[0]; ; first++)
+ if (first->is_minimal)
+ break;
+ apply_case (first, output);
+ advance_file (first, by);
+
+ /* Read additional cases and update the output case from
+ them. (Don't update the output case from any duplicate
+ cases in the master file.) */
+ for (file = first + (first == proc->files);
+ file < &proc->files[proc->n_files]; file++)
+ {
+ while (file->is_minimal)
+ {
+ apply_nonmissing_case (file, output);
+ advance_file (file, by);
+ }
+ }
+ casewriter_write (proc->output, output);
+
+ /* Write duplicate cases in the master file directly to the
+ output. */
+ if (first == proc->files && first->is_minimal)
+ {
+ n_duplicates++;
+ while (first->is_minimal)
+ {
+ output = create_output_case (proc);
+ apply_case (first, output);
+ advance_file (first, by);
+ casewriter_write (proc->output, output);
+ }
+ }
+ }
+
+ if (n_duplicates)
+ msg (SW, _("Encountered %zu sets of duplicate cases in the master file."),
+ n_duplicates);
+}
+
+/* Reads FILE, which must be of type COMB_TABLE, until it
+ encounters a case with BY or greater for its BY variables.
+ Returns true if a case with exactly BY for its BY variables
+ was found, otherwise false. */
+static bool
+scan_table (struct comb_file *file, union value by[])
+{
+ while (file->data != NULL)
+ {
+ int cmp = subcase_compare_3way_xc (&file->by_vars, by, file->data);
+ if (cmp > 0)
+ {
+ case_unref (file->data);
+ file->data = casereader_read (file->reader);
+ }
+ else
+ return cmp == 0;
+ }
+ return false;
+}
+
+/* Creates and returns an output case for PROC, initializing each
+ of its values to system-missing or blanks, except that the
+ values of IN variables are set to 0. */
+static struct ccase *
+create_output_case (const struct comb_proc *proc)
+{
+ size_t n_vars = dict_get_n_vars (proc->dict);
+ struct ccase *output = case_create (dict_get_proto (proc->dict));
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ struct variable *v = dict_get_var (proc->dict, i);
+ value_set_missing (case_data_rw (output, v), var_get_width (v));
+ }
+ for (size_t i = 0; i < proc->n_files; i++)
+ {
+ struct comb_file *file = &proc->files[i];
+ if (file->in_var != NULL)
+ *case_num_rw (output, file->in_var) = false;
+ }
+ return output;
+}
+
+static void
+mark_file_used (const struct comb_file *file, struct ccase *output)
+{
+ if (file->in_var != NULL)
+ *case_num_rw (output, file->in_var) = true;
+}
+
+/* Copies the data from FILE's case into output case OUTPUT.
+ If FILE has an IN variable, then it is set to 1 in OUTPUT. */
+static void
+apply_case (const struct comb_file *file, struct ccase *output)
+{
+ subcase_copy (&file->src, file->data, &file->dst, output);
+ mark_file_used (file, output);
+}
+
+/* Copies the data from FILE's case into output case OUTPUT,
+ skipping values that are missing or all spaces.
+
+ If FILE has an IN variable, then it is set to 1 in OUTPUT. */
+static void
+apply_nonmissing_case (const struct comb_file *file, struct ccase *output)
+{
+ for (size_t i = 0; i < subcase_get_n_fields (&file->src); i++)
+ {
+ const struct subcase_field *src_field = &file->src.fields[i];
+ const struct subcase_field *dst_field = &file->dst.fields[i];
+ const union value *src_value
+ = case_data_idx (file->data, src_field->case_index);
+ int width = src_field->width;
+
+ if (!mv_is_value_missing (file->mv[i], src_value)
+ && !(width > 0 && value_is_spaces (src_value, width)))
+ value_copy (case_data_rw_idx (output, dst_field->case_index),
+ src_value, width);
+ }
+ mark_file_used (file, output);
+}
+
+/* Advances FILE to its next case. If BY is nonnull, then FILE's is_minimal
+ member is updated based on whether the new case's BY values still match
+ those in BY. */
+static void
+advance_file (struct comb_file *file, union value by[])
+{
+ case_unref (file->data);
+ file->data = casereader_read (file->reader);
+ if (by)
+ file->is_minimal = (file->data != NULL
+ && subcase_equal_cx (&file->by_vars, file->data, by));
+}
+
+/* Writes OUTPUT, whose BY values has been extracted into BY, to
+ PROC's output file, first initializing any FIRST or LAST
+ variables in OUTPUT to the correct values. */
+static void
+output_case (struct comb_proc *proc, struct ccase *output, union value by[])
+{
+ if (proc->first == NULL && proc->last == NULL)
+ casewriter_write (proc->output, output);
+ else
+ {
+ /* It's harder with LAST, because we can't know whether
+ this case is the last in a group until we've prepared
+ the *next* case also. Thus, we buffer the previous
+ output case until the next one is ready. */
+ bool new_BY;
+ if (proc->prev_BY != NULL)
+ {
+ new_BY = !subcase_equal_xx (&proc->by_vars, proc->prev_BY, by);
+ if (proc->last != NULL)
+ *case_num_rw (proc->buffered_case, proc->last) = new_BY;
+ casewriter_write (proc->output, proc->buffered_case);
+ }
+ else
+ new_BY = true;
+
+ proc->buffered_case = output;
+ if (proc->first != NULL)
+ *case_num_rw (proc->buffered_case, proc->first) = new_BY;
+
+ if (new_BY)
+ {
+ size_t n_values = subcase_get_n_fields (&proc->by_vars);
+ const struct caseproto *proto = subcase_get_proto (&proc->by_vars);
+ if (proc->prev_BY == NULL)
+ {
+ proc->prev_BY = xmalloc (n_values * sizeof *proc->prev_BY);
+ caseproto_init_values (proto, proc->prev_BY);
+ }
+ caseproto_copy (subcase_get_proto (&proc->by_vars), 0, n_values,
+ proc->prev_BY, by);
+ }
+ }
+}
+
+/* Writes a trailing buffered case to the output, if FIRST or
+ LAST is in use. */
+static void
+output_buffered_case (struct comb_proc *proc)
+{
+ if (proc->prev_BY != NULL)
+ {
+ if (proc->last != NULL)
+ *case_num_rw (proc->buffered_case, proc->last) = 1.0;
+ casewriter_write (proc->output, proc->buffered_case);
+ proc->buffered_case = NULL;
+ }
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <stdint.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "data/vector.h"
+#include "language/command.h"
+#include "language/expressions/public.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+struct compute_trns;
+struct lvalue;
+
+/* COMPUTE or IF target variable or vector element.
+ For a variable, the `variable' member is non-null.
+ For a vector element, the `vector' member is non-null. */
+struct lvalue
+ {
+ struct msg_location *location; /* Syntax for variable or vector. */
+
+ struct variable *variable; /* Destination variable. */
+ bool is_new_variable; /* Did we create the variable? */
+
+ const struct vector *vector; /* Destination vector, if any, or NULL. */
+ struct expression *element; /* Destination vector element, or NULL. */
+ };
+
+/* Target of a COMPUTE or IF assignment, either a variable or a
+ vector element. */
+static struct lvalue *lvalue_parse (struct lexer *lexer, struct dataset *);
+static int lvalue_get_type (const struct lvalue *);
+static bool lvalue_is_vector (const struct lvalue *);
+static void lvalue_finalize (struct lvalue *,
+ struct compute_trns *, struct dictionary *);
+static void lvalue_destroy (struct lvalue *, struct dictionary *);
+
+/* COMPUTE and IF transformation. */
+struct compute_trns
+ {
+ /* Test expression (IF only). */
+ struct expression *test; /* Test expression. */
+
+ /* Variable lvalue, if variable != NULL. */
+ struct variable *variable; /* Destination variable, if any. */
+ int width; /* Lvalue string width; 0=numeric. */
+
+ /* Vector lvalue, if vector != NULL. */
+ const struct vector *vector; /* Destination vector, if any. */
+ struct expression *element; /* Destination vector element expr. */
+
+ struct msg_location *lvalue_location;
+
+ /* Rvalue. */
+ struct expression *rvalue; /* Rvalue expression. */
+ };
+
+static struct expression *parse_rvalue (struct lexer *lexer,
+ const struct lvalue *,
+ struct dataset *);
+
+static struct compute_trns *compute_trns_create (void);
+static bool compute_trns_free (void *compute_);
+static const struct trns_class *get_trns_class (const struct lvalue *);
+\f
+/* COMPUTE. */
+
+int
+cmd_compute (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct lvalue *lvalue = NULL;
+ struct compute_trns *compute = NULL;
+
+ compute = compute_trns_create ();
+
+ lvalue = lvalue_parse (lexer, ds);
+ if (lvalue == NULL)
+ goto fail;
+
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto fail;
+ compute->rvalue = parse_rvalue (lexer, lvalue, ds);
+ if (compute->rvalue == NULL)
+ goto fail;
+
+ add_transformation (ds, get_trns_class (lvalue), compute);
+
+ lvalue_finalize (lvalue, compute, dict);
+
+ return CMD_SUCCESS;
+
+ fail:
+ lvalue_destroy (lvalue, dict);
+ compute_trns_free (compute);
+ return CMD_CASCADING_FAILURE;
+}
+\f
+/* Transformation functions. */
+
+/* Handle COMPUTE or IF with numeric target variable. */
+static enum trns_result
+compute_num (void *compute_, struct ccase **c, casenumber case_num)
+{
+ struct compute_trns *compute = compute_;
+
+ if (compute->test == NULL
+ || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
+ {
+ *c = case_unshare (*c);
+ *case_num_rw (*c, compute->variable)
+ = expr_evaluate_num (compute->rvalue, *c, case_num);
+ }
+
+ return TRNS_CONTINUE;
+}
+
+/* Handle COMPUTE or IF with numeric vector element target
+ variable. */
+static enum trns_result
+compute_num_vec (void *compute_, struct ccase **c, casenumber case_num)
+{
+ struct compute_trns *compute = compute_;
+
+ if (compute->test == NULL
+ || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
+ {
+ double index; /* Index into the vector. */
+ int rindx; /* Rounded index value. */
+
+ index = expr_evaluate_num (compute->element, *c, case_num);
+ rindx = floor (index + EPSILON);
+ if (index == SYSMIS
+ || rindx < 1 || rindx > vector_get_n_vars (compute->vector))
+ {
+ if (index == SYSMIS)
+ msg_at (SW, compute->lvalue_location,
+ _("When executing COMPUTE: SYSMIS is not a valid value "
+ "as an index into vector %s."),
+ vector_get_name (compute->vector));
+ else
+ msg_at (SW, compute->lvalue_location,
+ _("When executing COMPUTE: %.*g is not a valid value as "
+ "an index into vector %s."),
+ DBL_DIG + 1, index, vector_get_name (compute->vector));
+ return TRNS_CONTINUE;
+ }
+
+ *c = case_unshare (*c);
+ *case_num_rw (*c, vector_get_var (compute->vector, rindx - 1))
+ = expr_evaluate_num (compute->rvalue, *c, case_num);
+ }
+
+ return TRNS_CONTINUE;
+}
+
+/* Handle COMPUTE or IF with string target variable. */
+static enum trns_result
+compute_str (void *compute_, struct ccase **c, casenumber case_num)
+{
+ struct compute_trns *compute = compute_;
+
+ if (compute->test == NULL
+ || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
+ {
+ char *s;
+
+ *c = case_unshare (*c);
+ s = CHAR_CAST_BUG (char *, case_str_rw (*c, compute->variable));
+ expr_evaluate_str (compute->rvalue, *c, case_num, s, compute->width);
+ }
+
+ return TRNS_CONTINUE;
+}
+
+/* Handle COMPUTE or IF with string vector element target
+ variable. */
+static enum trns_result
+compute_str_vec (void *compute_, struct ccase **c, casenumber case_num)
+{
+ struct compute_trns *compute = compute_;
+
+ if (compute->test == NULL
+ || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
+ {
+ double index; /* Index into the vector. */
+ int rindx; /* Rounded index value. */
+ struct variable *vr; /* Variable reference by indexed vector. */
+
+ index = expr_evaluate_num (compute->element, *c, case_num);
+ rindx = floor (index + EPSILON);
+ if (index == SYSMIS)
+ {
+ msg_at (SW, compute->lvalue_location,
+ _("When executing COMPUTE: SYSMIS is not a valid "
+ "value as an index into vector %s."),
+ vector_get_name (compute->vector));
+ return TRNS_CONTINUE;
+ }
+ else if (rindx < 1 || rindx > vector_get_n_vars (compute->vector))
+ {
+ msg_at (SW, compute->lvalue_location,
+ _("When executing COMPUTE: %.*g is not a valid value as "
+ "an index into vector %s."),
+ DBL_DIG + 1, index, vector_get_name (compute->vector));
+ return TRNS_CONTINUE;
+ }
+
+ vr = vector_get_var (compute->vector, rindx - 1);
+ *c = case_unshare (*c);
+ expr_evaluate_str (compute->rvalue, *c, case_num,
+ CHAR_CAST_BUG (char *, case_str_rw (*c, vr)),
+ var_get_width (vr));
+ }
+
+ return TRNS_CONTINUE;
+}
+\f
+/* IF. */
+
+int
+cmd_if (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct compute_trns *compute = NULL;
+ struct lvalue *lvalue = NULL;
+
+ compute = compute_trns_create ();
+
+ /* Test expression. */
+ compute->test = expr_parse_bool (lexer, ds);
+ if (compute->test == NULL)
+ goto fail;
+
+ /* Lvalue variable. */
+ lvalue = lvalue_parse (lexer, ds);
+ if (lvalue == NULL)
+ goto fail;
+
+ /* Rvalue expression. */
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto fail;
+ compute->rvalue = parse_rvalue (lexer, lvalue, ds);
+ if (compute->rvalue == NULL)
+ goto fail;
+
+ add_transformation (ds, get_trns_class (lvalue), compute);
+
+ lvalue_finalize (lvalue, compute, dict);
+
+ return CMD_SUCCESS;
+
+ fail:
+ lvalue_destroy (lvalue, dict);
+ compute_trns_free (compute);
+ return CMD_CASCADING_FAILURE;
+}
+\f
+/* Code common to COMPUTE and IF. */
+
+static const struct trns_class *
+get_trns_class (const struct lvalue *lvalue)
+{
+ static const struct trns_class classes[2][2] = {
+ [false][false] = {
+ .name = "COMPUTE",
+ .execute = compute_str,
+ .destroy = compute_trns_free
+ },
+ [false][true] = {
+ .name = "COMPUTE",
+ .execute = compute_str_vec,
+ .destroy = compute_trns_free
+ },
+ [true][false] = {
+ .name = "COMPUTE",
+ .execute = compute_num,
+ .destroy = compute_trns_free
+ },
+ [true][true] = {
+ .name = "COMPUTE",
+ .execute = compute_num_vec,
+ .destroy = compute_trns_free
+ },
+ };
+
+ bool is_numeric = lvalue_get_type (lvalue) == VAL_NUMERIC;
+ bool is_vector = lvalue_is_vector (lvalue);
+ return &classes[is_numeric][is_vector];
+}
+
+/* Parses and returns an rvalue expression of the same type as
+ LVALUE, or a null pointer on failure. */
+static struct expression *
+parse_rvalue (struct lexer *lexer,
+ const struct lvalue *lvalue, struct dataset *ds)
+{
+ if (lvalue->is_new_variable)
+ return expr_parse_new_variable (lexer, ds, var_get_name (lvalue->variable),
+ lvalue->location);
+ else
+ return expr_parse (lexer, ds, lvalue_get_type (lvalue));
+}
+
+/* Returns a new struct compute_trns after initializing its fields. */
+static struct compute_trns *
+compute_trns_create (void)
+{
+ struct compute_trns *compute = xmalloc (sizeof *compute);
+ *compute = (struct compute_trns) { .test = NULL };
+ return compute;
+}
+
+/* Deletes all the fields in COMPUTE. */
+static bool
+compute_trns_free (void *compute_)
+{
+ struct compute_trns *compute = compute_;
+
+ if (compute != NULL)
+ {
+ msg_location_destroy (compute->lvalue_location);
+ expr_free (compute->test);
+ expr_free (compute->element);
+ expr_free (compute->rvalue);
+ free (compute);
+ }
+ return true;
+}
+\f
+/* Parses the target variable or vector element into a new
+ `struct lvalue', which is returned. */
+static struct lvalue *
+lvalue_parse (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+
+ struct lvalue *lvalue = xmalloc (sizeof *lvalue);
+ *lvalue = (struct lvalue) { .variable = NULL };
+
+ if (!lex_force_id (lexer))
+ goto lossage;
+
+ int start_ofs = lex_ofs (lexer);
+ if (lex_next_token (lexer, 1) == T_LPAREN)
+ {
+ /* Vector. */
+ lvalue->vector = dict_lookup_vector (dict, lex_tokcstr (lexer));
+ if (lvalue->vector == NULL)
+ {
+ lex_error (lexer, _("There is no vector named %s."),
+ lex_tokcstr (lexer));
+ goto lossage;
+ }
+
+ /* Vector element. */
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto lossage;
+ lvalue->element = expr_parse (lexer, ds, VAL_NUMERIC);
+ if (lvalue->element == NULL)
+ goto lossage;
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto lossage;
+ }
+ else
+ {
+ /* Variable name. */
+ const char *var_name = lex_tokcstr (lexer);
+ lvalue->variable = dict_lookup_var (dict, var_name);
+ if (lvalue->variable == NULL)
+ {
+ lvalue->variable = dict_create_var_assert (dict, var_name, 0);
+ lvalue->is_new_variable = true;
+ }
+ lex_get (lexer);
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+ lvalue->location = lex_ofs_location (lexer, start_ofs, end_ofs);
+ return lvalue;
+
+ lossage:
+ lvalue_destroy (lvalue, dict);
+ return NULL;
+}
+
+/* Returns the type (NUMERIC or ALPHA) of the target variable or
+ vector in LVALUE. */
+static int
+lvalue_get_type (const struct lvalue *lvalue)
+{
+ return (lvalue->variable != NULL
+ ? var_get_type (lvalue->variable)
+ : vector_get_type (lvalue->vector));
+}
+
+/* Returns true if LVALUE has a vector as its target. */
+static bool
+lvalue_is_vector (const struct lvalue *lvalue)
+{
+ return lvalue->vector != NULL;
+}
+
+/* Finalizes making LVALUE the target of COMPUTE, by creating the
+ target variable if necessary and setting fields in COMPUTE. */
+static void
+lvalue_finalize (struct lvalue *lvalue,
+ struct compute_trns *compute,
+ struct dictionary *dict)
+{
+ compute->lvalue_location = lvalue->location;
+ lvalue->location = NULL;
+
+ if (lvalue->vector == NULL)
+ {
+ compute->variable = lvalue->variable;
+ compute->width = var_get_width (compute->variable);
+
+ /* Goofy behavior, but compatible: Turn off LEAVE. */
+ if (!var_must_leave (compute->variable))
+ var_set_leave (compute->variable, false);
+
+ /* Prevent lvalue_destroy from deleting variable. */
+ lvalue->is_new_variable = false;
+ }
+ else
+ {
+ compute->vector = lvalue->vector;
+ compute->element = lvalue->element;
+ lvalue->element = NULL;
+ }
+
+ lvalue_destroy (lvalue, dict);
+}
+
+/* Destroys LVALUE. */
+static void
+lvalue_destroy (struct lvalue *lvalue, struct dictionary *dict)
+{
+ if (lvalue == NULL)
+ return;
+
+ if (lvalue->is_new_variable)
+ dict_delete_var (dict, lvalue->variable);
+ expr_free (lvalue->element);
+ msg_location_destroy (lvalue->location);
+ free (lvalue);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_matrix.h>
+#include <math.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/correlation.h"
+#include "math/covariance.h"
+#include "math/moments.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+#include "gl/minmax.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+
+struct corr
+ {
+ size_t n_vars_total;
+ size_t n_vars1;
+
+ const struct variable **vars;
+ };
+
+
+/* Handling of missing values. */
+enum corr_missing_type
+ {
+ CORR_PAIRWISE, /* Handle missing values on a per-variable-pair basis. */
+ CORR_LISTWISE /* Discard entire case if any variable is missing. */
+ };
+
+struct corr_opts
+{
+ enum corr_missing_type missing_type;
+ enum mv_class exclude; /* Classes of missing values to exclude. */
+
+ bool sig; /* Flag significant values or not */
+ int tails; /* Report significance with how many tails ? */
+ bool descriptive_stats;
+ bool xprod_stats;
+
+ const struct variable *wv; /* The weight variable (if any) */
+};
+
+
+static void
+output_descriptives (const struct corr *corr, const struct corr_opts *opts,
+ const gsl_matrix *means,
+ const gsl_matrix *vars, const gsl_matrix *ns)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Descriptive Statistics"));
+ pivot_table_set_weight_var (table, opts->wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("N"), PIVOT_RC_COUNT);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t r = 0; r < corr->n_vars_total; ++r)
+ {
+ const struct variable *v = corr->vars[r];
+
+ int row = pivot_category_create_leaf (variables->root,
+ pivot_value_new_variable (v));
+
+ double mean = gsl_matrix_get (means, r, 0);
+ /* Here we want to display the non-biased estimator */
+ double n = gsl_matrix_get (ns, r, 0);
+ double stddev = sqrt (gsl_matrix_get (vars, r, 0) * n / (n - 1));
+ double entries[] = { mean, stddev, n };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+output_correlation (const struct corr *corr, const struct corr_opts *opts,
+ const gsl_matrix *cm, const gsl_matrix *samples,
+ const gsl_matrix *cv)
+{
+ struct pivot_table *table = pivot_table_create (N_("Correlations"));
+ pivot_table_set_weight_var (table, opts->wv);
+
+ /* Column variable dimension. */
+ struct pivot_dimension *columns = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Variables"));
+
+ size_t matrix_cols = (corr->n_vars_total > corr->n_vars1
+ ? corr->n_vars_total - corr->n_vars1
+ : corr->n_vars1);
+ for (size_t c = 0; c < matrix_cols; c++)
+ {
+ const struct variable *v = corr->n_vars_total > corr->n_vars1 ?
+ corr->vars[corr->n_vars1 + c] : corr->vars[c];
+ pivot_category_create_leaf (columns->root, pivot_value_new_variable (v));
+ }
+
+ /* Statistics dimension. */
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Pearson Correlation"), PIVOT_RC_CORRELATION,
+ opts->tails == 2 ? N_("Sig. (2-tailed)") : N_("Sig. (1-tailed)"),
+ PIVOT_RC_SIGNIFICANCE);
+
+ if (opts->xprod_stats)
+ pivot_category_create_leaves (statistics->root, N_("Cross-products"),
+ N_("Covariance"));
+
+ if (opts->missing_type != CORR_LISTWISE)
+ pivot_category_create_leaves (statistics->root, N_("N"), PIVOT_RC_COUNT);
+
+ /* Row variable dimension. */
+ struct pivot_dimension *rows = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+ for (size_t r = 0; r < corr->n_vars1; r++)
+ pivot_category_create_leaf (rows->root,
+ pivot_value_new_variable (corr->vars[r]));
+
+ struct pivot_footnote *sig_footnote = pivot_table_create_footnote (
+ table, pivot_value_new_text (N_("Significant at .05 level")));
+
+ for (size_t r = 0; r < corr->n_vars1; r++)
+ for (size_t c = 0; c < matrix_cols; c++)
+ {
+ const int col_index = (corr->n_vars_total > corr->n_vars1
+ ? corr->n_vars1 + c
+ : c);
+ double pearson = gsl_matrix_get (cm, r, col_index);
+ double w = gsl_matrix_get (samples, r, col_index);
+ double sig = opts->tails * significance_of_correlation (pearson, w);
+
+ double entries[5];
+ int n = 0;
+ entries[n++] = pearson;
+ entries[n++] = col_index != r ? sig : SYSMIS;
+ if (opts->xprod_stats)
+ {
+ double cov = gsl_matrix_get (cv, r, col_index);
+ const double xprod_dev = cov * w;
+ cov *= w / (w - 1.0);
+
+ entries[n++] = xprod_dev;
+ entries[n++] = cov;
+ }
+ if (opts->missing_type != CORR_LISTWISE)
+ entries[n++] = w;
+
+ for (int i = 0; i < n; i++)
+ if (entries[i] != SYSMIS)
+ {
+ struct pivot_value *v = pivot_value_new_number (entries[i]);
+ if (!i && opts->sig && col_index != r && sig < 0.05)
+ pivot_value_add_footnote (v, sig_footnote);
+ pivot_table_put3 (table, c, i, r, v);
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+run_corr (struct casereader *r, const struct corr_opts *opts, const struct corr *corr)
+{
+ struct covariance *cov = covariance_2pass_create (
+ corr->n_vars_total, corr->vars, NULL,opts->wv, opts->exclude, true);
+
+ struct casereader *rc = casereader_clone (r);
+ struct ccase *c;
+ for (; (c = casereader_read (r)); case_unref (c))
+ covariance_accumulate_pass1 (cov, c);
+ for (; (c = casereader_read (rc)); case_unref (c))
+ covariance_accumulate_pass2 (cov, c);
+ casereader_destroy (rc);
+
+ gsl_matrix *cov_matrix = covariance_calculate (cov);
+ if (!cov_matrix)
+ {
+ msg (SE, _("The data for the chosen variables are all missing or empty."));
+ covariance_destroy (cov);
+ return;
+ }
+
+ const gsl_matrix *samples_matrix = covariance_moments (cov, MOMENT_NONE);
+ const gsl_matrix *var_matrix = covariance_moments (cov, MOMENT_VARIANCE);
+ const gsl_matrix *mean_matrix = covariance_moments (cov, MOMENT_MEAN);
+
+ gsl_matrix *corr_matrix = correlation_from_covariance (cov_matrix, var_matrix);
+
+ if (opts->descriptive_stats)
+ output_descriptives (corr, opts, mean_matrix, var_matrix, samples_matrix);
+
+ output_correlation (corr, opts, corr_matrix, samples_matrix, cov_matrix);
+
+ covariance_destroy (cov);
+ gsl_matrix_free (corr_matrix);
+ gsl_matrix_free (cov_matrix);
+}
+
+int
+cmd_correlations (struct lexer *lexer, struct dataset *ds)
+{
+ size_t n_all_vars = 0; /* Total number of variables involved in this command */
+ const struct dictionary *dict = dataset_dict (ds);
+
+ struct corr *corrs = NULL;
+ size_t n_corrs = 0;
+ size_t allocated_corrs = 0;
+
+ struct corr_opts opts = {
+ .missing_type = CORR_PAIRWISE,
+ .wv = dict_get_weight (dict),
+ .tails = 2,
+ .exclude = MV_ANY,
+ };
+
+ /* Parse CORRELATIONS. */
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "PAIRWISE"))
+ opts.missing_type = CORR_PAIRWISE;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ opts.missing_type = CORR_LISTWISE;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ opts.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ opts.exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "PAIRWISE", "LISTWISE",
+ "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "PRINT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "TWOTAIL"))
+ opts.tails = 2;
+ else if (lex_match_id (lexer, "ONETAIL"))
+ opts.tails = 1;
+ else if (lex_match_id (lexer, "SIG"))
+ opts.sig = false;
+ else if (lex_match_id (lexer, "NOSIG"))
+ opts.sig = true;
+ else
+ {
+ lex_error_expecting (lexer, "TWOTAIL", "ONETAIL",
+ "SIG", "NOSIG");
+ goto error;
+ }
+
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "DESCRIPTIVES"))
+ opts.descriptive_stats = true;
+ else if (lex_match_id (lexer, "XPROD"))
+ opts.xprod_stats = true;
+ else if (lex_token (lexer) == T_ALL)
+ {
+ opts.descriptive_stats = opts.xprod_stats = true;
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "DESCRIPTIVES", "XPROD", "ALL");
+ goto error;
+ }
+
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else
+ {
+ if (lex_match_id (lexer, "VARIABLES"))
+ lex_match (lexer, T_EQUALS);
+
+ const struct variable **vars;
+ size_t n_vars1;
+ if (!parse_variables_const (lexer, dict, &vars, &n_vars1, PV_NUMERIC))
+ goto error;
+
+ size_t n_vars_total = n_vars1;
+ if (lex_match (lexer, T_WITH)
+ && !parse_variables_const (lexer, dict, &vars, &n_vars_total,
+ PV_NUMERIC | PV_APPEND))
+ goto error;
+
+ if (n_corrs >= allocated_corrs)
+ corrs = x2nrealloc (corrs, &allocated_corrs, sizeof *corrs);
+ corrs[n_corrs++] = (struct corr) {
+ .n_vars1 = n_vars1,
+ .n_vars_total = n_vars_total,
+ .vars = vars,
+ };
+
+ n_all_vars += n_vars_total;
+ }
+ }
+ if (n_corrs == 0)
+ {
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("No variables specified."));
+ goto error;
+ }
+
+ const struct variable **all_vars = xmalloc (n_all_vars * sizeof *all_vars);
+ const struct variable **vv = all_vars;
+ for (size_t i = 0; i < n_corrs; ++i)
+ {
+ const struct corr *c = &corrs[i];
+ for (size_t v = 0; v < c->n_vars_total; ++v)
+ *vv++ = c->vars[v];
+ }
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ for (size_t i = 0; i < n_corrs; ++i)
+ {
+ /* FIXME: No need to iterate the data multiple times */
+ struct casereader *r = casereader_clone (group);
+
+ if (opts.missing_type == CORR_LISTWISE)
+ r = casereader_create_filter_missing (r, all_vars, n_all_vars,
+ opts.exclude, NULL, NULL);
+
+
+ run_corr (r, &opts, &corrs[i]);
+ casereader_destroy (r);
+ }
+ casereader_destroy (group);
+ }
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ free (all_vars);
+
+ /* Done. */
+ for (size_t i = 0; i < n_corrs; i++)
+ free (corrs[i].vars);
+ free (corrs);
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+
+error:
+ for (size_t i = 0; i < n_corrs; i++)
+ free (corrs[i].vars);
+ free (corrs);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2015 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/compiler.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Value or range? */
+enum value_type
+ {
+ CNT_SINGLE, /* Single value. */
+ CNT_RANGE /* a <= x <= b. */
+ };
+
+/* Numeric count criteria. */
+struct num_value
+ {
+ enum value_type type; /* How to interpret a, b. */
+ double a, b; /* Values to count. */
+ };
+
+struct criteria
+ {
+ struct criteria *next;
+
+ /* Variables to count. */
+ const struct variable **vars;
+ size_t n_vars;
+
+ /* Count special values? */
+ bool count_system_missing; /* Count system missing? */
+ bool count_user_missing; /* Count user missing? */
+
+ /* Criterion values. */
+ size_t n_values;
+ union
+ {
+ struct num_value *num;
+ char **str;
+ }
+ values;
+ };
+
+struct dst_var
+ {
+ struct dst_var *next;
+ struct variable *var; /* Destination variable. */
+ char *name; /* Name of dest var. */
+ struct criteria *crit; /* The criteria specifications. */
+ };
+
+struct count_trns
+ {
+ struct dst_var *dst_vars;
+ struct pool *pool;
+ };
+
+static const struct trns_class count_trns_class;
+
+static bool parse_numeric_criteria (struct lexer *, struct pool *, struct criteria *);
+static bool parse_string_criteria (struct lexer *, struct pool *,
+ struct criteria *,
+ const char *dict_encoding);
+static bool count_trns_free (void *trns_);
+\f
+int
+cmd_count (struct lexer *lexer, struct dataset *ds)
+{
+ struct dst_var *dv; /* Destination var being parsed. */
+ struct count_trns *trns; /* Transformation. */
+
+ /* Parses each slash-delimited specification. */
+ trns = pool_create_container (struct count_trns, pool);
+ trns->dst_vars = dv = pool_alloc (trns->pool, sizeof *dv);
+ for (;;)
+ {
+ struct criteria *crit;
+
+ /* Initialize this struct dst_var to ensure proper cleanup. */
+ dv->next = NULL;
+ dv->var = NULL;
+ dv->crit = NULL;
+
+ /* Get destination variable, or at least its name. */
+ if (!lex_force_id (lexer))
+ goto fail;
+ dv->var = dict_lookup_var (dataset_dict (ds), lex_tokcstr (lexer));
+ if (dv->var != NULL)
+ {
+ if (var_is_alpha (dv->var))
+ {
+ lex_error (lexer, _("Destination cannot be a string variable."));
+ goto fail;
+ }
+ }
+ else
+ dv->name = pool_strdup (trns->pool, lex_tokcstr (lexer));
+
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto fail;
+
+ crit = dv->crit = pool_alloc (trns->pool, sizeof *crit);
+ for (;;)
+ {
+ struct dictionary *dict = dataset_dict (ds);
+ bool ok;
+
+ crit->next = NULL;
+ crit->vars = NULL;
+ if (!parse_variables_const (lexer, dict, &crit->vars,
+ &crit->n_vars,
+ PV_DUPLICATE | PV_SAME_TYPE))
+ goto fail;
+ pool_register (trns->pool, free, crit->vars);
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto fail;
+
+ crit->n_values = 0;
+ if (var_is_numeric (crit->vars[0]))
+ ok = parse_numeric_criteria (lexer, trns->pool, crit);
+ else
+ ok = parse_string_criteria (lexer, trns->pool, crit,
+ dict_get_encoding (dict));
+ if (!ok)
+ goto fail;
+
+ if (lex_token (lexer) == T_SLASH || lex_token (lexer) == T_ENDCMD)
+ break;
+
+ crit = crit->next = pool_alloc (trns->pool, sizeof *crit);
+ }
+
+ if (lex_token (lexer) == T_ENDCMD)
+ break;
+
+ if (!lex_force_match (lexer, T_SLASH))
+ goto fail;
+ dv = dv->next = pool_alloc (trns->pool, sizeof *dv);
+ }
+
+ /* Create all the nonexistent destination variables. */
+ for (dv = trns->dst_vars; dv; dv = dv->next)
+ if (dv->var == NULL)
+ {
+ /* It's valid, though motivationally questionable, to count to
+ the same dest var more than once. */
+ dv->var = dict_lookup_var (dataset_dict (ds), dv->name);
+
+ if (dv->var == NULL)
+ dv->var = dict_create_var_assert (dataset_dict (ds), dv->name, 0);
+ }
+
+ add_transformation (ds, &count_trns_class, trns);
+ return CMD_SUCCESS;
+
+fail:
+ count_trns_free (trns);
+ return CMD_FAILURE;
+}
+
+/* Parses a set of numeric criterion values. Returns success. */
+static bool
+parse_numeric_criteria (struct lexer *lexer, struct pool *pool, struct criteria *crit)
+{
+ size_t allocated = 0;
+
+ crit->values.num = NULL;
+ crit->count_system_missing = false;
+ crit->count_user_missing = false;
+ for (;;)
+ {
+ double low, high;
+
+ if (lex_match_id (lexer, "SYSMIS"))
+ crit->count_system_missing = true;
+ else if (lex_match_id (lexer, "MISSING"))
+ crit->count_system_missing = crit->count_user_missing = true;
+ else if (parse_num_range (lexer, &low, &high, NULL))
+ {
+ struct num_value *cur;
+
+ if (crit->n_values >= allocated)
+ crit->values.num = pool_2nrealloc (pool, crit->values.num,
+ &allocated,
+ sizeof *crit->values.num);
+ cur = &crit->values.num[crit->n_values++];
+ cur->type = low == high ? CNT_SINGLE : CNT_RANGE;
+ cur->a = low;
+ cur->b = high;
+ }
+ else
+ return false;
+
+ lex_match (lexer, T_COMMA);
+ if (lex_match (lexer, T_RPAREN))
+ break;
+ }
+ return true;
+}
+
+/* Parses a set of string criteria values. Returns success. */
+static bool
+parse_string_criteria (struct lexer *lexer, struct pool *pool,
+ struct criteria *crit, const char *dict_encoding)
+{
+ int len = 0;
+ size_t allocated = 0;
+ size_t i;
+
+ for (i = 0; i < crit->n_vars; i++)
+ if (var_get_width (crit->vars[i]) > len)
+ len = var_get_width (crit->vars[i]);
+
+ crit->values.str = NULL;
+ for (;;)
+ {
+ char **cur;
+ char *s;
+
+ if (crit->n_values >= allocated)
+ crit->values.str = pool_2nrealloc (pool, crit->values.str,
+ &allocated,
+ sizeof *crit->values.str);
+
+ if (!lex_force_string (lexer))
+ return false;
+
+ s = recode_string (dict_encoding, "UTF-8", lex_tokcstr (lexer),
+ ss_length (lex_tokss (lexer)));
+
+ cur = &crit->values.str[crit->n_values++];
+ *cur = pool_alloc (pool, len + 1);
+ str_copy_rpad (*cur, len + 1, s);
+ lex_get (lexer);
+
+ free (s);
+
+ lex_match (lexer, T_COMMA);
+ if (lex_match (lexer, T_RPAREN))
+ break;
+ }
+
+ return true;
+}
+\f
+/* Transformation. */
+
+/* Counts the number of values in case C matching CRIT. */
+static int
+count_numeric (struct criteria *crit, const struct ccase *c)
+{
+ int counter = 0;
+ size_t i;
+
+ for (i = 0; i < crit->n_vars; i++)
+ {
+ double x = case_num (c, crit->vars[i]);
+ struct num_value *v;
+
+ for (v = crit->values.num; v < crit->values.num + crit->n_values;
+ v++)
+ if (v->type == CNT_SINGLE ? x == v->a : x >= v->a && x <= v->b)
+ {
+ counter++;
+ break;
+ }
+
+ if (var_is_num_missing (crit->vars[i], x)
+ && (x == SYSMIS
+ ? crit->count_system_missing
+ : crit->count_user_missing))
+ {
+ counter++;
+ continue;
+ }
+
+ }
+
+ return counter;
+}
+
+/* Counts the number of values in case C matching CRIT. */
+static int
+count_string (struct criteria *crit, const struct ccase *c)
+{
+ int counter = 0;
+ size_t i;
+
+ for (i = 0; i < crit->n_vars; i++)
+ {
+ char **v;
+ for (v = crit->values.str; v < crit->values.str + crit->n_values; v++)
+ if (!memcmp (case_str (c, crit->vars[i]), *v,
+ var_get_width (crit->vars[i])))
+ {
+ counter++;
+ break;
+ }
+ }
+
+ return counter;
+}
+
+/* Performs the COUNT transformation T on case C. */
+static enum trns_result
+count_trns_proc (void *trns_, struct ccase **c,
+ casenumber case_num UNUSED)
+{
+ struct count_trns *trns = trns_;
+ struct dst_var *dv;
+
+ *c = case_unshare (*c);
+ for (dv = trns->dst_vars; dv; dv = dv->next)
+ {
+ struct criteria *crit;
+ int counter;
+
+ counter = 0;
+ for (crit = dv->crit; crit; crit = crit->next)
+ if (var_is_numeric (crit->vars[0]))
+ counter += count_numeric (crit, *c);
+ else
+ counter += count_string (crit, *c);
+ *case_num_rw (*c, dv->var) = counter;
+ }
+ return TRNS_CONTINUE;
+}
+
+/* Destroys all dynamic data structures associated with TRNS. */
+static bool
+count_trns_free (void *trns_)
+{
+ struct count_trns *trns = trns_;
+ pool_destroy (trns->pool);
+ return true;
+}
+
+static const struct trns_class count_trns_class = {
+ .name = "COUNT",
+ .execute = count_trns_proc,
+ .destroy = count_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012, 2013, 2014, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+/* FIXME:
+
+ - How to calculate significance of some symmetric and directional measures?
+ - How to calculate ASE for symmetric Somers ' d?
+ - How to calculate ASE for Goodman and Kruskal's tau?
+ - How to calculate approx. T of symmetric uncertainty coefficient?
+
+*/
+
+#include <config.h>
+
+#include <ctype.h>
+#include <float.h>
+#include <gsl/gsl_cdf.h>
+#include <stdlib.h>
+#include <stdio.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/freq.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmap.h"
+#include "libpspp/hmapx.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+#include "output/pivot-table.h"
+#include "output/charts/barchart.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc-oversized.h"
+#include "gl/xalloc.h"
+#include "gl/xsize.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+/* Kinds of cells in the crosstabulation. */
+#define CRS_CELLS \
+ C(COUNT, N_("Count"), PIVOT_RC_COUNT) \
+ C(EXPECTED, N_("Expected"), PIVOT_RC_OTHER) \
+ C(ROW, N_("Row %"), PIVOT_RC_PERCENT) \
+ C(COLUMN, N_("Column %"), PIVOT_RC_PERCENT) \
+ C(TOTAL, N_("Total %"), PIVOT_RC_PERCENT) \
+ C(RESIDUAL, N_("Residual"), PIVOT_RC_RESIDUAL) \
+ C(SRESIDUAL, N_("Std. Residual"), PIVOT_RC_RESIDUAL) \
+ C(ASRESIDUAL, N_("Adjusted Residual"), PIVOT_RC_RESIDUAL)
+enum crs_cell
+ {
+#define C(KEYWORD, STRING, RC) CRS_CL_##KEYWORD,
+ CRS_CELLS
+#undef C
+ };
+enum {
+#define C(KEYWORD, STRING, RC) + 1
+ CRS_N_CELLS = CRS_CELLS
+#undef C
+};
+#define CRS_ALL_CELLS ((1u << CRS_N_CELLS) - 1)
+
+/* Kinds of statistics. */
+#define CRS_STATISTICS \
+ S(CHISQ) \
+ S(PHI) \
+ S(CC) \
+ S(LAMBDA) \
+ S(UC) \
+ S(BTAU) \
+ S(CTAU) \
+ S(RISK) \
+ S(GAMMA) \
+ S(D) \
+ S(KAPPA) \
+ S(ETA) \
+ S(CORR)
+enum crs_statistic_index {
+#define S(KEYWORD) CRS_ST_##KEYWORD##_INDEX,
+ CRS_STATISTICS
+#undef S
+};
+enum crs_statistic_bit {
+#define S(KEYWORD) CRS_ST_##KEYWORD = 1u << CRS_ST_##KEYWORD##_INDEX,
+ CRS_STATISTICS
+#undef S
+};
+enum {
+#define S(KEYWORD) + 1
+ CRS_N_STATISTICS = CRS_STATISTICS
+#undef S
+};
+#define CRS_ALL_STATISTICS ((1u << CRS_N_STATISTICS) - 1)
+
+/* Number of chi-square statistics. */
+#define N_CHISQ 5
+
+/* Number of symmetric statistics. */
+#define N_SYMMETRIC 9
+
+/* Number of directional statistics. */
+#define N_DIRECTIONAL 13
+
+/* Indexes into the 'vars' member of struct crosstabulation and
+ struct crosstab member. */
+enum
+ {
+ ROW_VAR = 0, /* Row variable. */
+ COL_VAR = 1 /* Column variable. */
+ /* Higher indexes cause multiple tables to be output. */
+ };
+
+struct xtab_var
+ {
+ const struct variable *var;
+ union value *values;
+ size_t n_values;
+ };
+
+/* A crosstabulation of 2 or more variables. */
+struct crosstabulation
+ {
+ struct crosstabs_proc *proc;
+ struct fmt_spec weight_format; /* Format for weight variable. */
+ double missing; /* Weight of missing cases. */
+
+ /* Variables (2 or more). */
+ size_t n_vars;
+ struct xtab_var *vars;
+
+ /* Constants (0 or more). */
+ size_t n_consts;
+ struct xtab_var *const_vars;
+ size_t *const_indexes;
+
+ /* Data. */
+ struct hmap data;
+ struct freq **entries;
+ size_t n_entries;
+
+ /* Number of statistically interesting columns/rows
+ (columns/rows with data in them). */
+ size_t ns_cols, ns_rows;
+
+ /* Matrix contents. */
+ double *mat; /* Matrix proper. */
+ double *row_tot; /* Row totals. */
+ double *col_tot; /* Column totals. */
+ double total; /* Grand total. */
+
+ /* Syntax. */
+ int start_ofs;
+ int end_ofs;
+ };
+
+/* Integer mode variable info. */
+struct var_range
+ {
+ struct hmap_node hmap_node; /* In struct crosstabs_proc var_ranges map. */
+ const struct variable *var; /* The variable. */
+ int min; /* Minimum value. */
+ int max; /* Maximum value + 1. */
+ int count; /* max - min. */
+ };
+
+struct crosstabs_proc
+ {
+ const struct dictionary *dict;
+ enum { INTEGER, GENERAL } mode;
+ enum mv_class exclude;
+ bool barchart;
+ bool bad_warn;
+ struct fmt_spec weight_format;
+
+ /* Variables specifies on VARIABLES. */
+ const struct variable **variables;
+ size_t n_variables;
+ struct hmap var_ranges;
+
+ /* TABLES. */
+ struct crosstabulation *pivots;
+ size_t n_pivots;
+
+ /* CELLS. */
+ size_t n_cells; /* Number of cells requested. */
+ unsigned int cells; /* Bit k is 1 if cell k is requested. */
+ int a_cells[CRS_N_CELLS]; /* 0...n_cells-1 are the requested cells. */
+
+ /* Rounding of cells. */
+ bool round_case_weights; /* Round case weights? */
+ bool round_cells; /* If !round_case_weights, round cells? */
+ bool round_down; /* Round down? (otherwise to nearest) */
+
+ /* STATISTICS. */
+ unsigned int statistics; /* Bit k is 1 if statistic k is requested. */
+
+ bool descending; /* True if descending sort order is requested. */
+ };
+
+static bool parse_crosstabs_tables (struct lexer *, struct dataset *,
+ struct crosstabs_proc *);
+static bool parse_crosstabs_variables (struct lexer *, struct dataset *,
+ struct crosstabs_proc *);
+
+static const struct var_range *get_var_range (const struct crosstabs_proc *,
+ const struct variable *);
+
+static bool should_tabulate_case (const struct crosstabulation *,
+ const struct ccase *, enum mv_class exclude);
+static void tabulate_general_case (struct crosstabulation *, const struct ccase *,
+ double weight);
+static void tabulate_integer_case (struct crosstabulation *, const struct ccase *,
+ double weight);
+static void postcalc (struct crosstabs_proc *, struct lexer *);
+
+static double
+round_weight (const struct crosstabs_proc *proc, double weight)
+{
+ return proc->round_down ? floor (weight) : floor (weight + 0.5);
+}
+
+#define FOR_EACH_POPULATED_COLUMN(C, XT) \
+ for (size_t C = next_populated_column (0, XT); \
+ C < (XT)->vars[COL_VAR].n_values; \
+ C = next_populated_column (C + 1, XT))
+static size_t
+next_populated_column (size_t c, const struct crosstabulation *xt)
+{
+ size_t n_columns = xt->vars[COL_VAR].n_values;
+ for (; c < n_columns; c++)
+ if (xt->col_tot[c])
+ break;
+ return c;
+}
+
+#define FOR_EACH_POPULATED_ROW(R, XT) \
+ for (size_t R = next_populated_row (0, XT); R < (XT)->vars[ROW_VAR].n_values; \
+ R = next_populated_row (R + 1, XT))
+static size_t
+next_populated_row (size_t r, const struct crosstabulation *xt)
+{
+ size_t n_rows = xt->vars[ROW_VAR].n_values;
+ for (; r < n_rows; r++)
+ if (xt->row_tot[r])
+ break;
+ return r;
+}
+
+/* Parses and executes the CROSSTABS procedure. */
+int
+cmd_crosstabs (struct lexer *lexer, struct dataset *ds)
+{
+ int result = CMD_FAILURE;
+
+ struct crosstabs_proc proc = {
+ .dict = dataset_dict (ds),
+ .mode = GENERAL,
+ .exclude = MV_ANY,
+ .barchart = false,
+ .bad_warn = true,
+ .weight_format = *dict_get_weight_format (dataset_dict (ds)),
+
+ .variables = NULL,
+ .n_variables = 0,
+ .var_ranges = HMAP_INITIALIZER (proc.var_ranges),
+
+ .pivots = NULL,
+ .n_pivots = 0,
+
+ .cells = 1u << CRS_CL_COUNT,
+ /* n_cells and a_cells will be filled in later. */
+
+ .round_case_weights = false,
+ .round_cells = false,
+ .round_down = false,
+
+ .statistics = 0,
+
+ .descending = false,
+ };
+ bool show_tables = true;
+ int exclude_ofs = 0;
+ lex_match (lexer, T_SLASH);
+ for (;;)
+ {
+ if (lex_match_id (lexer, "VARIABLES"))
+ {
+ if (!parse_crosstabs_variables (lexer, ds, &proc))
+ goto exit;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ exclude_ofs = lex_ofs (lexer);
+ if (lex_match_id (lexer, "TABLE"))
+ proc.exclude = MV_ANY;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ proc.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "REPORT"))
+ proc.exclude = 0;
+ else
+ {
+ lex_error_expecting (lexer, "TABLE", "INCLUDE", "REPORT");
+ goto exit;
+ }
+ }
+ else if (lex_match_id (lexer, "COUNT"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ /* Default is CELL. */
+ proc.round_case_weights = false;
+ proc.round_cells = true;
+
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "ASIS"))
+ {
+ proc.round_case_weights = false;
+ proc.round_cells = false;
+ }
+ else if (lex_match_id (lexer, "CASE"))
+ {
+ proc.round_case_weights = true;
+ proc.round_cells = false;
+ }
+ else if (lex_match_id (lexer, "CELL"))
+ {
+ proc.round_case_weights = false;
+ proc.round_cells = true;
+ }
+ else if (lex_match_id (lexer, "ROUND"))
+ proc.round_down = false;
+ else if (lex_match_id (lexer, "TRUNCATE"))
+ proc.round_down = true;
+ else
+ {
+ lex_error_expecting (lexer, "ASIS", "CASE", "CELL",
+ "ROUND", "TRUNCATE");
+ goto exit;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "AVALUE"))
+ proc.descending = false;
+ else if (lex_match_id (lexer, "DVALUE"))
+ proc.descending = true;
+ else if (lex_match_id (lexer, "TABLES"))
+ show_tables = true;
+ else if (lex_match_id (lexer, "NOTABLES"))
+ show_tables = false;
+ else
+ {
+ lex_error_expecting (lexer, "AVALUE", "DVALUE",
+ "TABLES", "NOTABLES");
+ goto exit;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "BARCHART"))
+ proc.barchart = true;
+ else if (lex_match_id (lexer, "CELLS"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "NONE"))
+ proc.cells = 0;
+ else if (lex_match (lexer, T_ALL))
+ proc.cells = CRS_ALL_CELLS;
+ else
+ {
+ proc.cells = 0;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+#define C(KEYWORD, STRING, RC) \
+ if (lex_match_id (lexer, #KEYWORD)) \
+ { \
+ proc.cells |= 1u << CRS_CL_##KEYWORD; \
+ continue; \
+ }
+ CRS_CELLS
+#undef C
+
+ static const char *cells[] =
+ {
+#define C(KEYWORD, STRING, RC) #KEYWORD,
+ CRS_CELLS
+#undef C
+ };
+ lex_error_expecting_array (lexer, cells,
+ sizeof cells / sizeof *cells);
+ goto exit;
+ }
+ if (!proc.cells)
+ proc.cells = ((1u << CRS_CL_COUNT) | (1u << CRS_CL_ROW)
+ | (1u << CRS_CL_COLUMN) | (1u << CRS_CL_TOTAL));
+ }
+ }
+ else if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "NONE"))
+ proc.statistics = 0;
+ else if (lex_match (lexer, T_ALL))
+ proc.statistics = CRS_ALL_STATISTICS;
+ else
+ {
+ proc.statistics = 0;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+#define S(KEYWORD) \
+ if (lex_match_id (lexer, #KEYWORD)) \
+ { \
+ proc.statistics |= CRS_ST_##KEYWORD; \
+ continue; \
+ }
+ CRS_STATISTICS
+#undef S
+ static const char *stats[] =
+ {
+#define S(KEYWORD) #KEYWORD,
+ CRS_STATISTICS
+#undef S
+ };
+ lex_error_expecting_array (lexer, stats,
+ sizeof stats / sizeof *stats);
+ goto exit;
+ }
+ if (!proc.statistics)
+ proc.statistics = CRS_ST_CHISQ;
+ }
+ }
+ else if (!parse_crosstabs_tables (lexer, ds, &proc))
+ goto exit;
+
+ if (!lex_match (lexer, T_SLASH))
+ break;
+ }
+ if (!lex_end_of_command (lexer))
+ goto exit;
+
+ if (!proc.n_pivots)
+ {
+ msg (SE, _("At least one crosstabulation must be requested (using "
+ "the TABLES subcommand)."));
+ goto exit;
+ }
+
+ /* Cells. */
+ if (!show_tables)
+ proc.cells = 0;
+ for (size_t i = 0; i < CRS_N_CELLS; i++)
+ if (proc.cells & (1u << i))
+ proc.a_cells[proc.n_cells++] = i;
+ assert (proc.n_cells < CRS_N_CELLS);
+
+ /* Missing values. */
+ if (proc.mode == GENERAL && !proc.exclude)
+ {
+ lex_ofs_msg (lexer, SW, exclude_ofs, exclude_ofs,
+ _("Missing mode %s not allowed in general mode. "
+ "Assuming %s."), "REPORT", "MISSING=TABLE");
+ proc.exclude = MV_ANY;
+ }
+
+ struct casereader *input = casereader_create_filter_weight (proc_open (ds),
+ dataset_dict (ds),
+ NULL, NULL);
+ struct casegrouper *grouper = casegrouper_create_splits (input, dataset_dict (ds));
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ output_split_file_values_peek (ds, group);
+
+ /* Initialize hash tables. */
+ for (struct crosstabulation *xt = &proc.pivots[0];
+ xt < &proc.pivots[proc.n_pivots]; xt++)
+ hmap_init (&xt->data);
+
+ /* Tabulate. */
+ struct ccase *c;
+ for (; (c = casereader_read (group)) != NULL; case_unref (c))
+ for (struct crosstabulation *xt = &proc.pivots[0];
+ xt < &proc.pivots[proc.n_pivots]; xt++)
+ {
+ double weight = dict_get_case_weight (dataset_dict (ds), c,
+ &proc.bad_warn);
+ if (proc.round_case_weights)
+ {
+ weight = round_weight (&proc, weight);
+ if (weight == 0.)
+ continue;
+ }
+ if (should_tabulate_case (xt, c, proc.exclude))
+ {
+ if (proc.mode == GENERAL)
+ tabulate_general_case (xt, c, weight);
+ else
+ tabulate_integer_case (xt, c, weight);
+ }
+ else
+ xt->missing += weight;
+ }
+ casereader_destroy (group);
+
+ /* Output. */
+ postcalc (&proc, lexer);
+ }
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ result = ok ? CMD_SUCCESS : CMD_FAILURE;
+
+exit:
+ free (proc.variables);
+
+ struct var_range *range, *next_range;
+ HMAP_FOR_EACH_SAFE (range, next_range, struct var_range, hmap_node,
+ &proc.var_ranges)
+ {
+ hmap_delete (&proc.var_ranges, &range->hmap_node);
+ free (range);
+ }
+ for (struct crosstabulation *xt = &proc.pivots[0];
+ xt < &proc.pivots[proc.n_pivots]; xt++)
+ {
+ free (xt->vars);
+ free (xt->const_vars);
+ free (xt->const_indexes);
+ }
+ free (proc.pivots);
+
+ return result;
+}
+
+/* Parses the TABLES subcommand. */
+static bool
+parse_crosstabs_tables (struct lexer *lexer, struct dataset *ds,
+ struct crosstabs_proc *proc)
+{
+ const struct variable ***by = NULL;
+ size_t *by_nvar = NULL;
+ bool ok = false;
+
+ /* Ensure that this is a TABLES subcommand. */
+ if (!lex_match_id (lexer, "TABLES")
+ && (lex_token (lexer) != T_ID ||
+ dict_lookup_var (dataset_dict (ds), lex_tokcstr (lexer)) == NULL)
+ && lex_token (lexer) != T_ALL)
+ {
+ lex_error (lexer, _("Syntax error expecting subcommand name or "
+ "variable name."));
+ return false;
+ }
+ lex_match (lexer, T_EQUALS);
+
+ struct const_var_set *var_set
+ = (proc->variables
+ ? const_var_set_create_from_array (proc->variables,
+ proc->n_variables)
+ : const_var_set_create_from_dict (dataset_dict (ds)));
+
+ size_t nx = 1;
+ size_t n_by = 0;
+ int vars_start = lex_ofs (lexer);
+ do
+ {
+ by = xnrealloc (by, n_by + 1, sizeof *by);
+ by_nvar = xnrealloc (by_nvar, n_by + 1, sizeof *by_nvar);
+ if (!parse_const_var_set_vars (lexer, var_set, &by[n_by], &by_nvar[n_by],
+ PV_NO_DUPLICATE | PV_NO_SCRATCH))
+ goto done;
+ size_t n = by_nvar[n_by++];
+ if (xalloc_oversized (nx, n))
+ {
+ lex_ofs_error (
+ lexer, vars_start, lex_ofs (lexer) - 1,
+ _("Too many cross-tabulation variables or dimensions."));
+ goto done;
+ }
+ nx *= n;
+ }
+ while (lex_match (lexer, T_BY));
+ if (n_by < 2)
+ {
+ bool unused UNUSED = lex_force_match (lexer, T_BY);
+ goto done;
+ }
+ int vars_end = lex_ofs (lexer) - 1;
+
+ size_t *by_iter = XCALLOC (n_by, size_t);
+ proc->pivots = xnrealloc (proc->pivots,
+ proc->n_pivots + nx, sizeof *proc->pivots);
+ for (size_t i = 0; i < nx; i++)
+ {
+ struct crosstabulation *xt = &proc->pivots[proc->n_pivots++];
+
+ *xt = (struct crosstabulation) {
+ .proc = proc,
+ .weight_format = proc->weight_format,
+ .missing = 0.,
+ .n_vars = n_by,
+ .vars = xcalloc (n_by, sizeof *xt->vars),
+ .n_consts = 0,
+ .const_vars = NULL,
+ .const_indexes = NULL,
+ .start_ofs = vars_start,
+ .end_ofs = vars_end,
+ };
+
+ for (size_t j = 0; j < n_by; j++)
+ xt->vars[j].var = by[j][by_iter[j]];
+
+ for (int j = n_by - 1; j >= 0; j--)
+ {
+ if (++by_iter[j] < by_nvar[j])
+ break;
+ by_iter[j] = 0;
+ }
+ }
+ free (by_iter);
+ ok = true;
+
+done:
+ /* All return paths lead here. */
+ for (size_t i = 0; i < n_by; i++)
+ free (by[i]);
+ free (by);
+ free (by_nvar);
+
+ const_var_set_destroy (var_set);
+
+ return ok;
+}
+
+/* Parses the VARIABLES subcommand. */
+static bool
+parse_crosstabs_variables (struct lexer *lexer, struct dataset *ds,
+ struct crosstabs_proc *proc)
+{
+ if (proc->n_pivots)
+ {
+ lex_next_error (lexer, -1, -1, _("%s must be specified before %s."),
+ "VARIABLES", "TABLES");
+ return false;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ for (;;)
+ {
+ size_t orig_nv = proc->n_variables;
+
+ if (!parse_variables_const (lexer, dataset_dict (ds),
+ &proc->variables, &proc->n_variables,
+ (PV_APPEND | PV_NUMERIC
+ | PV_NO_DUPLICATE | PV_NO_SCRATCH)))
+ return false;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ if (!lex_force_int (lexer))
+ goto error;
+ long min = lex_integer (lexer);
+ lex_get (lexer);
+
+ lex_match (lexer, T_COMMA);
+
+ if (!lex_force_int_range (lexer, NULL, min, LONG_MAX))
+ goto error;
+ long max = lex_integer (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ for (size_t i = orig_nv; i < proc->n_variables; i++)
+ {
+ const struct variable *var = proc->variables[i];
+ struct var_range *vr = xmalloc (sizeof *vr);
+ *vr = (struct var_range) {
+ .var = var,
+ .min = min,
+ .max = max,
+ .count = max - min + 1,
+ };
+ hmap_insert (&proc->var_ranges, &vr->hmap_node,
+ hash_pointer (var, 0));
+ }
+
+ if (lex_token (lexer) == T_SLASH)
+ break;
+ }
+
+ proc->mode = INTEGER;
+ return true;
+
+ error:
+ free (proc->variables);
+ proc->variables = NULL;
+ proc->n_variables = 0;
+ return false;
+}
+\f
+/* Data file processing. */
+
+static const struct var_range *
+get_var_range (const struct crosstabs_proc *proc, const struct variable *var)
+{
+ if (!hmap_is_empty (&proc->var_ranges))
+ {
+ const struct var_range *range;
+
+ HMAP_FOR_EACH_IN_BUCKET (range, struct var_range, hmap_node,
+ hash_pointer (var, 0), &proc->var_ranges)
+ if (range->var == var)
+ return range;
+ }
+
+ return NULL;
+}
+
+static bool
+should_tabulate_case (const struct crosstabulation *xt, const struct ccase *c,
+ enum mv_class exclude)
+{
+ for (size_t j = 0; j < xt->n_vars; j++)
+ {
+ const struct variable *var = xt->vars[j].var;
+ const struct var_range *range = get_var_range (xt->proc, var);
+
+ if (var_is_value_missing (var, case_data (c, var)) & exclude)
+ return false;
+
+ if (range != NULL)
+ {
+ double num = case_num (c, var);
+ if (num < range->min || num >= range->max + 1.)
+ return false;
+ }
+ }
+ return true;
+}
+
+static void
+tabulate_integer_case (struct crosstabulation *xt, const struct ccase *c,
+ double weight)
+{
+ size_t hash = 0;
+ for (size_t j = 0; j < xt->n_vars; j++)
+ {
+ /* Throw away fractional parts of values. */
+ hash = hash_int (case_num (c, xt->vars[j].var), hash);
+ }
+
+ struct freq *te;
+ HMAP_FOR_EACH_WITH_HASH (te, struct freq, node, hash, &xt->data)
+ {
+ for (size_t j = 0; j < xt->n_vars; j++)
+ if ((int) case_num (c, xt->vars[j].var) != (int) te->values[j].f)
+ goto no_match;
+
+ /* Found an existing entry. */
+ te->count += weight;
+ return;
+
+ no_match: ;
+ }
+
+ /* No existing entry. Create a new one. */
+ te = xmalloc (table_entry_size (xt->n_vars));
+ te->count = weight;
+ for (size_t j = 0; j < xt->n_vars; j++)
+ te->values[j].f = (int) case_num (c, xt->vars[j].var);
+ hmap_insert (&xt->data, &te->node, hash);
+}
+
+static void
+tabulate_general_case (struct crosstabulation *xt, const struct ccase *c,
+ double weight)
+{
+ size_t hash = 0;
+ for (size_t j = 0; j < xt->n_vars; j++)
+ {
+ const struct variable *var = xt->vars[j].var;
+ hash = value_hash (case_data (c, var), var_get_width (var), hash);
+ }
+
+ struct freq *te;
+ HMAP_FOR_EACH_WITH_HASH (te, struct freq, node, hash, &xt->data)
+ {
+ for (size_t j = 0; j < xt->n_vars; j++)
+ {
+ const struct variable *var = xt->vars[j].var;
+ if (!value_equal (case_data (c, var), &te->values[j],
+ var_get_width (var)))
+ goto no_match;
+ }
+
+ /* Found an existing entry. */
+ te->count += weight;
+ return;
+
+ no_match: ;
+ }
+
+ /* No existing entry. Create a new one. */
+ te = xmalloc (table_entry_size (xt->n_vars));
+ te->count = weight;
+ for (size_t j = 0; j < xt->n_vars; j++)
+ {
+ const struct variable *var = xt->vars[j].var;
+ value_clone (&te->values[j], case_data (c, var), var_get_width (var));
+ }
+ hmap_insert (&xt->data, &te->node, hash);
+}
+\f
+/* Post-data reading calculations. */
+
+static int compare_table_entry_vars_3way (const struct freq *a,
+ const struct freq *b,
+ const struct crosstabulation *xt,
+ int idx0, int idx1);
+static int compare_table_entry_3way (const void *ap_, const void *bp_,
+ const void *xt_);
+static int compare_table_entry_3way_inv (const void *ap_, const void *bp_,
+ const void *xt_);
+
+static void enum_var_values (const struct crosstabulation *, int var_idx,
+ bool descending);
+static void free_var_values (const struct crosstabulation *, int var_idx);
+static void output_crosstabulation (struct crosstabs_proc *,
+ struct crosstabulation *,
+ struct lexer *);
+static void make_crosstabulation_subset (struct crosstabulation *xt,
+ size_t row0, size_t row1,
+ struct crosstabulation *subset);
+static void make_summary_table (struct crosstabs_proc *);
+static bool find_crosstab (struct crosstabulation *, size_t *row0p,
+ size_t *row1p);
+
+static void
+postcalc (struct crosstabs_proc *proc, struct lexer *lexer)
+{
+ /* Round hash table entries, if requested
+
+ If this causes any of the cell counts to fall to zero, delete those
+ cells. */
+ if (proc->round_cells)
+ for (struct crosstabulation *xt = proc->pivots;
+ xt < &proc->pivots[proc->n_pivots]; xt++)
+ {
+ struct freq *e, *next;
+ HMAP_FOR_EACH_SAFE (e, next, struct freq, node, &xt->data)
+ {
+ e->count = round_weight (proc, e->count);
+ if (e->count == 0.0)
+ {
+ hmap_delete (&xt->data, &e->node);
+ free (e);
+ }
+ }
+ }
+
+ /* Convert hash tables into sorted arrays of entries. */
+ for (struct crosstabulation *xt = proc->pivots;
+ xt < &proc->pivots[proc->n_pivots]; xt++)
+ {
+ xt->n_entries = hmap_count (&xt->data);
+ xt->entries = xnmalloc (xt->n_entries, sizeof *xt->entries);
+
+ size_t i = 0;
+ struct freq *e;
+ HMAP_FOR_EACH (e, struct freq, node, &xt->data)
+ xt->entries[i++] = e;
+
+ hmap_destroy (&xt->data);
+
+ sort (xt->entries, xt->n_entries, sizeof *xt->entries,
+ proc->descending ? compare_table_entry_3way_inv : compare_table_entry_3way,
+ xt);
+ }
+
+ make_summary_table (proc);
+
+ /* Output each pivot table. */
+ for (struct crosstabulation *xt = proc->pivots;
+ xt < &proc->pivots[proc->n_pivots]; xt++)
+ {
+ output_crosstabulation (proc, xt, lexer);
+ if (proc->barchart)
+ {
+ int n_vars = (xt->n_vars > 2 ? 2 : xt->n_vars);
+ const struct variable **vars = XCALLOC (n_vars, const struct variable*);
+ for (size_t i = 0; i < n_vars; i++)
+ vars[i] = xt->vars[i].var;
+ chart_submit (barchart_create (vars, n_vars, _("Count"),
+ false,
+ xt->entries, xt->n_entries));
+ free (vars);
+ }
+ }
+
+ /* Free output and prepare for next split file. */
+ for (struct crosstabulation *xt = proc->pivots;
+ xt < &proc->pivots[proc->n_pivots]; xt++)
+ {
+ xt->missing = 0.0;
+
+ /* Free the members that were allocated in this function(and the values
+ owned by the entries.
+
+ The other pointer members are either both allocated and destroyed at a
+ lower level (in output_crosstabulation), or both allocated and
+ destroyed at a higher level (in crs_custom_tables and free_proc,
+ respectively). */
+ for (size_t i = 0; i < xt->n_vars; i++)
+ {
+ int width = var_get_width (xt->vars[i].var);
+ if (value_needs_init (width))
+ for (size_t j = 0; j < xt->n_entries; j++)
+ value_destroy (&xt->entries[j]->values[i], width);
+ }
+
+ for (size_t i = 0; i < xt->n_entries; i++)
+ free (xt->entries[i]);
+ free (xt->entries);
+ }
+}
+
+static void
+make_crosstabulation_subset (struct crosstabulation *xt, size_t row0,
+ size_t row1, struct crosstabulation *subset)
+{
+ *subset = *xt;
+ if (xt->n_vars > 2)
+ {
+ assert (xt->n_consts == 0);
+ subset->n_vars = 2;
+ subset->vars = xt->vars;
+
+ subset->n_consts = xt->n_vars - 2;
+ subset->const_vars = xt->vars + 2;
+ subset->const_indexes = xcalloc (subset->n_consts,
+ sizeof *subset->const_indexes);
+ for (size_t i = 0; i < subset->n_consts; i++)
+ {
+ const union value *value = &xt->entries[row0]->values[2 + i];
+
+ for (size_t j = 0; j < xt->vars[2 + i].n_values; j++)
+ if (value_equal (&xt->vars[2 + i].values[j], value,
+ var_get_width (xt->vars[2 + i].var)))
+ {
+ subset->const_indexes[i] = j;
+ goto found;
+ }
+ NOT_REACHED ();
+ found: ;
+ }
+ }
+ subset->entries = &xt->entries[row0];
+ subset->n_entries = row1 - row0;
+}
+
+static int
+compare_table_entry_var_3way (const struct freq *a,
+ const struct freq *b,
+ const struct crosstabulation *xt,
+ int idx)
+{
+ return value_compare_3way (&a->values[idx], &b->values[idx],
+ var_get_width (xt->vars[idx].var));
+}
+
+static int
+compare_table_entry_vars_3way (const struct freq *a,
+ const struct freq *b,
+ const struct crosstabulation *xt,
+ int idx0, int idx1)
+{
+ for (int i = idx1 - 1; i >= idx0; i--)
+ {
+ int cmp = compare_table_entry_var_3way (a, b, xt, i);
+ if (cmp != 0)
+ return cmp;
+ }
+ return 0;
+}
+
+/* Compare the struct freq at *AP to the one at *BP and
+ return a strcmp()-type result. */
+static int
+compare_table_entry_3way (const void *ap_, const void *bp_, const void *xt_)
+{
+ const struct freq *const *ap = ap_;
+ const struct freq *const *bp = bp_;
+ const struct freq *a = *ap;
+ const struct freq *b = *bp;
+ const struct crosstabulation *xt = xt_;
+
+ int cmp = compare_table_entry_vars_3way (a, b, xt, 2, xt->n_vars);
+ if (cmp != 0)
+ return cmp;
+
+ cmp = compare_table_entry_var_3way (a, b, xt, ROW_VAR);
+ if (cmp != 0)
+ return cmp;
+
+ return compare_table_entry_var_3way (a, b, xt, COL_VAR);
+}
+
+/* Inverted version of compare_table_entry_3way */
+static int
+compare_table_entry_3way_inv (const void *ap_, const void *bp_, const void *xt_)
+{
+ return -compare_table_entry_3way (ap_, bp_, xt_);
+}
+
+/* Output a table summarizing the cases processed. */
+static void
+make_summary_table (struct crosstabs_proc *proc)
+{
+ struct pivot_table *table = pivot_table_create (N_("Summary"));
+ pivot_table_set_weight_var (table, dict_get_weight (proc->dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Percent"), PIVOT_RC_PERCENT);
+
+ struct pivot_dimension *cases = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Cases"),
+ N_("Valid"), N_("Missing"), N_("Total"));
+ cases->root->show_label = true;
+
+ struct pivot_dimension *tables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Crosstabulation"));
+ for (struct crosstabulation *xt = &proc->pivots[0];
+ xt < &proc->pivots[proc->n_pivots]; xt++)
+ {
+ struct string name = DS_EMPTY_INITIALIZER;
+ for (size_t i = 0; i < xt->n_vars; i++)
+ {
+ if (i > 0)
+ ds_put_cstr (&name, " × ");
+ ds_put_cstr (&name, var_to_string (xt->vars[i].var));
+ }
+
+ int row = pivot_category_create_leaf (
+ tables->root,
+ pivot_value_new_user_text_nocopy (ds_steal_cstr (&name)));
+
+ double valid = 0.;
+ for (size_t i = 0; i < xt->n_entries; i++)
+ valid += xt->entries[i]->count;
+
+ double n[3];
+ n[0] = valid;
+ n[1] = xt->missing;
+ n[2] = n[0] + n[1];
+ for (int i = 0; i < 3; i++)
+ {
+ pivot_table_put3 (table, 0, i, row, pivot_value_new_number (n[i]));
+ pivot_table_put3 (table, 1, i, row,
+ pivot_value_new_number (n[i] / n[2] * 100.0));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+\f
+/* Output. */
+
+static struct pivot_table *create_crosstab_table (
+ struct crosstabs_proc *, struct crosstabulation *,
+ size_t crs_leaves[CRS_N_CELLS]);
+static struct pivot_table *create_chisq_table (struct crosstabulation *);
+static struct pivot_table *create_sym_table (struct crosstabulation *);
+static struct pivot_table *create_risk_table (
+ struct crosstabulation *, struct pivot_dimension **risk_statistics);
+static struct pivot_table *create_direct_table (struct crosstabulation *);
+static void display_crosstabulation (struct crosstabs_proc *,
+ struct crosstabulation *,
+ struct pivot_table *,
+ size_t crs_leaves[CRS_N_CELLS]);
+static void display_chisq (struct crosstabulation *, struct pivot_table *);
+static void display_symmetric (struct crosstabs_proc *,
+ struct crosstabulation *, struct pivot_table *);
+static void display_risk (struct crosstabulation *, struct pivot_table *,
+ struct pivot_dimension *risk_statistics);
+static void display_directional (struct crosstabs_proc *,
+ struct crosstabulation *,
+ struct pivot_table *);
+static void delete_missing (struct crosstabulation *);
+static void build_matrix (struct crosstabulation *);
+
+/* Output pivot table XT in the context of PROC. */
+static void
+output_crosstabulation (struct crosstabs_proc *proc, struct crosstabulation *xt,
+ struct lexer *lexer)
+{
+ for (size_t i = 0; i < xt->n_vars; i++)
+ enum_var_values (xt, i, proc->descending);
+
+ if (xt->vars[COL_VAR].n_values == 0)
+ {
+ struct string vars;
+
+ ds_init_cstr (&vars, var_to_string (xt->vars[0].var));
+ for (size_t i = 1; i < xt->n_vars; i++)
+ ds_put_format (&vars, " × %s", var_to_string (xt->vars[i].var));
+
+ /* TRANSLATORS: The %s here describes a crosstabulation. It takes the
+ form "var1 * var2 * var3 * ...". */
+ lex_ofs_msg (lexer, SW, xt->start_ofs, xt->end_ofs,
+ _("Crosstabulation %s contained no non-missing cases."),
+ ds_cstr (&vars));
+
+ ds_destroy (&vars);
+ for (size_t i = 0; i < xt->n_vars; i++)
+ free_var_values (xt, i);
+ return;
+ }
+
+ size_t crs_leaves[CRS_N_CELLS];
+ struct pivot_table *table = (proc->cells
+ ? create_crosstab_table (proc, xt, crs_leaves)
+ : NULL);
+ struct pivot_table *chisq = (proc->statistics & CRS_ST_CHISQ
+ ? create_chisq_table (xt)
+ : NULL);
+ struct pivot_table *sym
+ = (proc->statistics & (CRS_ST_PHI | CRS_ST_CC | CRS_ST_BTAU | CRS_ST_CTAU
+ | CRS_ST_GAMMA | CRS_ST_CORR | CRS_ST_KAPPA)
+ ? create_sym_table (xt)
+ : NULL);
+ struct pivot_dimension *risk_statistics = NULL;
+ struct pivot_table *risk = (proc->statistics & CRS_ST_RISK
+ ? create_risk_table (xt, &risk_statistics)
+ : NULL);
+ struct pivot_table *direct
+ = (proc->statistics & (CRS_ST_LAMBDA | CRS_ST_UC | CRS_ST_D | CRS_ST_ETA)
+ ? create_direct_table (xt)
+ : NULL);
+
+ size_t row0 = 0;
+ size_t row1 = 0;
+ while (find_crosstab (xt, &row0, &row1))
+ {
+ struct crosstabulation x;
+
+ make_crosstabulation_subset (xt, row0, row1, &x);
+
+ size_t n_rows = x.vars[ROW_VAR].n_values;
+ size_t n_cols = x.vars[COL_VAR].n_values;
+ if (size_overflow_p (xtimes (xtimes (n_rows, n_cols), sizeof (double))))
+ xalloc_die ();
+ x.row_tot = xmalloc (n_rows * sizeof *x.row_tot);
+ x.col_tot = xmalloc (n_cols * sizeof *x.col_tot);
+ x.mat = xmalloc (n_rows * n_cols * sizeof *x.mat);
+
+ build_matrix (&x);
+
+ /* Find the first variable that differs from the last subtable. */
+ if (table)
+ display_crosstabulation (proc, &x, table, crs_leaves);
+
+ if (proc->exclude == 0)
+ delete_missing (&x);
+
+ if (chisq)
+ display_chisq (&x, chisq);
+
+ if (sym)
+ display_symmetric (proc, &x, sym);
+ if (risk)
+ display_risk (&x, risk, risk_statistics);
+ if (direct)
+ display_directional (proc, &x, direct);
+
+ free (x.mat);
+ free (x.row_tot);
+ free (x.col_tot);
+ free (x.const_indexes);
+ }
+
+ if (table)
+ pivot_table_submit (table);
+
+ if (chisq)
+ pivot_table_submit (chisq);
+
+ if (sym)
+ pivot_table_submit (sym);
+
+ if (risk)
+ {
+ if (!pivot_table_is_empty (risk))
+ pivot_table_submit (risk);
+ else
+ pivot_table_unref (risk);
+ }
+
+ if (direct)
+ pivot_table_submit (direct);
+
+ for (size_t i = 0; i < xt->n_vars; i++)
+ free_var_values (xt, i);
+}
+
+static void
+build_matrix (struct crosstabulation *x)
+{
+ const int col_var_width = var_get_width (x->vars[COL_VAR].var);
+ const int row_var_width = var_get_width (x->vars[ROW_VAR].var);
+ size_t n_rows = x->vars[ROW_VAR].n_values;
+ size_t n_cols = x->vars[COL_VAR].n_values;
+
+ double *mp = x->mat;
+ size_t col = 0;
+ size_t row = 0;
+ for (struct freq **p = x->entries; p < &x->entries[x->n_entries]; p++)
+ {
+ const struct freq *te = *p;
+
+ while (!value_equal (&x->vars[ROW_VAR].values[row],
+ &te->values[ROW_VAR], row_var_width))
+ {
+ for (; col < n_cols; col++)
+ *mp++ = 0.0;
+ col = 0;
+ row++;
+ }
+
+ while (!value_equal (&x->vars[COL_VAR].values[col],
+ &te->values[COL_VAR], col_var_width))
+ {
+ *mp++ = 0.0;
+ col++;
+ }
+
+ *mp++ = te->count;
+ if (++col >= n_cols)
+ {
+ col = 0;
+ row++;
+ }
+ }
+ while (mp < &x->mat[n_cols * n_rows])
+ *mp++ = 0.0;
+ assert (mp == &x->mat[n_cols * n_rows]);
+
+ /* Column totals, row totals, ns_rows. */
+ mp = x->mat;
+ for (col = 0; col < n_cols; col++)
+ x->col_tot[col] = 0.0;
+ for (row = 0; row < n_rows; row++)
+ x->row_tot[row] = 0.0;
+ x->ns_rows = 0;
+ for (row = 0; row < n_rows; row++)
+ {
+ bool row_is_empty = true;
+ for (col = 0; col < n_cols; col++)
+ {
+ if (*mp != 0.0)
+ {
+ row_is_empty = false;
+ x->col_tot[col] += *mp;
+ x->row_tot[row] += *mp;
+ }
+ mp++;
+ }
+ if (!row_is_empty)
+ x->ns_rows++;
+ }
+ assert (mp == &x->mat[n_cols * n_rows]);
+
+ /* ns_cols. */
+ x->ns_cols = 0;
+ for (col = 0; col < n_cols; col++)
+ for (row = 0; row < n_rows; row++)
+ if (x->mat[col + row * n_cols] != 0.0)
+ {
+ x->ns_cols++;
+ break;
+ }
+
+ /* Grand total. */
+ x->total = 0.0;
+ for (col = 0; col < n_cols; col++)
+ x->total += x->col_tot[col];
+}
+
+static void
+add_var_dimension (struct pivot_table *table, const struct xtab_var *var,
+ enum pivot_axis_type axis_type, bool total)
+{
+ struct pivot_dimension *d = pivot_dimension_create__ (
+ table, axis_type, pivot_value_new_variable (var->var));
+
+ struct pivot_footnote *missing_footnote = pivot_table_create_footnote (
+ table, pivot_value_new_text (N_("Missing value")));
+
+ struct pivot_category *group = pivot_category_create_group__ (
+ d->root, pivot_value_new_variable (var->var));
+ for (size_t j = 0; j < var->n_values; j++)
+ {
+ struct pivot_value *value = pivot_value_new_var_value (
+ var->var, &var->values[j]);
+ if (var_is_value_missing (var->var, &var->values[j]))
+ pivot_value_add_footnote (value, missing_footnote);
+ pivot_category_create_leaf (group, value);
+ }
+
+ if (total)
+ pivot_category_create_leaf (d->root, pivot_value_new_text (N_("Total")));
+}
+
+static struct pivot_table *
+create_crosstab_table (struct crosstabs_proc *proc, struct crosstabulation *xt,
+ size_t crs_leaves[CRS_N_CELLS])
+{
+ /* Title. */
+ struct string title = DS_EMPTY_INITIALIZER;
+ for (size_t i = 0; i < xt->n_vars; i++)
+ {
+ if (i)
+ ds_put_cstr (&title, " × ");
+ ds_put_cstr (&title, var_to_string (xt->vars[i].var));
+ }
+ for (size_t i = 0; i < xt->n_consts; i++)
+ {
+ const struct variable *var = xt->const_vars[i].var;
+ const union value *value = &xt->entries[0]->values[2 + i];
+ char *s;
+
+ ds_put_format (&title, ", %s=", var_to_string (var));
+
+ /* Insert the formatted value of VAR without any leading spaces. */
+ s = data_out (value, var_get_encoding (var), var_get_print_format (var),
+ settings_get_fmt_settings ());
+ ds_put_cstr (&title, s + strspn (s, " "));
+ free (s);
+ }
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_user_text_nocopy (ds_steal_cstr (&title)),
+ "Crosstabulation");
+ pivot_table_set_weight_format (table, &proc->weight_format);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"));
+
+ struct statistic
+ {
+ const char *label;
+ const char *rc;
+ };
+ static const struct statistic stats[CRS_N_CELLS] =
+ {
+#define C(KEYWORD, STRING, RC) { STRING, RC },
+ CRS_CELLS
+#undef C
+ };
+ for (size_t i = 0; i < CRS_N_CELLS; i++)
+ if (proc->cells & (1u << i) && stats[i].label)
+ crs_leaves[i] = pivot_category_create_leaf_rc (
+ statistics->root, pivot_value_new_text (stats[i].label),
+ stats[i].rc);
+
+ for (size_t i = 0; i < xt->n_vars; i++)
+ add_var_dimension (table, &xt->vars[i],
+ i == COL_VAR ? PIVOT_AXIS_COLUMN : PIVOT_AXIS_ROW,
+ true);
+
+ return table;
+}
+
+static struct pivot_table *
+create_chisq_table (struct crosstabulation *xt)
+{
+ struct pivot_table *chisq = pivot_table_create (N_("Chi-Square Tests"));
+ pivot_table_set_weight_format (chisq, &xt->weight_format);
+
+ pivot_dimension_create (
+ chisq, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Pearson Chi-Square"),
+ N_("Likelihood Ratio"),
+ N_("Fisher's Exact Test"),
+ N_("Continuity Correction"),
+ N_("Linear-by-Linear Association"),
+ N_("N of Valid Cases"), PIVOT_RC_COUNT);
+
+ pivot_dimension_create (
+ chisq, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Value"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_COUNT,
+ N_("Asymptotic Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ for (size_t i = 2; i < xt->n_vars; i++)
+ add_var_dimension (chisq, &xt->vars[i], PIVOT_AXIS_ROW, false);
+
+ return chisq;
+}
+
+/* Symmetric measures. */
+static struct pivot_table *
+create_sym_table (struct crosstabulation *xt)
+{
+ struct pivot_table *sym = pivot_table_create (N_("Symmetric Measures"));
+ pivot_table_set_weight_format (sym, &xt->weight_format);
+
+ pivot_dimension_create (
+ sym, PIVOT_AXIS_COLUMN, N_("Values"),
+ N_("Value"), PIVOT_RC_OTHER,
+ N_("Asymp. Std. Error"), PIVOT_RC_OTHER,
+ N_("Approx. T"), PIVOT_RC_OTHER,
+ N_("Approx. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ sym, PIVOT_AXIS_ROW, N_("Statistics"));
+ pivot_category_create_group (
+ statistics->root, N_("Nominal by Nominal"),
+ N_("Phi"), N_("Cramer's V"), N_("Contingency Coefficient"));
+ pivot_category_create_group (
+ statistics->root, N_("Ordinal by Ordinal"),
+ N_("Kendall's tau-b"), N_("Kendall's tau-c"),
+ N_("Gamma"), N_("Spearman Correlation"));
+ pivot_category_create_group (
+ statistics->root, N_("Interval by Interval"),
+ N_("Pearson's R"));
+ pivot_category_create_group (
+ statistics->root, N_("Measure of Agreement"),
+ N_("Kappa"));
+ pivot_category_create_leaves (statistics->root, N_("N of Valid Cases"),
+ PIVOT_RC_COUNT);
+
+ for (size_t i = 2; i < xt->n_vars; i++)
+ add_var_dimension (sym, &xt->vars[i], PIVOT_AXIS_ROW, false);
+
+ return sym;
+}
+
+/* Risk estimate. */
+static struct pivot_table *
+create_risk_table (struct crosstabulation *xt,
+ struct pivot_dimension **risk_statistics)
+{
+ struct pivot_table *risk = pivot_table_create (N_("Risk Estimate"));
+ pivot_table_set_weight_format (risk, &xt->weight_format);
+
+ struct pivot_dimension *values = pivot_dimension_create (
+ risk, PIVOT_AXIS_COLUMN, N_("Values"),
+ N_("Value"), PIVOT_RC_OTHER);
+ pivot_category_create_group (
+ /* xgettext:no-c-format */
+ values->root, N_("95% Confidence Interval"),
+ N_("Lower"), PIVOT_RC_OTHER,
+ N_("Upper"), PIVOT_RC_OTHER);
+
+ *risk_statistics = pivot_dimension_create (
+ risk, PIVOT_AXIS_ROW, N_("Statistics"));
+
+ for (size_t i = 2; i < xt->n_vars; i++)
+ add_var_dimension (risk, &xt->vars[i], PIVOT_AXIS_ROW, false);
+
+ return risk;
+}
+
+static void
+create_direct_stat (struct pivot_category *parent,
+ const struct crosstabulation *xt,
+ const char *name, bool symmetric)
+{
+ struct pivot_category *group = pivot_category_create_group (
+ parent, name);
+ if (symmetric)
+ pivot_category_create_leaf (group, pivot_value_new_text (N_("Symmetric")));
+
+ char *row_label = xasprintf (_("%s Dependent"),
+ var_to_string (xt->vars[ROW_VAR].var));
+ pivot_category_create_leaf (group, pivot_value_new_user_text_nocopy (
+ row_label));
+
+ char *col_label = xasprintf (_("%s Dependent"),
+ var_to_string (xt->vars[COL_VAR].var));
+ pivot_category_create_leaf (group, pivot_value_new_user_text_nocopy (
+ col_label));
+}
+
+/* Directional measures. */
+static struct pivot_table *
+create_direct_table (struct crosstabulation *xt)
+{
+ struct pivot_table *direct = pivot_table_create (N_("Directional Measures"));
+ pivot_table_set_weight_format (direct, &xt->weight_format);
+
+ pivot_dimension_create (
+ direct, PIVOT_AXIS_COLUMN, N_("Values"),
+ N_("Value"), PIVOT_RC_OTHER,
+ N_("Asymp. Std. Error"), PIVOT_RC_OTHER,
+ N_("Approx. T"), PIVOT_RC_OTHER,
+ N_("Approx. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ direct, PIVOT_AXIS_ROW, N_("Statistics"));
+ struct pivot_category *nn = pivot_category_create_group (
+ statistics->root, N_("Nominal by Nominal"));
+ create_direct_stat (nn, xt, N_("Lambda"), true);
+ create_direct_stat (nn, xt, N_("Goodman and Kruskal tau"), false);
+ create_direct_stat (nn, xt, N_("Uncertainty Coefficient"), true);
+ struct pivot_category *oo = pivot_category_create_group (
+ statistics->root, N_("Ordinal by Ordinal"));
+ create_direct_stat (oo, xt, N_("Somers' d"), true);
+ struct pivot_category *ni = pivot_category_create_group (
+ statistics->root, N_("Nominal by Interval"));
+ create_direct_stat (ni, xt, N_("Eta"), false);
+
+ for (size_t i = 2; i < xt->n_vars; i++)
+ add_var_dimension (direct, &xt->vars[i], PIVOT_AXIS_ROW, false);
+
+ return direct;
+}
+
+/* Delete missing rows and columns for statistical analysis when
+ /MISSING=REPORT. */
+static void
+delete_missing (struct crosstabulation *xt)
+{
+ size_t n_rows = xt->vars[ROW_VAR].n_values;
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+
+ for (size_t r = 0; r < n_rows; r++)
+ if (var_is_num_missing (xt->vars[ROW_VAR].var,
+ xt->vars[ROW_VAR].values[r].f) == MV_USER)
+ {
+ for (size_t c = 0; c < n_cols; c++)
+ xt->mat[c + r * n_cols] = 0.;
+ xt->ns_rows--;
+ }
+
+
+ for (size_t c = 0; c < n_cols; c++)
+ if (var_is_num_missing (xt->vars[COL_VAR].var,
+ xt->vars[COL_VAR].values[c].f) == MV_USER)
+ {
+ for (size_t r = 0; r < n_rows; r++)
+ xt->mat[c + r * n_cols] = 0.;
+ xt->ns_cols--;
+ }
+}
+
+static bool
+find_crosstab (struct crosstabulation *xt, size_t *row0p, size_t *row1p)
+{
+ size_t row0 = *row1p;
+ if (row0 >= xt->n_entries)
+ return false;
+
+ size_t row1;
+ for (row1 = row0 + 1; row1 < xt->n_entries; row1++)
+ {
+ struct freq *a = xt->entries[row0];
+ struct freq *b = xt->entries[row1];
+ if (compare_table_entry_vars_3way (a, b, xt, 2, xt->n_vars) != 0)
+ break;
+ }
+ *row0p = row0;
+ *row1p = row1;
+ return true;
+}
+
+/* Compares `union value's A_ and B_ and returns a strcmp()-like
+ result. WIDTH_ points to an int which is either 0 for a
+ numeric value or a string width for a string value. */
+static int
+compare_value_3way (const void *a_, const void *b_, const void *width_)
+{
+ const union value *a = a_;
+ const union value *b = b_;
+ const int *width = width_;
+
+ return value_compare_3way (a, b, *width);
+}
+
+/* Inverted version of the above */
+static int
+compare_value_3way_inv (const void *a_, const void *b_, const void *width_)
+{
+ return -compare_value_3way (a_, b_, width_);
+}
+
+
+/* Given an array of ENTRY_CNT table_entry structures starting at
+ ENTRIES, creates a sorted list of the values that the variable
+ with index VAR_IDX takes on. Stores the array of the values in
+ XT->values and the number of values in XT->n_values. */
+static void
+enum_var_values (const struct crosstabulation *xt, int var_idx,
+ bool descending)
+{
+ struct xtab_var *xv = &xt->vars[var_idx];
+ const struct var_range *range = get_var_range (xt->proc, xv->var);
+
+ if (range)
+ {
+ xv->values = xnmalloc (range->count, sizeof *xv->values);
+ xv->n_values = range->count;
+ for (size_t i = 0; i < range->count; i++)
+ xv->values[i].f = range->min + i;
+ }
+ else
+ {
+ int width = var_get_width (xv->var);
+ struct hmapx set = HMAPX_INITIALIZER (set);
+
+ for (size_t i = 0; i < xt->n_entries; i++)
+ {
+ const struct freq *te = xt->entries[i];
+ const union value *value = &te->values[var_idx];
+ size_t hash = value_hash (value, width, 0);
+
+ const union value *iter;
+ struct hmapx_node *node;
+ HMAPX_FOR_EACH_WITH_HASH (iter, node, hash, &set)
+ if (value_equal (iter, value, width))
+ goto next_entry;
+
+ hmapx_insert (&set, (union value *) value, hash);
+
+ next_entry: ;
+ }
+
+ xv->n_values = hmapx_count (&set);
+ xv->values = xnmalloc (xv->n_values, sizeof *xv->values);
+ size_t i = 0;
+ const union value *iter;
+ struct hmapx_node *node;
+ HMAPX_FOR_EACH (iter, node, &set)
+ xv->values[i++] = *iter;
+ hmapx_destroy (&set);
+
+ sort (xv->values, xv->n_values, sizeof *xv->values,
+ descending ? compare_value_3way_inv : compare_value_3way,
+ &width);
+ }
+}
+
+static void
+free_var_values (const struct crosstabulation *xt, int var_idx)
+{
+ struct xtab_var *xv = &xt->vars[var_idx];
+ free (xv->values);
+ xv->values = NULL;
+ xv->n_values = 0;
+}
+
+/* Displays the crosstabulation table. */
+static void
+display_crosstabulation (struct crosstabs_proc *proc,
+ struct crosstabulation *xt, struct pivot_table *table,
+ size_t crs_leaves[CRS_N_CELLS])
+{
+ size_t n_rows = xt->vars[ROW_VAR].n_values;
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+
+ size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
+ assert (xt->n_vars == 2);
+ for (size_t i = 0; i < xt->n_consts; i++)
+ indexes[i + 3] = xt->const_indexes[i];
+
+ /* Put in the actual cells. */
+ double *mp = xt->mat;
+ for (size_t r = 0; r < n_rows; r++)
+ {
+ if (!xt->row_tot[r] && proc->mode != INTEGER)
+ continue;
+
+ indexes[ROW_VAR + 1] = r;
+ for (size_t c = 0; c < n_cols; c++)
+ {
+ if (!xt->col_tot[c] && proc->mode != INTEGER)
+ continue;
+
+ indexes[COL_VAR + 1] = c;
+
+ double expected_value = xt->row_tot[r] * xt->col_tot[c] / xt->total;
+ double residual = *mp - expected_value;
+ double sresidual = residual / sqrt (expected_value);
+ double asresidual
+ = residual / sqrt (expected_value
+ * (1. - xt->row_tot[r] / xt->total)
+ * (1. - xt->col_tot[c] / xt->total));
+ double entries[CRS_N_CELLS] = {
+ [CRS_CL_COUNT] = *mp,
+ [CRS_CL_ROW] = *mp / xt->row_tot[r] * 100.,
+ [CRS_CL_COLUMN] = *mp / xt->col_tot[c] * 100.,
+ [CRS_CL_TOTAL] = *mp / xt->total * 100.,
+ [CRS_CL_EXPECTED] = expected_value,
+ [CRS_CL_RESIDUAL] = residual,
+ [CRS_CL_SRESIDUAL] = sresidual,
+ [CRS_CL_ASRESIDUAL] = asresidual,
+ };
+ for (size_t i = 0; i < proc->n_cells; i++)
+ {
+ int cell = proc->a_cells[i];
+ indexes[0] = crs_leaves[cell];
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (entries[cell]));
+ }
+
+ mp++;
+ }
+ }
+
+ /* Row totals. */
+ for (size_t r = 0; r < n_rows; r++)
+ {
+ if (!xt->row_tot[r] && proc->mode != INTEGER)
+ continue;
+
+ double expected_value = xt->row_tot[r] / xt->total;
+ double entries[CRS_N_CELLS] = {
+ [CRS_CL_COUNT] = xt->row_tot[r],
+ [CRS_CL_ROW] = 100.0,
+ [CRS_CL_COLUMN] = expected_value * 100.,
+ [CRS_CL_TOTAL] = expected_value * 100.,
+ [CRS_CL_EXPECTED] = expected_value,
+ [CRS_CL_RESIDUAL] = SYSMIS,
+ [CRS_CL_SRESIDUAL] = SYSMIS,
+ [CRS_CL_ASRESIDUAL] = SYSMIS,
+ };
+ for (size_t i = 0; i < proc->n_cells; i++)
+ {
+ int cell = proc->a_cells[i];
+ double entry = entries[cell];
+ if (entry != SYSMIS)
+ {
+ indexes[ROW_VAR + 1] = r;
+ indexes[COL_VAR + 1] = n_cols;
+ indexes[0] = crs_leaves[cell];
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (entry));
+ }
+ }
+ }
+
+ for (size_t c = 0; c <= n_cols; c++)
+ {
+ if (c < n_cols && !xt->col_tot[c] && proc->mode != INTEGER)
+ continue;
+
+ double ct = c < n_cols ? xt->col_tot[c] : xt->total;
+ double expected_value = ct / xt->total;
+ double entries[CRS_N_CELLS] = {
+ [CRS_CL_COUNT] = ct,
+ [CRS_CL_ROW] = expected_value * 100.0,
+ [CRS_CL_COLUMN] = 100.0,
+ [CRS_CL_TOTAL] = expected_value * 100.,
+ [CRS_CL_EXPECTED] = expected_value,
+ [CRS_CL_RESIDUAL] = SYSMIS,
+ [CRS_CL_SRESIDUAL] = SYSMIS,
+ [CRS_CL_ASRESIDUAL] = SYSMIS,
+ };
+ for (size_t i = 0; i < proc->n_cells; i++)
+ {
+ size_t cell = proc->a_cells[i];
+ double entry = entries[cell];
+ if (entry != SYSMIS)
+ {
+ indexes[ROW_VAR + 1] = n_rows;
+ indexes[COL_VAR + 1] = c;
+ indexes[0] = crs_leaves[cell];
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (entry));
+ }
+ }
+ }
+
+ free (indexes);
+}
+
+static void calc_r (struct crosstabulation *,
+ double *XT, double *Y, double *, double *, double *);
+static void calc_chisq (struct crosstabulation *,
+ double[N_CHISQ], int[N_CHISQ], double *, double *);
+
+/* Display chi-square statistics. */
+static void
+display_chisq (struct crosstabulation *xt, struct pivot_table *chisq)
+{
+ double chisq_v[N_CHISQ];
+ double fisher1, fisher2;
+ int df[N_CHISQ];
+ calc_chisq (xt, chisq_v, df, &fisher1, &fisher2);
+
+ size_t *indexes = xnmalloc (chisq->n_dimensions, sizeof *indexes);
+ assert (xt->n_vars == 2);
+ for (size_t i = 0; i < xt->n_consts; i++)
+ indexes[i + 2] = xt->const_indexes[i];
+ for (size_t i = 0; i < N_CHISQ; i++)
+ {
+ indexes[0] = i;
+
+ double entries[5] = { SYSMIS, SYSMIS, SYSMIS, SYSMIS, SYSMIS };
+ if (i == 2)
+ {
+ entries[3] = fisher2;
+ entries[4] = fisher1;
+ }
+ else if (chisq_v[i] != SYSMIS)
+ {
+ entries[0] = chisq_v[i];
+ entries[1] = df[i];
+ entries[2] = gsl_cdf_chisq_Q (chisq_v[i], df[i]);
+ }
+
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ if (entries[j] != SYSMIS)
+ {
+ indexes[1] = j;
+ pivot_table_put (chisq, indexes, chisq->n_dimensions,
+ pivot_value_new_number (entries[j]));
+ }
+ }
+
+ indexes[0] = 5;
+ indexes[1] = 0;
+ pivot_table_put (chisq, indexes, chisq->n_dimensions,
+ pivot_value_new_number (xt->total));
+
+ free (indexes);
+}
+
+static bool calc_symmetric (struct crosstabs_proc *, struct crosstabulation *,
+ double[N_SYMMETRIC], double[N_SYMMETRIC],
+ double[N_SYMMETRIC],
+ double[3], double[3], double[3]);
+
+/* Display symmetric measures. */
+static void
+display_symmetric (struct crosstabs_proc *proc, struct crosstabulation *xt,
+ struct pivot_table *sym)
+{
+ double sym_v[N_SYMMETRIC], sym_ase[N_SYMMETRIC], sym_t[N_SYMMETRIC];
+ double somers_d_v[3], somers_d_ase[3], somers_d_t[3];
+
+ if (!calc_symmetric (proc, xt, sym_v, sym_ase, sym_t,
+ somers_d_v, somers_d_ase, somers_d_t))
+ return;
+
+ size_t *indexes = xnmalloc (sym->n_dimensions, sizeof *indexes);
+ assert (xt->n_vars == 2);
+ for (size_t i = 0; i < xt->n_consts; i++)
+ indexes[i + 2] = xt->const_indexes[i];
+
+ for (size_t i = 0; i < N_SYMMETRIC; i++)
+ {
+ if (sym_v[i] == SYSMIS)
+ continue;
+
+ indexes[1] = i;
+
+ double entries[] = { sym_v[i], sym_ase[i], sym_t[i] };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ if (entries[j] != SYSMIS)
+ {
+ indexes[0] = j;
+ pivot_table_put (sym, indexes, sym->n_dimensions,
+ pivot_value_new_number (entries[j]));
+ }
+ }
+
+ indexes[1] = N_SYMMETRIC;
+ indexes[0] = 0;
+ struct pivot_value *total = pivot_value_new_number (xt->total);
+ pivot_value_set_rc (sym, total, PIVOT_RC_COUNT);
+ pivot_table_put (sym, indexes, sym->n_dimensions, total);
+
+ free (indexes);
+}
+
+static bool calc_risk (struct crosstabulation *,
+ double[], double[], double[], union value *,
+ double *);
+
+/* Display risk estimate. */
+static void
+display_risk (struct crosstabulation *xt, struct pivot_table *risk,
+ struct pivot_dimension *risk_statistics)
+{
+ double risk_v[3], lower[3], upper[3], n_valid;
+ union value c[2];
+ if (!calc_risk (xt, risk_v, upper, lower, c, &n_valid))
+ return;
+ assert (risk_statistics);
+
+ size_t *indexes = xnmalloc (risk->n_dimensions, sizeof *indexes);
+ assert (xt->n_vars == 2);
+ for (size_t i = 0; i < xt->n_consts; i++)
+ indexes[i + 2] = xt->const_indexes[i];
+
+ for (size_t i = 0; i < 3; i++)
+ {
+ const struct variable *cv = xt->vars[COL_VAR].var;
+ const struct variable *rv = xt->vars[ROW_VAR].var;
+
+ if (risk_v[i] == SYSMIS)
+ continue;
+
+ struct string label = DS_EMPTY_INITIALIZER;
+ switch (i)
+ {
+ case 0:
+ ds_put_format (&label, _("Odds Ratio for %s"), var_to_string (rv));
+ ds_put_cstr (&label, " (");
+ var_append_value_name (rv, &c[0], &label);
+ ds_put_cstr (&label, " / ");
+ var_append_value_name (rv, &c[1], &label);
+ ds_put_cstr (&label, ")");
+ break;
+ case 1:
+ case 2:
+ ds_put_format (&label, _("For cohort %s = "), var_to_string (cv));
+ var_append_value_name (cv, &xt->vars[ROW_VAR].values[i - 1], &label);
+ break;
+ }
+
+ indexes[1] = pivot_category_create_leaf (
+ risk_statistics->root,
+ pivot_value_new_user_text_nocopy (ds_steal_cstr (&label)));
+
+ double entries[] = { risk_v[i], lower[i], upper[i] };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ {
+ indexes[0] = j;
+ pivot_table_put (risk, indexes, risk->n_dimensions,
+ pivot_value_new_number (entries[i]));
+ }
+ }
+ indexes[1] = pivot_category_create_leaf (
+ risk_statistics->root,
+ pivot_value_new_text (N_("N of Valid Cases")));
+ indexes[0] = 0;
+ pivot_table_put (risk, indexes, risk->n_dimensions,
+ pivot_value_new_number (n_valid));
+ free (indexes);
+}
+
+static void calc_directional (struct crosstabs_proc *, struct crosstabulation *,
+ double[N_DIRECTIONAL], double[N_DIRECTIONAL],
+ double[N_DIRECTIONAL], double[N_DIRECTIONAL]);
+
+/* Display directional measures. */
+static void
+display_directional (struct crosstabs_proc *proc,
+ struct crosstabulation *xt, struct pivot_table *direct)
+{
+ double direct_v[N_DIRECTIONAL];
+ double direct_ase[N_DIRECTIONAL];
+ double direct_t[N_DIRECTIONAL];
+ double sig[N_DIRECTIONAL];
+ calc_directional (proc, xt, direct_v, direct_ase, direct_t, sig);
+
+ size_t *indexes = xnmalloc (direct->n_dimensions, sizeof *indexes);
+ assert (xt->n_vars == 2);
+ for (size_t i = 0; i < xt->n_consts; i++)
+ indexes[i + 2] = xt->const_indexes[i];
+
+ for (size_t i = 0; i < N_DIRECTIONAL; i++)
+ {
+ if (direct_v[i] == SYSMIS)
+ continue;
+
+ indexes[1] = i;
+
+ double entries[] = {
+ direct_v[i], direct_ase[i], direct_t[i], sig[i],
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ if (entries[j] != SYSMIS)
+ {
+ indexes[0] = j;
+ pivot_table_put (direct, indexes, direct->n_dimensions,
+ pivot_value_new_number (entries[j]));
+ }
+ }
+
+ free (indexes);
+}
+\f
+/* Statistical calculations. */
+
+/* Returns the value of the logarithm of gamma (factorial) function for an integer
+ argument XT. */
+static double
+log_gamma_int (double xt)
+{
+ double r = 0;
+ for (int i = 2; i < xt; i++)
+ r += log(i);
+ return r;
+}
+
+/* Calculate P_r as specified in _SPSS Statistical Algorithms_,
+ Appendix 5. */
+static inline double
+Pr (int a, int b, int c, int d)
+{
+ return exp (log_gamma_int (a + b + 1.) - log_gamma_int (a + 1.)
+ + log_gamma_int (c + d + 1.) - log_gamma_int (b + 1.)
+ + log_gamma_int (a + c + 1.) - log_gamma_int (c + 1.)
+ + log_gamma_int (b + d + 1.) - log_gamma_int (d + 1.)
+ - log_gamma_int (a + b + c + d + 1.));
+}
+
+/* Swap the contents of A and B. */
+static inline void
+swap (int *a, int *b)
+{
+ int t = *a;
+ *a = *b;
+ *b = t;
+}
+
+/* Calculate significance for Fisher's exact test as specified in
+ _SPSS Statistical Algorithms_, Appendix 5. */
+static void
+calc_fisher (int a, int b, int c, int d, double *fisher1, double *fisher2)
+{
+ if (MIN (c, d) < MIN (a, b))
+ swap (&a, &c), swap (&b, &d);
+ if (MIN (b, d) < MIN (a, c))
+ swap (&a, &b), swap (&c, &d);
+ if (b * c < a * d)
+ {
+ if (b < c)
+ swap (&a, &b), swap (&c, &d);
+ else
+ swap (&a, &c), swap (&b, &d);
+ }
+
+ double pn1 = Pr (a, b, c, d);
+ *fisher1 = pn1;
+ for (int xt = 1; xt <= a; xt++)
+ *fisher1 += Pr (a - xt, b + xt, c + xt, d - xt);
+
+ *fisher2 = *fisher1;
+ for (int xt = 1; xt <= b; xt++)
+ {
+ double p = Pr (a + xt, b - xt, c - xt, d + xt);
+ if (p < pn1)
+ *fisher2 += p;
+ }
+}
+
+/* Calculates chi-squares into CHISQ. MAT is a matrix with N_COLS
+ columns with values COLS and N_ROWS rows with values ROWS. Values
+ in the matrix sum to xt->total. */
+static void
+calc_chisq (struct crosstabulation *xt,
+ double chisq[N_CHISQ], int df[N_CHISQ],
+ double *fisher1, double *fisher2)
+{
+ chisq[0] = chisq[1] = 0.;
+ chisq[2] = chisq[3] = chisq[4] = SYSMIS;
+ *fisher1 = *fisher2 = SYSMIS;
+
+ df[0] = df[1] = (xt->ns_cols - 1) * (xt->ns_rows - 1);
+
+ if (xt->ns_rows <= 1 || xt->ns_cols <= 1)
+ {
+ chisq[0] = chisq[1] = SYSMIS;
+ return;
+ }
+
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+ FOR_EACH_POPULATED_ROW (r, xt)
+ FOR_EACH_POPULATED_COLUMN (c, xt)
+ {
+ const double expected = xt->row_tot[r] * xt->col_tot[c] / xt->total;
+ const double freq = xt->mat[n_cols * r + c];
+ const double residual = freq - expected;
+
+ chisq[0] += residual * residual / expected;
+ if (freq)
+ chisq[1] += freq * log (expected / freq);
+ }
+
+ if (chisq[0] == 0.)
+ chisq[0] = SYSMIS;
+
+ if (chisq[1] != 0.)
+ chisq[1] *= -2.;
+ else
+ chisq[1] = SYSMIS;
+
+ /* Calculate Yates and Fisher exact test. */
+ if (xt->ns_cols == 2 && xt->ns_rows == 2)
+ {
+ int nz_cols[2];
+
+ size_t j = 0;
+ FOR_EACH_POPULATED_COLUMN (c, xt)
+ {
+ nz_cols[j++] = c;
+ if (j == 2)
+ break;
+ }
+ assert (j == 2);
+
+ double f11 = xt->mat[nz_cols[0]];
+ double f12 = xt->mat[nz_cols[1]];
+ double f21 = xt->mat[nz_cols[0] + n_cols];
+ double f22 = xt->mat[nz_cols[1] + n_cols];
+
+ /* Yates. */
+ const double xt_ = fabs (f11 * f22 - f12 * f21) - 0.5 * xt->total;
+
+ if (xt_ > 0.)
+ chisq[3] = (xt->total * pow2 (xt_)
+ / (f11 + f12) / (f21 + f22)
+ / (f11 + f21) / (f12 + f22));
+ else
+ chisq[3] = 0.;
+
+ df[3] = 1.;
+
+ /* Fisher. */
+ calc_fisher (f11 + .5, f12 + .5, f21 + .5, f22 + .5, fisher1, fisher2);
+ }
+
+ /* Calculate Mantel-Haenszel. */
+ if (var_is_numeric (xt->vars[ROW_VAR].var)
+ && var_is_numeric (xt->vars[COL_VAR].var))
+ {
+ double r, ase_0, ase_1;
+ calc_r (xt, (double *) xt->vars[ROW_VAR].values,
+ (double *) xt->vars[COL_VAR].values,
+ &r, &ase_0, &ase_1);
+
+ chisq[4] = (xt->total - 1.) * r * r;
+ df[4] = 1;
+ }
+}
+
+/* Calculate the value of Pearson's r. r is stored into R, its T value into
+ T, and standard error into ERROR. The row and column values must be
+ passed in XT and Y. */
+static void
+calc_r (struct crosstabulation *xt,
+ double *XT, double *Y, double *r, double *t, double *error)
+{
+ size_t n_rows = xt->vars[ROW_VAR].n_values;
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+
+ double sum_XYf = 0;
+ for (size_t i = 0; i < n_rows; i++)
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ double fij = xt->mat[j + i * n_cols];
+ double product = XT[i] * Y[j];
+ double temp = fij * product;
+ sum_XYf += temp;
+ }
+
+ double sum_Xr = 0;
+ double sum_X2r = 0;
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ sum_Xr += XT[i] * xt->row_tot[i];
+ sum_X2r += pow2 (XT[i]) * xt->row_tot[i];
+ }
+ double Xbar = sum_Xr / xt->total;
+
+ double sum_Yc = 0;
+ double sum_Y2c = 0;
+ for (size_t i = 0; i < n_cols; i++)
+ {
+ sum_Yc += Y[i] * xt->col_tot[i];
+ sum_Y2c += Y[i] * Y[i] * xt->col_tot[i];
+ }
+ double Ybar = sum_Yc / xt->total;
+
+ double S = sum_XYf - sum_Xr * sum_Yc / xt->total;
+ double SX = sum_X2r - pow2 (sum_Xr) / xt->total;
+ double SY = sum_Y2c - pow2 (sum_Yc) / xt->total;
+ double T = sqrt (SX * SY);
+ *r = S / T;
+ *t = *r / sqrt (1 - pow2 (*r)) * sqrt (xt->total - 2);
+
+ double s = 0;
+ double c = 0;
+ for (size_t i = 0; i < n_rows; i++)
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ double Xresid = XT[i] - Xbar;
+ double Yresid = Y[j] - Ybar;
+ double temp = (T * Xresid * Yresid
+ - ((S / (2. * T))
+ * (Xresid * Xresid * SY + Yresid * Yresid * SX)));
+ double y = xt->mat[j + i * n_cols] * temp * temp - c;
+ double t = s + y;
+ c = (t - s) - y;
+ s = t;
+ }
+ *error = sqrt (s) / (T * T);
+}
+
+/* Calculate symmetric statistics and their asymptotic standard
+ errors. Returns false if none could be calculated. */
+static bool
+calc_symmetric (struct crosstabs_proc *proc, struct crosstabulation *xt,
+ double v[N_SYMMETRIC], double ase[N_SYMMETRIC],
+ double t[N_SYMMETRIC],
+ double somers_d_v[3], double somers_d_ase[3],
+ double somers_d_t[3])
+{
+ size_t n_rows = xt->vars[ROW_VAR].n_values;
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+
+ size_t q = MIN (xt->ns_rows, xt->ns_cols);
+ if (q <= 1)
+ return false;
+
+ for (size_t i = 0; i < N_SYMMETRIC; i++)
+ v[i] = ase[i] = t[i] = SYSMIS;
+
+ /* Phi, Cramer's V, contingency coefficient. */
+ if (proc->statistics & (CRS_ST_PHI | CRS_ST_CC))
+ {
+ double Xp = 0.; /* Pearson chi-square. */
+
+ FOR_EACH_POPULATED_ROW (r, xt)
+ FOR_EACH_POPULATED_COLUMN (c, xt)
+ {
+ double expected = xt->row_tot[r] * xt->col_tot[c] / xt->total;
+ double freq = xt->mat[n_cols * r + c];
+ double residual = freq - expected;
+
+ Xp += residual * residual / expected;
+ }
+
+ if (proc->statistics & CRS_ST_PHI)
+ {
+ v[0] = sqrt (Xp / xt->total);
+ v[1] = sqrt (Xp / (xt->total * (q - 1)));
+ }
+ if (proc->statistics & CRS_ST_CC)
+ v[2] = sqrt (Xp / (Xp + xt->total));
+ }
+
+ if (proc->statistics & (CRS_ST_BTAU | CRS_ST_CTAU
+ | CRS_ST_GAMMA | CRS_ST_D))
+ {
+ double Dr = pow2 (xt->total);
+ for (size_t r = 0; r < n_rows; r++)
+ Dr -= pow2 (xt->row_tot[r]);
+
+ double Dc = pow2 (xt->total);
+ for (size_t c = 0; c < n_cols; c++)
+ Dc -= pow2 (xt->col_tot[c]);
+
+ double *cum = xnmalloc (n_cols * n_rows, sizeof *cum);
+ for (size_t c = 0; c < n_cols; c++)
+ {
+ double ct = 0.;
+
+ for (size_t r = 0; r < n_rows; r++)
+ cum[c + r * n_cols] = ct += xt->mat[c + r * n_cols];
+ }
+
+ /* P and Q. */
+ double P = 0;
+ double Q = 0;
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ double Cij = 0;
+ for (size_t j = 1; j < n_cols; j++)
+ Cij += xt->col_tot[j] - cum[j + i * n_cols];
+
+ double Dij = 0;
+ if (i > 0)
+ for (size_t j = 1; j < n_cols; j++)
+ Dij += cum[j + (i - 1) * n_cols];
+
+ for (size_t j = 0;;)
+ {
+ double fij = xt->mat[j + i * n_cols];
+ P += fij * Cij;
+ Q += fij * Dij;
+
+ if (++j >= n_cols)
+ break;
+
+ Cij -= xt->col_tot[j] - cum[j + i * n_cols];
+ Dij += xt->col_tot[j - 1] - cum[j - 1 + i * n_cols];
+
+ if (i > 0)
+ {
+ Cij += cum[j - 1 + (i - 1) * n_cols];
+ Dij -= cum[j + (i - 1) * n_cols];
+ }
+ }
+ }
+
+ if (proc->statistics & CRS_ST_BTAU)
+ v[3] = (P - Q) / sqrt (Dr * Dc);
+ if (proc->statistics & CRS_ST_CTAU)
+ v[4] = (q * (P - Q)) / (pow2 (xt->total) * (q - 1));
+ if (proc->statistics & CRS_ST_GAMMA)
+ v[5] = (P - Q) / (P + Q);
+
+ /* ASE for tau-b, tau-c, gamma. Calculations could be
+ eliminated here, at expense of memory. */
+ double btau_cum = 0;
+ double ctau_cum = 0;
+ double gamma_cum = 0;
+ double d_yx_cum = 0;
+ double d_xy_cum = 0;
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ double Cij = 0;
+ for (size_t j = 1; j < n_cols; j++)
+ Cij += xt->col_tot[j] - cum[j + i * n_cols];
+
+ double Dij = 0;
+ if (i > 0)
+ for (size_t j = 1; j < n_cols; j++)
+ Dij += cum[j + (i - 1) * n_cols];
+
+ for (size_t j = 0;;)
+ {
+ double fij = xt->mat[j + i * n_cols];
+
+ if (proc->statistics & CRS_ST_BTAU)
+ btau_cum += fij * pow2 (2. * sqrt (Dr * Dc) * (Cij - Dij)
+ + v[3] * (xt->row_tot[i] * Dc
+ + xt->col_tot[j] * Dr));
+ ctau_cum += fij * pow2 (Cij - Dij);
+
+ if (proc->statistics & CRS_ST_GAMMA)
+ gamma_cum += fij * pow2 (Q * Cij - P * Dij);
+
+ if (proc->statistics & CRS_ST_D)
+ {
+ d_yx_cum += fij * pow2 (Dr * (Cij - Dij)
+ - (P - Q) * (xt->total - xt->row_tot[i]));
+ d_xy_cum += fij * pow2 (Dc * (Dij - Cij)
+ - (Q - P) * (xt->total - xt->col_tot[j]));
+ }
+
+ if (++j >= n_cols)
+ break;
+
+ Cij -= xt->col_tot[j] - cum[j + i * n_cols];
+ Dij += xt->col_tot[j - 1] - cum[j - 1 + i * n_cols];
+
+ if (i > 0)
+ {
+ Cij += cum[j - 1 + (i - 1) * n_cols];
+ Dij -= cum[j + (i - 1) * n_cols];
+ }
+ }
+ }
+
+ if (proc->statistics & CRS_ST_BTAU)
+ {
+ double btau_var = ((btau_cum
+ - (xt->total * pow2 (xt->total * (P - Q) / sqrt (Dr * Dc) * (Dr + Dc))))
+ / pow2 (Dr * Dc));
+ ase[3] = sqrt (btau_var);
+ t[3] = v[3] / (2 * sqrt ((ctau_cum - (P - Q) * (P - Q) / xt->total)
+ / (Dr * Dc)));
+ }
+ if (proc->statistics & CRS_ST_CTAU)
+ {
+ ase[4] = ((2 * q / ((q - 1) * pow2 (xt->total)))
+ * sqrt (ctau_cum - (P - Q) * (P - Q) / xt->total));
+ t[4] = v[4] / ase[4];
+ }
+ if (proc->statistics & CRS_ST_GAMMA)
+ {
+ ase[5] = ((4. / ((P + Q) * (P + Q))) * sqrt (gamma_cum));
+ t[5] = v[5] / (2. / (P + Q)
+ * sqrt (ctau_cum - (P - Q) * (P - Q) / xt->total));
+ }
+ if (proc->statistics & CRS_ST_D)
+ {
+ somers_d_v[0] = (P - Q) / (.5 * (Dc + Dr));
+ somers_d_ase[0] = SYSMIS;
+ somers_d_t[0] = (somers_d_v[0]
+ / (4 / (Dc + Dr)
+ * sqrt (ctau_cum - pow2 (P - Q) / xt->total)));
+ somers_d_v[1] = (P - Q) / Dc;
+ somers_d_ase[1] = 2. / pow2 (Dc) * sqrt (d_xy_cum);
+ somers_d_t[1] = (somers_d_v[1]
+ / (2. / Dc
+ * sqrt (ctau_cum - pow2 (P - Q) / xt->total)));
+ somers_d_v[2] = (P - Q) / Dr;
+ somers_d_ase[2] = 2. / pow2 (Dr) * sqrt (d_yx_cum);
+ somers_d_t[2] = (somers_d_v[2]
+ / (2. / Dr
+ * sqrt (ctau_cum - pow2 (P - Q) / xt->total)));
+ }
+
+ free (cum);
+ }
+
+ /* Spearman correlation, Pearson's r. */
+ if (proc->statistics & CRS_ST_CORR)
+ {
+ double *R = xmalloc (sizeof *R * n_rows);
+ double c = 0;
+ double s = 0;
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ R[i] = s + (xt->row_tot[i] + 1.) / 2.;
+ double y = xt->row_tot[i] - c;
+ double t = s + y;
+ c = (t - s) - y;
+ s = t;
+ }
+
+ double *C = xmalloc (sizeof *C * n_cols);
+ c = s = 0;
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ C[j] = s + (xt->col_tot[j] + 1.) / 2;
+ double y = xt->col_tot[j] - c;
+ double t = s + y;
+ c = (t - s) - y;
+ s = t;
+ }
+
+ calc_r (xt, R, C, &v[6], &t[6], &ase[6]);
+
+ free (R);
+ free (C);
+
+ calc_r (xt, (double *) xt->vars[ROW_VAR].values,
+ (double *) xt->vars[COL_VAR].values,
+ &v[7], &t[7], &ase[7]);
+ }
+
+ /* Cohen's kappa. */
+ if (proc->statistics & CRS_ST_KAPPA && xt->ns_rows == xt->ns_cols)
+ {
+ double sum_fii = 0;
+ double sum_rici = 0;
+ double sum_fiiri_ci = 0;
+ double sum_riciri_ci = 0;
+ for (size_t i = 0, j = 0; i < xt->ns_rows; i++, j++)
+ {
+ while (xt->col_tot[j] == 0.)
+ j++;
+
+ double prod = xt->row_tot[i] * xt->col_tot[j];
+ double sum = xt->row_tot[i] + xt->col_tot[j];
+
+ sum_fii += xt->mat[j + i * n_cols];
+ sum_rici += prod;
+ sum_fiiri_ci += xt->mat[j + i * n_cols] * sum;
+ sum_riciri_ci += prod * sum;
+ }
+
+ double sum_fijri_ci2 = 0;
+ for (size_t i = 0; i < xt->ns_rows; i++)
+ for (size_t j = 0; j < xt->ns_cols; j++)
+ {
+ double sum = xt->row_tot[i] + xt->col_tot[j];
+ sum_fijri_ci2 += xt->mat[j + i * n_cols] * sum * sum;
+ }
+
+ v[8] = (xt->total * sum_fii - sum_rici) / (pow2 (xt->total) - sum_rici);
+
+ double ase_under_h0 = sqrt ((pow2 (xt->total) * sum_rici
+ + sum_rici * sum_rici
+ - xt->total * sum_riciri_ci)
+ / (xt->total * (pow2 (xt->total) - sum_rici) * (pow2 (xt->total) - sum_rici)));
+
+ ase[8] = sqrt (xt->total * (((sum_fii * (xt->total - sum_fii))
+ / pow2 (pow2 (xt->total) - sum_rici))
+ + ((2. * (xt->total - sum_fii)
+ * (2. * sum_fii * sum_rici
+ - xt->total * sum_fiiri_ci))
+ / pow3 (pow2 (xt->total) - sum_rici))
+ + (pow2 (xt->total - sum_fii)
+ * (xt->total * sum_fijri_ci2 - 4.
+ * sum_rici * sum_rici)
+ / pow4 (pow2 (xt->total) - sum_rici))));
+
+ t[8] = v[8] / ase_under_h0;
+ }
+
+ return true;
+}
+
+/* Calculate risk estimate. */
+static bool
+calc_risk (struct crosstabulation *xt,
+ double *value, double *upper, double *lower, union value *c,
+ double *n_valid)
+{
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+
+ for (size_t i = 0; i < 3; i++)
+ value[i] = upper[i] = lower[i] = SYSMIS;
+
+ if (xt->ns_rows != 2 || xt->ns_cols != 2)
+ return false;
+
+ /* Find populated columns. */
+ size_t nz_cols[2];
+ size_t n = 0;
+ FOR_EACH_POPULATED_COLUMN (c, xt)
+ nz_cols[n++] = c;
+ assert (n == 2);
+
+ /* Find populated rows. */
+ size_t nz_rows[2];
+ n = 0;
+ FOR_EACH_POPULATED_ROW (r, xt)
+ nz_rows[n++] = r;
+ assert (n == 2);
+
+ double f11 = xt->mat[nz_cols[0] + n_cols * nz_rows[0]];
+ double f12 = xt->mat[nz_cols[1] + n_cols * nz_rows[0]];
+ double f21 = xt->mat[nz_cols[0] + n_cols * nz_rows[1]];
+ double f22 = xt->mat[nz_cols[1] + n_cols * nz_rows[1]];
+ *n_valid = f11 + f12 + f21 + f22;
+
+ c[0] = xt->vars[COL_VAR].values[nz_cols[0]];
+ c[1] = xt->vars[COL_VAR].values[nz_cols[1]];
+
+ value[0] = (f11 * f22) / (f12 * f21);
+ double v = sqrt (1. / f11 + 1. / f12 + 1. / f21 + 1. / f22);
+ lower[0] = value[0] * exp (-1.960 * v);
+ upper[0] = value[0] * exp (1.960 * v);
+
+ value[1] = (f11 * (f21 + f22)) / (f21 * (f11 + f12));
+ v = sqrt ((f12 / (f11 * (f11 + f12)))
+ + (f22 / (f21 * (f21 + f22))));
+ lower[1] = value[1] * exp (-1.960 * v);
+ upper[1] = value[1] * exp (1.960 * v);
+
+ value[2] = (f12 * (f21 + f22)) / (f22 * (f11 + f12));
+ v = sqrt ((f11 / (f12 * (f11 + f12)))
+ + (f21 / (f22 * (f21 + f22))));
+ lower[2] = value[2] * exp (-1.960 * v);
+ upper[2] = value[2] * exp (1.960 * v);
+
+ return true;
+}
+
+/* Calculate directional measures. */
+static void
+calc_directional (struct crosstabs_proc *proc, struct crosstabulation *xt,
+ double v[N_DIRECTIONAL], double ase[N_DIRECTIONAL],
+ double t[N_DIRECTIONAL], double sig[N_DIRECTIONAL])
+{
+ size_t n_rows = xt->vars[ROW_VAR].n_values;
+ size_t n_cols = xt->vars[COL_VAR].n_values;
+ for (size_t i = 0; i < N_DIRECTIONAL; i++)
+ v[i] = ase[i] = t[i] = sig[i] = SYSMIS;
+
+ /* Lambda. */
+ if (proc->statistics & CRS_ST_LAMBDA)
+ {
+ /* Find maximum for each row and their sum. */
+ double *fim = xnmalloc (n_rows, sizeof *fim);
+ size_t *fim_index = xnmalloc (n_rows, sizeof *fim_index);
+ double sum_fim = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ double max = xt->mat[i * n_cols];
+ size_t index = 0;
+
+ for (size_t j = 1; j < n_cols; j++)
+ if (xt->mat[j + i * n_cols] > max)
+ {
+ max = xt->mat[j + i * n_cols];
+ index = j;
+ }
+
+ fim[i] = max;
+ sum_fim += max;
+ fim_index[i] = index;
+ }
+
+ /* Find maximum for each column. */
+ double *fmj = xnmalloc (n_cols, sizeof *fmj);
+ size_t *fmj_index = xnmalloc (n_cols, sizeof *fmj_index);
+ double sum_fmj = 0.0;
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ double max = xt->mat[j];
+ size_t index = 0;
+
+ for (size_t i = 1; i < n_rows; i++)
+ if (xt->mat[j + i * n_cols] > max)
+ {
+ max = xt->mat[j + i * n_cols];
+ index = i;
+ }
+
+ fmj[j] = max;
+ sum_fmj += max;
+ fmj_index[j] = index;
+ }
+
+ /* Find maximum row total. */
+ double rm = xt->row_tot[0];
+ size_t rm_index = 0;
+ for (size_t i = 1; i < n_rows; i++)
+ if (xt->row_tot[i] > rm)
+ {
+ rm = xt->row_tot[i];
+ rm_index = i;
+ }
+
+ /* Find maximum column total. */
+ double cm = xt->col_tot[0];
+ size_t cm_index = 0;
+ for (size_t j = 1; j < n_cols; j++)
+ if (xt->col_tot[j] > cm)
+ {
+ cm = xt->col_tot[j];
+ cm_index = j;
+ }
+
+ v[0] = (sum_fim + sum_fmj - cm - rm) / (2. * xt->total - rm - cm);
+ v[1] = (sum_fmj - rm) / (xt->total - rm);
+ v[2] = (sum_fim - cm) / (xt->total - cm);
+
+ /* ASE1 for Y given XT. */
+ {
+ double accum = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ if (cm_index == fim_index[i])
+ accum += fim[i];
+ ase[2] = sqrt ((xt->total - sum_fim) * (sum_fim + cm - 2. * accum)
+ / pow3 (xt->total - cm));
+ }
+
+ /* ASE0 for Y given XT. */
+ {
+ double accum = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ if (cm_index != fim_index[i])
+ accum += (xt->mat[i * n_cols + fim_index[i]]
+ + xt->mat[i * n_cols + cm_index]);
+ t[2] = v[2] / (sqrt (accum - pow2 (sum_fim - cm) / xt->total) / (xt->total - cm));
+ }
+
+ /* ASE1 for XT given Y. */
+ {
+ double accum = 0.0;
+ for (size_t j = 0; j < n_cols; j++)
+ if (rm_index == fmj_index[j])
+ accum += fmj[j];
+ ase[1] = sqrt ((xt->total - sum_fmj) * (sum_fmj + rm - 2. * accum)
+ / pow3 (xt->total - rm));
+ }
+
+ /* ASE0 for XT given Y. */
+ {
+ double accum = 0.0;
+ for (size_t j = 0; j < n_cols; j++)
+ if (rm_index != fmj_index[j])
+ accum += (xt->mat[j + n_cols * fmj_index[j]]
+ + xt->mat[j + n_cols * rm_index]);
+ t[1] = v[1] / (sqrt (accum - pow2 (sum_fmj - rm) / xt->total) / (xt->total - rm));
+ }
+
+ /* Symmetric ASE0 and ASE1. */
+ {
+ double accum0 = 0.0;
+ double accum1 = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ int temp0 = (fmj_index[j] == i) + (fim_index[i] == j);
+ int temp1 = (i == rm_index) + (j == cm_index);
+ accum0 += xt->mat[j + i * n_cols] * pow2 (temp0 - temp1);
+ accum1 += (xt->mat[j + i * n_cols]
+ * pow2 (temp0 + (v[0] - 1.) * temp1));
+ }
+ ase[0] = sqrt (accum1 - 4. * xt->total * v[0] * v[0]) / (2. * xt->total - rm - cm);
+ t[0] = v[0] / (sqrt (accum0 - pow2 (sum_fim + sum_fmj - cm - rm) / xt->total)
+ / (2. * xt->total - rm - cm));
+ }
+
+ for (size_t i = 0; i < 3; i++)
+ sig[i] = 2 * gsl_cdf_ugaussian_Q (t[i]);
+
+ free (fim);
+ free (fim_index);
+ free (fmj);
+ free (fmj_index);
+
+ /* Tau. */
+ double sum_fij2_ri = 0.0;
+ double sum_fij2_ci = 0.0;
+ FOR_EACH_POPULATED_ROW (i, xt)
+ FOR_EACH_POPULATED_COLUMN (j, xt)
+ {
+ double temp = pow2 (xt->mat[j + i * n_cols]);
+ sum_fij2_ri += temp / xt->row_tot[i];
+ sum_fij2_ci += temp / xt->col_tot[j];
+ }
+
+ double sum_ri2 = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ sum_ri2 += pow2 (xt->row_tot[i]);
+
+ double sum_cj2 = 0.0;
+ for (size_t j = 0; j < n_cols; j++)
+ sum_cj2 += pow2 (xt->col_tot[j]);
+
+ v[3] = (xt->total * sum_fij2_ci - sum_ri2) / (pow2 (xt->total) - sum_ri2);
+ v[4] = (xt->total * sum_fij2_ri - sum_cj2) / (pow2 (xt->total) - sum_cj2);
+ }
+
+ if (proc->statistics & CRS_ST_UC)
+ {
+ double UX = 0.0;
+ FOR_EACH_POPULATED_ROW (i, xt)
+ UX -= xt->row_tot[i] / xt->total * log (xt->row_tot[i] / xt->total);
+
+ double UY = 0.0;
+ FOR_EACH_POPULATED_COLUMN (j, xt)
+ UY -= xt->col_tot[j] / xt->total * log (xt->col_tot[j] / xt->total);
+
+ double UXY = 0.0;
+ double P = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ double entry = xt->mat[j + i * n_cols];
+
+ if (entry <= 0.)
+ continue;
+
+ P += entry * pow2 (log (xt->col_tot[j] * xt->row_tot[i] / (xt->total * entry)));
+ UXY -= entry / xt->total * log (entry / xt->total);
+ }
+
+ double ase1_yx = 0.0;
+ double ase1_xy = 0.0;
+ double ase1_sym = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ double entry = xt->mat[j + i * n_cols];
+
+ if (entry <= 0.)
+ continue;
+
+ ase1_yx += entry * pow2 (UY * log (entry / xt->row_tot[i])
+ + (UX - UXY) * log (xt->col_tot[j] / xt->total));
+ ase1_xy += entry * pow2 (UX * log (entry / xt->col_tot[j])
+ + (UY - UXY) * log (xt->row_tot[i] / xt->total));
+ ase1_sym += entry * pow2 ((UXY
+ * log (xt->row_tot[i] * xt->col_tot[j] / pow2 (xt->total)))
+ - (UX + UY) * log (entry / xt->total));
+ }
+
+ v[5] = 2. * ((UX + UY - UXY) / (UX + UY));
+ ase[5] = (2. / (xt->total * pow2 (UX + UY))) * sqrt (ase1_sym);
+ t[5] = SYSMIS;
+
+ v[6] = (UX + UY - UXY) / UX;
+ ase[6] = sqrt (ase1_xy) / (xt->total * UX * UX);
+ t[6] = v[6] / (sqrt (P - xt->total * pow2 (UX + UY - UXY)) / (xt->total * UX));
+
+ v[7] = (UX + UY - UXY) / UY;
+ ase[7] = sqrt (ase1_yx) / (xt->total * UY * UY);
+ t[7] = v[7] / (sqrt (P - xt->total * pow2 (UX + UY - UXY)) / (xt->total * UY));
+ }
+
+ /* Somers' D. */
+ if (proc->statistics & CRS_ST_D)
+ {
+ double v_dummy[N_SYMMETRIC];
+ double ase_dummy[N_SYMMETRIC];
+ double t_dummy[N_SYMMETRIC];
+ double somers_d_v[3];
+ double somers_d_ase[3];
+ double somers_d_t[3];
+
+ if (calc_symmetric (proc, xt, v_dummy, ase_dummy, t_dummy,
+ somers_d_v, somers_d_ase, somers_d_t))
+ {
+ for (size_t i = 0; i < 3; i++)
+ {
+ v[8 + i] = somers_d_v[i];
+ ase[8 + i] = somers_d_ase[i];
+ t[8 + i] = somers_d_t[i];
+ sig[8 + i] = 2 * gsl_cdf_ugaussian_Q (fabs (somers_d_t[i]));
+ }
+ }
+ }
+
+ /* Eta. */
+ if (proc->statistics & CRS_ST_ETA)
+ {
+ /* X dependent. */
+ double sum_Xr = 0.0;
+ double sum_X2r = 0.0;
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ sum_Xr += xt->vars[ROW_VAR].values[i].f * xt->row_tot[i];
+ sum_X2r += pow2 (xt->vars[ROW_VAR].values[i].f) * xt->row_tot[i];
+ }
+ double SX = sum_X2r - pow2 (sum_Xr) / xt->total;
+
+ double SXW = 0.0;
+ FOR_EACH_POPULATED_COLUMN (j, xt)
+ {
+ double cum = 0.0;
+
+ for (size_t i = 0; i < n_rows; i++)
+ {
+ SXW += (pow2 (xt->vars[ROW_VAR].values[i].f)
+ * xt->mat[j + i * n_cols]);
+ cum += (xt->vars[ROW_VAR].values[i].f
+ * xt->mat[j + i * n_cols]);
+ }
+
+ SXW -= cum * cum / xt->col_tot[j];
+ }
+ v[11] = sqrt (1. - SXW / SX);
+
+ /* Y dependent. */
+ double sum_Yc = 0.0;
+ double sum_Y2c = 0.0;
+ for (size_t i = 0; i < n_cols; i++)
+ {
+ sum_Yc += xt->vars[COL_VAR].values[i].f * xt->col_tot[i];
+ sum_Y2c += pow2 (xt->vars[COL_VAR].values[i].f) * xt->col_tot[i];
+ }
+ double SY = sum_Y2c - pow2 (sum_Yc) / xt->total;
+
+ double SYW = 0.0;
+ FOR_EACH_POPULATED_ROW (i, xt)
+ {
+ double cum = 0.0;
+ for (size_t j = 0; j < n_cols; j++)
+ {
+ SYW += (pow2 (xt->vars[COL_VAR].values[j].f)
+ * xt->mat[j + i * n_cols]);
+ cum += (xt->vars[COL_VAR].values[j].f
+ * xt->mat[j + i * n_cols]);
+ }
+
+ SYW -= cum * cum / xt->row_tot[i];
+ }
+ v[12] = sqrt (1. - SYW / SY);
+ }
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2021 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <math.h>
+#include <errno.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/data-in.h"
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/mrset.h"
+#include "data/subcase.h"
+#include "data/value-labels.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/token.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmap.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/string-array.h"
+#include "math/mode.h"
+#include "math/moments.h"
+#include "math/percentiles.h"
+#include "math/sort.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+struct ctables;
+
+/* The three forms of weighting supported by CTABLES. */
+enum ctables_weighting
+ {
+ CTW_EFFECTIVE, /* Effective base weight (WEIGHT subcommand). */
+ CTW_DICTIONARY, /* Dictionary weight. */
+ CTW_UNWEIGHTED /* No weight. */
+#define N_CTWS 3
+ };
+\f
+/* CTABLES table areas. */
+
+enum ctables_area_type
+ {
+ /* Within a section, where stacked variables divide one section from
+ another.
+
+ Keep CTAT_LAYER after CTAT_LAYERROW and CTAT_LAYERCOL so that
+ parse_ctables_summary_function() parses correctly. */
+ CTAT_TABLE, /* All layers of a whole section. */
+ CTAT_LAYERROW, /* Row in one layer within a section. */
+ CTAT_LAYERCOL, /* Column in one layer within a section. */
+ CTAT_LAYER, /* One layer within a section. */
+
+ /* Within a subtable, where a subtable pairs an innermost row variable with
+ an innermost column variable within a single layer. */
+ CTAT_SUBTABLE, /* Whole subtable. */
+ CTAT_ROW, /* Row within a subtable. */
+ CTAT_COL, /* Column within a subtable. */
+#define N_CTATS 7
+ };
+
+static const char *ctables_area_type_name[N_CTATS] = {
+ [CTAT_TABLE] = "TABLE",
+ [CTAT_LAYER] = "LAYER",
+ [CTAT_LAYERROW] = "LAYERROW",
+ [CTAT_LAYERCOL] = "LAYERCOL",
+ [CTAT_SUBTABLE] = "SUBTABLE",
+ [CTAT_ROW] = "ROW",
+ [CTAT_COL] = "COL",
+};
+
+/* Summary statistics for an area. */
+struct ctables_area
+ {
+ struct hmap_node node;
+ const struct ctables_cell *example;
+
+ /* Sequence number used for CTSF_ID. */
+ size_t sequence;
+
+ /* Weights for CTSF_areaPCT_COUNT, CTSF_areaPCT_VALIDN, and
+ CTSF_areaPCT_TOTALN. */
+ double count[N_CTWS];
+ double valid[N_CTWS];
+ double total[N_CTWS];
+
+ /* Sums for CTSF_areaPCT_SUM. */
+ struct ctables_sum *sums;
+ };
+
+struct ctables_sum
+ {
+ double sum[N_CTWS];
+ };
+\f
+/* CTABLES summary functions. */
+
+enum ctables_function_type
+ {
+ /* A function that operates on data in a single cell. It operates on
+ effective weights. It does not have an unweighted version. */
+ CTFT_CELL,
+
+ /* A function that operates on data in a single cell. The function
+ operates on effective weights and has a U-prefixed unweighted
+ version. */
+ CTFT_UCELL,
+
+ /* A function that operates on data in a single cell. It operates on
+ dictionary weights, and has U-prefixed unweighted version and an
+ E-prefixed effective weight version. */
+ CTFT_UECELL,
+
+ /* A function that operates on an area of cells. It operates on effective
+ weights and has a U-prefixed unweighted version. */
+ CTFT_AREA,
+ };
+
+enum ctables_format
+ {
+ CTF_COUNT, /* F40.0. */
+ CTF_PERCENT, /* PCT40.1. */
+ CTF_GENERAL /* Variable's print format. */
+ };
+
+enum ctables_function_availability
+ {
+ CTFA_ALL, /* Any variables. */
+ CTFA_SCALE, /* Only scale variables, totals, and subtotals. */
+ //CTFA_MRSETS, /* Only multiple-response sets */
+ };
+
+enum ctables_summary_function
+ {
+#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) ENUM,
+#include "ctables.inc"
+#undef S
+ };
+
+enum {
+#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) +1
+ N_CTSF_FUNCTIONS =
+#include "ctables.inc"
+#undef S
+};
+
+struct ctables_function_info
+ {
+ struct substring basename;
+ enum ctables_function_type type;
+ enum ctables_format format;
+ enum ctables_function_availability availability;
+
+ bool u_prefix; /* Accepts a 'U' prefix (for unweighted)? */
+ bool e_prefix; /* Accepts an 'E' prefix (for effective)? */
+ bool is_area; /* Needs an area prefix. */
+ };
+static const struct ctables_function_info ctables_function_info[N_CTSF_FUNCTIONS] = {
+#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) \
+ [ENUM] = { \
+ .basename = SS_LITERAL_INITIALIZER (NAME), \
+ .type = TYPE, \
+ .format = FORMAT, \
+ .availability = AVAILABILITY, \
+ .u_prefix = (TYPE) == CTFT_UCELL || (TYPE) == CTFT_UECELL || (TYPE) == CTFT_AREA, \
+ .e_prefix = (TYPE) == CTFT_UECELL, \
+ .is_area = (TYPE) == CTFT_AREA \
+ },
+#include "ctables.inc"
+#undef S
+};
+
+static struct fmt_spec
+ctables_summary_default_format (enum ctables_summary_function function,
+ const struct variable *var)
+{
+ static const enum ctables_format default_formats[] = {
+#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) [ENUM] = FORMAT,
+#include "ctables.inc"
+#undef S
+ };
+ switch (default_formats[function])
+ {
+ case CTF_COUNT:
+ return (struct fmt_spec) { .type = FMT_F, .w = 40 };
+
+ case CTF_PERCENT:
+ return (struct fmt_spec) { .type = FMT_PCT, .w = 40, .d = 1 };
+
+ case CTF_GENERAL:
+ return *var_get_print_format (var);
+
+ default:
+ NOT_REACHED ();
+ }
+}
+
+static enum ctables_function_availability
+ctables_function_availability (enum ctables_summary_function f)
+{
+ static enum ctables_function_availability availability[] = {
+#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) [ENUM] = AVAILABILITY,
+#include "ctables.inc"
+#undef S
+ };
+
+ return availability[f];
+}
+
+static bool
+parse_ctables_summary_function (struct lexer *lexer,
+ enum ctables_summary_function *function,
+ enum ctables_weighting *weighting,
+ enum ctables_area_type *area)
+{
+ if (!lex_force_id (lexer))
+ return false;
+
+ struct substring name = lex_tokss (lexer);
+ if (ss_ends_with_case (name, ss_cstr (".LCL"))
+ || ss_ends_with_case (name, ss_cstr (".UCL"))
+ || ss_ends_with_case (name, ss_cstr (".SE")))
+ {
+ lex_error (lexer, _("Support for LCL, UCL, and SE summary functions "
+ "is not yet implemented."));
+ return false;
+ }
+
+ bool u = ss_match_byte (&name, 'U') || ss_match_byte (&name, 'u');
+ bool e = !u && (ss_match_byte (&name, 'E') || ss_match_byte (&name, 'e'));
+
+ bool has_area = false;
+ *area = 0;
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ if (ss_match_string_case (&name, ss_cstr (ctables_area_type_name[at])))
+ {
+ has_area = true;
+ *area = at;
+
+ if (ss_equals_case (name, ss_cstr ("PCT")))
+ {
+ /* Special case where .COUNT suffix is omitted. */
+ *function = CTSF_areaPCT_COUNT;
+ *weighting = CTW_EFFECTIVE;
+ lex_get (lexer);
+ return true;
+ }
+ break;
+ }
+
+ for (int f = 0; f < N_CTSF_FUNCTIONS; f++)
+ {
+ const struct ctables_function_info *cfi = &ctables_function_info[f];
+ if (ss_equals_case (cfi->basename, name))
+ {
+ *function = f;
+ if ((u && !cfi->u_prefix) || (e && !cfi->e_prefix) || (has_area != cfi->is_area))
+ break;
+
+ *weighting = (e ? CTW_EFFECTIVE
+ : u ? CTW_UNWEIGHTED
+ : cfi->e_prefix ? CTW_DICTIONARY
+ : CTW_EFFECTIVE);
+ lex_get (lexer);
+ return true;
+ }
+ }
+
+ lex_error (lexer, _("Syntax error expecting summary function name."));
+ return false;
+}
+
+static const char *
+ctables_summary_function_name (enum ctables_summary_function function,
+ enum ctables_weighting weighting,
+ enum ctables_area_type area,
+ char *buffer, size_t bufsize)
+{
+ const struct ctables_function_info *cfi = &ctables_function_info[function];
+ snprintf (buffer, bufsize, "%s%s%s",
+ (weighting == CTW_UNWEIGHTED ? "U"
+ : weighting == CTW_DICTIONARY ? ""
+ : cfi->e_prefix ? "E"
+ : ""),
+ cfi->is_area ? ctables_area_type_name[area] : "",
+ cfi->basename.string);
+ return buffer;
+}
+
+static const char *
+ctables_summary_function_label__ (enum ctables_summary_function function,
+ enum ctables_weighting weighting,
+ enum ctables_area_type area)
+{
+ bool w = weighting != CTW_UNWEIGHTED;
+ bool d = weighting == CTW_DICTIONARY;
+ enum ctables_area_type a = area;
+ switch (function)
+ {
+ case CTSF_COUNT:
+ return d ? N_("Count") : w ? N_("Adjusted Count") : N_("Unweighted Count");
+
+ case CTSF_areaPCT_COUNT:
+ switch (a)
+ {
+ case CTAT_TABLE: return w ? N_("Table %") : N_("Unweighted Table %");
+ case CTAT_LAYER: return w ? N_("Layer %") : N_("Unweighted Layer %");
+ case CTAT_LAYERROW: return w ? N_("Layer Row %") : N_("Unweighted Layer Row %");
+ case CTAT_LAYERCOL: return w ? N_("Layer Column %") : N_("Unweighted Layer Column %");
+ case CTAT_SUBTABLE: return w ? N_("Subtable %") : N_("Unweighted Subtable %");
+ case CTAT_ROW: return w ? N_("Row %") : N_("Unweighted Row %");
+ case CTAT_COL: return w ? N_("Column %") : N_("Unweighted Column %");
+ }
+ NOT_REACHED ();
+
+ case CTSF_areaPCT_VALIDN:
+ switch (a)
+ {
+ case CTAT_TABLE: return w ? N_("Table Valid N %") : N_("Unweighted Table Valid N %");
+ case CTAT_LAYER: return w ? N_("Layer Valid N %") : N_("Unweighted Layer Valid N %");
+ case CTAT_LAYERROW: return w ? N_("Layer Row Valid N %") : N_("Unweighted Layer Row Valid N %");
+ case CTAT_LAYERCOL: return w ? N_("Layer Column Valid N %") : N_("Unweighted Layer Column Valid N %");
+ case CTAT_SUBTABLE: return w ? N_("Subtable Valid N %") : N_("Unweighted Subtable Valid N %");
+ case CTAT_ROW: return w ? N_("Row Valid N %") : N_("Unweighted Row Valid N %");
+ case CTAT_COL: return w ? N_("Column Valid N %") : N_("Unweighted Column Valid N %");
+ }
+ NOT_REACHED ();
+
+ case CTSF_areaPCT_TOTALN:
+ switch (a)
+ {
+ case CTAT_TABLE: return w ? N_("Table Total N %") : N_("Unweighted Table Total N %");
+ case CTAT_LAYER: return w ? N_("Layer Total N %") : N_("Unweighted Layer Total N %");
+ case CTAT_LAYERROW: return w ? N_("Layer Row Total N %") : N_("Unweighted Layer Row Total N %");
+ case CTAT_LAYERCOL: return w ? N_("Layer Column Total N %") : N_("Unweighted Layer Column Total N %");
+ case CTAT_SUBTABLE: return w ? N_("Subtable Total N %") : N_("Unweighted Subtable Total N %");
+ case CTAT_ROW: return w ? N_("Row Total N %") : N_("Unweighted Row Total N %");
+ case CTAT_COL: return w ? N_("Column Total N %") : N_("Unweighted Column Total N %");
+ }
+ NOT_REACHED ();
+
+ case CTSF_MAXIMUM: return N_("Maximum");
+ case CTSF_MEAN: return w ? N_("Mean") : N_("Unweighted Mean");
+ case CTSF_MEDIAN: return w ? N_("Median") : N_("Unweighted Median");
+ case CTSF_MINIMUM: return N_("Minimum");
+ case CTSF_MISSING: return w ? N_("Missing") : N_("Unweighted Missing");
+ case CTSF_MODE: return w ? N_("Mode") : N_("Unweighted Mode");
+ case CTSF_PTILE: NOT_REACHED ();
+ case CTSF_RANGE: return N_("Range");
+ case CTSF_SEMEAN: return w ? N_("Std Error of Mean") : N_("Unweighted Std Error of Mean");
+ case CTSF_STDDEV: return w ? N_("Std Deviation") : N_("Unweighted Std Deviation");
+ case CTSF_SUM: return w ? N_("Sum") : N_("Unweighted Sum");
+ case CTSF_TOTALN: return (d ? N_("Total N")
+ : w ? N_("Adjusted Total N")
+ : N_("Unweighted Total N"));
+ case CTSF_VALIDN: return (d ? N_("Valid N")
+ : w ? N_("Adjusted Valid N")
+ : N_("Unweighted Valid N"));
+ case CTSF_VARIANCE: return w ? N_("Variance") : N_("Unweighted Variance");
+ case CTSF_areaPCT_SUM:
+ switch (a)
+ {
+ case CTAT_TABLE: return w ? N_("Table Sum %") : N_("Unweighted Table Sum %");
+ case CTAT_LAYER: return w ? N_("Layer Sum %") : N_("Unweighted Layer Sum %");
+ case CTAT_LAYERROW: return w ? N_("Layer Row Sum %") : N_("Unweighted Layer Row Sum %");
+ case CTAT_LAYERCOL: return w ? N_("Layer Column Sum %") : N_("Unweighted Layer Column Sum %");
+ case CTAT_SUBTABLE: return w ? N_("Subtable Sum %") : N_("Unweighted Subtable Sum %");
+ case CTAT_ROW: return w ? N_("Row Sum %") : N_("Unweighted Row Sum %");
+ case CTAT_COL: return w ? N_("Column Sum %") : N_("Unweighted Column Sum %");
+ }
+ NOT_REACHED ();
+
+ case CTSF_areaID:
+ switch (a)
+ {
+ /* Don't bother translating these: they are for developers only. */
+ case CTAT_TABLE: return "Table ID";
+ case CTAT_LAYER: return "Layer ID";
+ case CTAT_LAYERROW: return "Layer Row ID";
+ case CTAT_LAYERCOL: return "Layer Column ID";
+ case CTAT_SUBTABLE: return "Subtable ID";
+ case CTAT_ROW: return "Row ID";
+ case CTAT_COL: return "Column ID";
+ }
+ NOT_REACHED ();
+ }
+
+ NOT_REACHED ();
+}
+
+static struct pivot_value *
+ctables_summary_function_label (enum ctables_summary_function function,
+ enum ctables_weighting weighting,
+ enum ctables_area_type area,
+ double percentile)
+{
+ if (function == CTSF_PTILE)
+ {
+ char *s = (weighting != CTW_UNWEIGHTED
+ ? xasprintf (_("Percentile %.2f"), percentile)
+ : xasprintf (_("Unweighted Percentile %.2f"), percentile));
+ return pivot_value_new_user_text_nocopy (s);
+ }
+ else
+ return pivot_value_new_text (ctables_summary_function_label__ (
+ function, weighting, area));
+}
+\f
+/* CTABLES summaries. */
+
+struct ctables_summary_spec
+ {
+ /* The calculation to be performed.
+
+ 'function' is the function to calculate. 'weighted' specifies whether
+ to use weighted or unweighted data (for functions that do not support a
+ choice, it must be true). 'calc_area' is the area over which the
+ calculation takes place (for functions that target only an individual
+ cell, it must be 0). For CTSF_PTILE only, 'percentile' is the
+ percentile between 0 and 100 (for other functions it must be 0). */
+ enum ctables_summary_function function;
+ enum ctables_weighting weighting;
+ enum ctables_area_type calc_area;
+ double percentile; /* CTSF_PTILE only. */
+
+ /* How to display the result of the calculation.
+
+ 'label' is a user-specified label, NULL if the user didn't specify
+ one.
+
+ 'user_area' is usually the same as 'calc_area', but when category labels
+ are rotated from one axis to another it swaps rows and columns.
+
+ 'format' is the format for displaying the output. If
+ 'is_ctables_format' is true, then 'format.type' is one of the special
+ CTEF_* formats instead of the standard ones. */
+ char *label;
+ enum ctables_area_type user_area;
+ struct fmt_spec format;
+ bool is_ctables_format; /* Is 'format' one of CTEF_*? */
+
+ size_t axis_idx; /* Leaf index if summary dimension in use. */
+ size_t sum_var_idx; /* Offset into 'sums' in ctables_area. */
+ };
+
+static void
+ctables_summary_spec_clone (struct ctables_summary_spec *dst,
+ const struct ctables_summary_spec *src)
+{
+ *dst = *src;
+ dst->label = xstrdup_if_nonnull (src->label);
+}
+
+static void
+ctables_summary_spec_uninit (struct ctables_summary_spec *s)
+{
+ if (s)
+ free (s->label);
+}
+\f
+/* Collections of summary functions. */
+
+struct ctables_summary_spec_set
+ {
+ struct ctables_summary_spec *specs;
+ size_t n;
+ size_t allocated;
+
+ /* The variable to which the summary specs are applied. */
+ struct variable *var;
+
+ /* Whether the variable to which the summary specs are applied is a scale
+ variable for the purpose of summarization.
+
+ (VALIDN and TOTALN act differently for summarizing scale and categorical
+ variables.) */
+ bool is_scale;
+
+ /* If any of these optional additional scale variables are missing, then
+ treat 'var' as if it's missing too. This is for implementing
+ SMISSING=LISTWISE. */
+ struct variable **listwise_vars;
+ size_t n_listwise_vars;
+ };
+
+static void
+ctables_summary_spec_set_clone (struct ctables_summary_spec_set *dst,
+ const struct ctables_summary_spec_set *src)
+{
+ struct ctables_summary_spec *specs
+ = (src->n ? xnmalloc (src->n, sizeof *specs) : NULL);
+ for (size_t i = 0; i < src->n; i++)
+ ctables_summary_spec_clone (&specs[i], &src->specs[i]);
+
+ *dst = (struct ctables_summary_spec_set) {
+ .specs = specs,
+ .n = src->n,
+ .allocated = src->n,
+ .var = src->var,
+ .is_scale = src->is_scale,
+ };
+}
+
+static void
+ctables_summary_spec_set_uninit (struct ctables_summary_spec_set *set)
+{
+ for (size_t i = 0; i < set->n; i++)
+ ctables_summary_spec_uninit (&set->specs[i]);
+ free (set->listwise_vars);
+ free (set->specs);
+}
+
+static bool
+is_listwise_missing (const struct ctables_summary_spec_set *specs,
+ const struct ccase *c)
+{
+ for (size_t i = 0; i < specs->n_listwise_vars; i++)
+ {
+ const struct variable *var = specs->listwise_vars[i];
+ if (var_is_num_missing (var, case_num (c, var)))
+ return true;
+ }
+
+ return false;
+}
+\f
+/* CTABLES postcompute expressions. */
+
+struct ctables_postcompute
+ {
+ struct hmap_node hmap_node; /* In struct ctables's 'pcompute' hmap. */
+ char *name; /* Name, without leading &. */
+
+ struct msg_location *location; /* Location of definition. */
+ struct ctables_pcexpr *expr;
+ char *label;
+ struct ctables_summary_spec_set *specs;
+ bool hide_source_cats;
+ };
+
+struct ctables_pcexpr
+ {
+ /* Precedence table:
+
+ ()
+ **
+ -
+ * /
+ - +
+ */
+ enum ctables_pcexpr_op
+ {
+ /* Terminals. */
+ CTPO_CONSTANT, /* 5 */
+ CTPO_CAT_NUMBER, /* [5] */
+ CTPO_CAT_STRING, /* ["STRING"] */
+ CTPO_CAT_NRANGE, /* [LO THRU 5] */
+ CTPO_CAT_SRANGE, /* ["A" THRU "B"] */
+ CTPO_CAT_MISSING, /* MISSING */
+ CTPO_CAT_OTHERNM, /* OTHERNM */
+ CTPO_CAT_SUBTOTAL, /* SUBTOTAL */
+ CTPO_CAT_TOTAL, /* TOTAL */
+
+ /* Nonterminals. */
+ CTPO_ADD,
+ CTPO_SUB,
+ CTPO_MUL,
+ CTPO_DIV,
+ CTPO_POW,
+ CTPO_NEG,
+ }
+ op;
+
+ union
+ {
+ /* CTPO_CAT_NUMBER. */
+ double number;
+
+ /* CTPO_CAT_STRING, in dictionary encoding. */
+ struct substring string;
+
+ /* CTPO_CAT_NRANGE. */
+ double nrange[2];
+
+ /* CTPO_CAT_SRANGE. */
+ struct substring srange[2];
+
+ /* CTPO_CAT_SUBTOTAL. */
+ size_t subtotal_index;
+
+ /* Two elements: CTPO_ADD, CTPO_SUB, CTPO_MUL, CTPO_DIV, CTPO_POW.
+ One element: CTPO_NEG. */
+ struct ctables_pcexpr *subs[2];
+ };
+
+ /* Source location. */
+ struct msg_location *location;
+ };
+
+static struct ctables_postcompute *ctables_find_postcompute (struct ctables *,
+ const char *name);
+
+static struct ctables_pcexpr *ctables_pcexpr_allocate_binary (
+ enum ctables_pcexpr_op, struct ctables_pcexpr *sub0,
+ struct ctables_pcexpr *sub1);
+
+typedef struct ctables_pcexpr *parse_recursively_func (struct lexer *,
+ struct dictionary *);
+
+static void
+ctables_pcexpr_destroy (struct ctables_pcexpr *e)
+{
+ if (e)
+ {
+ switch (e->op)
+ {
+ case CTPO_CAT_STRING:
+ ss_dealloc (&e->string);
+ break;
+
+ case CTPO_CAT_SRANGE:
+ for (size_t i = 0; i < 2; i++)
+ ss_dealloc (&e->srange[i]);
+ break;
+
+ case CTPO_ADD:
+ case CTPO_SUB:
+ case CTPO_MUL:
+ case CTPO_DIV:
+ case CTPO_POW:
+ case CTPO_NEG:
+ for (size_t i = 0; i < 2; i++)
+ ctables_pcexpr_destroy (e->subs[i]);
+ break;
+
+ case CTPO_CONSTANT:
+ case CTPO_CAT_NUMBER:
+ case CTPO_CAT_NRANGE:
+ case CTPO_CAT_MISSING:
+ case CTPO_CAT_OTHERNM:
+ case CTPO_CAT_SUBTOTAL:
+ case CTPO_CAT_TOTAL:
+ break;
+ }
+
+ msg_location_destroy (e->location);
+ free (e);
+ }
+}
+
+static struct ctables_pcexpr *
+ctables_pcexpr_allocate_binary (enum ctables_pcexpr_op op,
+ struct ctables_pcexpr *sub0,
+ struct ctables_pcexpr *sub1)
+{
+ struct ctables_pcexpr *e = xmalloc (sizeof *e);
+ *e = (struct ctables_pcexpr) {
+ .op = op,
+ .subs = { sub0, sub1 },
+ .location = msg_location_merged (sub0->location, sub1->location),
+ };
+ return e;
+}
+
+/* How to parse an operator. */
+struct operator
+ {
+ enum token_type token;
+ enum ctables_pcexpr_op op;
+ };
+
+static const struct operator *
+ctables_pcexpr_match_operator (struct lexer *lexer,
+ const struct operator ops[], size_t n_ops)
+{
+ for (const struct operator *op = ops; op < ops + n_ops; op++)
+ if (lex_token (lexer) == op->token)
+ {
+ if (op->token != T_NEG_NUM)
+ lex_get (lexer);
+
+ return op;
+ }
+
+ return NULL;
+}
+
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_binary_operators__ (
+ struct lexer *lexer, struct dictionary *dict,
+ const struct operator ops[], size_t n_ops,
+ parse_recursively_func *parse_next_level,
+ const char *chain_warning, struct ctables_pcexpr *lhs)
+{
+ for (int op_count = 0; ; op_count++)
+ {
+ const struct operator *op
+ = ctables_pcexpr_match_operator (lexer, ops, n_ops);
+ if (!op)
+ {
+ if (op_count > 1 && chain_warning)
+ msg_at (SW, lhs->location, "%s", chain_warning);
+
+ return lhs;
+ }
+
+ struct ctables_pcexpr *rhs = parse_next_level (lexer, dict);
+ if (!rhs)
+ {
+ ctables_pcexpr_destroy (lhs);
+ return NULL;
+ }
+
+ lhs = ctables_pcexpr_allocate_binary (op->op, lhs, rhs);
+ }
+}
+
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_binary_operators (
+ struct lexer *lexer, struct dictionary *dict,
+ const struct operator ops[], size_t n_ops,
+ parse_recursively_func *parse_next_level, const char *chain_warning)
+{
+ struct ctables_pcexpr *lhs = parse_next_level (lexer, dict);
+ if (!lhs)
+ return NULL;
+
+ return ctables_pcexpr_parse_binary_operators__ (lexer, dict, ops, n_ops,
+ parse_next_level,
+ chain_warning, lhs);
+}
+
+static struct ctables_pcexpr *ctables_pcexpr_parse_add (struct lexer *,
+ struct dictionary *);
+
+static struct ctables_pcexpr
+ctpo_cat_nrange (double low, double high)
+{
+ return (struct ctables_pcexpr) {
+ .op = CTPO_CAT_NRANGE,
+ .nrange = { low, high },
+ };
+}
+
+static struct ctables_pcexpr
+ctpo_cat_srange (struct substring low, struct substring high)
+{
+ return (struct ctables_pcexpr) {
+ .op = CTPO_CAT_SRANGE,
+ .srange = { low, high },
+ };
+}
+
+static struct substring
+parse_substring (struct lexer *lexer, struct dictionary *dict)
+{
+ struct substring s = recode_substring_pool (
+ dict_get_encoding (dict), "UTF-8", lex_tokss (lexer), NULL);
+ ss_rtrim (&s, ss_cstr (" "));
+ lex_get (lexer);
+ return s;
+}
+
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_primary (struct lexer *lexer, struct dictionary *dict)
+{
+ int start_ofs = lex_ofs (lexer);
+ struct ctables_pcexpr e;
+ if (lex_is_number (lexer))
+ {
+ e = (struct ctables_pcexpr) { .op = CTPO_CONSTANT,
+ .number = lex_number (lexer) };
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ e = (struct ctables_pcexpr) { .op = CTPO_CAT_MISSING };
+ else if (lex_match_id (lexer, "OTHERNM"))
+ e = (struct ctables_pcexpr) { .op = CTPO_CAT_OTHERNM };
+ else if (lex_match_id (lexer, "TOTAL"))
+ e = (struct ctables_pcexpr) { .op = CTPO_CAT_TOTAL };
+ else if (lex_match_id (lexer, "SUBTOTAL"))
+ {
+ size_t subtotal_index = 0;
+ if (lex_match (lexer, T_LBRACK))
+ {
+ if (!lex_force_int_range (lexer, "SUBTOTAL", 1, LONG_MAX))
+ return NULL;
+ subtotal_index = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RBRACK))
+ return NULL;
+ }
+ e = (struct ctables_pcexpr) { .op = CTPO_CAT_SUBTOTAL,
+ .subtotal_index = subtotal_index };
+ }
+ else if (lex_match (lexer, T_LBRACK))
+ {
+ if (lex_match_id (lexer, "LO"))
+ {
+ if (!lex_force_match_id (lexer, "THRU"))
+ return false;
+
+ if (lex_is_string (lexer))
+ {
+ struct substring low = { .string = NULL };
+ struct substring high = parse_substring (lexer, dict);
+ e = ctpo_cat_srange (low, high);
+ }
+ else
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ e = ctpo_cat_nrange (-DBL_MAX, lex_number (lexer));
+ lex_get (lexer);
+ }
+ }
+ else if (lex_is_number (lexer))
+ {
+ double number = lex_number (lexer);
+ lex_get (lexer);
+ if (lex_match_id (lexer, "THRU"))
+ {
+ if (lex_match_id (lexer, "HI"))
+ e = ctpo_cat_nrange (number, DBL_MAX);
+ else
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ e = ctpo_cat_nrange (number, lex_number (lexer));
+ lex_get (lexer);
+ }
+ }
+ else
+ e = (struct ctables_pcexpr) { .op = CTPO_CAT_NUMBER,
+ .number = number };
+ }
+ else if (lex_is_string (lexer))
+ {
+ struct substring s = parse_substring (lexer, dict);
+
+ if (lex_match_id (lexer, "THRU"))
+ {
+ struct substring high;
+
+ if (lex_match_id (lexer, "HI"))
+ high = (struct substring) { .string = NULL };
+ else
+ {
+ if (!lex_force_string (lexer))
+ {
+ ss_dealloc (&s);
+ return false;
+ }
+ high = parse_substring (lexer, dict);
+ }
+
+ e = ctpo_cat_srange (s, high);
+ }
+ else
+ e = (struct ctables_pcexpr) { .op = CTPO_CAT_STRING, .string = s };
+ }
+ else
+ {
+ lex_error (lexer,
+ _("Syntax error expecting number or string or range."));
+ return NULL;
+ }
+
+ if (!lex_force_match (lexer, T_RBRACK))
+ {
+ if (e.op == CTPO_CAT_STRING)
+ ss_dealloc (&e.string);
+ else if (e.op == CTPO_CAT_SRANGE)
+ {
+ ss_dealloc (&e.srange[0]);
+ ss_dealloc (&e.srange[1]);
+ }
+ return NULL;
+ }
+ }
+ else if (lex_match (lexer, T_LPAREN))
+ {
+ struct ctables_pcexpr *ep = ctables_pcexpr_parse_add (lexer, dict);
+ if (!ep)
+ return NULL;
+ if (!lex_force_match (lexer, T_RPAREN))
+ {
+ ctables_pcexpr_destroy (ep);
+ return NULL;
+ }
+ return ep;
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error in postcompute expression."));
+ return NULL;
+ }
+
+ e.location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1);
+ return xmemdup (&e, sizeof e);
+}
+
+static struct ctables_pcexpr *
+ctables_pcexpr_allocate_neg (struct ctables_pcexpr *sub,
+ struct lexer *lexer, int start_ofs)
+{
+ struct ctables_pcexpr *e = xmalloc (sizeof *e);
+ *e = (struct ctables_pcexpr) {
+ .op = CTPO_NEG,
+ .subs = { sub },
+ .location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1),
+ };
+ return e;
+}
+
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_exp (struct lexer *lexer, struct dictionary *dict)
+{
+ static const struct operator op = { T_EXP, CTPO_POW };
+
+ const char *chain_warning =
+ _("The exponentiation operator (`**') is left-associative: "
+ "`a**b**c' equals `(a**b)**c', not `a**(b**c)'. "
+ "To disable this warning, insert parentheses.");
+
+ if (lex_token (lexer) != T_NEG_NUM || lex_next_token (lexer, 1) != T_EXP)
+ return ctables_pcexpr_parse_binary_operators (lexer, dict, &op, 1,
+ ctables_pcexpr_parse_primary,
+ chain_warning);
+
+ /* Special case for situations like "-5**6", which must be parsed as
+ -(5**6). */
+
+ int start_ofs = lex_ofs (lexer);
+ struct ctables_pcexpr *lhs = xmalloc (sizeof *lhs);
+ *lhs = (struct ctables_pcexpr) {
+ .op = CTPO_CONSTANT,
+ .number = -lex_tokval (lexer),
+ .location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer)),
+ };
+ lex_get (lexer);
+
+ struct ctables_pcexpr *node = ctables_pcexpr_parse_binary_operators__ (
+ lexer, dict, &op, 1,
+ ctables_pcexpr_parse_primary, chain_warning, lhs);
+ if (!node)
+ return NULL;
+
+ return ctables_pcexpr_allocate_neg (node, lexer, start_ofs);
+}
+
+/* Parses the unary minus level. */
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_neg (struct lexer *lexer, struct dictionary *dict)
+{
+ int start_ofs = lex_ofs (lexer);
+ if (!lex_match (lexer, T_DASH))
+ return ctables_pcexpr_parse_exp (lexer, dict);
+
+ struct ctables_pcexpr *inner = ctables_pcexpr_parse_neg (lexer, dict);
+ if (!inner)
+ return NULL;
+
+ return ctables_pcexpr_allocate_neg (inner, lexer, start_ofs);
+}
+
+/* Parses the multiplication and division level. */
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_mul (struct lexer *lexer, struct dictionary *dict)
+{
+ static const struct operator ops[] =
+ {
+ { T_ASTERISK, CTPO_MUL },
+ { T_SLASH, CTPO_DIV },
+ };
+
+ return ctables_pcexpr_parse_binary_operators (lexer, dict, ops,
+ sizeof ops / sizeof *ops,
+ ctables_pcexpr_parse_neg, NULL);
+}
+
+/* Parses the addition and subtraction level. */
+static struct ctables_pcexpr *
+ctables_pcexpr_parse_add (struct lexer *lexer, struct dictionary *dict)
+{
+ static const struct operator ops[] =
+ {
+ { T_PLUS, CTPO_ADD },
+ { T_DASH, CTPO_SUB },
+ { T_NEG_NUM, CTPO_ADD },
+ };
+
+ return ctables_pcexpr_parse_binary_operators (lexer, dict,
+ ops, sizeof ops / sizeof *ops,
+ ctables_pcexpr_parse_mul, NULL);
+}
+\f
+/* CTABLES axis expressions. */
+
+/* CTABLES has a number of extra formats that we implement via custom
+ currency specifications on an alternate fmt_settings. */
+#define CTEF_NEGPAREN FMT_CCA
+#define CTEF_NEQUAL FMT_CCB
+#define CTEF_PAREN FMT_CCC
+#define CTEF_PCTPAREN FMT_CCD
+
+enum ctables_summary_variant
+ {
+ CSV_CELL,
+ CSV_TOTAL
+#define N_CSVS 2
+ };
+
+struct ctables_axis
+ {
+ enum ctables_axis_op
+ {
+ /* Terminals. */
+ CTAO_VAR,
+
+ /* Nonterminals. */
+ CTAO_STACK, /* + */
+ CTAO_NEST, /* > */
+ }
+ op;
+
+ union
+ {
+ /* Terminals. */
+ struct
+ {
+ struct variable *var;
+ bool scale;
+ struct ctables_summary_spec_set specs[N_CSVS];
+ };
+
+ /* Nonterminals. */
+ struct ctables_axis *subs[2];
+ };
+
+ struct msg_location *loc;
+ };
+
+static void
+ctables_axis_destroy (struct ctables_axis *axis)
+{
+ if (!axis)
+ return;
+
+ switch (axis->op)
+ {
+ case CTAO_VAR:
+ for (size_t i = 0; i < N_CSVS; i++)
+ ctables_summary_spec_set_uninit (&axis->specs[i]);
+ break;
+
+ case CTAO_STACK:
+ case CTAO_NEST:
+ ctables_axis_destroy (axis->subs[0]);
+ ctables_axis_destroy (axis->subs[1]);
+ break;
+ }
+ msg_location_destroy (axis->loc);
+ free (axis);
+}
+
+static struct ctables_axis *
+ctables_axis_new_nonterminal (enum ctables_axis_op op,
+ struct ctables_axis *sub0,
+ struct ctables_axis *sub1,
+ struct lexer *lexer, int start_ofs)
+{
+ struct ctables_axis *axis = xmalloc (sizeof *axis);
+ *axis = (struct ctables_axis) {
+ .op = op,
+ .subs = { sub0, sub1 },
+ .loc = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1),
+ };
+ return axis;
+}
+
+struct ctables_axis_parse_ctx
+ {
+ struct lexer *lexer;
+ struct dictionary *dict;
+ };
+
+static struct pivot_value *
+ctables_summary_label (const struct ctables_summary_spec *spec, double cilevel)
+{
+ if (!spec->label)
+ return ctables_summary_function_label (spec->function, spec->weighting,
+ spec->user_area, spec->percentile);
+ else
+ {
+ struct substring in = ss_cstr (spec->label);
+ struct substring target = ss_cstr (")CILEVEL");
+
+ struct string out = DS_EMPTY_INITIALIZER;
+ for (;;)
+ {
+ size_t chunk = ss_find_substring (in, target);
+ ds_put_substring (&out, ss_head (in, chunk));
+ ss_advance (&in, chunk);
+ if (!in.length)
+ return pivot_value_new_user_text_nocopy (ds_steal_cstr (&out));
+
+ ss_advance (&in, target.length);
+ ds_put_format (&out, "%g", cilevel);
+ }
+ }
+}
+
+static bool
+add_summary_spec (struct ctables_axis *axis,
+ enum ctables_summary_function function,
+ enum ctables_weighting weighting,
+ enum ctables_area_type area, double percentile,
+ const char *label, const struct fmt_spec *format,
+ bool is_ctables_format, const struct msg_location *loc,
+ enum ctables_summary_variant sv)
+{
+ if (axis->op == CTAO_VAR)
+ {
+ char function_name[128];
+ ctables_summary_function_name (function, weighting, area,
+ function_name, sizeof function_name);
+ const char *var_name = var_get_name (axis->var);
+ switch (ctables_function_availability (function))
+ {
+#if 0
+ case CTFA_MRSETS:
+ msg_at (SE, loc, _("Summary function %s applies only to multiple "
+ "response sets."), function_name);
+ msg_at (SN, axis->loc, _("'%s' is not a multiple response set."),
+ var_name);
+ return false;
+#endif
+
+ case CTFA_SCALE:
+ if (!axis->scale && sv != CSV_TOTAL)
+ {
+ msg_at (SE, loc,
+ _("Summary function %s applies only to scale variables."),
+ function_name);
+ msg_at (SN, axis->loc, _("'%s' is not a scale variable."),
+ var_name);
+ return false;
+ }
+ break;
+
+ case CTFA_ALL:
+ break;
+ }
+
+ struct ctables_summary_spec_set *set = &axis->specs[sv];
+ if (set->n >= set->allocated)
+ set->specs = x2nrealloc (set->specs, &set->allocated,
+ sizeof *set->specs);
+
+ struct ctables_summary_spec *dst = &set->specs[set->n++];
+ *dst = (struct ctables_summary_spec) {
+ .function = function,
+ .weighting = weighting,
+ .calc_area = area,
+ .user_area = area,
+ .percentile = percentile,
+ .label = xstrdup_if_nonnull (label),
+ .format = (format ? *format
+ : ctables_summary_default_format (function, axis->var)),
+ .is_ctables_format = is_ctables_format,
+ };
+ return true;
+ }
+ else
+ {
+ for (size_t i = 0; i < 2; i++)
+ if (!add_summary_spec (axis->subs[i], function, weighting, area,
+ percentile, label, format, is_ctables_format,
+ loc, sv))
+ return false;
+ return true;
+ }
+}
+
+static struct ctables_axis *ctables_axis_parse_stack (
+ struct ctables_axis_parse_ctx *);
+
+static struct ctables_axis *
+ctables_axis_parse_primary (struct ctables_axis_parse_ctx *ctx)
+{
+ if (lex_match (ctx->lexer, T_LPAREN))
+ {
+ struct ctables_axis *sub = ctables_axis_parse_stack (ctx);
+ if (!sub || !lex_force_match (ctx->lexer, T_RPAREN))
+ {
+ ctables_axis_destroy (sub);
+ return NULL;
+ }
+ return sub;
+ }
+
+ if (!lex_force_id (ctx->lexer))
+ return NULL;
+
+ if (lex_tokcstr (ctx->lexer)[0] == '$')
+ {
+ lex_error (ctx->lexer,
+ _("Multiple response set support not implemented."));
+ return NULL;
+ }
+
+ int start_ofs = lex_ofs (ctx->lexer);
+ struct variable *var = parse_variable (ctx->lexer, ctx->dict);
+ if (!var)
+ return NULL;
+
+ struct ctables_axis *axis = xmalloc (sizeof *axis);
+ *axis = (struct ctables_axis) { .op = CTAO_VAR, .var = var };
+
+ axis->scale = (lex_match_phrase (ctx->lexer, "[S]") ? true
+ : lex_match_phrase (ctx->lexer, "[C]") ? false
+ : var_get_measure (var) == MEASURE_SCALE);
+ axis->loc = lex_ofs_location (ctx->lexer, start_ofs,
+ lex_ofs (ctx->lexer) - 1);
+ if (axis->scale && var_is_alpha (var))
+ {
+ msg_at (SE, axis->loc, _("Cannot use string variable %s as a scale "
+ "variable."),
+ var_get_name (var));
+ ctables_axis_destroy (axis);
+ return NULL;
+ }
+
+ return axis;
+}
+
+static bool
+has_digit (const char *s)
+{
+ return s[strcspn (s, "0123456789")] != '\0';
+}
+
+static bool
+parse_ctables_format_specifier (struct lexer *lexer, struct fmt_spec *format,
+ bool *is_ctables_format)
+{
+ char type[FMT_TYPE_LEN_MAX + 1];
+ if (!parse_abstract_format_specifier__ (lexer, type, &format->w, &format->d))
+ return false;
+
+ if (!strcasecmp (type, "NEGPAREN"))
+ format->type = CTEF_NEGPAREN;
+ else if (!strcasecmp (type, "NEQUAL"))
+ format->type = CTEF_NEQUAL;
+ else if (!strcasecmp (type, "PAREN"))
+ format->type = CTEF_PAREN;
+ else if (!strcasecmp (type, "PCTPAREN"))
+ format->type = CTEF_PCTPAREN;
+ else
+ {
+ *is_ctables_format = false;
+ if (!parse_format_specifier (lexer, format))
+ return false;
+
+ char *error = fmt_check_output__ (format);
+ if (!error)
+ error = fmt_check_type_compat__ (format, NULL, VAL_NUMERIC);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ return false;
+ }
+
+ return true;
+ }
+
+ lex_get (lexer);
+ if (format->w < 2)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Output format %s requires width 2 or greater."), type);
+ return false;
+ }
+ else if (format->d > format->w - 1)
+ {
+ lex_next_error (lexer, -1, -1, _("Output format %s requires width "
+ "greater than decimals."), type);
+ return false;
+ }
+ else
+ {
+ *is_ctables_format = true;
+ return true;
+ }
+}
+
+static struct ctables_axis *
+ctables_axis_parse_postfix (struct ctables_axis_parse_ctx *ctx)
+{
+ struct ctables_axis *sub = ctables_axis_parse_primary (ctx);
+ if (!sub || !lex_match (ctx->lexer, T_LBRACK))
+ return sub;
+
+ enum ctables_summary_variant sv = CSV_CELL;
+ for (;;)
+ {
+ int start_ofs = lex_ofs (ctx->lexer);
+
+ /* Parse function. */
+ enum ctables_summary_function function;
+ enum ctables_weighting weighting;
+ enum ctables_area_type area;
+ if (!parse_ctables_summary_function (ctx->lexer, &function, &weighting,
+ &area))
+ goto error;
+
+ /* Parse percentile. */
+ double percentile = 0;
+ if (function == CTSF_PTILE)
+ {
+ if (!lex_force_num_range_closed (ctx->lexer, "PTILE", 0, 100))
+ goto error;
+ percentile = lex_number (ctx->lexer);
+ lex_get (ctx->lexer);
+ }
+
+ /* Parse label. */
+ char *label = NULL;
+ if (lex_is_string (ctx->lexer))
+ {
+ label = ss_xstrdup (lex_tokss (ctx->lexer));
+ lex_get (ctx->lexer);
+ }
+
+ /* Parse format. */
+ struct fmt_spec format;
+ const struct fmt_spec *formatp;
+ bool is_ctables_format = false;
+ if (lex_token (ctx->lexer) == T_ID
+ && has_digit (lex_tokcstr (ctx->lexer)))
+ {
+ if (!parse_ctables_format_specifier (ctx->lexer, &format,
+ &is_ctables_format))
+ {
+ free (label);
+ goto error;
+ }
+ formatp = &format;
+ }
+ else
+ formatp = NULL;
+
+ struct msg_location *loc = lex_ofs_location (ctx->lexer, start_ofs,
+ lex_ofs (ctx->lexer) - 1);
+ add_summary_spec (sub, function, weighting, area, percentile, label,
+ formatp, is_ctables_format, loc, sv);
+ free (label);
+ msg_location_destroy (loc);
+
+ lex_match (ctx->lexer, T_COMMA);
+ if (sv == CSV_CELL && lex_match_id (ctx->lexer, "TOTALS"))
+ {
+ if (!lex_force_match (ctx->lexer, T_LBRACK))
+ goto error;
+ sv = CSV_TOTAL;
+ }
+ else if (lex_match (ctx->lexer, T_RBRACK))
+ {
+ if (sv == CSV_TOTAL && !lex_force_match (ctx->lexer, T_RBRACK))
+ goto error;
+ return sub;
+ }
+ }
+
+error:
+ ctables_axis_destroy (sub);
+ return NULL;
+}
+
+static const struct ctables_axis *
+find_scale (const struct ctables_axis *axis)
+{
+ if (!axis)
+ return NULL;
+ else if (axis->op == CTAO_VAR)
+ return axis->scale ? axis : NULL;
+ else
+ {
+ for (size_t i = 0; i < 2; i++)
+ {
+ const struct ctables_axis *scale = find_scale (axis->subs[i]);
+ if (scale)
+ return scale;
+ }
+ return NULL;
+ }
+}
+
+static const struct ctables_axis *
+find_categorical_summary_spec (const struct ctables_axis *axis)
+{
+ if (!axis)
+ return NULL;
+ else if (axis->op == CTAO_VAR)
+ return !axis->scale && axis->specs[CSV_CELL].n ? axis : NULL;
+ else
+ {
+ for (size_t i = 0; i < 2; i++)
+ {
+ const struct ctables_axis *sum
+ = find_categorical_summary_spec (axis->subs[i]);
+ if (sum)
+ return sum;
+ }
+ return NULL;
+ }
+}
+
+static struct ctables_axis *
+ctables_axis_parse_nest (struct ctables_axis_parse_ctx *ctx)
+{
+ int start_ofs = lex_ofs (ctx->lexer);
+ struct ctables_axis *lhs = ctables_axis_parse_postfix (ctx);
+ if (!lhs)
+ return NULL;
+
+ while (lex_match (ctx->lexer, T_GT))
+ {
+ struct ctables_axis *rhs = ctables_axis_parse_postfix (ctx);
+ if (!rhs)
+ {
+ ctables_axis_destroy (lhs);
+ return NULL;
+ }
+
+ struct ctables_axis *nest = ctables_axis_new_nonterminal (
+ CTAO_NEST, lhs, rhs, ctx->lexer, start_ofs);
+
+ const struct ctables_axis *outer_scale = find_scale (lhs);
+ const struct ctables_axis *inner_scale = find_scale (rhs);
+ if (outer_scale && inner_scale)
+ {
+ msg_at (SE, nest->loc, _("Cannot nest scale variables."));
+ msg_at (SN, outer_scale->loc, _("This is an outer scale variable."));
+ msg_at (SN, inner_scale->loc, _("This is an inner scale variable."));
+ ctables_axis_destroy (nest);
+ return NULL;
+ }
+
+ const struct ctables_axis *outer_sum = find_categorical_summary_spec (lhs);
+ if (outer_sum)
+ {
+ msg_at (SE, nest->loc,
+ _("Summaries may only be requested for categorical variables "
+ "at the innermost nesting level."));
+ msg_at (SN, outer_sum->loc,
+ _("This outer categorical variable has a summary."));
+ ctables_axis_destroy (nest);
+ return NULL;
+ }
+
+ lhs = nest;
+ }
+
+ return lhs;
+}
+
+static struct ctables_axis *
+ctables_axis_parse_stack (struct ctables_axis_parse_ctx *ctx)
+{
+ int start_ofs = lex_ofs (ctx->lexer);
+ struct ctables_axis *lhs = ctables_axis_parse_nest (ctx);
+ if (!lhs)
+ return NULL;
+
+ while (lex_match (ctx->lexer, T_PLUS))
+ {
+ struct ctables_axis *rhs = ctables_axis_parse_nest (ctx);
+ if (!rhs)
+ {
+ ctables_axis_destroy (lhs);
+ return NULL;
+ }
+
+ lhs = ctables_axis_new_nonterminal (CTAO_STACK, lhs, rhs,
+ ctx->lexer, start_ofs);
+ }
+
+ return lhs;
+}
+
+static bool
+ctables_axis_parse (struct lexer *lexer, struct dictionary *dict,
+ struct ctables_axis **axisp)
+{
+ *axisp = NULL;
+ if (lex_token (lexer) == T_BY
+ || lex_token (lexer) == T_SLASH
+ || lex_token (lexer) == T_ENDCMD)
+ return true;
+
+ struct ctables_axis_parse_ctx ctx = {
+ .lexer = lexer,
+ .dict = dict,
+ };
+ *axisp = ctables_axis_parse_stack (&ctx);
+ return *axisp;
+}
+\f
+/* CTABLES categories. */
+
+struct ctables_categories
+ {
+ size_t n_refs;
+ struct ctables_category *cats;
+ size_t n_cats;
+ };
+
+struct ctables_category
+ {
+ enum ctables_category_type
+ {
+ /* Explicit category lists. */
+ CCT_NUMBER,
+ CCT_STRING,
+ CCT_NRANGE, /* Numerical range. */
+ CCT_SRANGE, /* String range. */
+ CCT_MISSING,
+ CCT_OTHERNM,
+ CCT_POSTCOMPUTE,
+
+ /* Totals and subtotals. */
+ CCT_SUBTOTAL,
+ CCT_TOTAL,
+
+ /* Implicit category lists. */
+ CCT_VALUE,
+ CCT_LABEL,
+ CCT_FUNCTION,
+
+ /* For contributing to TOTALN. */
+ CCT_EXCLUDED_MISSING,
+ }
+ type;
+
+ struct ctables_category *subtotal;
+
+ bool hide;
+
+ union
+ {
+ double number; /* CCT_NUMBER. */
+ struct substring string; /* CCT_STRING, in dictionary encoding. */
+ double nrange[2]; /* CCT_NRANGE. */
+ struct substring srange[2]; /* CCT_SRANGE. */
+
+ struct
+ {
+ char *total_label; /* CCT_SUBTOTAL, CCT_TOTAL. */
+ bool hide_subcategories; /* CCT_SUBTOTAL. */
+ };
+
+ /* CCT_POSTCOMPUTE. */
+ struct
+ {
+ const struct ctables_postcompute *pc;
+ enum fmt_type parse_format;
+ };
+
+ /* CCT_VALUE, CCT_LABEL, CCT_FUNCTION. */
+ struct
+ {
+ bool include_missing;
+ bool sort_ascending;
+
+ /* CCT_FUNCTION. */
+ enum ctables_summary_function sort_function;
+ enum ctables_weighting weighting;
+ enum ctables_area_type area;
+ struct variable *sort_var;
+ double percentile;
+ };
+ };
+
+ /* Source location (sometimes NULL). */
+ struct msg_location *location;
+ };
+
+static void
+ctables_category_uninit (struct ctables_category *cat)
+{
+ if (!cat)
+ return;
+
+ msg_location_destroy (cat->location);
+ switch (cat->type)
+ {
+ case CCT_NUMBER:
+ case CCT_NRANGE:
+ case CCT_MISSING:
+ case CCT_OTHERNM:
+ case CCT_POSTCOMPUTE:
+ break;
+
+ case CCT_STRING:
+ ss_dealloc (&cat->string);
+ break;
+
+ case CCT_SRANGE:
+ ss_dealloc (&cat->srange[0]);
+ ss_dealloc (&cat->srange[1]);
+ break;
+
+ case CCT_SUBTOTAL:
+ case CCT_TOTAL:
+ free (cat->total_label);
+ break;
+
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ break;
+
+ case CCT_EXCLUDED_MISSING:
+ break;
+ }
+}
+
+static bool
+nullable_substring_equal (const struct substring *a,
+ const struct substring *b)
+{
+ return !a->string ? !b->string : b->string && ss_equals (*a, *b);
+}
+
+static bool
+ctables_category_equal (const struct ctables_category *a,
+ const struct ctables_category *b)
+{
+ if (a->type != b->type)
+ return false;
+
+ switch (a->type)
+ {
+ case CCT_NUMBER:
+ return a->number == b->number;
+
+ case CCT_STRING:
+ return ss_equals (a->string, b->string);
+
+ case CCT_NRANGE:
+ return a->nrange[0] == b->nrange[0] && a->nrange[1] == b->nrange[1];
+
+ case CCT_SRANGE:
+ return (nullable_substring_equal (&a->srange[0], &b->srange[0])
+ && nullable_substring_equal (&a->srange[1], &b->srange[1]));
+
+ case CCT_MISSING:
+ case CCT_OTHERNM:
+ return true;
+
+ case CCT_POSTCOMPUTE:
+ return a->pc == b->pc;
+
+ case CCT_SUBTOTAL:
+ case CCT_TOTAL:
+ return !strcmp (a->total_label, b->total_label);
+
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ return (a->include_missing == b->include_missing
+ && a->sort_ascending == b->sort_ascending
+ && a->sort_function == b->sort_function
+ && a->sort_var == b->sort_var
+ && a->percentile == b->percentile);
+
+ case CCT_EXCLUDED_MISSING:
+ return true;
+ }
+
+ NOT_REACHED ();
+}
+
+static void
+ctables_categories_unref (struct ctables_categories *c)
+{
+ if (!c)
+ return;
+
+ assert (c->n_refs > 0);
+ if (--c->n_refs)
+ return;
+
+ for (size_t i = 0; i < c->n_cats; i++)
+ ctables_category_uninit (&c->cats[i]);
+ free (c->cats);
+ free (c);
+}
+
+static bool
+ctables_categories_equal (const struct ctables_categories *a,
+ const struct ctables_categories *b)
+{
+ if (a->n_cats != b->n_cats)
+ return false;
+
+ for (size_t i = 0; i < a->n_cats; i++)
+ if (!ctables_category_equal (&a->cats[i], &b->cats[i]))
+ return false;
+
+ return true;
+}
+
+static struct ctables_category
+cct_nrange (double low, double high)
+{
+ return (struct ctables_category) {
+ .type = CCT_NRANGE,
+ .nrange = { low, high }
+ };
+}
+
+static struct ctables_category
+cct_srange (struct substring low, struct substring high)
+{
+ return (struct ctables_category) {
+ .type = CCT_SRANGE,
+ .srange = { low, high }
+ };
+}
+
+static bool
+ctables_table_parse_subtotal (struct lexer *lexer, bool hide_subcategories,
+ struct ctables_category *cat)
+{
+ char *total_label;
+ if (lex_match (lexer, T_EQUALS))
+ {
+ if (!lex_force_string (lexer))
+ return false;
+
+ total_label = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else
+ total_label = xstrdup (_("Subtotal"));
+
+ *cat = (struct ctables_category) {
+ .type = CCT_SUBTOTAL,
+ .hide_subcategories = hide_subcategories,
+ .total_label = total_label
+ };
+ return true;
+}
+
+static bool
+ctables_table_parse_explicit_category (struct lexer *lexer,
+ struct dictionary *dict,
+ struct ctables *ct,
+ struct ctables_category *cat)
+{
+ if (lex_match_id (lexer, "OTHERNM"))
+ *cat = (struct ctables_category) { .type = CCT_OTHERNM };
+ else if (lex_match_id (lexer, "MISSING"))
+ *cat = (struct ctables_category) { .type = CCT_MISSING };
+ else if (lex_match_id (lexer, "SUBTOTAL"))
+ return ctables_table_parse_subtotal (lexer, false, cat);
+ else if (lex_match_id (lexer, "HSUBTOTAL"))
+ return ctables_table_parse_subtotal (lexer, true, cat);
+ else if (lex_match_id (lexer, "LO"))
+ {
+ if (!lex_force_match_id (lexer, "THRU"))
+ return false;
+ if (lex_is_string (lexer))
+ {
+ struct substring sr0 = { .string = NULL };
+ struct substring sr1 = parse_substring (lexer, dict);
+ *cat = cct_srange (sr0, sr1);
+ }
+ else if (lex_force_num (lexer))
+ {
+ *cat = cct_nrange (-DBL_MAX, lex_number (lexer));
+ lex_get (lexer);
+ }
+ else
+ return false;
+ }
+ else if (lex_is_number (lexer))
+ {
+ double number = lex_number (lexer);
+ lex_get (lexer);
+ if (lex_match_id (lexer, "THRU"))
+ {
+ if (lex_match_id (lexer, "HI"))
+ *cat = cct_nrange (number, DBL_MAX);
+ else
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ *cat = cct_nrange (number, lex_number (lexer));
+ lex_get (lexer);
+ }
+ }
+ else
+ *cat = (struct ctables_category) {
+ .type = CCT_NUMBER,
+ .number = number
+ };
+ }
+ else if (lex_is_string (lexer))
+ {
+ struct substring s = parse_substring (lexer, dict);
+ if (lex_match_id (lexer, "THRU"))
+ {
+ if (lex_match_id (lexer, "HI"))
+ {
+ struct substring sr1 = { .string = NULL };
+ *cat = cct_srange (s, sr1);
+ }
+ else
+ {
+ if (!lex_force_string (lexer))
+ {
+ ss_dealloc (&s);
+ return false;
+ }
+ struct substring sr1 = parse_substring (lexer, dict);
+ *cat = cct_srange (s, sr1);
+ }
+ }
+ else
+ *cat = (struct ctables_category) { .type = CCT_STRING, .string = s };
+ }
+ else if (lex_match (lexer, T_AND))
+ {
+ if (!lex_force_id (lexer))
+ return false;
+ struct ctables_postcompute *pc = ctables_find_postcompute (
+ ct, lex_tokcstr (lexer));
+ if (!pc)
+ {
+ struct msg_location *loc = lex_get_location (lexer, -1, 0);
+ msg_at (SE, loc, _("Unknown postcompute &%s."),
+ lex_tokcstr (lexer));
+ msg_location_destroy (loc);
+ return false;
+ }
+ lex_get (lexer);
+
+ *cat = (struct ctables_category) { .type = CCT_POSTCOMPUTE, .pc = pc };
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting category specification."));
+ return false;
+ }
+
+ return true;
+}
+
+static bool
+parse_category_string (struct msg_location *location,
+ struct substring s, const struct dictionary *dict,
+ enum fmt_type format, double *n)
+{
+ union value v;
+ char *error = data_in (s, dict_get_encoding (dict), format,
+ settings_get_fmt_settings (), &v, 0, NULL);
+ if (error)
+ {
+ msg_at (SE, location,
+ _("Failed to parse category specification as format %s: %s."),
+ fmt_name (format), error);
+ free (error);
+ return false;
+ }
+
+ *n = v.f;
+ return true;
+}
+
+static struct ctables_category *
+ctables_find_category_for_postcompute__ (const struct ctables_categories *cats,
+ const struct ctables_pcexpr *e)
+{
+ struct ctables_category *best = NULL;
+ size_t n_subtotals = 0;
+ for (size_t i = 0; i < cats->n_cats; i++)
+ {
+ struct ctables_category *cat = &cats->cats[i];
+ switch (e->op)
+ {
+ case CTPO_CAT_NUMBER:
+ if (cat->type == CCT_NUMBER && cat->number == e->number)
+ best = cat;
+ break;
+
+ case CTPO_CAT_STRING:
+ if (cat->type == CCT_STRING && ss_equals (cat->string, e->string))
+ best = cat;
+ break;
+
+ case CTPO_CAT_NRANGE:
+ if (cat->type == CCT_NRANGE
+ && cat->nrange[0] == e->nrange[0]
+ && cat->nrange[1] == e->nrange[1])
+ best = cat;
+ break;
+
+ case CTPO_CAT_SRANGE:
+ if (cat->type == CCT_SRANGE
+ && nullable_substring_equal (&cat->srange[0], &e->srange[0])
+ && nullable_substring_equal (&cat->srange[1], &e->srange[1]))
+ best = cat;
+ break;
+
+ case CTPO_CAT_MISSING:
+ if (cat->type == CCT_MISSING)
+ best = cat;
+ break;
+
+ case CTPO_CAT_OTHERNM:
+ if (cat->type == CCT_OTHERNM)
+ best = cat;
+ break;
+
+ case CTPO_CAT_SUBTOTAL:
+ if (cat->type == CCT_SUBTOTAL)
+ {
+ n_subtotals++;
+ if (e->subtotal_index == n_subtotals)
+ return cat;
+ else if (e->subtotal_index == 0)
+ best = cat;
+ }
+ break;
+
+ case CTPO_CAT_TOTAL:
+ if (cat->type == CCT_TOTAL)
+ return cat;
+ break;
+
+ case CTPO_CONSTANT:
+ case CTPO_ADD:
+ case CTPO_SUB:
+ case CTPO_MUL:
+ case CTPO_DIV:
+ case CTPO_POW:
+ case CTPO_NEG:
+ NOT_REACHED ();
+ }
+ }
+ if (e->op == CTPO_CAT_SUBTOTAL && e->subtotal_index == 0 && n_subtotals > 1)
+ return NULL;
+ return best;
+}
+
+static struct ctables_category *
+ctables_find_category_for_postcompute (const struct dictionary *dict,
+ const struct ctables_categories *cats,
+ enum fmt_type parse_format,
+ const struct ctables_pcexpr *e)
+{
+ if (parse_format != FMT_F)
+ {
+ if (e->op == CTPO_CAT_STRING)
+ {
+ double number;
+ if (!parse_category_string (e->location, e->string, dict,
+ parse_format, &number))
+ return NULL;
+
+ struct ctables_pcexpr e2 = {
+ .op = CTPO_CAT_NUMBER,
+ .number = number,
+ .location = e->location,
+ };
+ return ctables_find_category_for_postcompute__ (cats, &e2);
+ }
+ else if (e->op == CTPO_CAT_SRANGE)
+ {
+ double nrange[2];
+ if (!e->srange[0].string)
+ nrange[0] = -DBL_MAX;
+ else if (!parse_category_string (e->location, e->srange[0], dict,
+ parse_format, &nrange[0]))
+ return NULL;
+
+ if (!e->srange[1].string)
+ nrange[1] = DBL_MAX;
+ else if (!parse_category_string (e->location, e->srange[1], dict,
+ parse_format, &nrange[1]))
+ return NULL;
+
+ struct ctables_pcexpr e2 = {
+ .op = CTPO_CAT_NRANGE,
+ .nrange = { nrange[0], nrange[1] },
+ .location = e->location,
+ };
+ return ctables_find_category_for_postcompute__ (cats, &e2);
+ }
+ }
+ return ctables_find_category_for_postcompute__ (cats, e);
+}
+
+static struct substring
+rtrim_value (const union value *v, const struct variable *var)
+{
+ struct substring s = ss_buffer (CHAR_CAST (char *, v->s),
+ var_get_width (var));
+ ss_rtrim (&s, ss_cstr (" "));
+ return s;
+}
+
+static bool
+in_string_range (const union value *v, const struct variable *var,
+ const struct substring *srange)
+{
+ struct substring s = rtrim_value (v, var);
+ return ((!srange[0].string || ss_compare (s, srange[0]) >= 0)
+ && (!srange[1].string || ss_compare (s, srange[1]) <= 0));
+}
+
+static const struct ctables_category *
+ctables_categories_match (const struct ctables_categories *c,
+ const union value *v, const struct variable *var)
+{
+ if (var_is_numeric (var) && v->f == SYSMIS)
+ return NULL;
+
+ const struct ctables_category *othernm = NULL;
+ for (size_t i = c->n_cats; i-- > 0; )
+ {
+ const struct ctables_category *cat = &c->cats[i];
+ switch (cat->type)
+ {
+ case CCT_NUMBER:
+ if (cat->number == v->f)
+ return cat;
+ break;
+
+ case CCT_STRING:
+ if (ss_equals (cat->string, rtrim_value (v, var)))
+ return cat;
+ break;
+
+ case CCT_NRANGE:
+ if ((cat->nrange[0] == -DBL_MAX || v->f >= cat->nrange[0])
+ && (cat->nrange[1] == DBL_MAX || v->f <= cat->nrange[1]))
+ return cat;
+ break;
+
+ case CCT_SRANGE:
+ if (in_string_range (v, var, cat->srange))
+ return cat;
+ break;
+
+ case CCT_MISSING:
+ if (var_is_value_missing (var, v))
+ return cat;
+ break;
+
+ case CCT_POSTCOMPUTE:
+ break;
+
+ case CCT_OTHERNM:
+ if (!othernm)
+ othernm = cat;
+ break;
+
+ case CCT_SUBTOTAL:
+ case CCT_TOTAL:
+ break;
+
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ return (cat->include_missing || !var_is_value_missing (var, v) ? cat
+ : NULL);
+
+ case CCT_EXCLUDED_MISSING:
+ break;
+ }
+ }
+
+ return var_is_value_missing (var, v) ? NULL : othernm;
+}
+
+static const struct ctables_category *
+ctables_categories_total (const struct ctables_categories *c)
+{
+ const struct ctables_category *first = &c->cats[0];
+ const struct ctables_category *last = &c->cats[c->n_cats - 1];
+ return (first->type == CCT_TOTAL ? first
+ : last->type == CCT_TOTAL ? last
+ : NULL);
+}
+
+static void
+ctables_category_format_number (double number, const struct variable *var,
+ struct string *s)
+{
+ struct pivot_value *pv = pivot_value_new_var_value (
+ var, &(union value) { .f = number });
+ pivot_value_format (pv, NULL, s);
+ pivot_value_destroy (pv);
+}
+
+static void
+ctables_category_format_string (struct substring string,
+ const struct variable *var, struct string *out)
+{
+ int width = var_get_width (var);
+ char *s = xmalloc (width);
+ buf_copy_rpad (s, width, string.string, string.length, ' ');
+ struct pivot_value *pv = pivot_value_new_var_value (
+ var, &(union value) { .s = CHAR_CAST (uint8_t *, s) });
+ pivot_value_format (pv, NULL, out);
+ pivot_value_destroy (pv);
+ free (s);
+}
+
+static bool
+ctables_category_format_label (const struct ctables_category *cat,
+ const struct variable *var,
+ struct string *s)
+{
+ switch (cat->type)
+ {
+ case CCT_NUMBER:
+ ctables_category_format_number (cat->number, var, s);
+ return true;
+
+ case CCT_STRING:
+ ctables_category_format_string (cat->string, var, s);
+ return true;
+
+ case CCT_NRANGE:
+ ctables_category_format_number (cat->nrange[0], var, s);
+ ds_put_format (s, " THRU ");
+ ctables_category_format_number (cat->nrange[1], var, s);
+ return true;
+
+ case CCT_SRANGE:
+ ctables_category_format_string (cat->srange[0], var, s);
+ ds_put_format (s, " THRU ");
+ ctables_category_format_string (cat->srange[1], var, s);
+ return true;
+
+ case CCT_MISSING:
+ ds_put_cstr (s, "MISSING");
+ return true;
+
+ case CCT_OTHERNM:
+ ds_put_cstr (s, "OTHERNM");
+ return true;
+
+ case CCT_POSTCOMPUTE:
+ ds_put_format (s, "&%s", cat->pc->name);
+ return true;
+
+ case CCT_TOTAL:
+ case CCT_SUBTOTAL:
+ ds_put_cstr (s, cat->total_label);
+ return true;
+
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ case CCT_EXCLUDED_MISSING:
+ return false;
+ }
+
+ return false;
+}
+
+static bool
+ctables_recursive_check_postcompute (struct dictionary *dict,
+ const struct ctables_pcexpr *e,
+ struct ctables_category *pc_cat,
+ const struct ctables_categories *cats,
+ const struct msg_location *cats_location)
+{
+ switch (e->op)
+ {
+ case CTPO_CAT_NUMBER:
+ case CTPO_CAT_STRING:
+ case CTPO_CAT_NRANGE:
+ case CTPO_CAT_SRANGE:
+ case CTPO_CAT_MISSING:
+ case CTPO_CAT_OTHERNM:
+ case CTPO_CAT_SUBTOTAL:
+ case CTPO_CAT_TOTAL:
+ {
+ struct ctables_category *cat = ctables_find_category_for_postcompute (
+ dict, cats, pc_cat->parse_format, e);
+ if (!cat)
+ {
+ if (e->op == CTPO_CAT_SUBTOTAL && e->subtotal_index == 0)
+ {
+ size_t n_subtotals = 0;
+ for (size_t i = 0; i < cats->n_cats; i++)
+ n_subtotals += cats->cats[i].type == CCT_SUBTOTAL;
+ if (n_subtotals > 1)
+ {
+ msg_at (SE, cats_location,
+ ngettext ("These categories include %zu instance "
+ "of SUBTOTAL or HSUBTOTAL, so references "
+ "from computed categories must refer to "
+ "subtotals by position, "
+ "e.g. SUBTOTAL[1].",
+ "These categories include %zu instances "
+ "of SUBTOTAL or HSUBTOTAL, so references "
+ "from computed categories must refer to "
+ "subtotals by position, "
+ "e.g. SUBTOTAL[1].",
+ n_subtotals),
+ n_subtotals);
+ msg_at (SN, e->location,
+ _("This is the reference that lacks a position."));
+ return NULL;
+ }
+ }
+
+ msg_at (SE, pc_cat->location,
+ _("Computed category &%s references a category not included "
+ "in the category list."),
+ pc_cat->pc->name);
+ msg_at (SN, e->location, _("This is the missing category."));
+ if (e->op == CTPO_CAT_SUBTOTAL)
+ msg_at (SN, cats_location,
+ _("To fix the problem, add subtotals to the "
+ "list of categories here."));
+ else if (e->op == CTPO_CAT_TOTAL)
+ msg (SN, _("To fix the problem, add TOTAL=YES to the variable's "
+ "CATEGORIES specification."));
+ else
+ msg_at (SN, cats_location,
+ _("To fix the problem, add the missing category to the "
+ "list of categories here."));
+ return false;
+ }
+ if (pc_cat->pc->hide_source_cats)
+ cat->hide = true;
+ return true;
+ }
+
+ case CTPO_CONSTANT:
+ return true;
+
+ case CTPO_ADD:
+ case CTPO_SUB:
+ case CTPO_MUL:
+ case CTPO_DIV:
+ case CTPO_POW:
+ case CTPO_NEG:
+ for (size_t i = 0; i < 2; i++)
+ if (e->subs[i] && !ctables_recursive_check_postcompute (
+ dict, e->subs[i], pc_cat, cats, cats_location))
+ return false;
+ return true;
+ }
+
+ NOT_REACHED ();
+}
+
+static struct pivot_value *
+ctables_postcompute_label (const struct ctables_categories *cats,
+ const struct ctables_category *cat,
+ const struct variable *var)
+{
+ struct substring in = ss_cstr (cat->pc->label);
+ struct substring target = ss_cstr (")LABEL[");
+
+ struct string out = DS_EMPTY_INITIALIZER;
+ for (;;)
+ {
+ size_t chunk = ss_find_substring (in, target);
+ if (chunk == SIZE_MAX)
+ {
+ if (ds_is_empty (&out))
+ return pivot_value_new_user_text (in.string, in.length);
+ else
+ {
+ ds_put_substring (&out, in);
+ return pivot_value_new_user_text_nocopy (ds_steal_cstr (&out));
+ }
+ }
+
+ ds_put_substring (&out, ss_head (in, chunk));
+ ss_advance (&in, chunk + target.length);
+
+ struct substring idx_s;
+ if (!ss_get_until (&in, ']', &idx_s))
+ goto error;
+ char *tail;
+ long int idx = strtol (idx_s.string, &tail, 10);
+ if (idx < 1 || idx > cats->n_cats || tail != ss_end (idx_s))
+ goto error;
+
+ struct ctables_category *cat2 = &cats->cats[idx - 1];
+ if (!ctables_category_format_label (cat2, var, &out))
+ goto error;
+ }
+
+error:
+ ds_destroy (&out);
+ return pivot_value_new_user_text (cat->pc->label, SIZE_MAX);
+}
+
+static struct pivot_value *
+ctables_category_create_value_label (const struct ctables_categories *cats,
+ const struct ctables_category *cat,
+ const struct variable *var,
+ const union value *value)
+{
+ return (cat->type == CCT_POSTCOMPUTE && cat->pc->label
+ ? ctables_postcompute_label (cats, cat, var)
+ : cat->type == CCT_TOTAL || cat->type == CCT_SUBTOTAL
+ ? pivot_value_new_user_text (cat->total_label, SIZE_MAX)
+ : pivot_value_new_var_value (var, value));
+}
+\f
+/* CTABLES variable nesting and stacking. */
+
+/* A nested sequence of variables, e.g. a > b > c. */
+struct ctables_nest
+ {
+ struct variable **vars;
+ size_t n;
+ size_t scale_idx;
+ size_t summary_idx;
+ size_t *areas[N_CTATS];
+ size_t n_areas[N_CTATS];
+ size_t group_head;
+
+ struct ctables_summary_spec_set specs[N_CSVS];
+ };
+
+/* A stack of nestings, e.g. nest1 + nest2 + ... + nestN. */
+struct ctables_stack
+ {
+ struct ctables_nest *nests;
+ size_t n;
+ };
+
+static void
+ctables_nest_uninit (struct ctables_nest *nest)
+{
+ free (nest->vars);
+ for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
+ ctables_summary_spec_set_uninit (&nest->specs[sv]);
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ free (nest->areas[at]);
+}
+
+static void
+ctables_stack_uninit (struct ctables_stack *stack)
+{
+ if (stack)
+ {
+ for (size_t i = 0; i < stack->n; i++)
+ ctables_nest_uninit (&stack->nests[i]);
+ free (stack->nests);
+ }
+}
+
+static struct ctables_stack
+nest_fts (struct ctables_stack s0, struct ctables_stack s1)
+{
+ if (!s0.n)
+ return s1;
+ else if (!s1.n)
+ return s0;
+
+ struct ctables_stack stack = { .nests = xnmalloc (s0.n, s1.n * sizeof *stack.nests) };
+ for (size_t i = 0; i < s0.n; i++)
+ for (size_t j = 0; j < s1.n; j++)
+ {
+ const struct ctables_nest *a = &s0.nests[i];
+ const struct ctables_nest *b = &s1.nests[j];
+
+ size_t allocate = a->n + b->n;
+ struct variable **vars = xnmalloc (allocate, sizeof *vars);
+ size_t n = 0;
+ for (size_t k = 0; k < a->n; k++)
+ vars[n++] = a->vars[k];
+ for (size_t k = 0; k < b->n; k++)
+ vars[n++] = b->vars[k];
+ assert (n == allocate);
+
+ const struct ctables_nest *summary_src;
+ if (!a->specs[CSV_CELL].var)
+ summary_src = b;
+ else if (!b->specs[CSV_CELL].var)
+ summary_src = a;
+ else
+ NOT_REACHED ();
+
+ struct ctables_nest *new = &stack.nests[stack.n++];
+ *new = (struct ctables_nest) {
+ .vars = vars,
+ .scale_idx = (a->scale_idx != SIZE_MAX ? a->scale_idx
+ : b->scale_idx != SIZE_MAX ? a->n + b->scale_idx
+ : SIZE_MAX),
+ .summary_idx = (a->summary_idx != SIZE_MAX ? a->summary_idx
+ : b->summary_idx != SIZE_MAX ? a->n + b->summary_idx
+ : SIZE_MAX),
+ .n = n,
+ };
+ for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
+ ctables_summary_spec_set_clone (&new->specs[sv], &summary_src->specs[sv]);
+ }
+ ctables_stack_uninit (&s0);
+ ctables_stack_uninit (&s1);
+ return stack;
+}
+
+static struct ctables_stack
+stack_fts (struct ctables_stack s0, struct ctables_stack s1)
+{
+ struct ctables_stack stack = { .nests = xnmalloc (s0.n + s1.n, sizeof *stack.nests) };
+ for (size_t i = 0; i < s0.n; i++)
+ stack.nests[stack.n++] = s0.nests[i];
+ for (size_t i = 0; i < s1.n; i++)
+ {
+ stack.nests[stack.n] = s1.nests[i];
+ stack.nests[stack.n].group_head += s0.n;
+ stack.n++;
+ }
+ assert (stack.n == s0.n + s1.n);
+ free (s0.nests);
+ free (s1.nests);
+ return stack;
+}
+
+static struct ctables_stack
+var_fts (const struct ctables_axis *a)
+{
+ struct variable **vars = xmalloc (sizeof *vars);
+ *vars = a->var;
+
+ bool is_summary = a->specs[CSV_CELL].n || a->scale;
+ struct ctables_nest *nest = xmalloc (sizeof *nest);
+ *nest = (struct ctables_nest) {
+ .vars = vars,
+ .n = 1,
+ .scale_idx = a->scale ? 0 : SIZE_MAX,
+ .summary_idx = is_summary ? 0 : SIZE_MAX,
+ };
+ if (is_summary)
+ for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
+ {
+ ctables_summary_spec_set_clone (&nest->specs[sv], &a->specs[sv]);
+ nest->specs[sv].var = a->var;
+ nest->specs[sv].is_scale = a->scale;
+ }
+ return (struct ctables_stack) { .nests = nest, .n = 1 };
+}
+
+static struct ctables_stack
+enumerate_fts (enum pivot_axis_type axis_type, const struct ctables_axis *a)
+{
+ if (!a)
+ return (struct ctables_stack) { .n = 0 };
+
+ switch (a->op)
+ {
+ case CTAO_VAR:
+ return var_fts (a);
+
+ case CTAO_STACK:
+ return stack_fts (enumerate_fts (axis_type, a->subs[0]),
+ enumerate_fts (axis_type, a->subs[1]));
+
+ case CTAO_NEST:
+ /* This should consider any of the scale variables found in the result to
+ be linked to each other listwise for SMISSING=LISTWISE. */
+ return nest_fts (enumerate_fts (axis_type, a->subs[0]),
+ enumerate_fts (axis_type, a->subs[1]));
+ }
+
+ NOT_REACHED ();
+}
+\f
+/* CTABLES summary calculation. */
+
+union ctables_summary
+ {
+ /* COUNT, VALIDN, TOTALN. */
+ double count;
+
+ /* MINIMUM, MAXIMUM, RANGE. */
+ struct
+ {
+ double min;
+ double max;
+ };
+
+ /* MEAN, SEMEAN, STDDEV, SUM, VARIANCE, *.SUM. */
+ struct moments1 *moments;
+
+ /* MEDIAN, MODE, PTILE. */
+ struct
+ {
+ struct casewriter *writer;
+ double ovalid;
+ double ovalue;
+ };
+ };
+
+static void
+ctables_summary_init (union ctables_summary *s,
+ const struct ctables_summary_spec *ss)
+{
+ switch (ss->function)
+ {
+ case CTSF_COUNT:
+ case CTSF_areaPCT_COUNT:
+ case CTSF_areaPCT_VALIDN:
+ case CTSF_areaPCT_TOTALN:
+ case CTSF_MISSING:
+ case CTSF_TOTALN:
+ case CTSF_VALIDN:
+ s->count = 0;
+ break;
+
+ case CTSF_areaID:
+ break;
+
+ case CTSF_MAXIMUM:
+ case CTSF_MINIMUM:
+ case CTSF_RANGE:
+ s->min = s->max = SYSMIS;
+ break;
+
+ case CTSF_MEAN:
+ case CTSF_SUM:
+ case CTSF_areaPCT_SUM:
+ s->moments = moments1_create (MOMENT_MEAN);
+ break;
+
+ case CTSF_SEMEAN:
+ case CTSF_STDDEV:
+ case CTSF_VARIANCE:
+ s->moments = moments1_create (MOMENT_VARIANCE);
+ break;
+
+ case CTSF_MEDIAN:
+ case CTSF_MODE:
+ case CTSF_PTILE:
+ {
+ struct caseproto *proto = caseproto_create ();
+ proto = caseproto_add_width (proto, 0);
+ proto = caseproto_add_width (proto, 0);
+
+ struct subcase ordering;
+ subcase_init (&ordering, 0, 0, SC_ASCEND);
+ s->writer = sort_create_writer (&ordering, proto);
+ subcase_uninit (&ordering);
+ caseproto_unref (proto);
+
+ s->ovalid = 0;
+ s->ovalue = SYSMIS;
+ }
+ break;
+ }
+}
+
+static void
+ctables_summary_uninit (union ctables_summary *s,
+ const struct ctables_summary_spec *ss)
+{
+ switch (ss->function)
+ {
+ case CTSF_COUNT:
+ case CTSF_areaPCT_COUNT:
+ case CTSF_areaPCT_VALIDN:
+ case CTSF_areaPCT_TOTALN:
+ case CTSF_MISSING:
+ case CTSF_TOTALN:
+ case CTSF_VALIDN:
+ break;
+
+ case CTSF_areaID:
+ break;
+
+ case CTSF_MAXIMUM:
+ case CTSF_MINIMUM:
+ case CTSF_RANGE:
+ break;
+
+ case CTSF_MEAN:
+ case CTSF_SEMEAN:
+ case CTSF_STDDEV:
+ case CTSF_SUM:
+ case CTSF_VARIANCE:
+ case CTSF_areaPCT_SUM:
+ moments1_destroy (s->moments);
+ break;
+
+ case CTSF_MEDIAN:
+ case CTSF_MODE:
+ case CTSF_PTILE:
+ casewriter_destroy (s->writer);
+ break;
+ }
+}
+
+static void
+ctables_summary_add (union ctables_summary *s,
+ const struct ctables_summary_spec *ss,
+ const union value *value,
+ bool is_missing, bool is_included,
+ double weight)
+{
+ /* To determine whether a case is included in a given table for a particular
+ kind of summary, consider the following charts for the variable being
+ summarized. Only if "yes" appears is the case counted.
+
+ Categorical variables: VALIDN other TOTALN
+ Valid values in included categories yes yes yes
+ Missing values in included categories --- yes yes
+ Missing values in excluded categories --- --- yes
+ Valid values in excluded categories --- --- ---
+
+ Scale variables: VALIDN other TOTALN
+ Valid value yes yes yes
+ Missing value --- yes yes
+
+ Missing values include both user- and system-missing. (The system-missing
+ value is always in an excluded category.)
+
+ One way to interpret the above table is that scale variables are like
+ categorical variables in which all values are in included categories.
+ */
+ switch (ss->function)
+ {
+ case CTSF_TOTALN:
+ case CTSF_areaPCT_TOTALN:
+ s->count += weight;
+ break;
+
+ case CTSF_COUNT:
+ case CTSF_areaPCT_COUNT:
+ if (is_included)
+ s->count += weight;
+ break;
+
+ case CTSF_VALIDN:
+ case CTSF_areaPCT_VALIDN:
+ if (!is_missing)
+ s->count += weight;
+ break;
+
+ case CTSF_areaID:
+ break;
+
+ case CTSF_MISSING:
+ if (is_missing)
+ s->count += weight;
+ break;
+
+ case CTSF_MAXIMUM:
+ case CTSF_MINIMUM:
+ case CTSF_RANGE:
+ if (!is_missing)
+ {
+ if (s->min == SYSMIS || value->f < s->min)
+ s->min = value->f;
+ if (s->max == SYSMIS || value->f > s->max)
+ s->max = value->f;
+ }
+ break;
+
+ case CTSF_MEAN:
+ case CTSF_SEMEAN:
+ case CTSF_STDDEV:
+ case CTSF_SUM:
+ case CTSF_VARIANCE:
+ if (!is_missing)
+ moments1_add (s->moments, value->f, weight);
+ break;
+
+ case CTSF_areaPCT_SUM:
+ if (!is_missing)
+ moments1_add (s->moments, value->f, weight);
+ break;
+
+ case CTSF_MEDIAN:
+ case CTSF_MODE:
+ case CTSF_PTILE:
+ if (!is_missing)
+ {
+ s->ovalid += weight;
+
+ struct ccase *c = case_create (casewriter_get_proto (s->writer));
+ *case_num_rw_idx (c, 0) = value->f;
+ *case_num_rw_idx (c, 1) = weight;
+ casewriter_write (s->writer, c);
+ }
+ break;
+ }
+}
+
+static double
+ctables_summary_value (struct ctables_area *areas[N_CTATS],
+ union ctables_summary *s,
+ const struct ctables_summary_spec *ss)
+{
+ switch (ss->function)
+ {
+ case CTSF_COUNT:
+ return s->count;
+
+ case CTSF_areaID:
+ return areas[ss->calc_area]->sequence;
+
+ case CTSF_areaPCT_COUNT:
+ {
+ const struct ctables_area *a = areas[ss->calc_area];
+ double a_count = a->count[ss->weighting];
+ return a_count ? s->count / a_count * 100 : SYSMIS;
+ }
+
+ case CTSF_areaPCT_VALIDN:
+ {
+ const struct ctables_area *a = areas[ss->calc_area];
+ double a_valid = a->valid[ss->weighting];
+ return a_valid ? s->count / a_valid * 100 : SYSMIS;
+ }
+
+ case CTSF_areaPCT_TOTALN:
+ {
+ const struct ctables_area *a = areas[ss->calc_area];
+ double a_total = a->total[ss->weighting];
+ return a_total ? s->count / a_total * 100 : SYSMIS;
+ }
+
+ case CTSF_MISSING:
+ case CTSF_TOTALN:
+ case CTSF_VALIDN:
+ return s->count;
+
+ case CTSF_MAXIMUM:
+ return s->max;
+
+ case CTSF_MINIMUM:
+ return s->min;
+
+ case CTSF_RANGE:
+ return s->max != SYSMIS && s->min != SYSMIS ? s->max - s->min : SYSMIS;
+
+ case CTSF_MEAN:
+ {
+ double mean;
+ moments1_calculate (s->moments, NULL, &mean, NULL, NULL, NULL);
+ return mean;
+ }
+
+ case CTSF_SEMEAN:
+ {
+ double weight, variance;
+ moments1_calculate (s->moments, &weight, NULL, &variance, NULL, NULL);
+ return calc_semean (variance, weight);
+ }
+
+ case CTSF_STDDEV:
+ {
+ double variance;
+ moments1_calculate (s->moments, NULL, NULL, &variance, NULL, NULL);
+ return variance != SYSMIS ? sqrt (variance) : SYSMIS;
+ }
+
+ case CTSF_SUM:
+ {
+ double weight, mean;
+ moments1_calculate (s->moments, &weight, &mean, NULL, NULL, NULL);
+ return weight != SYSMIS && mean != SYSMIS ? weight * mean : SYSMIS;
+ }
+
+ case CTSF_VARIANCE:
+ {
+ double variance;
+ moments1_calculate (s->moments, NULL, NULL, &variance, NULL, NULL);
+ return variance;
+ }
+
+ case CTSF_areaPCT_SUM:
+ {
+ double weight, mean;
+ moments1_calculate (s->moments, &weight, &mean, NULL, NULL, NULL);
+ if (weight == SYSMIS || mean == SYSMIS)
+ return SYSMIS;
+
+ const struct ctables_area *a = areas[ss->calc_area];
+ const struct ctables_sum *sum = &a->sums[ss->sum_var_idx];
+ double denom = sum->sum[ss->weighting];
+ return denom != 0 ? weight * mean / denom * 100 : SYSMIS;
+ }
+
+ case CTSF_MEDIAN:
+ case CTSF_PTILE:
+ if (s->writer)
+ {
+ struct casereader *reader = casewriter_make_reader (s->writer);
+ s->writer = NULL;
+
+ struct percentile *ptile = percentile_create (
+ ss->function == CTSF_PTILE ? ss->percentile : 0.5, s->ovalid);
+ struct order_stats *os = &ptile->parent;
+ order_stats_accumulate_idx (&os, 1, reader, 1, 0);
+ s->ovalue = percentile_calculate (ptile, PC_HAVERAGE);
+ statistic_destroy (&ptile->parent.parent);
+ }
+ return s->ovalue;
+
+ case CTSF_MODE:
+ if (s->writer)
+ {
+ struct casereader *reader = casewriter_make_reader (s->writer);
+ s->writer = NULL;
+
+ struct mode *mode = mode_create ();
+ struct order_stats *os = &mode->parent;
+ order_stats_accumulate_idx (&os, 1, reader, 1, 0);
+ s->ovalue = mode->mode;
+ statistic_destroy (&mode->parent.parent);
+ }
+ return s->ovalue;
+ }
+
+ NOT_REACHED ();
+}
+\f
+/* CTABLES occurrences. */
+
+struct ctables_occurrence
+ {
+ struct hmap_node node;
+ union value value;
+ };
+
+static void
+ctables_add_occurrence (const struct variable *var,
+ const union value *value,
+ struct hmap *occurrences)
+{
+ int width = var_get_width (var);
+ unsigned int hash = value_hash (value, width, 0);
+
+ struct ctables_occurrence *o;
+ HMAP_FOR_EACH_WITH_HASH (o, struct ctables_occurrence, node, hash,
+ occurrences)
+ if (value_equal (value, &o->value, width))
+ return;
+
+ o = xmalloc (sizeof *o);
+ value_clone (&o->value, value, width);
+ hmap_insert (occurrences, &o->node, hash);
+}
+\f
+enum ctables_vlabel
+ {
+ CTVL_NONE = SETTINGS_VALUE_SHOW_DEFAULT,
+ CTVL_NAME = SETTINGS_VALUE_SHOW_VALUE,
+ CTVL_LABEL = SETTINGS_VALUE_SHOW_LABEL,
+ CTVL_BOTH = SETTINGS_VALUE_SHOW_BOTH,
+ };
+
+struct ctables_cell
+ {
+ /* In struct ctables_section's 'cells' hmap. Indexed by all the values in
+ all the axes (except the scalar variable, if any). */
+ struct hmap_node node;
+ struct ctables_section *section;
+
+ /* The areas that contain this cell. */
+ uint32_t omit_areas;
+ struct ctables_area *areas[N_CTATS];
+
+ bool hide;
+
+ bool postcompute;
+ enum ctables_summary_variant sv;
+
+ struct ctables_cell_axis
+ {
+ struct ctables_cell_value
+ {
+ const struct ctables_category *category;
+ union value value;
+ }
+ *cvs;
+ int leaf;
+ }
+ axes[PIVOT_N_AXES];
+
+ union ctables_summary *summaries;
+ };
+
+struct ctables_section
+ {
+ /* Settings. */
+ struct ctables_table *table;
+ struct ctables_nest *nests[PIVOT_N_AXES];
+
+ /* Data. */
+ struct hmap *occurrences[PIVOT_N_AXES]; /* "struct ctables_occurrence"s. */
+ struct hmap cells; /* Contains "struct ctables_cell"s. */
+ struct hmap areas[N_CTATS]; /* Contains "struct ctables_area"s. */
+ };
+
+static void ctables_section_uninit (struct ctables_section *);
+
+struct ctables_table
+ {
+ struct ctables *ctables;
+ struct ctables_axis *axes[PIVOT_N_AXES];
+ struct ctables_stack stacks[PIVOT_N_AXES];
+ struct ctables_section *sections;
+ size_t n_sections;
+ enum pivot_axis_type summary_axis;
+ struct ctables_summary_spec_set summary_specs;
+ struct variable **sum_vars;
+ size_t n_sum_vars;
+
+ enum pivot_axis_type slabels_axis;
+ bool slabels_visible;
+
+ /* The innermost category labels for axis 'a' appear on axis label_axis[a].
+
+ Most commonly, label_axis[a] == a, and in particular we always have
+ label_axis{PIVOT_AXIS_LAYER] == PIVOT_AXIS_LAYER.
+
+ If ROWLABELS or COLLABELS is specified, then one of
+ label_axis[PIVOT_AXIS_ROW] or label_axis[PIVOT_AXIS_COLUMN] can be the
+ opposite axis or PIVOT_AXIS_LAYER. Only one of them will differ.
+
+ If any category labels are moved, then 'clabels_example' is one of the
+ variables being moved (and it is otherwise NULL). All of the variables
+ being moved have the same width, value labels, and categories, so this
+ example variable can be used to find those out.
+
+ The remaining members in this group are relevant only if category labels
+ are moved.
+
+ 'clabels_values_map' holds a "struct ctables_value" for all the values
+ that appear in all of the variables in the moved categories. It is
+ accumulated as the data is read. Once the data is fully read, its
+ sorted values are put into 'clabels_values' and 'n_clabels_values'.
+ */
+ enum pivot_axis_type label_axis[PIVOT_N_AXES];
+ enum pivot_axis_type clabels_from_axis;
+ enum pivot_axis_type clabels_to_axis;
+ int clabels_start_ofs, clabels_end_ofs;
+ const struct variable *clabels_example;
+ struct hmap clabels_values_map;
+ struct ctables_value **clabels_values;
+ size_t n_clabels_values;
+
+ /* Indexed by variable dictionary index. */
+ struct ctables_categories **categories;
+ size_t n_categories;
+ bool *show_empty;
+
+ double cilevel;
+
+ char *caption;
+ char *corner;
+ char *title;
+
+ struct ctables_chisq *chisq;
+ struct ctables_pairwise *pairwise;
+ };
+
+struct ctables_cell_sort_aux
+ {
+ const struct ctables_nest *nest;
+ enum pivot_axis_type a;
+ };
+
+static int
+ctables_cell_compare_3way (const void *a_, const void *b_, const void *aux_)
+{
+ const struct ctables_cell_sort_aux *aux = aux_;
+ struct ctables_cell *const *ap = a_;
+ struct ctables_cell *const *bp = b_;
+ const struct ctables_cell *a = *ap;
+ const struct ctables_cell *b = *bp;
+
+ const struct ctables_nest *nest = aux->nest;
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ const struct variable *var = nest->vars[i];
+ const struct ctables_cell_value *a_cv = &a->axes[aux->a].cvs[i];
+ const struct ctables_cell_value *b_cv = &b->axes[aux->a].cvs[i];
+ if (a_cv->category != b_cv->category)
+ return a_cv->category > b_cv->category ? 1 : -1;
+
+ const union value *a_val = &a_cv->value;
+ const union value *b_val = &b_cv->value;
+ switch (a_cv->category->type)
+ {
+ case CCT_NUMBER:
+ case CCT_STRING:
+ case CCT_SUBTOTAL:
+ case CCT_TOTAL:
+ case CCT_POSTCOMPUTE:
+ case CCT_EXCLUDED_MISSING:
+ /* Must be equal. */
+ continue;
+
+ case CCT_NRANGE:
+ case CCT_SRANGE:
+ case CCT_MISSING:
+ case CCT_OTHERNM:
+ {
+ int cmp = value_compare_3way (a_val, b_val, var_get_width (var));
+ if (cmp)
+ return cmp;
+ }
+ break;
+
+ case CCT_VALUE:
+ {
+ int cmp = value_compare_3way (a_val, b_val, var_get_width (var));
+ if (cmp)
+ return a_cv->category->sort_ascending ? cmp : -cmp;
+ }
+ break;
+
+ case CCT_LABEL:
+ {
+ const char *a_label = var_lookup_value_label (var, a_val);
+ const char *b_label = var_lookup_value_label (var, b_val);
+ int cmp;
+ if (a_label)
+ {
+ if (!b_label)
+ return -1;
+ cmp = strcmp (a_label, b_label);
+ }
+ else
+ {
+ if (b_label)
+ return 1;
+ cmp = value_compare_3way (a_val, b_val, var_get_width (var));
+ }
+ if (cmp)
+ return a_cv->category->sort_ascending ? cmp : -cmp;
+ }
+ break;
+
+ case CCT_FUNCTION:
+ NOT_REACHED ();
+ }
+ }
+ return 0;
+}
+
+static struct ctables_area *
+ctables_area_insert (struct ctables_cell *cell, enum ctables_area_type area)
+{
+ struct ctables_section *s = cell->section;
+ size_t hash = 0;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n_areas[area]; i++)
+ {
+ size_t v_idx = nest->areas[area][i];
+ struct ctables_cell_value *cv = &cell->axes[a].cvs[v_idx];
+ hash = hash_pointer (cv->category, hash);
+ if (cv->category->type != CCT_TOTAL
+ && cv->category->type != CCT_SUBTOTAL
+ && cv->category->type != CCT_POSTCOMPUTE)
+ hash = value_hash (&cv->value,
+ var_get_width (nest->vars[v_idx]), hash);
+ }
+ }
+
+ struct ctables_area *a;
+ HMAP_FOR_EACH_WITH_HASH (a, struct ctables_area, node, hash, &s->areas[area])
+ {
+ const struct ctables_cell *df = a->example;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n_areas[area]; i++)
+ {
+ size_t v_idx = nest->areas[area][i];
+ struct ctables_cell_value *cv1 = &df->axes[a].cvs[v_idx];
+ struct ctables_cell_value *cv2 = &cell->axes[a].cvs[v_idx];
+ if (cv1->category != cv2->category
+ || (cv1->category->type != CCT_TOTAL
+ && cv1->category->type != CCT_SUBTOTAL
+ && cv1->category->type != CCT_POSTCOMPUTE
+ && !value_equal (&cv1->value, &cv2->value,
+ var_get_width (nest->vars[v_idx]))))
+ goto not_equal;
+ }
+ }
+ return a;
+
+ not_equal: ;
+ }
+
+ struct ctables_sum *sums = (s->table->n_sum_vars
+ ? xzalloc (s->table->n_sum_vars * sizeof *sums)
+ : NULL);
+
+ a = xmalloc (sizeof *a);
+ *a = (struct ctables_area) { .example = cell, .sums = sums };
+ hmap_insert (&s->areas[area], &a->node, hash);
+ return a;
+}
+
+static struct ctables_cell *
+ctables_cell_insert__ (struct ctables_section *s, const struct ccase *c,
+ const struct ctables_category **cats[PIVOT_N_AXES])
+{
+ size_t hash = 0;
+ enum ctables_summary_variant sv = CSV_CELL;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ hash = hash_pointer (cats[a][i], hash);
+ if (cats[a][i]->type != CCT_TOTAL
+ && cats[a][i]->type != CCT_SUBTOTAL
+ && cats[a][i]->type != CCT_POSTCOMPUTE)
+ hash = value_hash (case_data (c, nest->vars[i]),
+ var_get_width (nest->vars[i]), hash);
+ else
+ sv = CSV_TOTAL;
+ }
+ }
+
+ struct ctables_cell *cell;
+ HMAP_FOR_EACH_WITH_HASH (cell, struct ctables_cell, node, hash, &s->cells)
+ {
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx
+ && (cats[a][i] != cell->axes[a].cvs[i].category
+ || (cats[a][i]->type != CCT_TOTAL
+ && cats[a][i]->type != CCT_SUBTOTAL
+ && cats[a][i]->type != CCT_POSTCOMPUTE
+ && !value_equal (case_data (c, nest->vars[i]),
+ &cell->axes[a].cvs[i].value,
+ var_get_width (nest->vars[i])))))
+ goto not_equal;
+ }
+
+ return cell;
+
+ not_equal: ;
+ }
+
+ cell = xmalloc (sizeof *cell);
+ cell->section = s;
+ cell->hide = false;
+ cell->sv = sv;
+ cell->omit_areas = 0;
+ cell->postcompute = false;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ cell->axes[a].cvs = (nest->n
+ ? xnmalloc (nest->n, sizeof *cell->axes[a].cvs)
+ : NULL);
+ for (size_t i = 0; i < nest->n; i++)
+ {
+ const struct ctables_category *cat = cats[a][i];
+ const struct variable *var = nest->vars[i];
+ const union value *value = case_data (c, var);
+ if (i != nest->scale_idx)
+ {
+ const struct ctables_category *subtotal = cat->subtotal;
+ if (cat->hide || (subtotal && subtotal->hide_subcategories))
+ cell->hide = true;
+
+ if (cat->type == CCT_TOTAL
+ || cat->type == CCT_SUBTOTAL
+ || cat->type == CCT_POSTCOMPUTE)
+ {
+ switch (a)
+ {
+ case PIVOT_AXIS_COLUMN:
+ cell->omit_areas |= ((1u << CTAT_TABLE) |
+ (1u << CTAT_LAYER) |
+ (1u << CTAT_LAYERCOL) |
+ (1u << CTAT_SUBTABLE) |
+ (1u << CTAT_COL));
+ break;
+ case PIVOT_AXIS_ROW:
+ cell->omit_areas |= ((1u << CTAT_TABLE) |
+ (1u << CTAT_LAYER) |
+ (1u << CTAT_LAYERROW) |
+ (1u << CTAT_SUBTABLE) |
+ (1u << CTAT_ROW));
+ break;
+ case PIVOT_AXIS_LAYER:
+ cell->omit_areas |= ((1u << CTAT_TABLE) |
+ (1u << CTAT_LAYER));
+ break;
+ }
+ }
+ if (cat->type == CCT_POSTCOMPUTE)
+ cell->postcompute = true;
+ }
+
+ cell->axes[a].cvs[i].category = cat;
+ value_clone (&cell->axes[a].cvs[i].value, value, var_get_width (var));
+ }
+ }
+
+ const struct ctables_nest *ss = s->nests[s->table->summary_axis];
+ const struct ctables_summary_spec_set *specs = &ss->specs[cell->sv];
+ cell->summaries = xmalloc (specs->n * sizeof *cell->summaries);
+ for (size_t i = 0; i < specs->n; i++)
+ ctables_summary_init (&cell->summaries[i], &specs->specs[i]);
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ cell->areas[at] = ctables_area_insert (cell, at);
+ hmap_insert (&s->cells, &cell->node, hash);
+ return cell;
+}
+
+static void
+add_weight (double dst[N_CTWS], const double src[N_CTWS])
+{
+ for (enum ctables_weighting wt = 0; wt < N_CTWS; wt++)
+ dst[wt] += src[wt];
+}
+
+static void
+ctables_cell_add__ (struct ctables_section *s, const struct ccase *c,
+ const struct ctables_category **cats[PIVOT_N_AXES],
+ bool is_included, double weight[N_CTWS])
+{
+ struct ctables_cell *cell = ctables_cell_insert__ (s, c, cats);
+ const struct ctables_nest *ss = s->nests[s->table->summary_axis];
+
+ const struct ctables_summary_spec_set *specs = &ss->specs[cell->sv];
+ const union value *value = case_data (c, specs->var);
+ bool is_missing = var_is_value_missing (specs->var, value);
+ bool is_scale_missing
+ = is_missing || (specs->is_scale && is_listwise_missing (specs, c));
+
+ for (size_t i = 0; i < specs->n; i++)
+ ctables_summary_add (&cell->summaries[i], &specs->specs[i], value,
+ is_scale_missing, is_included,
+ weight[specs->specs[i].weighting]);
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ if (!(cell->omit_areas && (1u << at)))
+ {
+ struct ctables_area *a = cell->areas[at];
+
+ add_weight (a->total, weight);
+ if (is_included)
+ add_weight (a->count, weight);
+ if (!is_missing)
+ {
+ add_weight (a->valid, weight);
+
+ if (!is_scale_missing)
+ for (size_t i = 0; i < s->table->n_sum_vars; i++)
+ {
+ const struct variable *var = s->table->sum_vars[i];
+ double addend = case_num (c, var);
+ if (!var_is_num_missing (var, addend))
+ for (enum ctables_weighting wt = 0; wt < N_CTWS; wt++)
+ a->sums[i].sum[wt] += addend * weight[wt];
+ }
+ }
+ }
+}
+
+static void
+recurse_totals (struct ctables_section *s, const struct ccase *c,
+ const struct ctables_category **cats[PIVOT_N_AXES],
+ bool is_included, double weight[N_CTWS],
+ enum pivot_axis_type start_axis, size_t start_nest)
+{
+ for (enum pivot_axis_type a = start_axis; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = start_nest; i < nest->n; i++)
+ {
+ if (i == nest->scale_idx)
+ continue;
+
+ const struct variable *var = nest->vars[i];
+
+ const struct ctables_category *total = ctables_categories_total (
+ s->table->categories[var_get_dict_index (var)]);
+ if (total)
+ {
+ const struct ctables_category *save = cats[a][i];
+ cats[a][i] = total;
+ ctables_cell_add__ (s, c, cats, is_included, weight);
+ recurse_totals (s, c, cats, is_included, weight, a, i + 1);
+ cats[a][i] = save;
+ }
+ }
+ start_nest = 0;
+ }
+}
+
+static void
+recurse_subtotals (struct ctables_section *s, const struct ccase *c,
+ const struct ctables_category **cats[PIVOT_N_AXES],
+ bool is_included, double weight[N_CTWS],
+ enum pivot_axis_type start_axis, size_t start_nest)
+{
+ for (enum pivot_axis_type a = start_axis; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = start_nest; i < nest->n; i++)
+ {
+ if (i == nest->scale_idx)
+ continue;
+
+ const struct ctables_category *save = cats[a][i];
+ if (save->subtotal)
+ {
+ cats[a][i] = save->subtotal;
+ ctables_cell_add__ (s, c, cats, is_included, weight);
+ recurse_subtotals (s, c, cats, is_included, weight, a, i + 1);
+ cats[a][i] = save;
+ }
+ }
+ start_nest = 0;
+ }
+}
+
+static void
+ctables_cell_insert (struct ctables_section *s, const struct ccase *c,
+ double weight[N_CTWS])
+{
+ const struct ctables_category *layer_cats[s->nests[PIVOT_AXIS_LAYER]->n];
+ const struct ctables_category *row_cats[s->nests[PIVOT_AXIS_ROW]->n];
+ const struct ctables_category *column_cats[s->nests[PIVOT_AXIS_COLUMN]->n];
+ const struct ctables_category **cats[PIVOT_N_AXES] =
+ {
+ [PIVOT_AXIS_LAYER] = layer_cats,
+ [PIVOT_AXIS_ROW] = row_cats,
+ [PIVOT_AXIS_COLUMN] = column_cats,
+ };
+
+ bool is_included = true;
+
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ const struct variable *var = nest->vars[i];
+ const union value *value = case_data (c, var);
+
+ cats[a][i] = ctables_categories_match (
+ s->table->categories[var_get_dict_index (var)], value, var);
+ if (!cats[a][i])
+ {
+ if (i != nest->summary_idx)
+ return;
+
+ if (!var_is_value_missing (var, value))
+ return;
+
+ static const struct ctables_category cct_excluded_missing = {
+ .type = CCT_EXCLUDED_MISSING,
+ .hide = true,
+ };
+ cats[a][i] = &cct_excluded_missing;
+ is_included = false;
+ }
+ }
+ }
+
+ if (is_included)
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ const struct variable *var = nest->vars[i];
+ const union value *value = case_data (c, var);
+ ctables_add_occurrence (var, value, &s->occurrences[a][i]);
+ }
+ }
+
+ ctables_cell_add__ (s, c, cats, is_included, weight);
+ recurse_totals (s, c, cats, is_included, weight, 0, 0);
+ recurse_subtotals (s, c, cats, is_included, weight, 0, 0);
+}
+\f
+struct ctables_value
+ {
+ struct hmap_node node;
+ union value value;
+ int leaf;
+ };
+
+static struct ctables_value *
+ctables_value_find__ (const struct ctables_table *t, const union value *value,
+ int width, unsigned int hash)
+{
+ struct ctables_value *clv;
+ HMAP_FOR_EACH_WITH_HASH (clv, struct ctables_value, node,
+ hash, &t->clabels_values_map)
+ if (value_equal (value, &clv->value, width))
+ return clv;
+ return NULL;
+}
+
+static void
+ctables_value_insert (struct ctables_table *t, const union value *value,
+ int width)
+{
+ unsigned int hash = value_hash (value, width, 0);
+ struct ctables_value *clv = ctables_value_find__ (t, value, width, hash);
+ if (!clv)
+ {
+ clv = xmalloc (sizeof *clv);
+ value_clone (&clv->value, value, width);
+ hmap_insert (&t->clabels_values_map, &clv->node, hash);
+ }
+}
+
+static const struct ctables_value *
+ctables_value_find (const struct ctables_cell *cell)
+{
+ const struct ctables_section *s = cell->section;
+ const struct ctables_table *t = s->table;
+ if (!t->clabels_example)
+ return NULL;
+
+ const struct ctables_nest *clabels_nest = s->nests[t->clabels_from_axis];
+ const struct variable *var = clabels_nest->vars[clabels_nest->n - 1];
+ const union value *value
+ = &cell->axes[t->clabels_from_axis].cvs[clabels_nest->n - 1].value;
+ int width = var_get_width (var);
+ const struct ctables_value *ctv = ctables_value_find__ (
+ t, value, width, value_hash (value, width, 0));
+ assert (ctv != NULL);
+ return ctv;
+}
+
+static int
+compare_ctables_values_3way (const void *a_, const void *b_, const void *width_)
+{
+ const struct ctables_value *const *ap = a_;
+ const struct ctables_value *const *bp = b_;
+ const struct ctables_value *a = *ap;
+ const struct ctables_value *b = *bp;
+ const int *width = width_;
+ return value_compare_3way (&a->value, &b->value, *width);
+}
+
+static void
+ctables_sort_clabels_values (struct ctables_table *t)
+{
+ const struct variable *v0 = t->clabels_example;
+ int width = var_get_width (v0);
+
+ size_t i0 = var_get_dict_index (v0);
+ struct ctables_categories *c0 = t->categories[i0];
+ if (t->show_empty[i0])
+ {
+ const struct val_labs *val_labs = var_get_value_labels (v0);
+ for (const struct val_lab *vl = val_labs_first (val_labs); vl;
+ vl = val_labs_next (val_labs, vl))
+ if (ctables_categories_match (c0, &vl->value, v0))
+ ctables_value_insert (t, &vl->value, width);
+ }
+
+ size_t n = hmap_count (&t->clabels_values_map);
+ t->clabels_values = xnmalloc (n, sizeof *t->clabels_values);
+
+ struct ctables_value *clv;
+ size_t i = 0;
+ HMAP_FOR_EACH (clv, struct ctables_value, node, &t->clabels_values_map)
+ t->clabels_values[i++] = clv;
+ t->n_clabels_values = n;
+ assert (i == n);
+
+ sort (t->clabels_values, n, sizeof *t->clabels_values,
+ compare_ctables_values_3way, &width);
+
+ for (size_t i = 0; i < n; i++)
+ t->clabels_values[i]->leaf = i;
+}
+\f
+struct ctables
+ {
+ const struct dictionary *dict;
+ struct pivot_table_look *look;
+
+ /* For CTEF_* formats. */
+ struct fmt_settings ctables_formats;
+
+ /* If this is NULL, zeros are displayed using the normal print format.
+ Otherwise, this string is displayed. */
+ char *zero;
+
+ /* If this is NULL, missing values are displayed using the normal print
+ format. Otherwise, this string is displayed. */
+ char *missing;
+
+ /* Indexed by variable dictionary index. */
+ enum ctables_vlabel *vlabels;
+
+ struct hmap postcomputes; /* Contains "struct ctables_postcompute"s. */
+
+ bool mrsets_count_duplicates; /* MRSETS. */
+ bool smissing_listwise; /* SMISSING. */
+ struct variable *e_weight; /* WEIGHT. */
+ int hide_threshold; /* HIDESMALLCOUNTS. */
+
+ struct ctables_table **tables;
+ size_t n_tables;
+ };
+\f
+static double
+ctpo_add (double a, double b)
+{
+ return a + b;
+}
+
+static double
+ctpo_sub (double a, double b)
+{
+ return a - b;
+}
+
+static double
+ctpo_mul (double a, double b)
+{
+ return a * b;
+}
+
+static double
+ctpo_div (double a, double b)
+{
+ return b ? a / b : SYSMIS;
+}
+
+static double
+ctpo_pow (double a, double b)
+{
+ int save_errno = errno;
+ errno = 0;
+ double result = pow (a, b);
+ if (errno)
+ result = SYSMIS;
+ errno = save_errno;
+ return result;
+}
+
+static double
+ctpo_neg (double a, double b UNUSED)
+{
+ return -a;
+}
+
+struct ctables_pcexpr_evaluate_ctx
+ {
+ const struct ctables_cell *cell;
+ const struct ctables_section *section;
+ const struct ctables_categories *cats;
+ enum pivot_axis_type pc_a;
+ size_t pc_a_idx;
+ size_t summary_idx;
+ enum fmt_type parse_format;
+ };
+
+static double ctables_pcexpr_evaluate (
+ const struct ctables_pcexpr_evaluate_ctx *, const struct ctables_pcexpr *);
+
+static double
+ctables_pcexpr_evaluate_nonterminal (
+ const struct ctables_pcexpr_evaluate_ctx *ctx,
+ const struct ctables_pcexpr *e, size_t n_args,
+ double evaluate (double, double))
+{
+ double args[2] = { 0, 0 };
+ for (size_t i = 0; i < n_args; i++)
+ {
+ args[i] = ctables_pcexpr_evaluate (ctx, e->subs[i]);
+ if (!isfinite (args[i]) || args[i] == SYSMIS)
+ return SYSMIS;
+ }
+ return evaluate (args[0], args[1]);
+}
+
+static double
+ctables_pcexpr_evaluate_category (const struct ctables_pcexpr_evaluate_ctx *ctx,
+ const struct ctables_cell_value *pc_cv)
+{
+ const struct ctables_section *s = ctx->section;
+
+ size_t hash = 0;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ const struct ctables_cell_value *cv
+ = (a == ctx->pc_a && i == ctx->pc_a_idx ? pc_cv
+ : &ctx->cell->axes[a].cvs[i]);
+ hash = hash_pointer (cv->category, hash);
+ if (cv->category->type != CCT_TOTAL
+ && cv->category->type != CCT_SUBTOTAL
+ && cv->category->type != CCT_POSTCOMPUTE)
+ hash = value_hash (&cv->value,
+ var_get_width (nest->vars[i]), hash);
+ }
+ }
+
+ struct ctables_cell *tc;
+ HMAP_FOR_EACH_WITH_HASH (tc, struct ctables_cell, node, hash, &s->cells)
+ {
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ const struct ctables_cell_value *p_cv
+ = (a == ctx->pc_a && i == ctx->pc_a_idx ? pc_cv
+ : &ctx->cell->axes[a].cvs[i]);
+ const struct ctables_cell_value *t_cv = &tc->axes[a].cvs[i];
+ if (p_cv->category != t_cv->category
+ || (p_cv->category->type != CCT_TOTAL
+ && p_cv->category->type != CCT_SUBTOTAL
+ && p_cv->category->type != CCT_POSTCOMPUTE
+ && !value_equal (&p_cv->value,
+ &t_cv->value,
+ var_get_width (nest->vars[i]))))
+ goto not_equal;
+ }
+ }
+
+ goto found;
+
+ not_equal: ;
+ }
+ return 0;
+
+found: ;
+ const struct ctables_table *t = s->table;
+ const struct ctables_nest *specs_nest = s->nests[t->summary_axis];
+ const struct ctables_summary_spec_set *specs = &specs_nest->specs[tc->sv];
+ return ctables_summary_value (tc->areas, &tc->summaries[ctx->summary_idx],
+ &specs->specs[ctx->summary_idx]);
+}
+
+static double
+ctables_pcexpr_evaluate (const struct ctables_pcexpr_evaluate_ctx *ctx,
+ const struct ctables_pcexpr *e)
+{
+ switch (e->op)
+ {
+ case CTPO_CONSTANT:
+ return e->number;
+
+ case CTPO_CAT_NRANGE:
+ case CTPO_CAT_SRANGE:
+ case CTPO_CAT_MISSING:
+ case CTPO_CAT_OTHERNM:
+ {
+ struct ctables_cell_value cv = {
+ .category = ctables_find_category_for_postcompute (ctx->section->table->ctables->dict, ctx->cats, ctx->parse_format, e)
+ };
+ assert (cv.category != NULL);
+
+ struct hmap *occurrences = &ctx->section->occurrences[ctx->pc_a][ctx->pc_a_idx];
+ const struct ctables_occurrence *o;
+
+ double sum = 0.0;
+ const struct variable *var = ctx->section->nests[ctx->pc_a]->vars[ctx->pc_a_idx];
+ HMAP_FOR_EACH (o, struct ctables_occurrence, node, occurrences)
+ if (ctables_categories_match (ctx->cats, &o->value, var) == cv.category)
+ {
+ cv.value = o->value;
+ sum += ctables_pcexpr_evaluate_category (ctx, &cv);
+ }
+ return sum;
+ }
+
+ case CTPO_CAT_NUMBER:
+ case CTPO_CAT_SUBTOTAL:
+ case CTPO_CAT_TOTAL:
+ {
+ struct ctables_cell_value cv = {
+ .category = ctables_find_category_for_postcompute (ctx->section->table->ctables->dict, ctx->cats, ctx->parse_format, e),
+ .value = { .f = e->number },
+ };
+ assert (cv.category != NULL);
+ return ctables_pcexpr_evaluate_category (ctx, &cv);
+ }
+
+ case CTPO_CAT_STRING:
+ {
+ int width = var_get_width (ctx->section->nests[ctx->pc_a]->vars[ctx->pc_a_idx]);
+ char *s = NULL;
+ if (width > e->string.length)
+ {
+ s = xmalloc (width);
+ buf_copy_rpad (s, width, e->string.string, e->string.length, ' ');
+ }
+
+ const struct ctables_category *category
+ = ctables_find_category_for_postcompute (
+ ctx->section->table->ctables->dict,
+ ctx->cats, ctx->parse_format, e);
+ assert (category != NULL);
+
+ struct ctables_cell_value cv = { .category = category };
+ if (category->type == CCT_NUMBER)
+ cv.value.f = category->number;
+ else if (category->type == CCT_STRING)
+ cv.value.s = CHAR_CAST (uint8_t *, s ? s : e->string.string);
+ else
+ NOT_REACHED ();
+
+ double retval = ctables_pcexpr_evaluate_category (ctx, &cv);
+ free (s);
+ return retval;
+ }
+
+ case CTPO_ADD:
+ return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_add);
+
+ case CTPO_SUB:
+ return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_sub);
+
+ case CTPO_MUL:
+ return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_mul);
+
+ case CTPO_DIV:
+ return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_div);
+
+ case CTPO_POW:
+ return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_pow);
+
+ case CTPO_NEG:
+ return ctables_pcexpr_evaluate_nonterminal (ctx, e, 1, ctpo_neg);
+ }
+
+ NOT_REACHED ();
+}
+
+static const struct ctables_category *
+ctables_cell_postcompute (const struct ctables_section *s,
+ const struct ctables_cell *cell,
+ enum pivot_axis_type *pc_a_p,
+ size_t *pc_a_idx_p)
+{
+ assert (cell->postcompute);
+ const struct ctables_category *pc_cat = NULL;
+ for (enum pivot_axis_type pc_a = 0; pc_a < PIVOT_N_AXES; pc_a++)
+ for (size_t pc_a_idx = 0; pc_a_idx < s->nests[pc_a]->n; pc_a_idx++)
+ {
+ const struct ctables_cell_value *cv = &cell->axes[pc_a].cvs[pc_a_idx];
+ if (cv->category->type == CCT_POSTCOMPUTE)
+ {
+ if (pc_cat)
+ {
+ /* Multiple postcomputes cross each other. The value is
+ undefined. */
+ return NULL;
+ }
+
+ pc_cat = cv->category;
+ if (pc_a_p)
+ *pc_a_p = pc_a;
+ if (pc_a_idx_p)
+ *pc_a_idx_p = pc_a_idx;
+ }
+ }
+
+ assert (pc_cat != NULL);
+ return pc_cat;
+}
+
+static double
+ctables_cell_calculate_postcompute (const struct ctables_section *s,
+ const struct ctables_cell *cell,
+ const struct ctables_summary_spec *ss,
+ struct fmt_spec *format,
+ bool *is_ctables_format,
+ size_t summary_idx)
+{
+ enum pivot_axis_type pc_a = 0;
+ size_t pc_a_idx = 0;
+ const struct ctables_category *pc_cat = ctables_cell_postcompute (
+ s, cell, &pc_a, &pc_a_idx);
+ if (!pc_cat)
+ return SYSMIS;
+
+ const struct ctables_postcompute *pc = pc_cat->pc;
+ if (pc->specs)
+ {
+ for (size_t i = 0; i < pc->specs->n; i++)
+ {
+ const struct ctables_summary_spec *ss2 = &pc->specs->specs[i];
+ if (ss->function == ss2->function
+ && ss->weighting == ss2->weighting
+ && ss->calc_area == ss2->calc_area
+ && ss->percentile == ss2->percentile)
+ {
+ *format = ss2->format;
+ *is_ctables_format = ss2->is_ctables_format;
+ break;
+ }
+ }
+ }
+
+ const struct variable *var = s->nests[pc_a]->vars[pc_a_idx];
+ const struct ctables_categories *cats = s->table->categories[
+ var_get_dict_index (var)];
+ struct ctables_pcexpr_evaluate_ctx ctx = {
+ .cell = cell,
+ .section = s,
+ .cats = cats,
+ .pc_a = pc_a,
+ .pc_a_idx = pc_a_idx,
+ .summary_idx = summary_idx,
+ .parse_format = pc_cat->parse_format,
+ };
+ return ctables_pcexpr_evaluate (&ctx, pc->expr);
+}
+\f
+/* Chi-square test (SIGTEST). */
+struct ctables_chisq
+ {
+ double alpha;
+ bool include_mrsets;
+ bool all_visible;
+ };
+
+/* Pairwise comparison test (COMPARETEST). */
+struct ctables_pairwise
+ {
+ enum { PROP, MEAN } type;
+ double alpha[2];
+ bool include_mrsets;
+ bool meansvariance_allcats;
+ bool all_visible;
+ enum { BONFERRONI = 1, BH } adjust;
+ bool merge;
+ bool apa_style;
+ bool show_sig;
+ };
+
+
+
+static bool
+parse_col_width (struct lexer *lexer, const char *name, double *width)
+{
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "DEFAULT"))
+ *width = SYSMIS;
+ else if (lex_force_num_range_closed (lexer, name, 0, DBL_MAX))
+ {
+ *width = lex_number (lexer);
+ lex_get (lexer);
+ }
+ else
+ return false;
+
+ return true;
+}
+
+static bool
+parse_bool (struct lexer *lexer, bool *b)
+{
+ if (lex_match_id (lexer, "NO"))
+ *b = false;
+ else if (lex_match_id (lexer, "YES"))
+ *b = true;
+ else
+ {
+ lex_error_expecting (lexer, "YES", "NO");
+ return false;
+ }
+ return true;
+}
+
+static void
+ctables_chisq_destroy (struct ctables_chisq *chisq)
+{
+ free (chisq);
+}
+
+static void
+ctables_pairwise_destroy (struct ctables_pairwise *pairwise)
+{
+ free (pairwise);
+}
+
+static void
+ctables_table_destroy (struct ctables_table *t)
+{
+ if (!t)
+ return;
+
+ for (size_t i = 0; i < t->n_sections; i++)
+ ctables_section_uninit (&t->sections[i]);
+ free (t->sections);
+
+ for (size_t i = 0; i < t->n_categories; i++)
+ ctables_categories_unref (t->categories[i]);
+ free (t->categories);
+ free (t->show_empty);
+
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ ctables_axis_destroy (t->axes[a]);
+ ctables_stack_uninit (&t->stacks[a]);
+ }
+ free (t->summary_specs.specs);
+
+ struct ctables_value *ctv, *next_ctv;
+ HMAP_FOR_EACH_SAFE (ctv, next_ctv, struct ctables_value, node,
+ &t->clabels_values_map)
+ {
+ value_destroy (&ctv->value, var_get_width (t->clabels_example));
+ hmap_delete (&t->clabels_values_map, &ctv->node);
+ free (ctv);
+ }
+ hmap_destroy (&t->clabels_values_map);
+ free (t->clabels_values);
+
+ free (t->sum_vars);
+ free (t->caption);
+ free (t->corner);
+ free (t->title);
+ ctables_chisq_destroy (t->chisq);
+ ctables_pairwise_destroy (t->pairwise);
+ free (t);
+}
+
+static void
+ctables_destroy (struct ctables *ct)
+{
+ if (!ct)
+ return;
+
+ struct ctables_postcompute *pc, *next_pc;
+ HMAP_FOR_EACH_SAFE (pc, next_pc, struct ctables_postcompute, hmap_node,
+ &ct->postcomputes)
+ {
+ free (pc->name);
+ msg_location_destroy (pc->location);
+ ctables_pcexpr_destroy (pc->expr);
+ free (pc->label);
+ if (pc->specs)
+ {
+ ctables_summary_spec_set_uninit (pc->specs);
+ free (pc->specs);
+ }
+ hmap_delete (&ct->postcomputes, &pc->hmap_node);
+ free (pc);
+ }
+ hmap_destroy (&ct->postcomputes);
+
+ fmt_settings_uninit (&ct->ctables_formats);
+ pivot_table_look_unref (ct->look);
+ free (ct->zero);
+ free (ct->missing);
+ free (ct->vlabels);
+ for (size_t i = 0; i < ct->n_tables; i++)
+ ctables_table_destroy (ct->tables[i]);
+ free (ct->tables);
+ free (ct);
+}
+
+static bool
+all_strings (struct variable **vars, size_t n_vars,
+ const struct ctables_category *cat)
+{
+ for (size_t j = 0; j < n_vars; j++)
+ if (var_is_numeric (vars[j]))
+ {
+ msg_at (SE, cat->location,
+ _("This category specification may be applied only to string "
+ "variables, but this subcommand tries to apply it to "
+ "numeric variable %s."),
+ var_get_name (vars[j]));
+ return false;
+ }
+ return true;
+}
+
+static bool
+ctables_table_parse_categories (struct lexer *lexer, struct dictionary *dict,
+ struct ctables *ct, struct ctables_table *t)
+{
+ if (!lex_force_match_id (lexer, "VARIABLES"))
+ return false;
+ lex_match (lexer, T_EQUALS);
+
+ struct variable **vars;
+ size_t n_vars;
+ if (!parse_variables (lexer, dict, &vars, &n_vars, PV_NO_SCRATCH))
+ return false;
+
+ const struct fmt_spec *common_format = var_get_print_format (vars[0]);
+ for (size_t i = 1; i < n_vars; i++)
+ {
+ const struct fmt_spec *f = var_get_print_format (vars[i]);
+ if (f->type != common_format->type)
+ {
+ common_format = NULL;
+ break;
+ }
+ }
+ bool parse_strings
+ = (common_format
+ && (fmt_get_category (common_format->type)
+ & (FMT_CAT_DATE | FMT_CAT_TIME | FMT_CAT_DATE_COMPONENT)));
+
+ struct ctables_categories *c = xmalloc (sizeof *c);
+ *c = (struct ctables_categories) { .n_refs = 1 };
+
+ bool set_categories = false;
+
+ size_t allocated_cats = 0;
+ int cats_start_ofs = -1;
+ int cats_end_ofs = -1;
+ if (lex_match (lexer, T_LBRACK))
+ {
+ set_categories = true;
+ cats_start_ofs = lex_ofs (lexer);
+ do
+ {
+ if (c->n_cats >= allocated_cats)
+ c->cats = x2nrealloc (c->cats, &allocated_cats, sizeof *c->cats);
+
+ int start_ofs = lex_ofs (lexer);
+ struct ctables_category *cat = &c->cats[c->n_cats];
+ if (!ctables_table_parse_explicit_category (lexer, dict, ct, cat))
+ goto error;
+ cat->location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1);
+ c->n_cats++;
+
+ lex_match (lexer, T_COMMA);
+ }
+ while (!lex_match (lexer, T_RBRACK));
+ cats_end_ofs = lex_ofs (lexer) - 1;
+ }
+
+ struct ctables_category cat = {
+ .type = CCT_VALUE,
+ .include_missing = false,
+ .sort_ascending = true,
+ };
+ bool show_totals = false;
+ char *total_label = NULL;
+ bool totals_before = false;
+ int key_start_ofs = 0;
+ int key_end_ofs = 0;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (!c->n_cats && lex_match_id (lexer, "ORDER"))
+ {
+ set_categories = true;
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "A"))
+ cat.sort_ascending = true;
+ else if (lex_match_id (lexer, "D"))
+ cat.sort_ascending = false;
+ else
+ {
+ lex_error_expecting (lexer, "A", "D");
+ goto error;
+ }
+ }
+ else if (!c->n_cats && lex_match_id (lexer, "KEY"))
+ {
+ set_categories = true;
+ key_start_ofs = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "VALUE"))
+ cat.type = CCT_VALUE;
+ else if (lex_match_id (lexer, "LABEL"))
+ cat.type = CCT_LABEL;
+ else
+ {
+ cat.type = CCT_FUNCTION;
+ if (!parse_ctables_summary_function (lexer, &cat.sort_function,
+ &cat.weighting, &cat.area))
+ goto error;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ cat.sort_var = parse_variable (lexer, dict);
+ if (!cat.sort_var)
+ goto error;
+
+ if (cat.sort_function == CTSF_PTILE)
+ {
+ lex_match (lexer, T_COMMA);
+ if (!lex_force_num_range_closed (lexer, "PTILE", 0, 100))
+ goto error;
+ cat.percentile = lex_number (lexer);
+ lex_get (lexer);
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (ctables_function_availability (cat.sort_function)
+ == CTFA_SCALE)
+ {
+ bool UNUSED b = lex_force_match (lexer, T_LPAREN);
+ goto error;
+ }
+ }
+ key_end_ofs = lex_ofs (lexer) - 1;
+
+ if (cat.type == CCT_FUNCTION)
+ {
+ lex_ofs_error (lexer, key_start_ofs, key_end_ofs,
+ _("Data-dependent sorting is not implemented."));
+ goto error;
+ }
+ }
+ else if (!c->n_cats && lex_match_id (lexer, "MISSING"))
+ {
+ set_categories = true;
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "INCLUDE"))
+ cat.include_missing = true;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ cat.include_missing = false;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "TOTAL"))
+ {
+ set_categories = true;
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &show_totals))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "LABEL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+ free (total_label);
+ total_label = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "POSITION"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "BEFORE"))
+ totals_before = true;
+ else if (lex_match_id (lexer, "AFTER"))
+ totals_before = false;
+ else
+ {
+ lex_error_expecting (lexer, "BEFORE", "AFTER");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "EMPTY"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ bool show_empty;
+ if (lex_match_id (lexer, "INCLUDE"))
+ show_empty = true;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ show_empty = false;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+
+ for (size_t i = 0; i < n_vars; i++)
+ t->show_empty[var_get_dict_index (vars[i])] = show_empty;
+ }
+ else
+ {
+ if (!c->n_cats)
+ lex_error_expecting (lexer, "ORDER", "KEY", "MISSING",
+ "TOTAL", "LABEL", "POSITION", "EMPTY");
+ else
+ lex_error_expecting (lexer, "TOTAL", "LABEL", "POSITION", "EMPTY");
+ goto error;
+ }
+ }
+
+ if (!c->n_cats)
+ {
+ if (key_start_ofs)
+ cat.location = lex_ofs_location (lexer, key_start_ofs, key_end_ofs);
+
+ if (c->n_cats >= allocated_cats)
+ c->cats = x2nrealloc (c->cats, &allocated_cats, sizeof *c->cats);
+ c->cats[c->n_cats++] = cat;
+ }
+
+ if (show_totals)
+ {
+ if (c->n_cats >= allocated_cats)
+ c->cats = x2nrealloc (c->cats, &allocated_cats, sizeof *c->cats);
+
+ struct ctables_category *totals;
+ if (totals_before)
+ {
+ insert_element (c->cats, c->n_cats, sizeof *c->cats, 0);
+ totals = &c->cats[0];
+ }
+ else
+ totals = &c->cats[c->n_cats];
+ c->n_cats++;
+
+ *totals = (struct ctables_category) {
+ .type = CCT_TOTAL,
+ .total_label = total_label ? total_label : xstrdup (_("Total")),
+ };
+ }
+
+ struct ctables_category *subtotal = NULL;
+ for (size_t i = totals_before ? 0 : c->n_cats;
+ totals_before ? i < c->n_cats : i-- > 0;
+ totals_before ? i++ : 0)
+ {
+ struct ctables_category *cat = &c->cats[i];
+ switch (cat->type)
+ {
+ case CCT_NUMBER:
+ case CCT_STRING:
+ case CCT_NRANGE:
+ case CCT_SRANGE:
+ case CCT_MISSING:
+ case CCT_OTHERNM:
+ cat->subtotal = subtotal;
+ break;
+
+ case CCT_POSTCOMPUTE:
+ break;
+
+ case CCT_SUBTOTAL:
+ subtotal = cat;
+ break;
+
+ case CCT_TOTAL:
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ case CCT_EXCLUDED_MISSING:
+ break;
+ }
+ }
+
+ if (cats_start_ofs != -1)
+ {
+ for (size_t i = 0; i < c->n_cats; i++)
+ {
+ struct ctables_category *cat = &c->cats[i];
+ switch (cat->type)
+ {
+ case CCT_POSTCOMPUTE:
+ cat->parse_format = parse_strings ? common_format->type : FMT_F;
+ struct msg_location *cats_location
+ = lex_ofs_location (lexer, cats_start_ofs, cats_end_ofs);
+ bool ok = ctables_recursive_check_postcompute (
+ dict, cat->pc->expr, cat, c, cats_location);
+ msg_location_destroy (cats_location);
+ if (!ok)
+ goto error;
+ break;
+
+ case CCT_NUMBER:
+ case CCT_NRANGE:
+ for (size_t j = 0; j < n_vars; j++)
+ if (var_is_alpha (vars[j]))
+ {
+ msg_at (SE, cat->location,
+ _("This category specification may be applied "
+ "only to numeric variables, but this "
+ "subcommand tries to apply it to string "
+ "variable %s."),
+ var_get_name (vars[j]));
+ goto error;
+ }
+ break;
+
+ case CCT_STRING:
+ if (parse_strings)
+ {
+ double n;
+ if (!parse_category_string (cat->location, cat->string, dict,
+ common_format->type, &n))
+ goto error;
+
+ ss_dealloc (&cat->string);
+
+ cat->type = CCT_NUMBER;
+ cat->number = n;
+ }
+ else if (!all_strings (vars, n_vars, cat))
+ goto error;
+ break;
+
+ case CCT_SRANGE:
+ if (parse_strings)
+ {
+ double n[2];
+
+ if (!cat->srange[0].string)
+ n[0] = -DBL_MAX;
+ else if (!parse_category_string (cat->location,
+ cat->srange[0], dict,
+ common_format->type, &n[0]))
+ goto error;
+
+ if (!cat->srange[1].string)
+ n[1] = DBL_MAX;
+ else if (!parse_category_string (cat->location,
+ cat->srange[1], dict,
+ common_format->type, &n[1]))
+ goto error;
+
+ ss_dealloc (&cat->srange[0]);
+ ss_dealloc (&cat->srange[1]);
+
+ cat->type = CCT_NRANGE;
+ cat->nrange[0] = n[0];
+ cat->nrange[1] = n[1];
+ }
+ else if (!all_strings (vars, n_vars, cat))
+ goto error;
+ break;
+
+ case CCT_MISSING:
+ case CCT_OTHERNM:
+ case CCT_SUBTOTAL:
+ case CCT_TOTAL:
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ case CCT_EXCLUDED_MISSING:
+ break;
+ }
+ }
+ }
+
+ if (set_categories)
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ struct ctables_categories **cp
+ = &t->categories[var_get_dict_index (vars[i])];
+ ctables_categories_unref (*cp);
+ *cp = c;
+ c->n_refs++;
+ }
+
+ ctables_categories_unref (c);
+ free (vars);
+ return true;
+
+error:
+ ctables_categories_unref (c);
+ free (vars);
+ return false;
+}
+\f
+
+struct merge_item
+ {
+ const struct ctables_summary_spec_set *set;
+ size_t ofs;
+ };
+
+static int
+merge_item_compare_3way (const struct merge_item *a, const struct merge_item *b)
+{
+ const struct ctables_summary_spec *as = &a->set->specs[a->ofs];
+ const struct ctables_summary_spec *bs = &b->set->specs[b->ofs];
+ if (as->function != bs->function)
+ return as->function > bs->function ? 1 : -1;
+ else if (as->weighting != bs->weighting)
+ return as->weighting > bs->weighting ? 1 : -1;
+ else if (as->calc_area != bs->calc_area)
+ return as->calc_area > bs->calc_area ? 1 : -1;
+ else if (as->percentile != bs->percentile)
+ return as->percentile < bs->percentile ? 1 : -1;
+
+ const char *as_label = as->label ? as->label : "";
+ const char *bs_label = bs->label ? bs->label : "";
+ return strcmp (as_label, bs_label);
+}
+
+static void
+ctables_table_add_section (struct ctables_table *t, enum pivot_axis_type a,
+ size_t ix[PIVOT_N_AXES])
+{
+ if (a < PIVOT_N_AXES)
+ {
+ size_t limit = MAX (t->stacks[a].n, 1);
+ for (ix[a] = 0; ix[a] < limit; ix[a]++)
+ ctables_table_add_section (t, a + 1, ix);
+ }
+ else
+ {
+ struct ctables_section *s = &t->sections[t->n_sections++];
+ *s = (struct ctables_section) {
+ .table = t,
+ .cells = HMAP_INITIALIZER (s->cells),
+ };
+ for (a = 0; a < PIVOT_N_AXES; a++)
+ if (t->stacks[a].n)
+ {
+ struct ctables_nest *nest = &t->stacks[a].nests[ix[a]];
+ s->nests[a] = nest;
+ s->occurrences[a] = xnmalloc (nest->n, sizeof *s->occurrences[a]);
+ for (size_t i = 0; i < nest->n; i++)
+ hmap_init (&s->occurrences[a][i]);
+ }
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ hmap_init (&s->areas[at]);
+ }
+}
+
+static char *
+ctables_format (double d, const struct fmt_spec *format,
+ const struct fmt_settings *settings)
+{
+ const union value v = { .f = d };
+ char *s = data_out_stretchy (&v, "UTF-8", format, settings, NULL);
+
+ /* The custom-currency specifications for NEQUAL, PAREN, and PCTPAREN don't
+ produce the results we want for negative numbers, putting the negative
+ sign in the wrong spot, before the prefix instead of after it. We can't,
+ in fact, produce the desired results using a custom-currency
+ specification. Instead, we postprocess the output, moving the negative
+ sign into place:
+
+ NEQUAL: "-N=3" => "N=-3"
+ PAREN: "-(3)" => "(-3)"
+ PCTPAREN: "-(3%)" => "(-3%)"
+
+ This transformation doesn't affect NEGPAREN. */
+ char *minus_src = strchr (s, '-');
+ if (minus_src && (minus_src == s || minus_src[-1] != 'E'))
+ {
+ char *n_equals = strstr (s, "N=");
+ char *lparen = strchr (s, '(');
+ char *minus_dst = n_equals ? n_equals + 1 : lparen;
+ if (minus_dst)
+ move_element (s, minus_dst - s + 1, 1, minus_src - s, minus_dst - s);
+ }
+ return s;
+}
+
+static bool
+all_hidden_vlabels (const struct ctables_table *t, enum pivot_axis_type a)
+{
+ for (size_t i = 0; i < t->stacks[a].n; i++)
+ {
+ struct ctables_nest *nest = &t->stacks[a].nests[i];
+ if (nest->n != 1 || nest->scale_idx != 0)
+ return false;
+
+ enum ctables_vlabel vlabel
+ = t->ctables->vlabels[var_get_dict_index (nest->vars[0])];
+ if (vlabel != CTVL_NONE)
+ return false;
+ }
+ return true;
+}
+
+static int
+compare_ints_3way (int a, int b)
+{
+ return a < b ? -1 : a > b;
+}
+
+static int
+ctables_cell_compare_leaf_3way (const void *a_, const void *b_,
+ const void *aux UNUSED)
+{
+ struct ctables_cell *const *ap = a_;
+ struct ctables_cell *const *bp = b_;
+ const struct ctables_cell *a = *ap;
+ const struct ctables_cell *b = *bp;
+
+ if (a == b)
+ {
+ assert (a_ == b_);
+ return 0;
+ }
+
+ for (enum pivot_axis_type axis = 0; axis < PIVOT_N_AXES; axis++)
+ {
+ int cmp = compare_ints_3way (a->axes[axis].leaf, b->axes[axis].leaf);
+ if (cmp)
+ return cmp;
+ }
+
+ const struct ctables_value *a_ctv = ctables_value_find (a);
+ const struct ctables_value *b_ctv = ctables_value_find (b);
+ if (a_ctv && b_ctv)
+ {
+ int cmp = compare_ints_3way (a_ctv->leaf, b_ctv->leaf);
+ if (cmp)
+ return cmp;
+ }
+ else
+ assert (!a_ctv && !b_ctv);
+ return 0;
+}
+
+static void
+ctables_table_output (struct ctables *ct, struct ctables_table *t)
+{
+ struct pivot_table *pt = pivot_table_create__ (
+ (t->title
+ ? pivot_value_new_user_text (t->title, SIZE_MAX)
+ : pivot_value_new_text (N_("Custom Tables"))),
+ "Custom Tables");
+ if (t->caption)
+ pivot_table_set_caption (
+ pt, pivot_value_new_user_text (t->caption, SIZE_MAX));
+ if (t->corner)
+ pivot_table_set_corner_text (
+ pt, pivot_value_new_user_text (t->corner, SIZE_MAX));
+
+ bool summary_dimension = (t->summary_axis != t->slabels_axis
+ || (!t->slabels_visible
+ && t->summary_specs.n > 1));
+ if (summary_dimension)
+ {
+ struct pivot_dimension *d = pivot_dimension_create (
+ pt, t->slabels_axis, N_("Statistics"));
+ const struct ctables_summary_spec_set *specs = &t->summary_specs;
+ if (!t->slabels_visible)
+ d->hide_all_labels = true;
+ for (size_t i = 0; i < specs->n; i++)
+ pivot_category_create_leaf (
+ d->root, ctables_summary_label (&specs->specs[i], t->cilevel));
+ }
+
+ bool categories_dimension = t->clabels_example != NULL;
+ if (categories_dimension)
+ {
+ struct pivot_dimension *d = pivot_dimension_create (
+ pt, t->label_axis[t->clabels_from_axis],
+ t->clabels_from_axis == PIVOT_AXIS_ROW
+ ? N_("Row Categories")
+ : N_("Column Categories"));
+ const struct variable *var = t->clabels_example;
+ const struct ctables_categories *c = t->categories[var_get_dict_index (var)];
+ for (size_t i = 0; i < t->n_clabels_values; i++)
+ {
+ const struct ctables_value *value = t->clabels_values[i];
+ const struct ctables_category *cat = ctables_categories_match (c, &value->value, var);
+ assert (cat != NULL);
+ pivot_category_create_leaf (
+ d->root, ctables_category_create_value_label (c, cat,
+ t->clabels_example,
+ &value->value));
+ }
+ }
+
+ pivot_table_set_look (pt, ct->look);
+ struct pivot_dimension *d[PIVOT_N_AXES];
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ static const char *names[] = {
+ [PIVOT_AXIS_ROW] = N_("Rows"),
+ [PIVOT_AXIS_COLUMN] = N_("Columns"),
+ [PIVOT_AXIS_LAYER] = N_("Layers"),
+ };
+ d[a] = (t->axes[a] || a == t->summary_axis
+ ? pivot_dimension_create (pt, a, names[a])
+ : NULL);
+ if (!d[a])
+ continue;
+
+ assert (t->axes[a]);
+
+ for (size_t i = 0; i < t->stacks[a].n; i++)
+ {
+ struct ctables_nest *nest = &t->stacks[a].nests[i];
+ struct ctables_section **sections = xnmalloc (t->n_sections,
+ sizeof *sections);
+ size_t n_sections = 0;
+
+ size_t n_total_cells = 0;
+ size_t max_depth = 0;
+ for (size_t j = 0; j < t->n_sections; j++)
+ if (t->sections[j].nests[a] == nest)
+ {
+ struct ctables_section *s = &t->sections[j];
+ sections[n_sections++] = s;
+ n_total_cells += hmap_count (&s->cells);
+
+ size_t depth = s->nests[a]->n;
+ max_depth = MAX (depth, max_depth);
+ }
+
+ struct ctables_cell **sorted = xnmalloc (n_total_cells,
+ sizeof *sorted);
+ size_t n_sorted = 0;
+
+ for (size_t j = 0; j < n_sections; j++)
+ {
+ struct ctables_section *s = sections[j];
+
+ struct ctables_cell *cell;
+ HMAP_FOR_EACH (cell, struct ctables_cell, node, &s->cells)
+ if (!cell->hide)
+ sorted[n_sorted++] = cell;
+ assert (n_sorted <= n_total_cells);
+ }
+
+ struct ctables_cell_sort_aux aux = { .nest = nest, .a = a };
+ sort (sorted, n_sorted, sizeof *sorted, ctables_cell_compare_3way, &aux);
+
+ struct ctables_level
+ {
+ enum ctables_level_type
+ {
+ CTL_VAR, /* Variable label for nest->vars[var_idx]. */
+ CTL_CATEGORY, /* Category for nest->vars[var_idx]. */
+ CTL_SUMMARY, /* Summary functions. */
+ }
+ type;
+
+ enum settings_value_show vlabel; /* CTL_VAR only. */
+ size_t var_idx;
+ };
+ struct ctables_level *levels = xnmalloc (1 + 2 * max_depth, sizeof *levels);
+ size_t n_levels = 0;
+ for (size_t k = 0; k < nest->n; k++)
+ {
+ enum ctables_vlabel vlabel = ct->vlabels[var_get_dict_index (nest->vars[k])];
+ if (vlabel == CTVL_NONE && nest->scale_idx == k)
+ vlabel = CTVL_NAME;
+ if (vlabel != CTVL_NONE)
+ {
+ levels[n_levels++] = (struct ctables_level) {
+ .type = CTL_VAR,
+ .vlabel = (enum settings_value_show) vlabel,
+ .var_idx = k,
+ };
+ }
+
+ if (nest->scale_idx != k
+ && (k != nest->n - 1 || t->label_axis[a] == a))
+ {
+ levels[n_levels++] = (struct ctables_level) {
+ .type = CTL_CATEGORY,
+ .var_idx = k,
+ };
+ }
+ }
+
+ if (!summary_dimension && a == t->slabels_axis)
+ {
+ levels[n_levels++] = (struct ctables_level) {
+ .type = CTL_SUMMARY,
+ .var_idx = SIZE_MAX,
+ };
+ }
+
+ /* Pivot categories:
+
+ - variable label for nest->vars[0], if vlabel != CTVL_NONE
+ - category for nest->vars[0], if nest->scale_idx != 0
+ - variable label for nest->vars[1], if vlabel != CTVL_NONE
+ - category for nest->vars[1], if nest->scale_idx != 1
+ ...
+ - variable label for nest->vars[n - 1], if vlabel != CTVL_NONE
+ - category for nest->vars[n - 1], if t->label_axis[a] == a && nest->scale_idx != n - 1.
+ - summary function, if 'a == t->slabels_axis && a ==
+ t->summary_axis'.
+
+ Additional dimensions:
+
+ - If 'a == t->slabels_axis && a != t->summary_axis', add a summary
+ dimension.
+ - If 't->label_axis[b] == a' for some 'b != a', add a category
+ dimension to 'a'.
+ */
+
+
+ struct pivot_category **groups = xnmalloc (1 + 2 * max_depth, sizeof *groups);
+ int prev_leaf = 0;
+ for (size_t j = 0; j < n_sorted; j++)
+ {
+ struct ctables_cell *cell = sorted[j];
+ struct ctables_cell *prev = j > 0 ? sorted[j - 1] : NULL;
+
+ size_t n_common = 0;
+ if (j > 0)
+ {
+ for (; n_common < n_levels; n_common++)
+ {
+ const struct ctables_level *level = &levels[n_common];
+ if (level->type == CTL_CATEGORY)
+ {
+ size_t var_idx = level->var_idx;
+ const struct ctables_category *c = cell->axes[a].cvs[var_idx].category;
+ if (prev->axes[a].cvs[var_idx].category != c)
+ break;
+ else if (c->type != CCT_SUBTOTAL
+ && c->type != CCT_TOTAL
+ && c->type != CCT_POSTCOMPUTE
+ && !value_equal (&prev->axes[a].cvs[var_idx].value,
+ &cell->axes[a].cvs[var_idx].value,
+ var_get_type (nest->vars[var_idx])))
+ break;
+ }
+ }
+ }
+
+ for (size_t k = n_common; k < n_levels; k++)
+ {
+ const struct ctables_level *level = &levels[k];
+ struct pivot_category *parent = k ? groups[k - 1] : d[a]->root;
+ if (level->type == CTL_SUMMARY)
+ {
+ assert (k == n_levels - 1);
+
+ const struct ctables_summary_spec_set *specs = &t->summary_specs;
+ for (size_t m = 0; m < specs->n; m++)
+ {
+ int leaf = pivot_category_create_leaf (
+ parent, ctables_summary_label (&specs->specs[m],
+ t->cilevel));
+ if (!m)
+ prev_leaf = leaf;
+ }
+ }
+ else
+ {
+ const struct variable *var = nest->vars[level->var_idx];
+ struct pivot_value *label;
+ if (level->type == CTL_VAR)
+ {
+ label = pivot_value_new_variable (var);
+ label->variable.show = level->vlabel;
+ }
+ else if (level->type == CTL_CATEGORY)
+ {
+ const struct ctables_cell_value *cv = &cell->axes[a].cvs[level->var_idx];
+ label = ctables_category_create_value_label (
+ t->categories[var_get_dict_index (var)],
+ cv->category, var, &cv->value);
+ }
+ else
+ NOT_REACHED ();
+
+ if (k == n_levels - 1)
+ prev_leaf = pivot_category_create_leaf (parent, label);
+ else
+ groups[k] = pivot_category_create_group__ (parent, label);
+ }
+ }
+
+ cell->axes[a].leaf = prev_leaf;
+ }
+ free (sorted);
+ free (groups);
+ free (levels);
+ free (sections);
+
+ }
+
+ d[a]->hide_all_labels = all_hidden_vlabels (t, a);
+ }
+
+ {
+ size_t n_total_cells = 0;
+ for (size_t j = 0; j < t->n_sections; j++)
+ n_total_cells += hmap_count (&t->sections[j].cells);
+
+ struct ctables_cell **sorted = xnmalloc (n_total_cells, sizeof *sorted);
+ size_t n_sorted = 0;
+ for (size_t j = 0; j < t->n_sections; j++)
+ {
+ const struct ctables_section *s = &t->sections[j];
+ struct ctables_cell *cell;
+ HMAP_FOR_EACH (cell, struct ctables_cell, node, &s->cells)
+ if (!cell->hide)
+ sorted[n_sorted++] = cell;
+ }
+ assert (n_sorted <= n_total_cells);
+ sort (sorted, n_sorted, sizeof *sorted, ctables_cell_compare_leaf_3way,
+ NULL);
+ size_t ids[N_CTATS];
+ memset (ids, 0, sizeof ids);
+ for (size_t j = 0; j < n_sorted; j++)
+ {
+ struct ctables_cell *cell = sorted[j];
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ {
+ struct ctables_area *area = cell->areas[at];
+ if (!area->sequence)
+ area->sequence = ++ids[at];
+ }
+ }
+
+ free (sorted);
+ }
+
+ for (size_t i = 0; i < t->n_sections; i++)
+ {
+ struct ctables_section *s = &t->sections[i];
+
+ struct ctables_cell *cell;
+ HMAP_FOR_EACH (cell, struct ctables_cell, node, &s->cells)
+ {
+ if (cell->hide)
+ continue;
+
+ const struct ctables_value *ctv = ctables_value_find (cell);
+ const struct ctables_nest *specs_nest = s->nests[t->summary_axis];
+ const struct ctables_summary_spec_set *specs = &specs_nest->specs[cell->sv];
+ for (size_t j = 0; j < specs->n; j++)
+ {
+ size_t dindexes[5];
+ size_t n_dindexes = 0;
+
+ if (summary_dimension)
+ dindexes[n_dindexes++] = specs->specs[j].axis_idx;
+
+ if (ctv)
+ dindexes[n_dindexes++] = ctv->leaf;
+
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ if (d[a])
+ {
+ int leaf = cell->axes[a].leaf;
+ if (a == t->summary_axis && !summary_dimension)
+ leaf += specs->specs[j].axis_idx;
+ dindexes[n_dindexes++] = leaf;
+ }
+
+ const struct ctables_summary_spec *ss = &specs->specs[j];
+
+ struct fmt_spec format = specs->specs[j].format;
+ bool is_ctables_format = ss->is_ctables_format;
+ double d = (cell->postcompute
+ ? ctables_cell_calculate_postcompute (
+ s, cell, ss, &format, &is_ctables_format, j)
+ : ctables_summary_value (cell->areas,
+ &cell->summaries[j], ss));
+
+ struct pivot_value *value;
+ if (ct->hide_threshold != 0
+ && d < ct->hide_threshold
+ && ss->function == CTSF_COUNT)
+ {
+ value = pivot_value_new_user_text_nocopy (
+ xasprintf ("<%d", ct->hide_threshold));
+ }
+ else if (d == 0 && ct->zero)
+ value = pivot_value_new_user_text (ct->zero, SIZE_MAX);
+ else if (d == SYSMIS && ct->missing)
+ value = pivot_value_new_user_text (ct->missing, SIZE_MAX);
+ else if (is_ctables_format)
+ value = pivot_value_new_user_text_nocopy (
+ ctables_format (d, &format, &ct->ctables_formats));
+ else
+ {
+ value = pivot_value_new_number (d);
+ value->numeric.format = format;
+ }
+ /* XXX should text values be right-justified? */
+ pivot_table_put (pt, dindexes, n_dindexes, value);
+ }
+ }
+ }
+
+ pivot_table_submit (pt);
+}
+
+static bool
+ctables_check_label_position (struct ctables_table *t, struct lexer *lexer,
+ enum pivot_axis_type a)
+{
+ enum pivot_axis_type label_pos = t->label_axis[a];
+ if (label_pos == a)
+ return true;
+
+ const struct ctables_stack *stack = &t->stacks[a];
+ if (!stack->n)
+ return true;
+
+ const struct ctables_nest *n0 = &stack->nests[0];
+ if (n0->n == 0)
+ {
+ assert (stack->n == 1);
+ return true;
+ }
+
+ const struct variable *v0 = n0->vars[n0->n - 1];
+ struct ctables_categories *c0 = t->categories[var_get_dict_index (v0)];
+ t->clabels_example = v0;
+
+ for (size_t i = 0; i < c0->n_cats; i++)
+ if (c0->cats[i].type == CCT_FUNCTION)
+ {
+ msg (SE, _("Category labels may not be moved to another axis when "
+ "sorting by a summary function."));
+ lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
+ _("This syntax moves category labels to another axis."));
+ msg_at (SN, c0->cats[i].location,
+ _("This syntax requests sorting by a summary function."));
+ return false;
+ }
+
+ for (size_t i = 0; i < stack->n; i++)
+ {
+ const struct ctables_nest *ni = &stack->nests[i];
+ assert (ni->n > 0);
+ const struct variable *vi = ni->vars[ni->n - 1];
+ if (n0->n - 1 == ni->scale_idx)
+ {
+ msg (SE, _("To move category labels from one axis to another, "
+ "the variables whose labels are to be moved must be "
+ "categorical, but %s is scale."), var_get_name (vi));
+ lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
+ _("This syntax moves category labels to another axis."));
+ return false;
+ }
+ }
+
+ for (size_t i = 1; i < stack->n; i++)
+ {
+ const struct ctables_nest *ni = &stack->nests[i];
+ assert (ni->n > 0);
+ const struct variable *vi = ni->vars[ni->n - 1];
+ struct ctables_categories *ci = t->categories[var_get_dict_index (vi)];
+
+ if (var_get_width (v0) != var_get_width (vi))
+ {
+ msg (SE, _("To move category labels from one axis to another, "
+ "the variables whose labels are to be moved must all "
+ "have the same width, but %s has width %d and %s has "
+ "width %d."),
+ var_get_name (v0), var_get_width (v0),
+ var_get_name (vi), var_get_width (vi));
+ lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
+ _("This syntax moves category labels to another axis."));
+ return false;
+ }
+ if (!val_labs_equal (var_get_value_labels (v0),
+ var_get_value_labels (vi)))
+ {
+ msg (SE, _("To move category labels from one axis to another, "
+ "the variables whose labels are to be moved must all "
+ "have the same value labels, but %s and %s have "
+ "different value labels."),
+ var_get_name (v0), var_get_name (vi));
+ lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
+ _("This syntax moves category labels to another axis."));
+ return false;
+ }
+ if (!ctables_categories_equal (c0, ci))
+ {
+ msg (SE, _("To move category labels from one axis to another, "
+ "the variables whose labels are to be moved must all "
+ "have the same category specifications, but %s and %s "
+ "have different category specifications."),
+ var_get_name (v0), var_get_name (vi));
+ lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
+ _("This syntax moves category labels to another axis."));
+ return false;
+ }
+ }
+
+ return true;
+}
+
+static size_t
+add_sum_var (struct variable *var,
+ struct variable ***sum_vars, size_t *n, size_t *allocated)
+{
+ for (size_t i = 0; i < *n; i++)
+ if (var == (*sum_vars)[i])
+ return i;
+
+ if (*n >= *allocated)
+ *sum_vars = x2nrealloc (*sum_vars, allocated, sizeof **sum_vars);
+ (*sum_vars)[*n] = var;
+ return (*n)++;
+}
+
+static enum ctables_area_type
+rotate_area (enum ctables_area_type area)
+{
+ return area;
+ switch (area)
+ {
+ case CTAT_TABLE:
+ case CTAT_LAYER:
+ case CTAT_SUBTABLE:
+ return area;
+
+ case CTAT_LAYERROW:
+ return CTAT_LAYERCOL;
+
+ case CTAT_LAYERCOL:
+ return CTAT_LAYERROW;
+
+ case CTAT_ROW:
+ return CTAT_COL;
+
+ case CTAT_COL:
+ return CTAT_ROW;
+ }
+
+ NOT_REACHED ();
+}
+
+static void
+enumerate_sum_vars (const struct ctables_axis *a,
+ struct variable ***sum_vars, size_t *n, size_t *allocated)
+{
+ if (!a)
+ return;
+
+ switch (a->op)
+ {
+ case CTAO_VAR:
+ for (size_t i = 0; i < N_CSVS; i++)
+ for (size_t j = 0; j < a->specs[i].n; j++)
+ {
+ struct ctables_summary_spec *spec = &a->specs[i].specs[j];
+ if (spec->function == CTSF_areaPCT_SUM)
+ spec->sum_var_idx = add_sum_var (a->var, sum_vars, n, allocated);
+ }
+ break;
+
+ case CTAO_STACK:
+ case CTAO_NEST:
+ for (size_t i = 0; i < 2; i++)
+ enumerate_sum_vars (a->subs[i], sum_vars, n, allocated);
+ break;
+ }
+}
+
+static bool
+ctables_prepare_table (struct ctables_table *t, struct lexer *lexer)
+{
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ if (t->axes[a])
+ {
+ t->stacks[a] = enumerate_fts (a, t->axes[a]);
+
+ for (size_t j = 0; j < t->stacks[a].n; j++)
+ {
+ struct ctables_nest *nest = &t->stacks[a].nests[j];
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ {
+ nest->areas[at] = xmalloc (nest->n * sizeof *nest->areas[at]);
+ nest->n_areas[at] = 0;
+
+ enum pivot_axis_type ata, atb;
+ if (at == CTAT_ROW || at == CTAT_LAYERROW)
+ {
+ ata = PIVOT_AXIS_ROW;
+ atb = PIVOT_AXIS_COLUMN;
+ }
+ else if (at == CTAT_COL || at == CTAT_LAYERCOL)
+ {
+ ata = PIVOT_AXIS_COLUMN;
+ atb = PIVOT_AXIS_ROW;
+ }
+
+ if (at == CTAT_LAYER
+ ? a != PIVOT_AXIS_LAYER && t->label_axis[a] == PIVOT_AXIS_LAYER
+ : at == CTAT_LAYERCOL || at == CTAT_LAYERROW
+ ? a == atb && t->label_axis[a] != a
+ : false)
+ {
+ for (size_t k = nest->n - 1; k < nest->n; k--)
+ if (k != nest->scale_idx)
+ {
+ nest->areas[at][nest->n_areas[at]++] = k;
+ break;
+ }
+ continue;
+ }
+
+ if (at == CTAT_LAYER ? a != PIVOT_AXIS_LAYER
+ : at == CTAT_LAYERROW || at == CTAT_LAYERCOL ? a == atb
+ : at == CTAT_TABLE ? true
+ : false)
+ continue;
+
+ for (size_t k = 0; k < nest->n; k++)
+ if (k != nest->scale_idx)
+ nest->areas[at][nest->n_areas[at]++] = k;
+
+ int n_drop;
+ switch (at)
+ {
+ case CTAT_SUBTABLE:
+#define L PIVOT_AXIS_LAYER
+ n_drop = (t->clabels_from_axis == L ? a != L
+ : t->clabels_to_axis == L ? (t->clabels_from_axis == a ? -1 : a != L)
+ : t->clabels_from_axis == a ? 2
+ : 0);
+#undef L
+ break;
+
+ case CTAT_LAYERROW:
+ case CTAT_LAYERCOL:
+ n_drop = a == ata && t->label_axis[ata] == atb;
+ break;
+
+ case CTAT_ROW:
+ case CTAT_COL:
+ n_drop = (a == ata ? t->label_axis[ata] == atb
+ : a != atb ? 0
+ : t->clabels_from_axis == atb ? -1
+ : t->clabels_to_axis != atb ? 1
+ : 0);
+ break;
+
+ case CTAT_LAYER:
+ case CTAT_TABLE:
+ n_drop = 0;
+ break;
+ }
+
+ if (n_drop < 0)
+ {
+ size_t n = nest->n_areas[at];
+ if (n > 1)
+ {
+ nest->areas[at][n - 2] = nest->areas[at][n - 1];
+ nest->n_areas[at]--;
+ }
+ }
+ else
+ {
+ for (int i = 0; i < n_drop; i++)
+ if (nest->n_areas[at] > 0)
+ nest->n_areas[at]--;
+ }
+ }
+ }
+ }
+ else
+ {
+ struct ctables_nest *nest = xmalloc (sizeof *nest);
+ *nest = (struct ctables_nest) {
+ .n = 0,
+ .scale_idx = SIZE_MAX,
+ .summary_idx = SIZE_MAX
+ };
+ t->stacks[a] = (struct ctables_stack) { .nests = nest, .n = 1 };
+
+ /* There's no point in moving labels away from an axis that has no
+ labels, so avoid dealing with the special cases around that. */
+ t->label_axis[a] = a;
+ }
+
+ struct ctables_stack *stack = &t->stacks[t->summary_axis];
+ for (size_t i = 0; i < stack->n; i++)
+ {
+ struct ctables_nest *nest = &stack->nests[i];
+ if (!nest->specs[CSV_CELL].n)
+ {
+ struct ctables_summary_spec_set *ss = &nest->specs[CSV_CELL];
+ ss->specs = xmalloc (sizeof *ss->specs);
+ ss->n = 1;
+
+ enum ctables_summary_function function
+ = ss->is_scale ? CTSF_MEAN : CTSF_COUNT;
+
+ if (!ss->var)
+ {
+ nest->summary_idx = nest->n - 1;
+ ss->var = nest->vars[nest->summary_idx];
+ }
+ *ss->specs = (struct ctables_summary_spec) {
+ .function = function,
+ .weighting = ss->is_scale ? CTW_EFFECTIVE : CTW_DICTIONARY,
+ .format = ctables_summary_default_format (function, ss->var),
+ };
+
+ ctables_summary_spec_set_clone (&nest->specs[CSV_TOTAL],
+ &nest->specs[CSV_CELL]);
+ }
+ else if (!nest->specs[CSV_TOTAL].n)
+ ctables_summary_spec_set_clone (&nest->specs[CSV_TOTAL],
+ &nest->specs[CSV_CELL]);
+
+ if (t->label_axis[PIVOT_AXIS_ROW] == PIVOT_AXIS_COLUMN
+ || t->label_axis[PIVOT_AXIS_COLUMN] == PIVOT_AXIS_ROW)
+ {
+ for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
+ for (size_t i = 0; i < nest->specs[sv].n; i++)
+ {
+ struct ctables_summary_spec *ss = &nest->specs[sv].specs[i];
+ const struct ctables_function_info *cfi =
+ &ctables_function_info[ss->function];
+ if (cfi->is_area)
+ ss->calc_area = rotate_area (ss->calc_area);
+ }
+ }
+
+ if (t->ctables->smissing_listwise)
+ {
+ struct variable **listwise_vars = NULL;
+ size_t n = 0;
+ size_t allocated = 0;
+
+ for (size_t j = nest->group_head; j < stack->n; j++)
+ {
+ const struct ctables_nest *other_nest = &stack->nests[j];
+ if (other_nest->group_head != nest->group_head)
+ break;
+
+ if (nest != other_nest && other_nest->scale_idx < other_nest->n)
+ {
+ if (n >= allocated)
+ listwise_vars = x2nrealloc (listwise_vars, &allocated,
+ sizeof *listwise_vars);
+ listwise_vars[n++] = other_nest->vars[other_nest->scale_idx];
+ }
+ }
+ for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
+ {
+ if (sv > 0)
+ listwise_vars = xmemdup (listwise_vars,
+ n * sizeof *listwise_vars);
+ nest->specs[sv].listwise_vars = listwise_vars;
+ nest->specs[sv].n_listwise_vars = n;
+ }
+ }
+ }
+
+ struct ctables_summary_spec_set *merged = &t->summary_specs;
+ struct merge_item *items = xnmalloc (N_CSVS * stack->n, sizeof *items);
+ size_t n_left = 0;
+ for (size_t j = 0; j < stack->n; j++)
+ {
+ const struct ctables_nest *nest = &stack->nests[j];
+ if (nest->n)
+ for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
+ items[n_left++] = (struct merge_item) { .set = &nest->specs[sv] };
+ }
+
+ while (n_left > 0)
+ {
+ struct merge_item min = items[0];
+ for (size_t j = 1; j < n_left; j++)
+ if (merge_item_compare_3way (&items[j], &min) < 0)
+ min = items[j];
+
+ if (merged->n >= merged->allocated)
+ merged->specs = x2nrealloc (merged->specs, &merged->allocated,
+ sizeof *merged->specs);
+ merged->specs[merged->n++] = min.set->specs[min.ofs];
+
+ for (size_t j = 0; j < n_left; )
+ {
+ if (merge_item_compare_3way (&items[j], &min) == 0)
+ {
+ struct merge_item *item = &items[j];
+ item->set->specs[item->ofs++].axis_idx = merged->n - 1;
+ if (item->ofs >= item->set->n)
+ {
+ items[j] = items[--n_left];
+ continue;
+ }
+ }
+ j++;
+ }
+ }
+ free (items);
+
+ size_t allocated_sum_vars = 0;
+ enumerate_sum_vars (t->axes[t->summary_axis],
+ &t->sum_vars, &t->n_sum_vars, &allocated_sum_vars);
+
+ return (ctables_check_label_position (t, lexer, PIVOT_AXIS_ROW)
+ && ctables_check_label_position (t, lexer, PIVOT_AXIS_COLUMN));
+}
+
+static void
+ctables_insert_clabels_values (struct ctables_table *t, const struct ccase *c,
+ enum pivot_axis_type a)
+{
+ struct ctables_stack *stack = &t->stacks[a];
+ for (size_t i = 0; i < stack->n; i++)
+ {
+ const struct ctables_nest *nest = &stack->nests[i];
+ const struct variable *var = nest->vars[nest->n - 1];
+ const union value *value = case_data (c, var);
+
+ if (var_is_numeric (var) && value->f == SYSMIS)
+ continue;
+
+ if (ctables_categories_match (t->categories [var_get_dict_index (var)],
+ value, var))
+ ctables_value_insert (t, value, var_get_width (var));
+ }
+}
+
+static void
+ctables_add_category_occurrences (const struct variable *var,
+ struct hmap *occurrences,
+ const struct ctables_categories *cats)
+{
+ const struct val_labs *val_labs = var_get_value_labels (var);
+
+ for (size_t i = 0; i < cats->n_cats; i++)
+ {
+ const struct ctables_category *c = &cats->cats[i];
+ switch (c->type)
+ {
+ case CCT_NUMBER:
+ ctables_add_occurrence (var, &(const union value) { .f = c->number },
+ occurrences);
+ break;
+
+ case CCT_STRING:
+ {
+ int width = var_get_width (var);
+ union value value;
+ value_init (&value, width);
+ value_copy_buf_rpad (&value, width,
+ CHAR_CAST (uint8_t *, c->string.string),
+ c->string.length, ' ');
+ ctables_add_occurrence (var, &value, occurrences);
+ value_destroy (&value, width);
+ }
+ break;
+
+ case CCT_NRANGE:
+ assert (var_is_numeric (var));
+ for (const struct val_lab *vl = val_labs_first (val_labs); vl;
+ vl = val_labs_next (val_labs, vl))
+ if (vl->value.f >= c->nrange[0] && vl->value.f <= c->nrange[1])
+ ctables_add_occurrence (var, &vl->value, occurrences);
+ break;
+
+ case CCT_SRANGE:
+ assert (var_is_alpha (var));
+ for (const struct val_lab *vl = val_labs_first (val_labs); vl;
+ vl = val_labs_next (val_labs, vl))
+ if (in_string_range (&vl->value, var, c->srange))
+ ctables_add_occurrence (var, &vl->value, occurrences);
+ break;
+
+ case CCT_MISSING:
+ for (const struct val_lab *vl = val_labs_first (val_labs); vl;
+ vl = val_labs_next (val_labs, vl))
+ if (var_is_value_missing (var, &vl->value))
+ ctables_add_occurrence (var, &vl->value, occurrences);
+ break;
+
+ case CCT_OTHERNM:
+ for (const struct val_lab *vl = val_labs_first (val_labs); vl;
+ vl = val_labs_next (val_labs, vl))
+ ctables_add_occurrence (var, &vl->value, occurrences);
+ break;
+
+ case CCT_POSTCOMPUTE:
+ break;
+
+ case CCT_SUBTOTAL:
+ case CCT_TOTAL:
+ break;
+
+ case CCT_VALUE:
+ case CCT_LABEL:
+ case CCT_FUNCTION:
+ for (const struct val_lab *vl = val_labs_first (val_labs); vl;
+ vl = val_labs_next (val_labs, vl))
+ if (c->include_missing || !var_is_value_missing (var, &vl->value))
+ ctables_add_occurrence (var, &vl->value, occurrences);
+ break;
+
+ case CCT_EXCLUDED_MISSING:
+ break;
+ }
+ }
+}
+
+static void
+ctables_section_recurse_add_empty_categories (
+ struct ctables_section *s,
+ const struct ctables_category **cats[PIVOT_N_AXES], struct ccase *c,
+ enum pivot_axis_type a, size_t a_idx)
+{
+ if (a >= PIVOT_N_AXES)
+ ctables_cell_insert__ (s, c, cats);
+ else if (!s->nests[a] || a_idx >= s->nests[a]->n)
+ ctables_section_recurse_add_empty_categories (s, cats, c, a + 1, 0);
+ else
+ {
+ const struct variable *var = s->nests[a]->vars[a_idx];
+ const struct ctables_categories *categories = s->table->categories[
+ var_get_dict_index (var)];
+ int width = var_get_width (var);
+ const struct hmap *occurrences = &s->occurrences[a][a_idx];
+ const struct ctables_occurrence *o;
+ HMAP_FOR_EACH (o, struct ctables_occurrence, node, occurrences)
+ {
+ union value *value = case_data_rw (c, var);
+ value_destroy (value, width);
+ value_clone (value, &o->value, width);
+ cats[a][a_idx] = ctables_categories_match (categories, value, var);
+ assert (cats[a][a_idx] != NULL);
+ ctables_section_recurse_add_empty_categories (s, cats, c, a, a_idx + 1);
+ }
+
+ for (size_t i = 0; i < categories->n_cats; i++)
+ {
+ const struct ctables_category *cat = &categories->cats[i];
+ if (cat->type == CCT_POSTCOMPUTE)
+ {
+ cats[a][a_idx] = cat;
+ ctables_section_recurse_add_empty_categories (s, cats, c, a, a_idx + 1);
+ }
+ }
+ }
+}
+
+static void
+ctables_section_add_empty_categories (struct ctables_section *s)
+{
+ bool show_empty = false;
+ for (size_t a = 0; a < PIVOT_N_AXES; a++)
+ if (s->nests[a])
+ for (size_t k = 0; k < s->nests[a]->n; k++)
+ if (k != s->nests[a]->scale_idx)
+ {
+ const struct variable *var = s->nests[a]->vars[k];
+ size_t idx = var_get_dict_index (var);
+ const struct ctables_categories *cats = s->table->categories[idx];
+ if (s->table->show_empty[idx])
+ {
+ show_empty = true;
+ ctables_add_category_occurrences (var, &s->occurrences[a][k], cats);
+ }
+ }
+ if (!show_empty)
+ return;
+
+ const struct ctables_category *layer_cats[s->nests[PIVOT_AXIS_LAYER]->n];
+ const struct ctables_category *row_cats[s->nests[PIVOT_AXIS_ROW]->n];
+ const struct ctables_category *column_cats[s->nests[PIVOT_AXIS_COLUMN]->n];
+ const struct ctables_category **cats[PIVOT_N_AXES] =
+ {
+ [PIVOT_AXIS_LAYER] = layer_cats,
+ [PIVOT_AXIS_ROW] = row_cats,
+ [PIVOT_AXIS_COLUMN] = column_cats,
+ };
+ struct ccase *c = case_create (dict_get_proto (s->table->ctables->dict));
+ ctables_section_recurse_add_empty_categories (s, cats, c, 0, 0);
+ case_unref (c);
+}
+
+static void
+ctables_section_clear (struct ctables_section *s)
+{
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ {
+ const struct variable *var = nest->vars[i];
+ int width = var_get_width (var);
+ struct ctables_occurrence *o, *next;
+ struct hmap *map = &s->occurrences[a][i];
+ HMAP_FOR_EACH_SAFE (o, next, struct ctables_occurrence, node, map)
+ {
+ value_destroy (&o->value, width);
+ hmap_delete (map, &o->node);
+ free (o);
+ }
+ hmap_shrink (map);
+ }
+ }
+
+ struct ctables_cell *cell, *next_cell;
+ HMAP_FOR_EACH_SAFE (cell, next_cell, struct ctables_cell, node, &s->cells)
+ {
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ const struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ if (i != nest->scale_idx)
+ value_destroy (&cell->axes[a].cvs[i].value,
+ var_get_width (nest->vars[i]));
+ free (cell->axes[a].cvs);
+ }
+
+ const struct ctables_nest *ss = s->nests[s->table->summary_axis];
+ const struct ctables_summary_spec_set *specs = &ss->specs[cell->sv];
+ for (size_t i = 0; i < specs->n; i++)
+ ctables_summary_uninit (&cell->summaries[i], &specs->specs[i]);
+ free (cell->summaries);
+
+ hmap_delete (&s->cells, &cell->node);
+ free (cell);
+ }
+ hmap_shrink (&s->cells);
+
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ {
+ struct ctables_area *area, *next_area;
+ HMAP_FOR_EACH_SAFE (area, next_area, struct ctables_area, node,
+ &s->areas[at])
+ {
+ free (area->sums);
+ hmap_delete (&s->areas[at], &area->node);
+ free (area);
+ }
+ hmap_shrink (&s->areas[at]);
+ }
+}
+
+static void
+ctables_section_uninit (struct ctables_section *s)
+{
+ ctables_section_clear (s);
+
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ struct ctables_nest *nest = s->nests[a];
+ for (size_t i = 0; i < nest->n; i++)
+ hmap_destroy (&s->occurrences[a][i]);
+ free (s->occurrences[a]);
+ }
+
+ hmap_destroy (&s->cells);
+ for (enum ctables_area_type at = 0; at < N_CTATS; at++)
+ hmap_destroy (&s->areas[at]);
+}
+
+static void
+ctables_table_clear (struct ctables_table *t)
+{
+ for (size_t i = 0; i < t->n_sections; i++)
+ ctables_section_clear (&t->sections[i]);
+
+ if (t->clabels_example)
+ {
+ int width = var_get_width (t->clabels_example);
+ struct ctables_value *value, *next_value;
+ HMAP_FOR_EACH_SAFE (value, next_value, struct ctables_value, node,
+ &t->clabels_values_map)
+ {
+ value_destroy (&value->value, width);
+ hmap_delete (&t->clabels_values_map, &value->node);
+ free (value);
+ }
+ hmap_shrink (&t->clabels_values_map);
+
+ free (t->clabels_values);
+ t->clabels_values = NULL;
+ t->n_clabels_values = 0;
+ }
+}
+
+static bool
+ctables_execute (struct dataset *ds, struct casereader *input,
+ struct ctables *ct)
+{
+ for (size_t i = 0; i < ct->n_tables; i++)
+ {
+ struct ctables_table *t = ct->tables[i];
+ t->sections = xnmalloc (MAX (1, t->stacks[PIVOT_AXIS_ROW].n) *
+ MAX (1, t->stacks[PIVOT_AXIS_COLUMN].n) *
+ MAX (1, t->stacks[PIVOT_AXIS_LAYER].n),
+ sizeof *t->sections);
+ size_t ix[PIVOT_N_AXES];
+ ctables_table_add_section (t, 0, ix);
+ }
+
+ struct dictionary *dict = dataset_dict (ds);
+
+ bool splitting = dict_get_split_type (dict) == SPLIT_SEPARATE;
+ struct casegrouper *grouper
+ = (splitting
+ ? casegrouper_create_splits (input, dict)
+ : casegrouper_create_vars (input, NULL, 0));
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ if (splitting)
+ output_split_file_values_peek (ds, group);
+
+ bool warn_on_invalid = true;
+ for (struct ccase *c = casereader_read (group); c;
+ case_unref (c), c = casereader_read (group))
+ {
+ double d_weight = dict_get_rounded_case_weight (dict, c, &warn_on_invalid);
+ double e_weight = (ct->e_weight
+ ? var_force_valid_weight (ct->e_weight,
+ case_num (c, ct->e_weight),
+ &warn_on_invalid)
+ : d_weight);
+ double weight[] = {
+ [CTW_DICTIONARY] = d_weight,
+ [CTW_EFFECTIVE] = e_weight,
+ [CTW_UNWEIGHTED] = 1.0,
+ };
+
+ for (size_t i = 0; i < ct->n_tables; i++)
+ {
+ struct ctables_table *t = ct->tables[i];
+
+ for (size_t j = 0; j < t->n_sections; j++)
+ ctables_cell_insert (&t->sections[j], c, weight);
+
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ if (t->label_axis[a] != a)
+ ctables_insert_clabels_values (t, c, a);
+ }
+ }
+ casereader_destroy (group);
+
+ for (size_t i = 0; i < ct->n_tables; i++)
+ {
+ struct ctables_table *t = ct->tables[i];
+
+ if (t->clabels_example)
+ ctables_sort_clabels_values (t);
+
+ for (size_t j = 0; j < t->n_sections; j++)
+ ctables_section_add_empty_categories (&t->sections[j]);
+
+ ctables_table_output (ct, t);
+ ctables_table_clear (t);
+ }
+ }
+ return casegrouper_destroy (grouper);
+}
+\f
+static struct ctables_postcompute *
+ctables_find_postcompute (struct ctables *ct, const char *name)
+{
+ struct ctables_postcompute *pc;
+ HMAP_FOR_EACH_WITH_HASH (pc, struct ctables_postcompute, hmap_node,
+ utf8_hash_case_string (name, 0), &ct->postcomputes)
+ if (!utf8_strcasecmp (pc->name, name))
+ return pc;
+ return NULL;
+}
+
+static bool
+ctables_parse_pcompute (struct lexer *lexer, struct dictionary *dict,
+ struct ctables *ct)
+{
+ int pcompute_start = lex_ofs (lexer) - 1;
+
+ if (!lex_match (lexer, T_AND))
+ {
+ lex_error_expecting (lexer, "&");
+ return false;
+ }
+ if (!lex_force_id (lexer))
+ return false;
+
+ char *name = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ if (!lex_force_match_phrase (lexer, "=EXPR("))
+ {
+ free (name);
+ return false;
+ }
+
+ int expr_start = lex_ofs (lexer);
+ struct ctables_pcexpr *expr = ctables_pcexpr_parse_add (lexer, dict);
+ int expr_end = lex_ofs (lexer) - 1;
+ if (!expr || !lex_force_match (lexer, T_RPAREN))
+ {
+ ctables_pcexpr_destroy (expr);
+ free (name);
+ return false;
+ }
+ int pcompute_end = lex_ofs (lexer) - 1;
+
+ struct msg_location *location = lex_ofs_location (lexer, pcompute_start,
+ pcompute_end);
+
+ struct ctables_postcompute *pc = ctables_find_postcompute (ct, name);
+ if (pc)
+ {
+ msg_at (SW, location, _("New definition of &%s will override the "
+ "previous definition."),
+ pc->name);
+ msg_at (SN, pc->location, _("This is the previous definition."));
+
+ ctables_pcexpr_destroy (pc->expr);
+ msg_location_destroy (pc->location);
+ free (name);
+ }
+ else
+ {
+ pc = xmalloc (sizeof *pc);
+ *pc = (struct ctables_postcompute) { .name = name };
+ hmap_insert (&ct->postcomputes, &pc->hmap_node,
+ utf8_hash_case_string (pc->name, 0));
+ }
+ pc->expr = expr;
+ pc->location = location;
+ if (!pc->label)
+ pc->label = lex_ofs_representation (lexer, expr_start, expr_end);
+ return true;
+}
+
+static bool
+ctables_parse_pproperties_format (struct lexer *lexer,
+ struct ctables_summary_spec_set *sss)
+{
+ *sss = (struct ctables_summary_spec_set) { .n = 0 };
+
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH
+ && !(lex_token (lexer) == T_ID
+ && (lex_id_match (ss_cstr ("LABEL"), lex_tokss (lexer))
+ || lex_id_match (ss_cstr ("HIDESOURCECATS"),
+ lex_tokss (lexer)))))
+ {
+ /* Parse function. */
+ enum ctables_summary_function function;
+ enum ctables_weighting weighting;
+ enum ctables_area_type area;
+ if (!parse_ctables_summary_function (lexer, &function, &weighting, &area))
+ goto error;
+
+ /* Parse percentile. */
+ double percentile = 0;
+ if (function == CTSF_PTILE)
+ {
+ if (!lex_force_num_range_closed (lexer, "PTILE", 0, 100))
+ goto error;
+ percentile = lex_number (lexer);
+ lex_get (lexer);
+ }
+
+ /* Parse format. */
+ struct fmt_spec format;
+ bool is_ctables_format;
+ if (!parse_ctables_format_specifier (lexer, &format, &is_ctables_format))
+ goto error;
+
+ if (sss->n >= sss->allocated)
+ sss->specs = x2nrealloc (sss->specs, &sss->allocated,
+ sizeof *sss->specs);
+ sss->specs[sss->n++] = (struct ctables_summary_spec) {
+ .function = function,
+ .weighting = weighting,
+ .calc_area = area,
+ .user_area = area,
+ .percentile = percentile,
+ .format = format,
+ .is_ctables_format = is_ctables_format,
+ };
+ }
+ return true;
+
+error:
+ ctables_summary_spec_set_uninit (sss);
+ return false;
+}
+
+static bool
+ctables_parse_pproperties (struct lexer *lexer, struct ctables *ct)
+{
+ struct ctables_postcompute **pcs = NULL;
+ size_t n_pcs = 0;
+ size_t allocated_pcs = 0;
+
+ while (lex_match (lexer, T_AND))
+ {
+ if (!lex_force_id (lexer))
+ goto error;
+ struct ctables_postcompute *pc
+ = ctables_find_postcompute (ct, lex_tokcstr (lexer));
+ if (!pc)
+ {
+ lex_error (lexer, _("Unknown computed category &%s."),
+ lex_tokcstr (lexer));
+ goto error;
+ }
+ lex_get (lexer);
+
+ if (n_pcs >= allocated_pcs)
+ pcs = x2nrealloc (pcs, &allocated_pcs, sizeof *pcs);
+ pcs[n_pcs++] = pc;
+ }
+
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "LABEL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ for (size_t i = 0; i < n_pcs; i++)
+ {
+ free (pcs[i]->label);
+ pcs[i]->label = ss_xstrdup (lex_tokss (lexer));
+ }
+
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ struct ctables_summary_spec_set sss;
+ if (!ctables_parse_pproperties_format (lexer, &sss))
+ goto error;
+
+ for (size_t i = 0; i < n_pcs; i++)
+ {
+ if (pcs[i]->specs)
+ ctables_summary_spec_set_uninit (pcs[i]->specs);
+ else
+ pcs[i]->specs = xmalloc (sizeof *pcs[i]->specs);
+ ctables_summary_spec_set_clone (pcs[i]->specs, &sss);
+ }
+ ctables_summary_spec_set_uninit (&sss);
+ }
+ else if (lex_match_id (lexer, "HIDESOURCECATS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ bool hide_source_cats;
+ if (!parse_bool (lexer, &hide_source_cats))
+ goto error;
+ for (size_t i = 0; i < n_pcs; i++)
+ pcs[i]->hide_source_cats = hide_source_cats;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "LABEL", "FORMAT", "HIDESOURCECATS");
+ goto error;
+ }
+ }
+ free (pcs);
+ return true;
+
+error:
+ free (pcs);
+ return false;
+}
+
+static void
+put_strftime (struct string *out, time_t now, const char *format)
+{
+ const struct tm *tm = localtime (&now);
+ char value[128];
+ strftime (value, sizeof value, format, tm);
+ ds_put_cstr (out, value);
+}
+
+static bool
+skip_prefix (struct substring *s, struct substring prefix)
+{
+ if (ss_starts_with (*s, prefix))
+ {
+ ss_advance (s, prefix.length);
+ return true;
+ }
+ else
+ return false;
+}
+
+static void
+put_table_expression (struct string *out, struct lexer *lexer,
+ struct dictionary *dict, int expr_start, int expr_end)
+{
+ size_t nest = 0;
+ for (int ofs = expr_start; ofs < expr_end; ofs++)
+ {
+ const struct token *t = lex_ofs_token (lexer, ofs);
+ if (t->type == T_LBRACK)
+ nest++;
+ else if (t->type == T_RBRACK && nest > 0)
+ nest--;
+ else if (nest > 0)
+ {
+ /* Nothing. */
+ }
+ else if (t->type == T_ID)
+ {
+ const struct variable *var
+ = dict_lookup_var (dict, t->string.string);
+ const char *label = var ? var_get_label (var) : NULL;
+ ds_put_cstr (out, label ? label : t->string.string);
+ }
+ else
+ {
+ if (ofs != expr_start && t->type != T_RPAREN && ds_last (out) != ' ')
+ ds_put_byte (out, ' ');
+
+ char *repr = lex_ofs_representation (lexer, ofs, ofs);
+ ds_put_cstr (out, repr);
+ free (repr);
+
+ if (ofs + 1 != expr_end && t->type != T_LPAREN)
+ ds_put_byte (out, ' ');
+ }
+ }
+}
+
+static void
+put_title_text (struct string *out, struct substring in, time_t now,
+ struct lexer *lexer, struct dictionary *dict,
+ int expr_start, int expr_end)
+{
+ for (;;)
+ {
+ size_t chunk = ss_find_byte (in, ')');
+ ds_put_substring (out, ss_head (in, chunk));
+ ss_advance (&in, chunk);
+ if (ss_is_empty (in))
+ return;
+
+ if (skip_prefix (&in, ss_cstr (")DATE")))
+ put_strftime (out, now, "%x");
+ else if (skip_prefix (&in, ss_cstr (")TIME")))
+ put_strftime (out, now, "%X");
+ else if (skip_prefix (&in, ss_cstr (")TABLE")))
+ put_table_expression (out, lexer, dict, expr_start, expr_end);
+ else
+ {
+ ds_put_byte (out, ')');
+ ss_advance (&in, 1);
+ }
+ }
+}
+
+int
+cmd_ctables (struct lexer *lexer, struct dataset *ds)
+{
+ struct casereader *input = NULL;
+
+ struct measure_guesser *mg = measure_guesser_create (ds);
+ if (mg)
+ {
+ input = proc_open (ds);
+ measure_guesser_run (mg, input);
+ measure_guesser_destroy (mg);
+ }
+
+ size_t n_vars = dict_get_n_vars (dataset_dict (ds));
+ enum ctables_vlabel *vlabels = xnmalloc (n_vars, sizeof *vlabels);
+ enum settings_value_show tvars = settings_get_show_variables ();
+ for (size_t i = 0; i < n_vars; i++)
+ vlabels[i] = (enum ctables_vlabel) tvars;
+
+ struct pivot_table_look *look = pivot_table_look_unshare (
+ pivot_table_look_ref (pivot_table_look_get_default ()));
+ look->omit_empty = false;
+
+ struct ctables *ct = xmalloc (sizeof *ct);
+ *ct = (struct ctables) {
+ .dict = dataset_dict (ds),
+ .look = look,
+ .ctables_formats = FMT_SETTINGS_INIT,
+ .vlabels = vlabels,
+ .postcomputes = HMAP_INITIALIZER (ct->postcomputes),
+ };
+
+ time_t now = time (NULL);
+
+ struct ctf
+ {
+ enum fmt_type type;
+ const char *dot_string;
+ const char *comma_string;
+ };
+ static const struct ctf ctfs[4] = {
+ { CTEF_NEGPAREN, "(,,,)", "(...)" },
+ { CTEF_NEQUAL, "-,N=,,", "-.N=.." },
+ { CTEF_PAREN, "-,(,),", "-.(.)." },
+ { CTEF_PCTPAREN, "-,(,%),", "-.(.%)." },
+ };
+ bool is_dot = settings_get_fmt_settings ()->decimal == '.';
+ for (size_t i = 0; i < 4; i++)
+ {
+ const char *s = is_dot ? ctfs[i].dot_string : ctfs[i].comma_string;
+ fmt_settings_set_cc (&ct->ctables_formats, ctfs[i].type,
+ fmt_number_style_from_string (s));
+ }
+
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+
+ while (!lex_match_id (lexer, "TABLE"))
+ {
+ if (lex_match_id (lexer, "FORMAT"))
+ {
+ double widths[2] = { SYSMIS, SYSMIS };
+ double units_per_inch = 72.0;
+
+ int start_ofs = lex_ofs (lexer);
+ while (lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "MINCOLWIDTH"))
+ {
+ if (!parse_col_width (lexer, "MINCOLWIDTH", &widths[0]))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "MAXCOLWIDTH"))
+ {
+ if (!parse_col_width (lexer, "MAXCOLWIDTH", &widths[1]))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "UNITS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "POINTS"))
+ units_per_inch = 72.0;
+ else if (lex_match_id (lexer, "INCHES"))
+ units_per_inch = 1.0;
+ else if (lex_match_id (lexer, "CM"))
+ units_per_inch = 2.54;
+ else
+ {
+ lex_error_expecting (lexer, "POINTS", "INCHES", "CM");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "EMPTY"))
+ {
+ free (ct->zero);
+ ct->zero = NULL;
+
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "ZERO"))
+ {
+ /* Nothing to do. */
+ }
+ else if (lex_match_id (lexer, "BLANK"))
+ ct->zero = xstrdup ("");
+ else if (lex_force_string (lexer))
+ {
+ ct->zero = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else
+ goto error;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (ct->missing);
+ ct->missing = (strcmp (lex_tokcstr (lexer), ".")
+ ? ss_xstrdup (lex_tokss (lexer))
+ : NULL);
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MINCOLWIDTH", "MAXCOLWIDTH",
+ "UNITS", "EMPTY", "MISSING");
+ goto error;
+ }
+ }
+
+ if (widths[0] != SYSMIS && widths[1] != SYSMIS
+ && widths[0] > widths[1])
+ {
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("MINCOLWIDTH must not be greater than "
+ "MAXCOLWIDTH."));
+ goto error;
+ }
+
+ for (size_t i = 0; i < 2; i++)
+ if (widths[i] != SYSMIS)
+ {
+ int *wr = ct->look->width_ranges[TABLE_HORZ];
+ wr[i] = widths[i] / units_per_inch * 96.0;
+ if (wr[0] > wr[1])
+ wr[!i] = wr[i];
+ }
+ }
+ else if (lex_match_id (lexer, "VLABELS"))
+ {
+ if (!lex_force_match_id (lexer, "VARIABLES"))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+
+ struct variable **vars;
+ size_t n_vars;
+ if (!parse_variables (lexer, dataset_dict (ds), &vars, &n_vars,
+ PV_NO_SCRATCH))
+ goto error;
+
+ if (!lex_force_match_id (lexer, "DISPLAY"))
+ {
+ free (vars);
+ goto error;
+ }
+ lex_match (lexer, T_EQUALS);
+
+ enum ctables_vlabel vlabel;
+ if (lex_match_id (lexer, "DEFAULT"))
+ vlabel = (enum ctables_vlabel) settings_get_show_variables ();
+ else if (lex_match_id (lexer, "NAME"))
+ vlabel = CTVL_NAME;
+ else if (lex_match_id (lexer, "LABEL"))
+ vlabel = CTVL_LABEL;
+ else if (lex_match_id (lexer, "BOTH"))
+ vlabel = CTVL_BOTH;
+ else if (lex_match_id (lexer, "NONE"))
+ vlabel = CTVL_NONE;
+ else
+ {
+ lex_error_expecting (lexer, "DEFAULT", "NAME", "LABEL",
+ "BOTH", "NONE");
+ free (vars);
+ goto error;
+ }
+
+ for (size_t i = 0; i < n_vars; i++)
+ ct->vlabels[var_get_dict_index (vars[i])] = vlabel;
+ free (vars);
+ }
+ else if (lex_match_id (lexer, "MRSETS"))
+ {
+ if (!lex_force_match_id (lexer, "COUNTDUPLICATES"))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &ct->mrsets_count_duplicates))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "SMISSING"))
+ {
+ if (lex_match_id (lexer, "VARIABLE"))
+ ct->smissing_listwise = false;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ ct->smissing_listwise = true;
+ else
+ {
+ lex_error_expecting (lexer, "VARIABLE", "LISTWISE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "PCOMPUTE"))
+ {
+ if (!ctables_parse_pcompute (lexer, dataset_dict (ds), ct))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "PPROPERTIES"))
+ {
+ if (!ctables_parse_pproperties (lexer, ct))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "WEIGHT"))
+ {
+ if (!lex_force_match_id (lexer, "VARIABLE"))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+ ct->e_weight = parse_variable (lexer, dataset_dict (ds));
+ if (!ct->e_weight)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "HIDESMALLCOUNTS"))
+ {
+ if (lex_match_id (lexer, "COUNT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "HIDESMALLCOUNTS COUNT",
+ 2, INT_MAX))
+ goto error;
+ ct->hide_threshold = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (ct->hide_threshold == 0)
+ ct->hide_threshold = 5;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "FORMAT", "VLABELS", "MRSETS",
+ "SMISSING", "PCOMPUTE", "PPROPERTIES",
+ "WEIGHT", "HIDESMALLCOUNTS", "TABLE");
+ if (lex_match_id (lexer, "SLABELS")
+ || lex_match_id (lexer, "CLABELS")
+ || lex_match_id (lexer, "CRITERIA")
+ || lex_match_id (lexer, "CATEGORIES")
+ || lex_match_id (lexer, "TITLES")
+ || lex_match_id (lexer, "SIGTEST")
+ || lex_match_id (lexer, "COMPARETEST"))
+ lex_next_msg (lexer, SN, -1, -1,
+ _("TABLE must appear before this subcommand."));
+ goto error;
+ }
+
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+ }
+
+ size_t allocated_tables = 0;
+ do
+ {
+ if (ct->n_tables >= allocated_tables)
+ ct->tables = x2nrealloc (ct->tables, &allocated_tables,
+ sizeof *ct->tables);
+
+ struct ctables_category *cat = xmalloc (sizeof *cat);
+ *cat = (struct ctables_category) {
+ .type = CCT_VALUE,
+ .include_missing = false,
+ .sort_ascending = true,
+ };
+
+ struct ctables_categories *c = xmalloc (sizeof *c);
+ size_t n_vars = dict_get_n_vars (dataset_dict (ds));
+ *c = (struct ctables_categories) {
+ .n_refs = n_vars,
+ .cats = cat,
+ .n_cats = 1,
+ };
+
+ struct ctables_categories **categories = xnmalloc (n_vars,
+ sizeof *categories);
+ for (size_t i = 0; i < n_vars; i++)
+ categories[i] = c;
+
+ bool *show_empty = xmalloc (n_vars);
+ memset (show_empty, true, n_vars);
+
+ struct ctables_table *t = xmalloc (sizeof *t);
+ *t = (struct ctables_table) {
+ .ctables = ct,
+ .slabels_axis = PIVOT_AXIS_COLUMN,
+ .slabels_visible = true,
+ .clabels_values_map = HMAP_INITIALIZER (t->clabels_values_map),
+ .label_axis = {
+ [PIVOT_AXIS_ROW] = PIVOT_AXIS_ROW,
+ [PIVOT_AXIS_COLUMN] = PIVOT_AXIS_COLUMN,
+ [PIVOT_AXIS_LAYER] = PIVOT_AXIS_LAYER,
+ },
+ .clabels_from_axis = PIVOT_AXIS_LAYER,
+ .clabels_to_axis = PIVOT_AXIS_LAYER,
+ .categories = categories,
+ .n_categories = n_vars,
+ .show_empty = show_empty,
+ .cilevel = 95,
+ };
+ ct->tables[ct->n_tables++] = t;
+
+ lex_match (lexer, T_EQUALS);
+ int expr_start = lex_ofs (lexer);
+ if (!ctables_axis_parse (lexer, dataset_dict (ds),
+ &t->axes[PIVOT_AXIS_ROW]))
+ goto error;
+ if (lex_match (lexer, T_BY))
+ {
+ if (!ctables_axis_parse (lexer, dataset_dict (ds),
+ &t->axes[PIVOT_AXIS_COLUMN]))
+ goto error;
+
+ if (lex_match (lexer, T_BY))
+ {
+ if (!ctables_axis_parse (lexer, dataset_dict (ds),
+ &t->axes[PIVOT_AXIS_LAYER]))
+ goto error;
+ }
+ }
+ int expr_end = lex_ofs (lexer);
+
+ if (!t->axes[PIVOT_AXIS_ROW] && !t->axes[PIVOT_AXIS_COLUMN]
+ && !t->axes[PIVOT_AXIS_LAYER])
+ {
+ lex_error (lexer, _("At least one variable must be specified."));
+ goto error;
+ }
+
+ const struct ctables_axis *scales[PIVOT_N_AXES];
+ size_t n_scales = 0;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ scales[a] = find_scale (t->axes[a]);
+ if (scales[a])
+ n_scales++;
+ }
+ if (n_scales > 1)
+ {
+ msg (SE, _("Scale variables may appear only on one axis."));
+ if (scales[PIVOT_AXIS_ROW])
+ msg_at (SN, scales[PIVOT_AXIS_ROW]->loc,
+ _("This scale variable appears on the rows axis."));
+ if (scales[PIVOT_AXIS_COLUMN])
+ msg_at (SN, scales[PIVOT_AXIS_COLUMN]->loc,
+ _("This scale variable appears on the columns axis."));
+ if (scales[PIVOT_AXIS_LAYER])
+ msg_at (SN, scales[PIVOT_AXIS_LAYER]->loc,
+ _("This scale variable appears on the layer axis."));
+ goto error;
+ }
+
+ const struct ctables_axis *summaries[PIVOT_N_AXES];
+ size_t n_summaries = 0;
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ {
+ summaries[a] = (scales[a]
+ ? scales[a]
+ : find_categorical_summary_spec (t->axes[a]));
+ if (summaries[a])
+ n_summaries++;
+ }
+ if (n_summaries > 1)
+ {
+ msg (SE, _("Summaries may appear only on one axis."));
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ if (summaries[a])
+ {
+ msg_at (SN, summaries[a]->loc,
+ a == PIVOT_AXIS_ROW
+ ? _("This variable on the rows axis has a summary.")
+ : a == PIVOT_AXIS_COLUMN
+ ? _("This variable on the columns axis has a summary.")
+ : _("This variable on the layers axis has a summary."));
+ if (scales[a])
+ msg_at (SN, summaries[a]->loc,
+ _("This is a scale variable, so it always has a "
+ "summary even if the syntax does not explicitly "
+ "specify one."));
+ }
+ goto error;
+ }
+ for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
+ if (n_summaries ? summaries[a] : t->axes[a])
+ {
+ t->summary_axis = a;
+ break;
+ }
+
+ if (lex_token (lexer) == T_ENDCMD)
+ {
+ if (!ctables_prepare_table (t, lexer))
+ goto error;
+ break;
+ }
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+
+ while (!lex_match_id (lexer, "TABLE") && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "SLABELS"))
+ {
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "POSITION"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "COLUMN"))
+ t->slabels_axis = PIVOT_AXIS_COLUMN;
+ else if (lex_match_id (lexer, "ROW"))
+ t->slabels_axis = PIVOT_AXIS_ROW;
+ else if (lex_match_id (lexer, "LAYER"))
+ t->slabels_axis = PIVOT_AXIS_LAYER;
+ else
+ {
+ lex_error_expecting (lexer, "COLUMN", "ROW", "LAYER");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "VISIBLE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &t->slabels_visible))
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "POSITION", "VISIBLE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "CLABELS"))
+ {
+ int start_ofs = lex_ofs (lexer) - 1;
+ if (lex_match_id (lexer, "AUTO"))
+ {
+ t->label_axis[PIVOT_AXIS_ROW] = PIVOT_AXIS_ROW;
+ t->label_axis[PIVOT_AXIS_COLUMN] = PIVOT_AXIS_COLUMN;
+ }
+ else if (lex_match_id (lexer, "ROWLABELS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "OPPOSITE"))
+ t->label_axis[PIVOT_AXIS_ROW] = PIVOT_AXIS_COLUMN;
+ else if (lex_match_id (lexer, "LAYER"))
+ t->label_axis[PIVOT_AXIS_ROW] = PIVOT_AXIS_LAYER;
+ else
+ {
+ lex_error_expecting (lexer, "OPPOSITE", "LAYER");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "COLLABELS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "OPPOSITE"))
+ t->label_axis[PIVOT_AXIS_COLUMN] = PIVOT_AXIS_ROW;
+ else if (lex_match_id (lexer, "LAYER"))
+ t->label_axis[PIVOT_AXIS_COLUMN] = PIVOT_AXIS_LAYER;
+ else
+ {
+ lex_error_expecting (lexer, "OPPOSITE", "LAYER");
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "AUTO", "ROWLABELS",
+ "COLLABELS");
+ goto error;
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ if (t->label_axis[PIVOT_AXIS_ROW] != PIVOT_AXIS_ROW
+ && t->label_axis[PIVOT_AXIS_COLUMN] != PIVOT_AXIS_COLUMN)
+ {
+ msg (SE, _("ROWLABELS and COLLABELS may not both be "
+ "specified."));
+
+ lex_ofs_msg (lexer, SN, t->clabels_start_ofs,
+ t->clabels_end_ofs,
+ _("This is the first specification."));
+ lex_ofs_msg (lexer, SN, start_ofs, end_ofs,
+ _("This is the second specification."));
+ goto error;
+ }
+
+ t->clabels_start_ofs = start_ofs;
+ t->clabels_end_ofs = end_ofs;
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ if (!lex_force_match_id (lexer, "CILEVEL"))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_num_range_halfopen (lexer, "CILEVEL", 0, 100))
+ goto error;
+ t->cilevel = lex_number (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "CATEGORIES"))
+ {
+ if (!ctables_table_parse_categories (lexer, dataset_dict (ds),
+ ct, t))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "TITLES"))
+ {
+ do
+ {
+ char **textp;
+ if (lex_match_id (lexer, "CAPTIONS"))
+ textp = &t->caption;
+ else if (lex_match_id (lexer, "CORNERS"))
+ textp = &t->corner;
+ else if (lex_match_id (lexer, "TITLES"))
+ textp = &t->title;
+ else
+ {
+ lex_error_expecting (lexer, "CAPTION", "CORNER", "TITLE");
+ goto error;
+ }
+ lex_match (lexer, T_EQUALS);
+
+ struct string s = DS_EMPTY_INITIALIZER;
+ while (lex_is_string (lexer))
+ {
+ if (!ds_is_empty (&s))
+ ds_put_byte (&s, '\n');
+ put_title_text (&s, lex_tokss (lexer), now,
+ lexer, dataset_dict (ds),
+ expr_start, expr_end);
+ lex_get (lexer);
+ }
+ free (*textp);
+ *textp = ds_steal_cstr (&s);
+ }
+ while (lex_token (lexer) != T_SLASH
+ && lex_token (lexer) != T_ENDCMD);
+ }
+ else if (lex_match_id (lexer, "SIGTEST"))
+ {
+ int start_ofs = lex_ofs (lexer) - 1;
+ if (!t->chisq)
+ {
+ t->chisq = xmalloc (sizeof *t->chisq);
+ *t->chisq = (struct ctables_chisq) {
+ .alpha = .05,
+ .include_mrsets = true,
+ .all_visible = true,
+ };
+ }
+
+ do
+ {
+ if (lex_match_id (lexer, "TYPE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_match_id (lexer, "CHISQUARE"))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ALPHA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_num_range_halfopen (lexer, "ALPHA", 0, 1))
+ goto error;
+ t->chisq->alpha = lex_number (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "INCLUDEMRSETS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &t->chisq->include_mrsets))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "CATEGORIES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "ALLVISIBLE"))
+ t->chisq->all_visible = true;
+ else if (lex_match_id (lexer, "SUBTOTALS"))
+ t->chisq->all_visible = false;
+ else
+ {
+ lex_error_expecting (lexer,
+ "ALLVISIBLE", "SUBTOTALS");
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "TYPE", "ALPHA",
+ "INCLUDEMRSETS", "CATEGORIES");
+ goto error;
+ }
+ }
+ while (lex_token (lexer) != T_SLASH
+ && lex_token (lexer) != T_ENDCMD);
+
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("Support for SIGTEST not yet implemented."));
+ goto error;
+ }
+ else if (lex_match_id (lexer, "COMPARETEST"))
+ {
+ int start_ofs = lex_ofs (lexer) - 1;
+ if (!t->pairwise)
+ {
+ t->pairwise = xmalloc (sizeof *t->pairwise);
+ *t->pairwise = (struct ctables_pairwise) {
+ .type = PROP,
+ .alpha = { .05, .05 },
+ .adjust = BONFERRONI,
+ .include_mrsets = true,
+ .meansvariance_allcats = true,
+ .all_visible = true,
+ .merge = false,
+ .apa_style = true,
+ .show_sig = false,
+ };
+ }
+
+ do
+ {
+ if (lex_match_id (lexer, "TYPE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "PROP"))
+ t->pairwise->type = PROP;
+ else if (lex_match_id (lexer, "MEAN"))
+ t->pairwise->type = MEAN;
+ else
+ {
+ lex_error_expecting (lexer, "PROP", "MEAN");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "ALPHA"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_num_range_open (lexer, "ALPHA", 0, 1))
+ goto error;
+ double a0 = lex_number (lexer);
+ lex_get (lexer);
+
+ lex_match (lexer, T_COMMA);
+ if (lex_is_number (lexer))
+ {
+ if (!lex_force_num_range_open (lexer, "ALPHA", 0, 1))
+ goto error;
+ double a1 = lex_number (lexer);
+ lex_get (lexer);
+
+ t->pairwise->alpha[0] = MIN (a0, a1);
+ t->pairwise->alpha[1] = MAX (a0, a1);
+ }
+ else
+ t->pairwise->alpha[0] = t->pairwise->alpha[1] = a0;
+ }
+ else if (lex_match_id (lexer, "ADJUST"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "BONFERRONI"))
+ t->pairwise->adjust = BONFERRONI;
+ else if (lex_match_id (lexer, "BH"))
+ t->pairwise->adjust = BH;
+ else if (lex_match_id (lexer, "NONE"))
+ t->pairwise->adjust = 0;
+ else
+ {
+ lex_error_expecting (lexer, "BONFERRONI", "BH",
+ "NONE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "INCLUDEMRSETS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &t->pairwise->include_mrsets))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "MEANSVARIANCE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "ALLCATS"))
+ t->pairwise->meansvariance_allcats = true;
+ else if (lex_match_id (lexer, "TESTEDCATS"))
+ t->pairwise->meansvariance_allcats = false;
+ else
+ {
+ lex_error_expecting (lexer, "ALLCATS", "TESTEDCATS");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "CATEGORIES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "ALLVISIBLE"))
+ t->pairwise->all_visible = true;
+ else if (lex_match_id (lexer, "SUBTOTALS"))
+ t->pairwise->all_visible = false;
+ else
+ {
+ lex_error_expecting (lexer, "ALLVISIBLE",
+ "SUBTOTALS");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "MERGE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &t->pairwise->merge))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "STYLE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "APA"))
+ t->pairwise->apa_style = true;
+ else if (lex_match_id (lexer, "SIMPLE"))
+ t->pairwise->apa_style = false;
+ else
+ {
+ lex_error_expecting (lexer, "APA", "SIMPLE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "SHOWSIG"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_bool (lexer, &t->pairwise->show_sig))
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "TYPE", "ALPHA", "ADJUST",
+ "INCLUDEMRSETS", "MEANSVARIANCE",
+ "CATEGORIES", "MERGE", "STYLE",
+ "SHOWSIG");
+ goto error;
+ }
+ }
+ while (lex_token (lexer) != T_SLASH
+ && lex_token (lexer) != T_ENDCMD);
+
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("Support for COMPARETEST not yet implemented."));
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "TABLE", "SLABELS", "CLABELS",
+ "CRITERIA", "CATEGORIES", "TITLES",
+ "SIGTEST", "COMPARETEST");
+ if (lex_match_id (lexer, "FORMAT")
+ || lex_match_id (lexer, "VLABELS")
+ || lex_match_id (lexer, "MRSETS")
+ || lex_match_id (lexer, "SMISSING")
+ || lex_match_id (lexer, "PCOMPUTE")
+ || lex_match_id (lexer, "PPROPERTIES")
+ || lex_match_id (lexer, "WEIGHT")
+ || lex_match_id (lexer, "HIDESMALLCOUNTS"))
+ lex_next_msg (lexer, SN, -1, -1,
+ _("This subcommand must appear before TABLE."));
+ goto error;
+ }
+
+ if (!lex_match (lexer, T_SLASH))
+ break;
+ }
+
+ if (t->label_axis[PIVOT_AXIS_ROW] != PIVOT_AXIS_ROW)
+ t->clabels_from_axis = PIVOT_AXIS_ROW;
+ else if (t->label_axis[PIVOT_AXIS_COLUMN] != PIVOT_AXIS_COLUMN)
+ t->clabels_from_axis = PIVOT_AXIS_COLUMN;
+ t->clabels_to_axis = t->label_axis[t->clabels_from_axis];
+
+ if (!ctables_prepare_table (t, lexer))
+ goto error;
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ if (!input)
+ input = proc_open (ds);
+ bool ok = ctables_execute (ds, input, ct);
+ ok = proc_commit (ds) && ok;
+
+ ctables_destroy (ct);
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+
+error:
+ if (input)
+ proc_commit (ds);
+ ctables_destroy (ct);
+ return CMD_FAILURE;
+}
+
--- /dev/null
+/* -*- c -*- */
+
+/* Summary functions for all variables. */
+S(CTSF_COUNT, "COUNT", CTFT_UECELL, CTF_COUNT, CTFA_ALL)
+S(CTSF_areaPCT_COUNT, "PCT.COUNT", CTFT_AREA, CTF_PERCENT, CTFA_ALL)
+S(CTSF_areaPCT_VALIDN, "PCT.VALIDN", CTFT_AREA, CTF_PERCENT, CTFA_ALL)
+S(CTSF_areaPCT_TOTALN, "PCT.TOTALN", CTFT_AREA, CTF_PERCENT, CTFA_ALL)
+
+/* Scale variables, totals, and subtotals. */
+S(CTSF_MAXIMUM, "MAXIMUM", CTFT_CELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_MEAN, "MEAN", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_MEDIAN, "MEDIAN", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_MINIMUM, "MINIMUM", CTFT_CELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_MISSING, "MISSING", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_MODE, "MODE", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_PTILE, "PTILE", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_RANGE, "RANGE", CTFT_CELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_SEMEAN, "SEMEAN", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_STDDEV, "STDDEV", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_SUM, "SUM", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_TOTALN, "TOTALN", CTFT_UECELL, CTF_COUNT, CTFA_SCALE)
+S(CTSF_VALIDN, "VALIDN", CTFT_UECELL, CTF_COUNT, CTFA_SCALE)
+S(CTSF_VARIANCE, "VARIANCE", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
+S(CTSF_areaPCT_SUM, "PCT.SUM", CTFT_AREA, CTF_PERCENT, CTFA_SCALE)
+
+/* Debugging and troubleshooting. */
+S(CTSF_areaID, "ID", CTFT_AREA, CTF_COUNT, CTFA_ALL)
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2007, 2009, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <float.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/data-in.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/settings.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/data-parser.h"
+#include "language/commands/data-reader.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/inpt-pgm.h"
+#include "language/commands/placement-parser.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+
+#include "gl/xsize.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+\f
+/* DATA LIST transformation data. */
+struct data_list_trns
+ {
+ struct data_parser *parser; /* Parser. */
+ struct dictionary *dict; /* Dictionary. */
+ struct dfm_reader *reader; /* Data file reader. */
+ struct variable *end; /* Variable specified on END subcommand. */
+ };
+
+static bool parse_fixed (struct lexer *, struct dictionary *,
+ struct pool *, struct data_parser *);
+static bool parse_free (struct lexer *, struct dictionary *,
+ struct pool *, struct data_parser *);
+
+static const struct trns_class data_list_trns_class;
+
+int
+cmd_data_list (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = (in_input_program ()
+ ? dataset_dict (ds)
+ : dict_create (get_default_encoding ()));
+ struct data_parser *parser = data_parser_create ();
+ struct dfm_reader *reader = NULL;
+
+ struct variable *end = NULL;
+ struct file_handle *fh = NULL;
+
+ char *encoding = NULL;
+ int encoding_start = 0, encoding_end = 0;
+
+ int table = -1; /* Print table if nonzero, -1=undecided. */
+
+ bool has_type = false;
+
+ int end_start = 0, end_end = 0;
+ while (lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "FILE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ fh_unref (fh);
+ fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
+ if (fh == NULL)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ENCODING"))
+ {
+ encoding_start = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ encoding_end = lex_ofs (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "RECORDS"))
+ {
+ if (data_parser_get_records (parser) > 0)
+ {
+ lex_sbc_only_once (lexer, "RECORDS");
+ goto error;
+ }
+ lex_match (lexer, T_EQUALS);
+ lex_match (lexer, T_LPAREN);
+ if (!lex_force_int_range (lexer, "RECORDS", 0, INT_MAX))
+ goto error;
+ data_parser_set_records (parser, lex_integer (lexer));
+ lex_get (lexer);
+ lex_match (lexer, T_RPAREN);
+ }
+ else if (lex_match_id (lexer, "SKIP"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "SKIP", 0, INT_MAX))
+ goto error;
+ data_parser_set_skip (parser, lex_integer (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "END"))
+ {
+ if (!in_input_program ())
+ {
+ lex_next_error (lexer, -1, -1,
+ _("The %s subcommand may only be used within %s."),
+ "END", "INPUT PROGRAM");
+ goto error;
+ }
+ if (end)
+ {
+ lex_sbc_only_once (lexer, "END");
+ goto error;
+ }
+
+ end_start = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_id (lexer))
+ goto error;
+ end_end = lex_ofs (lexer);
+
+ end = dict_lookup_var (dict, lex_tokcstr (lexer));
+ if (!end)
+ end = dict_create_var_assert (dict, lex_tokcstr (lexer), 0);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "NOTABLE"))
+ table = 0;
+ else if (lex_match_id (lexer, "TABLE"))
+ table = 1;
+ else if (lex_token (lexer) == T_ID)
+ {
+ if (lex_match_id (lexer, "FIXED"))
+ data_parser_set_type (parser, DP_FIXED);
+ else if (lex_match_id (lexer, "FREE"))
+ {
+ data_parser_set_type (parser, DP_DELIMITED);
+ data_parser_set_span (parser, true);
+ }
+ else if (lex_match_id (lexer, "LIST"))
+ {
+ data_parser_set_type (parser, DP_DELIMITED);
+ data_parser_set_span (parser, false);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "FILE", "ENCODING", "RECORDS",
+ "SKIP", "END", "NOTABLE", "TABLE",
+ "FIXED", "FREE", "LIST");
+ goto error;
+ }
+
+ if (has_type)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Only one of FIXED, FREE, or LIST may "
+ "be specified."));
+ goto error;
+ }
+ has_type = true;
+
+ if (data_parser_get_type (parser) == DP_DELIMITED)
+ {
+ if (lex_match (lexer, T_LPAREN))
+ {
+ struct string delims = DS_EMPTY_INITIALIZER;
+
+ do
+ {
+ int delim;
+
+ if (lex_match_id (lexer, "TAB"))
+ delim = '\t';
+ else if (lex_is_string (lexer)
+ && ss_length (lex_tokss (lexer)) == 1)
+ {
+ delim = ss_first (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else
+ {
+ /* XXX should support multibyte UTF-8 characters */
+ lex_error (lexer, _("Syntax error expecting TAB "
+ "or delimiter string."));
+ ds_destroy (&delims);
+ goto error;
+ }
+ ds_put_byte (&delims, delim);
+
+ lex_match (lexer, T_COMMA);
+ }
+ while (!lex_match (lexer, T_RPAREN));
+
+ data_parser_set_empty_line_has_field (parser, true);
+ data_parser_set_quotes (parser, ss_empty ());
+ data_parser_set_soft_delimiters (parser, ss_empty ());
+ data_parser_set_hard_delimiters (parser, ds_ss (&delims));
+ ds_destroy (&delims);
+ }
+ else
+ {
+ data_parser_set_empty_line_has_field (parser, false);
+ data_parser_set_quotes (parser, ss_cstr ("'\""));
+ data_parser_set_soft_delimiters (parser,
+ ss_cstr (CC_SPACES));
+ const char decimal = settings_get_fmt_settings ()->decimal;
+ data_parser_set_hard_delimiters (parser,
+ ss_buffer (",", (decimal == '.') ? 1 : 0));
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "FILE", "ENCODING", "RECORDS",
+ "SKIP", "END", "NOTABLE", "TABLE",
+ "FIXED", "FREE", "LIST");
+ goto error;
+ }
+ }
+
+ if (!fh)
+ {
+ fh = fh_inline_file ();
+
+ if (encoding)
+ lex_ofs_msg (lexer, SW, encoding_start, encoding_end,
+ _("Encoding should not be specified for inline data. "
+ "It will be ignored."));
+ }
+ fh_set_default_handle (fh);
+
+ enum data_parser_type type = data_parser_get_type (parser);
+ if (type != DP_FIXED && end != NULL)
+ {
+ lex_ofs_error (lexer, end_start, end_end,
+ _("The %s subcommand may be used only with %s."),
+ "END", "DATA LIST FIXED");
+ goto error;
+ }
+
+ struct pool *tmp_pool = pool_create ();
+ bool ok = (type == DP_FIXED
+ ? parse_fixed (lexer, dict, tmp_pool, parser)
+ : parse_free (lexer, dict, tmp_pool, parser));
+ pool_destroy (tmp_pool);
+ if (!ok)
+ goto error;
+ assert (data_parser_any_fields (parser));
+
+ if (lex_end_of_command (lexer) != CMD_SUCCESS)
+ goto error;
+
+ if (table == -1)
+ table = type == DP_FIXED || !data_parser_get_span (parser);
+ if (table)
+ data_parser_output_description (parser, fh);
+
+ reader = dfm_open_reader (fh, lexer, encoding);
+ if (reader == NULL)
+ goto error;
+
+ if (in_input_program ())
+ {
+ struct data_list_trns *trns = xmalloc (sizeof *trns);
+ *trns = (struct data_list_trns) {
+ .parser = parser,
+ .dict = dict_ref (dict),
+ .reader = reader,
+ .end = end,
+ };
+ add_transformation (ds, &data_list_trns_class, trns);
+ }
+ else
+ data_parser_make_active_file (parser, ds, reader, dict, NULL, NULL);
+
+ fh_unref (fh);
+ free (encoding);
+
+ data_list_seen ();
+
+ return CMD_SUCCESS;
+
+ error:
+ data_parser_destroy (parser);
+ if (!in_input_program ())
+ dict_unref (dict);
+ fh_unref (fh);
+ free (encoding);
+ return CMD_CASCADING_FAILURE;
+}
+\f
+/* Fixed-format parsing. */
+
+/* Parses all the variable specifications for DATA LIST FIXED,
+ storing them into DLS. Uses TMP_POOL for temporary storage;
+ the caller may destroy it. Returns true only if
+ successful. */
+static bool
+parse_fixed (struct lexer *lexer, struct dictionary *dict,
+ struct pool *tmp_pool, struct data_parser *parser)
+{
+ int max_records = data_parser_get_records (parser);
+ int record = 0;
+ int column = 1;
+
+ do
+ {
+ /* Parse everything. */
+ int records_start = lex_ofs (lexer);
+ if (!parse_record_placement (lexer, &record, &column))
+ return false;
+
+ int vars_start = lex_ofs (lexer);
+ char **names;
+ size_t n_names;
+ if (!parse_DATA_LIST_vars_pool (lexer, dict, tmp_pool,
+ &names, &n_names, PV_NONE))
+ return false;
+ int vars_end = lex_ofs (lexer) - 1;
+ struct fmt_spec *formats;
+ size_t n_formats;
+ if (!parse_var_placements (lexer, tmp_pool, n_names, FMT_FOR_INPUT,
+ &formats, &n_formats))
+ return false;
+ int placements_end = lex_ofs (lexer) - 1;
+
+ /* Create variables and var specs. */
+ size_t name_idx = 0;
+ for (struct fmt_spec *f = formats; f < &formats[n_formats]; f++)
+ if (!execute_placement_format (f, &record, &column))
+ {
+ /* Create variable. */
+ const char *name = names[name_idx++];
+ int width = fmt_var_width (f);
+ struct variable *v = dict_create_var (dict, name, width);
+ if (v != NULL)
+ {
+ /* Success. */
+ struct fmt_spec output = fmt_for_output_from_input (
+ f, settings_get_fmt_settings ());
+ var_set_both_formats (v, &output);
+ }
+ else
+ {
+ /* Failure.
+ This can be acceptable if we're in INPUT
+ PROGRAM, but only if the existing variable has
+ the same width as the one we would have
+ created. */
+ if (!in_input_program ())
+ {
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("%s is a duplicate variable name."), name);
+ return false;
+ }
+
+ v = dict_lookup_var_assert (dict, name);
+ if ((width != 0) != (var_get_width (v) != 0))
+ {
+ lex_ofs_error (lexer, vars_start, placements_end,
+ _("There is already a variable %s of a "
+ "different type."), name);
+ return false;
+ }
+ if (width != 0 && width != var_get_width (v))
+ {
+ lex_ofs_error (lexer, vars_start, placements_end,
+ _("There is already a string variable %s of "
+ "a different width."), name);
+ return false;
+ }
+ }
+
+ if (max_records && record > max_records)
+ {
+ lex_ofs_error (lexer, records_start, vars_end,
+ _("Cannot place variable %s on record %d when "
+ "RECORDS=%d is specified."),
+ var_get_name (v), record,
+ data_parser_get_records (parser));
+ return false;
+ }
+
+ data_parser_add_fixed_field (parser, f,
+ var_get_case_index (v),
+ var_get_name (v), record, column);
+
+ column += f->w;
+ }
+ assert (name_idx == n_names);
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ return true;
+}
+\f
+/* Free-format parsing. */
+
+/* Parses variable specifications for DATA LIST FREE and adds
+ them to DLS. Uses TMP_POOL for temporary storage; the caller
+ may destroy it. Returns true only if successful. */
+static bool
+parse_free (struct lexer *lexer, struct dictionary *dict,
+ struct pool *tmp_pool, struct data_parser *parser)
+{
+ lex_get (lexer);
+ do
+ {
+ char **names;
+ size_t n_names;
+
+ int vars_start = lex_ofs (lexer);
+ if (!parse_DATA_LIST_vars_pool (lexer, dict, tmp_pool,
+ &names, &n_names, PV_NONE))
+ return false;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ struct fmt_spec input, output;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ char type[FMT_TYPE_LEN_MAX + 1];
+
+ if (!parse_abstract_format_specifier (lexer, type, &input.w,
+ &input.d))
+ return NULL;
+ if (!fmt_from_name (type, &input.type))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Unknown format type `%s'."), type);
+ return NULL;
+ }
+
+ /* If no width was included, use the minimum width for the type.
+ This isn't quite right, because DATETIME by itself seems to become
+ DATETIME20 (see bug #30690), whereas this will become
+ DATETIME17. The correct behavior is not documented. */
+ if (input.w == 0)
+ {
+ input.w = fmt_min_input_width (input.type);
+ input.d = 0;
+ }
+
+ char *error = fmt_check_input__ (&input);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ return NULL;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ return NULL;
+
+ /* As a special case, N format is treated as F format
+ for free-field input. */
+ if (input.type == FMT_N)
+ input.type = FMT_F;
+
+ output = fmt_for_output_from_input (&input,
+ settings_get_fmt_settings ());
+ }
+ else
+ {
+ lex_match (lexer, T_ASTERISK);
+ input = fmt_for_input (FMT_F, 8, 0);
+ output = *settings_get_format ();
+ }
+
+ for (size_t i = 0; i < n_names; i++)
+ {
+ struct variable *v = dict_create_var (dict, names[i],
+ fmt_var_width (&input));
+ if (!v)
+ {
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("%s is a duplicate variable name."), names[i]);
+ return false;
+ }
+ var_set_both_formats (v, &output);
+
+ data_parser_add_delimited_field (parser,
+ &input, var_get_case_index (v),
+ var_get_name (v));
+ }
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ return true;
+}
+\f
+/* Input procedure. */
+
+/* Destroys DATA LIST transformation TRNS.
+ Returns true if successful, false if an I/O error occurred. */
+static bool
+data_list_trns_free (void *trns_)
+{
+ struct data_list_trns *trns = trns_;
+ data_parser_destroy (trns->parser);
+ dfm_close_reader (trns->reader);
+ dict_unref (trns->dict);
+ free (trns);
+ return true;
+}
+
+/* Handle DATA LIST transformation TRNS, parsing data into *C. */
+static enum trns_result
+data_list_trns_proc (void *trns_, struct ccase **c, casenumber case_num UNUSED)
+{
+ struct data_list_trns *trns = trns_;
+ enum trns_result retval;
+
+ *c = case_unshare (*c);
+ if (data_parser_parse (trns->parser, trns->reader, trns->dict, *c))
+ retval = TRNS_CONTINUE;
+ else if (dfm_reader_error (trns->reader) || dfm_eof (trns->reader) > 1)
+ {
+ /* An I/O error, or encountering end of file for a second
+ time, should be escalated into a more serious error. */
+ retval = TRNS_ERROR;
+ }
+ else
+ retval = TRNS_END_FILE;
+
+ /* If there was an END subcommand handle it. */
+ if (trns->end != NULL)
+ {
+ double *end = case_num_rw (*c, trns->end);
+ if (retval == TRNS_END_FILE)
+ {
+ *end = 1.0;
+ retval = TRNS_CONTINUE;
+ }
+ else
+ *end = 0.0;
+ }
+
+ return retval;
+}
+
+static const struct trns_class data_list_trns_class = {
+ .name = "DATA LIST",
+ .execute = data_list_trns_proc,
+ .destroy = data_list_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2007, 2009, 2010, 2011, 2012, 2013, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/data-parser.h"
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#include "data/casereader-provider.h"
+#include "data/data-in.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/file-handle-def.h"
+#include "data/settings.h"
+#include "language/commands/data-reader.h"
+#include "libpspp/intern.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "libpspp/string-array.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+/* Data parser for textual data like that read by DATA LIST. */
+struct data_parser
+ {
+ enum data_parser_type type; /* Type of data to parse. */
+ int skip_records; /* Records to skip before first real data. */
+
+ struct field *fields; /* Fields to parse. */
+ size_t n_fields; /* Number of fields. */
+ size_t field_allocated; /* Number of fields spaced allocated for. */
+
+ /* DP_DELIMITED parsers only. */
+ bool span; /* May cases span multiple records? */
+ bool empty_line_has_field; /* Does an empty line have an (empty) field? */
+ bool warn_missing_fields; /* Should missing fields be considered errors? */
+ struct substring quotes; /* Characters that can quote separators. */
+ bool quote_escape; /* Doubled quote acts as escape? */
+ struct substring soft_seps; /* Two soft separators act like just one. */
+ struct substring hard_seps; /* Two hard separators yield empty fields. */
+ struct string any_sep; /* Concatenation of soft_seps and hard_seps. */
+
+ /* DP_FIXED parsers only. */
+ int records_per_case; /* Number of records in each case. */
+ };
+
+/* How to parse one variable. */
+struct field
+ {
+ struct fmt_spec format; /* Input format of this field. */
+ int case_idx; /* First value in case. */
+ char *name; /* Var name for error messages and tables. */
+
+ /* DP_FIXED only. */
+ int record; /* Record number (1-based). */
+ int first_column; /* First column in record (1-based). */
+ };
+
+static void set_any_sep (struct data_parser *parser);
+
+/* Creates and returns a new data parser. */
+struct data_parser *
+data_parser_create (void)
+{
+ struct data_parser *parser = xmalloc (sizeof *parser);
+ *parser = (struct data_parser) {
+ .type = DP_FIXED,
+ .span = true,
+ .warn_missing_fields = true,
+ .quotes = ss_clone (ss_cstr ("\"'")),
+ .soft_seps = ss_clone (ss_cstr (CC_SPACES)),
+ .hard_seps = ss_clone (ss_cstr (",")),
+ };
+ set_any_sep (parser);
+
+ return parser;
+}
+
+/* Destroys PARSER. */
+void
+data_parser_destroy (struct data_parser *parser)
+{
+ if (parser != NULL)
+ {
+ size_t i;
+
+ for (i = 0; i < parser->n_fields; i++)
+ free (parser->fields[i].name);
+ free (parser->fields);
+ ss_dealloc (&parser->quotes);
+ ss_dealloc (&parser->soft_seps);
+ ss_dealloc (&parser->hard_seps);
+ ds_destroy (&parser->any_sep);
+ free (parser);
+ }
+}
+
+/* Returns the type of PARSER (either DP_DELIMITED or DP_FIXED). */
+enum data_parser_type
+data_parser_get_type (const struct data_parser *parser)
+{
+ return parser->type;
+}
+
+/* Sets the type of PARSER to TYPE (either DP_DELIMITED or
+ DP_FIXED). */
+void
+data_parser_set_type (struct data_parser *parser, enum data_parser_type type)
+{
+ assert (parser->n_fields == 0);
+ assert (type == DP_FIXED || type == DP_DELIMITED);
+ parser->type = type;
+}
+
+/* Configures PARSER to skip the specified number of
+ INITIAL_RECORDS_TO_SKIP before parsing any data. By default,
+ no records are skipped. */
+void
+data_parser_set_skip (struct data_parser *parser, int initial_records_to_skip)
+{
+ assert (initial_records_to_skip >= 0);
+ parser->skip_records = initial_records_to_skip;
+}
+
+/* Returns true if PARSER is configured to allow cases to span
+ multiple records. */
+bool
+data_parser_get_span (const struct data_parser *parser)
+{
+ return parser->span;
+}
+
+/* If MAY_CASES_SPAN_RECORDS is true, configures PARSER to allow
+ a single case to span multiple records and multiple cases to
+ occupy a single record. If MAY_CASES_SPAN_RECORDS is false,
+ configures PARSER to require each record to contain exactly
+ one case.
+
+ This setting affects parsing of DP_DELIMITED files only. */
+void
+data_parser_set_span (struct data_parser *parser, bool may_cases_span_records)
+{
+ parser->span = may_cases_span_records;
+}
+
+/* If EMPTY_LINE_HAS_FIELD is true, configures PARSER to parse an
+ empty line as an empty field and to treat a hard delimiter
+ followed by end-of-line as an empty field. If
+ EMPTY_LINE_HAS_FIELD is false, PARSER will skip empty lines
+ and hard delimiters at the end of lines without emitting empty
+ fields.
+
+ This setting affects parsing of DP_DELIMITED files only. */
+void
+data_parser_set_empty_line_has_field (struct data_parser *parser,
+ bool empty_line_has_field)
+{
+ parser->empty_line_has_field = empty_line_has_field;
+}
+
+
+/* If WARN_MISSING_FIELDS is true, configures PARSER to emit a warning
+ and cause an error condition when a missing field is encountered.
+ If WARN_MISSING_FIELDS is false, PARSER will silently fill such
+ fields with the system missing value.
+
+ This setting affects parsing of DP_DELIMITED files only. */
+void
+data_parser_set_warn_missing_fields (struct data_parser *parser,
+ bool warn_missing_fields)
+{
+ parser->warn_missing_fields = warn_missing_fields;
+}
+
+
+/* Sets the characters that may be used for quoting field
+ contents to QUOTES. If QUOTES is empty, quoting will be
+ disabled.
+
+ The caller retains ownership of QUOTES.
+
+ This setting affects parsing of DP_DELIMITED files only. */
+void
+data_parser_set_quotes (struct data_parser *parser, struct substring quotes)
+{
+ ss_dealloc (&parser->quotes);
+ parser->quotes = ss_clone (quotes);
+}
+
+/* If ESCAPE is false (the default setting), a character used for
+ quoting cannot itself be embedded within a quoted field. If
+ ESCAPE is true, then a quote character can be embedded within
+ a quoted field by doubling it.
+
+ This setting affects parsing of DP_DELIMITED files only, and
+ only when at least one quote character has been set (with
+ data_parser_set_quotes). */
+void
+data_parser_set_quote_escape (struct data_parser *parser, bool escape)
+{
+ parser->quote_escape = escape;
+}
+
+/* Sets PARSER's soft delimiters to DELIMITERS. Soft delimiters
+ separate fields, but consecutive soft delimiters do not yield
+ empty fields. (Ordinarily, only white space characters are
+ appropriate soft delimiters.)
+
+ The caller retains ownership of DELIMITERS.
+
+ This setting affects parsing of DP_DELIMITED files only. */
+void
+data_parser_set_soft_delimiters (struct data_parser *parser,
+ struct substring delimiters)
+{
+ ss_dealloc (&parser->soft_seps);
+ parser->soft_seps = ss_clone (delimiters);
+ set_any_sep (parser);
+}
+
+/* Sets PARSER's hard delimiters to DELIMITERS. Hard delimiters
+ separate fields. A consecutive pair of hard delimiters yield
+ an empty field.
+
+ The caller retains ownership of DELIMITERS.
+
+ This setting affects parsing of DP_DELIMITED files only. */
+void
+data_parser_set_hard_delimiters (struct data_parser *parser,
+ struct substring delimiters)
+{
+ ss_dealloc (&parser->hard_seps);
+ parser->hard_seps = ss_clone (delimiters);
+ set_any_sep (parser);
+}
+
+/* Returns the number of records per case. */
+int
+data_parser_get_records (const struct data_parser *parser)
+{
+ return parser->records_per_case;
+}
+
+/* Sets the number of records per case to RECORDS_PER_CASE.
+
+ This setting affects parsing of DP_FIXED files only. */
+void
+data_parser_set_records (struct data_parser *parser, int records_per_case)
+{
+ assert (records_per_case >= 0);
+ assert (records_per_case >= parser->records_per_case);
+ parser->records_per_case = records_per_case;
+}
+
+static void
+add_field (struct data_parser *p, const struct fmt_spec *format, int case_idx,
+ const char *name, int record, int first_column)
+{
+ struct field *field;
+
+ if (p->n_fields == p->field_allocated)
+ p->fields = x2nrealloc (p->fields, &p->field_allocated, sizeof *p->fields);
+ field = &p->fields[p->n_fields++];
+ field->format = *format;
+ field->case_idx = case_idx;
+ field->name = xstrdup (name);
+ field->record = record;
+ field->first_column = first_column;
+}
+
+/* Adds a delimited field to the field parsed by PARSER, which
+ must be configured as a DP_DELIMITED parser. The field is
+ parsed as input format FORMAT. Its data will be stored into case
+ index CASE_INDEX. Errors in input data will be reported
+ against variable NAME. */
+void
+data_parser_add_delimited_field (struct data_parser *parser,
+ const struct fmt_spec *format, int case_idx,
+ const char *name)
+{
+ assert (parser->type == DP_DELIMITED);
+ add_field (parser, format, case_idx, name, 0, 0);
+}
+
+/* Adds a fixed field to the field parsed by PARSER, which
+ must be configured as a DP_FIXED parser. The field is
+ parsed as input format FORMAT. Its data will be stored into case
+ index CASE_INDEX. Errors in input data will be reported
+ against variable NAME. The field will be drawn from the
+ FORMAT->w columns in 1-based RECORD starting at 1-based
+ column FIRST_COLUMN.
+
+ RECORD must be at least as great as that of any field already
+ added; that is, fields must be added in increasing order of
+ record number. If RECORD is greater than the current number
+ of records per case, the number of records per case are
+ increased as needed. */
+void
+data_parser_add_fixed_field (struct data_parser *parser,
+ const struct fmt_spec *format, int case_idx,
+ const char *name,
+ int record, int first_column)
+{
+ assert (parser->type == DP_FIXED);
+ assert (parser->n_fields == 0
+ || record >= parser->fields[parser->n_fields - 1].record);
+ if (record > parser->records_per_case)
+ parser->records_per_case = record;
+ add_field (parser, format, case_idx, name, record, first_column);
+}
+
+/* Returns true if any fields have been added to PARSER, false
+ otherwise. */
+bool
+data_parser_any_fields (const struct data_parser *parser)
+{
+ return parser->n_fields > 0;
+}
+
+static void
+set_any_sep (struct data_parser *parser)
+{
+ ds_assign_substring (&parser->any_sep, parser->soft_seps);
+ ds_put_substring (&parser->any_sep, parser->hard_seps);
+}
+\f
+static bool parse_delimited_span (const struct data_parser *,
+ struct dfm_reader *,
+ struct dictionary *, struct ccase *);
+static bool parse_delimited_no_span (const struct data_parser *,
+ struct dfm_reader *,
+ struct dictionary *, struct ccase *);
+static bool parse_fixed (const struct data_parser *, struct dfm_reader *,
+ struct dictionary *, struct ccase *);
+
+/* Reads a case from DFM into C, which matches dictionary DICT, parsing it with
+ PARSER. Returns true if successful, false at end of file or on I/O error.
+
+ Case C must not be shared. */
+bool
+data_parser_parse (struct data_parser *parser, struct dfm_reader *reader,
+ struct dictionary *dict, struct ccase *c)
+{
+ bool retval;
+
+ assert (!case_is_shared (c));
+ assert (data_parser_any_fields (parser));
+
+ /* Skip the requested number of records before reading the
+ first case. */
+ for (; parser->skip_records > 0; parser->skip_records--)
+ {
+ if (dfm_eof (reader))
+ return false;
+ dfm_forward_record (reader);
+ }
+
+ /* Limit cases. */
+ if (parser->type == DP_DELIMITED)
+ {
+ if (parser->span)
+ retval = parse_delimited_span (parser, reader, dict, c);
+ else
+ retval = parse_delimited_no_span (parser, reader, dict, c);
+ }
+ else
+ retval = parse_fixed (parser, reader, dict, c);
+
+ return retval;
+}
+
+static void
+cut_field__ (const struct data_parser *parser, const struct substring *line,
+ struct substring *p, size_t *n_columns,
+ struct string *tmp, struct substring *field)
+{
+ bool quoted = ss_find_byte (parser->quotes, ss_first (*p)) != SIZE_MAX;
+ if (quoted)
+ {
+ /* Quoted field. */
+ int quote = ss_get_byte (p);
+ if (!ss_get_until (p, quote, field))
+ msg (DW, _("Quoted string extends beyond end of line."));
+ if (parser->quote_escape && ss_first (*p) == quote)
+ {
+ ds_assign_substring (tmp, *field);
+ while (ss_match_byte (p, quote))
+ {
+ struct substring ss;
+ ds_put_byte (tmp, quote);
+ if (!ss_get_until (p, quote, &ss))
+ msg (DW, _("Quoted string extends beyond end of line."));
+ ds_put_substring (tmp, ss);
+ }
+ *field = ds_ss (tmp);
+ }
+ *n_columns = ss_length (*line) - ss_length (*p);
+ }
+ else
+ {
+ /* Regular field. */
+ ss_get_bytes (p, ss_cspan (*p, ds_ss (&parser->any_sep)), field);
+ *n_columns = ss_length (*field);
+ }
+
+ /* Skip trailing soft separator and a single hard separator if present. */
+ size_t length_before_separators = ss_length (*p);
+ ss_ltrim (p, parser->soft_seps);
+ if (!ss_is_empty (*p)
+ && ss_find_byte (parser->hard_seps, ss_first (*p)) != SIZE_MAX)
+ {
+ ss_advance (p, 1);
+ ss_ltrim (p, parser->soft_seps);
+ }
+
+ if (!ss_is_empty (*p) && quoted && length_before_separators == ss_length (*p))
+ msg (DW, _("Missing delimiter following quoted string."));
+}
+
+/* Extracts a delimited field from the current position in the
+ current record according to PARSER, reading data from READER.
+
+ *FIELD is set to the field content. The caller must not or
+ destroy this constant string.
+
+ Sets *FIRST_COLUMN to the 1-based column number of the start of
+ the extracted field, and *LAST_COLUMN to the end of the extracted
+ field.
+
+ Returns true on success, false on failure. */
+static bool
+cut_field (const struct data_parser *parser, struct dfm_reader *reader,
+ int *first_column, int *last_column, struct string *tmp,
+ struct substring *field)
+{
+ struct substring line, p;
+
+ if (dfm_eof (reader))
+ return false;
+ if (ss_is_empty (parser->hard_seps))
+ dfm_expand_tabs (reader);
+ line = p = dfm_get_record (reader);
+
+ /* Skip leading soft separators. */
+ ss_ltrim (&p, parser->soft_seps);
+
+ /* Handle empty or completely consumed lines. */
+ if (ss_is_empty (p))
+ {
+ if (!parser->empty_line_has_field || dfm_columns_past_end (reader) > 0)
+ return false;
+ else
+ {
+ *field = p;
+ *first_column = dfm_column_start (reader);
+ *last_column = *first_column + 1;
+ dfm_forward_columns (reader, 1);
+ return true;
+ }
+ }
+
+ size_t n_columns;
+ cut_field__ (parser, &line, &p, &n_columns, tmp, field);
+ *first_column = dfm_column_start (reader);
+ *last_column = *first_column + n_columns;
+
+ if (ss_is_empty (p))
+ dfm_forward_columns (reader, 1);
+ dfm_forward_columns (reader, ss_length (line) - ss_length (p));
+
+ return true;
+}
+
+static void
+parse_error (const struct dfm_reader *reader, const struct field *field,
+ int first_column, int last_column, char *error)
+{
+ int line_number = dfm_get_line_number (reader);
+ struct msg_location *location = xmalloc (sizeof *location);
+ *location = (struct msg_location) {
+ .file_name = intern_new (dfm_get_file_name (reader)),
+ .start = { .line = line_number, .column = first_column },
+ .end = { .line = line_number, .column = last_column - 1 },
+ };
+ struct msg *m = xmalloc (sizeof *m);
+ *m = (struct msg) {
+ .category = MSG_C_DATA,
+ .severity = MSG_S_WARNING,
+ .location = location,
+ .text = xasprintf (_("Data for variable %s is not valid as format %s: %s"),
+ field->name, fmt_name (field->format.type), error),
+ };
+ msg_emit (m);
+
+ free (error);
+}
+
+/* Reads a case from READER into C, which matches DICT, parsing it according to
+ fixed-format syntax rules in PARSER. Returns true if successful, false at
+ end of file or on I/O error. */
+static bool
+parse_fixed (const struct data_parser *parser, struct dfm_reader *reader,
+ struct dictionary *dict, struct ccase *c)
+{
+ const char *input_encoding = dfm_reader_get_encoding (reader);
+ const char *output_encoding = dict_get_encoding (dict);
+ struct field *f;
+ int row;
+
+ if (dfm_eof (reader))
+ return false;
+
+ f = parser->fields;
+ for (row = 1; row <= parser->records_per_case; row++)
+ {
+ struct substring line;
+
+ if (dfm_eof (reader))
+ {
+ msg (DW, _("Partial case of %d of %d records discarded."),
+ row - 1, parser->records_per_case);
+ return false;
+ }
+ dfm_expand_tabs (reader);
+ line = dfm_get_record (reader);
+
+ for (; f < &parser->fields[parser->n_fields] && f->record == row; f++)
+ {
+ struct substring s = ss_substr (line, f->first_column - 1,
+ f->format.w);
+ union value *value = case_data_rw_idx (c, f->case_idx);
+ char *error = data_in (s, input_encoding, f->format.type,
+ settings_get_fmt_settings (),
+ value, fmt_var_width (&f->format),
+ output_encoding);
+
+ if (error == NULL)
+ data_in_imply_decimals (s, input_encoding, f->format.type,
+ f->format.d, settings_get_fmt_settings (),
+ value);
+ else
+ parse_error (reader, f, f->first_column,
+ f->first_column + f->format.w, error);
+ }
+
+ dfm_forward_record (reader);
+ }
+
+ return true;
+}
+
+/* Splits the data line in LINE into individual text fields and returns the
+ number of fields. If SA is nonnull, appends each field to SA; the caller
+ retains ownership of SA and its contents. */
+size_t
+data_parser_split (const struct data_parser *parser,
+ struct substring line, struct string_array *sa)
+{
+ size_t n = 0;
+
+ struct string tmp = DS_EMPTY_INITIALIZER;
+ for (;;)
+ {
+ struct substring p = line;
+ ss_ltrim (&p, parser->soft_seps);
+ if (ss_is_empty (p))
+ {
+ ds_destroy (&tmp);
+ return n;
+ }
+
+ size_t n_columns;
+ struct substring field;
+
+ msg_disable ();
+ cut_field__ (parser, &line, &p, &n_columns, &tmp, &field);
+ msg_enable ();
+
+ if (sa)
+ string_array_append_nocopy (sa, ss_xstrdup (field));
+ n++;
+ line = p;
+ }
+}
+
+/* Reads a case from READER into C, which matches dictionary DICT, parsing it
+ according to free-format syntax rules in PARSER. Returns true if
+ successful, false at end of file or on I/O error. */
+static bool
+parse_delimited_span (const struct data_parser *parser,
+ struct dfm_reader *reader,
+ struct dictionary *dict, struct ccase *c)
+{
+ const char *output_encoding = dict_get_encoding (dict);
+ struct string tmp = DS_EMPTY_INITIALIZER;
+ struct field *f;
+
+ for (f = parser->fields; f < &parser->fields[parser->n_fields]; f++)
+ {
+ struct substring s;
+ int first_column, last_column;
+ char *error;
+
+ /* Cut out a field and read in a new record if necessary. */
+ while (!cut_field (parser, reader,
+ &first_column, &last_column, &tmp, &s))
+ {
+ if (!dfm_eof (reader))
+ dfm_forward_record (reader);
+ if (dfm_eof (reader))
+ {
+ if (f > parser->fields)
+ msg (DW, _("Partial case discarded. The first variable "
+ "missing was %s."), f->name);
+ ds_destroy (&tmp);
+ return false;
+ }
+ }
+
+ const char *input_encoding = dfm_reader_get_encoding (reader);
+ error = data_in (s, input_encoding, f->format.type,
+ settings_get_fmt_settings (),
+ case_data_rw_idx (c, f->case_idx),
+ fmt_var_width (&f->format), output_encoding);
+ if (error != NULL)
+ parse_error (reader, f, first_column, last_column, error);
+ }
+ ds_destroy (&tmp);
+ return true;
+}
+
+/* Reads a case from READER into C, which matches dictionary DICT, parsing it
+ according to delimited syntax rules with one case per record in PARSER.
+ Returns true if successful, false at end of file or on I/O error. */
+static bool
+parse_delimited_no_span (const struct data_parser *parser,
+ struct dfm_reader *reader,
+ struct dictionary *dict, struct ccase *c)
+{
+ const char *output_encoding = dict_get_encoding (dict);
+ struct string tmp = DS_EMPTY_INITIALIZER;
+ struct substring s;
+ struct field *f, *end;
+
+ if (dfm_eof (reader))
+ return false;
+
+ end = &parser->fields[parser->n_fields];
+ for (f = parser->fields; f < end; f++)
+ {
+ int first_column, last_column;
+ char *error;
+
+ if (!cut_field (parser, reader, &first_column, &last_column, &tmp, &s))
+ {
+ if (f < end - 1 && settings_get_undefined () && parser->warn_missing_fields)
+ msg (DW, _("Missing value(s) for all variables from %s onward. "
+ "These will be filled with the system-missing value "
+ "or blanks, as appropriate."),
+ f->name);
+ for (; f < end; f++)
+ value_set_missing (case_data_rw_idx (c, f->case_idx),
+ fmt_var_width (&f->format));
+ goto exit;
+ }
+
+ const char *input_encoding = dfm_reader_get_encoding (reader);
+ error = data_in (s, input_encoding, f->format.type,
+ settings_get_fmt_settings (),
+ case_data_rw_idx (c, f->case_idx),
+ fmt_var_width (&f->format), output_encoding);
+ if (error != NULL)
+ parse_error (reader, f, first_column, last_column, error);
+ }
+
+ s = dfm_get_record (reader);
+ ss_ltrim (&s, parser->soft_seps);
+ if (!ss_is_empty (s))
+ msg (DW, _("Record ends in data not part of any field."));
+
+exit:
+ dfm_forward_record (reader);
+ ds_destroy (&tmp);
+ return true;
+}
+\f
+/* Displays a table giving information on fixed-format variable
+ parsing on DATA LIST. */
+static void
+dump_fixed_table (const struct data_parser *parser,
+ const struct file_handle *fh)
+{
+ /* XXX This should not be preformatted. */
+ char *title = xasprintf (ngettext ("Reading %d record from %s.",
+ "Reading %d records from %s.",
+ parser->records_per_case),
+ parser->records_per_case, fh_get_name (fh));
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_user_text (title, -1), "Fixed Data Records");
+ free (title);
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Attributes"),
+ N_("Record"), N_("Columns"), N_("Format"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+ variables->root->show_label = true;
+ for (size_t i = 0; i < parser->n_fields; i++)
+ {
+ struct field *f = &parser->fields[i];
+
+ /* XXX It would be better to have the actual variable here. */
+ int variable_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_user_text (f->name, -1));
+
+ pivot_table_put2 (table, 0, variable_idx,
+ pivot_value_new_integer (f->record));
+
+ int first_column = f->first_column;
+ int last_column = f->first_column + f->format.w - 1;
+ char *columns = xasprintf ("%d-%d", first_column, last_column);
+ pivot_table_put2 (table, 1, variable_idx,
+ pivot_value_new_user_text (columns, -1));
+ free (columns);
+
+ char str[FMT_STRING_LEN_MAX + 1];
+ pivot_table_put2 (table, 2, variable_idx,
+ pivot_value_new_user_text (
+ fmt_to_string (&f->format, str), -1));
+
+ }
+
+ pivot_table_submit (table);
+}
+
+/* Displays a table giving information on free-format variable parsing
+ on DATA LIST. */
+static void
+dump_delimited_table (const struct data_parser *parser,
+ const struct file_handle *fh)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("Reading free-form data from %s."),
+ fh_get_name (fh)),
+ "Free-Form Data Records");
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Attributes"), N_("Format"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+ variables->root->show_label = true;
+ for (size_t i = 0; i < parser->n_fields; i++)
+ {
+ struct field *f = &parser->fields[i];
+
+ /* XXX It would be better to have the actual variable here. */
+ int variable_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_user_text (f->name, -1));
+
+ char str[FMT_STRING_LEN_MAX + 1];
+ pivot_table_put2 (table, 0, variable_idx,
+ pivot_value_new_user_text (
+ fmt_to_string (&f->format, str), -1));
+ }
+
+ pivot_table_submit (table);
+}
+
+/* Displays a table giving information on how PARSER will read
+ data from FH. */
+void
+data_parser_output_description (struct data_parser *parser,
+ const struct file_handle *fh)
+{
+ if (parser->type == DP_FIXED)
+ dump_fixed_table (parser, fh);
+ else
+ dump_delimited_table (parser, fh);
+}
+\f
+/* Data parser input program. */
+struct data_parser_casereader
+ {
+ struct data_parser *parser; /* Parser. */
+ struct dictionary *dict; /* Dictionary. */
+ struct dfm_reader *reader; /* Data file reader. */
+ struct caseproto *proto; /* Format of cases. */
+ };
+
+static const struct casereader_class data_parser_casereader_class;
+
+/* Replaces DS's active dataset by an input program that reads data
+ from READER according to the rules in PARSER, using DICT as
+ the underlying dictionary. Ownership of PARSER and READER is
+ transferred to the input program, and ownership of DICT is
+ transferred to the dataset. */
+void
+data_parser_make_active_file (struct data_parser *parser, struct dataset *ds,
+ struct dfm_reader *reader,
+ struct dictionary *dict,
+ struct casereader* (*func)(struct casereader *,
+ const struct dictionary *,
+ void *),
+ void *ud)
+{
+ struct data_parser_casereader *r;
+ struct casereader *casereader0;
+ struct casereader *casereader1;
+
+ r = xmalloc (sizeof *r);
+ r->parser = parser;
+ r->dict = dict_ref (dict);
+ r->reader = reader;
+ r->proto = caseproto_ref (dict_get_proto (dict));
+ casereader0 = casereader_create_sequential (NULL, r->proto,
+ CASENUMBER_MAX,
+ &data_parser_casereader_class, r);
+
+ if (func)
+ casereader1 = func (casereader0, dict, ud);
+ else
+ casereader1 = casereader0;
+
+ dataset_set_dict (ds, dict);
+ dataset_set_source (ds, casereader1);
+}
+
+
+static struct ccase *
+data_parser_casereader_read (struct casereader *reader UNUSED, void *r_)
+{
+ struct data_parser_casereader *r = r_;
+ struct ccase *c = case_create (r->proto);
+ if (data_parser_parse (r->parser, r->reader, r->dict, c))
+ return c;
+ else
+ {
+ case_unref (c);
+ return NULL;
+ }
+}
+
+static void
+data_parser_casereader_destroy (struct casereader *reader, void *r_)
+{
+ struct data_parser_casereader *r = r_;
+ if (dfm_reader_error (r->reader))
+ casereader_force_error (reader);
+ dfm_close_reader (r->reader);
+ caseproto_unref (r->proto);
+ dict_unref (r->dict);
+ data_parser_destroy (r->parser);
+ free (r);
+}
+
+static const struct casereader_class data_parser_casereader_class =
+ {
+ data_parser_casereader_read,
+ data_parser_casereader_destroy,
+ NULL,
+ NULL,
+ };
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2007, 2011, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef LANGUAGE_DATA_IO_DATA_PARSER_H
+#define LANGUAGE_DATA_IO_DATA_PARSER_H
+
+/* Abstraction of a DATA LIST or GET DATA TYPE=TXT data parser. */
+
+#include <stdbool.h>
+#include "data/case.h"
+#include "libpspp/str.h"
+
+struct dataset;
+struct dfm_reader;
+struct dictionary;
+struct file_handle;
+struct fmt_spec;
+struct string_array;
+struct substring;
+
+/* Type of data read by a data parser. */
+enum data_parser_type
+ {
+ DP_FIXED, /* Fields in fixed column positions. */
+ DP_DELIMITED /* Fields delimited by e.g. commas. */
+ };
+
+/* Creating and configuring any parser. */
+struct data_parser *data_parser_create (void);
+void data_parser_destroy (struct data_parser *);
+
+enum data_parser_type data_parser_get_type (const struct data_parser *);
+void data_parser_set_type (struct data_parser *, enum data_parser_type);
+
+void data_parser_set_skip (struct data_parser *, int initial_records_to_skip);
+
+/* For configuring delimited parsers only. */
+bool data_parser_get_span (const struct data_parser *);
+void data_parser_set_span (struct data_parser *, bool may_cases_span_records);
+
+void data_parser_set_empty_line_has_field (struct data_parser *,
+ bool empty_line_has_field);
+void data_parser_set_warn_missing_fields (struct data_parser *parser,
+ bool warn_missing_fields);
+
+void data_parser_set_quotes (struct data_parser *, struct substring);
+void data_parser_set_quote_escape (struct data_parser *, bool escape);
+void data_parser_set_soft_delimiters (struct data_parser *, struct substring);
+void data_parser_set_hard_delimiters (struct data_parser *, struct substring);
+
+/* For configuring fixed parsers only. */
+int data_parser_get_records (const struct data_parser *);
+void data_parser_set_records (struct data_parser *, int records_per_case);
+
+/* Field setup and parsing. */
+void data_parser_add_delimited_field (struct data_parser *,
+ const struct fmt_spec *, int fv,
+ const char *name);
+void data_parser_add_fixed_field (struct data_parser *,
+ const struct fmt_spec *, int fv,
+ const char *name,
+ int record, int first_column);
+bool data_parser_any_fields (const struct data_parser *);
+bool data_parser_parse (struct data_parser *, struct dfm_reader *,
+ struct dictionary *, struct ccase *);
+size_t data_parser_split (const struct data_parser *, struct substring line,
+ struct string_array *);
+
+/* Uses for a configured parser. */
+void data_parser_output_description (struct data_parser *,
+ const struct file_handle *);
+struct casereader;
+void data_parser_make_active_file (struct data_parser *, struct dataset *,
+ struct dfm_reader *, struct dictionary *,
+ struct casereader* (*func)(struct casereader *,
+ const struct dictionary *,
+ void *),
+ void *ud);
+
+
+#endif /* language/commands/data-parser.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-2004, 2006, 2010, 2011, 2012, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/data-reader.h"
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/file-handle-def.h"
+#include "data/file-name.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/encoding-guesser.h"
+#include "libpspp/integer-format.h"
+#include "libpspp/line-reader.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+/* Flags for DFM readers. */
+enum dfm_reader_flags
+ {
+ DFM_ADVANCE = 002, /* Read next line on dfm_get_record() call? */
+ DFM_SAW_BEGIN_DATA = 004, /* For inline_file only, whether we've
+ already read a BEGIN DATA line. */
+ DFM_TABS_EXPANDED = 010, /* Tabs have been expanded. */
+ DFM_CONSUME = 020 /* read_inline_record() should get a token? */
+ };
+
+/* Data file reader. */
+struct dfm_reader
+ {
+ struct file_handle *fh; /* File handle. */
+ struct fh_lock *lock; /* Mutual exclusion lock for file. */
+ int line_number; /* Current line or record number. */
+ struct string line; /* Current line. */
+ struct string scratch; /* Extra line buffer. */
+ enum dfm_reader_flags flags; /* Zero or more of DFM_*. */
+ FILE *file; /* Associated file. */
+ size_t pos; /* Offset in line of current character. */
+ unsigned n_eofs; /* # of attempts to advance past EOF. */
+ struct lexer *lexer; /* The lexer reading the file */
+ char *encoding; /* Current encoding. */
+
+ /* For FH_MODE_TEXT only. */
+ struct line_reader *line_reader;
+
+ /* For FH_MODE_360_VARIABLE and FH_MODE_360_SPANNED files only. */
+ size_t block_left; /* Bytes left in current block. */
+ };
+
+/* Closes reader R opened by dfm_open_reader(). */
+void
+dfm_close_reader (struct dfm_reader *r)
+{
+ if (r == NULL)
+ return;
+
+ if (fh_unlock (r->lock))
+ {
+ /* File is still locked by another client. */
+ return;
+ }
+
+ /* This was the last client, so close the underlying file. */
+ if (fh_get_referent (r->fh) != FH_REF_INLINE)
+ fn_close (r->fh, r->file);
+ else
+ {
+ /* Skip any remaining data on the inline file. */
+ if (r->flags & DFM_SAW_BEGIN_DATA)
+ {
+ dfm_reread_record (r, 0);
+ while (!dfm_eof (r))
+ dfm_forward_record (r);
+ }
+ }
+
+ line_reader_free (r->line_reader);
+ free (r->encoding);
+ fh_unref (r->fh);
+ ds_destroy (&r->line);
+ ds_destroy (&r->scratch);
+ free (r);
+}
+
+/* Opens the file designated by file handle FH for reading as a data file.
+ Returns a reader if successful, or a null pointer otherwise.
+
+ If FH is fh_inline_file() then the new reader reads data included inline in
+ the command file between BEGIN FILE and END FILE, obtaining data from LEXER.
+ LEXER must remain valid as long as the new reader is in use. ENCODING is
+ ignored.
+
+ If FH is not fh_inline_file(), then the encoding of the file read is by
+ default that of FH itself. If ENCODING is nonnull, then it overrides the
+ default encoding. LEXER is ignored. */
+struct dfm_reader *
+dfm_open_reader (struct file_handle *fh, struct lexer *lexer,
+ const char *encoding)
+{
+ struct dfm_reader *r;
+ struct fh_lock *lock;
+
+ /* TRANSLATORS: this fragment will be interpolated into
+ messages in fh_lock() that identify types of files. */
+ lock = fh_lock (fh, FH_REF_FILE | FH_REF_INLINE, N_("data file"),
+ FH_ACC_READ, false);
+ if (lock == NULL)
+ return NULL;
+
+ r = fh_lock_get_aux (lock);
+ if (r != NULL)
+ return r;
+
+ r = xmalloc (sizeof *r);
+ r->fh = fh_ref (fh);
+ r->lock = lock;
+ r->lexer = lexer;
+ ds_init_empty (&r->line);
+ ds_init_empty (&r->scratch);
+ r->flags = DFM_ADVANCE;
+ r->n_eofs = 0;
+ r->block_left = 0;
+ if (fh_get_referent (fh) != FH_REF_INLINE)
+ {
+ r->line_number = 0;
+ r->file = fn_open (fh, "rb");
+ if (r->file == NULL)
+ {
+ msg (ME, _("Could not open `%s' for reading as a data file: %s."),
+ fh_get_file_name (r->fh), strerror (errno));
+ goto error;
+ }
+ }
+ fh_lock_set_aux (lock, r);
+
+ if (encoding == NULL)
+ encoding = fh_get_encoding (fh);
+ if (fh_get_referent (fh) == FH_REF_FILE && fh_get_mode (fh) == FH_MODE_TEXT)
+ {
+ r->line_reader = line_reader_for_fd (encoding, fileno (r->file));
+ if (r->line_reader == NULL)
+ {
+ msg (ME, _("Could not read `%s' as a text file with encoding `%s': "
+ "%s."),
+ fh_get_file_name (r->fh), encoding, strerror (errno));
+ goto error;
+ }
+ r->encoding = xstrdup (line_reader_get_encoding (r->line_reader));
+ }
+ else
+ {
+ r->line_reader = NULL;
+ r->encoding = xstrdup (encoding_guess_parse_encoding (encoding));
+ }
+
+ return r;
+
+error:
+ fh_unlock (r->lock);
+ fh_unref (fh);
+ free (r);
+ return NULL;
+}
+
+/* Returns true if an I/O error occurred on READER, false otherwise. */
+bool
+dfm_reader_error (const struct dfm_reader *r)
+{
+ return (fh_get_referent (r->fh) == FH_REF_FILE
+ && (r->line_reader != NULL
+ ? line_reader_error (r->line_reader) != 0
+ : ferror (r->file)));
+}
+
+/* Reads a record from the inline file into R.
+ Returns true if successful, false on failure. */
+static bool
+read_inline_record (struct dfm_reader *r)
+{
+ if ((r->flags & DFM_SAW_BEGIN_DATA) == 0)
+ {
+ r->flags |= DFM_SAW_BEGIN_DATA;
+ r->flags &= ~DFM_CONSUME;
+
+ while (lex_token (r->lexer) == T_ENDCMD)
+ lex_get (r->lexer);
+
+ if (!lex_force_match_phrase (r->lexer, "BEGIN DATA"))
+ return false;
+
+ lex_match (r->lexer, T_ENDCMD);
+ }
+
+ if (r->flags & DFM_CONSUME)
+ lex_get (r->lexer);
+
+ if (!lex_is_string (r->lexer))
+ {
+ if (!lex_match_id (r->lexer, "END") || !lex_match_id (r->lexer, "DATA"))
+ {
+ msg (SE, _("Missing %s while reading inline data. "
+ "This probably indicates a missing or incorrectly "
+ "formatted %s command. %s must appear "
+ "by itself on a single line with exactly one space "
+ "between words."), "END DATA", "END DATA", "END DATA");
+ lex_discard_rest_of_command (r->lexer);
+ }
+ return false;
+ }
+
+ ds_assign_substring (&r->line, lex_tokss (r->lexer));
+ r->flags |= DFM_CONSUME;
+
+ return true;
+}
+
+/* Report a read error on R. */
+static void
+read_error (struct dfm_reader *r)
+{
+ msg (ME, _("Error reading file %s: %s."),
+ fh_get_name (r->fh), strerror (errno));
+}
+
+/* Report a partial read at end of file reading R. */
+static void
+partial_record (struct dfm_reader *r)
+{
+ msg (ME, _("Unexpected end of file in partial record reading %s."),
+ fh_get_name (r->fh));
+}
+
+/* Tries to read SIZE bytes from R into BUFFER. Returns 1 if
+ successful, 0 if end of file was reached before any bytes
+ could be read, and -1 if some bytes were read but fewer than
+ SIZE due to end of file or an error mid-read. In the latter
+ case, reports an error. */
+static int
+try_to_read_fully (struct dfm_reader *r, void *buffer, size_t size)
+{
+ size_t bytes_read = fread (buffer, 1, size, r->file);
+ if (bytes_read == size)
+ return 1;
+ else if (bytes_read == 0)
+ return 0;
+ else
+ {
+ partial_record (r);
+ return -1;
+ }
+}
+
+/* Type of a descriptor word. */
+enum descriptor_type
+ {
+ BLOCK,
+ RECORD
+ };
+
+/* Reads a block descriptor word or record descriptor word
+ (according to TYPE) from R. Returns 1 if successful, 0 if
+ end of file was reached before any bytes could be read, -1 if
+ an error occurred. Reports an error in the latter case.
+
+ If successful, stores the number of remaining bytes in the
+ block or record (that is, the block or record length, minus
+ the 4 bytes in the BDW or RDW itself) into *REMAINING_SIZE.
+ If SEGMENT is nonnull, also stores the segment control
+ character (SCC) into *SEGMENT. */
+static int
+read_descriptor_word (struct dfm_reader *r, enum descriptor_type type,
+ size_t *remaining_size, int *segment)
+{
+ uint8_t raw_descriptor[4];
+ int status;
+
+ status = try_to_read_fully (r, raw_descriptor, sizeof raw_descriptor);
+ if (status <= 0)
+ return status;
+
+ *remaining_size = (raw_descriptor[0] << 8) | raw_descriptor[1];
+ if (segment != NULL)
+ *segment = raw_descriptor[2];
+
+ if (*remaining_size < 4)
+ {
+ msg (ME,
+ (type == BLOCK
+ ? _("Corrupt block descriptor word at offset 0x%lx in %s.")
+ : _("Corrupt record descriptor word at offset 0x%lx in %s.")),
+ (long) ftello (r->file) - 4, fh_get_name (r->fh));
+ return -1;
+ }
+
+ *remaining_size -= 4;
+ return 1;
+}
+
+/* Reports that reader R has read a corrupt record size. */
+static void
+corrupt_size (struct dfm_reader *r)
+{
+ msg (ME, _("Corrupt record size at offset 0x%lx in %s."),
+ (long) ftello (r->file) - 4, fh_get_name (r->fh));
+}
+
+/* Reads a 32-byte little-endian signed number from R and stores
+ its value into *SIZE_OUT. Returns 1 if successful, 0 if end
+ of file was reached before any bytes could be read, -1 if an
+ error occurred. Reports an error in the latter case. Numbers
+ less than 0 are considered errors. */
+static int
+read_size (struct dfm_reader *r, size_t *size_out)
+{
+ int32_t size;
+ int status;
+
+ status = try_to_read_fully (r, &size, sizeof size);
+ if (status <= 0)
+ return status;
+
+ integer_convert (INTEGER_LSB_FIRST, &size, INTEGER_NATIVE, &size,
+ sizeof size);
+ if (size < 0)
+ {
+ corrupt_size (r);
+ return -1;
+ }
+
+ *size_out = size;
+ return 1;
+}
+
+static bool
+read_text_record (struct dfm_reader *r)
+{
+ bool is_auto;
+ bool ok;
+
+ /* Read a line. If the line reader's encoding changes, update r->encoding to
+ match. */
+ is_auto = line_reader_is_auto (r->line_reader);
+ ok = line_reader_read (r->line_reader, &r->line, SIZE_MAX);
+ if (is_auto && !line_reader_is_auto (r->line_reader))
+ {
+ free (r->encoding);
+ r->encoding = xstrdup (line_reader_get_encoding (r->line_reader));
+ }
+
+ /* Detect and report read error. */
+ if (!ok)
+ {
+ int error = line_reader_error (r->line_reader);
+ if (error != 0)
+ msg (ME, _("Error reading file %s: %s."),
+ fh_get_name (r->fh), strerror (error));
+ }
+
+ return ok;
+}
+
+/* Reads a record from a disk file into R.
+ Returns true if successful, false on error or at end of file. */
+static bool
+read_file_record (struct dfm_reader *r)
+{
+ assert (r->fh != fh_inline_file ());
+
+ ds_clear (&r->line);
+ switch (fh_get_mode (r->fh))
+ {
+ case FH_MODE_TEXT:
+ return read_text_record (r);
+
+ case FH_MODE_FIXED:
+ if (ds_read_stream (&r->line, 1, fh_get_record_width (r->fh), r->file))
+ return true;
+ else
+ {
+ if (ferror (r->file))
+ read_error (r);
+ else if (!ds_is_empty (&r->line))
+ partial_record (r);
+ return false;
+ }
+
+ case FH_MODE_VARIABLE:
+ {
+ size_t leading_size;
+ size_t trailing_size;
+ int status;
+
+ /* Read leading record size. */
+ status = read_size (r, &leading_size);
+ if (status <= 0)
+ return false;
+
+ /* Read record data. */
+ if (!ds_read_stream (&r->line, leading_size, 1, r->file))
+ {
+ if (ferror (r->file))
+ read_error (r);
+ else
+ partial_record (r);
+ return false;
+ }
+
+ /* Read trailing record size and check that it's the same
+ as the leading record size. */
+ status = read_size (r, &trailing_size);
+ if (status <= 0)
+ {
+ if (status == 0)
+ partial_record (r);
+ return false;
+ }
+ if (leading_size != trailing_size)
+ {
+ corrupt_size (r);
+ return false;
+ }
+
+ return true;
+ }
+
+ case FH_MODE_360_VARIABLE:
+ case FH_MODE_360_SPANNED:
+ for (;;)
+ {
+ size_t record_size;
+ int segment;
+ int status;
+
+ /* If we've exhausted our current block, start another
+ one by reading the new block descriptor word. */
+ if (r->block_left == 0)
+ {
+ status = read_descriptor_word (r, BLOCK, &r->block_left, NULL);
+ if (status < 0)
+ return false;
+ else if (status == 0)
+ return !ds_is_empty (&r->line);
+ }
+
+ /* Read record descriptor. */
+ if (r->block_left < 4)
+ {
+ partial_record (r);
+ return false;
+ }
+ r->block_left -= 4;
+ status = read_descriptor_word (r, RECORD, &record_size, &segment);
+ if (status <= 0)
+ {
+ if (status == 0)
+ partial_record (r);
+ return false;
+ }
+ if (record_size > r->block_left)
+ {
+ msg (ME, _("Record exceeds remaining block length."));
+ return false;
+ }
+
+ /* Read record data. */
+ if (!ds_read_stream (&r->line, record_size, 1, r->file))
+ {
+ if (ferror (r->file))
+ read_error (r);
+ else
+ partial_record (r);
+ return false;
+ }
+ r->block_left -= record_size;
+
+ /* In variable mode, read only a single record.
+ In spanned mode, a segment value of 0 should
+ designate a whole record without spanning, 1 the
+ first segment in a record, 2 the last segment in a
+ record, and 3 an intermediate segment in a record.
+ For compatibility, though, we actually pay attention
+ only to whether the segment value is even or odd. */
+ if (fh_get_mode (r->fh) == FH_MODE_360_VARIABLE
+ || (segment & 1) == 0)
+ return true;
+ }
+ }
+
+ NOT_REACHED ();
+}
+
+/* Reads a record from R, setting the current position to the
+ start of the line. If an error occurs or end-of-file is
+ encountered, the current line is set to null. */
+static bool
+read_record (struct dfm_reader *r)
+{
+ if (fh_get_referent (r->fh) == FH_REF_FILE)
+ {
+ bool ok = read_file_record (r);
+ if (ok)
+ r->line_number++;
+ return ok;
+ }
+ else
+ return read_inline_record (r);
+}
+
+/* Returns the number of attempts, thus far, to advance past
+ end-of-file in reader R. Reads forward in HANDLE's file, if
+ necessary, to find out.
+
+ Normally, the user stops attempting to read from the file the
+ first time EOF is reached (a return value of 1). If the user
+ tries to read past EOF again (a return value of 2 or more),
+ an error message is issued, and the caller should more
+ forcibly abort to avoid an infinite loop. */
+unsigned
+dfm_eof (struct dfm_reader *r)
+{
+ if (r->flags & DFM_ADVANCE)
+ {
+ r->flags &= ~DFM_ADVANCE;
+
+ if (r->n_eofs == 0 && read_record (r))
+ {
+ r->pos = 0;
+ return 0;
+ }
+
+ r->n_eofs++;
+ if (r->n_eofs == 2)
+ {
+ if (r->fh != fh_inline_file ())
+ msg (ME, _("Attempt to read beyond end-of-file on file %s."),
+ fh_get_name (r->fh));
+ else
+ msg (ME, _("Attempt to read beyond %s."), "END DATA");
+ }
+ }
+
+ return r->n_eofs;
+}
+
+/* Returns the current record in the file corresponding to
+ HANDLE. Aborts if reading from the file is necessary or at
+ end of file, so call dfm_eof() first. */
+struct substring
+dfm_get_record (struct dfm_reader *r)
+{
+ assert ((r->flags & DFM_ADVANCE) == 0);
+ assert (r->n_eofs == 0);
+
+ return ds_substr (&r->line, r->pos, SIZE_MAX);
+}
+
+/* Expands tabs in the current line into the equivalent number of
+ spaces, if appropriate for this kind of file. Aborts if
+ reading from the file is necessary or at end of file, so call
+ dfm_eof() first.*/
+void
+dfm_expand_tabs (struct dfm_reader *r)
+{
+ size_t ofs, new_pos, tab_width;
+
+ assert ((r->flags & DFM_ADVANCE) == 0);
+ assert (r->n_eofs == 0);
+
+ if (r->flags & DFM_TABS_EXPANDED)
+ return;
+ r->flags |= DFM_TABS_EXPANDED;
+
+ if (r->fh != fh_inline_file ()
+ && (fh_get_mode (r->fh) != FH_MODE_TEXT
+ || fh_get_tab_width (r->fh) == 0
+ || ds_find_byte (&r->line, '\t') == SIZE_MAX))
+ return;
+
+ /* Expand tabs from r->line into r->scratch, and figure out
+ new value for r->pos. */
+ tab_width = fh_get_tab_width (r->fh);
+ ds_clear (&r->scratch);
+ new_pos = SIZE_MAX;
+ for (ofs = 0; ofs < ds_length (&r->line); ofs++)
+ {
+ unsigned char c;
+
+ if (ofs == r->pos)
+ new_pos = ds_length (&r->scratch);
+
+ c = ds_data (&r->line)[ofs];
+ if (c != '\t')
+ ds_put_byte (&r->scratch, c);
+ else
+ {
+ do
+ ds_put_byte (&r->scratch, ' ');
+ while (ds_length (&r->scratch) % tab_width != 0);
+ }
+ }
+ if (new_pos == SIZE_MAX)
+ {
+ /* Maintain the same relationship between position and line
+ length that we had before. DATA LIST uses a
+ beyond-the-end position to deal with an empty field at
+ the end of the line. */
+ assert (r->pos >= ds_length (&r->line));
+ new_pos = (r->pos - ds_length (&r->line)) + ds_length (&r->scratch);
+ }
+
+ /* Swap r->line and r->scratch and set new r->pos. */
+ ds_swap (&r->line, &r->scratch);
+ r->pos = new_pos;
+}
+
+/* Returns the character encoding of data read from READER. */
+const char *
+dfm_reader_get_encoding (const struct dfm_reader *reader)
+{
+ return reader->encoding;
+}
+
+/* Causes dfm_get_record() or dfm_get_whole_record() to read in
+ the next record the next time it is executed on file
+ HANDLE. */
+void
+dfm_forward_record (struct dfm_reader *r)
+{
+ r->flags |= DFM_ADVANCE;
+}
+
+/* Cancels the effect of any previous dfm_fwd_record() executed
+ on file HANDLE. Sets the current line to begin in the 1-based
+ column COLUMN. */
+void
+dfm_reread_record (struct dfm_reader *r, size_t column)
+{
+ r->flags &= ~DFM_ADVANCE;
+ r->pos = MAX (column, 1) - 1;
+}
+
+/* Sets the current line to begin COLUMNS characters following
+ the current start. */
+void
+dfm_forward_columns (struct dfm_reader *r, size_t columns)
+{
+ dfm_reread_record (r, (r->pos + 1) + columns);
+}
+
+/* Returns the 1-based column to which the line pointer in HANDLE
+ is set. Unless dfm_reread_record() or dfm_forward_columns()
+ have been called, this is 1. */
+size_t
+dfm_column_start (const struct dfm_reader *r)
+{
+ return r->pos + 1;
+}
+
+/* Returns the number of columns we are currently beyond the end
+ of the line. At or before end-of-line, this is 0; one column
+ after end-of-line, this is 1; and so on. */
+size_t
+dfm_columns_past_end (const struct dfm_reader *r)
+{
+ return r->pos < ds_length (&r->line) ? 0 : ds_length (&r->line) - r->pos;
+}
+
+/* Returns the 1-based column within the current line that P
+ designates. */
+size_t
+dfm_get_column (const struct dfm_reader *r, const char *p)
+{
+ return ds_pointer_to_position (&r->line, p) + 1;
+}
+
+const char *
+dfm_get_file_name (const struct dfm_reader *r)
+{
+ enum fh_referent referent = fh_get_referent (r->fh);
+ return (referent == FH_REF_FILE ? fh_get_file_name (r->fh)
+ : referent == FH_REF_INLINE ? lex_get_file_name (r->lexer)
+ : NULL);
+}
+
+int
+dfm_get_line_number (const struct dfm_reader *r)
+{
+ switch (fh_get_referent (r->fh))
+ {
+ case FH_REF_FILE:
+ return r->line_number;
+
+ case FH_REF_INLINE:
+ return lex_ofs_start_point (r->lexer, lex_ofs (r->lexer)).line;
+
+ case FH_REF_DATASET:
+ default:
+ return -1;
+ }
+}
+\f
+/* BEGIN DATA...END DATA procedure. */
+
+/* Perform BEGIN DATA...END DATA as a procedure in itself. */
+int
+cmd_begin_data (struct lexer *lexer, struct dataset *ds)
+{
+ struct dfm_reader *r;
+ bool ok;
+
+ if (!fh_is_locked (fh_inline_file (), FH_ACC_READ))
+ {
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("This command is not valid here since the current "
+ "input program does not access the inline file."));
+ return CMD_CASCADING_FAILURE;
+ }
+ lex_match (lexer, T_ENDCMD);
+
+ /* Open inline file. */
+ r = dfm_open_reader (fh_inline_file (), lexer, NULL);
+ r->flags |= DFM_SAW_BEGIN_DATA;
+ r->flags &= ~DFM_CONSUME;
+
+ /* Input procedure reads from inline file. */
+ casereader_destroy (proc_open (ds));
+ ok = proc_commit (ds);
+ dfm_close_reader (r);
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011, 2012, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef DFM_READ_H
+#define DFM_READ_H
+
+/* Data file manager (dfm).
+
+ This module is in charge of reading and writing data files (other
+ than system files). dfm is an fhuser, so see file-handle.h for the
+ fhuser interface. */
+
+#include <stdbool.h>
+#include <stddef.h>
+
+struct file_handle;
+struct string;
+struct lexer;
+
+/* Input. */
+struct dfm_reader *dfm_open_reader (struct file_handle *, struct lexer *,
+ const char *encoding);
+void dfm_close_reader (struct dfm_reader *);
+bool dfm_reader_error (const struct dfm_reader *);
+unsigned dfm_eof (struct dfm_reader *);
+struct substring dfm_get_record (struct dfm_reader *);
+void dfm_expand_tabs (struct dfm_reader *);
+const char *dfm_reader_get_encoding (const struct dfm_reader *);
+
+/* Line control. */
+void dfm_forward_record (struct dfm_reader *);
+void dfm_reread_record (struct dfm_reader *, size_t column);
+void dfm_forward_columns (struct dfm_reader *, size_t columns);
+size_t dfm_column_start (const struct dfm_reader *);
+size_t dfm_columns_past_end (const struct dfm_reader *);
+size_t dfm_get_column (const struct dfm_reader *, const char *);
+
+/* Information. */
+const char *dfm_get_file_name (const struct dfm_reader *);
+int dfm_get_line_number (const struct dfm_reader *);
+
+#endif /* data-reader.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-2004, 2006, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/data-writer.h"
+
+#include <assert.h>
+#include <errno.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <sys/stat.h>
+
+#include "data/file-name.h"
+#include "data/make-file.h"
+#include "language/commands/file-handle.h"
+#include "libpspp/assertion.h"
+#include "libpspp/encoding-guesser.h"
+#include "libpspp/integer-format.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+/* Data file writer. */
+struct dfm_writer
+ {
+ struct file_handle *fh; /* File handle. */
+ struct fh_lock *lock; /* Exclusive access to file. */
+ FILE *file; /* Associated file. */
+ struct replace_file *rf; /* Atomic file replacement support. */
+ char *encoding; /* Encoding. */
+ enum fh_line_ends line_ends; /* Line ends for text files. */
+
+ int unit; /* Unit width, in bytes. */
+ char cr[MAX_UNIT]; /* \r in encoding, 'unit' bytes long. */
+ char lf[MAX_UNIT]; /* \n in encoding, 'unit' bytes long. */
+ char spaces[32]; /* 32 bytes worth of ' ' in encoding. */
+ };
+
+/* Opens a file handle for writing as a data file.
+
+ The encoding of the file written is by default that of FH itself. If
+ ENCODING is nonnull, then it overrides the default encoding.
+
+ *However*: ENCODING directly affects only text strings written by the data
+ writer code itself, that is, new-lines in FH_MODE_TEXT and space padding in
+ FH_MODE_FIXED mode. The client must do its own encoding translation for the
+ data that it writes. (This is unavoidable because sometimes the data
+ written includes binary data that reencoding would mangle.) The client can
+ obtain the encoding to re-encode into with dfm_writer_get_encoding(). */
+struct dfm_writer *
+dfm_open_writer (struct file_handle *fh, const char *encoding)
+{
+ struct encoding_info ei;
+ struct dfm_writer *w;
+ struct fh_lock *lock;
+ int ofs;
+
+ lock = fh_lock (fh, FH_REF_FILE, N_("data file"), FH_ACC_WRITE, false);
+ if (lock == NULL)
+ return NULL;
+
+ w = fh_lock_get_aux (lock);
+ if (w != NULL)
+ return w;
+
+ encoding = encoding_guess_parse_encoding (encoding != NULL
+ ? encoding
+ : fh_get_encoding (fh));
+ get_encoding_info (&ei, encoding);
+
+ w = xmalloc (sizeof *w);
+ w->fh = fh_ref (fh);
+ w->lock = lock;
+ w->rf = replace_file_start (w->fh, "wb", 0666, &w->file);
+ w->encoding = xstrdup (encoding);
+ w->line_ends = fh_get_line_ends (fh);
+ w->unit = ei.unit;
+ memcpy (w->cr, ei.cr, sizeof w->cr);
+ memcpy (w->lf, ei.lf, sizeof w->lf);
+ for (ofs = 0; ofs + ei.unit <= sizeof w->spaces; ofs += ei.unit)
+ memcpy (&w->spaces[ofs], ei.space, ei.unit);
+
+ if (w->rf == NULL)
+ {
+ msg (ME, _("An error occurred while opening `%s' for writing "
+ "as a data file: %s."),
+ fh_get_file_name (w->fh), strerror (errno));
+ dfm_close_writer (w);
+ return NULL;
+ }
+ fh_lock_set_aux (lock, w);
+
+ return w;
+}
+
+/* Returns true if an I/O error occurred on WRITER, false otherwise. */
+bool
+dfm_write_error (const struct dfm_writer *writer)
+{
+ return ferror (writer->file);
+}
+
+/* Writes record REC (which need not be null-terminated) having
+ length LEN to the file corresponding to HANDLE. Adds any
+ needed formatting, such as a trailing new-line. Returns true
+ on success, false on failure. */
+bool
+dfm_put_record (struct dfm_writer *w, const char *rec, size_t len)
+{
+ assert (w != NULL);
+
+ if (dfm_write_error (w))
+ return false;
+
+ switch (fh_get_mode (w->fh))
+ {
+ case FH_MODE_TEXT:
+ fwrite (rec, len, 1, w->file);
+ if (w->line_ends == FH_END_CRLF)
+ fwrite (w->cr, w->unit, 1, w->file);
+ fwrite (w->lf, w->unit, 1, w->file);
+ break;
+
+ case FH_MODE_FIXED:
+ {
+ size_t record_width = fh_get_record_width (w->fh);
+ size_t write_bytes = MIN (len, record_width);
+ size_t pad_bytes = record_width - write_bytes;
+ fwrite (rec, write_bytes, 1, w->file);
+ while (pad_bytes > 0)
+ {
+ size_t chunk = MIN (pad_bytes, sizeof w->spaces);
+ fwrite (w->spaces, chunk, 1, w->file);
+ pad_bytes -= chunk;
+ }
+ }
+ break;
+
+ case FH_MODE_VARIABLE:
+ {
+ uint32_t size = len;
+ integer_convert (INTEGER_NATIVE, &size, INTEGER_LSB_FIRST, &size,
+ sizeof size);
+ fwrite (&size, sizeof size, 1, w->file);
+ fwrite (rec, len, 1, w->file);
+ fwrite (&size, sizeof size, 1, w->file);
+ }
+ break;
+
+ case FH_MODE_360_VARIABLE:
+ case FH_MODE_360_SPANNED:
+ {
+ size_t ofs = 0;
+ if (fh_get_mode (w->fh) == FH_MODE_360_VARIABLE)
+ len = MIN (65527, len);
+ while (ofs < len)
+ {
+ size_t chunk = MIN (65527, len - ofs);
+ uint32_t bdw = (chunk + 8) << 16;
+ int scc = (ofs == 0 && chunk == len ? 0
+ : ofs == 0 ? 1
+ : ofs + chunk == len ? 2
+ : 3);
+ uint32_t rdw = ((chunk + 4) << 16) | (scc << 8);
+
+ integer_convert (INTEGER_NATIVE, &bdw, INTEGER_MSB_FIRST, &bdw,
+ sizeof bdw);
+ integer_convert (INTEGER_NATIVE, &rdw, INTEGER_MSB_FIRST, &rdw,
+ sizeof rdw);
+ fwrite (&bdw, 1, sizeof bdw, w->file);
+ fwrite (&rdw, 1, sizeof rdw, w->file);
+ fwrite (rec + ofs, 1, chunk, w->file);
+ ofs += chunk;
+ }
+ }
+ break;
+
+ default:
+ NOT_REACHED ();
+ }
+
+ return !dfm_write_error (w);
+}
+
+/* Writes record REC (which need not be null-terminated) having length LEN to
+ the file corresponding to HANDLE. REC is encoded in UTF-8, which this
+ function recodes to the correct encoding for W before writing. Adds any
+ needed formatting, such as a trailing new-line. Returns true on success,
+ false on failure. */
+bool
+dfm_put_record_utf8 (struct dfm_writer *w, const char *rec, size_t len)
+{
+ if (is_encoding_utf8 (w->encoding))
+ return dfm_put_record (w, rec, len);
+ else
+ {
+ char *recoded = recode_string (w->encoding, UTF8, rec, len);
+ bool ok = dfm_put_record (w, recoded, strlen (recoded));
+ free (recoded);
+ return ok;
+ }
+}
+
+/* Closes data file writer W. */
+bool
+dfm_close_writer (struct dfm_writer *w)
+{
+ bool ok;
+
+ if (w == NULL)
+ return true;
+ if (fh_unlock (w->lock))
+ return true;
+
+ ok = true;
+ if (w->file != NULL)
+ {
+ const char *file_name = fh_get_file_name (w->fh);
+ ok = !dfm_write_error (w) && !fn_close (w->fh, w->file);
+
+ if (!ok)
+ msg (ME, _("I/O error occurred writing data file `%s'."), file_name);
+
+ if (ok ? !replace_file_commit (w->rf) : !replace_file_abort (w->rf))
+ ok = false;
+ }
+ fh_unref (w->fh);
+ free (w->encoding);
+ free (w);
+
+ return ok;
+}
+
+/* Returns the encoding of data written to WRITER. */
+const char *
+dfm_writer_get_encoding (const struct dfm_writer *writer)
+{
+ return writer->encoding;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2011, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef DFM_WRITE_H
+#define DFM_WRITE_H
+
+/* Writing data files. */
+
+#include <stdbool.h>
+#include <stddef.h>
+
+struct file_handle;
+struct dfm_writer *dfm_open_writer (struct file_handle *,
+ const char *encoding);
+bool dfm_close_writer (struct dfm_writer *);
+bool dfm_write_error (const struct dfm_writer *);
+bool dfm_put_record (struct dfm_writer *, const char *rec, size_t len);
+bool dfm_put_record_utf8 (struct dfm_writer *, const char *rec, size_t len);
+const char *dfm_writer_get_encoding (const struct dfm_writer *);
+
+#endif /* data-writer.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/command.h"
+
+#include "data/dataset.h"
+#include "data/session.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+\f
+static int
+parse_window (struct lexer *lexer, unsigned int allowed,
+ enum dataset_display def)
+{
+ if (!lex_match_id (lexer, "WINDOW"))
+ return def;
+ lex_match (lexer, T_EQUALS);
+
+ if (allowed & (1 << DATASET_MINIMIZED) && lex_match_id (lexer, "MINIMIZED"))
+ return DATASET_MINIMIZED;
+ else if (allowed & (1 << DATASET_ASIS) && lex_match_id (lexer, "ASIS"))
+ return DATASET_ASIS;
+ else if (allowed & (1 << DATASET_FRONT) && lex_match_id (lexer, "FRONT"))
+ return DATASET_FRONT;
+ else if (allowed & (1 << DATASET_HIDDEN) && lex_match_id (lexer, "HIDDEN"))
+ return DATASET_HIDDEN;
+
+ const char *allowed_s[4];
+ size_t n_allowed = 0;
+ if (allowed & (1 << DATASET_MINIMIZED))
+ allowed_s[n_allowed++] = "MINIMIZED";
+ if (allowed & (1 << DATASET_ASIS))
+ allowed_s[n_allowed++] = "ASIS";
+ if (allowed & (1 << DATASET_FRONT))
+ allowed_s[n_allowed++] = "FRONT";
+ if (allowed & (1 << DATASET_HIDDEN))
+ allowed_s[n_allowed++] = "HIDDEN";
+ lex_error_expecting_array (lexer, allowed_s, n_allowed);
+ return -1;
+}
+
+static struct dataset *
+parse_dataset_name (struct lexer *lexer, struct session *session)
+{
+ if (!lex_force_id (lexer))
+ return NULL;
+
+ struct dataset *ds = session_lookup_dataset (session, lex_tokcstr (lexer));
+ if (ds != NULL)
+ lex_get (lexer);
+ else
+ lex_error (lexer, _("There is no dataset named %s."), lex_tokcstr (lexer));
+ return ds;
+}
+
+int
+cmd_dataset_name (struct lexer *lexer, struct dataset *active)
+{
+ if (!lex_force_id (lexer))
+ return CMD_FAILURE;
+ dataset_set_name (active, lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ int display = parse_window (lexer, (1 << DATASET_ASIS) | (1 << DATASET_FRONT),
+ DATASET_ASIS);
+ if (display < 0)
+ return CMD_FAILURE;
+ else if (display != DATASET_ASIS)
+ dataset_set_display (active, display);
+
+ return CMD_SUCCESS;
+}
+
+int
+cmd_dataset_activate (struct lexer *lexer, struct dataset *active)
+{
+ struct session *session = dataset_session (active);
+ struct dataset *ds;
+ int display;
+
+ ds = parse_dataset_name (lexer, session);
+ if (ds == NULL)
+ return CMD_FAILURE;
+
+ if (ds != active)
+ {
+ proc_execute (active);
+ session_set_active_dataset (session, ds);
+ if (dataset_name (active)[0] == '\0')
+ dataset_destroy (active);
+ return CMD_SUCCESS;
+ }
+
+ display = parse_window (lexer, (1 << DATASET_ASIS) | (1 << DATASET_FRONT),
+ DATASET_ASIS);
+ if (display < 0)
+ return CMD_FAILURE;
+ else if (display != DATASET_ASIS)
+ dataset_set_display (ds, display);
+
+ return CMD_SUCCESS;
+}
+
+int
+cmd_dataset_copy (struct lexer *lexer, struct dataset *old)
+{
+ struct session *session = dataset_session (old);
+
+ /* Parse the entire command first. proc_execute() can attempt to parse
+ BEGIN DATA...END DATA and it will fail confusingly if we are in the
+ middle of the command at the point. */
+ if (!lex_force_id (lexer))
+ return CMD_FAILURE;
+ char *name = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ int display = parse_window (lexer, ((1 << DATASET_MINIMIZED)
+ | (1 << DATASET_HIDDEN)
+ | (1 << DATASET_FRONT)),
+ DATASET_MINIMIZED);
+ if (display < 0)
+ {
+ free (name);
+ return CMD_FAILURE;
+ }
+
+ struct dataset *new;
+ if (session_lookup_dataset (session, name) == old)
+ {
+ new = old;
+ dataset_set_name (old, "");
+ }
+ else
+ {
+ proc_execute (old);
+ new = dataset_clone (old, name);
+ }
+ dataset_set_display (new, display);
+
+ free (name);
+ return CMD_SUCCESS;
+}
+
+int
+cmd_dataset_declare (struct lexer *lexer, struct dataset *ds)
+{
+ struct session *session = dataset_session (ds);
+
+ if (!lex_force_id (lexer))
+ return CMD_FAILURE;
+
+ struct dataset *new = session_lookup_dataset (session, lex_tokcstr (lexer));
+ if (new == NULL)
+ new = dataset_create (session, lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ int display = parse_window (lexer, ((1 << DATASET_MINIMIZED)
+ | (1 << DATASET_HIDDEN)
+ | (1 << DATASET_FRONT)),
+ DATASET_MINIMIZED);
+ if (display < 0)
+ return CMD_FAILURE;
+ dataset_set_display (new, display);
+
+ return CMD_SUCCESS;
+}
+
+static void
+dataset_close_cb (struct dataset *ds, void *session_)
+{
+ struct session *session = session_;
+
+ if (ds != session_active_dataset (session))
+ dataset_destroy (ds);
+}
+
+int
+cmd_dataset_close (struct lexer *lexer, struct dataset *ds)
+{
+ struct session *session = dataset_session (ds);
+
+ if (lex_match (lexer, T_ALL))
+ {
+ session_for_each_dataset (session, dataset_close_cb, session);
+ dataset_set_name (session_active_dataset (session), "");
+ }
+ else
+ {
+ if (!lex_match (lexer, T_ASTERISK))
+ {
+ ds = parse_dataset_name (lexer, session);
+ if (ds == NULL)
+ return CMD_FAILURE;
+ }
+
+ if (ds == session_active_dataset (session))
+ dataset_set_name (ds, "");
+ else
+ dataset_destroy (ds);
+ }
+
+ return CMD_SUCCESS;
+}
+
+static void
+dataset_display_cb (struct dataset *ds, void *p_)
+{
+ struct dataset ***p = p_;
+ **p = ds;
+ (*p)++;
+}
+
+static int
+sort_datasets (const void *a_, const void *b_)
+{
+ struct dataset *const *a = a_;
+ struct dataset *const *b = b_;
+
+ return strcmp (dataset_name (*a), dataset_name (*b));
+}
+
+int
+cmd_dataset_display (struct lexer *lexer UNUSED, struct dataset *ds)
+{
+ struct session *session = dataset_session (ds);
+ size_t n = session_n_datasets (session);
+ struct dataset **datasets = xmalloc (n * sizeof *datasets);
+ struct dataset **p = datasets;
+ session_for_each_dataset (session, dataset_display_cb, &p);
+ qsort (datasets, n, sizeof *datasets, sort_datasets);
+
+ struct pivot_table *table = pivot_table_create (N_("Datasets"));
+
+ struct pivot_dimension *datasets_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dataset"));
+ datasets_dim->hide_all_labels = true;
+
+ for (size_t i = 0; i < n; i++)
+ {
+ struct dataset *ds = datasets[i];
+ const char *name;
+
+ name = dataset_name (ds);
+ if (name[0] == '\0')
+ name = _("unnamed dataset");
+
+ char *text = (ds == session_active_dataset (session)
+ ? xasprintf ("%s (%s)", name, _("active dataset"))
+ : xstrdup (name));
+
+ int dataset_idx = pivot_category_create_leaf (
+ datasets_dim->root, pivot_value_new_integer (i));
+
+ pivot_table_put1 (table, dataset_idx,
+ pivot_value_new_user_text_nocopy (text));
+ }
+
+ free (datasets);
+
+ pivot_table_submit (table);
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2004, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Stub for USE command. */
+int
+cmd_use (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ if (lex_match (lexer, T_ALL))
+ return CMD_SUCCESS;
+
+ lex_msg (lexer, SW, _("Only %s is currently implemented."), "USE ALL");
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2021 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <limits.h>
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/macro.h"
+#include "language/lexer/scan.h"
+#include "language/lexer/token.h"
+#include "libpspp/intern.h"
+#include "libpspp/message.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+static bool
+match_macro_id (struct lexer *lexer, const char *keyword)
+{
+ if (keyword[0] != '!')
+ return lex_match_id (lexer, keyword);
+ else if (lex_token (lexer) == T_MACRO_ID
+ && lex_id_match_n (ss_cstr (keyword), lex_tokss (lexer), 4))
+ {
+ lex_get (lexer);
+ return true;
+ }
+ else
+ return false;
+}
+
+/* Obtains a quoted string from LEXER and then tokenizes the quoted string's
+ content to obtain a single TOKEN. Returns true if successful, false
+ otherwise. The caller takes ownership of TOKEN on success, otherwise TOKEN
+ is indeterminate. */
+static bool
+parse_quoted_token (struct lexer *lexer, struct token *token)
+{
+ if (!lex_force_string (lexer))
+ return false;
+
+ struct substring s = lex_tokss (lexer);
+ struct string_lexer slex;
+ string_lexer_init (&slex, s.string, s.length, SEG_MODE_INTERACTIVE, true);
+ struct token another_token = { .type = T_STOP };
+ if (string_lexer_next (&slex, token) != SLR_TOKEN
+ || string_lexer_next (&slex, &another_token) != SLR_END)
+ {
+ token_uninit (token);
+ token_uninit (&another_token);
+ lex_error (lexer, _("String must contain exactly one token."));
+ return false;
+ }
+ lex_get (lexer);
+ return true;
+}
+
+static bool
+dup_arg_type (struct lexer *lexer, bool *saw_arg_type)
+{
+ if (*saw_arg_type)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Only one of !TOKENS, !CHAREND, !ENCLOSE, or "
+ "!CMDEND is allowed."));
+ return false;
+ }
+ else
+ {
+ *saw_arg_type = true;
+ return true;
+ }
+}
+
+static bool
+parse_macro_body (struct lexer *lexer, struct macro_tokens *mts)
+{
+ *mts = (struct macro_tokens) { .n = 0 };
+ struct string body = DS_EMPTY_INITIALIZER;
+ struct msg_point start = lex_ofs_start_point (lexer, lex_ofs (lexer));
+ while (!match_macro_id (lexer, "!ENDDEFINE"))
+ {
+ if (lex_token (lexer) != T_STRING)
+ {
+ lex_error (lexer,
+ _("Syntax error expecting macro body or !ENDDEFINE."));
+ ds_destroy (&body);
+ return false;
+ }
+
+ ds_put_substring (&body, lex_tokss (lexer));
+ ds_put_byte (&body, '\n');
+ lex_get (lexer);
+ }
+
+ struct segmenter segmenter = segmenter_init (lex_get_syntax_mode (lexer),
+ true);
+ struct substring p = body.ss;
+ bool ok = true;
+ while (p.length > 0)
+ {
+ enum segment_type type;
+ int seg_len = segmenter_push (&segmenter, p.string,
+ p.length, true, &type);
+ assert (seg_len >= 0);
+
+ struct macro_token mt = {
+ .token = { .type = T_STOP },
+ .syntax = ss_head (p, seg_len),
+ };
+ enum tokenize_result result
+ = token_from_segment (type, mt.syntax, &mt.token);
+ ss_advance (&p, seg_len);
+
+ switch (result)
+ {
+ case TOKENIZE_EMPTY:
+ break;
+
+ case TOKENIZE_TOKEN:
+ macro_tokens_add (mts, &mt);
+ break;
+
+ case TOKENIZE_ERROR:
+ {
+ size_t start_offset = mt.syntax.string - body.ss.string;
+ size_t end_offset = start_offset + (mt.syntax.length ? mt.syntax.length - 1 : 0);
+
+ const struct msg_location loc = {
+ .file_name = intern_new_if_nonnull (lex_get_file_name (lexer)),
+ .start = msg_point_advance (start, ss_buffer (body.ss.string, start_offset)),
+ .end = msg_point_advance (start, ss_buffer (body.ss.string, end_offset)),
+ .src = CONST_CAST (struct lex_source *, lex_source (lexer)),
+ };
+ msg_at (SE, &loc, "%s", mt.token.string.string);
+ intern_unref (loc.file_name);
+
+ ok = false;
+ }
+ break;
+ }
+
+ token_uninit (&mt.token);
+ }
+ ds_destroy (&body);
+ return ok;
+}
+
+int
+cmd_define (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ /* Parse macro name.
+
+ The macro name is a T_STRING token, even though it's an identifier,
+ because that's the way that the segmenter prevents it from getting
+ macro-expanded. */
+ if (lex_token (lexer) != T_STRING)
+ {
+ lex_error (lexer, _("Syntax error expecting identifier."));
+ return CMD_FAILURE;
+ }
+ const char *name = lex_tokcstr (lexer);
+ if (!id_is_plausible (name + (name[0] == '!')))
+ {
+ lex_error (lexer, _("Syntax error expecting identifier."));
+ return CMD_FAILURE;
+ }
+
+ struct macro *m = xmalloc (sizeof *m);
+ *m = (struct macro) { .name = xstrdup (name) };
+ struct msg_point macro_start = lex_ofs_start_point (lexer, lex_ofs (lexer));
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ size_t allocated_params = 0;
+ int keyword_ofs = 0;
+ while (!lex_match (lexer, T_RPAREN))
+ {
+ if (m->n_params >= allocated_params)
+ m->params = x2nrealloc (m->params, &allocated_params,
+ sizeof *m->params);
+
+ size_t param_index = m->n_params++;
+ struct macro_param *p = &m->params[param_index];
+ *p = (struct macro_param) { .expand_arg = true };
+
+ /* Parse parameter name. */
+ if (match_macro_id (lexer, "!POSITIONAL"))
+ {
+ if (param_index > 0 && !m->params[param_index - 1].positional)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Positional parameters must precede "
+ "keyword parameters."));
+ lex_ofs_msg (lexer, SN, keyword_ofs, keyword_ofs,
+ _("Here is a previous keyword parameter."));
+ goto error;
+ }
+
+ p->positional = true;
+ p->name = xasprintf ("!%zu", param_index + 1);
+ }
+ else
+ {
+ if (keyword_ofs == 0)
+ keyword_ofs = lex_ofs (lexer);
+ if (lex_token (lexer) == T_MACRO_ID)
+ {
+ lex_error (lexer, _("Keyword macro parameter must be named in "
+ "definition without \"!\" prefix."));
+ goto error;
+ }
+ if (!lex_force_id (lexer))
+ goto error;
+
+ if (is_macro_keyword (lex_tokss (lexer)))
+ {
+ lex_error (lexer, _("Cannot use macro keyword \"%s\" "
+ "as an argument name."),
+ lex_tokcstr (lexer));
+ goto error;
+ }
+
+ p->positional = false;
+ p->name = xasprintf ("!%s", lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ lex_match (lexer, T_EQUALS);
+
+ bool saw_default = false;
+ bool saw_arg_type = false;
+ for (;;)
+ {
+ if (match_macro_id (lexer, "!DEFAULT"))
+ {
+ if (saw_default)
+ {
+ lex_next_error (
+ lexer, -1, -1,
+ _("!DEFAULT is allowed only once per argument."));
+ goto error;
+ }
+ saw_default = true;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ /* XXX Should this handle balanced inner parentheses? */
+ while (!lex_match (lexer, T_RPAREN))
+ {
+ if (lex_token (lexer) == T_ENDCMD)
+ {
+ lex_error_expecting (lexer, ")");
+ goto error;
+ }
+ char *syntax = lex_next_representation (lexer, 0, 0);
+ const struct macro_token mt = {
+ .token = *lex_next (lexer, 0),
+ .syntax = ss_cstr (syntax),
+ };
+ macro_tokens_add (&p->def, &mt);
+ free (syntax);
+
+ lex_get (lexer);
+ }
+ }
+ else if (match_macro_id (lexer, "!NOEXPAND"))
+ p->expand_arg = false;
+ else if (match_macro_id (lexer, "!TOKENS"))
+ {
+ if (!dup_arg_type (lexer, &saw_arg_type)
+ || !lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "!TOKENS", 1, INT_MAX))
+ goto error;
+ p->arg_type = ARG_N_TOKENS;
+ p->n_tokens = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (match_macro_id (lexer, "!CHAREND"))
+ {
+ if (!dup_arg_type (lexer, &saw_arg_type))
+ goto error;
+
+ p->arg_type = ARG_CHAREND;
+
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !parse_quoted_token (lexer, &p->end)
+ || !lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (match_macro_id (lexer, "!ENCLOSE"))
+ {
+ if (!dup_arg_type (lexer, &saw_arg_type))
+ goto error;
+
+ p->arg_type = ARG_ENCLOSE;
+
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !parse_quoted_token (lexer, &p->start)
+ || !lex_force_match (lexer, T_COMMA)
+ || !parse_quoted_token (lexer, &p->end)
+ || !lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (match_macro_id (lexer, "!CMDEND"))
+ {
+ if (!dup_arg_type (lexer, &saw_arg_type))
+ goto error;
+
+ p->arg_type = ARG_CMDEND;
+ }
+ else
+ break;
+ }
+ if (!saw_arg_type)
+ {
+ lex_error_expecting (lexer, "!TOKENS", "!CHAREND", "!ENCLOSE",
+ "!CMDEND");
+ goto error;
+ }
+
+ if (lex_token (lexer) != T_RPAREN && !lex_force_match (lexer, T_SLASH))
+ goto error;
+ }
+
+ if (!parse_macro_body (lexer, &m->body))
+ goto error;
+
+ struct msg_point macro_end = lex_ofs_end_point (lexer, lex_ofs (lexer) - 1);
+ m->location = xmalloc (sizeof *m->location);
+ *m->location = (struct msg_location) {
+ .file_name = intern_new_if_nonnull (lex_get_file_name (lexer)),
+ .start = { .line = macro_start.line },
+ .end = { .line = macro_end.line },
+ };
+
+ lex_define_macro (lexer, m);
+
+ return CMD_SUCCESS;
+
+error:
+ macro_destroy (m);
+ return CMD_FAILURE;
+}
+
+int
+cmd_debug_expand (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ settings_set_mprint (true);
+
+ while (lex_token (lexer) != T_STOP)
+ {
+ if (!lex_next_is_from_macro (lexer, 0) && lex_token (lexer) != T_ENDCMD)
+ {
+ char *rep = lex_next_representation (lexer, 0, 0);
+ msg (MN, "unexpanded token \"%s\"", rep);
+ free (rep);
+ }
+ lex_get (lexer);
+ }
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2007, 2010, 2011, 2013 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Performs DELETE VARIABLES command. */
+int
+cmd_delete_variables (struct lexer *lexer, struct dataset *ds)
+{
+ struct variable **vars;
+ size_t n_vars;
+ bool ok;
+
+ if (proc_make_temporary_transformations_permanent (ds))
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("%s may not be used after %s. "
+ "Temporary transformations will be made permanent."),
+ "DELETE VARIABLES", "TEMPORARY");
+
+ if (!parse_variables (lexer, dataset_dict (ds), &vars, &n_vars, PV_NONE))
+ goto error;
+ if (n_vars == dict_get_n_vars (dataset_dict (ds)))
+ {
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("%s may not be used to delete all variables "
+ "from the active dataset dictionary. "
+ "Use %s instead."), "DELETE VARIABLES", "NEW FILE");
+ goto error;
+ }
+
+ ok = casereader_destroy (proc_open_filtering (ds, false));
+ ok = proc_commit (ds) && ok;
+ if (!ok)
+ goto error;
+
+ dict_delete_vars (dataset_dict (ds), vars, n_vars);
+
+ /* XXX A bunch of bugs conspire to make executing transformations again here
+ necessary, even though it shouldn't be.
+
+ Consider the following (which is included in delete-variables.at):
+
+ DATA LIST NOTABLE /s1 TO s2 1-2(A).
+ BEGIN DATA
+ 12
+ END DATA.
+ DELETE VARIABLES s1.
+ NUMERIC n1.
+ LIST.
+
+ The DATA LIST gives us a caseproto with widths 1,1. DELETE VARIABLES
+ deletes the first variable so we now have -1,1. This already is
+ technically a problem because proc_casereader_read() calls
+ case_unshare_and_resize() from the former to the latter caseproto, and
+ these caseprotos are not conformable (which is a requirement for
+ case_resize()). It doesn't cause an assert by default because
+ case_resize() uses expensive_assert() to check for it though. However, in
+ practice we don't see a problem yet because case_resize() only does work
+ if the number of widths in the source and dest caseproto are different.
+
+ Executing NUMERIC adds a third variable, though, so we have -1,1,0. This
+ makes caseproto_resize() notice that there are fewer strings in the new
+ caseproto. Therefore it destroys the second one (s2). It should destroy
+ the first one (s1), but if the caseprotos were really conformable then it
+ would have destroyed the right one. This mistake eventually causes a bad
+ memory reference.
+
+ Executing transformations a second time after DELETE VARIABLES, like we do
+ below, works around the problem because we can never run into a situation
+ where we've got both new variables (triggering a resize) and deleted
+ variables (triggering the bad free).
+
+ We should fix this in a better way. Doing it cleanly seems hard. This
+ seems to work for now. */
+ ok = casereader_destroy (proc_open_filtering (ds, false));
+ ok = proc_commit (ds) && ok;
+ if (!ok)
+ goto error;
+
+ free (vars);
+
+ return CMD_SUCCESS;
+
+ error:
+ free (vars);
+ return CMD_CASCADING_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-2000, 2009-2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <limits.h>
+#include <math.h>
+#include <stdlib.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/subcase.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "math/moments.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+/* DESCRIPTIVES private data. */
+
+/* Handling of missing values. */
+enum dsc_missing_type
+ {
+ DSC_VARIABLE, /* Handle missing values on a per-variable basis. */
+ DSC_LISTWISE /* Discard entire case if any variable is missing. */
+ };
+
+/* Describes properties of a distribution for the purpose of
+ calculating a Z-score. */
+struct dsc_z_score
+ {
+ const struct variable *src_var; /* Variable on which z-score is based. */
+ struct variable *z_var; /* New z-score variable. */
+ double mean; /* Distribution mean. */
+ double std_dev; /* Distribution standard deviation. */
+ };
+
+/* DESCRIPTIVES transformation (for calculating Z-scores). */
+struct dsc_trns
+ {
+ struct dsc_z_score *z_scores; /* Array of Z-scores. */
+ size_t n_z_scores; /* Number of Z-scores. */
+ const struct variable **vars; /* Variables for listwise missing checks. */
+ size_t n_vars; /* Number of variables. */
+ enum dsc_missing_type missing_type; /* Treatment of missing values. */
+ enum mv_class exclude; /* Classes of missing values to exclude. */
+ const struct variable *filter; /* Dictionary FILTER BY variable. */
+ struct casereader *z_reader; /* Reader for count, mean, stddev. */
+ casenumber count; /* Number left in this SPLIT FILE group.*/
+ bool ok;
+ };
+
+/* Statistics. Used as bit indexes, so must be 32 or fewer. */
+enum dsc_statistic
+ {
+ DSC_MEAN = 0, DSC_SEMEAN, DSC_STDDEV, DSC_VARIANCE, DSC_KURTOSIS,
+ DSC_SEKURT, DSC_SKEWNESS, DSC_SESKEW, DSC_RANGE, DSC_MIN,
+ DSC_MAX, DSC_SUM, DSC_N_STATS,
+
+ /* Only valid as sort criteria. */
+ DSC_NAME = -2, /* Sort by name. */
+ DSC_NONE = -1 /* Unsorted. */
+ };
+
+/* Describes one statistic. */
+struct dsc_statistic_info
+ {
+ const char *identifier; /* Identifier. */
+ const char *name; /* Full name. */
+ enum moment moment; /* Highest moment needed to calculate. */
+ };
+
+/* Table of statistics, indexed by DSC_*. */
+static const struct dsc_statistic_info dsc_info[DSC_N_STATS] =
+ {
+ {"MEAN", N_("Mean"), MOMENT_MEAN},
+ {"SEMEAN", N_("S.E. Mean"), MOMENT_VARIANCE},
+ {"STDDEV", N_("Std Dev"), MOMENT_VARIANCE},
+ {"VARIANCE", N_("Variance"), MOMENT_VARIANCE},
+ {"KURTOSIS", N_("Kurtosis"), MOMENT_KURTOSIS},
+ {"SEKURTOSIS", N_("S.E. Kurt"), MOMENT_NONE},
+ {"SKEWNESS", N_("Skewness"), MOMENT_SKEWNESS},
+ {"SESKEWNESS", N_("S.E. Skew"), MOMENT_NONE},
+ {"RANGE", N_("Range"), MOMENT_NONE},
+ {"MINIMUM", N_("Minimum"), MOMENT_NONE},
+ {"MAXIMUM", N_("Maximum"), MOMENT_NONE},
+ {"SUM", N_("Sum"), MOMENT_MEAN},
+ };
+
+/* Statistics calculated by default if none are explicitly
+ requested. */
+#define DEFAULT_STATS \
+ ((1UL << DSC_MEAN) | (1UL << DSC_STDDEV) | (1UL << DSC_MIN) \
+ | (1UL << DSC_MAX))
+
+/* A variable specified on DESCRIPTIVES. */
+struct dsc_var
+ {
+ const struct variable *v; /* Variable to calculate on. */
+ char *z_name; /* Name for z-score variable. */
+ double valid, missing; /* Valid, missing counts. */
+ struct moments *moments; /* Moments. */
+ double min, max; /* Maximum and mimimum values. */
+ double stats[DSC_N_STATS]; /* All the stats' values. */
+ };
+
+/* A DESCRIPTIVES procedure. */
+struct dsc_proc
+ {
+ /* Per-variable info. */
+ struct dictionary *dict; /* Dictionary. */
+ struct dsc_var *vars; /* Variables. */
+ size_t n_vars; /* Number of variables. */
+
+ /* User options. */
+ enum dsc_missing_type missing_type; /* Treatment of missing values. */
+ enum mv_class exclude; /* Classes of missing values to exclude. */
+
+ /* Accumulated results. */
+ double missing_listwise; /* Sum of weights of cases missing listwise. */
+ double valid; /* Sum of weights of valid cases. */
+ bool bad_warn; /* Warn if bad weight found. */
+ enum dsc_statistic sort_by_stat; /* Statistic to sort by; -1: name. */
+ enum subcase_direction sort_direction;
+ unsigned long show_stats; /* Statistics to display. */
+ unsigned long calc_stats; /* Statistics to calculate. */
+ enum moment max_moment; /* Highest moment needed for stats. */
+
+ /* Z scores. */
+ struct casewriter *z_writer; /* Mean and stddev per SPLIT FILE group. */
+ };
+
+/* Parsing. */
+static enum dsc_statistic match_statistic (struct lexer *);
+static void free_dsc_proc (struct dsc_proc *);
+
+/* Z-score functions. */
+static bool try_name (const struct dictionary *dict,
+ struct dsc_proc *dsc, const char *name);
+static char *generate_z_varname (const struct dictionary *dict,
+ struct dsc_proc *dsc,
+ const char *name, int *n_zs);
+static void dump_z_table (struct dsc_proc *);
+static void setup_z_trns (struct dsc_proc *, struct dataset *);
+
+/* Procedure execution functions. */
+static void calc_descriptives (struct dsc_proc *, struct casereader *,
+ struct dataset *);
+static void display (struct dsc_proc *dsc);
+\f
+/* Parser and outline. */
+
+/* Handles DESCRIPTIVES. */
+int
+cmd_descriptives (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ const struct variable **vars = NULL;
+ size_t n_vars = 0;
+ bool save_z_scores = false;
+ int n_zs = 0;
+
+ /* Create and initialize dsc. */
+ struct dsc_proc *dsc = xmalloc (sizeof *dsc);
+ *dsc = (struct dsc_proc) {
+ .dict = dict,
+ .missing_type = DSC_VARIABLE,
+ .exclude = MV_ANY,
+ .bad_warn = 1,
+ .sort_by_stat = DSC_NONE,
+ .sort_direction = SC_ASCEND,
+ .show_stats = DEFAULT_STATS,
+ .calc_stats = DEFAULT_STATS,
+ };
+
+ /* Parse DESCRIPTIVES. */
+ int z_ofs = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "VARIABLE"))
+ dsc->missing_type = DSC_VARIABLE;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ dsc->missing_type = DSC_LISTWISE;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ dsc->exclude = MV_SYSTEM;
+ else
+ {
+ lex_error_expecting (lexer, "VARIABLE", "LISTWISE",
+ "INCLUDE");
+ goto error;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "SAVE"))
+ {
+ save_z_scores = true;
+ z_ofs = lex_ofs (lexer) - 1;
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "LABELS")
+ || lex_match_id (lexer, "NOLABELS")
+ || lex_match_id (lexer, "INDEX")
+ || lex_match_id (lexer, "NOINDEX")
+ || lex_match_id (lexer, "LINE")
+ || lex_match_id (lexer, "SERIAL"))
+ {
+ /* Ignore. */
+ }
+ else
+ {
+ lex_error_expecting (lexer, "LABELS", "NOLABELS",
+ "INDEX", "NOINDEX", "LINE", "SERIAL");
+ goto error;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ dsc->show_stats = 0;
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match (lexer, T_ALL))
+ dsc->show_stats |= (1UL << DSC_N_STATS) - 1;
+ else if (lex_match_id (lexer, "DEFAULT"))
+ dsc->show_stats |= DEFAULT_STATS;
+ else
+ {
+ enum dsc_statistic s = match_statistic (lexer);
+ if (s == DSC_NONE)
+ goto error;
+ dsc->show_stats |= 1UL << s;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ if (dsc->show_stats == 0)
+ dsc->show_stats = DEFAULT_STATS;
+ }
+ else if (lex_match_id (lexer, "SORT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "NAME"))
+ dsc->sort_by_stat = DSC_NAME;
+ else
+ {
+ dsc->sort_by_stat = match_statistic (lexer);
+ if (dsc->sort_by_stat == DSC_NONE)
+ dsc->sort_by_stat = DSC_MEAN;
+ }
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (lex_match_id (lexer, "A"))
+ dsc->sort_direction = SC_ASCEND;
+ else if (lex_match_id (lexer, "D"))
+ dsc->sort_direction = SC_DESCEND;
+ else
+ {
+ lex_error_expecting (lexer, "A", "D");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+ else if (n_vars == 0)
+ {
+ lex_match_phrase (lexer, "VARIABLES=");
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (!parse_variables_const (lexer, dict, &vars, &n_vars,
+ PV_APPEND | PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+
+ dsc->vars = xnrealloc ((void *)dsc->vars, n_vars, sizeof *dsc->vars);
+ for (size_t i = dsc->n_vars; i < n_vars; i++)
+ dsc->vars[i] = (struct dsc_var) { .v = vars[i] };
+ dsc->n_vars = n_vars;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_id (lexer))
+ goto error;
+ z_ofs = lex_ofs (lexer);
+ if (try_name (dict, dsc, lex_tokcstr (lexer)))
+ {
+ struct dsc_var *dsc_var = &dsc->vars[dsc->n_vars - 1];
+ dsc_var->z_name = xstrdup (lex_tokcstr (lexer));
+ n_zs++;
+ }
+ else
+ lex_error (lexer, _("Z-score variable name %s would be "
+ "a duplicate variable name."),
+ lex_tokcstr (lexer));
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MISSING", "SAVE", "FORMAT", "STATISTICS",
+ "SORT", "VARIABLES");
+ goto error;
+ }
+
+ lex_match (lexer, T_SLASH);
+ }
+ if (n_vars == 0)
+ {
+ msg (SE, _("No variables specified."));
+ goto error;
+ }
+
+ /* Construct z-score varnames, show translation table. */
+ if (n_zs || save_z_scores)
+ {
+ if (save_z_scores)
+ {
+ int n_gens = 0;
+
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dsc_var = &dsc->vars[i];
+ if (dsc_var->z_name == NULL)
+ {
+ const char *name = var_get_name (dsc_var->v);
+ dsc_var->z_name = generate_z_varname (dict, dsc, name,
+ &n_gens);
+ if (dsc_var->z_name == NULL)
+ goto error;
+
+ n_zs++;
+ }
+ }
+ }
+
+ /* It would be better to handle Z scores correctly (however we define
+ that) when TEMPORARY is in effect, but in the meantime this at least
+ prevents a use-after-free error. See bug #38786. */
+ if (proc_make_temporary_transformations_permanent (ds))
+ lex_ofs_msg (lexer, SW, z_ofs, z_ofs,
+ _("DESCRIPTIVES with Z scores ignores TEMPORARY. "
+ "Temporary transformations will be made permanent."));
+
+ struct caseproto *proto = caseproto_create ();
+ for (size_t i = 0; i < 1 + 2 * n_zs; i++)
+ proto = caseproto_add_width (proto, 0);
+ dsc->z_writer = autopaging_writer_create (proto);
+ caseproto_unref (proto);
+
+ dump_z_table (dsc);
+ }
+
+ /* Figure out statistics to display. */
+ if (dsc->show_stats & (1UL << DSC_SKEWNESS))
+ dsc->show_stats |= 1UL << DSC_SESKEW;
+ if (dsc->show_stats & (1UL << DSC_KURTOSIS))
+ dsc->show_stats |= 1UL << DSC_SEKURT;
+
+ /* Figure out which statistics to calculate. */
+ dsc->calc_stats = dsc->show_stats;
+ if (n_zs > 0)
+ dsc->calc_stats |= (1UL << DSC_MEAN) | (1UL << DSC_STDDEV);
+ if (dsc->sort_by_stat >= 0)
+ dsc->calc_stats |= 1UL << dsc->sort_by_stat;
+ if (dsc->show_stats & (1UL << DSC_SESKEW))
+ dsc->calc_stats |= 1UL << DSC_SKEWNESS;
+ if (dsc->show_stats & (1UL << DSC_SEKURT))
+ dsc->calc_stats |= 1UL << DSC_KURTOSIS;
+
+ /* Figure out maximum moment needed and allocate moments for
+ the variables. */
+ dsc->max_moment = MOMENT_NONE;
+ for (size_t i = 0; i < DSC_N_STATS; i++)
+ if (dsc->calc_stats & (1UL << i) && dsc_info[i].moment > dsc->max_moment)
+ dsc->max_moment = dsc_info[i].moment;
+ if (dsc->max_moment != MOMENT_NONE)
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ dsc->vars[i].moments = moments_create (dsc->max_moment);
+
+ /* Data pass. */
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open_filtering (
+ ds, false), dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ calc_descriptives (dsc, group, ds);
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ /* Z-scoring! */
+ if (ok && n_zs)
+ setup_z_trns (dsc, ds);
+
+ /* Done. */
+ free (vars);
+ free_dsc_proc (dsc);
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+
+ error:
+ free (vars);
+ free_dsc_proc (dsc);
+ return CMD_FAILURE;
+}
+
+/* Returns the statistic named by the current token and skips past the token.
+ Returns DSC_NONE if no statistic is given (e.g., subcommand with no
+ specifiers). Emits an error if the current token ID does not name a
+ statistic. */
+static enum dsc_statistic
+match_statistic (struct lexer *lexer)
+{
+ if (lex_token (lexer) == T_ID)
+ {
+ for (enum dsc_statistic stat = 0; stat < DSC_N_STATS; stat++)
+ if (lex_match_id (lexer, dsc_info[stat].identifier))
+ return stat;
+
+ const char *stat_names[DSC_N_STATS];
+ for (enum dsc_statistic stat = 0; stat < DSC_N_STATS; stat++)
+ stat_names[stat] = dsc_info[stat].identifier;
+ lex_error_expecting_array (lexer, stat_names,
+ sizeof stat_names / sizeof *stat_names);
+ lex_get (lexer);
+ }
+
+ return DSC_NONE;
+}
+
+/* Frees DSC. */
+static void
+free_dsc_proc (struct dsc_proc *dsc)
+{
+ if (dsc == NULL)
+ return;
+
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dsc_var = &dsc->vars[i];
+ free (dsc_var->z_name);
+ moments_destroy (dsc_var->moments);
+ }
+ casewriter_destroy (dsc->z_writer);
+ free (dsc->vars);
+ free (dsc);
+}
+\f
+/* Z scores. */
+
+/* Returns false if NAME is a duplicate of any existing variable name or
+ of any previously-declared z-var name; otherwise returns true. */
+static bool
+try_name (const struct dictionary *dict, struct dsc_proc *dsc,
+ const char *name)
+{
+ if (dict_lookup_var (dict, name) != NULL)
+ return false;
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dsc_var = &dsc->vars[i];
+ if (dsc_var->z_name != NULL && !utf8_strcasecmp (dsc_var->z_name, name))
+ return false;
+ }
+ return true;
+}
+
+/* Generates a name for a Z-score variable based on a variable
+ named VAR_NAME, given that *Z_CNT generated variable names are
+ known to already exist. If successful, returns the new name
+ as a dynamically allocated string. On failure, returns NULL. */
+static char *
+generate_z_varname (const struct dictionary *dict, struct dsc_proc *dsc,
+ const char *var_name, int *n_zs)
+{
+ /* Try a name based on the original variable name. */
+ char *z_name = xasprintf ("Z%s", var_name);
+ char *trunc_name = utf8_encoding_trunc (z_name, dict_get_encoding (dict),
+ ID_MAX_LEN);
+ free (z_name);
+ if (try_name (dict, dsc, trunc_name))
+ return trunc_name;
+ free (trunc_name);
+
+ /* Generate a synthetic name. */
+ for (;;)
+ {
+ char name[16];
+
+ (*n_zs)++;
+
+ if (*n_zs <= 99)
+ sprintf (name, "ZSC%03d", *n_zs);
+ else if (*n_zs <= 108)
+ sprintf (name, "STDZ%02d", *n_zs - 99);
+ else if (*n_zs <= 117)
+ sprintf (name, "ZZZZ%02d", *n_zs - 108);
+ else if (*n_zs <= 126)
+ sprintf (name, "ZQZQ%02d", *n_zs - 117);
+ else
+ {
+ msg (SE, _("Ran out of generic names for Z-score variables. "
+ "There are only 126 generic names: ZSC001-ZSC099, "
+ "STDZ01-STDZ09, ZZZZ01-ZZZZ09, ZQZQ01-ZQZQ09."));
+ return NULL;
+ }
+
+ if (try_name (dict, dsc, name))
+ return xstrdup (name);
+ }
+ NOT_REACHED();
+}
+
+/* Outputs a table describing the mapping between source
+ variables and Z-score variables. */
+static void
+dump_z_table (struct dsc_proc *dsc)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Mapping of Variables to Z-scores"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Names"),
+ N_("Source"), N_("Target"));
+
+ struct pivot_dimension *names = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+ names->hide_all_labels = true;
+
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ if (dsc->vars[i].z_name != NULL)
+ {
+ int row = pivot_category_create_leaf (names->root,
+ pivot_value_new_number (i));
+
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_variable (dsc->vars[i].v));
+ pivot_table_put2 (table, 1, row,
+ pivot_value_new_user_text (dsc->vars[i].z_name, -1));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+descriptives_set_all_sysmis_zscores (const struct dsc_trns *t, struct ccase *c)
+{
+ for (const struct dsc_z_score *z = t->z_scores;
+ z < t->z_scores + t->n_z_scores; z++)
+ *case_num_rw (c, z->z_var) = SYSMIS;
+}
+
+/* Transformation function to calculate Z-scores. Will return SYSMIS if any of
+ the following are true: 1) mean or standard deviation is SYSMIS 2) score is
+ SYSMIS 3) score is user missing and they were not included in the original
+ analyis. 4) any of the variables in the original analysis were missing
+ (either system or user-missing values that weren't included).
+*/
+static enum trns_result
+descriptives_trns_proc (void *trns_, struct ccase **c,
+ casenumber case_idx UNUSED)
+{
+ struct dsc_trns *t = trns_;
+
+ *c = case_unshare (*c);
+
+ if (t->filter)
+ {
+ double f = case_num (*c, t->filter);
+ if (f == 0.0 || var_is_num_missing (t->filter, f))
+ {
+ descriptives_set_all_sysmis_zscores (t, *c);
+ return TRNS_CONTINUE;
+ }
+ }
+
+ if (t->count <= 0)
+ {
+ struct ccase *z_case = casereader_read (t->z_reader);
+ if (z_case)
+ {
+ size_t z_idx = 0;
+
+ t->count = case_num_idx (z_case, z_idx++);
+ for (struct dsc_z_score *z = t->z_scores;
+ z < t->z_scores + t->n_z_scores; z++)
+ {
+ z->mean = case_num_idx (z_case, z_idx++);
+ z->std_dev = case_num_idx (z_case, z_idx++);
+ }
+ case_unref (z_case);
+ }
+ else
+ {
+ if (t->ok)
+ {
+ msg (SE, _("Internal error processing Z scores. "
+ "Please report this to %s."),
+ PACKAGE_BUGREPORT);
+ t->ok = false;
+ }
+ descriptives_set_all_sysmis_zscores (t, *c);
+ return TRNS_CONTINUE;
+ }
+ }
+ t->count--;
+
+ if (t->missing_type == DSC_LISTWISE)
+ {
+ assert (t->vars != NULL);
+ for (const struct variable **vars = t->vars; vars < t->vars + t->n_vars;
+ vars++)
+ {
+ double score = case_num (*c, *vars);
+ if (var_is_num_missing (*vars, score) & t->exclude)
+ {
+ descriptives_set_all_sysmis_zscores (t, *c);
+ return TRNS_CONTINUE;
+ }
+ }
+ }
+
+ for (struct dsc_z_score *z = t->z_scores; z < t->z_scores + t->n_z_scores;
+ z++)
+ {
+ double input = case_num (*c, z->src_var);
+ double *output = case_num_rw (*c, z->z_var);
+
+ if (z->mean == SYSMIS || z->std_dev == SYSMIS
+ || var_is_num_missing (z->src_var, input) & t->exclude)
+ *output = SYSMIS;
+ else
+ *output = (input - z->mean) / z->std_dev;
+ }
+ return TRNS_CONTINUE;
+}
+
+/* Frees a descriptives_trns struct. */
+static bool
+descriptives_trns_free (void *trns_)
+{
+ struct dsc_trns *t = trns_;
+ bool ok = t->ok && !casereader_error (t->z_reader);
+
+ free (t->z_scores);
+ casereader_destroy (t->z_reader);
+ assert ((t->missing_type != DSC_LISTWISE) != (t->vars != NULL));
+ free (t->vars);
+ free (t);
+
+ return ok;
+}
+
+static const struct trns_class descriptives_trns_class = {
+ .name = "DESCRIPTIVES (Z scores)",
+ .execute = descriptives_trns_proc,
+ .destroy = descriptives_trns_free,
+};
+
+/* Sets up a transformation to calculate Z scores. */
+static void
+setup_z_trns (struct dsc_proc *dsc, struct dataset *ds)
+{
+ size_t n = 0;
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ if (dsc->vars[i].z_name != NULL)
+ n++;
+
+ struct dsc_trns *t = xmalloc (sizeof *t);
+ *t = (struct dsc_trns) {
+ .z_scores = xmalloc (n * sizeof *t->z_scores),
+ .n_z_scores = n,
+ .missing_type = dsc->missing_type,
+ .exclude = dsc->exclude,
+ .filter = dict_get_filter (dataset_dict (ds)),
+ .z_reader = casewriter_make_reader (dsc->z_writer),
+ .ok = true,
+ };
+ if (t->missing_type == DSC_LISTWISE)
+ {
+ t->n_vars = dsc->n_vars;
+ t->vars = xnmalloc (t->n_vars, sizeof *t->vars);
+ for (size_t i = 0; i < t->n_vars; i++)
+ t->vars[i] = dsc->vars[i].v;
+ }
+ dsc->z_writer = NULL;
+
+ n = 0;
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dv = &dsc->vars[i];
+ if (dv->z_name != NULL)
+ {
+ struct variable *dst_var = dict_create_var_assert (dataset_dict (ds),
+ dv->z_name, 0);
+
+ char *label = xasprintf (_("Z-score of %s"), var_to_string (dv->v));
+ var_set_label (dst_var, label);
+ free (label);
+
+ struct dsc_z_score *z = &t->z_scores[n++];
+ *z = (struct dsc_z_score) {
+ .src_var = dv->v,
+ .z_var = dst_var,
+ };
+ }
+ }
+
+ add_transformation (ds, &descriptives_trns_class, t);
+}
+\f
+/* Statistical calculation. */
+
+static bool listwise_missing (struct dsc_proc *dsc, const struct ccase *c);
+
+/* Calculates and displays descriptive statistics for the cases
+ in CF. */
+static void
+calc_descriptives (struct dsc_proc *dsc, struct casereader *group,
+ struct dataset *ds)
+{
+ output_split_file_values_peek (ds, group);
+ group = casereader_create_filter_weight (group, dataset_dict (ds),
+ NULL, NULL);
+
+ struct casereader *pass1 = group;
+ struct casereader *pass2 = (dsc->max_moment <= MOMENT_MEAN ? NULL
+ : casereader_clone (pass1));
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dv = &dsc->vars[i];
+
+ dv->valid = dv->missing = 0.0;
+ if (dv->moments != NULL)
+ moments_clear (dv->moments);
+ dv->min = DBL_MAX;
+ dv->max = -DBL_MAX;
+ }
+ dsc->missing_listwise = 0.;
+ dsc->valid = 0.;
+
+ /* First pass to handle most of the work. */
+ casenumber count = 0;
+ const struct variable *filter = dict_get_filter (dataset_dict (ds));
+ struct ccase *c;
+ for (; (c = casereader_read (pass1)) != NULL; case_unref (c))
+ {
+ double weight = dict_get_case_weight (dataset_dict (ds), c, NULL);
+
+ if (filter)
+ {
+ double f = case_num (c, filter);
+ if (f == 0.0 || var_is_num_missing (filter, f))
+ continue;
+ }
+
+ /* Check for missing values. */
+ if (listwise_missing (dsc, c))
+ {
+ dsc->missing_listwise += weight;
+ if (dsc->missing_type == DSC_LISTWISE)
+ continue;
+ }
+ dsc->valid += weight;
+
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dv = &dsc->vars[i];
+ double x = case_num (c, dv->v);
+
+ if (var_is_num_missing (dv->v, x) & dsc->exclude)
+ {
+ dv->missing += weight;
+ continue;
+ }
+
+ if (dv->moments != NULL)
+ moments_pass_one (dv->moments, x, weight);
+
+ if (x < dv->min)
+ dv->min = x;
+ if (x > dv->max)
+ dv->max = x;
+ }
+
+ count++;
+ }
+ if (!casereader_destroy (pass1))
+ {
+ casereader_destroy (pass2);
+ return;
+ }
+
+ /* Second pass for higher-order moments. */
+ if (dsc->max_moment > MOMENT_MEAN)
+ {
+ for (; (c = casereader_read (pass2)) != NULL; case_unref (c))
+ {
+ double weight = dict_get_case_weight (dataset_dict (ds), c, NULL);
+
+ if (filter)
+ {
+ double f = case_num (c, filter);
+ if (f == 0.0 || var_is_num_missing (filter, f))
+ continue;
+ }
+
+ /* Check for missing values. */
+ if (dsc->missing_type == DSC_LISTWISE && listwise_missing (dsc, c))
+ continue;
+
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dv = &dsc->vars[i];
+ double x = case_num (c, dv->v);
+
+ if (var_is_num_missing (dv->v, x) & dsc->exclude)
+ continue;
+
+ if (dv->moments != NULL)
+ moments_pass_two (dv->moments, x, weight);
+ }
+ }
+ if (!casereader_destroy (pass2))
+ return;
+ }
+
+ /* Calculate results. */
+ size_t z_idx = 0;
+ if (dsc->z_writer && count > 0)
+ {
+ c = case_create (casewriter_get_proto (dsc->z_writer));
+ *case_num_rw_idx (c, z_idx++) = count;
+ }
+ else
+ c = NULL;
+
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dv = &dsc->vars[i];
+
+ for (size_t j = 0; j < DSC_N_STATS; j++)
+ dv->stats[j] = SYSMIS;
+
+ double W = dsc->valid - dv->missing;
+ dv->valid = W;
+
+ if (dv->moments != NULL)
+ moments_calculate (dv->moments, NULL,
+ &dv->stats[DSC_MEAN], &dv->stats[DSC_VARIANCE],
+ &dv->stats[DSC_SKEWNESS], &dv->stats[DSC_KURTOSIS]);
+ if (dsc->calc_stats & (1UL << DSC_SEMEAN)
+ && dv->stats[DSC_VARIANCE] != SYSMIS && W > 0.)
+ dv->stats[DSC_SEMEAN] = sqrt (dv->stats[DSC_VARIANCE]) / sqrt (W);
+ if (dsc->calc_stats & (1UL << DSC_STDDEV)
+ && dv->stats[DSC_VARIANCE] != SYSMIS)
+ dv->stats[DSC_STDDEV] = sqrt (dv->stats[DSC_VARIANCE]);
+ if (dsc->calc_stats & (1UL << DSC_SEKURT))
+ if (dv->stats[DSC_KURTOSIS] != SYSMIS)
+ dv->stats[DSC_SEKURT] = calc_sekurt (W);
+ if (dsc->calc_stats & (1UL << DSC_SESKEW)
+ && dv->stats[DSC_SKEWNESS] != SYSMIS)
+ dv->stats[DSC_SESKEW] = calc_seskew (W);
+ dv->stats[DSC_RANGE] = ((dv->min == DBL_MAX || dv->max == -DBL_MAX)
+ ? SYSMIS : dv->max - dv->min);
+ dv->stats[DSC_MIN] = dv->min == DBL_MAX ? SYSMIS : dv->min;
+ dv->stats[DSC_MAX] = dv->max == -DBL_MAX ? SYSMIS : dv->max;
+ if (dsc->calc_stats & (1UL << DSC_SUM))
+ dv->stats[DSC_SUM] = W * dv->stats[DSC_MEAN];
+
+ if (dv->z_name && c != NULL)
+ {
+ *case_num_rw_idx (c, z_idx++) = dv->stats[DSC_MEAN];
+ *case_num_rw_idx (c, z_idx++) = dv->stats[DSC_STDDEV];
+ }
+ }
+
+ if (c != NULL)
+ casewriter_write (dsc->z_writer, c);
+
+ /* Output results. */
+ display (dsc);
+}
+
+/* Returns true if any of the descriptives variables in DSC's
+ variable list have missing values in case C, false otherwise. */
+static bool
+listwise_missing (struct dsc_proc *dsc, const struct ccase *c)
+{
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ struct dsc_var *dv = &dsc->vars[i];
+ double x = case_num (c, dv->v);
+
+ if (var_is_num_missing (dv->v, x) & dsc->exclude)
+ return true;
+ }
+ return false;
+}
+\f
+/* Statistical display. */
+
+static algo_compare_func descriptives_compare_dsc_vars;
+
+/* Displays a table of descriptive statistics for DSC. */
+static void
+display (struct dsc_proc *dsc)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Descriptive Statistics"));
+ pivot_table_set_weight_var (table, dict_get_weight (dsc->dict));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ pivot_category_create_leaf_rc (
+ statistics->root, pivot_value_new_text (N_("N")), PIVOT_RC_COUNT);
+ for (int i = 0; i < DSC_N_STATS; i++)
+ if (dsc->show_stats & (1UL << i))
+ pivot_category_create_leaf (statistics->root,
+ pivot_value_new_text (dsc_info[i].name));
+
+ if (dsc->sort_by_stat != DSC_NONE)
+ sort (dsc->vars, dsc->n_vars, sizeof *dsc->vars,
+ descriptives_compare_dsc_vars, dsc);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+ for (size_t i = 0; i < dsc->n_vars; i++)
+ {
+ const struct dsc_var *dv = &dsc->vars[i];
+
+ int row = pivot_category_create_leaf (variables->root,
+ pivot_value_new_variable (dv->v));
+
+ int column = 0;
+ pivot_table_put2 (table, column++, row,
+ pivot_value_new_number (dv->valid));
+
+ for (int j = 0; j < DSC_N_STATS; j++)
+ if (dsc->show_stats & (1UL << j))
+ {
+ union value v = { .f = dv->stats[j] };
+ struct pivot_value *pv = (j == DSC_MIN || j == DSC_MAX
+ ? pivot_value_new_var_value (dv->v, &v)
+ : pivot_value_new_number (dv->stats[j]));
+ pivot_table_put2 (table, column++, row, pv);
+ }
+ }
+
+ int row = pivot_category_create_leaves (
+ variables->root, N_("Valid N (listwise)"), N_("Missing N (listwise)"));
+ pivot_table_put2 (table, 0, row, pivot_value_new_number (dsc->valid));
+ pivot_table_put2 (table, 0, row + 1,
+ pivot_value_new_number (dsc->missing_listwise));
+ pivot_table_submit (table);
+}
+
+/* Compares `struct dsc_var's A and B according to the ordering
+ specified by CMD. */
+static int
+descriptives_compare_dsc_vars (const void *a_, const void *b_, const void *dsc_)
+{
+ const struct dsc_var *a = a_;
+ const struct dsc_var *b = b_;
+ const struct dsc_proc *dsc = dsc_;
+
+ int result;
+
+ if (dsc->sort_by_stat == DSC_NAME)
+ result = utf8_strcasecmp (var_get_name (a->v), var_get_name (b->v));
+ else
+ {
+ double as = a->stats[dsc->sort_by_stat];
+ double bs = b->stats[dsc->sort_by_stat];
+
+ result = as < bs ? -1 : as > bs;
+ }
+
+ if (dsc->sort_direction == SC_DESCEND)
+ result = -result;
+
+ return result;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009-2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/transformations.h"
+#include "language/command.h"
+#include "language/commands/inpt-pgm.h"
+#include "language/expressions/public.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/compiler.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* A conditional clause. */
+struct clause
+ {
+ struct msg_location *location;
+ struct expression *condition; /* Test expression; NULL for ELSE clause. */
+ struct trns_chain xforms;
+ };
+
+/* DO IF transformation. */
+struct do_if_trns
+ {
+ struct clause *clauses; /* Clauses. */
+ size_t n_clauses; /* Number of clauses. */
+
+ const struct trns_chain *resume;
+ size_t ofs;
+ };
+
+static const struct trns_class do_if_trns_class;
+
+static void
+start_clause (struct lexer *lexer, struct dataset *ds,
+ bool condition, struct do_if_trns *do_if,
+ size_t *allocated_clauses, bool *ok)
+{
+ if (*ok && do_if->n_clauses > 0
+ && !do_if->clauses[do_if->n_clauses - 1].condition)
+ {
+ if (condition)
+ lex_ofs_error (lexer, 0, 1,
+ _("ELSE IF is not allowed following ELSE "
+ "within DO IF...END IF."));
+ else
+ lex_ofs_error (lexer, 0, 0,
+ _("Only one ELSE is allowed within DO IF...END IF."));
+
+ msg_at (SN, do_if->clauses[do_if->n_clauses - 1].location,
+ _("This is the location of the previous ELSE clause."));
+
+ msg_at (SN, do_if->clauses[0].location,
+ _("This is the location of the DO IF command."));
+
+ *ok = false;
+ }
+
+ if (do_if->n_clauses >= *allocated_clauses)
+ do_if->clauses = x2nrealloc (do_if->clauses, allocated_clauses,
+ sizeof *do_if->clauses);
+ struct clause *clause = &do_if->clauses[do_if->n_clauses++];
+
+ *clause = (struct clause) { .location = NULL };
+ if (condition)
+ {
+ clause->condition = expr_parse_bool (lexer, ds);
+ if (!clause->condition)
+ lex_discard_rest_of_command (lexer);
+ }
+ clause->location = lex_ofs_location (lexer, 0, lex_ofs (lexer));
+
+ lex_end_of_command (lexer);
+ lex_get (lexer);
+
+ proc_push_transformations (ds);
+}
+
+static void
+finish_clause (struct dataset *ds, struct do_if_trns *do_if)
+{
+ struct clause *clause = &do_if->clauses[do_if->n_clauses - 1];
+ proc_pop_transformations (ds, &clause->xforms);
+}
+
+/* Parse DO IF. */
+int
+cmd_do_if (struct lexer *lexer, struct dataset *ds)
+{
+ struct do_if_trns *do_if = xmalloc (sizeof *do_if);
+ *do_if = (struct do_if_trns) { .n_clauses = 0 };
+
+ size_t allocated_clauses = 0;
+ bool ok = true;
+
+ start_clause (lexer, ds, true, do_if, &allocated_clauses, &ok);
+ while (!lex_match_phrase (lexer, "END IF"))
+ {
+ if (lex_token (lexer) == T_STOP)
+ {
+ lex_error_expecting (lexer, "END IF");
+ break;
+ }
+ else if (lex_match_phrase (lexer, "ELSE IF"))
+ {
+ finish_clause (ds, do_if);
+ start_clause (lexer, ds, true, do_if, &allocated_clauses, &ok);
+ }
+ else if (lex_match_id (lexer, "ELSE"))
+ {
+ finish_clause (ds, do_if);
+ start_clause (lexer, ds, false, do_if, &allocated_clauses, &ok);
+ }
+ else
+ cmd_parse_in_state (lexer, ds,
+ (in_input_program ()
+ ? CMD_STATE_NESTED_INPUT_PROGRAM
+ : CMD_STATE_NESTED_DATA));
+ }
+ finish_clause (ds, do_if);
+
+ add_transformation (ds, &do_if_trns_class, do_if);
+
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+}
+
+int
+cmd_inside_do_if (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("This command cannot appear outside DO IF...END IF."));
+ return CMD_FAILURE;
+}
+
+static const struct trns_chain *
+do_if_find_clause (const struct do_if_trns *do_if,
+ struct ccase *c, casenumber case_num)
+{
+ for (size_t i = 0; i < do_if->n_clauses; i++)
+ {
+ const struct clause *clause = &do_if->clauses[i];
+ if (!clause->condition)
+ return &clause->xforms;
+
+ double boolean = expr_evaluate_num (clause->condition, c, case_num);
+ if (boolean != 0.0)
+ return boolean == SYSMIS ? NULL : &clause->xforms;
+ }
+ return NULL;
+}
+
+static enum trns_result
+do_if_trns_proc (void *do_if_, struct ccase **c, casenumber case_num)
+{
+ struct do_if_trns *do_if = do_if_;
+
+ const struct trns_chain *chain;
+ size_t start;
+ if (do_if->resume)
+ {
+ chain = do_if->resume;
+ start = do_if->ofs;
+ do_if->resume = NULL;
+ do_if->ofs = 0;
+ }
+ else
+ {
+ chain = do_if_find_clause (do_if, *c, case_num);
+ if (!chain)
+ return TRNS_CONTINUE;
+ start = 0;
+ }
+
+ for (size_t i = start; i < chain->n; i++)
+ {
+ const struct transformation *trns = &chain->xforms[i];
+ enum trns_result r = trns->class->execute (trns->aux, c, case_num);
+ switch (r)
+ {
+ case TRNS_CONTINUE:
+ break;
+
+ case TRNS_BREAK:
+ case TRNS_DROP_CASE:
+ case TRNS_ERROR:
+ case TRNS_END_FILE:
+ return r;
+
+ case TRNS_END_CASE:
+ do_if->resume = chain;
+ do_if->ofs = i;
+ return r;
+ }
+ }
+ return TRNS_CONTINUE;
+}
+
+static bool
+do_if_trns_free (void *do_if_)
+{
+ struct do_if_trns *do_if = do_if_;
+
+ for (size_t i = 0; i < do_if->n_clauses; i++)
+ {
+ struct clause *clause = &do_if->clauses[i];
+
+ msg_location_destroy (clause->location);
+ expr_free (clause->condition);
+
+ trns_chain_uninit (&clause->xforms);
+ }
+ free (do_if->clauses);
+ free (do_if);
+ return true;
+}
+
+static const struct trns_class do_if_trns_class = {
+ .name = "DO IF",
+ .execute = do_if_trns_proc,
+ .destroy = do_if_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2005, 2009, 2010 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "output/output-item.h"
+#include "output/driver.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Echos a string to the output stream */
+int
+cmd_echo (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ if (!lex_force_string (lexer))
+ return CMD_FAILURE;
+
+ output_log ("%s", lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/*
+ PSPP - a program for statistical analysis.
+ Copyright (C) 2012, 2013, 2016, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <config.h>
+
+#include <math.h>
+#include <gsl/gsl_cdf.h>
+
+#include "data/casegrouper.h"
+#include "data/caseproto.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+#include "math/box-whisker.h"
+#include "math/categoricals.h"
+#include "math/chart-geometry.h"
+#include "math/histogram.h"
+#include "math/interaction.h"
+#include "math/moments.h"
+#include "math/np.h"
+#include "math/order-stats.h"
+#include "math/percentiles.h"
+#include "math/shapiro-wilk.h"
+#include "math/sort.h"
+#include "math/trimmed-mean.h"
+#include "math/tukey-hinges.h"
+#include "output/charts/boxplot.h"
+#include "output/charts/np-plot.h"
+#include "output/charts/plot-hist.h"
+#include "output/charts/spreadlevel-plot.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+static void
+append_value_name (const struct variable *var, const union value *val, struct string *str)
+{
+ var_append_value_name (var, val, str);
+ if (var_is_value_missing (var, val))
+ ds_put_cstr (str, _(" (missing)"));
+}
+
+enum bp_mode
+ {
+ BP_GROUPS,
+ BP_VARIABLES
+ };
+
+/* Indices for the ex_proto member (below) */
+enum
+ {
+ EX_VAL, /* value */
+ EX_ID, /* identity */
+ EX_WT /* weight */
+ };
+
+
+struct examine
+{
+ struct pool *pool;
+
+ /* A caseproto used to contain the data subsets under examination,
+ see (enum above) */
+ struct caseproto *ex_proto;
+
+ size_t n_dep_vars;
+ const struct variable **dep_vars;
+
+ size_t n_iacts;
+ struct interaction **iacts;
+
+ enum mv_class dep_excl;
+ enum mv_class fctr_excl;
+
+ const struct dictionary *dict;
+
+ struct categoricals *cats;
+
+ /* how many extremities to display */
+ int disp_extremes;
+ int calc_extremes;
+ bool descriptives;
+
+ double conf;
+
+ bool missing_pw;
+
+ /* The case index of the ID value (or -1) if not applicable */
+ size_t id_idx;
+ int id_width;
+
+ enum pc_alg pc_alg;
+ double *ptiles;
+ size_t n_percentiles;
+
+ bool plot_histogram;
+ bool plot_boxplot;
+ bool plot_npplot;
+ bool plot_spreadlevel;
+ float sl_power;
+
+ enum bp_mode boxplot_mode;
+
+ const struct variable *id_var;
+
+ const struct variable *wv;
+};
+
+struct extremity
+{
+ /* The value of this extremity */
+ double val;
+
+ /* Either the casenumber or the value of the variable specified
+ by the /ID subcommand which corresponds to this extremity */
+ union value identity;
+};
+
+struct exploratory_stats
+{
+ double missing;
+ double non_missing;
+
+ struct moments *mom;
+
+ /* Most operations need a sorted reader/writer */
+ struct casewriter *sorted_writer;
+ struct casereader *sorted_reader;
+
+ struct extremity *minima;
+ struct extremity *maxima;
+
+ /*
+ Minimum should alway equal mimima[0].val.
+ Likewise, maximum should alway equal maxima[0].val.
+ This redundancy exists as an optimisation effort.
+ Some statistics (eg histogram) require early calculation
+ of the min and max
+ */
+ double minimum;
+ double maximum;
+
+ struct trimmed_mean *trimmed_mean;
+ struct percentile *quartiles[3];
+ struct percentile **percentiles;
+ struct shapiro_wilk *shapiro_wilk;
+
+ struct tukey_hinges *hinges;
+
+ /* The data for the NP Plots */
+ struct np *np;
+
+ struct histogram *histogram;
+
+ /* The data for the box plots */
+ struct box_whisker *box_whisker;
+
+ /* Total weight */
+ double cc;
+
+ /* The minimum weight */
+ double cmin;
+};
+
+static void
+show_boxplot_grouped (const struct examine *cmd, int iact_idx)
+{
+ int v;
+
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ double y_min = DBL_MAX;
+ double y_max = -DBL_MAX;
+ int grp;
+ struct boxplot *boxplot;
+ struct string title;
+ ds_init_empty (&title);
+
+ if (iact->n_vars > 0)
+ {
+ struct string istr;
+ ds_init_empty (&istr);
+ interaction_to_string (iact, &istr);
+ ds_put_format (&title, _("Boxplot of %s vs. %s"),
+ var_to_string (cmd->dep_vars[v]),
+ ds_cstr (&istr));
+ ds_destroy (&istr);
+ }
+ else
+ ds_put_format (&title, _("Boxplot of %s"), var_to_string (cmd->dep_vars[v]));
+
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ const struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ if (y_min > es[v].minimum)
+ y_min = es[v].minimum;
+
+ if (y_max < es[v].maximum)
+ y_max = es[v].maximum;
+ }
+
+ boxplot = boxplot_create (y_min, y_max, ds_cstr (&title));
+
+ ds_destroy (&title);
+
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ int ivar_idx;
+ struct string label;
+
+ const struct ccase *c =
+ categoricals_get_case_by_category_real (cmd->cats, iact_idx, grp);
+
+ struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ ds_init_empty (&label);
+ for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
+ {
+ struct string l;
+ const struct variable *ivar = iact->vars[ivar_idx];
+ const union value *val = case_data (c, ivar);
+ ds_init_empty (&l);
+
+ append_value_name (ivar, val, &l);
+ ds_ltrim (&l, ss_cstr (" "));
+
+ ds_put_substring (&label, l.ss);
+ if (ivar_idx < iact->n_vars - 1)
+ ds_put_cstr (&label, "; ");
+
+ ds_destroy (&l);
+ }
+
+ boxplot_add_box (boxplot, es[v].box_whisker, ds_cstr (&label));
+ es[v].box_whisker = NULL;
+
+ ds_destroy (&label);
+ }
+
+ boxplot_submit (boxplot);
+ }
+}
+
+static void
+show_boxplot_variabled (const struct examine *cmd, int iact_idx)
+{
+ int grp;
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ struct boxplot *boxplot;
+ int v;
+ double y_min = DBL_MAX;
+ double y_max = -DBL_MAX;
+
+ const struct ccase *c =
+ categoricals_get_case_by_category_real (cmd->cats, iact_idx, grp);
+
+ struct string title;
+ ds_init_empty (&title);
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ const struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ if (y_min > es[v].minimum)
+ y_min = es[v].minimum;
+
+ if (y_max < es[v].maximum)
+ y_max = es[v].maximum;
+ }
+
+ if (iact->n_vars == 0)
+ ds_put_format (&title, _("Boxplot"));
+ else
+ {
+ int ivar_idx;
+ struct string label;
+ ds_init_empty (&label);
+ for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
+ {
+ const struct variable *ivar = iact->vars[ivar_idx];
+ const union value *val = case_data (c, ivar);
+
+ ds_put_cstr (&label, var_to_string (ivar));
+ ds_put_cstr (&label, " = ");
+ append_value_name (ivar, val, &label);
+ ds_put_cstr (&label, "; ");
+ }
+
+ ds_put_format (&title, _("Boxplot of %s"),
+ ds_cstr (&label));
+
+ ds_destroy (&label);
+ }
+
+ boxplot = boxplot_create (y_min, y_max, ds_cstr (&title));
+
+ ds_destroy (&title);
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ boxplot_add_box (boxplot, es[v].box_whisker,
+ var_to_string (cmd->dep_vars[v]));
+ es[v].box_whisker = NULL;
+ }
+
+ boxplot_submit (boxplot);
+ }
+}
+
+
+static void
+show_npplot (const struct examine *cmd, int iact_idx)
+{
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+
+ int v;
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ int grp;
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ struct chart *npp, *dnpp;
+ struct casereader *reader;
+ struct np *np;
+
+ int ivar_idx;
+ const struct ccase *c =
+ categoricals_get_case_by_category_real (cmd->cats,
+ iact_idx, grp);
+
+ const struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ struct string label;
+ ds_init_cstr (&label,
+ var_to_string (cmd->dep_vars[v]));
+
+ if (iact->n_vars > 0)
+ {
+ ds_put_cstr (&label, " (");
+ for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
+ {
+ const struct variable *ivar = iact->vars[ivar_idx];
+ const union value *val = case_data (c, ivar);
+
+ ds_put_cstr (&label, var_to_string (ivar));
+ ds_put_cstr (&label, " = ");
+ append_value_name (ivar, val, &label);
+ ds_put_cstr (&label, "; ");
+
+ }
+ ds_put_cstr (&label, ")");
+ }
+
+ np = es[v].np;
+ reader = casewriter_make_reader (np->writer);
+ np->writer = NULL;
+
+ npp = np_plot_create (np, reader, ds_cstr (&label));
+ dnpp = dnp_plot_create (np, reader, ds_cstr (&label));
+
+ if (npp == NULL || dnpp == NULL)
+ {
+ msg (MW, _("Not creating NP plot because data set is empty."));
+ chart_unref (npp);
+ chart_unref (dnpp);
+ }
+ else
+ {
+ chart_submit (npp);
+ chart_submit (dnpp);
+ }
+ casereader_destroy (reader);
+
+ ds_destroy (&label);
+ }
+ }
+}
+
+static void
+show_spreadlevel (const struct examine *cmd, int iact_idx)
+{
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+
+ int v;
+
+ /* Spreadlevel when there are no levels is not useful */
+ if (iact->n_vars == 0)
+ return;
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ int grp;
+ struct chart *sl;
+
+ struct string label;
+ ds_init_cstr (&label,
+ var_to_string (cmd->dep_vars[v]));
+
+ if (iact->n_vars > 0)
+ {
+ ds_put_cstr (&label, " (");
+ interaction_to_string (iact, &label);
+ ds_put_cstr (&label, ")");
+ }
+
+ sl = spreadlevel_plot_create (ds_cstr (&label), cmd->sl_power);
+
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ const struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ double median = percentile_calculate (es[v].quartiles[1], cmd->pc_alg);
+
+ double iqr = percentile_calculate (es[v].quartiles[2], cmd->pc_alg) -
+ percentile_calculate (es[v].quartiles[0], cmd->pc_alg);
+
+ spreadlevel_plot_add (sl, iqr, median);
+ }
+
+ if (sl == NULL)
+ msg (MW, _("Not creating spreadlevel chart for %s"), ds_cstr (&label));
+ else
+ chart_submit (sl);
+
+ ds_destroy (&label);
+ }
+}
+
+
+static void
+show_histogram (const struct examine *cmd, int iact_idx)
+{
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+
+ int v;
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ int grp;
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ double n, mean, var;
+ int ivar_idx;
+ const struct ccase *c =
+ categoricals_get_case_by_category_real (cmd->cats,
+ iact_idx, grp);
+
+ const struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
+
+ struct string label;
+
+ if (es[v].histogram == NULL)
+ continue;
+
+ ds_init_cstr (&label,
+ var_to_string (cmd->dep_vars[v]));
+
+ if (iact->n_vars > 0)
+ {
+ ds_put_cstr (&label, " (");
+ for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
+ {
+ const struct variable *ivar = iact->vars[ivar_idx];
+ const union value *val = case_data (c, ivar);
+
+ ds_put_cstr (&label, var_to_string (ivar));
+ ds_put_cstr (&label, " = ");
+ append_value_name (ivar, val, &label);
+ ds_put_cstr (&label, "; ");
+
+ }
+ ds_put_cstr (&label, ")");
+ }
+
+
+ moments_calculate (es[v].mom, &n, &mean, &var, NULL, NULL);
+
+ chart_submit
+ (histogram_chart_create (es[v].histogram->gsl_hist,
+ ds_cstr (&label), n, mean,
+ sqrt (var), false));
+
+
+ ds_destroy (&label);
+ }
+ }
+}
+
+static struct pivot_value *
+new_value_with_missing_footnote (const struct variable *var,
+ const union value *value,
+ struct pivot_footnote *missing_footnote)
+{
+ struct pivot_value *pv = pivot_value_new_var_value (var, value);
+ if (var_is_value_missing (var, value) == MV_USER)
+ pivot_value_add_footnote (pv, missing_footnote);
+ return pv;
+}
+
+static void
+create_interaction_dimensions (struct pivot_table *table,
+ const struct categoricals *cats,
+ const struct interaction *iact,
+ struct pivot_footnote *missing_footnote)
+{
+ for (size_t i = iact->n_vars; i-- > 0;)
+ {
+ const struct variable *var = iact->vars[i];
+ struct pivot_dimension *d = pivot_dimension_create__ (
+ table, PIVOT_AXIS_ROW, pivot_value_new_variable (var));
+ d->root->show_label = true;
+
+ size_t n;
+ union value *values = categoricals_get_var_values (cats, var, &n);
+ for (size_t j = 0; j < n; j++)
+ pivot_category_create_leaf (
+ d->root, new_value_with_missing_footnote (var, &values[j],
+ missing_footnote));
+ }
+}
+
+static struct pivot_footnote *
+create_missing_footnote (struct pivot_table *table)
+{
+ return pivot_table_create_footnote (
+ table, pivot_value_new_text (N_("User-missing value.")));
+}
+
+static void
+percentiles_report (const struct examine *cmd, int iact_idx)
+{
+ struct pivot_table *table = pivot_table_create (N_("Percentiles"));
+
+ struct pivot_dimension *percentiles = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Percentiles"));
+ percentiles->root->show_label = true;
+ for (int i = 0; i < cmd->n_percentiles; ++i)
+ pivot_category_create_leaf (
+ percentiles->root,
+ pivot_value_new_user_text_nocopy (xasprintf ("%g", cmd->ptiles[i])));
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Weighted Average"), N_("Tukey's Hinges"));
+
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ struct pivot_footnote *missing_footnote = create_missing_footnote (table);
+ create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
+
+ struct pivot_dimension *dep_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
+
+ size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
+ dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
+
+ for (size_t i = 0; i < n_cats; ++i)
+ {
+ for (size_t j = 0; j < iact->n_vars; j++)
+ {
+ int idx = categoricals_get_value_index_by_category_real (
+ cmd->cats, iact_idx, i, j);
+ indexes[table->n_dimensions - 2 - j] = idx;
+ }
+
+ const struct exploratory_stats *ess
+ = categoricals_get_user_data_by_category_real (
+ cmd->cats, iact_idx, i);
+ const struct exploratory_stats *es = ess + v;
+
+ double hinges[3];
+ tukey_hinges_calculate (es->hinges, hinges);
+
+ for (size_t pc_idx = 0; pc_idx < cmd->n_percentiles; ++pc_idx)
+ {
+ indexes[0] = pc_idx;
+
+ indexes[1] = 0;
+ double value = percentile_calculate (es->percentiles[pc_idx],
+ cmd->pc_alg);
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (value));
+
+ double hinge = (cmd->ptiles[pc_idx] == 25.0 ? hinges[0]
+ : cmd->ptiles[pc_idx] == 50.0 ? hinges[1]
+ : cmd->ptiles[pc_idx] == 75.0 ? hinges[2]
+ : SYSMIS);
+ if (hinge != SYSMIS)
+ {
+ indexes[1] = 1;
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (hinge));
+ }
+ }
+ }
+
+ }
+ free (indexes);
+
+ pivot_table_submit (table);
+}
+
+static void
+normality_report (const struct examine *cmd, int iact_idx)
+{
+ struct pivot_table *table = pivot_table_create (N_("Tests of Normality"));
+
+ struct pivot_dimension *test =
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Shapiro-Wilk"),
+ N_("Statistic"),
+ N_("df"), PIVOT_RC_COUNT,
+ N_("Sig."));
+
+ test->root->show_label = true;
+
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ struct pivot_footnote *missing_footnote = create_missing_footnote (table);
+ create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
+
+ struct pivot_dimension *dep_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
+
+ size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ indexes[table->n_dimensions - 1] =
+ pivot_category_create_leaf (dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
+
+ for (size_t i = 0; i < n_cats; ++i)
+ {
+ indexes[1] = i;
+
+ const struct exploratory_stats *es
+ = categoricals_get_user_data_by_category_real (
+ cmd->cats, iact_idx, i);
+
+ struct shapiro_wilk *sw = es[v].shapiro_wilk;
+
+ if (sw == NULL)
+ continue;
+
+ double w = shapiro_wilk_calculate (sw);
+
+ int j = 0;
+ indexes[0] = j;
+
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (w));
+
+ indexes[0] = ++j;
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (sw->n));
+
+ indexes[0] = ++j;
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (shapiro_wilk_significance (sw->n, w)));
+ }
+ }
+
+ free (indexes);
+
+ pivot_table_submit (table);
+}
+
+
+static void
+descriptives_report (const struct examine *cmd, int iact_idx)
+{
+ struct pivot_table *table = pivot_table_create (N_("Descriptives"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Aspect"),
+ N_("Statistic"), N_("Std. Error"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"), N_("Mean"));
+ struct pivot_category *interval = pivot_category_create_group__ (
+ statistics->root,
+ pivot_value_new_text_format (N_("%g%% Confidence Interval for Mean"),
+ cmd->conf * 100.0));
+ pivot_category_create_leaves (interval, N_("Lower Bound"),
+ N_("Upper Bound"));
+ pivot_category_create_leaves (
+ statistics->root, N_("5% Trimmed Mean"), N_("Median"), N_("Variance"),
+ N_("Std. Deviation"), N_("Minimum"), N_("Maximum"), N_("Range"),
+ N_("Interquartile Range"), N_("Skewness"), N_("Kurtosis"));
+
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ struct pivot_footnote *missing_footnote = create_missing_footnote (table);
+ create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
+
+ struct pivot_dimension *dep_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
+
+ size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
+ dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
+
+ for (size_t i = 0; i < n_cats; ++i)
+ {
+ for (size_t j = 0; j < iact->n_vars; j++)
+ {
+ int idx = categoricals_get_value_index_by_category_real (
+ cmd->cats, iact_idx, i, j);
+ indexes[table->n_dimensions - 2 - j] = idx;
+ }
+
+ const struct exploratory_stats *ess
+ = categoricals_get_user_data_by_category_real (cmd->cats,
+ iact_idx, i);
+ const struct exploratory_stats *es = ess + v;
+
+ double m0, m1, m2, m3, m4;
+ moments_calculate (es->mom, &m0, &m1, &m2, &m3, &m4);
+ double tval = gsl_cdf_tdist_Qinv ((1.0 - cmd->conf) / 2.0, m0 - 1.0);
+
+ struct entry
+ {
+ int stat_idx;
+ int aspect_idx;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, m1 },
+ { 0, 1, calc_semean (m2, m0) },
+ { 1, 0, m1 - tval * calc_semean (m2, m0) },
+ { 2, 0, m1 + tval * calc_semean (m2, m0) },
+ { 3, 0, trimmed_mean_calculate (es->trimmed_mean) },
+ { 4, 0, percentile_calculate (es->quartiles[1], cmd->pc_alg) },
+ { 5, 0, m2 },
+ { 6, 0, sqrt (m2) },
+ { 7, 0, es->minima[0].val },
+ { 8, 0, es->maxima[0].val },
+ { 9, 0, es->maxima[0].val - es->minima[0].val },
+ { 10, 0, (percentile_calculate (es->quartiles[2], cmd->pc_alg) -
+ percentile_calculate (es->quartiles[0], cmd->pc_alg)) },
+ { 11, 0, m3 },
+ { 11, 1, calc_seskew (m0) },
+ { 12, 0, m4 },
+ { 12, 1, calc_sekurt (m0) },
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ {
+ const struct entry *e = &entries[j];
+ indexes[0] = e->aspect_idx;
+ indexes[1] = e->stat_idx;
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (e->x));
+ }
+ }
+ }
+
+ free (indexes);
+
+ pivot_table_submit (table);
+}
+
+
+static void
+extremes_report (const struct examine *cmd, int iact_idx)
+{
+ struct pivot_table *table = pivot_table_create (N_("Extreme Values"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ pivot_category_create_leaf (statistics->root,
+ (cmd->id_var
+ ? pivot_value_new_variable (cmd->id_var)
+ : pivot_value_new_text (N_("Case Number"))));
+ pivot_category_create_leaves (statistics->root, N_("Value"));
+
+ struct pivot_dimension *order = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Order"));
+ for (size_t i = 0; i < cmd->disp_extremes; i++)
+ pivot_category_create_leaf (order->root, pivot_value_new_integer (i + 1));
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW,
+ /* TRANSLATORS: This is a noun, not an adjective. */
+ N_("Extreme"),
+ N_("Highest"), N_("Lowest"));
+
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ struct pivot_footnote *missing_footnote = create_missing_footnote (table);
+ create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
+
+ struct pivot_dimension *dep_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
+
+ size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
+ dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
+
+ for (size_t i = 0; i < n_cats; ++i)
+ {
+ for (size_t j = 0; j < iact->n_vars; j++)
+ {
+ int idx = categoricals_get_value_index_by_category_real (
+ cmd->cats, iact_idx, i, j);
+ indexes[table->n_dimensions - 2 - j] = idx;
+ }
+
+ const struct exploratory_stats *ess
+ = categoricals_get_user_data_by_category_real (cmd->cats,
+ iact_idx, i);
+ const struct exploratory_stats *es = ess + v;
+
+ for (int e = 0; e < cmd->disp_extremes; ++e)
+ {
+ indexes[1] = e;
+
+ for (size_t j = 0; j < 2; j++)
+ {
+ const struct extremity *extremity
+ = j ? &es->minima[e] : &es->maxima[e];
+ indexes[2] = j;
+
+ indexes[0] = 0;
+ pivot_table_put (
+ table, indexes, table->n_dimensions,
+ (cmd->id_var
+ ? new_value_with_missing_footnote (cmd->id_var,
+ &extremity->identity,
+ missing_footnote)
+ : pivot_value_new_integer (extremity->identity.f)));
+
+ indexes[0] = 1;
+ union value val = { .f = extremity->val };
+ pivot_table_put (
+ table, indexes, table->n_dimensions,
+ new_value_with_missing_footnote (cmd->dep_vars[v], &val,
+ missing_footnote));
+ }
+ }
+ }
+ }
+ free (indexes);
+
+ pivot_table_submit (table);
+}
+
+
+static void
+summary_report (const struct examine *cmd, int iact_idx)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Case Processing Summary"));
+ pivot_table_set_weight_var (table, dict_get_weight (cmd->dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Percent"), PIVOT_RC_PERCENT);
+ struct pivot_dimension *cases = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Cases"), N_("Valid"), N_("Missing"),
+ N_("Total"));
+ cases->root->show_label = true;
+
+ const struct interaction *iact = cmd->iacts[iact_idx];
+ struct pivot_footnote *missing_footnote = create_missing_footnote (table);
+ create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
+
+ struct pivot_dimension *dep_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
+
+ size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
+ dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
+
+ for (size_t i = 0; i < n_cats; ++i)
+ {
+ for (size_t j = 0; j < iact->n_vars; j++)
+ {
+ int idx = categoricals_get_value_index_by_category_real (
+ cmd->cats, iact_idx, i, j);
+ indexes[table->n_dimensions - 2 - j] = idx;
+ }
+
+ const struct exploratory_stats *es
+ = categoricals_get_user_data_by_category_real (
+ cmd->cats, iact_idx, i);
+
+ double total = es[v].missing + es[v].non_missing;
+ struct entry
+ {
+ int stat_idx;
+ int case_idx;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, es[v].non_missing },
+ { 1, 0, 100.0 * es[v].non_missing / total },
+ { 0, 1, es[v].missing },
+ { 1, 1, 100.0 * es[v].missing / total },
+ { 0, 2, total },
+ { 1, 2, 100.0 },
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ {
+ const struct entry *e = &entries[j];
+ indexes[0] = e->stat_idx;
+ indexes[1] = e->case_idx;
+ pivot_table_put (table, indexes, table->n_dimensions,
+ pivot_value_new_number (e->x));
+ }
+ }
+ }
+
+ free (indexes);
+
+ pivot_table_submit (table);
+}
+
+/* Attempt to parse an interaction from LEXER */
+static struct interaction *
+parse_interaction (struct lexer *lexer, struct examine *ex)
+{
+ const struct variable *v;
+ if (!lex_match_variable (lexer, ex->dict, &v))
+ return NULL;
+
+ struct interaction *iact = interaction_create (v);
+ while (lex_match (lexer, T_BY))
+ {
+ if (!lex_match_variable (lexer, ex->dict, &v))
+ {
+ interaction_destroy (iact);
+ return NULL;
+ }
+ interaction_add_variable (iact, v);
+ }
+ lex_match (lexer, T_COMMA);
+ return iact;
+}
+
+
+static void *
+create_n (const void *aux1, void *aux2 UNUSED)
+{
+ int v;
+
+ const struct examine *examine = aux1;
+ struct exploratory_stats *es = pool_calloc (examine->pool, examine->n_dep_vars, sizeof (*es));
+ struct subcase ordering;
+ subcase_init (&ordering, 0, 0, SC_ASCEND);
+
+ for (v = 0; v < examine->n_dep_vars; v++)
+ {
+ es[v].sorted_writer = sort_create_writer (&ordering, examine->ex_proto);
+ es[v].sorted_reader = NULL;
+
+ es[v].mom = moments_create (MOMENT_KURTOSIS);
+ es[v].cmin = DBL_MAX;
+
+ es[v].maximum = -DBL_MAX;
+ es[v].minimum = DBL_MAX;
+ }
+
+ subcase_uninit (&ordering);
+ return es;
+}
+
+static void
+update_n (const void *aux1, void *aux2 UNUSED, void *user_data,
+ const struct ccase *c, double weight)
+{
+ int v;
+ const struct examine *examine = aux1;
+ struct exploratory_stats *es = user_data;
+
+ bool this_case_is_missing = false;
+ /* LISTWISE missing must be dealt with here */
+ if (!examine->missing_pw)
+ {
+ for (v = 0; v < examine->n_dep_vars; v++)
+ {
+ const struct variable *var = examine->dep_vars[v];
+
+ if (var_is_value_missing (var, case_data (c, var))
+ & examine->dep_excl)
+ {
+ es[v].missing += weight;
+ this_case_is_missing = true;
+ }
+ }
+ }
+
+ if (this_case_is_missing)
+ return;
+
+ for (v = 0; v < examine->n_dep_vars; v++)
+ {
+ struct ccase *outcase;
+ const struct variable *var = examine->dep_vars[v];
+ const double x = case_num (c, var);
+
+ if (var_is_value_missing (var, case_data (c, var)) & examine->dep_excl)
+ {
+ es[v].missing += weight;
+ continue;
+ }
+
+ outcase = case_create (examine->ex_proto);
+
+ if (x > es[v].maximum)
+ es[v].maximum = x;
+
+ if (x < es[v].minimum)
+ es[v].minimum = x;
+
+ es[v].non_missing += weight;
+
+ moments_pass_one (es[v].mom, x, weight);
+
+ /* Save the value and the ID to the writer */
+ assert (examine->id_idx != -1);
+ *case_num_rw_idx (outcase, EX_VAL) = x;
+ value_copy (case_data_rw_idx (outcase, EX_ID),
+ case_data_idx (c, examine->id_idx), examine->id_width);
+
+ *case_num_rw_idx (outcase, EX_WT) = weight;
+
+ es[v].cc += weight;
+
+ if (es[v].cmin > weight)
+ es[v].cmin = weight;
+
+ casewriter_write (es[v].sorted_writer, outcase);
+ }
+}
+
+static void
+calculate_n (const void *aux1, void *aux2 UNUSED, void *user_data)
+{
+ int v;
+ const struct examine *examine = aux1;
+ struct exploratory_stats *es = user_data;
+
+ for (v = 0; v < examine->n_dep_vars; v++)
+ {
+ int i;
+ casenumber imin = 0;
+ casenumber imax;
+ struct casereader *reader;
+ struct ccase *c;
+
+ if (examine->plot_histogram && es[v].non_missing > 0)
+ {
+ /* Sturges Rule */
+ double bin_width = fabs (es[v].minimum - es[v].maximum)
+ / (1 + log2 (es[v].cc));
+
+ es[v].histogram =
+ histogram_create (bin_width, es[v].minimum, es[v].maximum);
+ }
+
+ es[v].sorted_reader = casewriter_make_reader (es[v].sorted_writer);
+ es[v].sorted_writer = NULL;
+
+ imax = casereader_get_n_cases (es[v].sorted_reader);
+
+ es[v].maxima = pool_calloc (examine->pool, examine->calc_extremes, sizeof (*es[v].maxima));
+ es[v].minima = pool_calloc (examine->pool, examine->calc_extremes, sizeof (*es[v].minima));
+ for (i = 0; i < examine->calc_extremes; ++i)
+ {
+ value_init_pool (examine->pool, &es[v].maxima[i].identity, examine->id_width);
+ value_init_pool (examine->pool, &es[v].minima[i].identity, examine->id_width);
+ }
+
+ bool warn = true;
+ for (reader = casereader_clone (es[v].sorted_reader);
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ const double val = case_num_idx (c, EX_VAL);
+ double wt = case_num_idx (c, EX_WT);
+ wt = var_force_valid_weight (examine->wv, wt, &warn);
+
+ moments_pass_two (es[v].mom, val, wt);
+
+ if (es[v].histogram)
+ histogram_add (es[v].histogram, val, wt);
+
+ if (imin < examine->calc_extremes)
+ {
+ int x;
+ for (x = imin; x < examine->calc_extremes; ++x)
+ {
+ struct extremity *min = &es[v].minima[x];
+ min->val = val;
+ value_copy (&min->identity, case_data_idx (c, EX_ID), examine->id_width);
+ }
+ imin ++;
+ }
+
+ imax --;
+ if (imax < examine->calc_extremes)
+ {
+ int x;
+
+ for (x = imax; x < imax + 1; ++x)
+ {
+ struct extremity *max;
+
+ if (x >= examine->calc_extremes)
+ break;
+
+ max = &es[v].maxima[x];
+ max->val = val;
+ value_copy (&max->identity, case_data_idx (c, EX_ID), examine->id_width);
+ }
+ }
+ }
+ casereader_destroy (reader);
+
+ if (examine->calc_extremes > 0 && es[v].non_missing > 0)
+ {
+ assert (es[v].minima[0].val == es[v].minimum);
+ assert (es[v].maxima[0].val == es[v].maximum);
+ }
+
+ {
+ const int n_os = 5 + examine->n_percentiles;
+ es[v].percentiles = pool_calloc (examine->pool, examine->n_percentiles, sizeof (*es[v].percentiles));
+
+ es[v].trimmed_mean = trimmed_mean_create (es[v].cc, 0.05);
+ es[v].shapiro_wilk = NULL;
+
+ struct order_stats **os = XCALLOC (n_os, struct order_stats *);
+ os[0] = &es[v].trimmed_mean->parent;
+
+ es[v].quartiles[0] = percentile_create (0.25, es[v].cc);
+ es[v].quartiles[1] = percentile_create (0.5, es[v].cc);
+ es[v].quartiles[2] = percentile_create (0.75, es[v].cc);
+
+ os[1] = &es[v].quartiles[0]->parent;
+ os[2] = &es[v].quartiles[1]->parent;
+ os[3] = &es[v].quartiles[2]->parent;
+
+ es[v].hinges = tukey_hinges_create (es[v].cc, es[v].cmin);
+ os[4] = &es[v].hinges->parent;
+
+ for (i = 0; i < examine->n_percentiles; ++i)
+ {
+ es[v].percentiles[i] = percentile_create (examine->ptiles[i] / 100.00, es[v].cc);
+ os[5 + i] = &es[v].percentiles[i]->parent;
+ }
+
+ order_stats_accumulate_idx (os, n_os,
+ casereader_clone (es[v].sorted_reader),
+ EX_WT, EX_VAL);
+
+ free (os);
+ }
+
+ if (examine->plot_boxplot)
+ {
+ struct order_stats *os;
+
+ es[v].box_whisker = box_whisker_create (es[v].hinges,
+ EX_ID, examine->id_var);
+
+ os = &es[v].box_whisker->parent;
+ order_stats_accumulate_idx (&os, 1,
+ casereader_clone (es[v].sorted_reader),
+ EX_WT, EX_VAL);
+ }
+
+ if (examine->plot_boxplot || examine->plot_histogram
+ || examine->plot_npplot || examine->plot_spreadlevel)
+ {
+ double mean;
+
+ moments_calculate (es[v].mom, NULL, &mean, NULL, NULL, NULL);
+
+ es[v].shapiro_wilk = shapiro_wilk_create (es[v].non_missing, mean);
+
+ if (es[v].shapiro_wilk)
+ {
+ struct order_stats *os = &es[v].shapiro_wilk->parent;
+ order_stats_accumulate_idx (&os, 1,
+ casereader_clone (es[v].sorted_reader),
+ EX_WT, EX_VAL);
+ }
+ }
+
+ if (examine->plot_npplot)
+ {
+ double n, mean, var;
+ struct order_stats *os;
+
+ moments_calculate (es[v].mom, &n, &mean, &var, NULL, NULL);
+
+ es[v].np = np_create (n, mean, var);
+
+ os = &es[v].np->parent;
+
+ order_stats_accumulate_idx (&os, 1,
+ casereader_clone (es[v].sorted_reader),
+ EX_WT, EX_VAL);
+ }
+
+ }
+}
+
+static void
+cleanup_exploratory_stats (struct examine *cmd)
+{
+ int i;
+ for (i = 0; i < cmd->n_iacts; ++i)
+ {
+ int v;
+ const size_t n_cats = categoricals_n_count (cmd->cats, i);
+
+ for (v = 0; v < cmd->n_dep_vars; ++v)
+ {
+ int grp;
+ for (grp = 0; grp < n_cats; ++grp)
+ {
+ int q;
+ const struct exploratory_stats *es =
+ categoricals_get_user_data_by_category_real (cmd->cats, i, grp);
+
+ struct order_stats *os = &es[v].hinges->parent;
+ struct statistic *stat = &os->parent;
+ stat->destroy (stat);
+
+ for (q = 0; q < 3; q++)
+ {
+ os = &es[v].quartiles[q]->parent;
+ stat = &os->parent;
+ stat->destroy (stat);
+ }
+
+ for (q = 0; q < cmd->n_percentiles; q++)
+ {
+ os = &es[v].percentiles[q]->parent;
+ stat = &os->parent;
+ stat->destroy (stat);
+ }
+
+ if (es[v].shapiro_wilk)
+ {
+ stat = &es[v].shapiro_wilk->parent.parent;
+ stat->destroy (stat);
+ }
+
+ os = &es[v].trimmed_mean->parent;
+ stat = &os->parent;
+ stat->destroy (stat);
+
+ os = &es[v].np->parent;
+ if (os)
+ {
+ stat = &os->parent;
+ stat->destroy (stat);
+ }
+
+ statistic_destroy (&es[v].histogram->parent);
+ moments_destroy (es[v].mom);
+
+ if (es[v].box_whisker)
+ {
+ stat = &es[v].box_whisker->parent.parent;
+ stat->destroy (stat);
+ }
+
+ casereader_destroy (es[v].sorted_reader);
+ }
+ }
+ }
+}
+
+
+static void
+run_examine (struct examine *cmd, struct casereader *input)
+{
+ int i;
+ struct ccase *c;
+ struct casereader *reader;
+
+ struct payload payload;
+ payload.create = create_n;
+ payload.update = update_n;
+ payload.calculate = calculate_n;
+ payload.destroy = NULL;
+
+ cmd->wv = dict_get_weight (cmd->dict);
+
+ cmd->cats
+ = categoricals_create (cmd->iacts, cmd->n_iacts, cmd->wv, cmd->fctr_excl);
+
+ categoricals_set_payload (cmd->cats, &payload, cmd, NULL);
+
+ if (cmd->id_var == NULL)
+ {
+ struct ccase *c = casereader_peek (input, 0);
+
+ cmd->id_idx = case_get_n_values (c);
+ input = casereader_create_arithmetic_sequence (input, 1.0, 1.0);
+
+ case_unref (c);
+ }
+
+ for (reader = input;
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ categoricals_update (cmd->cats, c);
+ }
+ casereader_destroy (reader);
+ categoricals_done (cmd->cats);
+
+ for (i = 0; i < cmd->n_iacts; ++i)
+ {
+ summary_report (cmd, i);
+
+ const size_t n_cats = categoricals_n_count (cmd->cats, i);
+ if (n_cats == 0)
+ continue;
+
+ if (cmd->disp_extremes > 0)
+ extremes_report (cmd, i);
+
+ if (cmd->n_percentiles > 0)
+ percentiles_report (cmd, i);
+
+ if (cmd->plot_boxplot)
+ {
+ switch (cmd->boxplot_mode)
+ {
+ case BP_GROUPS:
+ show_boxplot_grouped (cmd, i);
+ break;
+ case BP_VARIABLES:
+ show_boxplot_variabled (cmd, i);
+ break;
+ default:
+ NOT_REACHED ();
+ break;
+ }
+ }
+
+ if (cmd->plot_histogram)
+ show_histogram (cmd, i);
+
+ if (cmd->plot_npplot)
+ show_npplot (cmd, i);
+
+ if (cmd->plot_spreadlevel)
+ show_spreadlevel (cmd, i);
+
+ if (cmd->descriptives)
+ descriptives_report (cmd, i);
+
+ if (cmd->plot_histogram || cmd->plot_npplot
+ || cmd->plot_spreadlevel || cmd->plot_boxplot)
+ normality_report (cmd, i);
+ }
+
+ cleanup_exploratory_stats (cmd);
+ categoricals_destroy (cmd->cats);
+}
+
+static void
+add_interaction (struct examine *examine, struct interaction *iact,
+ size_t *allocated_iacts)
+{
+ if (examine->n_iacts >= *allocated_iacts)
+ examine->iacts = pool_2nrealloc (examine->pool, examine->iacts,
+ allocated_iacts, sizeof *examine->iacts);
+ examine->iacts[examine->n_iacts++] = iact;
+}
+
+int
+cmd_examine (struct lexer *lexer, struct dataset *ds)
+{
+ bool nototals_seen = false;
+ bool totals_seen = false;
+
+ bool percentiles_seen = false;
+
+ size_t allocated_iacts = 0;
+ struct examine examine = {
+ .pool = pool_create (),
+ .dict = dataset_dict (ds),
+
+ .conf = 0.95,
+ .pc_alg = PC_HAVERAGE,
+ .id_idx = -1,
+ .boxplot_mode = BP_GROUPS,
+
+ .ex_proto = caseproto_create (),
+
+ .dep_excl = MV_ANY,
+ .fctr_excl = MV_ANY,
+ };
+
+ /* Allocate space for the first interaction.
+ This is interaction is an empty one (for the totals).
+ If no totals are requested, we will simply ignore this
+ interaction.
+ */
+ add_interaction (&examine, interaction_create (NULL), &allocated_iacts);
+
+ /* Accept an optional, completely pointless "/VARIABLES=" */
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "VARIABLES") && !lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ if (!parse_variables_const (lexer, examine.dict,
+ &examine.dep_vars, &examine.n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+
+ if (lex_match (lexer, T_BY))
+ {
+ for (;;)
+ {
+ struct interaction *iact = parse_interaction (lexer, &examine);
+ if (!iact)
+ break;
+
+ add_interaction (&examine, iact, &allocated_iacts);
+ }
+ }
+
+ int nototals_ofs = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "DESCRIPTIVES"))
+ examine.descriptives = true;
+ else if (lex_match_id (lexer, "EXTREME"))
+ {
+ int extr = 5;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_int_range (lexer, "EXTREME", 0, INT_MAX))
+ goto error;
+ extr = lex_integer (lexer);
+
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ examine.disp_extremes = extr;
+ }
+ else if (lex_match_id (lexer, "NONE"))
+ {
+ }
+ else if (lex_match (lexer, T_ALL))
+ {
+ if (examine.disp_extremes == 0)
+ examine.disp_extremes = 5;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "DESCRIPTIVES", "EXTREME",
+ "NONE", "ALL");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "PERCENTILES"))
+ {
+ percentiles_seen = true;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ size_t allocated_percentiles = examine.n_percentiles;
+ while (lex_is_number (lexer))
+ {
+ if (!lex_force_num_range_open (lexer, "PERCENTILES", 0, 100))
+ goto error;
+ double p = lex_number (lexer);
+
+ if (examine.n_percentiles >= allocated_percentiles)
+ examine.ptiles = x2nrealloc (examine.ptiles,
+ &allocated_percentiles,
+ sizeof *examine.ptiles);
+ examine.ptiles[examine.n_percentiles++] = p;
+
+ lex_get (lexer);
+ lex_match (lexer, T_COMMA);
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "HAVERAGE"))
+ examine.pc_alg = PC_HAVERAGE;
+ else if (lex_match_id (lexer, "WAVERAGE"))
+ examine.pc_alg = PC_WAVERAGE;
+ else if (lex_match_id (lexer, "ROUND"))
+ examine.pc_alg = PC_ROUND;
+ else if (lex_match_id (lexer, "EMPIRICAL"))
+ examine.pc_alg = PC_EMPIRICAL;
+ else if (lex_match_id (lexer, "AEMPIRICAL"))
+ examine.pc_alg = PC_AEMPIRICAL;
+ else if (lex_match_id (lexer, "NONE"))
+ examine.pc_alg = PC_NONE;
+ else
+ {
+ lex_error_expecting (lexer, "HAVERAGE", "WAVERAGE",
+ "ROUND", "EMPIRICAL", "AEMPIRICAL",
+ "NONE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "TOTAL"))
+ totals_seen = true;
+ else if (lex_match_id (lexer, "NOTOTAL"))
+ {
+ nototals_seen = true;
+ nototals_ofs = lex_ofs (lexer) - 1;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "LISTWISE"))
+ examine.missing_pw = false;
+ else if (lex_match_id (lexer, "PAIRWISE"))
+ examine.missing_pw = true;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ examine.dep_excl = MV_ANY;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ examine.dep_excl = MV_SYSTEM;
+ else if (lex_match_id (lexer, "REPORT"))
+ examine.fctr_excl = 0;
+ else if (lex_match_id (lexer, "NOREPORT"))
+ examine.fctr_excl = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "LISTWISE", "PAIRWISE",
+ "EXCLUDE", "INCLUDE", "REPORT",
+ "NOREPORT");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "COMPARE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "VARIABLES"))
+ examine.boxplot_mode = BP_VARIABLES;
+ else if (lex_match_id (lexer, "GROUPS"))
+ examine.boxplot_mode = BP_GROUPS;
+ else
+ {
+ lex_error_expecting (lexer, "VARIABLES", "GROUPS");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "PLOT"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "BOXPLOT"))
+ examine.plot_boxplot = true;
+ else if (lex_match_id (lexer, "NPPLOT"))
+ examine.plot_npplot = true;
+ else if (lex_match_id (lexer, "HISTOGRAM"))
+ examine.plot_histogram = true;
+ else if (lex_match_id (lexer, "SPREADLEVEL"))
+ {
+ examine.plot_spreadlevel = true;
+ examine.sl_power = 0;
+ if (lex_match (lexer, T_LPAREN) && lex_force_num (lexer))
+ {
+ examine.sl_power = lex_number (lexer);
+
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "NONE"))
+ examine.plot_boxplot = examine.plot_npplot
+ = examine.plot_histogram = examine.plot_spreadlevel = false;
+ else if (lex_match (lexer, T_ALL))
+ examine.plot_boxplot = examine.plot_npplot
+ = examine.plot_histogram = examine.plot_spreadlevel = true;
+ else
+ {
+ lex_error_expecting (lexer, "BOXPLOT", "NPPLOT",
+ "HISTOGRAM", "SPREADLEVEL",
+ "NONE", "ALL");
+ goto error;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "CINTERVAL"))
+ {
+ if (!lex_force_num (lexer))
+ goto error;
+
+ examine.conf = lex_number (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "ID"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ examine.id_var = parse_variable_const (lexer, examine.dict);
+ if (!examine.id_var)
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "STATISTICS", "PERCENTILES",
+ "TOTAL", "NOTOTAL", "MISSING", "COMPARE",
+ "PLOT", "CINTERVAL", "ID");
+ goto error;
+ }
+ }
+
+
+ if (totals_seen && nototals_seen)
+ {
+ lex_ofs_error (lexer, nototals_ofs, nototals_ofs,
+ _("%s and %s are mutually exclusive."),
+ "TOTAL", "NOTOTAL");
+ goto error;
+ }
+
+ /* If totals have been requested or if there are no factors
+ in this analysis, then the totals need to be included. */
+ if (nototals_seen && examine.n_iacts > 1)
+ {
+ interaction_destroy (examine.iacts[0]);
+ examine.iacts++;
+ examine.n_iacts--;
+ }
+
+ if (examine.id_var)
+ {
+ examine.id_idx = var_get_case_index (examine.id_var);
+ examine.id_width = var_get_width (examine.id_var);
+ }
+
+ examine.ex_proto = caseproto_add_width (examine.ex_proto, 0); /* value */
+ examine.ex_proto = caseproto_add_width (examine.ex_proto, examine.id_width); /* id */
+ examine.ex_proto = caseproto_add_width (examine.ex_proto, 0); /* weight */
+
+ if (examine.disp_extremes > 0)
+ examine.calc_extremes = examine.disp_extremes;
+
+ if (examine.descriptives && examine.calc_extremes == 0)
+ {
+ /* Descriptives always displays the max and min */
+ examine.calc_extremes = 1;
+ }
+
+ if (percentiles_seen && examine.n_percentiles == 0)
+ {
+ examine.n_percentiles = 7;
+ examine.ptiles = xmalloc (examine.n_percentiles * sizeof *examine.ptiles);
+
+ examine.ptiles[0] = 5;
+ examine.ptiles[1] = 10;
+ examine.ptiles[2] = 25;
+ examine.ptiles[3] = 50;
+ examine.ptiles[4] = 75;
+ examine.ptiles[5] = 90;
+ examine.ptiles[6] = 95;
+ }
+
+ assert (examine.calc_extremes >= examine.disp_extremes);
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), examine.dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ run_examine (&examine, group);
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ caseproto_unref (examine.ex_proto);
+
+ for (size_t i = 0; i < examine.n_iacts; ++i)
+ interaction_destroy (examine.iacts[i]);
+ free (examine.ptiles);
+ free (examine.dep_vars);
+ pool_destroy (examine.pool);
+
+ return CMD_SUCCESS;
+
+ error:
+ caseproto_unref (examine.ex_proto);
+ for (size_t i = 0; i < examine.n_iacts; ++i)
+ interaction_destroy (examine.iacts[i]);
+ free (examine.dep_vars);
+ free (examine.ptiles);
+ pool_destroy (examine.pool);
+
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009, 2010, 2011, 2012, 2014, 2015,
+ 2016, 2017 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_vector.h>
+#include <gsl/gsl_linalg.h>
+#include <gsl/gsl_matrix.h>
+#include <gsl/gsl_eigen.h>
+#include <gsl/gsl_blas.h>
+#include <gsl/gsl_sort_vector.h>
+#include <gsl/gsl_cdf.h>
+
+#include "data/any-reader.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/matrix-reader.h"
+#include "libpspp/cast.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/correlation.h"
+#include "math/covariance.h"
+#include "math/moments.h"
+#include "output/charts/scree.h"
+#include "output/pivot-table.h"
+
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+enum method
+ {
+ METHOD_CORR,
+ METHOD_COV
+ };
+
+enum missing_type
+ {
+ MISS_LISTWISE,
+ MISS_PAIRWISE,
+ MISS_MEANSUB,
+ };
+
+enum extraction_method
+ {
+ EXTRACTION_PC,
+ EXTRACTION_PAF,
+ };
+
+enum plot_opts
+ {
+ PLOT_SCREE = 0x0001,
+ PLOT_ROTATION = 0x0002
+ };
+
+enum print_opts
+ {
+ PRINT_UNIVARIATE = 1 << 0,
+ PRINT_DETERMINANT = 1 << 1,
+ PRINT_INV = 1 << 2,
+ PRINT_AIC = 1 << 3,
+ PRINT_SIG = 1 << 4,
+ PRINT_COVARIANCE = 1 << 5,
+ PRINT_CORRELATION = 1 << 6,
+ PRINT_ROTATION = 1 << 7,
+ PRINT_EXTRACTION = 1 << 8,
+ PRINT_INITIAL = 1 << 9,
+ PRINT_KMO = 1 << 10,
+ PRINT_REPR = 1 << 11,
+ PRINT_FSCORE = 1 << 12
+ };
+
+enum rotation_type
+ {
+ ROT_VARIMAX = 0,
+ ROT_EQUAMAX,
+ ROT_QUARTIMAX,
+ ROT_PROMAX,
+ ROT_NONE
+ };
+
+typedef void (*rotation_coefficients) (double *x, double *y,
+ double a, double b, double c, double d,
+ const gsl_matrix *loadings);
+
+
+static void
+varimax_coefficients (double *x, double *y,
+ double a, double b, double c, double d,
+ const gsl_matrix *loadings)
+{
+ *x = d - 2 * a * b / loadings->size1;
+ *y = c - (a * a - b * b) / loadings->size1;
+}
+
+static void
+equamax_coefficients (double *x, double *y,
+ double a, double b, double c, double d,
+ const gsl_matrix *loadings)
+{
+ *x = d - loadings->size2 * a * b / loadings->size1;
+ *y = c - loadings->size2 * (a * a - b * b) / (2 * loadings->size1);
+}
+
+static void
+quartimax_coefficients (double *x, double *y,
+ double a UNUSED, double b UNUSED, double c, double d,
+ const gsl_matrix *loadings UNUSED)
+{
+ *x = d;
+ *y = c;
+}
+
+static const rotation_coefficients rotation_coeff[] = {
+ varimax_coefficients,
+ equamax_coefficients,
+ quartimax_coefficients,
+ varimax_coefficients /* PROMAX is identical to VARIMAX */
+};
+
+
+/* return diag (C'C) ^ {-0.5} */
+static gsl_matrix *
+diag_rcp_sqrt (const gsl_matrix *C)
+{
+ gsl_matrix *d = gsl_matrix_calloc (C->size1, C->size2);
+ gsl_matrix *r = gsl_matrix_calloc (C->size1, C->size2);
+
+ assert (C->size1 == C->size2);
+
+ gsl_linalg_matmult_mod (C, GSL_LINALG_MOD_TRANSPOSE,
+ C, GSL_LINALG_MOD_NONE,
+ d);
+
+ for (int j = 0; j < d->size2; ++j)
+ {
+ double e = gsl_matrix_get (d, j, j);
+ e = 1.0 / sqrt (e);
+ gsl_matrix_set (r, j, j, e);
+ }
+
+ gsl_matrix_free (d);
+
+ return r;
+}
+
+
+
+/* return diag ((C'C)^-1) ^ {-0.5} */
+static gsl_matrix *
+diag_rcp_inv_sqrt (const gsl_matrix *CCinv)
+{
+ gsl_matrix *r = gsl_matrix_calloc (CCinv->size1, CCinv->size2);
+
+ assert (CCinv->size1 == CCinv->size2);
+
+ for (int j = 0; j < CCinv->size2; ++j)
+ {
+ double e = gsl_matrix_get (CCinv, j, j);
+ e = 1.0 / sqrt (e);
+ gsl_matrix_set (r, j, j, e);
+ }
+
+ return r;
+}
+
+
+
+
+
+struct cmd_factor
+{
+ size_t n_vars;
+ const struct variable **vars;
+
+ const struct variable *wv;
+
+ enum method method;
+ enum missing_type missing_type;
+ enum mv_class exclude;
+ enum print_opts print;
+ enum extraction_method extraction;
+ enum plot_opts plot;
+ enum rotation_type rotation;
+ int rotation_iterations;
+ int promax_power;
+
+ /* Extraction Criteria */
+ int n_factors;
+ double min_eigen;
+ double econverge;
+ int extraction_iterations;
+
+ double rconverge;
+
+ /* Format */
+ double blank;
+ bool sort;
+};
+
+
+struct idata
+{
+ /* Intermediate values used in calculation */
+ struct matrix_material mm;
+
+ gsl_matrix *analysis_matrix; /* A pointer to either mm.corr or mm.cov */
+
+ gsl_vector *eval; /* The eigenvalues */
+ gsl_matrix *evec; /* The eigenvectors */
+
+ int n_extractions;
+
+ gsl_vector *msr; /* Multiple Squared Regressions */
+
+ double detR; /* The determinant of the correlation matrix */
+
+ gsl_matrix *ai_cov; /* The anti-image covariance matrix */
+ gsl_matrix *ai_cor; /* The anti-image correlation matrix */
+ struct covariance *cvm;
+};
+
+static struct idata *
+idata_alloc (size_t n_vars)
+{
+ struct idata *id = XZALLOC (struct idata);
+
+ id->n_extractions = 0;
+ id->msr = gsl_vector_alloc (n_vars);
+
+ id->eval = gsl_vector_alloc (n_vars);
+ id->evec = gsl_matrix_alloc (n_vars, n_vars);
+
+ return id;
+}
+
+static void
+idata_free (struct idata *id)
+{
+ gsl_vector_free (id->msr);
+ gsl_vector_free (id->eval);
+ gsl_matrix_free (id->evec);
+ gsl_matrix_free (id->ai_cov);
+ gsl_matrix_free (id->ai_cor);
+
+ free (id);
+}
+
+/* Return the sum of squares of all the elements in row J excluding column J */
+static double
+ssq_row_od_n (const gsl_matrix *m, int j)
+{
+ assert (m->size1 == m->size2);
+ assert (j < m->size1);
+
+ double ss = 0;
+ for (int i = 0; i < m->size1; ++i)
+ if (i != j)
+ ss += pow2 (gsl_matrix_get (m, i, j));
+ return ss;
+}
+
+/* Return the sum of squares of all the elements excluding row N */
+static double
+ssq_od_n (const gsl_matrix *m, int n)
+{
+ assert (m->size1 == m->size2);
+ assert (n < m->size1);
+
+ double ss = 0;
+ for (int i = 0; i < m->size1; ++i)
+ for (int j = 0; j < m->size2; ++j)
+ if (i != j)
+ ss += pow2 (gsl_matrix_get (m, i, j));
+ return ss;
+}
+
+
+static gsl_matrix *
+anti_image_corr (const gsl_matrix *m, const struct idata *idata)
+{
+ assert (m->size1 == m->size2);
+
+ gsl_matrix *a = gsl_matrix_alloc (m->size1, m->size2);
+ for (int i = 0; i < m->size1; ++i)
+ for (int j = 0; j < m->size2; ++j)
+ {
+ double *p = gsl_matrix_ptr (a, i, j);
+ *p = gsl_matrix_get (m, i, j);
+ *p /= sqrt (gsl_matrix_get (m, i, i) *
+ gsl_matrix_get (m, j, j));
+ }
+
+ for (int i = 0; i < m->size1; ++i)
+ {
+ double r = ssq_row_od_n (idata->mm.corr, i);
+ double u = ssq_row_od_n (a, i);
+ gsl_matrix_set (a, i, i, r / (r + u));
+ }
+
+ return a;
+}
+
+static gsl_matrix *
+anti_image_cov (const gsl_matrix *m)
+{
+ assert (m->size1 == m->size2);
+
+ gsl_matrix *a = gsl_matrix_alloc (m->size1, m->size2);
+ for (int i = 0; i < m->size1; ++i)
+ for (int j = 0; j < m->size2; ++j)
+ {
+ double *p = gsl_matrix_ptr (a, i, j);
+ *p = gsl_matrix_get (m, i, j);
+ *p /= gsl_matrix_get (m, i, i);
+ *p /= gsl_matrix_get (m, j, j);
+ }
+
+ return a;
+}
+
+#if 0
+static void
+dump_matrix (const gsl_matrix *m)
+{
+ for (int i = 0; i < m->size1; ++i)
+ {
+ for (int j = 0; j < m->size2; ++j)
+ printf ("%02f ", gsl_matrix_get (m, i, j));
+ printf ("\n");
+ }
+}
+
+static void
+dump_matrix_permute (const gsl_matrix *m, const gsl_permutation *p)
+{
+ for (int i = 0; i < m->size1; ++i)
+ {
+ for (int j = 0; j < m->size2; ++j)
+ printf ("%02f ", gsl_matrix_get (m, gsl_permutation_get (p, i), j));
+ printf ("\n");
+ }
+}
+
+
+static void
+dump_vector (const gsl_vector *v)
+{
+ for (size_t i = 0; i < v->size; ++i)
+ printf ("%02f\n", gsl_vector_get (v, i));
+ printf ("\n");
+}
+#endif
+
+
+static int
+n_extracted_factors (const struct cmd_factor *factor, struct idata *idata)
+{
+ /* If there is a cached value, then return that. */
+ if (idata->n_extractions != 0)
+ return idata->n_extractions;
+
+ /* Otherwise, if the number of factors has been explicitly requested,
+ use that. */
+ if (factor->n_factors > 0)
+ {
+ idata->n_extractions = factor->n_factors;
+ goto finish;
+ }
+
+ /* Use the MIN_EIGEN setting. */
+ for (int i = 0; i < idata->eval->size; ++i)
+ {
+ double evali = fabs (gsl_vector_get (idata->eval, i));
+
+ idata->n_extractions = i;
+
+ if (evali < factor->min_eigen)
+ goto finish;
+ }
+
+ finish:
+ return idata->n_extractions;
+}
+
+
+/* Returns a newly allocated matrix identical to M.
+ It is the callers responsibility to free the returned value.
+*/
+static gsl_matrix *
+matrix_dup (const gsl_matrix *m)
+{
+ gsl_matrix *n = gsl_matrix_alloc (m->size1, m->size2);
+ gsl_matrix_memcpy (n, m);
+ return n;
+}
+
+
+struct smr_workspace
+{
+ /* Copy of the subject */
+ gsl_matrix *m;
+
+ gsl_matrix *inverse;
+
+ gsl_permutation *perm;
+
+ gsl_matrix *result1;
+ gsl_matrix *result2;
+};
+
+
+static struct smr_workspace *ws_create (const gsl_matrix *input)
+{
+ struct smr_workspace *ws = xmalloc (sizeof (*ws));
+
+ ws->m = gsl_matrix_alloc (input->size1, input->size2);
+ ws->inverse = gsl_matrix_calloc (input->size1 - 1, input->size2 - 1);
+ ws->perm = gsl_permutation_alloc (input->size1 - 1);
+ ws->result1 = gsl_matrix_calloc (input->size1 - 1, 1);
+ ws->result2 = gsl_matrix_calloc (1, 1);
+
+ return ws;
+}
+
+static void
+ws_destroy (struct smr_workspace *ws)
+{
+ gsl_matrix_free (ws->result2);
+ gsl_matrix_free (ws->result1);
+ gsl_permutation_free (ws->perm);
+ gsl_matrix_free (ws->inverse);
+ gsl_matrix_free (ws->m);
+
+ free (ws);
+}
+
+
+/*
+ Return the square of the regression coefficient for VAR regressed against all other variables.
+ */
+static double
+squared_multiple_correlation (const gsl_matrix *corr, int var, struct smr_workspace *ws)
+{
+ /* For an explanation of what this is doing, see
+ http://www.visualstatistics.net/Visual%20Statistics%20Multimedia/multiple_regression_analysis.htm
+ */
+
+ gsl_matrix_memcpy (ws->m, corr);
+
+ gsl_matrix_swap_rows (ws->m, 0, var);
+ gsl_matrix_swap_columns (ws->m, 0, var);
+
+ gsl_matrix_view rxx = gsl_matrix_submatrix (ws->m, 1, 1, ws->m->size1 - 1, ws->m->size1 - 1);
+
+ int signum = 0;
+ gsl_linalg_LU_decomp (&rxx.matrix, ws->perm, &signum);
+
+ gsl_linalg_LU_invert (&rxx.matrix, ws->perm, ws->inverse);
+
+ gsl_matrix_const_view rxy = gsl_matrix_const_submatrix (ws->m, 1, 0, ws->m->size1 - 1, 1);
+ gsl_matrix_const_view ryx = gsl_matrix_const_submatrix (ws->m, 0, 1, 1, ws->m->size1 - 1);
+
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans,
+ 1.0, ws->inverse, &rxy.matrix, 0.0, ws->result1);
+
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans,
+ 1.0, &ryx.matrix, ws->result1, 0.0, ws->result2);
+
+ return gsl_matrix_get (ws->result2, 0, 0);
+}
+
+
+
+static double the_communality (const gsl_matrix *evec, const gsl_vector *eval, int n, int n_factors);
+
+
+struct factor_matrix_workspace
+{
+ size_t n_factors;
+ gsl_eigen_symmv_workspace *eigen_ws;
+
+ gsl_vector *eval;
+ gsl_matrix *evec;
+
+ gsl_matrix *gamma;
+
+ gsl_matrix *r;
+};
+
+static struct factor_matrix_workspace *
+factor_matrix_workspace_alloc (size_t n, size_t nf)
+{
+ struct factor_matrix_workspace *ws = xmalloc (sizeof (*ws));
+
+ ws->n_factors = nf;
+ ws->gamma = gsl_matrix_calloc (nf, nf);
+ ws->eigen_ws = gsl_eigen_symmv_alloc (n);
+ ws->eval = gsl_vector_alloc (n);
+ ws->evec = gsl_matrix_alloc (n, n);
+ ws->r = gsl_matrix_alloc (n, n);
+
+ return ws;
+}
+
+static void
+factor_matrix_workspace_free (struct factor_matrix_workspace *ws)
+{
+ gsl_eigen_symmv_free (ws->eigen_ws);
+ gsl_vector_free (ws->eval);
+ gsl_matrix_free (ws->evec);
+ gsl_matrix_free (ws->gamma);
+ gsl_matrix_free (ws->r);
+ free (ws);
+}
+
+/*
+ Shift P left by OFFSET places, and overwrite TARGET
+ with the shifted result.
+ Positions in TARGET less than OFFSET are unchanged.
+*/
+static void
+perm_shift_apply (gsl_permutation *target, const gsl_permutation *p,
+ size_t offset)
+{
+ assert (target->size == p->size);
+ assert (offset <= target->size);
+
+ for (size_t i = 0; i < target->size - offset; ++i)
+ target->data[i] = p->data [i + offset];
+}
+
+
+/*
+ Indirectly sort the rows of matrix INPUT, storing the sort order in PERM.
+ The sort criteria are as follows:
+
+ Rows are sorted on the first column, until the absolute value of an
+ element in a subsequent column is greater than that of the first
+ column. Thereafter, rows will be sorted on the second column,
+ until the absolute value of an element in a subsequent column
+ exceeds that of the second column ...
+*/
+static void
+sort_matrix_indirect (const gsl_matrix *input, gsl_permutation *perm)
+{
+ assert (perm->size == input->size1);
+
+ const size_t n = perm->size;
+ const size_t m = input->size2;
+ gsl_permutation *p = gsl_permutation_alloc (n);
+
+ /* Copy INPUT into MAT, discarding the sign */
+ gsl_matrix *mat = gsl_matrix_alloc (n, m);
+ for (int i = 0; i < mat->size1; ++i)
+ for (int j = 0; j < mat->size2; ++j)
+ gsl_matrix_set (mat, i, j, fabs (gsl_matrix_get (input, i, j)));
+
+ int column_n = 0;
+ int row_n = 0;
+ while (column_n < m && row_n < n)
+ {
+ gsl_vector_const_view columni = gsl_matrix_const_column (mat, column_n);
+ gsl_sort_vector_index (p, &columni.vector);
+
+ int i;
+ for (i = 0; i < n; ++i)
+ {
+ gsl_vector_view row = gsl_matrix_row (mat, p->data[n - 1 - i]);
+ size_t maxindex = gsl_vector_max_index (&row.vector);
+
+ if (maxindex > column_n)
+ break;
+
+ /* All subsequent elements of this row, are of no interest.
+ So set them all to a highly negative value */
+ for (int j = column_n + 1; j < row.vector.size; ++j)
+ gsl_vector_set (&row.vector, j, -DBL_MAX);
+ }
+
+ perm_shift_apply (perm, p, row_n);
+ row_n += i;
+
+ column_n++;
+ }
+
+ gsl_permutation_free (p);
+ gsl_matrix_free (mat);
+
+ assert (0 == gsl_permutation_valid (perm));
+
+ /* We want the biggest value to be first */
+ gsl_permutation_reverse (perm);
+}
+
+
+static void
+drot_go (double phi, double *l0, double *l1)
+{
+ double r0 = cos (phi) * *l0 + sin (phi) * *l1;
+ double r1 = - sin (phi) * *l0 + cos (phi) * *l1;
+
+ *l0 = r0;
+ *l1 = r1;
+}
+
+
+static gsl_matrix *
+clone_matrix (const gsl_matrix *m)
+{
+ gsl_matrix *c = gsl_matrix_calloc (m->size1, m->size2);
+
+ for (int j = 0; j < c->size1; ++j)
+ for (int k = 0; k < c->size2; ++k)
+ gsl_matrix_set (c, j, k, gsl_matrix_get (m, j, k));
+
+ return c;
+}
+
+
+static double
+initial_sv (const gsl_matrix *fm)
+{
+ double sv = 0.0;
+ for (int j = 0; j < fm->size2; ++j)
+ {
+ double l4s = 0;
+ double l2s = 0;
+
+ for (int k = j + 1; k < fm->size2; ++k)
+ {
+ double lambda = gsl_matrix_get (fm, k, j);
+ double lambda_sq = lambda * lambda;
+ double lambda_4 = lambda_sq * lambda_sq;
+
+ l4s += lambda_4;
+ l2s += lambda_sq;
+ }
+ sv += (fm->size1 * l4s - (l2s * l2s)) / (fm->size1 * fm->size1);
+ }
+ return sv;
+}
+
+static void
+rotate (const struct cmd_factor *cf, const gsl_matrix *unrot,
+ const gsl_vector *communalities,
+ gsl_matrix *result,
+ gsl_vector *rotated_loadings,
+ gsl_matrix *pattern_matrix,
+ gsl_matrix *factor_correlation_matrix)
+{
+ /* First get a normalised version of UNROT */
+ gsl_matrix *normalised = gsl_matrix_calloc (unrot->size1, unrot->size2);
+ gsl_matrix *h_sqrt = gsl_matrix_calloc (communalities->size, communalities->size);
+ gsl_matrix *h_sqrt_inv;
+
+ /* H is the diagonal matrix containing the absolute values of the communalities */
+ for (int i = 0; i < communalities->size; ++i)
+ {
+ double *ptr = gsl_matrix_ptr (h_sqrt, i, i);
+ *ptr = fabs (gsl_vector_get (communalities, i));
+ }
+
+ /* Take the square root of the communalities */
+ gsl_linalg_cholesky_decomp (h_sqrt);
+
+ /* Save a copy of h_sqrt and invert it */
+ h_sqrt_inv = clone_matrix (h_sqrt);
+ gsl_linalg_cholesky_decomp (h_sqrt_inv);
+ gsl_linalg_cholesky_invert (h_sqrt_inv);
+
+ /* normalised vertion is H^{1/2} x UNROT */
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, h_sqrt_inv, unrot, 0.0, normalised);
+
+ gsl_matrix_free (h_sqrt_inv);
+
+ /* Now perform the rotation iterations */
+ double prev_sv = initial_sv (normalised);
+ for (int i = 0; i < cf->rotation_iterations; ++i)
+ {
+ double sv = 0.0;
+ for (int j = 0; j < normalised->size2; ++j)
+ {
+ /* These variables relate to the convergence criterium */
+ double l4s = 0;
+ double l2s = 0;
+
+ for (int k = j + 1; k < normalised->size2; ++k)
+ {
+ double a = 0.0;
+ double b = 0.0;
+ double c = 0.0;
+ double d = 0.0;
+ for (int p = 0; p < normalised->size1; ++p)
+ {
+ double jv = gsl_matrix_get (normalised, p, j);
+ double kv = gsl_matrix_get (normalised, p, k);
+
+ double u = jv * jv - kv * kv;
+ double v = 2 * jv * kv;
+ a += u;
+ b += v;
+ c += u * u - v * v;
+ d += 2 * u * v;
+ }
+
+ double x, y;
+ rotation_coeff [cf->rotation] (&x, &y, a, b, c, d, normalised);
+ double phi = atan2 (x, y) / 4.0;
+
+ /* Don't bother rotating if the angle is small */
+ if (fabs (sin (phi)) <= pow (10.0, -15.0))
+ continue;
+
+ for (int p = 0; p < normalised->size1; ++p)
+ {
+ double *lambda0 = gsl_matrix_ptr (normalised, p, j);
+ double *lambda1 = gsl_matrix_ptr (normalised, p, k);
+ drot_go (phi, lambda0, lambda1);
+ }
+
+ /* Calculate the convergence criterium */
+ double lambda = gsl_matrix_get (normalised, k, j);
+ double lambda_sq = lambda * lambda;
+ double lambda_4 = lambda_sq * lambda_sq;
+
+ l4s += lambda_4;
+ l2s += lambda_sq;
+ }
+ sv += (normalised->size1 * l4s - (l2s * l2s)) / (normalised->size1 * normalised->size1);
+ }
+
+ if (fabs (sv - prev_sv) <= cf->rconverge)
+ break;
+
+ prev_sv = sv;
+ }
+
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0,
+ h_sqrt, normalised, 0.0, result);
+
+ gsl_matrix_free (h_sqrt);
+ gsl_matrix_free (normalised);
+
+ if (cf->rotation == ROT_PROMAX)
+ {
+ /* general purpose m by m matrix, where m is the number of factors */
+ gsl_matrix *mm1 = gsl_matrix_calloc (unrot->size2, unrot->size2);
+ gsl_matrix *mm2 = gsl_matrix_calloc (unrot->size2, unrot->size2);
+
+ /* general purpose m by p matrix, where p is the number of variables */
+ gsl_matrix *mp1 = gsl_matrix_calloc (unrot->size2, unrot->size1);
+
+ gsl_matrix *pm1 = gsl_matrix_calloc (unrot->size1, unrot->size2);
+
+ gsl_permutation *perm = gsl_permutation_alloc (unrot->size2);
+
+
+ /* The following variables follow the notation by SPSS Statistical
+ Algorithms page 342. */
+ gsl_matrix *L = gsl_matrix_calloc (unrot->size2, unrot->size2);
+ gsl_matrix *P = clone_matrix (result);
+
+ /* Vector of length p containing (indexed by i)
+ \Sum^m_j {\lambda^2_{ij}} */
+ gsl_vector *rssq = gsl_vector_calloc (unrot->size1);
+
+ for (int i = 0; i < P->size1; ++i)
+ {
+ double sum = 0;
+ for (int j = 0; j < P->size2; ++j)
+ sum += gsl_matrix_get (result, i, j) * gsl_matrix_get (result, i, j);
+ gsl_vector_set (rssq, i, sqrt (sum));
+ }
+
+ for (int i = 0; i < P->size1; ++i)
+ {
+ for (int j = 0; j < P->size2; ++j)
+ {
+ double l = gsl_matrix_get (result, i, j);
+ double r = gsl_vector_get (rssq, i);
+ gsl_matrix_set (P, i, j, pow (fabs (l / r), cf->promax_power + 1) * r / l);
+ }
+ }
+
+ gsl_vector_free (rssq);
+
+ gsl_linalg_matmult_mod (result,
+ GSL_LINALG_MOD_TRANSPOSE,
+ result,
+ GSL_LINALG_MOD_NONE,
+ mm1);
+
+ int signum;
+ gsl_linalg_LU_decomp (mm1, perm, &signum);
+ gsl_linalg_LU_invert (mm1, perm, mm2);
+
+ gsl_linalg_matmult_mod (mm2, GSL_LINALG_MOD_NONE,
+ result, GSL_LINALG_MOD_TRANSPOSE,
+ mp1);
+
+ gsl_linalg_matmult_mod (mp1, GSL_LINALG_MOD_NONE,
+ P, GSL_LINALG_MOD_NONE,
+ L);
+
+ gsl_matrix *D = diag_rcp_sqrt (L);
+ gsl_matrix *Q = gsl_matrix_calloc (unrot->size2, unrot->size2);
+
+ gsl_linalg_matmult_mod (L, GSL_LINALG_MOD_NONE,
+ D, GSL_LINALG_MOD_NONE,
+ Q);
+
+ gsl_matrix *QQinv = gsl_matrix_calloc (unrot->size2, unrot->size2);
+
+ gsl_linalg_matmult_mod (Q, GSL_LINALG_MOD_TRANSPOSE,
+ Q, GSL_LINALG_MOD_NONE,
+ QQinv);
+
+ gsl_linalg_cholesky_decomp (QQinv);
+ gsl_linalg_cholesky_invert (QQinv);
+
+
+ gsl_matrix *C = diag_rcp_inv_sqrt (QQinv);
+ gsl_matrix *Cinv = clone_matrix (C);
+
+ gsl_linalg_cholesky_decomp (Cinv);
+ gsl_linalg_cholesky_invert (Cinv);
+
+
+ gsl_linalg_matmult_mod (result, GSL_LINALG_MOD_NONE,
+ Q, GSL_LINALG_MOD_NONE,
+ pm1);
+
+ gsl_linalg_matmult_mod (pm1, GSL_LINALG_MOD_NONE,
+ Cinv, GSL_LINALG_MOD_NONE,
+ pattern_matrix);
+
+
+ gsl_linalg_matmult_mod (C, GSL_LINALG_MOD_NONE,
+ QQinv, GSL_LINALG_MOD_NONE,
+ mm1);
+
+ gsl_linalg_matmult_mod (mm1, GSL_LINALG_MOD_NONE,
+ C, GSL_LINALG_MOD_TRANSPOSE,
+ factor_correlation_matrix);
+
+ gsl_linalg_matmult_mod (pattern_matrix, GSL_LINALG_MOD_NONE,
+ factor_correlation_matrix, GSL_LINALG_MOD_NONE,
+ pm1);
+
+ gsl_matrix_memcpy (result, pm1);
+
+
+ gsl_matrix_free (QQinv);
+ gsl_matrix_free (C);
+ gsl_matrix_free (Cinv);
+
+ gsl_matrix_free (D);
+ gsl_matrix_free (Q);
+ gsl_matrix_free (L);
+ gsl_matrix_free (P);
+
+ gsl_permutation_free (perm);
+
+ gsl_matrix_free (mm1);
+ gsl_matrix_free (mm2);
+ gsl_matrix_free (mp1);
+ gsl_matrix_free (pm1);
+ }
+
+
+ /* reflect negative sums and populate the rotated loadings vector*/
+ for (int i = 0; i < result->size2; ++i)
+ {
+ double ssq = 0.0;
+ double sum = 0.0;
+ for (int j = 0; j < result->size1; ++j)
+ {
+ double s = gsl_matrix_get (result, j, i);
+ ssq += s * s;
+ sum += s;
+ }
+
+ gsl_vector_set (rotated_loadings, i, ssq);
+
+ if (sum < 0)
+ for (int j = 0; j < result->size1; ++j)
+ {
+ double *lambda = gsl_matrix_ptr (result, j, i);
+ *lambda = - *lambda;
+ }
+ }
+}
+
+/*
+ Get an approximation for the factor matrix into FACTORS, and the communalities into COMMUNALITIES.
+ R is the matrix to be analysed.
+ WS is a pointer to a structure which must have been initialised with factor_matrix_workspace_init.
+ */
+static void
+iterate_factor_matrix (const gsl_matrix *r, gsl_vector *communalities, gsl_matrix *factors,
+ struct factor_matrix_workspace *ws)
+{
+ assert (r->size1 == r->size2);
+ assert (r->size1 == communalities->size);
+
+ assert (factors->size1 == r->size1);
+ assert (factors->size2 == ws->n_factors);
+
+ gsl_matrix_memcpy (ws->r, r);
+
+ /* Apply Communalities to diagonal of correlation matrix */
+ for (size_t i = 0; i < communalities->size; ++i)
+ {
+ double *x = gsl_matrix_ptr (ws->r, i, i);
+ *x = gsl_vector_get (communalities, i);
+ }
+
+ gsl_eigen_symmv (ws->r, ws->eval, ws->evec, ws->eigen_ws);
+
+ gsl_matrix_view mv = gsl_matrix_submatrix (ws->evec, 0, 0, ws->evec->size1, ws->n_factors);
+
+ /* Gamma is the diagonal matrix containing the absolute values of the eigenvalues */
+ for (size_t i = 0; i < ws->n_factors; ++i)
+ {
+ double *ptr = gsl_matrix_ptr (ws->gamma, i, i);
+ *ptr = fabs (gsl_vector_get (ws->eval, i));
+ }
+
+ /* Take the square root of gamma */
+ gsl_linalg_cholesky_decomp (ws->gamma);
+
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, &mv.matrix, ws->gamma, 0.0, factors);
+
+ for (size_t i = 0; i < r->size1; ++i)
+ {
+ double h = the_communality (ws->evec, ws->eval, i, ws->n_factors);
+ gsl_vector_set (communalities, i, h);
+ }
+}
+
+
+
+static bool run_factor (struct dataset *ds, const struct cmd_factor *factor);
+
+static void do_factor_by_matrix (const struct cmd_factor *factor, struct idata *idata);
+
+
+
+int
+cmd_factor (struct lexer *lexer, struct dataset *ds)
+{
+ int n_iterations = 25;
+
+ struct cmd_factor factor = {
+ .n_vars = 0,
+ .vars = NULL,
+ .method = METHOD_CORR,
+ .missing_type = MISS_LISTWISE,
+ .exclude = MV_ANY,
+ .print = PRINT_INITIAL | PRINT_EXTRACTION | PRINT_ROTATION,
+ .extraction = EXTRACTION_PC,
+ .n_factors = 0,
+ .min_eigen = SYSMIS,
+ .extraction_iterations = 25,
+ .rotation_iterations = 25,
+ .econverge = 0.001,
+
+ .blank = 0,
+ .sort = false,
+ .plot = 0,
+ .rotation = ROT_VARIMAX,
+ .wv = NULL,
+
+ .rconverge = 0.0001,
+ };
+
+ lex_match (lexer, T_SLASH);
+
+ struct dictionary *dict = NULL;
+ struct matrix_reader *mr = NULL;
+ struct casereader *matrix_reader = NULL;
+
+ int vars_start, vars_end;
+ if (lex_match_id (lexer, "VARIABLES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ dict = dataset_dict (ds);
+ factor.wv = dict_get_weight (dict);
+
+ vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, dict, &factor.vars, &factor.n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ vars_end = lex_ofs (lexer) - 1;
+ }
+ else if (lex_match_id (lexer, "MATRIX"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_match_phrase (lexer, "IN("))
+ goto error;
+ if (!lex_match_id (lexer, "CORR") && !lex_match_id (lexer, "COV"))
+ {
+ lex_error (lexer, _("Matrix input for %s must be either COV or CORR"),
+ "FACTOR");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+ vars_start = lex_ofs (lexer);
+ if (lex_match (lexer, T_ASTERISK))
+ {
+ dict = dataset_dict (ds);
+ matrix_reader = casereader_clone (dataset_source (ds));
+ }
+ else
+ {
+ struct file_handle *fh = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (fh == NULL)
+ goto error;
+
+ matrix_reader = any_reader_open_and_decode (fh, NULL, &dict, NULL);
+
+ if (!(matrix_reader && dict))
+ goto error;
+ }
+ vars_end = lex_ofs (lexer) - 1;
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ {
+ casereader_destroy (matrix_reader);
+ goto error;
+ }
+
+ mr = matrix_reader_create (dict, matrix_reader);
+ factor.vars = xmemdup (mr->cvars, mr->n_cvars * sizeof *mr->cvars);
+ factor.n_vars = mr->n_cvars;
+ }
+ else
+ goto error;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "ANALYSIS"))
+ {
+ struct const_var_set *vs;
+ const struct variable **vars;
+ size_t n_vars;
+
+ lex_match (lexer, T_EQUALS);
+
+ vars_start = lex_ofs (lexer);
+ vs = const_var_set_create_from_array (factor.vars, factor.n_vars);
+ vars_end = lex_ofs (lexer) - 1;
+ bool ok = parse_const_var_set_vars (lexer, vs, &vars, &n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC);
+ const_var_set_destroy (vs);
+
+ if (!ok)
+ goto error;
+
+ free (factor.vars);
+ factor.vars = vars;
+ factor.n_vars = n_vars;
+
+ if (mr)
+ {
+ free (mr->cvars);
+ mr->cvars = xmemdup (vars, n_vars * sizeof *vars);
+ mr->n_cvars = n_vars;
+ }
+ }
+ else if (lex_match_id (lexer, "PLOT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "EIGEN"))
+ {
+ factor.plot |= PLOT_SCREE;
+ }
+#if FACTOR_FULLY_IMPLEMENTED
+ else if (lex_match_id (lexer, "ROTATION"))
+ {
+ }
+#endif
+ else
+ {
+ lex_error_expecting (lexer, "EIGEN"
+#if FACTOR_FULLY_IMPLEMENTED
+ , "ROTATION"
+#endif
+ );
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "METHOD"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "COVARIANCE"))
+ factor.method = METHOD_COV;
+ else if (lex_match_id (lexer, "CORRELATION"))
+ factor.method = METHOD_CORR;
+ else
+ {
+ lex_error_expecting (lexer, "COVARIANCE", "CORRELATION");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "ROTATION"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ /* VARIMAX and DEFAULT are defaults */
+ if (lex_match_id (lexer, "VARIMAX") || lex_match_id (lexer, "DEFAULT"))
+ factor.rotation = ROT_VARIMAX;
+ else if (lex_match_id (lexer, "EQUAMAX"))
+ factor.rotation = ROT_EQUAMAX;
+ else if (lex_match_id (lexer, "QUARTIMAX"))
+ factor.rotation = ROT_QUARTIMAX;
+ else if (lex_match_id (lexer, "PROMAX"))
+ {
+ factor.promax_power = 5;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_int (lexer))
+ goto error;
+ factor.promax_power = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ factor.rotation = ROT_PROMAX;
+ }
+ else if (lex_match_id (lexer, "NOROTATE"))
+ factor.rotation = ROT_NONE;
+ else
+ {
+ lex_error_expecting (lexer, "DEFAULT", "VARIMAX", "EQUAMAX",
+ "QUARTIMAX", "PROMAX", "NOROTATE");
+ goto error;
+ }
+ }
+ factor.rotation_iterations = n_iterations;
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "FACTORS"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int (lexer))
+ goto error;
+ factor.n_factors = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "MINEIGEN"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ factor.min_eigen = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ECONVERGE"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ factor.econverge = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "RCONVERGE"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ factor.rconverge = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ITERATE"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "ITERATE", 0, INT_MAX))
+ goto error;
+ n_iterations = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "DEFAULT"))
+ {
+ factor.n_factors = 0;
+ factor.min_eigen = 1;
+ n_iterations = 25;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "FACTORS", "MINEIGEN",
+ "ECONVERGE", "RCONVERGE", "ITERATE",
+ "DEFAULT");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "EXTRACTION"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "PAF"))
+ factor.extraction = EXTRACTION_PAF;
+ else if (lex_match_id (lexer, "PC"))
+ factor.extraction = EXTRACTION_PC;
+ else if (lex_match_id (lexer, "PA1"))
+ factor.extraction = EXTRACTION_PC;
+ else if (lex_match_id (lexer, "DEFAULT"))
+ factor.extraction = EXTRACTION_PC;
+ else
+ {
+ lex_error_expecting (lexer, "PAF", "PC", "PA1", "DEFAULT");
+ goto error;
+ }
+ }
+ factor.extraction_iterations = n_iterations;
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "SORT"))
+ factor.sort = true;
+ else if (lex_match_id (lexer, "BLANK"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ factor.blank = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "DEFAULT"))
+ {
+ factor.blank = 0;
+ factor.sort = false;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "SORT", "BLANK", "DEFAULT");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "PRINT"))
+ {
+ factor.print = 0;
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "UNIVARIATE"))
+ factor.print |= PRINT_UNIVARIATE;
+ else if (lex_match_id (lexer, "DET"))
+ factor.print |= PRINT_DETERMINANT;
+#if FACTOR_FULLY_IMPLEMENTED
+ else if (lex_match_id (lexer, "INV"))
+ {
+ }
+#endif
+ else if (lex_match_id (lexer, "AIC"))
+ factor.print |= PRINT_AIC;
+ else if (lex_match_id (lexer, "SIG"))
+ factor.print |= PRINT_SIG;
+ else if (lex_match_id (lexer, "CORRELATION"))
+ factor.print |= PRINT_CORRELATION;
+ else if (lex_match_id (lexer, "COVARIANCE"))
+ factor.print |= PRINT_COVARIANCE;
+ else if (lex_match_id (lexer, "ROTATION"))
+ factor.print |= PRINT_ROTATION;
+ else if (lex_match_id (lexer, "EXTRACTION"))
+ factor.print |= PRINT_EXTRACTION;
+ else if (lex_match_id (lexer, "INITIAL"))
+ factor.print |= PRINT_INITIAL;
+ else if (lex_match_id (lexer, "KMO"))
+ factor.print |= PRINT_KMO;
+#if FACTOR_FULLY_IMPLEMENTED
+ else if (lex_match_id (lexer, "REPR"))
+ {
+ }
+ else if (lex_match_id (lexer, "FSCORE"))
+ {
+ }
+#endif
+ else if (lex_match (lexer, T_ALL))
+ factor.print = -1;
+ else if (lex_match_id (lexer, "DEFAULT"))
+ {
+ factor.print |= PRINT_INITIAL;
+ factor.print |= PRINT_EXTRACTION;
+ factor.print |= PRINT_ROTATION;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "UNIVARIATE", "DET", "AIC", "SIG",
+ "CORRELATION", "COVARIANCE", "ROTATION",
+ "EXTRACTION", "INITIAL", "KMO", "ALL",
+ "DEFAULT");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ factor.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ factor.exclude = MV_ANY;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ factor.missing_type = MISS_LISTWISE;
+ else if (lex_match_id (lexer, "PAIRWISE"))
+ factor.missing_type = MISS_PAIRWISE;
+ else if (lex_match_id (lexer, "MEANSUB"))
+ factor.missing_type = MISS_MEANSUB;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE", "LISTWISE",
+ "PAIRRWISE", "MEANSUB");
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "ANALYSIS", "PLOT", "METHOD", "ROTATION",
+ "CRITERIA", "EXTRACTION", "FORMAT", "PRINT",
+ "MISSING");
+ goto error;
+ }
+ }
+
+ if (factor.rotation == ROT_NONE)
+ factor.print &= ~PRINT_ROTATION;
+
+ assert (factor.n_vars > 0);
+ if (factor.n_vars < 2)
+ lex_ofs_msg (lexer, SW, vars_start, vars_end,
+ _("Factor analysis on a single variable is not useful."));
+
+ if (matrix_reader)
+ {
+ struct idata *id = idata_alloc (factor.n_vars);
+
+ while (matrix_reader_next (&id->mm, mr, NULL))
+ {
+ do_factor_by_matrix (&factor, id);
+
+ gsl_matrix_free (id->ai_cov);
+ id->ai_cov = NULL;
+ gsl_matrix_free (id->ai_cor);
+ id->ai_cor = NULL;
+
+ matrix_material_uninit (&id->mm);
+ }
+
+ idata_free (id);
+ }
+ else
+ if (!run_factor (ds, &factor))
+ goto error;
+
+ matrix_reader_destroy (mr);
+ free (factor.vars);
+ return CMD_SUCCESS;
+
+error:
+ matrix_reader_destroy (mr);
+ free (factor.vars);
+ return CMD_FAILURE;
+}
+
+static void do_factor (const struct cmd_factor *factor, struct casereader *group);
+
+
+static bool
+run_factor (struct dataset *ds, const struct cmd_factor *factor)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ bool ok;
+ struct casereader *group;
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
+
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ if (factor->missing_type == MISS_LISTWISE)
+ group = casereader_create_filter_missing (group, factor->vars, factor->n_vars,
+ factor->exclude,
+ NULL, NULL);
+ do_factor (factor, group);
+ }
+
+ ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ return ok;
+}
+
+
+/* Return the communality of variable N, calculated to N_FACTORS */
+static double
+the_communality (const gsl_matrix *evec, const gsl_vector *eval, int n, int n_factors)
+{
+ assert (n >= 0);
+ assert (n < eval->size);
+ assert (n < evec->size1);
+ assert (n_factors <= eval->size);
+
+ double comm = 0;
+ for (size_t i = 0; i < n_factors; ++i)
+ {
+ double evali = fabs (gsl_vector_get (eval, i));
+
+ double eveci = gsl_matrix_get (evec, n, i);
+
+ comm += pow2 (eveci) * evali;
+ }
+
+ return comm;
+}
+
+/* Return the communality of variable N, calculated to N_FACTORS */
+static double
+communality (const struct idata *idata, int n, int n_factors)
+{
+ return the_communality (idata->evec, idata->eval, n, n_factors);
+}
+
+
+static void
+show_scree (const struct cmd_factor *f, const struct idata *idata)
+{
+ struct scree *s;
+ const char *label;
+
+ if (!(f->plot & PLOT_SCREE))
+ return;
+
+
+ label = f->extraction == EXTRACTION_PC ? _("Component Number") : _("Factor Number");
+
+ s = scree_create (idata->eval, label);
+
+ scree_submit (s);
+}
+
+static void
+show_communalities (const struct cmd_factor * factor,
+ const gsl_vector *initial, const gsl_vector *extracted)
+{
+ if (!(factor->print & (PRINT_INITIAL | PRINT_EXTRACTION)))
+ return;
+
+ struct pivot_table *table = pivot_table_create (N_("Communalities"));
+
+ struct pivot_dimension *communalities = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Communalities"));
+ if (factor->print & PRINT_INITIAL)
+ pivot_category_create_leaves (communalities->root, N_("Initial"));
+ if (factor->print & PRINT_EXTRACTION)
+ pivot_category_create_leaves (communalities->root, N_("Extraction"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ {
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (factor->vars[i]));
+
+ int col = 0;
+ if (factor->print & PRINT_INITIAL)
+ pivot_table_put2 (table, col++, row, pivot_value_new_number (
+ gsl_vector_get (initial, i)));
+ if (factor->print & PRINT_EXTRACTION)
+ pivot_table_put2 (table, col++, row, pivot_value_new_number (
+ gsl_vector_get (extracted, i)));
+ }
+
+ pivot_table_submit (table);
+}
+
+static struct pivot_dimension *
+create_numeric_dimension (struct pivot_table *table,
+ enum pivot_axis_type axis_type, const char *name,
+ size_t n, bool show_label)
+{
+ struct pivot_dimension *d = pivot_dimension_create (table, axis_type, name);
+ d->root->show_label = show_label;
+ for (int i = 0; i < n; ++i)
+ pivot_category_create_leaf (d->root, pivot_value_new_integer (i + 1));
+ return d;
+}
+
+static void
+show_factor_matrix (const struct cmd_factor *factor, const struct idata *idata, const char *title, const gsl_matrix *fm)
+{
+ struct pivot_table *table = pivot_table_create (title);
+
+ const int n_factors = idata->n_extractions;
+ create_numeric_dimension (
+ table, PIVOT_AXIS_COLUMN,
+ factor->extraction == EXTRACTION_PC ? N_("Component") : N_("Factor"),
+ n_factors, true);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ /* Initialise to the identity permutation */
+ gsl_permutation *perm = gsl_permutation_calloc (factor->n_vars);
+
+ if (factor->sort)
+ sort_matrix_indirect (fm, perm);
+
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ {
+ const int matrix_row = perm->data[i];
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (factor->vars[matrix_row]));
+
+ for (size_t j = 0; j < n_factors; ++j)
+ {
+ double x = gsl_matrix_get (fm, matrix_row, j);
+ if (fabs (x) < factor->blank)
+ continue;
+
+ pivot_table_put2 (table, j, var_idx, pivot_value_new_number (x));
+ }
+ }
+
+ gsl_permutation_free (perm);
+
+ pivot_table_submit (table);
+}
+
+static void
+put_variance (struct pivot_table *table, int row, int phase_idx,
+ double lambda, double percent, double cum)
+{
+ double entries[] = { lambda, percent, cum };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put3 (table, i, phase_idx, row,
+ pivot_value_new_number (entries[i]));
+}
+
+static void
+show_explained_variance (const struct cmd_factor * factor,
+ const struct idata *idata,
+ const gsl_vector *initial_eigenvalues,
+ const gsl_vector *extracted_eigenvalues,
+ const gsl_vector *rotated_loadings)
+{
+ if (!(factor->print & (PRINT_INITIAL | PRINT_EXTRACTION | PRINT_ROTATION)))
+ return;
+
+ struct pivot_table *table = pivot_table_create (
+ N_("Total Variance Explained"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Total"), PIVOT_RC_OTHER,
+ /* xgettext:no-c-format */
+ N_("% of Variance"), PIVOT_RC_PERCENT,
+ /* xgettext:no-c-format */
+ N_("Cumulative %"), PIVOT_RC_PERCENT);
+
+ struct pivot_dimension *phase = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Phase"));
+ if (factor->print & PRINT_INITIAL)
+ pivot_category_create_leaves (phase->root, N_("Initial Eigenvalues"));
+
+ if (factor->print & PRINT_EXTRACTION)
+ pivot_category_create_leaves (phase->root,
+ N_("Extraction Sums of Squared Loadings"));
+
+ if (factor->print & PRINT_ROTATION)
+ pivot_category_create_leaves (phase->root,
+ N_("Rotation Sums of Squared Loadings"));
+
+ struct pivot_dimension *components = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW,
+ factor->extraction == EXTRACTION_PC ? N_("Component") : N_("Factor"));
+
+ double i_total = 0.0;
+ for (size_t i = 0; i < initial_eigenvalues->size; ++i)
+ i_total += gsl_vector_get (initial_eigenvalues, i);
+
+ double e_total = (factor->extraction == EXTRACTION_PAF
+ ? factor->n_vars
+ : i_total);
+
+ double i_cum = 0.0;
+ double e_cum = 0.0;
+ double r_cum = 0.0;
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ {
+ const double i_lambda = gsl_vector_get (initial_eigenvalues, i);
+ double i_percent = 100.0 * i_lambda / i_total;
+ i_cum += i_percent;
+
+ const double e_lambda = gsl_vector_get (extracted_eigenvalues, i);
+ double e_percent = 100.0 * e_lambda / e_total;
+ e_cum += e_percent;
+
+ int row = pivot_category_create_leaf (
+ components->root, pivot_value_new_integer (i + 1));
+
+ int phase_idx = 0;
+
+ /* Initial Eigenvalues */
+ if (factor->print & PRINT_INITIAL)
+ put_variance (table, row, phase_idx++, i_lambda, i_percent, i_cum);
+
+ if (i < idata->n_extractions)
+ {
+ if (factor->print & PRINT_EXTRACTION)
+ put_variance (table, row, phase_idx++, e_lambda, e_percent, e_cum);
+
+ if (rotated_loadings != NULL && factor->print & PRINT_ROTATION)
+ {
+ double r_lambda = gsl_vector_get (rotated_loadings, i);
+ double r_percent = 100.0 * r_lambda / e_total;
+ if (factor->rotation == ROT_PROMAX)
+ r_lambda = r_percent = SYSMIS;
+
+ r_cum += r_percent;
+ put_variance (table, row, phase_idx++, r_lambda, r_percent,
+ r_cum);
+ }
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_factor_correlation (const struct cmd_factor * factor, const gsl_matrix *fcm)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Factor Correlation Matrix"));
+
+ create_numeric_dimension (
+ table, PIVOT_AXIS_ROW,
+ factor->extraction == EXTRACTION_PC ? N_("Component") : N_("Factor"),
+ fcm->size2, true);
+
+ create_numeric_dimension (table, PIVOT_AXIS_COLUMN, N_("Factor 2"),
+ fcm->size1, false);
+
+ for (size_t i = 0; i < fcm->size1; ++i)
+ for (size_t j = 0; j < fcm->size2; ++j)
+ pivot_table_put2 (table, j, i,
+ pivot_value_new_number (gsl_matrix_get (fcm, i, j)));
+
+ pivot_table_submit (table);
+}
+
+static void
+add_var_dims (struct pivot_table *table, const struct cmd_factor *factor)
+{
+ for (int i = 0; i < 2; i++)
+ {
+ struct pivot_dimension *d = pivot_dimension_create (
+ table, i ? PIVOT_AXIS_ROW : PIVOT_AXIS_COLUMN,
+ N_("Variables"));
+
+ for (size_t j = 0; j < factor->n_vars; j++)
+ pivot_category_create_leaf (
+ d->root, pivot_value_new_variable (factor->vars[j]));
+ }
+}
+
+static void
+show_aic (const struct cmd_factor *factor, const struct idata *idata)
+{
+ if ((factor->print & PRINT_AIC) == 0)
+ return;
+
+ struct pivot_table *table = pivot_table_create (N_("Anti-Image Matrices"));
+
+ add_var_dims (table, factor);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Anti-image Covariance"),
+ N_("Anti-image Correlation"));
+
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ for (size_t j = 0; j < factor->n_vars; ++j)
+ {
+ double cov = gsl_matrix_get (idata->ai_cov, i, j);
+ pivot_table_put3 (table, i, j, 0, pivot_value_new_number (cov));
+
+ double corr = gsl_matrix_get (idata->ai_cor, i, j);
+ pivot_table_put3 (table, i, j, 1, pivot_value_new_number (corr));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_correlation_matrix (const struct cmd_factor *factor, const struct idata *idata)
+{
+ if (!(factor->print & (PRINT_CORRELATION | PRINT_SIG | PRINT_DETERMINANT)))
+ return;
+
+ struct pivot_table *table = pivot_table_create (N_("Correlation Matrix"));
+
+ if (factor->print & (PRINT_CORRELATION | PRINT_SIG))
+ {
+ add_var_dims (table, factor);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"));
+ if (factor->print & PRINT_CORRELATION)
+ pivot_category_create_leaves (statistics->root, N_("Correlation"),
+ PIVOT_RC_CORRELATION);
+ if (factor->print & PRINT_SIG)
+ pivot_category_create_leaves (statistics->root, N_("Sig. (1-tailed)"),
+ PIVOT_RC_SIGNIFICANCE);
+
+ int stat_idx = 0;
+ if (factor->print & PRINT_CORRELATION)
+ {
+ for (int i = 0; i < factor->n_vars; ++i)
+ for (int j = 0; j < factor->n_vars; ++j)
+ {
+ double corr = gsl_matrix_get (idata->mm.corr, i, j);
+ pivot_table_put3 (table, j, i, stat_idx,
+ pivot_value_new_number (corr));
+ }
+ stat_idx++;
+ }
+
+ if (factor->print & PRINT_SIG)
+ {
+ for (int i = 0; i < factor->n_vars; ++i)
+ for (int j = 0; j < factor->n_vars; ++j)
+ if (i != j)
+ {
+ double rho = gsl_matrix_get (idata->mm.corr, i, j);
+ double w = gsl_matrix_get (idata->mm.n, i, j);
+ double sig = significance_of_correlation (rho, w);
+ pivot_table_put3 (table, j, i, stat_idx,
+ pivot_value_new_number (sig));
+ }
+ stat_idx++;
+ }
+ }
+
+ if (factor->print & PRINT_DETERMINANT)
+ {
+ struct pivot_value *caption = pivot_value_new_user_text_nocopy (
+ xasprintf ("%s: %.2f", _("Determinant"), idata->detR));
+ pivot_table_set_caption (table, caption);
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_covariance_matrix (const struct cmd_factor *factor, const struct idata *idata)
+{
+ if (!(factor->print & PRINT_COVARIANCE))
+ return;
+
+ struct pivot_table *table = pivot_table_create (N_("Covariance Matrix"));
+ add_var_dims (table, factor);
+
+ for (int i = 0; i < factor->n_vars; ++i)
+ for (int j = 0; j < factor->n_vars; ++j)
+ {
+ double cov = gsl_matrix_get (idata->mm.cov, i, j);
+ pivot_table_put2 (table, j, i, pivot_value_new_number (cov));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+do_factor (const struct cmd_factor *factor, struct casereader *r)
+{
+ struct ccase *c;
+ struct idata *idata = idata_alloc (factor->n_vars);
+
+ idata->cvm = covariance_1pass_create (factor->n_vars, factor->vars,
+ factor->wv, factor->exclude, true);
+
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ covariance_accumulate (idata->cvm, c);
+ }
+
+ idata->mm.cov = covariance_calculate (idata->cvm);
+
+ if (idata->mm.cov == NULL)
+ {
+ msg (MW, _("The dataset contains no complete observations. No analysis will be performed."));
+ covariance_destroy (idata->cvm);
+ goto finish;
+ }
+
+ idata->mm.var_matrix = covariance_moments (idata->cvm, MOMENT_VARIANCE);
+ idata->mm.mean_matrix = covariance_moments (idata->cvm, MOMENT_MEAN);
+ idata->mm.n = covariance_moments (idata->cvm, MOMENT_NONE);
+
+ do_factor_by_matrix (factor, idata);
+
+ finish:
+ gsl_matrix_free (idata->mm.corr);
+ gsl_matrix_free (idata->mm.cov);
+
+ idata_free (idata);
+ casereader_destroy (r);
+}
+
+static void
+do_factor_by_matrix (const struct cmd_factor *factor, struct idata *idata)
+{
+ if (!idata->mm.cov && !(idata->mm.corr && idata->mm.var_matrix))
+ {
+ msg (ME, _("The dataset has no covariance matrix or a "
+ "correlation matrix along with standard deviations."));
+ return;
+ }
+
+ if (idata->mm.cov && !idata->mm.corr)
+ idata->mm.corr = correlation_from_covariance (idata->mm.cov, idata->mm.var_matrix);
+ if (idata->mm.corr && !idata->mm.cov)
+ idata->mm.cov = covariance_from_correlation (idata->mm.corr, idata->mm.var_matrix);
+ if (factor->method == METHOD_CORR)
+ idata->analysis_matrix = idata->mm.corr;
+ else
+ idata->analysis_matrix = idata->mm.cov;
+
+ gsl_matrix *r_inv;
+ r_inv = clone_matrix (idata->mm.corr);
+ gsl_linalg_cholesky_decomp (r_inv);
+ gsl_linalg_cholesky_invert (r_inv);
+
+ idata->ai_cov = anti_image_cov (r_inv);
+ idata->ai_cor = anti_image_corr (r_inv, idata);
+
+ double sum_ssq_r = 0;
+ double sum_ssq_a = 0;
+ for (int i = 0; i < r_inv->size1; ++i)
+ {
+ sum_ssq_r += ssq_od_n (idata->mm.corr, i);
+ sum_ssq_a += ssq_od_n (idata->ai_cor, i);
+ }
+
+ gsl_matrix_free (r_inv);
+
+ if (factor->print & PRINT_DETERMINANT
+ || factor->print & PRINT_KMO)
+ {
+ int sign = 0;
+
+ const int size = idata->mm.corr->size1;
+ gsl_permutation *p = gsl_permutation_calloc (size);
+ gsl_matrix *tmp = gsl_matrix_calloc (size, size);
+ gsl_matrix_memcpy (tmp, idata->mm.corr);
+
+ gsl_linalg_LU_decomp (tmp, p, &sign);
+ idata->detR = gsl_linalg_LU_det (tmp, sign);
+ gsl_permutation_free (p);
+ gsl_matrix_free (tmp);
+ }
+
+ if (factor->print & PRINT_UNIVARIATE
+ && idata->mm.n && idata->mm.mean_matrix && idata->mm.var_matrix)
+ {
+ struct pivot_table *table = pivot_table_create (
+ N_("Descriptive Statistics"));
+ pivot_table_set_weight_var (table, factor->wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("Analysis N"), PIVOT_RC_COUNT);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ {
+ const struct variable *v = factor->vars[i];
+
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (v));
+
+ double entries[] = {
+ gsl_matrix_get (idata->mm.mean_matrix, i, i),
+ sqrt (gsl_matrix_get (idata->mm.var_matrix, i, i)),
+ gsl_matrix_get (idata->mm.n, i, i),
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, row,
+ pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+ }
+
+ if (factor->print & PRINT_KMO && idata->mm.n)
+ {
+ struct pivot_table *table = pivot_table_create (
+ N_("KMO and Bartlett's Test"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Kaiser-Meyer-Olkin Measure of Sampling Adequacy"), PIVOT_RC_OTHER);
+ pivot_category_create_group (
+ statistics->root, N_("Bartlett's Test of Sphericity"),
+ N_("Approx. Chi-Square"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ /* The literature doesn't say what to do for the value of W when
+ missing values are involved. The best thing I can think of
+ is to take the mean average. */
+ double w = 0;
+ for (int i = 0; i < idata->mm.n->size1; ++i)
+ w += gsl_matrix_get (idata->mm.n, i, i);
+ w /= idata->mm.n->size1;
+
+ double xsq = ((w - 1 - (2 * factor->n_vars + 5) / 6.0)
+ * -log (idata->detR));
+ double df = factor->n_vars * (factor->n_vars - 1) / 2;
+ double entries[] = {
+ sum_ssq_r / (sum_ssq_r + sum_ssq_a),
+ xsq,
+ df,
+ gsl_cdf_chisq_Q (xsq, df)
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
+
+ pivot_table_submit (table);
+ }
+
+ show_correlation_matrix (factor, idata);
+ show_covariance_matrix (factor, idata);
+ if (idata->cvm)
+ covariance_destroy (idata->cvm);
+
+ {
+ gsl_matrix *am = matrix_dup (idata->analysis_matrix);
+ gsl_eigen_symmv_workspace *workspace = gsl_eigen_symmv_alloc (factor->n_vars);
+
+ gsl_eigen_symmv (am, idata->eval, idata->evec, workspace);
+
+ gsl_eigen_symmv_free (workspace);
+ gsl_matrix_free (am);
+ }
+
+ gsl_eigen_symmv_sort (idata->eval, idata->evec, GSL_EIGEN_SORT_ABS_DESC);
+
+ idata->n_extractions = n_extracted_factors (factor, idata);
+
+ if (idata->n_extractions == 0)
+ {
+ msg (MW, _("The %s criteria result in zero factors extracted. Therefore no analysis will be performed."), "FACTOR");
+ return;
+ }
+
+ if (idata->n_extractions > factor->n_vars)
+ {
+ msg (MW,
+ _("The %s criteria result in more factors than variables, which is not meaningful. No analysis will be performed."),
+ "FACTOR");
+ return;
+ }
+
+ {
+ gsl_matrix *rotated_factors = NULL;
+ gsl_matrix *pattern_matrix = NULL;
+ gsl_matrix *fcm = NULL;
+ gsl_vector *rotated_loadings = NULL;
+
+ const gsl_vector *extracted_eigenvalues = NULL;
+ gsl_vector *initial_communalities = gsl_vector_alloc (factor->n_vars);
+ gsl_vector *extracted_communalities = gsl_vector_alloc (factor->n_vars);
+ struct factor_matrix_workspace *fmw = factor_matrix_workspace_alloc (idata->msr->size, idata->n_extractions);
+ gsl_matrix *factor_matrix = gsl_matrix_calloc (factor->n_vars, fmw->n_factors);
+
+ if (factor->extraction == EXTRACTION_PAF)
+ {
+ gsl_vector *diff = gsl_vector_alloc (idata->msr->size);
+ struct smr_workspace *ws = ws_create (idata->analysis_matrix);
+
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ {
+ double r2 = squared_multiple_correlation (idata->analysis_matrix, i, ws);
+
+ gsl_vector_set (idata->msr, i, r2);
+ }
+ ws_destroy (ws);
+
+ gsl_vector_memcpy (initial_communalities, idata->msr);
+
+ for (size_t i = 0; i < factor->extraction_iterations; ++i)
+ {
+ double min, max;
+ gsl_vector_memcpy (diff, idata->msr);
+
+ iterate_factor_matrix (idata->analysis_matrix, idata->msr, factor_matrix, fmw);
+
+ gsl_vector_sub (diff, idata->msr);
+
+ gsl_vector_minmax (diff, &min, &max);
+
+ if (fabs (min) < factor->econverge && fabs (max) < factor->econverge)
+ break;
+ }
+ gsl_vector_free (diff);
+
+
+
+ gsl_vector_memcpy (extracted_communalities, idata->msr);
+ extracted_eigenvalues = fmw->eval;
+ }
+ else if (factor->extraction == EXTRACTION_PC)
+ {
+ for (size_t i = 0; i < factor->n_vars; ++i)
+ gsl_vector_set (initial_communalities, i, communality (idata, i, factor->n_vars));
+
+ gsl_vector_memcpy (extracted_communalities, initial_communalities);
+
+ iterate_factor_matrix (idata->analysis_matrix, extracted_communalities, factor_matrix, fmw);
+
+
+ extracted_eigenvalues = idata->eval;
+ }
+
+
+ show_aic (factor, idata);
+ show_communalities (factor, initial_communalities, extracted_communalities);
+
+ if (factor->rotation != ROT_NONE)
+ {
+ rotated_factors = gsl_matrix_calloc (factor_matrix->size1, factor_matrix->size2);
+ rotated_loadings = gsl_vector_calloc (factor_matrix->size2);
+ if (factor->rotation == ROT_PROMAX)
+ {
+ pattern_matrix = gsl_matrix_calloc (factor_matrix->size1, factor_matrix->size2);
+ fcm = gsl_matrix_calloc (factor_matrix->size2, factor_matrix->size2);
+ }
+
+
+ rotate (factor, factor_matrix, extracted_communalities, rotated_factors, rotated_loadings, pattern_matrix, fcm);
+ }
+
+ show_explained_variance (factor, idata, idata->eval, extracted_eigenvalues, rotated_loadings);
+
+ factor_matrix_workspace_free (fmw);
+
+ show_scree (factor, idata);
+
+ show_factor_matrix (factor, idata,
+ (factor->extraction == EXTRACTION_PC
+ ? N_("Component Matrix") : N_("Factor Matrix")),
+ factor_matrix);
+
+ if (factor->rotation == ROT_PROMAX)
+ {
+ show_factor_matrix (factor, idata, N_("Pattern Matrix"),
+ pattern_matrix);
+ gsl_matrix_free (pattern_matrix);
+ }
+
+ if (factor->rotation != ROT_NONE)
+ {
+ show_factor_matrix (factor, idata,
+ (factor->rotation == ROT_PROMAX
+ ? N_("Structure Matrix")
+ : factor->extraction == EXTRACTION_PC
+ ? N_("Rotated Component Matrix")
+ : N_("Rotated Factor Matrix")),
+ rotated_factors);
+
+ gsl_matrix_free (rotated_factors);
+ }
+
+ if (factor->rotation == ROT_PROMAX)
+ {
+ show_factor_correlation (factor, fcm);
+ gsl_matrix_free (fcm);
+ }
+
+ gsl_matrix_free (factor_matrix);
+ gsl_vector_free (rotated_loadings);
+ gsl_vector_free (initial_communalities);
+ gsl_vector_free (extracted_communalities);
+ }
+}
+
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2007, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/dataset.h"
+#include "data/transformations.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+\f
+/* A transformation which is guaranteed to fail. */
+
+static enum trns_result
+trns_fail (void *x UNUSED, struct ccase **c UNUSED,
+ casenumber n UNUSED)
+{
+ msg (SE, "DEBUG XFORM FAIL transformation executed");
+ return TRNS_ERROR;
+}
+
+int
+cmd_debug_xform_fail (struct lexer *lexer UNUSED, struct dataset *ds)
+{
+ static const struct trns_class fail_trns_class = {
+ .name = "DEBUG XFORM FAIL",
+ .execute = trns_fail
+ };
+ add_transformation (ds, &fail_trns_class, NULL);
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-2000, 2006, 2010-2013, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "data/file-handle-def.h"
+
+#include <limits.h>
+#include <errno.h>
+#include <stdlib.h>
+
+#include "data/file-name.h"
+#include "data/session.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_file_handle (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ enum cmd_result result = CMD_CASCADING_FAILURE;
+ char *handle_name = NULL;
+ char *file_name = NULL;
+ int lrecl = 0;
+ int tabwidth = -1;
+ enum { MODE_DEFAULT, MODE_CHARACTER, MODE_BINARY, MODE_IMAGE, MODE_360 }
+ mode = MODE_DEFAULT;
+ int ends = -1;
+ enum { RECFORM_FIXED = 1, RECFORM_VARIABLE, RECFORM_SPANNED } recform = 0;
+ char *encoding = NULL;
+
+ if (!lex_force_id (lexer))
+ goto exit;
+
+ handle_name = xstrdup (lex_tokcstr (lexer));
+ struct file_handle *fh = fh_from_id (handle_name);
+ if (fh)
+ {
+ fh_unref (fh);
+ lex_error (lexer, _("File handle %s is already defined. "
+ "Use %s before redefining a file handle."),
+ handle_name, "CLOSE FILE HANDLE");
+ goto exit;
+ }
+
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_SLASH))
+ goto exit;
+
+ int mode_start = 0;
+ int mode_end = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "NAME"))
+ {
+ if (file_name)
+ {
+ lex_sbc_only_once (lexer, "NAME");
+ goto exit;
+ }
+
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto exit;
+ free (file_name);
+ file_name = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "LRECL"))
+ {
+ if (lrecl)
+ {
+ lex_sbc_only_once (lexer, "LRECL");
+ goto exit;
+ }
+
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "LRECL", 1, (1UL << 31) - 1))
+ goto exit;
+ lrecl = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "TABWIDTH"))
+ {
+ if (tabwidth >= 0)
+ {
+ lex_sbc_only_once (lexer, "TABWIDTH");
+ goto exit;
+ }
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_int_range (lexer, "TABWIDTH", 1, INT_MAX))
+ goto exit;
+ tabwidth = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "MODE"))
+ {
+ if (mode != MODE_DEFAULT)
+ {
+ lex_sbc_only_once (lexer, "MODE");
+ goto exit;
+ }
+ mode_start = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "CHARACTER"))
+ mode = MODE_CHARACTER;
+ else if (lex_match_id (lexer, "BINARY"))
+ mode = MODE_BINARY;
+ else if (lex_match_id (lexer, "IMAGE"))
+ mode = MODE_IMAGE;
+ else if (lex_match_int (lexer, 360))
+ mode = MODE_360;
+ else
+ {
+ lex_error_expecting (lexer, "CHARACTER", "BINARY",
+ "IMAGE", "360");
+ goto exit;
+ }
+ mode_end = lex_ofs (lexer) - 1;
+ }
+ else if (lex_match_id (lexer, "ENDS"))
+ {
+ if (ends >= 0)
+ {
+ lex_sbc_only_once (lexer, "ENDS");
+ goto exit;
+ }
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "LF"))
+ ends = FH_END_LF;
+ else if (lex_match_id (lexer, "CRLF"))
+ ends = FH_END_CRLF;
+ else
+ {
+ lex_error_expecting (lexer, "LF", "CRLF");
+ goto exit;
+ }
+ }
+ else if (lex_match_id (lexer, "RECFORM"))
+ {
+ if (recform)
+ {
+ lex_sbc_only_once (lexer, "RECFORM");
+ goto exit;
+ }
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "FIXED") || lex_match_id (lexer, "F"))
+ recform = RECFORM_FIXED;
+ else if (lex_match_id (lexer, "VARIABLE")
+ || lex_match_id (lexer, "V"))
+ recform = RECFORM_VARIABLE;
+ else if (lex_match_id (lexer, "SPANNED")
+ || lex_match_id (lexer, "VS"))
+ recform = RECFORM_SPANNED;
+ else
+ {
+ lex_error_expecting (lexer, "FIXED", "VARIABLE", "SPANNED");
+ goto exit;
+ }
+ }
+ else if (lex_match_id (lexer, "ENCODING"))
+ {
+ if (encoding)
+ {
+ lex_sbc_only_once (lexer, "ENCODING");
+ goto exit;
+ }
+
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto exit;
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ if (!lex_match (lexer, T_SLASH))
+ break;
+ }
+
+ if (lex_end_of_command (lexer) != CMD_SUCCESS)
+ goto exit;
+
+ struct fh_properties properties = *fh_default_properties ();
+ if (file_name == NULL)
+ {
+ lex_sbc_missing (lexer, "NAME");
+ goto exit;
+ }
+
+ switch (mode)
+ {
+ case MODE_DEFAULT:
+ case MODE_CHARACTER:
+ properties.mode = FH_MODE_TEXT;
+ if (tabwidth >= 0)
+ properties.tab_width = tabwidth;
+ if (ends)
+ properties.line_ends = ends;
+ break;
+ case MODE_IMAGE:
+ properties.mode = FH_MODE_FIXED;
+ break;
+ case MODE_BINARY:
+ properties.mode = FH_MODE_VARIABLE;
+ break;
+ case MODE_360:
+ properties.encoding = CONST_CAST (char *, "EBCDIC-US");
+ if (recform == RECFORM_FIXED)
+ properties.mode = FH_MODE_FIXED;
+ else if (recform == RECFORM_VARIABLE)
+ {
+ properties.mode = FH_MODE_360_VARIABLE;
+ properties.record_width = 8192;
+ }
+ else if (recform == RECFORM_SPANNED)
+ {
+ properties.mode = FH_MODE_360_SPANNED;
+ properties.record_width = 8192;
+ }
+ else
+ {
+ lex_ofs_error (lexer, mode_start, mode_end,
+ _("%s must be specified with %s."),
+ "RECFORM", "MODE=360");
+ goto exit;
+ }
+ break;
+ default:
+ NOT_REACHED ();
+ }
+
+ if (properties.mode == FH_MODE_FIXED || lrecl)
+ {
+ if (!lrecl)
+ msg (SE, _("The specified file mode requires LRECL. "
+ "Assuming %zu-character records."),
+ properties.record_width);
+ else
+ properties.record_width = lrecl;
+ }
+
+ if (encoding)
+ properties.encoding = encoding;
+
+ fh_create_file (handle_name, file_name, lex_get_encoding (lexer),
+ &properties);
+
+ result = CMD_SUCCESS;
+
+exit:
+ free (handle_name);
+ free (file_name);
+ free (encoding);
+ return result;
+}
+
+int
+cmd_close_file_handle (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ struct file_handle *handle;
+
+ if (!lex_force_id (lexer))
+ return CMD_CASCADING_FAILURE;
+ handle = fh_from_id (lex_tokcstr (lexer));
+ if (handle == NULL)
+ return CMD_CASCADING_FAILURE;
+
+ fh_unname (handle);
+ return CMD_SUCCESS;
+}
+
+/* Returns the name for REFERENT. */
+static const char *
+referent_name (enum fh_referent referent)
+{
+ switch (referent)
+ {
+ case FH_REF_FILE:
+ return _("file");
+ case FH_REF_INLINE:
+ return _("inline file");
+ case FH_REF_DATASET:
+ return _("dataset");
+ default:
+ NOT_REACHED ();
+ }
+}
+
+/* Parses a file handle name:
+
+ - If SESSION is nonnull, then the parsed syntax may be the name of a
+ dataset within SESSION. Dataset names take precedence over file handle
+ names.
+
+ - If REFERENT_MASK includes FH_REF_FILE, the parsed syntax may be a file
+ name as a string or a file handle name as an identifier.
+
+ - If REFERENT_MASK includes FH_REF_INLINE, the parsed syntax may be the
+ identifier INLINE to represent inline data.
+
+ Returns the file handle when successful, a null pointer on failure.
+
+ The caller is responsible for fh_unref()'ing the returned file handle when
+ it is no longer needed. */
+struct file_handle *
+fh_parse (struct lexer *lexer, enum fh_referent referent_mask,
+ struct session *session)
+{
+ if (session != NULL && lex_token (lexer) == T_ID)
+ {
+ struct dataset *ds;
+
+ ds = session_lookup_dataset (session, lex_tokcstr (lexer));
+ if (ds != NULL)
+ {
+ lex_get (lexer);
+ return fh_create_dataset (ds);
+ }
+ }
+
+ int start_ofs = lex_ofs (lexer);
+ struct file_handle *handle;
+ if (lex_match_id (lexer, "INLINE"))
+ handle = fh_inline_file ();
+ else
+ {
+ if (lex_token (lexer) != T_ID && !lex_is_string (lexer))
+ {
+ lex_error (lexer,
+ _("Syntax error expecting a file name or handle name."));
+ return NULL;
+ }
+
+ handle = NULL;
+ if (lex_token (lexer) == T_ID)
+ handle = fh_from_id (lex_tokcstr (lexer));
+ if (handle == NULL)
+ handle = fh_create_file (NULL, lex_tokcstr (lexer), lex_get_encoding (lexer),
+ fh_default_properties ());
+ lex_get (lexer);
+ }
+
+ if (!(fh_get_referent (handle) & referent_mask))
+ {
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("Handle for %s not allowed here."),
+ referent_name (fh_get_referent (handle)));
+ fh_unref (handle);
+ return NULL;
+ }
+
+ return handle;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef LANGUAGE_DATA_IO_FILE_HANDLE_H
+#define LANGUAGE_DATA_IO_FILE_HANDLE_H 1
+
+/* Parsing file handles. */
+
+#include <stdbool.h>
+#include <stddef.h>
+#include "data/file-handle-def.h"
+
+struct lexer;
+struct session;
+
+struct file_handle *fh_parse (struct lexer *, enum fh_referent,
+ struct session *);
+
+#endif /* language/commands/file-handle.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2013 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <errno.h>
+#include <float.h>
+#include <limits.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/casereader-provider.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/settings.h"
+#include "data/short-names.h"
+#include "data/value.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+#include "data/data-in.h"
+#include "data/data-out.h"
+
+#include "gl/intprops.h"
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* List of variable names. */
+struct var_names
+ {
+ const char **names;
+ size_t n_names, allocated_names;
+ };
+
+static void var_names_init (struct var_names *);
+static void var_names_add (struct pool *, struct var_names *, const char *);
+
+/* Represents a FLIP input program. */
+struct flip_pgm
+ {
+ struct pool *pool; /* Pool containing FLIP data. */
+ size_t n_vars; /* Pre-flip number of variables. */
+ int n_cases; /* Pre-flip number of cases. */
+
+ struct variable *new_names_var; /* Variable with new variable names. */
+ const char *encoding; /* Variable names' encoding. */
+ struct var_names old_names; /* Variable names before FLIP. */
+ struct var_names new_names; /* Variable names after FLIP. */
+
+ FILE *file; /* Temporary file containing data. */
+ size_t cases_read; /* Number of cases already read. */
+ bool error; /* Error reading temporary file? */
+ };
+
+static const struct casereader_class flip_casereader_class;
+
+static void destroy_flip_pgm (struct flip_pgm *);
+static bool flip_file (struct flip_pgm *);
+static void make_new_var (struct dictionary *, const char *name);
+
+/* Parses and executes FLIP. */
+int
+cmd_flip (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *old_dict = dataset_dict (ds);
+ struct dictionary *new_dict = NULL;
+ const struct variable **vars;
+ struct flip_pgm *flip;
+ struct casereader *input, *reader;
+ struct ccase *c;
+ size_t i;
+ bool ok;
+
+ if (proc_make_temporary_transformations_permanent (ds))
+ lex_ofs_msg (lexer, SW, 0, lex_ofs (lexer) - 1,
+ _("%s ignores %s. "
+ "Temporary transformations will be made permanent."),
+ "FLIP", "TEMPORARY");
+
+ flip = pool_create_container (struct flip_pgm, pool);
+ flip->n_vars = 0;
+ flip->n_cases = 0;
+ flip->new_names_var = NULL;
+ var_names_init (&flip->old_names);
+ var_names_init (&flip->new_names);
+ flip->file = NULL;
+ flip->cases_read = 0;
+ flip->error = false;
+
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "VARIABLES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_variables_const (lexer, old_dict, &vars, &flip->n_vars,
+ PV_NO_DUPLICATE))
+ goto error;
+ lex_match (lexer, T_SLASH);
+ }
+ else
+ dict_get_vars (old_dict, &vars, &flip->n_vars, DC_SYSTEM);
+ pool_register (flip->pool, free, vars);
+
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "NEWNAMES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ flip->new_names_var = parse_variable (lexer, old_dict);
+ if (!flip->new_names_var)
+ goto error;
+ }
+ else
+ flip->new_names_var = dict_lookup_var (old_dict, "CASE_LBL");
+
+ if (flip->new_names_var)
+ {
+ for (i = 0; i < flip->n_vars; i++)
+ if (vars[i] == flip->new_names_var)
+ {
+ remove_element (vars, flip->n_vars, sizeof *vars, i);
+ flip->n_vars--;
+ break;
+ }
+ }
+ if (flip->n_vars <= 0)
+ goto error;
+
+ flip->file = pool_create_temp_file (flip->pool);
+ if (flip->file == NULL)
+ {
+ msg (SE, _("Could not create temporary file for %s."), "FLIP");
+ goto error;
+ }
+
+ /* Save old variable names for use as values of CASE_LBL
+ variable in flipped file. */
+ for (i = 0; i < flip->n_vars; i++)
+ var_names_add (flip->pool, &flip->old_names,
+ pool_strdup (flip->pool, var_get_name (vars[i])));
+
+ /* Read the active dataset into a flip_sink. */
+ proc_discard_output (ds);
+
+ /* Save old dictionary. */
+ new_dict = dict_clone (old_dict);
+ flip->encoding = dict_get_encoding (new_dict);
+ dict_clear (new_dict);
+
+ input = proc_open_filtering (ds, false);
+ while ((c = casereader_read (input)) != NULL)
+ {
+ flip->n_cases++;
+ for (i = 0; i < flip->n_vars; i++)
+ {
+ const struct variable *v = vars[i];
+ double out = var_is_numeric (v) ? case_num (c, v) : SYSMIS;
+ fwrite (&out, sizeof out, 1, flip->file);
+ }
+ if (flip->new_names_var != NULL)
+ {
+ const union value *value = case_data (c, flip->new_names_var);
+ const char *name;
+ if (var_is_numeric (flip->new_names_var))
+ {
+ double f = value->f;
+ name = (f == SYSMIS ? "VSYSMIS"
+ : f < INT_MIN ? "VNEGINF"
+ : f > INT_MAX ? "VPOSINF"
+ : pool_asprintf (flip->pool, "V%d", (int) f));
+ }
+ else
+ {
+ name = data_out_pool (value, dict_get_encoding (old_dict),
+ var_get_write_format (flip->new_names_var),
+ settings_get_fmt_settings (), flip->pool);
+ }
+ var_names_add (flip->pool, &flip->new_names, name);
+ }
+ case_unref (c);
+ }
+ ok = casereader_destroy (input);
+ ok = proc_commit (ds) && ok;
+
+ /* Flip the data we read. */
+ if (!ok || !flip_file (flip))
+ {
+ dataset_clear (ds);
+ goto error;
+ }
+
+ /* Flip the dictionary. */
+ dict_create_var_assert (new_dict, "CASE_LBL", 8);
+ for (i = 0; i < flip->n_cases; i++)
+ if (flip->new_names.n_names)
+ make_new_var (new_dict, flip->new_names.names[i]);
+ else
+ {
+ char s[3 + INT_STRLEN_BOUND (i) + 1];
+ sprintf (s, "VAR%03zu", i);
+ dict_create_var_assert (new_dict, s, 0);
+ }
+
+ /* Set up flipped data for reading. */
+ reader = casereader_create_sequential (NULL, dict_get_proto (new_dict),
+ flip->n_vars,
+ &flip_casereader_class, flip);
+ dataset_set_dict (ds, new_dict);
+ dataset_set_source (ds, reader);
+ return CMD_SUCCESS;
+
+ error:
+ dict_unref (new_dict);
+ destroy_flip_pgm (flip);
+ return CMD_CASCADING_FAILURE;
+}
+
+/* Destroys FLIP. */
+static void
+destroy_flip_pgm (struct flip_pgm *flip)
+{
+ if (flip != NULL)
+ pool_destroy (flip->pool);
+}
+
+/* Make a new variable with base name NAME, which is bowdlerized and
+ mangled until acceptable. */
+static void
+make_new_var (struct dictionary *dict, const char *name_)
+{
+ char *name = xstrdup (name_);
+ char *cp;
+
+ /* Trim trailing spaces. */
+ cp = strchr (name, '\0');
+ while (cp > name && isspace ((unsigned char) cp[-1]))
+ *--cp = '\0';
+
+ /* Fix invalid characters. */
+ for (cp = name; *cp && cp < name + ID_MAX_LEN; cp++)
+ if (cp == name)
+ {
+ if (!lex_is_id1 (*cp) || *cp == '$')
+ *cp = 'V';
+ }
+ else
+ {
+ if (!lex_is_idn (*cp))
+ *cp = '_';
+ }
+ *cp = '\0';
+
+ if (strlen (name) == 0)
+ {
+ free (name);
+ name = xstrdup ("v");
+ }
+
+ /* Use the mangled name, if it is available, or add numeric
+ extensions until we find one that is. */
+ if (!id_is_plausible (name) || !dict_create_var (dict, name, 0))
+ {
+ int len = strlen (name);
+ int i;
+ for (i = 1; ; i++)
+ {
+ char n[ID_MAX_LEN + 1];
+ int ofs = MIN (ID_MAX_LEN - 1 - intlog10 (i), len);
+ strncpy (n, name, ofs);
+ sprintf (&n[ofs], "%d", i);
+
+ if (id_is_plausible (n) && dict_create_var (dict, n, 0))
+ break;
+ }
+ }
+ free (name);
+}
+
+/* Transposes the external file into a new file. */
+static bool
+flip_file (struct flip_pgm *flip)
+{
+ size_t case_bytes;
+ size_t case_capacity;
+ size_t case_idx;
+ double *input_buf, *output_buf;
+ FILE *input_file, *output_file;
+
+ /* Allocate memory for many cases. */
+ case_bytes = flip->n_vars * sizeof *input_buf;
+ case_capacity = settings_get_workspace () / case_bytes;
+ if (case_capacity > flip->n_cases * 2)
+ case_capacity = flip->n_cases * 2;
+ if (case_capacity < 2)
+ case_capacity = 2;
+ for (;;)
+ {
+ size_t bytes = case_bytes * case_capacity;
+ if (case_capacity > 2)
+ input_buf = malloc (bytes);
+ else
+ input_buf = xmalloc (bytes);
+ if (input_buf != NULL)
+ break;
+
+ case_capacity /= 2;
+ if (case_capacity < 2)
+ case_capacity = 2;
+ }
+ pool_register (flip->pool, free, input_buf);
+
+ /* Use half the allocated memory for input_buf, half for
+ output_buf. */
+ case_capacity /= 2;
+ output_buf = input_buf + flip->n_vars * case_capacity;
+
+ input_file = flip->file;
+ if (fseeko (input_file, 0, SEEK_SET) != 0)
+ {
+ msg (SE, _("Error rewinding %s file: %s."), "FLIP", strerror (errno));
+ return false;
+ }
+
+ output_file = pool_create_temp_file (flip->pool);
+ if (output_file == NULL)
+ {
+ msg (SE, _("Error creating %s source file."), "FLIP");
+ return false;
+ }
+
+ for (case_idx = 0; case_idx < flip->n_cases;)
+ {
+ unsigned long read_cases = MIN (flip->n_cases - case_idx,
+ case_capacity);
+ size_t i;
+
+ if (read_cases != fread (input_buf, case_bytes, read_cases, input_file))
+ {
+ if (ferror (input_file))
+ msg (SE, _("Error reading %s file: %s."), "FLIP", strerror (errno));
+ else
+ msg (SE, _("Unexpected end of file reading %s file."), "FLIP");
+ return false;
+ }
+
+ for (i = 0; i < flip->n_vars; i++)
+ {
+ unsigned long j;
+
+ for (j = 0; j < read_cases; j++)
+ output_buf[j] = input_buf[i + j * flip->n_vars];
+
+ if (fseeko (output_file,
+ sizeof *input_buf * (case_idx
+ + (off_t) i * flip->n_cases),
+ SEEK_SET) != 0)
+ {
+ msg (SE, _("Error seeking %s source file: %s."), "FLIP",
+ strerror (errno));
+ return false;
+ }
+
+ if (fwrite (output_buf, sizeof *output_buf, read_cases, output_file)
+ != read_cases)
+ {
+ msg (SE, _("Error writing %s source file: %s."), "FLIP",
+ strerror (errno));
+ return false;
+ }
+ }
+
+ case_idx += read_cases;
+ }
+
+ pool_fclose_temp_file (flip->pool, input_file);
+ pool_unregister (flip->pool, input_buf);
+ free (input_buf);
+
+ if (fseeko (output_file, 0, SEEK_SET) != 0)
+ {
+ msg (SE, _("Error rewinding %s source file: %s."), "FLIP", strerror (errno));
+ return false;
+ }
+ flip->file = output_file;
+
+ return true;
+}
+
+/* Reads and returns one case.
+ Returns a null pointer at end of file or if an I/O error occurred. */
+static struct ccase *
+flip_casereader_read (struct casereader *reader, void *flip_)
+{
+ struct flip_pgm *flip = flip_;
+ struct ccase *c;
+ size_t i;
+
+ if (flip->error || flip->cases_read >= flip->n_vars)
+ return false;
+
+ c = case_create (casereader_get_proto (reader));
+ data_in (ss_cstr (flip->old_names.names[flip->cases_read]), flip->encoding,
+ FMT_A, settings_get_fmt_settings (), case_data_rw_idx (c, 0),
+ 8, flip->encoding);
+
+ for (i = 0; i < flip->n_cases; i++)
+ {
+ double in;
+ if (fread (&in, sizeof in, 1, flip->file) != 1)
+ {
+ case_unref (c);
+ if (ferror (flip->file))
+ msg (SE, _("Error reading %s temporary file: %s."), "FLIP",
+ strerror (errno));
+ else if (feof (flip->file))
+ msg (SE, _("Unexpected end of file reading %s temporary file."), "FLIP");
+ else
+ NOT_REACHED ();
+ flip->error = true;
+ return NULL;
+ }
+ *case_num_rw_idx (c, i + 1) = in;
+ }
+
+ flip->cases_read++;
+
+ return c;
+}
+
+/* Destroys the source.
+ Returns true if successful read, false if an I/O occurred
+ during destruction or previously. */
+static void
+flip_casereader_destroy (struct casereader *reader, void *flip_)
+{
+ struct flip_pgm *flip = flip_;
+ if (flip->error)
+ casereader_force_error (reader);
+ destroy_flip_pgm (flip);
+}
+
+static const struct casereader_class flip_casereader_class =
+ {
+ flip_casereader_read,
+ flip_casereader_destroy,
+ NULL,
+ NULL,
+ };
+\f
+static void
+var_names_init (struct var_names *vn)
+{
+ vn->names = NULL;
+ vn->n_names = 0;
+ vn->allocated_names = 0;
+}
+
+static void
+var_names_add (struct pool *pool, struct var_names *vn, const char *name)
+{
+ if (vn->n_names >= vn->allocated_names)
+ vn->names = pool_2nrealloc (pool, vn->names, &vn->allocated_names,
+ sizeof *vn->names);
+ vn->names[vn->n_names++] = name;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/str.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+static int cmd_formats__ (struct lexer *, struct dataset *ds,
+ bool print_format, bool write_format);
+
+int
+cmd_print_formats (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_formats__ (lexer, ds, true, false);
+}
+
+int
+cmd_write_formats (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_formats__ (lexer, ds, false, true);
+}
+
+int
+cmd_formats (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_formats__ (lexer, ds, true, true);
+}
+
+static int
+cmd_formats__ (struct lexer *lexer, struct dataset *ds,
+ bool print_format, bool write_format)
+{
+ /* Variables. */
+ struct variable **v;
+ size_t cv;
+
+ for (;;)
+ {
+ struct fmt_spec f;
+ int width;
+ size_t i;
+
+ lex_match (lexer, T_SLASH);
+
+ if (lex_token (lexer) == T_ENDCMD)
+ break;
+
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &cv, PV_SAME_WIDTH))
+ return CMD_FAILURE;
+ width = var_get_width (v[0]);
+
+ if (!lex_match (lexer, T_LPAREN))
+ {
+ lex_error_expecting (lexer, "`('");
+ goto fail;
+ }
+ if (!parse_format_specifier (lexer, &f))
+ goto fail;
+ char *error = fmt_check_output__ (&f);
+ if (!error)
+ error = fmt_check_width_compat__ (&f, var_get_name (v[0]), width);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ goto fail;
+ }
+
+ if (!lex_match (lexer, T_RPAREN))
+ {
+ lex_error_expecting (lexer, "`)'");
+ goto fail;
+ }
+
+ for (i = 0; i < cv; i++)
+ {
+ if (print_format)
+ var_set_print_format (v[i], &f);
+ if (write_format)
+ var_set_write_format (v[i], &f);
+ }
+ free (v);
+ v = NULL;
+ }
+ return CMD_SUCCESS;
+
+fail:
+ free (v);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2009, 2010 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/freq.h"
+
+#include <stdlib.h>
+
+#include "data/variable.h"
+#include "data/value.h"
+#include "libpspp/array.h"
+#include "libpspp/compiler.h"
+
+struct freq *
+freq_clone (const struct freq *in, int values, int *widths)
+{
+ int i;
+ struct freq *f = xmalloc (sizeof (struct freq) +
+ (sizeof (union value) * (values - 1)));
+
+ f->node = in->node;
+ f->count = in->count;
+ for (i = 0; i < values; ++i)
+ {
+ value_init (&f->values[i], widths[i]);
+ value_copy (&f->values[i], &in->values[i], widths[i]);
+ }
+
+ return f;
+}
+
+void
+freq_destroy (struct freq *f, int values, int *widths)
+{
+ int i;
+ for (i = 0; i < values; ++i)
+ {
+ value_destroy (&f->values[i], widths[i]);
+ }
+
+ free (f);
+}
+
+
+
+void
+freq_hmap_destroy (struct hmap *hmap, int width)
+{
+ struct freq *f, *next;
+
+ HMAP_FOR_EACH_SAFE (f, next, struct freq, node, hmap)
+ {
+ value_destroy (&f->values[0], width);
+ hmap_delete (hmap, &f->node);
+ free (f);
+ }
+ hmap_destroy (hmap);
+}
+
+struct freq *
+freq_hmap_search (struct hmap *hmap,
+ const union value *value, int width, size_t hash)
+{
+ struct freq *f;
+
+ HMAP_FOR_EACH_WITH_HASH (f, struct freq, node, hash, hmap)
+ if (value_equal (value, &f->values[0], width))
+ return f;
+
+ return NULL;
+}
+
+struct freq *
+freq_hmap_insert (struct hmap *hmap,
+ const union value *value, int width, size_t hash)
+{
+ struct freq *f = xmalloc (sizeof *f);
+ value_clone (&f->values[0], value, width);
+ f->count = 0;
+ hmap_insert (hmap, &f->node, hash);
+ return f;
+}
+
+int
+compare_freq_ptr_3way (const void *a_, const void *b_, const void *width_)
+{
+ const struct freq *const *ap = a_;
+ const struct freq *const *bp = b_;
+ const int *widthp = width_;
+
+ return value_compare_3way (&(*ap)->values[0], &(*bp)->values[0], *widthp);
+}
+
+struct freq **
+freq_hmap_sort (struct hmap *hmap, int width)
+{
+ size_t n_entries = hmap_count (hmap);
+ struct freq **entries;
+ struct freq *f;
+ size_t i;
+
+ entries = xnmalloc (n_entries, sizeof *entries);
+ i = 0;
+ HMAP_FOR_EACH (f, struct freq, node, hmap)
+ entries[i++] = f;
+ assert (i == n_entries);
+
+ sort (entries, n_entries, sizeof *entries, compare_freq_ptr_3way, &width);
+
+ return entries;
+}
+
+struct freq *
+freq_hmap_extract (struct hmap *hmap)
+{
+ struct freq *freqs, *f;
+ size_t n_freqs;
+ size_t i;
+
+ n_freqs = hmap_count (hmap);
+ freqs = xnmalloc (n_freqs, sizeof *freqs);
+ i = 0;
+ HMAP_FOR_EACH (f, struct freq, node, hmap)
+ freqs[i++] = *f;
+ assert (i == n_freqs);
+
+ return freqs;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2010, 2015 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef LANGUAGE_STATS_FREQ_H
+#define LANGUAGE_STATS_FREQ_H 1
+
+#include "data/value.h"
+#include "libpspp/hmap.h"
+
+/* Frequency table entry. */
+struct freq
+ {
+ struct hmap_node node; /* Element in hash table. */
+ double count; /* The number of occurrences of the value. */
+ union value values[1]; /* The value. */
+ };
+
+
+struct freq *freq_clone (const struct freq *, int values, int *widths);
+void freq_destroy (struct freq *f, int values, int *widths);
+
+
+static inline size_t
+table_entry_size (size_t n_values)
+{
+ return (offsetof (struct freq, values)
+ + n_values * sizeof (union value));
+}
+
+
+int compare_freq_ptr_3way (const void *a_, const void *b_, const void *width_);
+
+
+
+void freq_hmap_destroy (struct hmap *, int width);
+
+struct freq *freq_hmap_search (struct hmap *, const union value *, int width,
+ size_t hash);
+struct freq *freq_hmap_insert (struct hmap *, const union value *, int width,
+ size_t hash);
+
+struct freq **freq_hmap_sort (struct hmap *, int width);
+struct freq *freq_hmap_extract (struct hmap *);
+
+
+
+
+#endif /* language/commands/freq.h */
--- /dev/null
+/*
+ PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2007, 2009, 2010, 2011, 2014, 2015 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <config.h>
+#include <stdlib.h>
+#include <gsl/gsl_histogram.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "data/settings.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+
+#include "language/commands/split-file.h"
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/freq.h"
+
+#include "libpspp/array.h"
+#include "libpspp/bit-vector.h"
+#include "libpspp/compiler.h"
+#include "libpspp/hmap.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+
+#include "math/histogram.h"
+#include "math/moments.h"
+#include "math/chart-geometry.h"
+
+
+#include "output/charts/barchart.h"
+#include "output/charts/piechart.h"
+#include "output/charts/plot-hist.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+/* Percentiles to calculate. */
+
+struct percentile
+ {
+ double p; /* The percentile to calculate, between 0 and 1. */
+ bool show; /* True to show this percentile in the statistics box. */
+ };
+
+static int
+percentile_compare_3way (const void *a_, const void *b_)
+{
+ const struct percentile *a = a_;
+ const struct percentile *b = b_;
+
+ return a->p < b->p ? -1 : a->p > b->p;
+}
+
+enum
+ {
+ FRQ_FREQ,
+ FRQ_PERCENT
+ };
+
+enum sortprops
+ {
+ FRQ_AFREQ,
+ FRQ_DFREQ,
+ FRQ_AVALUE,
+ FRQ_DVALUE
+ };
+
+#define STATISTICS \
+ S(FRQ_ST_MEAN, "MEAN", N_("Mean")) \
+ S(FRQ_ST_SEMEAN, "SEMEAN", N_("S.E. Mean")) \
+ S(FRQ_ST_MEDIAN, "MEDIAN", N_("Median")) \
+ S(FRQ_ST_MODE, "MODE", N_("Mode")) \
+ S(FRQ_ST_STDDEV, "STDDEV", N_("Std Dev")) \
+ S(FRQ_ST_VARIANCE, "VARIANCE", N_("Variance")) \
+ S(FRQ_ST_KURTOSIS, "KURTOSIS", N_("Kurtosis")) \
+ S(FRQ_ST_SEKURTOSIS, "SEKURTOSIS",N_("S.E. Kurt")) \
+ S(FRQ_ST_SKEWNESS, "SKEWNESS", N_("Skewness")) \
+ S(FRQ_ST_SESKEWNESS, "SESKEWNESS",N_("S.E. Skew")) \
+ S(FRQ_ST_RANGE, "RANGE", N_("Range")) \
+ S(FRQ_ST_MINIMUM, "MINIMUM", N_("Minimum")) \
+ S(FRQ_ST_MAXIMUM, "MAXIMUM", N_("Maximum")) \
+ S(FRQ_ST_SUM, "SUM", N_("Sum"))
+
+enum frq_statistic
+ {
+#define S(ENUM, KEYWORD, NAME) ENUM,
+STATISTICS
+#undef S
+ };
+
+enum {
+#define S(ENUM, KEYWORD, NAME) +1
+ FRQ_ST_count = STATISTICS,
+#undef S
+};
+
+static const char *st_keywords[FRQ_ST_count] = {
+#define S(ENUM, KEYWORD, NAME) KEYWORD,
+ STATISTICS
+#undef S
+};
+
+static const char *st_names[FRQ_ST_count] = {
+#define S(ENUM, KEYWORD, NAME) NAME,
+ STATISTICS
+#undef S
+};
+
+struct freq_tab
+ {
+ struct hmap data; /* Hash table for accumulating counts. */
+ struct freq *valid; /* Valid freqs. */
+ size_t n_valid; /* Number of total freqs. */
+ const struct dictionary *dict; /* Source of entries in the table. */
+
+ struct freq *missing; /* Missing freqs. */
+ size_t n_missing; /* Number of missing freqs. */
+
+ /* Statistics. */
+ double total_cases; /* Sum of weights of all cases. */
+ double valid_cases; /* Sum of weights of valid cases. */
+ };
+
+struct frq_chart
+ {
+ double x_min; /* X axis minimum value. */
+ double x_max; /* X axis maximum value. */
+ int y_scale; /* Y axis scale: FRQ_FREQ or FRQ_PERCENT. */
+
+ /* Histograms only. */
+ double y_max; /* Y axis maximum value. */
+ bool draw_normal; /* Whether to draw normal curve. */
+
+ /* Pie charts only. */
+ bool include_missing; /* Whether to include missing values. */
+ };
+
+/* Per-variable frequency data. */
+struct var_freqs
+ {
+ const struct variable *var;
+
+ /* Freqency table. */
+ struct freq_tab tab; /* Frequencies table to use. */
+
+ /* Statistics. */
+ double stat[FRQ_ST_count];
+ double *percentiles;
+
+ /* Variable attributes. */
+ int width;
+ };
+
+struct frq_proc
+ {
+ struct var_freqs *vars;
+ size_t n_vars;
+
+ /* Percentiles to calculate and possibly display. */
+ struct percentile *percentiles;
+ size_t median_idx;
+ size_t n_percentiles;
+
+ /* Frequency table display. */
+ long int max_categories; /* Maximum categories to show. */
+ int sort; /* FRQ_AVALUE or FRQ_DVALUE
+ or FRQ_AFREQ or FRQ_DFREQ. */
+
+ /* Statistics. */
+ unsigned long stats;
+
+ /* Histogram and pie chart settings. */
+ struct frq_chart *hist, *pie, *bar;
+
+ bool warn;
+ };
+
+
+struct freq_compare_aux
+ {
+ bool by_freq;
+ bool ascending_freq;
+
+ int width;
+ bool ascending_value;
+ };
+
+static void calc_stats (const struct frq_proc *,
+ const struct var_freqs *, double d[FRQ_ST_count]);
+
+static void do_piechart(const struct frq_chart *pie,
+ const struct variable *var,
+ const struct freq_tab *frq_tab);
+
+static void do_barchart(const struct frq_chart *bar,
+ const struct variable **var,
+ const struct freq_tab *frq_tab);
+
+static struct frq_stats_table *frq_stats_table_submit (
+ struct frq_stats_table *, const struct frq_proc *,
+ const struct dictionary *, const struct variable *wv,
+ const struct ccase *example);
+static void frq_stats_table_destroy (struct frq_stats_table *);
+
+static int
+compare_freq (const void *a_, const void *b_, const void *aux_)
+{
+ const struct freq_compare_aux *aux = aux_;
+ const struct freq *a = a_;
+ const struct freq *b = b_;
+
+ if (aux->by_freq && a->count != b->count)
+ {
+ int cmp = a->count > b->count ? 1 : -1;
+ return aux->ascending_freq ? cmp : -cmp;
+ }
+ else
+ {
+ int cmp = value_compare_3way (a->values, b->values, aux->width);
+ return aux->ascending_value ? cmp : -cmp;
+ }
+}
+
+/* Create a gsl_histogram from a freq_tab */
+static struct histogram *freq_tab_to_hist (const struct frq_proc *,
+ const struct var_freqs *);
+
+static void
+put_freq_row (struct pivot_table *table, int var_idx,
+ double frequency, double percent,
+ double valid_percent, double cum_percent)
+{
+ double entries[] = { frequency, percent, valid_percent, cum_percent };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ if (entries[i] != SYSMIS)
+ pivot_table_put2 (table, i, var_idx,
+ pivot_value_new_number (entries[i]));
+}
+
+/* Displays a full frequency table for variable V. */
+static void
+dump_freq_table (const struct var_freqs *vf, const struct variable *wv)
+{
+ const struct freq_tab *ft = &vf->tab;
+
+ struct pivot_table *table = pivot_table_create__ (pivot_value_new_variable (
+ vf->var), "Frequencies");
+ pivot_table_set_weight_var (table, wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Frequency"), PIVOT_RC_COUNT,
+ N_("Percent"), PIVOT_RC_PERCENT,
+ N_("Valid Percent"), PIVOT_RC_PERCENT,
+ N_("Cumulative Percent"), PIVOT_RC_PERCENT);
+
+ struct pivot_dimension *variable = pivot_dimension_create__ (
+ table, PIVOT_AXIS_ROW, pivot_value_new_variable (vf->var));
+
+ double cum_freq = 0.0;
+ double cum_percent = 0.0;
+ struct pivot_category *valid = NULL;
+ for (const struct freq *f = ft->valid; f < ft->missing; f++)
+ {
+ cum_freq += f->count;
+ double valid_percent = f->count / ft->valid_cases * 100.0;
+ cum_percent += valid_percent;
+
+ if (!valid)
+ valid = pivot_category_create_group (variable->root, N_("Valid"));
+ int var_idx = pivot_category_create_leaf (
+ valid, pivot_value_new_var_value (vf->var, &f->values[0]));
+ put_freq_row (table, var_idx, f->count,
+ f->count / ft->total_cases * 100.0,
+ valid_percent, cum_percent);
+ }
+
+ struct pivot_category *missing = NULL;
+ size_t n_categories = ft->n_valid + ft->n_missing;
+ for (const struct freq *f = ft->missing; f < &ft->valid[n_categories]; f++)
+ {
+ cum_freq += f->count;
+
+ if (!missing)
+ missing = pivot_category_create_group (variable->root, N_("Missing"));
+ int var_idx = pivot_category_create_leaf (
+ missing, pivot_value_new_var_value (vf->var, &f->values[0]));
+ put_freq_row (table, var_idx, f->count,
+ f->count / ft->total_cases * 100.0, SYSMIS, SYSMIS);
+ }
+
+ int var_idx = pivot_category_create_leaf (
+ variable->root, pivot_value_new_text (N_("Total")));
+ put_freq_row (table, var_idx, cum_freq, cum_percent, SYSMIS, SYSMIS);
+
+ pivot_table_submit (table);
+}
+\f
+/* Statistical display. */
+
+static double
+calc_percentile (double p, double valid_cases, double x1, double x2)
+{
+ double s, dummy;
+
+ s = (settings_get_algorithm () != COMPATIBLE
+ ? modf ((valid_cases - 1) * p, &dummy)
+ : modf ((valid_cases + 1) * p - 1, &dummy));
+
+ return x1 + (x2 - x1) * s;
+}
+
+/* Calculates all of the percentiles for VF within FRQ. */
+static void
+calc_percentiles (const struct frq_proc *frq, struct var_freqs *vf)
+{
+ if (!frq->n_percentiles)
+ return;
+
+ if (!vf->percentiles)
+ vf->percentiles = xnmalloc (frq->n_percentiles, sizeof *vf->percentiles);
+
+ const struct freq_tab *ft = &vf->tab;
+ const double W = ft->valid_cases;
+ size_t idx = 0;
+
+ double rank = 0;
+ for (const struct freq *f = ft->valid; f < ft->missing; f++)
+ {
+ rank += f->count;
+ for (; idx < frq->n_percentiles; idx++)
+ {
+ struct percentile *pc = &frq->percentiles[idx];
+ double tp;
+
+ tp = (settings_get_algorithm () == ENHANCED
+ ? (W - 1) * pc->p
+ : (W + 1) * pc->p - 1);
+
+ if (rank <= tp)
+ break;
+
+ if (tp + 1 < rank || f + 1 >= ft->missing)
+ vf->percentiles[idx] = f->values[0].f;
+ else
+ vf->percentiles[idx] = calc_percentile (pc->p, W, f->values[0].f,
+ f[1].values[0].f);
+ }
+ }
+ for (; idx < frq->n_percentiles; idx++)
+ vf->percentiles[idx] = (ft->n_valid > 0
+ ? ft->valid[ft->n_valid - 1].values[0].f
+ : SYSMIS);
+}
+
+/* Returns true iff the value in struct freq F is non-missing
+ for variable V. */
+static bool
+not_missing (const void *f_, const void *v_)
+{
+ const struct freq *f = f_;
+ const struct variable *v = v_;
+
+ return !var_is_value_missing (v, f->values);
+}
+
+/* Summarizes the frequency table data for variable V. */
+static void
+postprocess_freq_tab (const struct frq_proc *frq, struct var_freqs *vf)
+{
+ struct freq_tab *ft = &vf->tab;
+
+ /* Extract data from hash table. */
+ size_t count = hmap_count (&ft->data);
+ struct freq *freqs = freq_hmap_extract (&ft->data);
+
+ /* Put data into ft. */
+ ft->valid = freqs;
+ ft->n_valid = partition (freqs, count, sizeof *freqs, not_missing, vf->var);
+ ft->missing = freqs + ft->n_valid;
+ ft->n_missing = count - ft->n_valid;
+
+ /* Sort data. */
+ struct freq_compare_aux aux = {
+ .by_freq = frq->sort == FRQ_AFREQ || frq->sort == FRQ_DFREQ,
+ .ascending_freq = frq->sort != FRQ_DFREQ,
+ .width = vf->width,
+ .ascending_value = frq->sort != FRQ_DVALUE,
+ };
+ sort (ft->valid, ft->n_valid, sizeof *ft->valid, compare_freq, &aux);
+ sort (ft->missing, ft->n_missing, sizeof *ft->missing, compare_freq, &aux);
+
+ /* Summary statistics. */
+ ft->valid_cases = 0.0;
+ for (size_t i = 0; i < ft->n_valid; ++i)
+ ft->valid_cases += ft->valid[i].count;
+
+ ft->total_cases = ft->valid_cases;
+ for (size_t i = 0; i < ft->n_missing; ++i)
+ ft->total_cases += ft->missing[i].count;
+}
+
+/* Add data from case C to the frequency table. */
+static void
+calc (struct frq_proc *frq, const struct ccase *c, const struct dataset *ds)
+{
+ double weight = dict_get_case_weight (dataset_dict (ds), c, &frq->warn);
+ for (size_t i = 0; i < frq->n_vars; i++)
+ {
+ struct var_freqs *vf = &frq->vars[i];
+ const union value *value = case_data (c, vf->var);
+ size_t hash = value_hash (value, vf->width, 0);
+ struct freq *f;
+
+ f = freq_hmap_search (&vf->tab.data, value, vf->width, hash);
+ if (f == NULL)
+ f = freq_hmap_insert (&vf->tab.data, value, vf->width, hash);
+
+ f->count += weight;
+ }
+}
+
+static void
+output_splits_once (bool *need_splits, const struct dataset *ds,
+ const struct ccase *c)
+{
+ if (*need_splits)
+ {
+ output_split_file_values (ds, c);
+ *need_splits = false;
+ }
+}
+
+/* Finishes up with the variables after frequencies have been
+ calculated. Displays statistics, percentiles, ... */
+static struct frq_stats_table *
+postcalc (struct frq_proc *frq, const struct dataset *ds,
+ struct ccase *example, struct frq_stats_table *fst)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct variable *wv = dict_get_weight (dict);
+
+ for (size_t i = 0; i < frq->n_vars; i++)
+ {
+ struct var_freqs *vf = &frq->vars[i];
+ postprocess_freq_tab (frq, vf);
+ calc_percentiles (frq, vf);
+ }
+
+ enum split_type st = dict_get_split_type (dict);
+ bool need_splits = true;
+ if (frq->stats)
+ {
+ if (st != SPLIT_LAYERED)
+ output_splits_once (&need_splits, ds, example);
+ fst = frq_stats_table_submit (fst, frq, dict, wv, example);
+ }
+
+ for (size_t i = 0; i < frq->n_vars; i++)
+ {
+ struct var_freqs *vf = &frq->vars[i];
+
+ /* Frequencies tables. */
+ if (vf->tab.n_valid + vf->tab.n_missing <= frq->max_categories)
+ {
+ output_splits_once (&need_splits, ds, example);
+ dump_freq_table (vf, wv);
+ }
+
+ if (frq->hist && var_is_numeric (vf->var) && vf->tab.n_valid > 0)
+ {
+ double d[FRQ_ST_count];
+ struct histogram *histogram;
+
+ calc_stats (frq, vf, d);
+
+ histogram = freq_tab_to_hist (frq, vf);
+
+ if (histogram)
+ {
+ output_splits_once (&need_splits, ds, example);
+ chart_submit (histogram_chart_create (
+ histogram->gsl_hist, var_to_string(vf->var),
+ vf->tab.valid_cases,
+ d[FRQ_ST_MEAN],
+ d[FRQ_ST_STDDEV],
+ frq->hist->draw_normal));
+
+ statistic_destroy (&histogram->parent);
+ }
+ }
+
+ if (frq->pie)
+ {
+ output_splits_once (&need_splits, ds, example);
+ do_piechart(frq->pie, vf->var, &vf->tab);
+ }
+
+ if (frq->bar)
+ {
+ output_splits_once (&need_splits, ds, example);
+ do_barchart(frq->bar, &vf->var, &vf->tab);
+ }
+
+ free (vf->tab.valid);
+ freq_hmap_destroy (&vf->tab.data, vf->width);
+ }
+
+ return fst;
+}
+
+static void
+frq_run (struct frq_proc *frq, struct dataset *ds)
+{
+ struct frq_stats_table *fst = NULL;
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds),
+ dataset_dict (ds));
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ for (size_t i = 0; i < frq->n_vars; i++)
+ hmap_init (&frq->vars[i].tab.data);
+
+ struct ccase *example = casereader_peek (group, 0);
+
+ struct ccase *c;
+ for (; (c = casereader_read (group)) != NULL; case_unref (c))
+ calc (frq, c, ds);
+ fst = postcalc (frq, ds, example, fst);
+ casereader_destroy (group);
+
+ case_unref (example);
+ }
+ frq_stats_table_destroy (fst);
+ casegrouper_destroy (grouper);
+ proc_commit (ds);
+}
+
+static void
+add_percentile (struct frq_proc *frq, double p, bool show,
+ size_t *allocated_percentiles)
+{
+ if (frq->n_percentiles >= *allocated_percentiles)
+ frq->percentiles = x2nrealloc (frq->percentiles, allocated_percentiles,
+ sizeof *frq->percentiles);
+ frq->percentiles[frq->n_percentiles++] = (struct percentile) {
+ .p = p,
+ .show = show,
+ };
+}
+
+int
+cmd_frequencies (struct lexer *lexer, struct dataset *ds)
+{
+ bool ok = false;
+ const struct variable **vars = NULL;
+
+ size_t allocated_percentiles = 0;
+
+ const unsigned long DEFAULT_STATS = (BIT_INDEX (FRQ_ST_MEAN)
+ | BIT_INDEX (FRQ_ST_STDDEV)
+ | BIT_INDEX (FRQ_ST_MINIMUM)
+ | BIT_INDEX (FRQ_ST_MAXIMUM));
+ struct frq_proc frq = {
+ .sort = FRQ_AVALUE,
+ .stats = DEFAULT_STATS,
+ .max_categories = LONG_MAX,
+ .median_idx = SIZE_MAX,
+ .warn = true,
+ };
+
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "VARIABLES") && !lex_force_match (lexer, T_EQUALS))
+ goto done;
+
+ if (!parse_variables_const (lexer, dataset_dict (ds),
+ &vars, &frq.n_vars, PV_NO_DUPLICATE))
+ goto done;
+
+ frq.vars = xcalloc (frq.n_vars, sizeof *frq.vars);
+ for (size_t i = 0; i < frq.n_vars; ++i)
+ {
+ frq.vars[i].var = vars[i];
+ frq.vars[i].width = var_get_width (vars[i]);
+ }
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ frq.stats = 0;
+
+ int ofs = lex_ofs (lexer);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ for (int s = 0; s < FRQ_ST_count; s++)
+ if (lex_match_id (lexer, st_keywords[s]))
+ {
+ frq.stats |= 1 << s;
+ goto next;
+ }
+
+ if (lex_match_id (lexer, "DEFAULT"))
+ frq.stats = DEFAULT_STATS;
+ else if (lex_match (lexer, T_ALL))
+ frq.stats = (1 << FRQ_ST_count) - 1;
+ else if (lex_match_id (lexer, "NONE"))
+ frq.stats = 0;
+ else
+ {
+#define S(ENUM, KEYWORD, NAME) KEYWORD,
+ lex_error_expecting (lexer,
+ STATISTICS
+ "DEFAULT", "ALL", "NONE");
+#undef S
+ goto done;
+ }
+
+ next:;
+ }
+
+ if (lex_ofs (lexer) == ofs)
+ frq.stats = DEFAULT_STATS;
+ }
+ else if (lex_match_id (lexer, "PERCENTILES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (!lex_force_num_range_closed (lexer, "PERCENTILES", 0, 100))
+ goto done;
+ add_percentile (&frq, lex_number (lexer) / 100.0, true,
+ &allocated_percentiles);
+ lex_get (lexer);
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "TABLE"))
+ {
+ }
+ else if (lex_match_id (lexer, "NOTABLE"))
+ frq.max_categories = 0;
+ else if (lex_match_id (lexer, "LIMIT"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "LIMIT", 0, INT_MAX))
+ goto done;
+
+ frq.max_categories = lex_integer (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else if (lex_match_id (lexer, "AVALUE"))
+ frq.sort = FRQ_AVALUE;
+ else if (lex_match_id (lexer, "DVALUE"))
+ frq.sort = FRQ_DVALUE;
+ else if (lex_match_id (lexer, "AFREQ"))
+ frq.sort = FRQ_AFREQ;
+ else if (lex_match_id (lexer, "DFREQ"))
+ frq.sort = FRQ_DFREQ;
+ else
+ {
+ lex_error_expecting (lexer, "TABLE", "NOTABLE",
+ "LIMIT", "AVALUE", "DVALUE",
+ "AFREQ", "DFREQ");
+ goto done;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "NTILES"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_int_range (lexer, "NTILES", 0, INT_MAX))
+ goto done;
+
+ int n = lex_integer (lexer);
+ lex_get (lexer);
+ for (int i = 0; i < n + 1; ++i)
+ add_percentile (&frq, i / (double) n, true, &allocated_percentiles);
+ }
+ else if (lex_match_id (lexer, "ALGORITHM"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "COMPATIBLE"))
+ settings_set_cmd_algorithm (COMPATIBLE);
+ else if (lex_match_id (lexer, "ENHANCED"))
+ settings_set_cmd_algorithm (ENHANCED);
+ else
+ {
+ lex_error_expecting (lexer, "COMPATIBLE", "ENHANCED");
+ goto done;
+ }
+ }
+ else if (lex_match_id (lexer, "HISTOGRAM"))
+ {
+ double hi_min = -DBL_MAX;
+ double hi_max = DBL_MAX;
+ int hi_scale = FRQ_FREQ;
+ int hi_freq = INT_MIN;
+ int hi_pcnt = INT_MIN;
+ bool hi_draw_normal = false;
+
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "NORMAL"))
+ hi_draw_normal = true;
+ else if (lex_match_id (lexer, "NONORMAL"))
+ hi_draw_normal = false;
+ else if (lex_match_id (lexer, "FREQ"))
+ {
+ hi_scale = FRQ_FREQ;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_int_range (lexer, "FREQ", 1, INT_MAX))
+ goto done;
+ hi_freq = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ }
+ else if (lex_match_id (lexer, "PERCENT"))
+ {
+ hi_scale = FRQ_PERCENT;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_int_range (lexer, "PERCENT", 1, INT_MAX))
+ goto done;
+ hi_pcnt = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ }
+ else if (lex_match_id (lexer, "MINIMUM"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "MINIMUM",
+ -DBL_MAX, hi_max))
+ goto done;
+ hi_min = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else if (lex_match_id (lexer, "MAXIMUM"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "MAXIMUM",
+ hi_min, DBL_MAX))
+ goto done;
+ hi_max = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "NORMAL", "NONORMAL",
+ "FREQ", "PERCENT", "MINIMUM", "MAXIMUM");
+ goto done;
+ }
+ }
+
+ free (frq.hist);
+ frq.hist = xmalloc (sizeof *frq.hist);
+ *frq.hist = (struct frq_chart) {
+ .x_min = hi_min,
+ .x_max = hi_max,
+ .y_scale = hi_scale,
+ .y_max = hi_scale == FRQ_FREQ ? hi_freq : hi_pcnt,
+ .draw_normal = hi_draw_normal,
+ .include_missing = false,
+ };
+
+ add_percentile (&frq, .25, false, &allocated_percentiles);
+ add_percentile (&frq, .75, false, &allocated_percentiles);
+ }
+ else if (lex_match_id (lexer, "PIECHART"))
+ {
+ double pie_min = -DBL_MAX;
+ double pie_max = DBL_MAX;
+ bool pie_missing = true;
+
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "MINIMUM"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "MINIMUM",
+ -DBL_MAX, pie_max))
+ goto done;
+ pie_min = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else if (lex_match_id (lexer, "MAXIMUM"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "MAXIMUM",
+ pie_min, DBL_MAX))
+ goto done;
+ pie_max = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ pie_missing = true;
+ else if (lex_match_id (lexer, "NOMISSING"))
+ pie_missing = false;
+ else
+ {
+ lex_error_expecting (lexer, "MINIMUM", "MAXIMUM",
+ "MISSING", "NOMISSING");
+ goto done;
+ }
+ }
+
+ free (frq.pie);
+ frq.pie = xmalloc (sizeof *frq.pie);
+ *frq.pie = (struct frq_chart) {
+ .x_min = pie_min,
+ .x_max = pie_max,
+ .include_missing = pie_missing,
+ };
+ }
+ else if (lex_match_id (lexer, "BARCHART"))
+ {
+ double bar_min = -DBL_MAX;
+ double bar_max = DBL_MAX;
+ bool bar_freq = true;
+
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "MINIMUM"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "MINIMUM",
+ -DBL_MAX, bar_max))
+ goto done;
+ bar_min = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else if (lex_match_id (lexer, "MAXIMUM"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "MAXIMUM",
+ bar_min, DBL_MAX))
+ goto done;
+ bar_max = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ else if (lex_match_id (lexer, "FREQ"))
+ {
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num_range_open (lexer, "FREQ", 0, DBL_MAX))
+ goto done;
+ /* XXX TODO */
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ bar_freq = true;
+ }
+ else if (lex_match_id (lexer, "PERCENT"))
+ {
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num_range_open (lexer, "PERCENT",
+ 0, DBL_MAX))
+ goto done;
+ /* XXX TODO */
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+ }
+ bar_freq = false;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MINIMUM", "MAXIMUM",
+ "FREQ", "PERCENT");
+ goto done;
+ }
+ }
+
+ free (frq.bar);
+ frq.bar = xmalloc (sizeof *frq.bar);
+ *frq.bar = (struct frq_chart) {
+ .x_min = bar_min,
+ .x_max = bar_max,
+ .include_missing = false,
+ .y_scale = bar_freq ? FRQ_FREQ : FRQ_PERCENT,
+ };
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "EXCLUDE"))
+ {
+ /* XXX TODO */
+ }
+ else if (lex_match_id (lexer, "INCLUDE"))
+ {
+ /* XXX TODO */
+ }
+ else
+ {
+ lex_error_expecting (lexer, "EXCLUDE", "INCLUDE");
+ goto done;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "ORDER"))
+ {
+ lex_match (lexer, T_EQUALS);
+ /* XXX TODO */
+ if (!lex_match_id (lexer, "ANALYSIS")
+ && !lex_match_id (lexer, "VARIABLE"))
+ {
+ lex_error_expecting (lexer, "ANALYSIS", "VARIABLE");
+ goto done;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "STATISTICS", "PERCENTILES", "FORMAT",
+ "NTILES", "ALGORITHM", "HISTOGRAM", "PIECHART",
+ "BARCHART", "MISSING", "ORDER");
+ goto done;
+ }
+ }
+
+ if (frq.stats & BIT_INDEX (FRQ_ST_MEDIAN))
+ add_percentile (&frq, .5, false, &allocated_percentiles);
+
+ if (frq.n_percentiles > 0)
+ {
+ qsort (frq.percentiles, frq.n_percentiles, sizeof *frq.percentiles,
+ percentile_compare_3way);
+
+ /* Combine equal percentiles. */
+ size_t o = 1;
+ for (int i = 1; i < frq.n_percentiles; ++i)
+ {
+ struct percentile *prev = &frq.percentiles[o - 1];
+ struct percentile *this = &frq.percentiles[i];
+ if (this->p != prev->p)
+ frq.percentiles[o++] = *this;
+ else if (this->show)
+ prev->show = true;
+ }
+ frq.n_percentiles = o;
+
+ for (size_t i = 0; i < frq.n_percentiles; i++)
+ if (frq.percentiles[i].p == 0.5)
+ {
+ frq.median_idx = i;
+ break;
+ }
+ }
+
+ frq_run (&frq, ds);
+ ok = true;
+
+done:
+ free (vars);
+ for (size_t i = 0; i < frq.n_vars; i++)
+ free (frq.vars[i].percentiles);
+ free (frq.vars);
+ free (frq.bar);
+ free (frq.pie);
+ free (frq.hist);
+ free (frq.percentiles);
+
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+}
+
+static double
+calculate_iqr (const struct frq_proc *frq, const struct var_freqs *vf)
+{
+ double q1 = SYSMIS;
+ double q3 = SYSMIS;
+
+ /* This cannot work unless the 25th and 75th percentile are calculated */
+ assert (frq->n_percentiles >= 2);
+ for (int i = 0; i < frq->n_percentiles; i++)
+ {
+ struct percentile *pc = &frq->percentiles[i];
+
+ if (fabs (0.25 - pc->p) < DBL_EPSILON)
+ q1 = vf->percentiles[i];
+ else if (fabs (0.75 - pc->p) < DBL_EPSILON)
+ q3 = vf->percentiles[i];
+ }
+
+ return q1 == SYSMIS || q3 == SYSMIS ? SYSMIS : q3 - q1;
+}
+
+static bool
+chart_includes_value (const struct frq_chart *chart,
+ const struct variable *var,
+ const union value *value)
+{
+ if (!chart->include_missing && var_is_value_missing (var, value))
+ return false;
+
+ if (var_is_numeric (var)
+ && ((chart->x_min != SYSMIS && value->f < chart->x_min)
+ || (chart->x_max != SYSMIS && value->f > chart->x_max)))
+ return false;
+
+ return true;
+}
+
+/* Create a gsl_histogram from a freq_tab */
+static struct histogram *
+freq_tab_to_hist (const struct frq_proc *frq, const struct var_freqs *vf)
+{
+ /* Find out the extremes of the x value, within the range to be included in
+ the histogram, and sum the total frequency of those values. */
+ double x_min = DBL_MAX;
+ double x_max = -DBL_MAX;
+ double valid_freq = 0;
+ for (size_t i = 0; i < vf->tab.n_valid; i++)
+ {
+ const struct freq *f = &vf->tab.valid[i];
+ if (chart_includes_value (frq->hist, vf->var, f->values))
+ {
+ x_min = MIN (x_min, f->values[0].f);
+ x_max = MAX (x_max, f->values[0].f);
+ valid_freq += f->count;
+ }
+ }
+
+ if (valid_freq <= 0)
+ return NULL;
+
+ double iqr = calculate_iqr (frq, vf);
+
+ double bin_width =
+ (iqr > 0
+ ? 2 * iqr / pow (valid_freq, 1.0 / 3.0) /* Freedman-Diaconis. */
+ : (x_max - x_min) / (1 + log2 (valid_freq))); /* Sturges */
+
+ struct histogram *histogram = histogram_create (bin_width, x_min, x_max);
+ if (histogram == NULL)
+ return NULL;
+
+ for (size_t i = 0; i < vf->tab.n_valid; i++)
+ {
+ const struct freq *f = &vf->tab.valid[i];
+ if (chart_includes_value (frq->hist, vf->var, f->values))
+ histogram_add (histogram, f->values[0].f, f->count);
+ }
+
+ return histogram;
+}
+
+
+/* Allocate an array of struct freqs and fill them from the data in FRQ_TAB,
+ according to the parameters of CATCHART
+ N_SLICES will contain the number of slices allocated.
+ The caller is responsible for freeing slices
+*/
+static struct freq *
+pick_cat_counts (const struct frq_chart *catchart,
+ const struct freq_tab *frq_tab,
+ int *n_slicesp)
+{
+ int n_slices = 0;
+ struct freq *slices = xnmalloc (frq_tab->n_valid + frq_tab->n_missing, sizeof *slices);
+
+ for (size_t i = 0; i < frq_tab->n_valid; i++)
+ {
+ struct freq *f = &frq_tab->valid[i];
+ if (f->count >= catchart->x_min && f->count <= catchart->x_max)
+ slices[n_slices++] = *f;
+ }
+
+
+ if (catchart->include_missing)
+ {
+ for (size_t i = 0; i < frq_tab->n_missing; i++)
+ {
+ const struct freq *f = &frq_tab->missing[i];
+ slices[n_slices].count += f->count;
+
+ if (i == 0)
+ slices[n_slices].values[0] = f->values[0];
+ }
+
+ if (frq_tab->n_missing > 0)
+ n_slices++;
+ }
+
+ *n_slicesp = n_slices;
+ return slices;
+}
+
+
+/* Allocate an array of struct freqs and fill them from the data in FRQ_TAB,
+ according to the parameters of CATCHART
+ N_SLICES will contain the number of slices allocated.
+ The caller is responsible for freeing slices
+*/
+static struct freq **
+pick_cat_counts_ptr (const struct frq_chart *catchart,
+ const struct freq_tab *frq_tab,
+ int *n_slicesp)
+{
+ int n_slices = 0;
+ struct freq **slices = xnmalloc (frq_tab->n_valid + frq_tab->n_missing, sizeof *slices);
+
+ for (size_t i = 0; i < frq_tab->n_valid; i++)
+ {
+ struct freq *f = &frq_tab->valid[i];
+ if (f->count >= catchart->x_min && f->count <= catchart->x_max)
+ slices[n_slices++] = f;
+ }
+
+ if (catchart->include_missing)
+ for (size_t i = 0; i < frq_tab->n_missing; i++)
+ {
+ const struct freq *f = &frq_tab->missing[i];
+ if (i == 0)
+ {
+ slices[n_slices] = xmalloc (sizeof *slices[n_slices]);
+ slices[n_slices]->values[0] = f->values[0];
+ }
+
+ slices[n_slices]->count += f->count;
+ }
+
+ *n_slicesp = n_slices;
+ return slices;
+}
+
+static void
+do_piechart (const struct frq_chart *pie, const struct variable *var,
+ const struct freq_tab *frq_tab)
+{
+ int n_slices;
+ struct freq *slices = pick_cat_counts (pie, frq_tab, &n_slices);
+
+ if (n_slices < 2)
+ msg (SW, _("Omitting pie chart for %s, which has only %d unique values."),
+ var_get_name (var), n_slices);
+ else if (n_slices > 50)
+ msg (SW, _("Omitting pie chart for %s, which has over 50 unique values."),
+ var_get_name (var));
+ else
+ chart_submit (piechart_create (var, slices, n_slices));
+
+ free (slices);
+}
+
+static void
+do_barchart (const struct frq_chart *bar, const struct variable **var,
+ const struct freq_tab *frq_tab)
+{
+ int n_slices;
+ struct freq **slices = pick_cat_counts_ptr (bar, frq_tab, &n_slices);
+
+ if (n_slices < 1)
+ msg (SW, _("Omitting bar chart, which has no values."));
+ else
+ chart_submit (barchart_create (
+ var, 1,
+ bar->y_scale == FRQ_FREQ ? _("Count") : _("Percent"),
+ bar->y_scale == FRQ_PERCENT,
+ slices, n_slices));
+ free (slices);
+}
+
+/* Calculates all the pertinent statistics for VF, putting them in array
+ D[]. */
+static void
+calc_stats (const struct frq_proc *frq, const struct var_freqs *vf,
+ double d[FRQ_ST_count])
+{
+ const struct freq_tab *ft = &vf->tab;
+
+ /* Calculate the mode. If there is more than one mode, we take the
+ smallest. */
+ int most_often = -1;
+ double X_mode = SYSMIS;
+ for (const struct freq *f = ft->valid; f < ft->missing; f++)
+ if (most_often < f->count)
+ {
+ most_often = f->count;
+ X_mode = f->values[0].f;
+ }
+
+ /* Calculate moments. */
+ struct moments *m = moments_create (MOMENT_KURTOSIS);
+ for (const struct freq *f = ft->valid; f < ft->missing; f++)
+ moments_pass_one (m, f->values[0].f, f->count);
+ for (const struct freq *f = ft->valid; f < ft->missing; f++)
+ moments_pass_two (m, f->values[0].f, f->count);
+ moments_calculate (m, NULL, &d[FRQ_ST_MEAN], &d[FRQ_ST_VARIANCE],
+ &d[FRQ_ST_SKEWNESS], &d[FRQ_ST_KURTOSIS]);
+ moments_destroy (m);
+
+ /* Formulae below are taken from _SPSS Statistical Algorithms_. */
+ double W = ft->valid_cases;
+ if (ft->n_valid > 0)
+ {
+ d[FRQ_ST_MINIMUM] = ft->valid[0].values[0].f;
+ d[FRQ_ST_MAXIMUM] = ft->valid[ft->n_valid - 1].values[0].f;
+ d[FRQ_ST_RANGE] = d[FRQ_ST_MAXIMUM] - d[FRQ_ST_MINIMUM];
+ }
+ else
+ {
+ d[FRQ_ST_MINIMUM] = SYSMIS;
+ d[FRQ_ST_MAXIMUM] = SYSMIS;
+ d[FRQ_ST_RANGE] = SYSMIS;
+ }
+ d[FRQ_ST_MODE] = X_mode;
+ d[FRQ_ST_SUM] = d[FRQ_ST_MEAN] * W;
+ d[FRQ_ST_STDDEV] = sqrt (d[FRQ_ST_VARIANCE]);
+ d[FRQ_ST_SEMEAN] = d[FRQ_ST_STDDEV] / sqrt (W);
+ d[FRQ_ST_SESKEWNESS] = calc_seskew (W);
+ d[FRQ_ST_SEKURTOSIS] = calc_sekurt (W);
+ d[FRQ_ST_MEDIAN] = (frq->median_idx != SIZE_MAX
+ ? vf->percentiles[frq->median_idx]
+ : SYSMIS);
+}
+
+static bool
+all_string_variables (const struct frq_proc *frq)
+{
+ for (size_t i = 0; i < frq->n_vars; i++)
+ if (var_is_numeric (frq->vars[i].var))
+ return false;
+
+ return true;
+}
+\f
+struct frq_stats_table
+ {
+ struct pivot_table *table;
+ struct pivot_splits *splits;
+ };
+
+/* Displays a table of all the statistics requested. */
+static struct frq_stats_table *
+frq_stats_table_create (const struct frq_proc *frq,
+ const struct dictionary *dict,
+ const struct variable *wv)
+{
+ if (all_string_variables (frq))
+ return NULL;
+
+ struct pivot_table *table = pivot_table_create (N_("Statistics"));
+ pivot_table_set_weight_var (table, wv);
+
+ struct pivot_dimension *variables
+ = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Variables"));
+ for (size_t i = 0; i < frq->n_vars; i++)
+ if (!var_is_alpha (frq->vars[i].var))
+ pivot_category_create_leaf (variables->root,
+ pivot_value_new_variable (frq->vars[i].var));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"));
+ struct pivot_category *n = pivot_category_create_group (
+ statistics->root, N_("N"));
+ pivot_category_create_leaves (n,
+ N_("Valid"), PIVOT_RC_COUNT,
+ N_("Missing"), PIVOT_RC_COUNT);
+ for (int i = 0; i < FRQ_ST_count; i++)
+ if (frq->stats & BIT_INDEX (i))
+ pivot_category_create_leaf (statistics->root,
+ pivot_value_new_text (st_names[i]));
+ struct pivot_category *percentiles = NULL;
+ for (size_t i = 0; i < frq->n_percentiles; i++)
+ {
+ const struct percentile *pc = &frq->percentiles[i];
+
+ if (!pc->show)
+ continue;
+
+ if (!percentiles)
+ percentiles = pivot_category_create_group (
+ statistics->root, N_("Percentiles"));
+ pivot_category_create_leaf (percentiles, pivot_value_new_integer (
+ pc->p * 100.0));
+ }
+
+ struct pivot_splits *splits = pivot_splits_create (table, PIVOT_AXIS_COLUMN,
+ dict);
+
+ struct frq_stats_table *fst = xmalloc (sizeof *fst);
+ *fst = (struct frq_stats_table) { .table = table, .splits = splits };
+ return fst;
+}
+
+static struct frq_stats_table *
+frq_stats_table_submit (struct frq_stats_table *fst,
+ const struct frq_proc *frq,
+ const struct dictionary *dict,
+ const struct variable *wv,
+ const struct ccase *example)
+{
+ if (!fst)
+ {
+ fst = frq_stats_table_create (frq, dict, wv);
+ if (!fst)
+ return NULL;
+ }
+ pivot_splits_new_split (fst->splits, example);
+
+ int var_idx = 0;
+ for (size_t i = 0; i < frq->n_vars; i++)
+ {
+ struct var_freqs *vf = &frq->vars[i];
+ if (var_is_alpha (vf->var))
+ continue;
+
+ const struct freq_tab *ft = &vf->tab;
+
+ int row = 0;
+ pivot_splits_put2 (fst->splits, fst->table, var_idx, row++,
+ pivot_value_new_number (ft->valid_cases));
+ pivot_splits_put2 (fst->splits, fst->table, var_idx, row++,
+ pivot_value_new_number (
+ ft->total_cases - ft->valid_cases));
+
+ double stat_values[FRQ_ST_count];
+ calc_stats (frq, vf, stat_values);
+ for (int j = 0; j < FRQ_ST_count; j++)
+ {
+ if (!(frq->stats & BIT_INDEX (j)))
+ continue;
+
+ union value v = { .f = vf->tab.n_valid ? stat_values[j] : SYSMIS };
+ struct pivot_value *pv
+ = (j == FRQ_ST_MODE || j == FRQ_ST_MINIMUM || j == FRQ_ST_MAXIMUM
+ ? pivot_value_new_var_value (vf->var, &v)
+ : pivot_value_new_number (v.f));
+ pivot_splits_put2 (fst->splits, fst->table, var_idx, row++, pv);
+ }
+
+ for (size_t j = 0; j < frq->n_percentiles; j++)
+ {
+ const struct percentile *pc = &frq->percentiles[j];
+ if (!pc->show)
+ continue;
+
+ union value v = {
+ .f = vf->tab.n_valid ? vf->percentiles[j] : SYSMIS
+ };
+ pivot_splits_put2 (fst->splits, fst->table, var_idx, row++,
+ pivot_value_new_var_value (vf->var, &v));
+ }
+
+ var_idx++;
+ }
+
+ if (!fst->splits)
+ {
+ frq_stats_table_destroy (fst);
+ return NULL;
+ }
+ return fst;
+}
+
+static void
+frq_stats_table_destroy (struct frq_stats_table *fst)
+{
+ if (!fst)
+ return;
+
+ pivot_table_submit (fst->table);
+ pivot_splits_destroy (fst->splits);
+ free (fst);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis. -*-c-*-
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <config.h>
+
+#include "language/commands/friedman.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct friedman
+{
+ double *rank_sum;
+ double cc;
+ double chi_sq;
+ double w;
+ const struct dictionary *dict;
+};
+
+static void show_ranks_box (const struct one_sample_test *ost,
+ const struct friedman *fr);
+
+static void show_sig_box (const struct one_sample_test *ost,
+ const struct friedman *fr);
+
+struct datum
+{
+ long posn;
+ double x;
+};
+
+static int
+cmp_x (const void *a_, const void *b_)
+{
+ const struct datum *a = a_;
+ const struct datum *b = b_;
+
+ if (a->x < b->x)
+ return -1;
+
+ return (a->x > b->x);
+}
+
+static int
+cmp_posn (const void *a_, const void *b_)
+{
+ const struct datum *a = a_;
+ const struct datum *b = b_;
+
+ if (a->posn < b->posn)
+ return -1;
+
+ return (a->posn > b->posn);
+}
+
+void
+friedman_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ double numerator = 0.0;
+ double denominator = 0.0;
+ int v;
+ struct ccase *c;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct variable *weight = dict_get_weight (dict);
+
+ struct one_sample_test *ost = UP_CAST (test, struct one_sample_test, parent);
+ struct friedman_test *ft = UP_CAST (ost, struct friedman_test, parent);
+ bool warn = true;
+
+ double sigma_t = 0.0;
+ struct datum *row = XCALLOC (ost->n_vars, struct datum);
+ double rsq;
+ struct friedman fr;
+ fr.rank_sum = xcalloc (ost->n_vars, sizeof *fr.rank_sum);
+ fr.cc = 0.0;
+ fr.dict = dict;
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ row[v].posn = v;
+ fr.rank_sum[v] = 0.0;
+ }
+
+ input = casereader_create_filter_weight (input, dict, &warn, NULL);
+ input = casereader_create_filter_missing (input,
+ ost->vars, ost->n_vars,
+ exclude, 0, 0);
+
+ for (; (c = casereader_read (input)); case_unref (c))
+ {
+ double prev_x = SYSMIS;
+ int run_length = 0;
+
+ const double w = weight ? case_num (c, weight) : 1.0;
+
+ fr.cc += w;
+
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+ const union value *val = case_data (c, var);
+ row[v].x = val->f;
+ }
+
+ qsort (row, ost->n_vars, sizeof *row, cmp_x);
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ double x = row[v].x;
+ /* Replace value by the Rank */
+ if (prev_x == x)
+ {
+ /* Deal with ties */
+ int i;
+ run_length++;
+ for (i = v - run_length; i < v; ++i)
+ {
+ row[i].x *= run_length ;
+ row[i].x += v + 1;
+ row[i].x /= run_length + 1;
+ }
+ row[v].x = row[v-1].x;
+ }
+ else
+ {
+ row[v].x = v + 1;
+ if (run_length > 0)
+ {
+ double t = run_length + 1;
+ sigma_t += w * (pow3 (t) - t);
+ }
+ run_length = 0;
+ }
+ prev_x = x;
+ }
+ if (run_length > 0)
+ {
+ double t = run_length + 1;
+ sigma_t += w * (pow3 (t) - t);
+ }
+
+ qsort (row, ost->n_vars, sizeof *row, cmp_posn);
+
+ for (v = 0; v < ost->n_vars; ++v)
+ fr.rank_sum[v] += row[v].x * w;
+ }
+ casereader_destroy (input);
+ free (row);
+
+
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ numerator += pow2 (fr.rank_sum[v]);
+ }
+
+ rsq = numerator;
+
+ numerator *= 12.0 / (fr.cc * ost->n_vars * (ost->n_vars + 1));
+ numerator -= 3 * fr.cc * (ost->n_vars + 1);
+
+ denominator = 1 - sigma_t / (fr.cc * ost->n_vars * (pow2 (ost->n_vars) - 1));
+
+ fr.chi_sq = numerator / denominator;
+
+ if (ft->kendalls_w)
+ {
+ fr.w = 12 * rsq ;
+ fr.w -= 3 * pow2 (fr.cc) *
+ ost->n_vars * pow2 (ost->n_vars + 1);
+
+ fr.w /= pow2 (fr.cc) * (pow3 (ost->n_vars) - ost->n_vars)
+ - fr.cc * sigma_t;
+ }
+ else
+ fr.w = SYSMIS;
+
+ show_ranks_box (ost, &fr);
+ show_sig_box (ost, &fr);
+
+ free (fr.rank_sum);
+}
+
+\f
+
+
+static void
+show_ranks_box (const struct one_sample_test *ost, const struct friedman *fr)
+{
+ struct pivot_table *table = pivot_table_create (N_("Ranks"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Mean Rank"),
+ N_("Mean Rank"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t i = 0 ; i < ost->n_vars ; ++i)
+ {
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (ost->vars[i]));
+
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_number (fr->rank_sum[i] / fr->cc));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+show_sig_box (const struct one_sample_test *ost, const struct friedman *fr)
+{
+ const struct friedman_test *ft = UP_CAST (ost, const struct friedman_test, parent);
+
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+ pivot_table_set_weight_var (table, dict_get_weight (fr->dict));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT);
+ if (ft->kendalls_w)
+ pivot_category_create_leaves (statistics->root, N_("Kendall's W"),
+ PIVOT_RC_OTHER);
+ pivot_category_create_leaves (statistics->root,
+ N_("Chi-Square"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ double entries[5];
+ int n = 0;
+
+ entries[n++] = fr->cc;
+ if (ft->kendalls_w)
+ entries[n++] = fr->w;
+ entries[n++] = fr->chi_sq;
+ entries[n++] = ost->n_vars - 1;
+ entries[n++] = gsl_cdf_chisq_Q (fr->chi_sq, ost->n_vars - 1);
+ assert (n <= sizeof entries / sizeof *entries);
+
+ for (size_t i = 0; i < n; i++)
+ pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !friedman_h
+#define friedman_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+struct friedman_test
+{
+ struct one_sample_test parent;
+
+ /* Calculate and display the Kendall W statistic */
+ bool kendalls_w;
+};
+
+
+void friedman_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool,
+ double);
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012,
+ 2013, 2015, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include <string.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/gnumeric-reader.h"
+#include "data/ods-reader.h"
+#include "data/spreadsheet-reader.h"
+#include "data/psql-reader.h"
+#include "data/settings.h"
+#include "language/command.h"
+#include "language/commands/data-parser.h"
+#include "language/commands/data-reader.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/placement-parser.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/cast.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+static bool parse_spreadsheet (struct lexer *lexer, char **filename,
+ struct spreadsheet_read_options *opts);
+
+static void destroy_spreadsheet_read_info (struct spreadsheet_read_options *);
+
+static int parse_get_txt (struct lexer *, struct dataset *);
+static int parse_get_psql (struct lexer *, struct dataset *);
+static int parse_get_spreadsheet (struct lexer *, struct dataset *,
+ struct spreadsheet *(*probe)(
+ const char *filename, bool report_errors));
+
+int
+cmd_get_data (struct lexer *lexer, struct dataset *ds)
+{
+ if (!lex_force_match_phrase (lexer, "/TYPE="))
+ return CMD_FAILURE;
+
+ if (lex_match_id (lexer, "TXT"))
+ return parse_get_txt (lexer, ds);
+ else if (lex_match_id (lexer, "PSQL"))
+ return parse_get_psql (lexer, ds);
+ else if (lex_match_id (lexer, "GNM"))
+ return parse_get_spreadsheet (lexer, ds, gnumeric_probe);
+ else if (lex_match_id (lexer, "ODS"))
+ return parse_get_spreadsheet (lexer, ds, ods_probe);
+ else
+ {
+ lex_error_expecting (lexer, "TXT", "PSQL", "GNM", "ODS");
+ return CMD_FAILURE;
+ }
+}
+
+static int
+parse_get_spreadsheet (struct lexer *lexer, struct dataset *ds,
+ struct spreadsheet *(*probe)(
+ const char *filename, bool report_errors))
+{
+ struct spreadsheet_read_options opts;
+ char *filename;
+ if (!parse_spreadsheet (lexer, &filename, &opts))
+ return CMD_FAILURE;
+
+ bool ok = false;
+ struct spreadsheet *spreadsheet = probe (filename, true);
+ if (!spreadsheet)
+ {
+ msg (SE, _("error reading file `%s'"), filename);
+ goto done;
+ }
+
+ struct casereader *reader = spreadsheet_make_reader (spreadsheet, &opts);
+ if (reader)
+ {
+ dataset_set_dict (ds, dict_clone (spreadsheet->dict));
+ dataset_set_source (ds, reader);
+ ok = true;
+ }
+ spreadsheet_unref (spreadsheet);
+
+done:
+ free (filename);
+ destroy_spreadsheet_read_info (&opts);
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+}
+
+static int
+parse_get_psql (struct lexer *lexer, struct dataset *ds)
+{
+ if (!lex_force_match_phrase (lexer, "/CONNECT=") || !lex_force_string (lexer))
+ return CMD_FAILURE;
+
+ struct psql_read_info psql = {
+ .str_width = -1,
+ .bsize = -1,
+ .conninfo = ss_xstrdup (lex_tokss (lexer)),
+ };
+ bool ok = false;
+
+ lex_get (lexer);
+
+ while (lex_match (lexer, T_SLASH))
+ {
+ if (lex_match_id (lexer, "ASSUMEDSTRWIDTH"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "ASSUMEDSTRWIDTH", 1, 32767))
+ goto done;
+ psql.str_width = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "BSIZE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "BSIZE", 1, INT_MAX))
+ goto done;
+ psql.bsize = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "UNENCRYPTED"))
+ psql.allow_clear = true;
+ else if (lex_match_id (lexer, "SQL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto done;
+
+ free (psql.sql);
+ psql.sql = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ }
+
+ struct dictionary *dict = NULL;
+ struct casereader *reader = psql_open_reader (&psql, &dict);
+ if (reader)
+ {
+ dataset_set_dict (ds, dict);
+ dataset_set_source (ds, reader);
+ }
+
+ done:
+ free (psql.conninfo);
+ free (psql.sql);
+
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+}
+
+static bool
+parse_spreadsheet (struct lexer *lexer, char **filename,
+ struct spreadsheet_read_options *opts)
+{
+ *opts = (struct spreadsheet_read_options) {
+ .sheet_index = 1,
+ .read_names = true,
+ .asw = -1,
+ };
+ *filename = NULL;
+
+ if (!lex_force_match_phrase (lexer, "/FILE=") || !lex_force_string (lexer))
+ goto error;
+
+ *filename = utf8_to_filename (lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ while (lex_match (lexer, T_SLASH))
+ {
+ if (lex_match_id (lexer, "ASSUMEDSTRWIDTH"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "ASSUMEDSTRWIDTH", 1, 32767))
+ goto error;
+ opts->asw = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "SHEET"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "NAME"))
+ {
+ if (!lex_force_string (lexer))
+ goto error;
+
+ opts->sheet_name = ss_xstrdup (lex_tokss (lexer));
+ opts->sheet_index = -1;
+
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "INDEX"))
+ {
+ if (!lex_force_int_range (lexer, "INDEX", 1, INT_MAX))
+ goto error;
+ opts->sheet_index = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "NAME", "INDEX");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "CELLRANGE"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "FULL"))
+ opts->cell_range = NULL;
+ else if (lex_match_id (lexer, "RANGE"))
+ {
+ if (!lex_force_string (lexer))
+ goto error;
+
+ opts->cell_range = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "FULL", "RANGE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "READNAMES"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "ON"))
+ opts->read_names = true;
+ else if (lex_match_id (lexer, "OFF"))
+ opts->read_names = false;
+ else
+ {
+ lex_error_expecting (lexer, "ON", "OFF");
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "ASSUMEDSTRWIDTH", "SHEET", "CELLRANGE",
+ "READNAMES");
+ goto error;
+ }
+ }
+
+ return true;
+
+ error:
+ destroy_spreadsheet_read_info (opts);
+ free (*filename);
+ return false;
+}
+
+
+static bool
+set_type (struct lexer *lexer, struct data_parser *parser,
+ enum data_parser_type type,
+ int type_start, int type_end, int *type_startp, int *type_endp)
+{
+ if (!*type_startp)
+ {
+ data_parser_set_type (parser, type);
+ *type_startp = type_start;
+ *type_endp = type_end;
+ }
+ else if (type != data_parser_get_type (parser))
+ {
+ msg (SE, _("FIXED and DELIMITED arrangements are mutually exclusive."));
+ lex_ofs_msg (lexer, SN, type_start, type_end,
+ _("This syntax requires %s arrangement."),
+ type == DP_FIXED ? "FIXED" : "DELIMITED");
+ lex_ofs_msg (lexer, SN, *type_startp, *type_endp,
+ _("This syntax requires %s arrangement."),
+ type == DP_FIXED ? "DELIMITED" : "FIXED");
+ return false;
+ }
+ return true;
+}
+
+static int
+parse_get_txt (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dict_create (get_default_encoding ());
+ struct data_parser *parser = data_parser_create ();
+ struct file_handle *fh = NULL;
+ char *encoding = NULL;
+ char *name = NULL;
+
+ if (!lex_force_match_phrase (lexer, "/FILE="))
+ goto error;
+ fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
+ if (fh == NULL)
+ goto error;
+
+ int type_start = 0, type_end = 0;
+ data_parser_set_type (parser, DP_DELIMITED);
+ data_parser_set_span (parser, false);
+ data_parser_set_quotes (parser, ss_empty ());
+ data_parser_set_quote_escape (parser, true);
+ data_parser_set_empty_line_has_field (parser, true);
+
+ for (;;)
+ {
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+
+ if (lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "ARRANGEMENT"))
+ {
+ bool ok;
+
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "FIXED"))
+ ok = set_type (lexer, parser, DP_FIXED,
+ lex_ofs (lexer) - 3, lex_ofs (lexer) - 1,
+ &type_start, &type_end);
+ else if (lex_match_id (lexer, "DELIMITED"))
+ ok = set_type (lexer, parser, DP_DELIMITED,
+ lex_ofs (lexer) - 3, lex_ofs (lexer) - 1,
+ &type_start, &type_end);
+ else
+ {
+ lex_error_expecting (lexer, "FIXED", "DELIMITED");
+ goto error;
+ }
+ if (!ok)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "FIRSTCASE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "FIRSTCASE", 1, INT_MAX))
+ goto error;
+ data_parser_set_skip (parser, lex_integer (lexer) - 1);
+ lex_get (lexer);
+ }
+ else if (lex_match_id_n (lexer, "DELCASE", 4))
+ {
+ if (!set_type (lexer, parser, DP_DELIMITED,
+ lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
+ &type_start, &type_end))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "LINE"))
+ data_parser_set_span (parser, false);
+ else if (lex_match_id (lexer, "VARIABLES"))
+ {
+ data_parser_set_span (parser, true);
+
+ /* VARIABLES takes an integer argument, but for no
+ good reason. We just ignore it. */
+ if (!lex_force_int (lexer))
+ goto error;
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "LINE", "VARIABLES");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "FIXCASE"))
+ {
+ if (!set_type (lexer, parser, DP_FIXED,
+ lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
+ &type_start, &type_end))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "FIXCASE", 1, INT_MAX))
+ goto error;
+ data_parser_set_records (parser, lex_integer (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "IMPORTCASES"))
+ {
+ int start_ofs = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+ if (lex_match (lexer, T_ALL))
+ {
+ /* Nothing to do. */
+ }
+ else if (lex_match_id (lexer, "FIRST"))
+ {
+ if (!lex_force_int (lexer))
+ goto error;
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "PERCENT"))
+ {
+ if (!lex_force_int (lexer))
+ goto error;
+ lex_get (lexer);
+ }
+ lex_ofs_msg (lexer, SW, start_ofs, lex_ofs (lexer) - 1,
+ _("Ignoring obsolete IMPORTCASES subcommand. (N OF "
+ "CASES or SAMPLE may be used to substitute.)"));
+ }
+ else if (lex_match_id_n (lexer, "DELIMITERS", 4))
+ {
+ if (!set_type (lexer, parser, DP_DELIMITED,
+ lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
+ &type_start, &type_end))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_string (lexer))
+ goto error;
+
+ /* XXX should support multibyte UTF-8 characters */
+ struct substring s = lex_tokss (lexer);
+ struct string hard_seps = DS_EMPTY_INITIALIZER;
+ const char *soft_seps = "";
+ if (ss_match_string (&s, ss_cstr ("\\t")))
+ ds_put_cstr (&hard_seps, "\t");
+ if (ss_match_string (&s, ss_cstr ("\\\\")))
+ ds_put_cstr (&hard_seps, "\\");
+ int c;
+ while ((c = ss_get_byte (&s)) != EOF)
+ if (c == ' ')
+ soft_seps = " ";
+ else
+ ds_put_byte (&hard_seps, c);
+ data_parser_set_soft_delimiters (parser, ss_cstr (soft_seps));
+ data_parser_set_hard_delimiters (parser, ds_ss (&hard_seps));
+ ds_destroy (&hard_seps);
+
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "QUALIFIERS"))
+ {
+ if (!set_type (lexer, parser, DP_DELIMITED,
+ lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
+ &type_start, &type_end))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_string (lexer))
+ goto error;
+
+ /* XXX should support multibyte UTF-8 characters */
+ if (settings_get_syntax () == COMPATIBLE
+ && ss_length (lex_tokss (lexer)) != 1)
+ {
+ lex_error (lexer, _("In compatible syntax mode, the QUALIFIER "
+ "string must contain exactly one character."));
+ goto error;
+ }
+
+ data_parser_set_quotes (parser, lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "VARIABLES"))
+ break;
+ else
+ {
+ lex_error_expecting (lexer, "VARIABLES");
+ goto error;
+ }
+ }
+ lex_match (lexer, T_EQUALS);
+
+ int record = 1;
+ enum data_parser_type type = data_parser_get_type (parser);
+ do
+ {
+ while (type == DP_FIXED && lex_match (lexer, T_SLASH))
+ {
+ if (!lex_force_int_range (lexer, NULL, record,
+ data_parser_get_records (parser)))
+ goto error;
+ record = lex_integer (lexer);
+ lex_get (lexer);
+ }
+
+ int name_ofs = lex_ofs (lexer);
+ if (!lex_force_id (lexer))
+ goto error;
+ name = xstrdup (lex_tokcstr (lexer));
+ char *error = dict_id_is_valid__ (dict, name);
+ if (error)
+ {
+ lex_error (lexer, "%s", error);
+ free (error);
+ goto error;
+ }
+ lex_get (lexer);
+
+ struct fmt_spec input, output;
+ int fc, lc;
+ if (type == DP_DELIMITED)
+ {
+ if (!parse_format_specifier (lexer, &input))
+ goto error;
+ error = fmt_check_input__ (&input);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ goto error;
+ }
+ output = fmt_for_output_from_input (&input,
+ settings_get_fmt_settings ());
+ }
+ else
+ {
+ int start_ofs = lex_ofs (lexer);
+ if (!parse_column_range (lexer, 0, &fc, &lc, NULL))
+ goto error;
+
+ /* Accept a format (e.g. F8.2) or just a type name (e.g. DOLLAR). */
+ char fmt_type_name[FMT_TYPE_LEN_MAX + 1];
+ uint16_t w;
+ uint8_t d;
+ if (!parse_abstract_format_specifier (lexer, fmt_type_name, &w, &d))
+ goto error;
+
+ enum fmt_type fmt_type;
+ if (!fmt_from_name (fmt_type_name, &fmt_type))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Unknown format type `%s'."), fmt_type_name);
+ goto error;
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ /* Compose input format. */
+ input = (struct fmt_spec) { .type = fmt_type, .w = lc - fc + 1 };
+ error = fmt_check_input__ (&input);
+ if (error)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs, "%s", error);
+ free (error);
+ goto error;
+ }
+
+ /* Compose output format. */
+ if (w != 0)
+ {
+ output = (struct fmt_spec) { .type = fmt_type, .w = w, .d = d };
+ error = fmt_check_output__ (&output);
+ if (error)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs, "%s", error);
+ free (error);
+ goto error;
+ }
+ }
+ else
+ output = fmt_for_output_from_input (&input,
+ settings_get_fmt_settings ());
+ }
+ struct variable *v = dict_create_var (dict, name, fmt_var_width (&input));
+ if (!v)
+ {
+ lex_ofs_error (lexer, name_ofs, name_ofs,
+ _("%s is a duplicate variable name."), name);
+ goto error;
+ }
+ var_set_both_formats (v, &output);
+ if (type == DP_DELIMITED)
+ data_parser_add_delimited_field (parser, &input,
+ var_get_case_index (v),
+ name);
+ else
+ data_parser_add_fixed_field (parser, &input, var_get_case_index (v),
+ name, record, fc);
+ free (name);
+ name = NULL;
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ struct dfm_reader *reader = dfm_open_reader (fh, lexer, encoding);
+ if (!reader)
+ goto error;
+
+ data_parser_make_active_file (parser, ds, reader, dict, NULL, NULL);
+ fh_unref (fh);
+ free (encoding);
+ return CMD_SUCCESS;
+
+ error:
+ data_parser_destroy (parser);
+ dict_unref (dict);
+ fh_unref (fh);
+ free (name);
+ free (encoding);
+ return CMD_CASCADING_FAILURE;
+}
+
+static void
+destroy_spreadsheet_read_info (struct spreadsheet_read_options *opts)
+{
+ free (opts->cell_range);
+ free (opts->sheet_name);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006-2007, 2010-15 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/any-reader.h"
+#include "data/case-map.h"
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/por-file-writer.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/trim.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/compiler.h"
+#include "libpspp/misc.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+\f
+/* Reading system and portable files. */
+
+/* Type of command. */
+enum reader_command
+ {
+ GET_CMD,
+ IMPORT_CMD
+ };
+
+static int parse_read_command (struct lexer *, struct dataset *,
+ enum reader_command);
+
+/* GET. */
+int
+cmd_get (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_read_command (lexer, ds, GET_CMD);
+}
+
+/* IMPORT. */
+int
+cmd_import (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_read_command (lexer, ds, IMPORT_CMD);
+}
+
+/* Parses a GET or IMPORT command. */
+static int
+parse_read_command (struct lexer *lexer, struct dataset *ds,
+ enum reader_command command)
+{
+ struct casereader *reader = NULL;
+ struct file_handle *fh = NULL;
+ struct dictionary *dict = NULL;
+ struct case_map *map = NULL;
+ struct case_map_stage *stage = NULL;
+ char *encoding = NULL;
+
+ for (;;)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "FILE") || lex_is_string (lexer))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ fh_unref (fh);
+ fh = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (fh == NULL)
+ goto error;
+ }
+ else if (command == GET_CMD && lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ }
+ else if (command == IMPORT_CMD && lex_match_id (lexer, "TYPE"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_match_id (lexer, "COMM")
+ && !lex_match_id (lexer, "TAPE"))
+ {
+ lex_error_expecting (lexer, "COMM", "TAPE");
+ goto error;
+ }
+ }
+ else
+ break;
+ }
+
+ if (fh == NULL)
+ {
+ lex_sbc_missing (lexer, "FILE");
+ goto error;
+ }
+
+ reader = any_reader_open_and_decode (fh, encoding, &dict, NULL);
+ if (reader == NULL)
+ goto error;
+
+ if (dict_get_n_vars (dict) == 0)
+ {
+ msg (SE, _("%s: Data file dictionary has no variables."),
+ fh_get_name (fh));
+ goto error;
+ }
+
+ stage = case_map_stage_create (dict);
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+ if (!parse_dict_trim (lexer, dict))
+ goto error;
+ }
+ dict_compact_values (dict);
+
+ map = case_map_stage_get_case_map (stage);
+ case_map_stage_destroy (stage);
+ if (map != NULL)
+ reader = case_map_create_input_translator (map, reader);
+
+ dataset_set_dict (ds, dict);
+ dataset_set_source (ds, reader);
+
+ fh_unref (fh);
+ free (encoding);
+ return CMD_SUCCESS;
+
+ error:
+ case_map_stage_destroy (stage);
+ fh_unref (fh);
+ casereader_destroy (reader);
+ if (dict != NULL)
+ dict_unref (dict);
+ free (encoding);
+ return CMD_CASCADING_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_matrix.h>
+#include <gsl/gsl_combination.h>
+#include <math.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/ll.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/taint.h"
+#include "linreg/sweep.h"
+#include "math/categoricals.h"
+#include "math/covariance.h"
+#include "math/interaction.h"
+#include "math/moments.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+struct glm_spec
+ {
+ const struct variable **dep_vars;
+ size_t n_dep_vars;
+
+ const struct variable **factor_vars;
+ size_t n_factor_vars;
+
+ struct interaction **interactions;
+ size_t n_interactions;
+
+ enum mv_class exclude;
+
+ const struct variable *wv; /* The weight variable */
+
+ const struct dictionary *dict;
+
+ int ss_type;
+ bool intercept;
+
+ double alpha;
+
+ bool dump_coding;
+ };
+
+struct glm_workspace
+ {
+ double total_ssq;
+ struct moments *totals;
+
+ struct categoricals *cats;
+
+ /*
+ Sums of squares due to different variables. Element 0 is the SSE
+ for the entire model. For i > 0, element i is the SS due to
+ variable i.
+ */
+ gsl_vector *ssq;
+ };
+
+/* Default design: all possible interactions */
+static void
+design_full (struct glm_spec *glm)
+{
+ size_t n = (1 << glm->n_factor_vars) - 1;
+ glm->interactions = xnmalloc (n, sizeof *glm->interactions);
+
+ /* All subsets, with exception of the empty set, of [0, glm->n_factor_vars) */
+ for (size_t sz = 1; sz <= glm->n_factor_vars; ++sz)
+ {
+ gsl_combination *c = gsl_combination_calloc (glm->n_factor_vars, sz);
+
+ do
+ {
+ struct interaction *iact = interaction_create (NULL);
+ for (int e = 0; e < gsl_combination_k (c); ++e)
+ interaction_add_variable (
+ iact, glm->factor_vars [gsl_combination_get (c, e)]);
+
+ glm->interactions[glm->n_interactions++] = iact;
+ }
+ while (gsl_combination_next (c) == GSL_SUCCESS);
+
+ gsl_combination_free (c);
+ }
+ assert (glm->n_interactions == n);
+}
+
+static void output_glm (const struct glm_spec *,
+ const struct glm_workspace *ws);
+static void run_glm (struct glm_spec *cmd, struct casereader *input,
+ const struct dataset *ds);
+
+static struct interaction *parse_design_term (struct lexer *,
+ const struct dictionary *);
+
+int
+cmd_glm (struct lexer *lexer, struct dataset *ds)
+{
+ struct const_var_set *factors = NULL;
+ bool design = false;
+ struct dictionary *dict = dataset_dict (ds);
+ struct glm_spec glm = {
+ .dict = dict,
+ .exclude = MV_ANY,
+ .intercept = true,
+ .wv = dict_get_weight (dict),
+ .alpha = 0.05,
+ .ss_type = 3,
+ };
+
+ int dep_vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, glm.dict,
+ &glm.dep_vars, &glm.n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ int dep_vars_end = lex_ofs (lexer) - 1;
+
+ if (!lex_force_match (lexer, T_BY))
+ goto error;
+
+ if (!parse_variables_const (lexer, glm.dict,
+ &glm.factor_vars, &glm.n_factor_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+
+ if (glm.n_dep_vars > 1)
+ {
+ lex_ofs_error (lexer, dep_vars_start, dep_vars_end,
+ _("Multivariate analysis is not yet implemented."));
+ goto error;
+ }
+
+ factors = const_var_set_create_from_array (glm.factor_vars, glm.n_factor_vars);
+
+ size_t allocated_interactions = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ glm.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ glm.exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "INTERCEPT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ glm.intercept = true;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ glm.intercept = false;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_match_phrase (lexer, "ALPHA(")
+ || !lex_force_num (lexer))
+ goto error;
+ glm.alpha = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "METHOD"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_match_phrase (lexer, "SSTYPE(")
+ || !lex_force_int_range (lexer, "SSTYPE", 1, 3))
+ goto error;
+
+ glm.ss_type = lex_integer (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "DESIGN"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ do
+ {
+ struct interaction *iact = parse_design_term (lexer, glm.dict);
+ if (!iact)
+ goto error;
+
+ if (glm.n_interactions >= allocated_interactions)
+ glm.interactions = x2nrealloc (glm.interactions,
+ &allocated_interactions,
+ sizeof *glm.interactions);
+ glm.interactions[glm.n_interactions++] = iact;
+
+ lex_match (lexer, T_COMMA);
+ }
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH);
+
+ if (glm.n_interactions > 0)
+ design = true;
+ }
+ else if (lex_match_id (lexer, "SHOWCODES"))
+ {
+ /* Undocumented debug option */
+ glm.dump_coding = true;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MISSING", "INTERCEPT", "CRITERIA",
+ "METHOD", "DESIGN");
+ goto error;
+ }
+ }
+
+ if (!design)
+ design_full (&glm);
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), glm.dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ run_glm (&glm, group, ds);
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ const_var_set_destroy (factors);
+ free (glm.factor_vars);
+ for (size_t i = 0; i < glm.n_interactions; ++i)
+ interaction_destroy (glm.interactions[i]);
+
+ free (glm.interactions);
+ free (glm.dep_vars);
+
+ return CMD_SUCCESS;
+
+error:
+ const_var_set_destroy (factors);
+ free (glm.factor_vars);
+ for (size_t i = 0; i < glm.n_interactions; ++i)
+ interaction_destroy (glm.interactions[i]);
+
+ free (glm.interactions);
+ free (glm.dep_vars);
+
+ return CMD_FAILURE;
+}
+
+static inline bool
+not_dropped (size_t j, const bool *ff)
+{
+ return !ff[j];
+}
+
+static void
+fill_submatrix (const gsl_matrix * cov, gsl_matrix * submatrix, bool *dropped_f)
+{
+ size_t i;
+ size_t j;
+ size_t n = 0;
+ size_t m = 0;
+
+ for (i = 0; i < cov->size1; i++)
+ {
+ if (not_dropped (i, dropped_f))
+ {
+ m = 0;
+ for (j = 0; j < cov->size2; j++)
+ {
+ if (not_dropped (j, dropped_f))
+ {
+ gsl_matrix_set (submatrix, n, m,
+ gsl_matrix_get (cov, i, j));
+ m++;
+ }
+ }
+ n++;
+ }
+ }
+}
+
+
+/*
+ Type 1 sums of squares.
+ Populate SSQ with the Type 1 sums of squares according to COV
+ */
+static void
+ssq_type1 (struct covariance *cov, gsl_vector *ssq, const struct glm_spec *cmd)
+{
+ const gsl_matrix *cm = covariance_calculate_unnormalized (cov);
+ size_t i;
+ size_t k;
+ bool *model_dropped = XCALLOC (covariance_dim (cov), bool);
+ bool *submodel_dropped = XCALLOC (covariance_dim (cov), bool);
+ const struct categoricals *cats = covariance_get_categoricals (cov);
+
+ size_t n_dropped_model = 0;
+ size_t n_dropped_submodel = 0;
+
+ for (i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
+ {
+ n_dropped_model++;
+ n_dropped_submodel++;
+ model_dropped[i] = true;
+ submodel_dropped[i] = true;
+ }
+
+ for (k = 0; k < cmd->n_interactions; k++)
+ {
+ gsl_matrix *model_cov = NULL;
+ gsl_matrix *submodel_cov = NULL;
+
+ n_dropped_submodel = n_dropped_model;
+ for (i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
+ submodel_dropped[i] = model_dropped[i];
+
+ for (i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
+ {
+ const struct interaction * x =
+ categoricals_get_interaction_by_subscript (cats, i - cmd->n_dep_vars);
+
+ if (x == cmd->interactions [k])
+ {
+ model_dropped[i] = false;
+ n_dropped_model--;
+ }
+ }
+
+ model_cov = gsl_matrix_alloc (cm->size1 - n_dropped_model, cm->size2 - n_dropped_model);
+ submodel_cov = gsl_matrix_alloc (cm->size1 - n_dropped_submodel, cm->size2 - n_dropped_submodel);
+
+ fill_submatrix (cm, model_cov, model_dropped);
+ fill_submatrix (cm, submodel_cov, submodel_dropped);
+
+ reg_sweep (model_cov, 0);
+ reg_sweep (submodel_cov, 0);
+
+ gsl_vector_set (ssq, k + 1,
+ gsl_matrix_get (submodel_cov, 0, 0) - gsl_matrix_get (model_cov, 0, 0)
+ );
+
+ gsl_matrix_free (model_cov);
+ gsl_matrix_free (submodel_cov);
+ }
+
+ free (model_dropped);
+ free (submodel_dropped);
+}
+
+/*
+ Type 2 sums of squares.
+ Populate SSQ with the Type 2 sums of squares according to COV
+ */
+static void
+ssq_type2 (struct covariance *cov, gsl_vector *ssq, const struct glm_spec *cmd)
+{
+ const gsl_matrix *cm = covariance_calculate_unnormalized (cov);
+ bool *model_dropped = XCALLOC (covariance_dim (cov), bool);
+ bool *submodel_dropped = XCALLOC (covariance_dim (cov), bool);
+ const struct categoricals *cats = covariance_get_categoricals (cov);
+
+ for (size_t k = 0; k < cmd->n_interactions; k++)
+ {
+ gsl_matrix *model_cov = NULL;
+ gsl_matrix *submodel_cov = NULL;
+ size_t n_dropped_model = 0;
+ size_t n_dropped_submodel = 0;
+ for (size_t i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
+ {
+ const struct interaction * x =
+ categoricals_get_interaction_by_subscript (cats, i - cmd->n_dep_vars);
+
+ model_dropped[i] = false;
+ submodel_dropped[i] = false;
+ if (interaction_is_subset (cmd->interactions [k], x))
+ {
+ assert (n_dropped_submodel < covariance_dim (cov));
+ n_dropped_submodel++;
+ submodel_dropped[i] = true;
+
+ if (cmd->interactions [k]->n_vars < x->n_vars)
+ {
+ assert (n_dropped_model < covariance_dim (cov));
+ n_dropped_model++;
+ model_dropped[i] = true;
+ }
+ }
+ }
+
+ model_cov = gsl_matrix_alloc (cm->size1 - n_dropped_model, cm->size2 - n_dropped_model);
+ submodel_cov = gsl_matrix_alloc (cm->size1 - n_dropped_submodel, cm->size2 - n_dropped_submodel);
+
+ fill_submatrix (cm, model_cov, model_dropped);
+ fill_submatrix (cm, submodel_cov, submodel_dropped);
+
+ reg_sweep (model_cov, 0);
+ reg_sweep (submodel_cov, 0);
+
+ gsl_vector_set (ssq, k + 1,
+ gsl_matrix_get (submodel_cov, 0, 0) - gsl_matrix_get (model_cov, 0, 0)
+ );
+
+ gsl_matrix_free (model_cov);
+ gsl_matrix_free (submodel_cov);
+ }
+
+ free (model_dropped);
+ free (submodel_dropped);
+}
+
+/*
+ Type 3 sums of squares.
+ Populate SSQ with the Type 2 sums of squares according to COV
+ */
+static void
+ssq_type3 (struct covariance *cov, gsl_vector *ssq, const struct glm_spec *cmd)
+{
+ const gsl_matrix *cm = covariance_calculate_unnormalized (cov);
+ bool *model_dropped = XCALLOC (covariance_dim (cov), bool);
+ bool *submodel_dropped = XCALLOC (covariance_dim (cov), bool);
+ const struct categoricals *cats = covariance_get_categoricals (cov);
+
+ gsl_matrix *submodel_cov = gsl_matrix_alloc (cm->size1, cm->size2);
+ fill_submatrix (cm, submodel_cov, submodel_dropped);
+ reg_sweep (submodel_cov, 0);
+ double ss0 = gsl_matrix_get (submodel_cov, 0, 0);
+ gsl_matrix_free (submodel_cov);
+ free (submodel_dropped);
+
+ for (size_t k = 0; k < cmd->n_interactions; k++)
+ {
+ size_t n_dropped_model = 0;
+ for (size_t i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
+ {
+ const struct interaction * x =
+ categoricals_get_interaction_by_subscript (cats, i - cmd->n_dep_vars);
+
+ model_dropped[i] = false;
+
+ if (cmd->interactions [k] == x)
+ {
+ assert (n_dropped_model < covariance_dim (cov));
+ n_dropped_model++;
+ model_dropped[i] = true;
+ }
+ }
+
+ gsl_matrix *model_cov = gsl_matrix_alloc (cm->size1 - n_dropped_model,
+ cm->size2 - n_dropped_model);
+
+ fill_submatrix (cm, model_cov, model_dropped);
+
+ reg_sweep (model_cov, 0);
+
+ gsl_vector_set (ssq, k + 1, gsl_matrix_get (model_cov, 0, 0) - ss0);
+
+ gsl_matrix_free (model_cov);
+ }
+ free (model_dropped);
+}
+
+static void
+run_glm (struct glm_spec *cmd, struct casereader *input,
+ const struct dataset *ds)
+{
+ bool warn_bad_weight = true;
+ struct dictionary *dict = dataset_dict (ds);
+
+
+ input = casereader_create_filter_missing (input,
+ cmd->dep_vars, cmd->n_dep_vars,
+ cmd->exclude,
+ NULL, NULL);
+
+ input = casereader_create_filter_missing (input,
+ cmd->factor_vars, cmd->n_factor_vars,
+ cmd->exclude,
+ NULL, NULL);
+
+ struct glm_workspace ws = {
+ .cats = categoricals_create (cmd->interactions, cmd->n_interactions,
+ cmd->wv, MV_ANY)
+ };
+
+ struct covariance *cov = covariance_2pass_create (
+ cmd->n_dep_vars, cmd->dep_vars, ws.cats, cmd->wv, cmd->exclude, true);
+
+ output_split_file_values_peek (ds, input);
+
+ struct taint *taint = taint_clone (casereader_get_taint (input));
+
+ ws.totals = moments_create (MOMENT_VARIANCE);
+
+ struct casereader *reader = casereader_clone (input);
+ struct ccase *c;
+ for (; (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ double weight = dict_get_case_weight (dict, c, &warn_bad_weight);
+
+ for (int v = 0; v < cmd->n_dep_vars; ++v)
+ moments_pass_one (ws.totals, case_num (c, cmd->dep_vars[v]), weight);
+
+ covariance_accumulate_pass1 (cov, c);
+ }
+ casereader_destroy (reader);
+
+ if (cmd->dump_coding)
+ reader = casereader_clone (input);
+ else
+ reader = input;
+
+ for (; (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ double weight = dict_get_case_weight (dict, c, &warn_bad_weight);
+
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ moments_pass_two (ws.totals, case_num (c, cmd->dep_vars[v]), weight);
+
+ covariance_accumulate_pass2 (cov, c);
+ }
+ casereader_destroy (reader);
+
+
+ if (cmd->dump_coding)
+ {
+ struct pivot_table *t = covariance_dump_enc_header (cov);
+ for (reader = input;
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ covariance_dump_enc (cov, c, t);
+ }
+
+ pivot_table_submit (t);
+ }
+
+ {
+ const gsl_matrix *ucm = covariance_calculate_unnormalized (cov);
+ gsl_matrix *cm = gsl_matrix_alloc (ucm->size1, ucm->size2);
+ gsl_matrix_memcpy (cm, ucm);
+
+ // dump_matrix (cm);
+
+ ws.total_ssq = gsl_matrix_get (cm, 0, 0);
+
+ reg_sweep (cm, 0);
+
+ /*
+ Store the overall SSE.
+ */
+ ws.ssq = gsl_vector_alloc (cm->size1);
+ gsl_vector_set (ws.ssq, 0, gsl_matrix_get (cm, 0, 0));
+ switch (cmd->ss_type)
+ {
+ case 1:
+ ssq_type1 (cov, ws.ssq, cmd);
+ break;
+ case 2:
+ ssq_type2 (cov, ws.ssq, cmd);
+ break;
+ case 3:
+ ssq_type3 (cov, ws.ssq, cmd);
+ break;
+ default:
+ NOT_REACHED ();
+ break;
+ }
+ // dump_matrix (cm);
+ gsl_matrix_free (cm);
+ }
+
+ if (!taint_has_tainted_successor (taint))
+ output_glm (cmd, &ws);
+
+ gsl_vector_free (ws.ssq);
+
+ covariance_destroy (cov);
+ moments_destroy (ws.totals);
+
+ taint_destroy (taint);
+}
+
+static void
+put_glm_row (struct pivot_table *table, int row,
+ double a, double b, double c, double d, double e)
+{
+ double entries[] = { a, b, c, d, e };
+
+ for (size_t col = 0; col < sizeof entries / sizeof *entries; col++)
+ if (entries[col] != SYSMIS)
+ pivot_table_put2 (table, col, row,
+ pivot_value_new_number (entries[col]));
+}
+
+static void
+output_glm (const struct glm_spec *cmd, const struct glm_workspace *ws)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Tests of Between-Subjects Effects"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ (cmd->ss_type == 1 ? N_("Type I Sum Of Squares")
+ : cmd->ss_type == 2 ? N_("Type II Sum Of Squares")
+ : N_("Type III Sum Of Squares")), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_COUNT,
+ N_("Mean Square"), PIVOT_RC_OTHER,
+ N_("F"), PIVOT_RC_OTHER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *source = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Source"),
+ cmd->intercept ? N_("Corrected Model") : N_("Model"));
+
+ double n_total, mean;
+ moments_calculate (ws->totals, &n_total, &mean, NULL, NULL, NULL);
+
+ double df_corr = 1.0 + categoricals_df_total (ws->cats);
+
+ double mse = gsl_vector_get (ws->ssq, 0) / (n_total - df_corr);
+ double intercept_ssq = pow2 (mean * n_total) / n_total;
+ if (cmd->intercept)
+ {
+ int row = pivot_category_create_leaf (
+ source->root, pivot_value_new_text (N_("Intercept")));
+
+ /* The intercept for unbalanced models is of limited use and
+ nobody knows how to calculate it properly */
+ if (categoricals_isbalanced (ws->cats))
+ {
+ const double df = 1.0;
+ const double F = intercept_ssq / df / mse;
+ put_glm_row (table, row, intercept_ssq, 1.0, intercept_ssq / df,
+ F, gsl_cdf_fdist_Q (F, df, n_total - df_corr));
+ }
+ }
+
+ double ssq_effects = 0.0;
+ for (int f = 0; f < cmd->n_interactions; ++f)
+ {
+ double df = categoricals_df (ws->cats, f);
+ double ssq = gsl_vector_get (ws->ssq, f + 1);
+ ssq_effects += ssq;
+ if (!cmd->intercept)
+ {
+ df++;
+ ssq += intercept_ssq;
+ }
+ double F = ssq / df / mse;
+
+ struct string str = DS_EMPTY_INITIALIZER;
+ interaction_to_string (cmd->interactions[f], &str);
+ int row = pivot_category_create_leaf (
+ source->root, pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
+
+ put_glm_row (table, row, ssq, df, ssq / df, F,
+ gsl_cdf_fdist_Q (F, df, n_total - df_corr));
+ }
+
+ {
+ /* Model / Corrected Model */
+ double df = df_corr;
+ double ssq = ws->total_ssq - gsl_vector_get (ws->ssq, 0);
+ if (cmd->intercept)
+ df--;
+ else
+ ssq += intercept_ssq;
+ double F = ssq / df / mse;
+ put_glm_row (table, 0, ssq, df, ssq / df, F,
+ gsl_cdf_fdist_Q (F, df, n_total - df_corr));
+ }
+
+ {
+ int row = pivot_category_create_leaf (source->root,
+ pivot_value_new_text (N_("Error")));
+ const double df = n_total - df_corr;
+ const double ssq = gsl_vector_get (ws->ssq, 0);
+ const double mse = ssq / df;
+ put_glm_row (table, row, ssq, df, mse, SYSMIS, SYSMIS);
+ }
+
+ {
+ int row = pivot_category_create_leaf (source->root,
+ pivot_value_new_text (N_("Total")));
+ put_glm_row (table, row, ws->total_ssq + intercept_ssq, n_total,
+ SYSMIS, SYSMIS, SYSMIS);
+ }
+
+ if (cmd->intercept)
+ {
+ int row = pivot_category_create_leaf (
+ source->root, pivot_value_new_text (N_("Corrected Total")));
+ put_glm_row (table, row, ws->total_ssq, n_total - 1.0, SYSMIS,
+ SYSMIS, SYSMIS);
+ }
+
+ pivot_table_submit (table);
+}
+
+#if 0
+static void
+dump_matrix (const gsl_matrix * m)
+{
+ size_t i, j;
+ for (i = 0; i < m->size1; ++i)
+ {
+ for (j = 0; j < m->size2; ++j)
+ {
+ double x = gsl_matrix_get (m, i, j);
+ printf ("%.3f ", x);
+ }
+ printf ("\n");
+ }
+ printf ("\n");
+}
+#endif
+
+
+\f
+static struct interaction *
+parse_design_term (struct lexer *lexer, const struct dictionary *dict)
+{
+ struct interaction *iact = interaction_create (NULL);
+ do
+ {
+ struct variable *var = parse_variable (lexer, dict);
+ if (!var)
+ goto error;
+ interaction_add_variable (iact, var);
+
+ if (lex_match (lexer, T_LPAREN) || lex_match_id (lexer, "WITHIN"))
+ {
+ lex_next_error (lexer, -1, -1,
+ "Nested variables are not yet implemented.");
+ goto error;
+ }
+ }
+ while (lex_match (lexer, T_ASTERISK));
+
+ return iact;
+
+error:
+ interaction_destroy (iact);
+ return NULL;
+}
--- /dev/null
+/*
+ PSPP - a program for statistical analysis.
+ Copyright (C) 2012, 2013, 2015, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+/*
+ * This module implements the graph command
+ */
+
+#include <config.h>
+
+#include <math.h>
+#include "gl/xalloc.h"
+#include <gsl/gsl_cdf.h>
+
+#include "libpspp/assertion.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/caseproto.h"
+#include "data/subcase.h"
+
+
+#include "data/format.h"
+
+#include "math/chart-geometry.h"
+#include "math/histogram.h"
+#include "math/moments.h"
+#include "math/sort.h"
+#include "math/order-stats.h"
+#include "output/charts/plot-hist.h"
+#include "output/charts/scatterplot.h"
+#include "output/charts/barchart.h"
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/freq.h"
+#include "language/commands/chart-category.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+enum chart_type
+ {
+ CT_NONE,
+ CT_BAR,
+ CT_LINE,
+ CT_PIE,
+ CT_ERRORBAR,
+ CT_HILO,
+ CT_HISTOGRAM,
+ CT_SCATTERPLOT,
+ CT_PARETO
+ };
+
+enum scatter_type
+ {
+ ST_BIVARIATE,
+ ST_OVERLAY,
+ ST_MATRIX,
+ ST_XYZ
+ };
+
+enum bar_type
+ {
+ CBT_SIMPLE,
+ CBT_GROUPED,
+ CBT_STACKED,
+ CBT_RANGE
+ };
+
+
+/* Variable index for histogram case */
+enum
+ {
+ HG_IDX_X,
+ HG_IDX_WT
+ };
+
+struct exploratory_stats
+{
+ double missing;
+ double non_missing;
+
+ struct moments *mom;
+
+ double minimum;
+ double maximum;
+
+ /* Total weight */
+ double cc;
+
+ /* The minimum weight */
+ double cmin;
+};
+
+
+struct graph
+{
+ struct pool *pool;
+
+ size_t n_dep_vars;
+ const struct variable **dep_vars;
+ struct exploratory_stats *es;
+
+ enum mv_class dep_excl;
+ enum mv_class fctr_excl;
+
+ const struct dictionary *dict;
+
+ bool missing_pw;
+
+ /* ------------ Graph ---------------- */
+ bool normal; /* For histograms, draw the normal curve */
+
+ enum chart_type chart_type;
+ enum scatter_type scatter_type;
+ enum bar_type bar_type;
+ const struct variable *by_var[2];
+ size_t n_by_vars;
+
+ struct subcase ordering; /* Ordering for aggregation */
+ int agr; /* Index into ag_func */
+
+ /* A caseproto that contains the plot data */
+ struct caseproto *gr_proto;
+};
+
+
+
+
+static double
+calc_mom1 (double acc, double x, double w)
+{
+ return acc + x * w;
+}
+
+static double
+calc_mom0 (double acc, double x UNUSED, double w)
+{
+ return acc + w;
+}
+
+static double
+pre_low_extreme (void)
+{
+ return -DBL_MAX;
+}
+
+static double
+calc_max (double acc, double x, double w UNUSED)
+{
+ return (acc > x) ? acc : x;
+}
+
+static double
+pre_high_extreme (void)
+{
+ return DBL_MAX;
+}
+
+static double
+calc_min (double acc, double x, double w UNUSED)
+{
+ return (acc < x) ? acc : x;
+}
+
+static double
+post_normalise (double acc, double cc)
+{
+ return acc / cc;
+}
+
+static double
+post_percentage (double acc, double ccc)
+{
+ return acc / ccc * 100.0;
+}
+
+const struct ag_func ag_func[] =
+ {
+ {"COUNT", N_("Count"), 0, 0, NULL, calc_mom0, 0, 0},
+ {"PCT", N_("Percentage"), 0, 0, NULL, calc_mom0, 0, post_percentage},
+ {"CUFREQ", N_("Cumulative Count"), 0, 1, NULL, calc_mom0, 0, 0},
+ {"CUPCT", N_("Cumulative Percent"), 0, 1, NULL, calc_mom0, 0,
+ post_percentage},
+
+ {"MEAN", N_("Mean"), 1, 0, NULL, calc_mom1, post_normalise, 0},
+ {"SUM", N_("Sum"), 1, 0, NULL, calc_mom1, 0, 0},
+ {"MAXIMUM", N_("Maximum"), 1, 0, pre_low_extreme, calc_max, 0, 0},
+ {"MINIMUM", N_("Minimum"), 1, 0, pre_high_extreme, calc_min, 0, 0},
+ };
+
+const int N_AG_FUNCS = sizeof (ag_func) / sizeof (ag_func[0]);
+
+static bool
+parse_function_name (struct lexer *lexer, int *agr)
+{
+ for (size_t i = 0; i < N_AG_FUNCS; ++i)
+ {
+ if (lex_match_id (lexer, ag_func[i].name))
+ {
+ *agr = i;
+ return true;
+ }
+ }
+
+ const char *ag_func_names[N_AG_FUNCS];
+ for (size_t i = 0; i < N_AG_FUNCS; ++i)
+ ag_func_names[i] = ag_func[i].name;
+ lex_error_expecting_array (lexer, ag_func_names, N_AG_FUNCS);
+ return false;
+}
+
+static bool
+parse_function (struct lexer *lexer, struct graph *graph)
+{
+ if (!parse_function_name (lexer, &graph->agr))
+ return false;
+
+ size_t arity = ag_func[graph->agr].arity;
+ graph->n_dep_vars = arity;
+ if (arity > 0)
+ {
+ if (!lex_force_match (lexer, T_LPAREN))
+ return false;
+
+ graph->dep_vars = xcalloc (graph->n_dep_vars, sizeof (graph->dep_vars));
+ for (int v = 0; v < arity; ++v)
+ {
+ graph->dep_vars[v] = parse_variable (lexer, graph->dict);
+ if (!graph->dep_vars[v])
+ return false;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+
+ if (!lex_force_match (lexer, T_BY))
+ return false;
+
+ graph->by_var[0] = parse_variable (lexer, graph->dict);
+ if (!graph->by_var[0])
+ return false;
+ subcase_add_var (&graph->ordering, graph->by_var[0], SC_ASCEND);
+ graph->n_by_vars++;
+
+ if (lex_match (lexer, T_BY))
+ {
+ graph->by_var[1] = parse_variable (lexer, graph->dict);
+ if (!graph->by_var[1])
+ return false;
+ subcase_add_var (&graph->ordering, graph->by_var[1], SC_ASCEND);
+ graph->n_by_vars++;
+ }
+
+ return true;
+}
+
+static void
+show_scatterplot (const struct graph *cmd, struct casereader *input)
+{
+ struct scatterplot_chart *scatterplot;
+ bool byvar_overflow = false;
+
+ char *title = (cmd->n_by_vars > 0
+ ? xasprintf (_("%s vs. %s by %s"),
+ var_to_string (cmd->dep_vars[1]),
+ var_to_string (cmd->dep_vars[0]),
+ var_to_string (cmd->by_var[0]))
+ : xasprintf (_("%s vs. %s"),
+ var_to_string (cmd->dep_vars[1]),
+ var_to_string (cmd->dep_vars[0])));;
+
+ scatterplot = scatterplot_create (input,
+ var_to_string(cmd->dep_vars[0]),
+ var_to_string(cmd->dep_vars[1]),
+ (cmd->n_by_vars > 0) ? cmd->by_var[0]
+ : NULL,
+ &byvar_overflow,
+ title,
+ cmd->es[0].minimum, cmd->es[0].maximum,
+ cmd->es[1].minimum, cmd->es[1].maximum);
+ scatterplot_chart_submit (scatterplot);
+ free (title);
+
+ if (byvar_overflow)
+ msg (MW, _("Maximum number of scatterplot categories reached. "
+ "Your BY variable has too many distinct values. "
+ "The coloring of the plot will not be correct."));
+}
+
+static void
+show_histogr (const struct graph *cmd, struct casereader *input)
+{
+ struct histogram *histogram;
+
+ if (cmd->es[0].cc <= 0)
+ {
+ casereader_destroy (input);
+ return;
+ }
+
+ /* Sturges Rule */
+ double bin_width = fabs (cmd->es[0].minimum - cmd->es[0].maximum)
+ / (1 + log2 (cmd->es[0].cc));
+ histogram = histogram_create (bin_width,
+ cmd->es[0].minimum, cmd->es[0].maximum);
+ if (!histogram)
+ {
+ casereader_destroy (input);
+ return;
+ }
+
+ struct ccase *c;
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ {
+ const double x = case_num_idx (c, HG_IDX_X);
+ const double weight = case_num_idx (c, HG_IDX_WT);
+ moments_pass_two (cmd->es[0].mom, x, weight);
+ histogram_add (histogram, x, weight);
+ }
+ casereader_destroy (input);
+
+ const char *label = var_to_string (cmd->dep_vars[0]);
+ double n, mean, var;
+ moments_calculate (cmd->es[0].mom, &n, &mean, &var, NULL, NULL);
+ chart_submit (histogram_chart_create (histogram->gsl_hist, label, n, mean,
+ sqrt (var), cmd->normal));
+
+ statistic_destroy (&histogram->parent);
+}
+
+static void
+cleanup_exploratory_stats (struct graph *cmd)
+{
+ for (size_t v = 0; v < cmd->n_dep_vars; ++v)
+ moments_destroy (cmd->es[v].mom);
+}
+
+static bool
+any_categorical_missing (const struct graph *cmd, const struct ccase *c)
+{
+ for (size_t v = 0; v < cmd->n_by_vars; ++v)
+ if (var_is_value_missing (cmd->by_var[v], case_data (c, cmd->by_var[v]))
+ & cmd->fctr_excl)
+ return true;
+ return false;
+}
+
+static struct freq *
+find_fcol (struct hmap *columns, const union value *value, size_t hash,
+ int width)
+{
+ struct freq *fcol;
+ HMAP_FOR_EACH_WITH_HASH (fcol, struct freq, node, hash, columns)
+ if (value_equal (value, &fcol->values[0], width))
+ return fcol;
+ return NULL;
+}
+
+static void
+run_barchart (struct graph *cmd, struct casereader *input)
+{
+ double ccc = 0.0;
+
+ if (cmd->missing_pw == false)
+ input = casereader_create_filter_missing (input,
+ cmd->dep_vars,
+ cmd->n_dep_vars,
+ cmd->dep_excl,
+ NULL,
+ NULL);
+
+
+ input = sort_execute (input, &cmd->ordering);
+
+ struct freq **cells = NULL;
+ size_t n_cells = 0;
+ size_t allocated_cells = 0;
+
+ struct hmap columns = HMAP_INITIALIZER (columns);
+ assert (cmd->n_by_vars <= 2);
+ struct casegrouper *grouper = casegrouper_create_vars (input, cmd->by_var,
+ cmd->n_by_vars);
+ struct casereader *group;
+ for (; casegrouper_get_next_group (grouper, &group);
+ casereader_destroy (group))
+ {
+ struct ccase *c = casereader_peek (group, 0);
+ if (any_categorical_missing (cmd, c))
+ {
+ case_unref (c);
+ continue;
+ }
+
+ if (n_cells >= allocated_cells)
+ cells = x2nrealloc (cells, &allocated_cells, sizeof *cells);
+ cells[n_cells++] = xzalloc (table_entry_size (cmd->n_by_vars));
+
+ if (ag_func[cmd->agr].cumulative && n_cells >= 2)
+ cells[n_cells - 1]->count = cells[n_cells - 2]->count;
+ else
+ cells[n_cells - 1]->count = 0;
+ if (ag_func[cmd->agr].pre)
+ cells[n_cells - 1]->count = ag_func[cmd->agr].pre();
+
+ if (cmd->n_by_vars > 1)
+ {
+ const union value *vv = case_data (c, cmd->by_var[1]);
+ const double weight = dict_get_case_weight (cmd->dict, c, NULL);
+ int v1_width = var_get_width (cmd->by_var[1]);
+ size_t hash = value_hash (vv, v1_width, 0);
+
+ struct freq *fcol = find_fcol (&columns, vv, hash, v1_width);
+ if (!fcol)
+ {
+ fcol = xzalloc (sizeof *fcol);
+ value_clone (&fcol->values[0], vv, v1_width);
+ hmap_insert (&columns, &fcol->node, hash);
+ }
+ fcol->count += weight;
+ }
+
+ for (size_t v = 0; v < cmd->n_by_vars; ++v)
+ value_clone (&cells[n_cells - 1]->values[v],
+ case_data (c, cmd->by_var[v]),
+ var_get_width (cmd->by_var[v]));
+ case_unref (c);
+
+ double cc = 0;
+ for (; (c = casereader_read (group)) != NULL; case_unref (c))
+ {
+ const double weight = dict_get_case_weight (cmd->dict, c, NULL);
+ const double x = (cmd->n_dep_vars > 0
+ ? case_num (c, cmd->dep_vars[0]) : SYSMIS);
+
+ cc += weight;
+ cells[n_cells - 1]->count
+ = ag_func[cmd->agr].calc (cells[n_cells - 1]->count, x, weight);
+ }
+
+ if (ag_func[cmd->agr].post)
+ cells[n_cells - 1]->count
+ = ag_func[cmd->agr].post (cells[n_cells - 1]->count, cc);
+
+ ccc += cc;
+ }
+
+ casegrouper_destroy (grouper);
+
+ for (int i = 0; i < n_cells; ++i)
+ {
+ if (ag_func[cmd->agr].ppost)
+ {
+ struct freq *cell = cells[i];
+ if (cmd->n_by_vars > 1)
+ {
+ const union value *vv = &cell->values[1];
+
+ int v1_width = var_get_width (cmd->by_var[1]);
+ size_t hash = value_hash (vv, v1_width, 0);
+
+ struct freq *fcol = find_fcol (&columns, vv, hash, v1_width);
+ cell->count = ag_func[cmd->agr].ppost (cell->count, fcol->count);
+ }
+ else
+ cell->count = ag_func[cmd->agr].ppost (cell->count, ccc);
+ }
+ }
+
+ if (cmd->n_by_vars > 1)
+ {
+ struct freq *cell, *next;
+ HMAP_FOR_EACH_SAFE (cell, next, struct freq, node, &columns)
+ {
+ value_destroy (cell->values, var_get_width (cmd->by_var[1]));
+ free (cell);
+ }
+ }
+ hmap_destroy (&columns);
+
+ char *label = (cmd->n_dep_vars > 0
+ ? xasprintf (_("%s of %s"),
+ ag_func[cmd->agr].description,
+ var_get_name (cmd->dep_vars[0]))
+ : xstrdup (ag_func[cmd->agr].description));
+ chart_submit (barchart_create (cmd->by_var, cmd->n_by_vars, label, false,
+ cells, n_cells));
+ free (label);
+
+ for (int i = 0; i < n_cells; ++i)
+ free (cells[i]);
+
+ free (cells);
+}
+
+static void
+run_graph (struct graph *cmd, struct casereader *input)
+{
+ cmd->es = pool_nmalloc (cmd->pool, cmd->n_dep_vars, sizeof *cmd->es);
+ for (int v = 0; v < cmd->n_dep_vars; v++)
+ cmd->es[v] = (struct exploratory_stats) {
+ .mom = moments_create (MOMENT_KURTOSIS),
+ .cmin = DBL_MAX,
+ .maximum = -DBL_MAX,
+ .minimum = DBL_MAX,
+ };
+
+ /* Always remove cases listwise. This is correct for the histogram because
+ there is only one variable and a simple bivariate scatterplot. */
+ input = casereader_create_filter_missing (input,
+ cmd->dep_vars,
+ cmd->n_dep_vars,
+ cmd->dep_excl,
+ NULL,
+ NULL);
+
+ struct casewriter *writer = autopaging_writer_create (cmd->gr_proto);
+
+ /* The case data is copied to a new writer.
+ The setup of the case depends on the chart type.
+
+ For Scatterplot:
+ - x is assumed in dep_vars[0].
+ - y is assumed in dep_vars[1].
+
+ For Histogram:
+ - x is assumed in dep_vars[0]. */
+ assert (SP_IDX_X == 0 && SP_IDX_Y == 1 && HG_IDX_X == 0);
+
+ struct ccase *c;
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ {
+ struct ccase *outcase = case_create (cmd->gr_proto);
+ const double weight = dict_get_case_weight (cmd->dict, c, NULL);
+ if (cmd->chart_type == CT_HISTOGRAM)
+ *case_num_rw_idx (outcase, HG_IDX_WT) = weight;
+ if (cmd->chart_type == CT_SCATTERPLOT && cmd->n_by_vars > 0)
+ value_copy (case_data_rw_idx (outcase, SP_IDX_BY),
+ case_data (c, cmd->by_var[0]),
+ var_get_width (cmd->by_var[0]));
+ for (int v = 0; v < cmd->n_dep_vars; v++)
+ {
+ const struct variable *var = cmd->dep_vars[v];
+ const double x = case_num (c, var);
+
+ if (var_is_value_missing (var, case_data (c, var)) & cmd->dep_excl)
+ {
+ cmd->es[v].missing += weight;
+ continue;
+ }
+
+ /* Magically v value fits to SP_IDX_X, SP_IDX_Y, HG_IDX_X. */
+ *case_num_rw_idx (outcase, v) = x;
+
+ if (x > cmd->es[v].maximum)
+ cmd->es[v].maximum = x;
+
+ if (x < cmd->es[v].minimum)
+ cmd->es[v].minimum = x;
+
+ cmd->es[v].non_missing += weight;
+
+ moments_pass_one (cmd->es[v].mom, x, weight);
+
+ cmd->es[v].cc += weight;
+
+ if (cmd->es[v].cmin > weight)
+ cmd->es[v].cmin = weight;
+ }
+ casewriter_write (writer, outcase);
+ }
+
+ struct casereader *reader = casewriter_make_reader (writer);
+ switch (cmd->chart_type)
+ {
+ case CT_HISTOGRAM:
+ show_histogr (cmd,reader);
+ break;
+
+ case CT_SCATTERPLOT:
+ show_scatterplot (cmd,reader);
+ break;
+
+ case CT_NONE:
+ case CT_BAR:
+ case CT_LINE:
+ case CT_PIE:
+ case CT_ERRORBAR:
+ case CT_HILO:
+ case CT_PARETO:
+ NOT_REACHED ();
+ }
+
+ casereader_destroy (input);
+ cleanup_exploratory_stats (cmd);
+}
+
+int
+cmd_graph (struct lexer *lexer, struct dataset *ds)
+{
+ struct graph graph = {
+ .missing_pw = false,
+
+ .pool = pool_create (),
+
+ .dep_excl = MV_ANY,
+ .fctr_excl = MV_ANY,
+
+ .dict = dataset_dict (ds),
+
+ .chart_type = CT_NONE,
+ .scatter_type = ST_BIVARIATE,
+ .gr_proto = caseproto_create (),
+ .ordering = SUBCASE_EMPTY_INITIALIZER,
+ };
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "HISTOGRAM"))
+ {
+ if (graph.chart_type != CT_NONE)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Only one chart type is allowed."));
+ goto error;
+ }
+ graph.normal = false;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_match_phrase (lexer, "NORMAL)"))
+ goto error;
+
+ graph.normal = true;
+ }
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+ graph.chart_type = CT_HISTOGRAM;
+ int vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, graph.dict,
+ &graph.dep_vars, &graph.n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ if (graph.n_dep_vars > 1)
+ {
+ lex_ofs_error (lexer, vars_start, lex_ofs (lexer) - 1,
+ _("Only one variable is allowed."));
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "BAR"))
+ {
+ if (graph.chart_type != CT_NONE)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Only one chart type is allowed."));
+ goto error;
+ }
+ graph.chart_type = CT_BAR;
+ graph.bar_type = CBT_SIMPLE;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (lex_match_id (lexer, "SIMPLE"))
+ {
+ /* This is the default anyway */
+ }
+ else if (lex_match_id (lexer, "GROUPED"))
+ {
+ graph.bar_type = CBT_GROUPED;
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."), "GROUPED");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "STACKED"))
+ {
+ graph.bar_type = CBT_STACKED;
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."), "STACKED");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "RANGE"))
+ {
+ graph.bar_type = CBT_RANGE;
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."), "RANGE");
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "SIMPLE", "GROUPED",
+ "STACKED", "RANGE");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ if (!parse_function (lexer, &graph))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "SCATTERPLOT"))
+ {
+ if (graph.chart_type != CT_NONE)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Only one chart type is allowed."));
+ goto error;
+ }
+ graph.chart_type = CT_SCATTERPLOT;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (lex_match_id (lexer, "BIVARIATE"))
+ {
+ /* This is the default anyway */
+ }
+ else if (lex_match_id (lexer, "OVERLAY"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"OVERLAY");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "MATRIX"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"MATRIX");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "XYZ"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"XYZ");
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "BIVARIATE", "OVERLAY",
+ "MATRIX", "XYZ");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ int vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, graph.dict,
+ &graph.dep_vars, &graph.n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+
+ if (graph.scatter_type == ST_BIVARIATE && graph.n_dep_vars != 1)
+ {
+ lex_ofs_error (lexer, vars_start, lex_ofs (lexer) - 1,
+ _("Only one variable is allowed."));
+ goto error;
+ }
+
+ if (!lex_force_match (lexer, T_WITH))
+ goto error;
+
+ vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, graph.dict,
+ &graph.dep_vars, &graph.n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC | PV_APPEND))
+ goto error;
+
+ if (graph.scatter_type == ST_BIVARIATE && graph.n_dep_vars != 2)
+ {
+ lex_ofs_error (lexer, vars_start, lex_ofs (lexer) - 1,
+ _("Only one variable is allowed."));
+ goto error;
+ }
+
+ if (lex_match (lexer, T_BY))
+ {
+ const struct variable *v = NULL;
+ if (!lex_match_variable (lexer,graph.dict,&v))
+ {
+ lex_error (lexer, _("Syntax error expecting variable name."));
+ goto error;
+ }
+ graph.by_var[0] = v;
+ graph.n_by_vars = 1;
+ }
+ }
+ else if (lex_match_id (lexer, "LINE"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"LINE");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "PIE"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"PIE");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ERRORBAR"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"ERRORBAR");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "PARETO"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"PARETO");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "TITLE"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"TITLE");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "SUBTITLE"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"SUBTITLE");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "FOOTNOTE"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not yet implemented."),"FOOTNOTE");
+ goto error;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "LISTWISE"))
+ graph.missing_pw = false;
+ else if (lex_match_id (lexer, "VARIABLE"))
+ graph.missing_pw = true;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ graph.dep_excl = MV_ANY;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ graph.dep_excl = MV_SYSTEM;
+ else if (lex_match_id (lexer, "REPORT"))
+ graph.fctr_excl = 0;
+ else if (lex_match_id (lexer, "NOREPORT"))
+ graph.fctr_excl = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "LISTWISE", "VARIABLE",
+ "EXCLUDE", "INCLUDE",
+ "REPORT", "NOREPORT");
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "HISTOGRAM", "BAR", "SCATTERPLOT", "LINE",
+ "PIE", "ERRORBAR", "PARETO", "TITLE", "SUBTITLE",
+ "FOOTNOTE", "MISSING");
+ goto error;
+ }
+ }
+
+ switch (graph.chart_type)
+ {
+ case CT_SCATTERPLOT:
+ /* See scatterplot.h for the setup of the case prototype */
+
+ /* x value - SP_IDX_X*/
+ graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
+
+ /* y value - SP_IDX_Y*/
+ graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
+ /* The by_var contains the plot categories for the different xy
+ plot colors */
+ if (graph.n_by_vars > 0) /* SP_IDX_BY */
+ graph.gr_proto = caseproto_add_width (graph.gr_proto,
+ var_get_width(graph.by_var[0]));
+ break;
+
+ case CT_HISTOGRAM:
+ /* x value */
+ graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
+ /* weight value */
+ graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
+ break;
+
+ case CT_BAR:
+ break;
+
+ case CT_NONE:
+ lex_error_expecting (lexer, "HISTOGRAM", "SCATTERPLOT", "BAR");
+ goto error;
+
+ default:
+ NOT_REACHED ();
+ break;
+ }
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), graph.dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ if (graph.chart_type == CT_BAR)
+ run_barchart (&graph, group);
+ else
+ run_graph (&graph, group);
+ }
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ subcase_uninit (&graph.ordering);
+ free (graph.dep_vars);
+ pool_destroy (graph.pool);
+ caseproto_unref (graph.gr_proto);
+
+ return CMD_SUCCESS;
+
+ error:
+ subcase_uninit (&graph.ordering);
+ caseproto_unref (graph.gr_proto);
+ free (graph.dep_vars);
+ pool_destroy (graph.pool);
+
+ return CMD_FAILURE;
+}
--- /dev/null
+/* pspp - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <unistd.h>
+#if HAVE_SYS_WAIT_H
+#include <sys/wait.h>
+#endif
+
+#include "data/settings.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "libpspp/string-array.h"
+#include "libpspp/temp-file.h"
+#include "output/driver.h"
+
+#include "gl/error.h"
+#include "gl/intprops.h"
+#include "gl/localcharset.h"
+#include "gl/read-file.h"
+#include "gl/timespec.h"
+#include "gl/xalloc.h"
+#include "gl/xmalloca.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+\f
+#if !HAVE_FORK
+#define TIME_LIMIT_SUPPORTED 0
+static bool
+run_commands (const struct string_array *commands, double time_limit)
+{
+ assert (time_limit == DBL_MAX);
+
+ for (size_t i = 0; i < commands->n; i++)
+ {
+ /* XXX No way to capture command output */
+ char *s = recode_string (locale_charset (), "UTF-8",
+ commands->strings[i], -1);
+ int retval = system (s);
+ free (s);
+
+ if (retval)
+ {
+ msg (SE, _("%s: Command exited with status %d."),
+ commands->strings[i], retval);
+ return false;
+ }
+ }
+ return true;
+}
+#else
+#define TIME_LIMIT_SUPPORTED 1
+static bool
+run_command (const char *command, struct timespec timeout)
+{
+ /* Same exit codes used by 'sh'. */
+ enum {
+ EXIT_CANNOT_INVOKE = 126,
+ EXIT_ENOENT = 127,
+ };
+
+ /* Create a temporary file to capture command output. */
+ FILE *output_file = create_temp_file ();
+ if (!output_file)
+ {
+ msg (SE, _("Failed to create temporary file (%s)."), strerror (errno));
+ return false;
+ }
+
+ int dev_null_fd = open ("/dev/null", O_RDONLY);
+ if (dev_null_fd < 0)
+ {
+ msg (SE, _("/dev/null: Failed to open (%s)."), strerror (errno));
+ fclose (output_file);
+ return false;
+ }
+
+ char *locale_command = recode_string (locale_charset (), "UTF-8",
+ command, -1);
+
+ pid_t pid = fork ();
+ if (pid < 0)
+ {
+ close (dev_null_fd);
+ fclose (output_file);
+ free (locale_command);
+
+ msg (SE, _("Couldn't fork: %s."), strerror (errno));
+ return false;
+ }
+ else if (!pid)
+ {
+ /* Running in the child. */
+
+#if __GNU__
+ /* Hurd doesn't support inheriting process timers in a way that works. */
+ if (setpgid (0, 0) < 0)
+ error (1, errno, _("Failed to set process group."));
+#else
+ /* Set up timeout. */
+ if (timeout.tv_sec < TYPE_MAXIMUM (time_t))
+ {
+ signal (SIGALRM, SIG_DFL);
+
+ struct timespec left = timespec_sub (timeout, current_timespec ());
+ if (timespec_sign (left) <= 0)
+ raise (SIGALRM);
+
+ struct itimerval it = {
+ .it_value = {
+ .tv_sec = left.tv_sec,
+ .tv_usec = left.tv_nsec / 1000
+ }
+ };
+ if (setitimer (ITIMER_REAL, &it, NULL) < 0)
+ error (1, errno, _("Failed to set timeout."));
+ }
+#endif
+
+ /* Set up file descriptors:
+ - /dev/null for stdin
+ - Temporary file to capture stdout and stderr.
+ - Close everything else.
+ */
+ dup2 (dev_null_fd, 0);
+ dup2 (fileno (output_file), 1);
+ dup2 (fileno (output_file), 2);
+ close (dev_null_fd);
+ for (int fd = 3; fd < 256; fd++)
+ close (fd);
+
+ /* Choose the shell. */
+ const char *shell = getenv ("SHELL");
+ if (shell == NULL)
+ shell = "/bin/sh";
+
+ /* Run subprocess. */
+ execl (shell, shell, "-c", locale_command, NULL);
+
+ /* Failed to start the shell. */
+ _exit (errno == ENOENT ? EXIT_ENOENT : EXIT_CANNOT_INVOKE);
+ }
+
+ /* Running in the parent. */
+ close (dev_null_fd);
+ free (locale_command);
+
+ /* Wait for child to exit. */
+ int status = 0;
+ int error = 0;
+ for (;;)
+ {
+#if __GNU__
+ if (timespec_cmp (current_timespec (), timeout) >= 0)
+ kill (-pid, SIGALRM);
+
+ int flags = WNOHANG;
+#else
+ int flags = 0;
+#endif
+ pid_t retval = waitpid (pid, &status, flags);
+ if (retval == pid)
+ break;
+ else if (retval < 0)
+ {
+ if (errno != EINTR)
+ {
+ error = errno;
+ break;
+ }
+ }
+#if __GNU__
+ else if (retval == 0)
+ sleep (1);
+#endif
+ else
+ NOT_REACHED ();
+ }
+
+ bool ok = true;
+ if (error)
+ {
+ msg (SW, _("While running \"%s\", waiting for child process "
+ "failed (%s)."),
+ command, strerror (errno));
+ ok = false;
+ }
+
+ if (WIFSIGNALED (status))
+ {
+ int signum = WTERMSIG (status);
+ if (signum == SIGALRM)
+ msg (SW, _("Command \"%s\" timed out."), command);
+ else
+ msg (SW, _("Command \"%s\" terminated by signal %d."), command, signum);
+ ok = false;
+ }
+ else if (WIFEXITED (status) && WEXITSTATUS (status))
+ {
+ int exit_code = WEXITSTATUS (status);
+ const char *detail = (exit_code == EXIT_ENOENT
+ ? _("Command or shell not found")
+ : exit_code == EXIT_CANNOT_INVOKE
+ ? _("Could not invoke command or shell")
+ : NULL);
+ if (detail)
+ msg (SW, _("Command \"%s\" exited with status %d (%s)."),
+ command, exit_code, detail);
+ else
+ msg (SW, _("Command \"%s\" exited with status %d."),
+ command, exit_code);
+ ok = false;
+ }
+
+ rewind (output_file);
+ size_t length;
+ char *locale_output = fread_file (output_file, 0, &length);
+ if (!locale_output)
+ {
+ msg (SW, _("Command \"%s\" output could not be read (%s)."),
+ command, strerror (errno));
+ ok = false;
+ }
+ else if (length > 0)
+ {
+ char *output = recode_string ("UTF-8", locale_charset (),
+ locale_output, -1);
+
+ /* Drop final new-line, if any. */
+ char *end = strchr (output, '\0');
+ if (end > output && end[-1] == '\n')
+ end[-1] = '\0';
+
+ output_log_nocopy (output);
+ }
+ free (locale_output);
+
+ return ok;
+}
+
+static bool
+run_commands (const struct string_array *commands, double time_limit)
+{
+ struct timespec timeout = timespec_add (dtotimespec (time_limit),
+ current_timespec ());
+
+ for (size_t i = 0; i < commands->n; i++)
+ {
+ if (!run_command (commands->strings[i], timeout))
+ return false;
+ }
+
+ return true;
+}
+#endif
+
+int
+cmd_host (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ if (settings_get_safer_mode ())
+ {
+ lex_next_error (lexer, -1, -1,
+ _("This command not allowed when the %s option is set."),
+ "SAFER");
+ return CMD_FAILURE;
+ }
+
+ if (!lex_force_match_phrase (lexer, "COMMAND=[")
+ || !lex_force_string (lexer))
+ return CMD_FAILURE;
+
+ struct string_array commands = STRING_ARRAY_INITIALIZER;
+ while (lex_token (lexer) == T_STRING)
+ {
+ string_array_append (&commands, lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ if (!lex_force_match (lexer, T_RBRACK))
+ {
+ string_array_destroy (&commands);
+ return CMD_FAILURE;
+ }
+
+ double time_limit = DBL_MAX;
+ if (lex_match_id (lexer, "TIMELIMIT"))
+ {
+ int time_limit_start = lex_ofs (lexer) - 1;
+ if (!lex_force_match (lexer, T_EQUALS)
+ || !lex_force_num (lexer))
+ {
+ string_array_destroy (&commands);
+ return CMD_FAILURE;
+ }
+
+ double num = lex_number (lexer);
+ lex_get (lexer);
+ time_limit = num < 0.0 ? 0.0 : num;
+
+ int time_limit_end = lex_ofs (lexer) - 1;
+ if (!TIME_LIMIT_SUPPORTED)
+ {
+ lex_ofs_error (lexer, time_limit_start, time_limit_end,
+ _("Time limit not supported on this platform."));
+ string_array_destroy (&commands);
+ return false;
+ }
+ }
+
+ enum cmd_result result = lex_end_of_command (lexer);
+ if (result == CMD_SUCCESS && !run_commands (&commands, time_limit))
+ result = CMD_FAILURE;
+ string_array_destroy (&commands);
+ return result;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2007, 2010, 2011, 2020 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "data/dataset.h"
+#include "data/session.h"
+#include "language/command.h"
+#include "language/lexer/include-path.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/dirname.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+enum variant
+ {
+ INSERT,
+ INCLUDE
+ };
+
+static int
+do_insert (struct lexer *lexer, struct dataset *ds, enum variant variant)
+{
+ /* Skip optional FILE=. */
+ if (lex_match_id (lexer, "FILE"))
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_string_or_id (lexer))
+ return CMD_FAILURE;
+
+ char *relative_name = utf8_to_filename (lex_tokcstr (lexer));
+ char *filename = include_path_search (relative_name);
+ free (relative_name);
+
+ if (!filename)
+ {
+ msg (SE, _("Can't find `%s' in include file search path."),
+ lex_tokcstr (lexer));
+ return CMD_FAILURE;
+ }
+ lex_get (lexer);
+
+ enum segmenter_mode syntax_mode = SEG_MODE_INTERACTIVE;
+ enum lex_error_mode error_mode = LEX_ERROR_CONTINUE;
+ bool cd = false;
+ int status = CMD_FAILURE;
+ char *encoding = xstrdup (session_get_default_syntax_encoding (
+ dataset_session (ds)));
+ while (T_ENDCMD != lex_token (lexer))
+ {
+ if (lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto exit;
+
+ free (encoding);
+ encoding = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ else if (variant == INSERT && lex_match_id (lexer, "SYNTAX"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "INTERACTIVE"))
+ syntax_mode = SEG_MODE_INTERACTIVE;
+ else if (lex_match_id (lexer, "BATCH"))
+ syntax_mode = SEG_MODE_BATCH;
+ else if (lex_match_id (lexer, "AUTO"))
+ syntax_mode = SEG_MODE_AUTO;
+ else
+ {
+ lex_error_expecting (lexer, "BATCH", "INTERACTIVE", "AUTO");
+ goto exit;
+ }
+ }
+ else if (variant == INSERT && lex_match_id (lexer, "CD"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "YES"))
+ cd = true;
+ else if (lex_match_id (lexer, "NO"))
+ cd = false;
+ else
+ {
+ lex_error_expecting (lexer, "YES", "NO");
+ goto exit;
+ }
+ }
+ else if (variant == INSERT && lex_match_id (lexer, "ERROR"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "CONTINUE"))
+ error_mode = LEX_ERROR_CONTINUE;
+ else if (lex_match_id (lexer, "STOP"))
+ error_mode = LEX_ERROR_STOP;
+ else if (settings_get_testing_mode ()
+ && lex_match_id (lexer, "IGNORE"))
+ error_mode = LEX_ERROR_IGNORE;
+ else
+ {
+ lex_error_expecting (lexer, "CONTINUE", "STOP");
+ goto exit;
+ }
+ }
+ else
+ {
+ if (variant == INSERT)
+ lex_error_expecting (lexer, "ENCODING", "SYNTAX", "CD", "ERROR");
+ else
+ lex_error_expecting (lexer, "ENCODING");
+ goto exit;
+ }
+ }
+ status = lex_end_of_command (lexer);
+
+ if (status == CMD_SUCCESS)
+ {
+ struct lex_reader *reader = lex_reader_for_file (filename, encoding,
+ syntax_mode, error_mode);
+ if (reader != NULL)
+ {
+ lex_discard_rest_of_command (lexer);
+ lex_include (lexer, reader);
+
+ if (cd)
+ {
+ char *directory = dir_name (filename);
+ if (chdir (directory))
+ {
+ int err = errno;
+ msg (SE, _("Cannot change directory to %s: %s"), directory,
+ strerror (err));
+ status = CMD_FAILURE;
+ }
+
+ free (directory);
+ }
+ }
+ }
+
+exit:
+ free (encoding);
+ free (filename);
+ return status;
+}
+
+int
+cmd_include (struct lexer *lexer, struct dataset *ds)
+{
+ return do_insert (lexer, ds, INCLUDE);
+}
+
+int
+cmd_insert (struct lexer *lexer, struct dataset *ds)
+{
+ return do_insert (lexer, ds, INSERT);
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/caseinit.h"
+#include "data/casereader-provider.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/session.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/data-reader.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/inpt-pgm.h"
+#include "language/expressions/public.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Indicates how a `union value' should be initialized. */
+struct input_program_pgm
+ {
+ struct session *session;
+ struct dataset *ds;
+
+ struct trns_chain xforms;
+ size_t idx;
+ bool eof;
+
+ casenumber case_nr; /* Incremented by END CASE transformation. */
+
+ struct caseinit *init;
+ struct caseproto *proto;
+ };
+
+static void destroy_input_program (struct input_program_pgm *);
+static const struct trns_class end_case_trns_class;
+static const struct trns_class reread_trns_class;
+static const struct trns_class end_file_trns_class;
+
+static const struct casereader_class input_program_casereader_class;
+
+static bool inside_input_program;
+static bool saw_END_CASE;
+static bool saw_END_FILE;
+static bool saw_DATA_LIST;
+
+/* Returns true if we're parsing the inside of a INPUT
+ PROGRAM...END INPUT PROGRAM construct, false otherwise. */
+bool
+in_input_program (void)
+{
+ return inside_input_program;
+}
+
+void
+data_list_seen (void)
+{
+ saw_DATA_LIST = true;
+}
+
+/* Emits an END CASE transformation for INP. */
+static void
+emit_END_CASE (struct dataset *ds)
+{
+ add_transformation (ds, &end_case_trns_class, xzalloc (sizeof (bool)));
+}
+
+int
+cmd_input_program (struct lexer *lexer, struct dataset *ds)
+{
+ struct msg_location *location = lex_ofs_location (lexer, 0, 1);
+ if (!lex_match (lexer, T_ENDCMD))
+ {
+ msg_location_destroy (location);
+ return lex_end_of_command (lexer);
+ }
+
+ struct session *session = session_create (dataset_session (ds));
+ struct dataset *inp_ds = dataset_create (session, "INPUT PROGRAM");
+
+ struct input_program_pgm *inp = xmalloc (sizeof *inp);
+ *inp = (struct input_program_pgm) { .session = session, .ds = inp_ds };
+
+ proc_push_transformations (inp->ds);
+ inside_input_program = true;
+ saw_END_CASE = saw_END_FILE = saw_DATA_LIST = false;
+ while (!lex_match_phrase (lexer, "END INPUT PROGRAM"))
+ {
+ enum cmd_result result;
+
+ result = cmd_parse_in_state (lexer, inp->ds, CMD_STATE_INPUT_PROGRAM);
+ if (result == CMD_EOF
+ || result == CMD_FINISH
+ || result == CMD_CASCADING_FAILURE)
+ {
+ proc_pop_transformations (inp->ds, &inp->xforms);
+
+ if (result == CMD_EOF)
+ msg (SE, _("Unexpected end-of-file within %s."), "INPUT PROGRAM");
+ inside_input_program = false;
+ destroy_input_program (inp);
+ msg_location_destroy (location);
+ return result;
+ }
+ }
+ if (!saw_END_CASE)
+ emit_END_CASE (inp->ds);
+ inside_input_program = false;
+ proc_pop_transformations (inp->ds, &inp->xforms);
+
+ struct msg_location *end = lex_ofs_location (lexer, 0, 2);
+ msg_location_merge (&location, end);
+ location->omit_underlines = true;
+ msg_location_destroy (end);
+
+ if (!saw_DATA_LIST && !saw_END_FILE)
+ {
+ msg_at (SE, location, _("Input program does not contain %s or %s."),
+ "DATA LIST", "END FILE");
+ destroy_input_program (inp);
+ msg_location_destroy (location);
+ return CMD_FAILURE;
+ }
+ if (dict_get_next_value_idx (dataset_dict (inp->ds)) == 0)
+ {
+ msg_at (SE, location, _("Input program did not create any variables."));
+ destroy_input_program (inp);
+ msg_location_destroy (location);
+ return CMD_FAILURE;
+ }
+ msg_location_destroy (location);
+
+ /* Figure out how to initialize each input case. */
+ inp->init = caseinit_create ();
+ caseinit_mark_for_init (inp->init, dataset_dict (inp->ds));
+ inp->proto = caseproto_ref (dict_get_proto (dataset_dict (inp->ds)));
+
+ dataset_set_dict (ds, dict_clone (dataset_dict (inp->ds)));
+ dataset_set_source (
+ ds, casereader_create_sequential (NULL, inp->proto, CASENUMBER_MAX,
+ &input_program_casereader_class, inp));
+
+ return CMD_SUCCESS;
+}
+
+/* Reads and returns one case.
+ Returns the case if successful, null at end of file or if an
+ I/O error occurred. */
+static struct ccase *
+input_program_casereader_read (struct casereader *reader UNUSED, void *inp_)
+{
+ struct input_program_pgm *inp = inp_;
+
+ if (inp->eof || !inp->xforms.n)
+ return NULL;
+
+ struct ccase *c = case_create (inp->proto);
+ caseinit_init_vars (inp->init, c);
+
+ for (size_t i = inp->idx < inp->xforms.n ? inp->idx : 0; ; i++)
+ {
+ if (i >= inp->xforms.n)
+ {
+ i = 0;
+ c = case_unshare (c);
+ caseinit_update_left_vars (inp->init, c);
+ caseinit_init_vars (inp->init, c);
+ }
+
+ const struct transformation *trns = &inp->xforms.xforms[i];
+ switch (trns->class->execute (trns->aux, &c, inp->case_nr))
+ {
+ case TRNS_END_CASE:
+ inp->case_nr++;
+ inp->idx = i;
+ return c;
+
+ case TRNS_ERROR:
+ casereader_force_error (reader);
+ /* Fall through. */
+ case TRNS_END_FILE:
+ inp->eof = true;
+ case_unref (c);
+ return NULL;
+
+ case TRNS_CONTINUE:
+ break;
+
+ default:
+ NOT_REACHED ();
+ }
+ }
+}
+
+static void
+destroy_input_program (struct input_program_pgm *pgm)
+{
+ if (pgm != NULL)
+ {
+ session_destroy (pgm->session);
+ trns_chain_uninit (&pgm->xforms);
+ caseinit_destroy (pgm->init);
+ caseproto_unref (pgm->proto);
+ free (pgm);
+ }
+}
+
+/* Destroys the casereader. */
+static void
+input_program_casereader_destroy (struct casereader *reader UNUSED, void *inp_)
+{
+ struct input_program_pgm *inp = inp_;
+ destroy_input_program (inp);
+}
+
+static const struct casereader_class input_program_casereader_class =
+ {
+ input_program_casereader_read,
+ input_program_casereader_destroy,
+ NULL,
+ NULL,
+ };
+\f
+int
+cmd_end_case (struct lexer *lexer UNUSED, struct dataset *ds)
+{
+ assert (in_input_program ());
+ emit_END_CASE (ds);
+ saw_END_CASE = true;
+ return CMD_SUCCESS;
+}
+
+/* Outputs the current case */
+static enum trns_result
+end_case_trns_proc (void *resume_, struct ccase **c UNUSED,
+ casenumber case_nr UNUSED)
+{
+ bool *resume = resume_;
+ enum trns_result retval = *resume ? TRNS_CONTINUE : TRNS_END_CASE;
+ *resume = !*resume;
+ return retval;
+}
+
+static bool
+end_case_trns_free (void *resume)
+{
+ free (resume);
+ return true;
+}
+
+static const struct trns_class end_case_trns_class = {
+ .name = "END CASE",
+ .execute = end_case_trns_proc,
+ .destroy = end_case_trns_free,
+};
+
+/* REREAD transformation. */
+struct reread_trns
+ {
+ struct dfm_reader *reader; /* File to move file pointer back on. */
+ struct expression *column; /* Column to reset file pointer to. */
+ };
+
+/* Parses REREAD command. */
+int
+cmd_reread (struct lexer *lexer, struct dataset *ds)
+{
+ char *encoding = NULL;
+ struct file_handle *fh = fh_get_default_handle ();
+ struct expression *e = NULL;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "COLUMN"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (e)
+ {
+ lex_sbc_only_once (lexer, "COLUMN");
+ goto error;
+ }
+
+ e = expr_parse (lexer, ds, VAL_NUMERIC);
+ if (!e)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "FILE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ fh_unref (fh);
+ fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
+ if (fh == NULL)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "COLUMN", "FILE", "ENCODING");
+ goto error;
+ }
+ }
+
+ struct reread_trns *t = xmalloc (sizeof *t);
+ *t = (struct reread_trns) {
+ .reader = dfm_open_reader (fh, lexer, encoding),
+ .column = e,
+ };
+ add_transformation (ds, &reread_trns_class, t);
+
+ fh_unref (fh);
+ free (encoding);
+ return CMD_SUCCESS;
+
+error:
+ expr_free (e);
+ free (encoding);
+ return CMD_CASCADING_FAILURE;
+}
+
+/* Executes a REREAD transformation. */
+static enum trns_result
+reread_trns_proc (void *t_, struct ccase **c, casenumber case_num)
+{
+ struct reread_trns *t = t_;
+
+ if (t->column == NULL)
+ dfm_reread_record (t->reader, 1);
+ else
+ {
+ double column = expr_evaluate_num (t->column, *c, case_num);
+ if (!isfinite (column) || column < 1)
+ {
+ msg (SE, _("REREAD: Column numbers must be positive finite "
+ "numbers. Column set to 1."));
+ dfm_reread_record (t->reader, 1);
+ }
+ else
+ dfm_reread_record (t->reader, column);
+ }
+ return TRNS_CONTINUE;
+}
+
+/* Frees a REREAD transformation.
+ Returns true if successful, false if an I/O error occurred. */
+static bool
+reread_trns_free (void *t_)
+{
+ struct reread_trns *t = t_;
+ expr_free (t->column);
+ dfm_close_reader (t->reader);
+ return true;
+}
+
+static const struct trns_class reread_trns_class = {
+ .name = "REREAD",
+ .execute = reread_trns_proc,
+ .destroy = reread_trns_free,
+};
+
+/* Parses END FILE command. */
+int
+cmd_end_file (struct lexer *lexer UNUSED, struct dataset *ds)
+{
+ assert (in_input_program ());
+
+ add_transformation (ds, &end_file_trns_class, NULL);
+ saw_END_FILE = true;
+
+ return CMD_SUCCESS;
+}
+
+/* Executes an END FILE transformation. */
+static enum trns_result
+end_file_trns_proc (void *trns_ UNUSED, struct ccase **c UNUSED,
+ casenumber case_num UNUSED)
+{
+ return TRNS_END_FILE;
+}
+
+static const struct trns_class end_file_trns_class = {
+ .name = "END FILE",
+ .execute = end_file_trns_proc,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef INPT_PGM_H
+#define INPT_PGM_H 1
+
+#include <stdbool.h>
+
+bool in_input_program (void);
+
+void data_list_seen (void);
+
+#endif /* inpt-pgm.h */
--- /dev/null
+/* Pspp - a program for statistical analysis.
+ Copyright (C) 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+
+#include <config.h>
+
+#include "jonckheere-terpstra.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "libpspp/assertion.h"
+#include "libpspp/hmap.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/sort.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+/* Returns true iff the independent variable lies in the
+ between val1 and val2. Regardless of which is the greater value.
+*/
+static bool
+include_func_bi (const struct ccase *c, void *aux)
+{
+ const struct n_sample_test *nst = aux;
+ const union value *bigger = NULL;
+ const union value *smaller = NULL;
+
+ if (0 > value_compare_3way (&nst->val1, &nst->val2, var_get_width (nst->indep_var)))
+ {
+ bigger = &nst->val2;
+ smaller = &nst->val1;
+ }
+ else
+ {
+ smaller = &nst->val2;
+ bigger = &nst->val1;
+ }
+
+ if (0 < value_compare_3way (smaller, case_data (c, nst->indep_var), var_get_width (nst->indep_var)))
+ return false;
+
+ if (0 > value_compare_3way (bigger, case_data (c, nst->indep_var), var_get_width (nst->indep_var)))
+ return false;
+
+ return true;
+}
+
+struct group_data
+{
+ /* The total of the caseweights in the group */
+ double cc;
+
+ /* A casereader containing the group data.
+ This casereader contains just two values:
+ 0: The raw value of the data
+ 1: The cumulative caseweight
+ */
+ struct casereader *reader;
+};
+
+
+static double
+u (const struct group_data *grp0, const struct group_data *grp1)
+{
+ struct ccase *c0;
+
+ struct casereader *r0 = casereader_clone (grp0->reader);
+ double usum = 0;
+ double prev_cc0 = 0.0;
+ for (; (c0 = casereader_read (r0)); case_unref (c0))
+ {
+ struct ccase *c1;
+ struct casereader *r1 = casereader_clone (grp1->reader);
+ double x0 = case_num_idx (c0, 0);
+ double cc0 = case_num_idx (c0, 1);
+ double w0 = cc0 - prev_cc0;
+
+ double prev_cc1 = 0;
+
+ for (; (c1 = casereader_read (r1)); case_unref (c1))
+ {
+ double x1 = case_num_idx (c1, 0);
+ double cc1 = case_num_idx (c1, 1);
+
+ if (x0 > x1)
+ {
+ /* Do nothing */
+ }
+ else if (x0 < x1)
+ {
+ usum += w0 * (grp1->cc - prev_cc1);
+ case_unref (c1);
+ break;
+ }
+ else
+ {
+#if 1
+ usum += w0 * ((grp1->cc - prev_cc1) / 2.0);
+#else
+ usum += w0 * (grp1->cc - (prev_cc1 + cc1) / 2.0);
+#endif
+ case_unref (c1);
+ break;
+ }
+
+ prev_cc1 = cc1;
+ }
+ casereader_destroy (r1);
+ prev_cc0 = cc0;
+ }
+ casereader_destroy (r0);
+
+ return usum;
+}
+
+
+typedef double func_f (double e_l);
+
+/*
+ These 3 functions are used repeatedly in the calculation of the
+ variance of the JT statistic.
+ Having them explicitly defined makes the variance calculation
+ a lot simpler.
+*/
+static double
+ff1 (double e)
+{
+ return e * (e - 1) * (2*e + 5);
+}
+
+static double
+ff2 (double e)
+{
+ return e * (e - 1) * (e - 2);
+}
+
+static double
+ff3 (double e)
+{
+ return e * (e - 1) ;
+}
+
+static func_f *mff[3] =
+ {
+ ff1, ff2, ff3
+ };
+
+
+/*
+ This function does the following:
+ It creates an ordered set of *distinct* values from IR.
+ For each case in that set, it calls f[0..N] passing it the caseweight.
+ It returns the sum of f[j] in result[j].
+
+ result and f must be allocated prior to calling this function.
+ */
+static
+void variance_calculation (struct casereader *ir, const struct variable *var,
+ const struct dictionary *dict,
+ func_f **f, double *result, size_t n)
+{
+ int i;
+ struct casereader *r = casereader_clone (ir);
+ struct ccase *c;
+ const struct variable *wv = dict_get_weight (dict);
+ const int w_idx = wv ?
+ var_get_case_index (wv) :
+ caseproto_get_n_widths (casereader_get_proto (r)) ;
+
+ r = sort_execute_1var (r, var);
+
+ r = casereader_create_distinct (r, var, dict_get_weight (dict));
+
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = case_num_idx (c, w_idx);
+
+ for (i = 0; i < n; ++i)
+ result[i] += f[i] (w);
+ }
+
+ casereader_destroy (r);
+}
+
+struct jt
+{
+ int levels;
+ double n;
+ double obs;
+ double mean;
+ double stddev;
+};
+
+static void show_jt (const struct n_sample_test *, const struct jt *,
+ const struct fmt_spec *wfmt);
+
+
+void
+jonckheere_terpstra_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ int v;
+ bool warn = true;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test, parent);
+
+ struct caseproto *proto = caseproto_create ();
+ proto = caseproto_add_width (proto, 0);
+ proto = caseproto_add_width (proto, 0);
+
+ /* If the independent variable is missing, then we ignore the case */
+ input = casereader_create_filter_missing (input,
+ &nst->indep_var, 1,
+ exclude,
+ NULL, NULL);
+
+ /* Remove cases with invalid weigths */
+ input = casereader_create_filter_weight (input, dict, &warn, NULL);
+
+ /* Remove all those cases which are outside the range (val1, val2) */
+ input = casereader_create_filter_func (input, include_func_bi, NULL,
+ CONST_CAST (struct n_sample_test *, nst), NULL);
+
+ /* Sort the data by the independent variable */
+ input = sort_execute_1var (input, nst->indep_var);
+
+ for (v = 0; v < nst->n_vars ; ++v)
+ {
+ struct jt jt;
+ double variance;
+ int g0;
+ double nn = 0;
+ int i;
+ double sums[3] = {0,0,0};
+ double e_sum[3] = {0,0,0};
+
+ struct group_data *grp = NULL;
+ double ccsq_sum = 0;
+
+ struct casegrouper *grouper;
+ struct casereader *group;
+ struct casereader *vreader= casereader_clone (input);
+
+ /* Get a few values into e_sum - we'll be needing these later */
+ variance_calculation (vreader, nst->vars[v], dict, mff, e_sum, 3);
+
+ grouper =
+ casegrouper_create_vars (vreader, &nst->indep_var, 1);
+
+ jt.obs = 0;
+ jt.levels = 0;
+ jt.n = 0;
+ for (; casegrouper_get_next_group (grouper, &group);
+ casereader_destroy (group))
+ {
+ struct casewriter *writer = autopaging_writer_create (proto);
+ struct ccase *c;
+ double cc = 0;
+
+ group = sort_execute_1var (group, nst->vars[v]);
+ for (; (c = casereader_read (group)); case_unref (c))
+ {
+ struct ccase *c_out = case_create (proto);
+
+ *case_num_rw_idx (c_out, 0) = case_num (c, nst->vars[v]);
+
+ cc += dict_get_case_weight (dict, c, &warn);
+ *case_num_rw_idx (c_out, 1) = cc;
+ casewriter_write (writer, c_out);
+ }
+
+ grp = xrealloc (grp, sizeof *grp * (jt.levels + 1));
+
+ grp[jt.levels].reader = casewriter_make_reader (writer);
+ grp[jt.levels].cc = cc;
+
+ jt.levels++;
+ jt.n += cc;
+ ccsq_sum += pow2 (cc);
+ }
+
+ casegrouper_destroy (grouper);
+
+ for (g0 = 0; g0 < jt.levels; ++g0)
+ {
+ int g1;
+ for (g1 = g0 +1 ; g1 < jt.levels; ++g1)
+ {
+ double uu = u (&grp[g0], &grp[g1]);
+ jt.obs += uu;
+ }
+ nn += pow2 (grp[g0].cc) * (2 * grp[g0].cc + 3);
+
+ for (i = 0; i < 3; ++i)
+ sums[i] += mff[i] (grp[g0].cc);
+
+ casereader_destroy (grp[g0].reader);
+ }
+
+ free (grp);
+
+ variance = (mff[0](jt.n) - sums[0] - e_sum[0]) / 72.0;
+ variance += sums[1] * e_sum[1] / (36.0 * mff[1] (jt.n));
+ variance += sums[2] * e_sum[2] / (8.0 * mff[2] (jt.n));
+
+ jt.stddev = sqrt (variance);
+
+ jt.mean = (pow2 (jt.n) - ccsq_sum) / 4.0;
+
+ show_jt (nst, &jt, dict_get_weight_format (dict));
+ }
+
+ casereader_destroy (input);
+ caseproto_unref (proto);
+}
+\f
+static void
+show_jt (const struct n_sample_test *nst, const struct jt *jt,
+ const struct fmt_spec *wfmt)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Jonckheere-Terpstra Test"));
+ pivot_table_set_weight_format (table, wfmt);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ pivot_category_create_leaf_rc (
+ statistics->root,
+ pivot_value_new_text_format (N_("Number of levels in %s"),
+ var_to_string (nst->indep_var)),
+ PIVOT_RC_INTEGER);
+ pivot_category_create_leaves (
+ statistics->root,
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Observed J-T Statistic"), PIVOT_RC_OTHER,
+ N_("Mean J-T Statistic"), PIVOT_RC_OTHER,
+ N_("Std. Deviation of J-T Statistic"), PIVOT_RC_OTHER,
+ N_("Std. J-T Statistic"), PIVOT_RC_OTHER,
+ N_("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t i = 0; i < nst->n_vars; ++i)
+ {
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (nst->vars[i]));
+
+ double std_jt = (jt[0].obs - jt[0].mean) / jt[0].stddev;
+ double sig = (2.0 * (std_jt > 0
+ ? gsl_cdf_ugaussian_Q (std_jt)
+ : gsl_cdf_ugaussian_P (std_jt)));
+ double entries[] = {
+ jt[0].levels,
+ jt[0].n,
+ jt[0].obs,
+ jt[0].mean,
+ jt[0].stddev,
+ std_jt,
+ sig,
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, row, pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !jonckheere_terpstra_h
+#define jonckheere_terpstra_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "data/case.h"
+#include "language/commands/npar.h"
+
+struct jonckheere_terpstra_test
+{
+ struct two_sample_test parent;
+};
+
+struct casereader;
+struct dataset;
+
+void jonckheere_terpstra_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer
+ );
+
+#endif
--- /dev/null
+/* Pspp - a program for statistical analysis.
+ Copyright (C) 2010, 2011, 2022 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+
+#include <config.h>
+
+#include "kruskal-wallis.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "libpspp/assertion.h"
+#include "libpspp/hmap.h"
+#include "libpspp/bt.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/sort.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+/* Returns true iff the independent variable lies between nst->val1 and nst->val2 */
+static bool
+include_func (const struct ccase *c, void *aux)
+{
+ const struct n_sample_test *nst = aux;
+
+ const union value *smaller = 0;
+ const union value *larger = 0;
+ int x = value_compare_3way (&nst->val1, &nst->val2, var_get_width (nst->indep_var));
+ if (x < 0)
+ {
+ smaller = &nst->val1;
+ larger = &nst->val2;
+ }
+ else
+ {
+ smaller = &nst->val2;
+ larger = &nst->val1;
+ }
+
+ if (0 < value_compare_3way (smaller, case_data (c, nst->indep_var),
+ var_get_width (nst->indep_var)))
+ return false;
+
+ if (0 > value_compare_3way (larger, case_data (c, nst->indep_var),
+ var_get_width (nst->indep_var)))
+ return false;
+
+ return true;
+}
+
+
+struct rank_entry
+{
+ struct hmap_node node;
+ struct bt_node btn;
+ union value group;
+
+ double sum_of_ranks;
+ double n;
+};
+
+
+static int
+compare_rank_entries_3way (const struct bt_node *a,
+ const struct bt_node *b,
+ const void *aux)
+{
+ const struct variable *var = aux;
+ const struct rank_entry *rea = BT_DATA (a, struct rank_entry, btn);
+ const struct rank_entry *reb = BT_DATA (b, struct rank_entry, btn);
+
+ return value_compare_3way (&rea->group, &reb->group, var_get_width (var));
+}
+
+
+/* Return the entry with the key GROUP or null if there is no such entry */
+static struct rank_entry *
+find_rank_entry (const struct hmap *map, const union value *group, size_t width)
+{
+ struct rank_entry *re = NULL;
+ size_t hash = value_hash (group, width, 0);
+
+ HMAP_FOR_EACH_WITH_HASH (re, struct rank_entry, node, hash, map)
+ {
+ if (0 == value_compare_3way (group, &re->group, width))
+ return re;
+ }
+
+ return re;
+}
+
+/* Calculates the adjustment necessary for tie compensation */
+static void
+distinct_callback (double v UNUSED, casenumber t, double w UNUSED, void *aux)
+{
+ double *tiebreaker = aux;
+
+ *tiebreaker += pow3 (t) - t;
+}
+
+
+struct kw
+{
+ struct hmap map;
+ double h;
+};
+
+static void show_ranks_box (const struct n_sample_test *, const struct kw *);
+static void show_sig_box (const struct n_sample_test *, const struct kw *);
+
+void
+kruskal_wallis_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ int i;
+ struct ccase *c;
+ bool warn = true;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test, parent);
+ const struct caseproto *proto ;
+ size_t rank_idx ;
+
+ int total_n_groups = 0.0;
+
+ struct kw *kw = XCALLOC (nst->n_vars, struct kw);
+
+ /* If the independent variable is missing, then we ignore the case */
+ input = casereader_create_filter_missing (input,
+ &nst->indep_var, 1,
+ exclude,
+ NULL, NULL);
+
+ input = casereader_create_filter_weight (input, dict, &warn, NULL);
+
+ /* Remove all those cases which are outside the range (val1, val2) */
+ input = casereader_create_filter_func (input, include_func, NULL,
+ CONST_CAST (struct n_sample_test *, nst), NULL);
+
+ proto = casereader_get_proto (input);
+ rank_idx = caseproto_get_n_widths (proto);
+
+ /* Rank cases by the v value */
+ for (i = 0; i < nst->n_vars; ++i)
+ {
+ double tiebreaker = 0.0;
+ bool warn = true;
+ enum rank_error rerr = 0;
+ struct casereader *rr;
+ struct casereader *r = casereader_clone (input);
+
+ r = sort_execute_1var (r, nst->vars[i]);
+
+ /* Ignore missings in the test variable */
+ r = casereader_create_filter_missing (r, &nst->vars[i], 1,
+ exclude,
+ NULL, NULL);
+
+ rr = casereader_create_append_rank (r,
+ nst->vars[i],
+ dict_get_weight (dict),
+ &rerr,
+ distinct_callback, &tiebreaker);
+
+ hmap_init (&kw[i].map);
+ for (; (c = casereader_read (rr)); case_unref (c))
+ {
+ const union value *group = case_data (c, nst->indep_var);
+ const size_t group_var_width = var_get_width (nst->indep_var);
+ struct rank_entry *rank = find_rank_entry (&kw[i].map, group, group_var_width);
+
+ if (NULL == rank)
+ {
+ rank = xzalloc (sizeof *rank);
+ value_clone (&rank->group, group, group_var_width);
+
+ hmap_insert (&kw[i].map, &rank->node,
+ value_hash (&rank->group, group_var_width, 0));
+ }
+
+ rank->sum_of_ranks += case_num_idx (c, rank_idx);
+ rank->n += dict_get_case_weight (dict, c, &warn);
+
+ /* If this assertion fires, then either the data wasn't sorted or some other
+ problem occurred */
+ assert (rerr == 0);
+ }
+
+ casereader_destroy (rr);
+
+ /* Calculate the value of h */
+ {
+ struct rank_entry *mre;
+ double n = 0.0;
+
+ HMAP_FOR_EACH (mre, struct rank_entry, node, &kw[i].map)
+ {
+ kw[i].h += pow2 (mre->sum_of_ranks) / mre->n;
+ n += mre->n;
+
+ total_n_groups ++;
+ }
+ kw[i].h *= 12 / (n * (n + 1));
+ kw[i].h -= 3 * (n + 1) ;
+
+ kw[i].h /= 1 - tiebreaker/ (pow3 (n) - n);
+ }
+ }
+
+ casereader_destroy (input);
+
+ show_ranks_box (nst, kw);
+ show_sig_box (nst, kw);
+
+ /* Cleanup allocated memory */
+ for (i = 0 ; i < nst->n_vars; ++i)
+ {
+ struct rank_entry *mre, *next;
+ HMAP_FOR_EACH_SAFE (mre, next, struct rank_entry, node, &kw[i].map)
+ {
+ hmap_delete (&kw[i].map, &mre->node);
+ free (mre);
+ }
+ hmap_destroy (&kw[i].map);
+ }
+
+ free (kw);
+}
+\f
+static void
+show_ranks_box (const struct n_sample_test *nst, const struct kw *kw)
+{
+ struct pivot_table *table = pivot_table_create (N_("Ranks"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_INTEGER,
+ N_("Mean Rank"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0 ; i < nst->n_vars ; ++i)
+ {
+ /* Sort the rank entries, by iteratin the hash and putting the entries
+ into a binary tree. */
+ struct bt bt = BT_INITIALIZER(compare_rank_entries_3way, nst->vars[i]);
+ struct rank_entry *re_x;
+ HMAP_FOR_EACH (re_x, struct rank_entry, node, &kw[i].map)
+ bt_insert (&bt, &re_x->btn);
+
+ /* Report the rank entries in sorted order. */
+ struct pivot_category *group = pivot_category_create_group__ (
+ variables->root, pivot_value_new_variable (nst->vars[i]));
+ int tot = 0;
+ const struct rank_entry *re;
+ BT_FOR_EACH (re, struct rank_entry, btn, &bt)
+ {
+ struct string str = DS_EMPTY_INITIALIZER;
+ var_append_value_name (nst->indep_var, &re->group, &str);
+ int row = pivot_category_create_leaf (
+ group, pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
+
+ double entries[] = { re->n, re->sum_of_ranks / re->n };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, row,
+ pivot_value_new_number (entries[j]));
+
+ tot += re->n;
+ }
+
+ int row = pivot_category_create_leaves (group, N_("Total"));
+ pivot_table_put2 (table, 0, row, pivot_value_new_number (tot));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_sig_box (const struct n_sample_test *nst, const struct kw *kw)
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Chi-Square"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Variables"));
+
+ for (size_t i = 0 ; i < nst->n_vars; ++i)
+ {
+ int col = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (nst->vars[i]));
+
+ double df = hmap_count (&kw[i].map) - 1;
+ double sig = gsl_cdf_chisq_Q (kw[i].h, df);
+ double entries[] = { kw[i].h, df, sig };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, col, pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011, 2022 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !kruskal_wallis_h
+#define kruskal_wallis_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "data/case.h"
+#include "language/commands/npar.h"
+
+struct kruskal_wallis_test
+{
+ struct n_sample_test parent;
+};
+
+struct casereader;
+struct dataset;
+
+void kruskal_wallis_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer
+ );
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/ks-one-sample.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+#include <stdlib.h>
+
+
+#include "math/sort.h"
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/commands/freq.h"
+#include "language/commands/npar.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/compiler.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+/* The per test variable statistics */
+struct ks
+{
+ double obs_cc;
+
+ double test_min ;
+ double test_max;
+ double mu;
+ double sigma;
+
+ double diff_pos;
+ double diff_neg;
+
+ double ssq;
+ double sum;
+};
+
+typedef double theoretical (const struct ks *ks, double x);
+typedef theoretical *theoreticalfp;
+
+static double
+theoretical_uniform (const struct ks *ks, double x)
+{
+ return gsl_cdf_flat_P (x, ks->test_min, ks->test_max);
+}
+
+static double
+theoretical_normal (const struct ks *ks, double x)
+{
+ return gsl_cdf_gaussian_P (x - ks->mu, ks->sigma);
+}
+
+static double
+theoretical_poisson (const struct ks *ks, double x)
+{
+ return gsl_cdf_poisson_P (x, ks->mu);
+}
+
+static double
+theoretical_exponential (const struct ks *ks, double x)
+{
+ return gsl_cdf_exponential_P (x, 1/ks->mu);
+}
+
+
+static const theoreticalfp theoreticalf[4] =
+{
+ theoretical_normal,
+ theoretical_uniform,
+ theoretical_poisson,
+ theoretical_exponential
+};
+
+/*
+ Return the assymptotic approximation to the significance of Z
+ */
+static double
+ks_asymp_sig (double z)
+{
+ if (z < 0.27)
+ return 1;
+
+ if (z >= 3.1)
+ return 0;
+
+ if (z < 1)
+ {
+ double q = exp (-1.233701 * pow (z, -2));
+ return 1 - 2.506628 * (q + pow (q, 9) + pow (q, 25))/ z ;
+ }
+ else
+ {
+ double q = exp (-2 * z * z);
+ return 2 * (q - pow (q, 4) + pow (q, 9) - pow (q, 16))/ z ;
+ }
+}
+
+static void show_results (const struct ks *, const struct ks_one_sample_test *, const struct fmt_spec *);
+
+
+void
+ks_one_sample_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool x UNUSED, double y UNUSED)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct ks_one_sample_test *kst = UP_CAST (test, const struct ks_one_sample_test, parent.parent);
+ const struct one_sample_test *ost = &kst->parent;
+ struct ccase *c;
+ const struct fmt_spec *wfmt = dict_get_weight_format (dict);
+ bool warn = true;
+ int v;
+ struct casereader *r = casereader_clone (input);
+
+ struct ks *ks = XCALLOC (ost->n_vars, struct ks);
+
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ ks[v].obs_cc = 0;
+ ks[v].test_min = DBL_MAX;
+ ks[v].test_max = -DBL_MAX;
+ ks[v].diff_pos = -DBL_MAX;
+ ks[v].diff_neg = DBL_MAX;
+ ks[v].sum = 0;
+ ks[v].ssq = 0;
+ }
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ const double weight = dict_get_case_weight (dict, c, &warn);
+
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+ const union value *val = case_data (c, var);
+
+ if (var_is_value_missing (var, val) & exclude)
+ continue;
+
+ minimize (&ks[v].test_min, val->f);
+ maximize (&ks[v].test_max, val->f);
+
+ ks[v].obs_cc += weight;
+ ks[v].sum += val->f;
+ ks[v].ssq += pow2 (val->f);
+ }
+ }
+ casereader_destroy (r);
+
+ for (v = 0; v < ost->n_vars; ++v)
+ {
+ const struct variable *var = ost->vars[v];
+ double cc = 0;
+ double prev_empirical = 0;
+
+ switch (kst->dist)
+ {
+ case KS_UNIFORM:
+ if (kst->p[0] != SYSMIS)
+ ks[v].test_min = kst->p[0];
+
+ if (kst->p[1] != SYSMIS)
+ ks[v].test_max = kst->p[1];
+ break;
+ case KS_NORMAL:
+ if (kst->p[0] != SYSMIS)
+ ks[v].mu = kst->p[0];
+ else
+ ks[v].mu = ks[v].sum / ks[v].obs_cc;
+
+ if (kst->p[1] != SYSMIS)
+ ks[v].sigma = kst->p[1];
+ else
+ {
+ ks[v].sigma = ks[v].ssq - pow2 (ks[v].sum) / ks[v].obs_cc;
+ ks[v].sigma /= ks[v].obs_cc - 1;
+ ks[v].sigma = sqrt (ks[v].sigma);
+ }
+
+ break;
+ case KS_POISSON:
+ case KS_EXPONENTIAL:
+ if (kst->p[0] != SYSMIS)
+ ks[v].mu = ks[v].sigma = kst->p[0];
+ else
+ ks[v].mu = ks[v].sigma = ks[v].sum / ks[v].obs_cc;
+ break;
+ default:
+ NOT_REACHED ();
+ }
+
+ r = sort_execute_1var (casereader_clone (input), var);
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ double theoretical, empirical;
+ double d, dp;
+ const double weight = dict_get_case_weight (dict, c, &warn);
+ const union value *val = case_data (c, var);
+
+ if (var_is_value_missing (var, val) & exclude)
+ continue;
+
+ cc += weight;
+
+ empirical = cc / ks[v].obs_cc;
+
+ theoretical = theoreticalf[kst->dist] (&ks[v], val->f);
+
+ d = empirical - theoretical;
+ dp = prev_empirical - theoretical;
+
+ if (d > 0)
+ maximize (&ks[v].diff_pos, d);
+ else
+ minimize (&ks[v].diff_neg, d);
+
+ if (dp > 0)
+ maximize (&ks[v].diff_pos, dp);
+ else
+ minimize (&ks[v].diff_neg, dp);
+
+ prev_empirical = empirical;
+ }
+
+ casereader_destroy (r);
+ }
+
+ show_results (ks, kst, wfmt);
+
+ free (ks);
+ casereader_destroy (input);
+}
+
+
+static void
+show_results (const struct ks *ks,
+ const struct ks_one_sample_test *kst,
+ const struct fmt_spec *wfmt)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("One-Sample Kolmogorov-Smirnov Test"));
+ pivot_table_set_weight_format (table, wfmt);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT);
+
+ switch (kst->dist)
+ {
+ case KS_UNIFORM:
+ pivot_category_create_group (statistics->root, N_("Uniform Parameters"),
+ N_("Minimum"), N_("Maximum"));
+ break;
+
+ case KS_NORMAL:
+ pivot_category_create_group (statistics->root, N_("Normal Parameters"),
+ N_("Mean"), N_("Std. Deviation"));
+ break;
+
+ case KS_POISSON:
+ pivot_category_create_group (statistics->root, N_("Poisson Parameters"),
+ N_("Lambda"));
+ break;
+
+ case KS_EXPONENTIAL:
+ pivot_category_create_group (statistics->root,
+ N_("Exponential Parameters"), N_("Scale"));
+ break;
+
+ default:
+ NOT_REACHED ();
+ }
+
+ pivot_category_create_group (
+ statistics->root, N_("Most Extreme Differences"),
+ N_("Absolute"), N_("Positive"), N_("Negative"));
+
+ pivot_category_create_leaves (
+ statistics->root, N_("Kolmogorov-Smirnov Z"),
+ _("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Variables"));
+
+ for (size_t i = 0; i < kst->parent.n_vars; ++i)
+ {
+ int col = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (kst->parent.vars[i]));
+
+ double values[10];
+ size_t n = 0;
+
+ values[n++] = ks[i].obs_cc;
+
+ switch (kst->dist)
+ {
+ case KS_UNIFORM:
+ values[n++] = ks[i].test_min;
+ values[n++] = ks[i].test_max;
+ break;
+
+ case KS_NORMAL:
+ values[n++] = ks[i].mu;
+ values[n++] = ks[i].sigma;
+ break;
+
+ case KS_POISSON:
+ case KS_EXPONENTIAL:
+ values[n++] = ks[i].mu;
+ break;
+
+ default:
+ NOT_REACHED ();
+ }
+
+ double abs = ks[i].diff_pos;
+ maximize (&abs, -ks[i].diff_neg);
+
+ double z = sqrt (ks[i].obs_cc) * abs;
+
+ values[n++] = abs;
+ values[n++] = ks[i].diff_pos;
+ values[n++] = ks[i].diff_neg;
+ values[n++] = z;
+ values[n++] = ks_asymp_sig (z);
+
+ for (size_t j = 0; j < n; j++)
+ pivot_table_put2 (table, j, col, pivot_value_new_number (values[j]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !ks_one_sample_h
+#define ks_one_sample_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+enum dist
+ {
+ KS_NORMAL,
+ KS_UNIFORM,
+ KS_POISSON,
+ KS_EXPONENTIAL
+ };
+
+struct ks_one_sample_test
+{
+ struct one_sample_test parent;
+
+ double p[2];
+ enum dist dist;
+};
+
+struct casereader;
+struct dataset;
+
+
+void ks_one_sample_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool, double);
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2009-2011, 2013, 2014, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/data-out.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/compiler.h"
+#include "libpspp/ll.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "output/pivot-table.h"
+
+#include "gl/intprops.h"
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+#include "gl/xmalloca.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+struct lst_cmd
+ {
+ long first;
+ long last;
+ long step;
+ const struct variable **vars;
+ size_t n_vars;
+ bool number_cases;
+ };
+
+static int
+list_execute (const struct lst_cmd *lcmd, struct dataset *ds)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+
+ struct subcase sc;
+ subcase_init_empty (&sc);
+ for (size_t i = 0; i < lcmd->n_vars; i++)
+ subcase_add_var (&sc, lcmd->vars[i], SC_ASCEND);
+
+ struct casegrouper *grouper;
+ struct casereader *group;
+ grouper = casegrouper_create_splits (proc_open (ds), dict);
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ output_split_file_values_peek (ds, group);
+ group = casereader_project (group, &sc);
+ group = casereader_select (group, lcmd->first - 1,
+ (lcmd->last != LONG_MAX ? lcmd->last
+ : CASENUMBER_MAX), lcmd->step);
+
+ struct pivot_table *table = pivot_table_create (N_("Data List"));
+ table->show_values = table->show_variables = SETTINGS_VALUE_SHOW_VALUE;
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Variables"));
+ for (size_t i = 0; i < lcmd->n_vars; i++)
+ pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (lcmd->vars[i]));
+
+ struct pivot_dimension *cases = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Case Number"));
+ if (lcmd->number_cases)
+ cases->root->show_label = true;
+ else
+ cases->hide_all_labels = true;
+
+ casenumber case_num = lcmd->first;
+ struct ccase *c;
+ for (; (c = casereader_read (group)) != NULL; case_unref (c))
+ {
+ int case_idx = pivot_category_create_leaf (
+ cases->root, pivot_value_new_integer (case_num));
+ case_num += lcmd->step;
+
+ for (size_t i = 0; i < lcmd->n_vars; i++)
+ pivot_table_put2 (table, i, case_idx,
+ pivot_value_new_var_value (
+ lcmd->vars[i], case_data_idx (c, i)));
+ }
+ casereader_destroy (group);
+
+ pivot_table_submit (table);
+ }
+
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ subcase_uninit (&sc);
+ free (lcmd->vars);
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+}
+
+
+/* Parses and executes the LIST procedure. */
+int
+cmd_list (struct lexer *lexer, struct dataset *ds)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+
+ struct lst_cmd cmd = {
+ .step = 1,
+ .first = 1,
+ .last = LONG_MAX,
+ };
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "VARIABLES"))
+ {
+ lex_match (lexer, T_EQUALS);
+ free (cmd.vars);
+ cmd.vars = NULL;
+ if (!parse_variables_const (lexer, dict, &cmd.vars, &cmd.n_vars,
+ PV_DUPLICATE))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "NUMBERED"))
+ cmd.number_cases = true;
+ else if (lex_match_id (lexer, "UNNUMBERED"))
+ cmd.number_cases = false;
+ else
+ {
+ lex_error_expecting (lexer, "NUMBERED", "UNNUMBERED");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "CASES"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "FROM"))
+ {
+ if (!lex_force_int_range (lexer, "FROM", 1, LONG_MAX))
+ goto error;
+ cmd.first = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ cmd.first = 1;
+
+ if (lex_match (lexer, T_TO) || lex_is_integer (lexer))
+ {
+ if (!lex_force_int_range (lexer, "TO", cmd.first, LONG_MAX))
+ goto error;
+ cmd.last = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ cmd.last = LONG_MAX;
+
+ if (lex_match (lexer, T_BY))
+ {
+ if (!lex_force_int_range (lexer, "TO", 1, LONG_MAX))
+ goto error;
+ cmd.step = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ cmd.step = 1;
+ }
+ else
+ {
+ free (cmd.vars);
+ cmd.vars = NULL;
+ if (!parse_variables_const (lexer, dict, &cmd.vars, &cmd.n_vars,
+ PV_DUPLICATE))
+ goto error;
+ }
+ }
+
+ if (!cmd.n_vars)
+ dict_get_vars (dict, &cmd.vars, &cmd.n_vars, DC_SYSTEM | DC_SCRATCH);
+ return list_execute (&cmd, ds);
+
+ error:
+ free (cmd.vars);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* pspp - a program for statistical analysis.
+ Copyright (C) 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+
+/*
+ References:
+ 1. "Coding Logistic Regression with Newton-Raphson", James McCaffrey
+ http://msdn.microsoft.com/en-us/magazine/jj618304.aspx
+
+ 2. "SPSS Statistical Algorithms" Chapter LOGISTIC REGRESSION Algorithms
+
+
+ The Newton Raphson method finds successive approximations to $\bf b$ where
+ approximation ${\bf b}_t$ is (hopefully) better than the previous ${\bf b}_{t-1}$.
+
+ $ {\bf b}_t = {\bf b}_{t -1} + ({\bf X}^T{\bf W}_{t-1}{\bf X})^{-1}{\bf X}^T({\bf y} - {\bf \pi}_{t-1})$
+ where:
+
+ $\bf X$ is the $n \times p$ design matrix, $n$ being the number of cases,
+ $p$ the number of parameters, \par
+ $\bf W$ is the diagonal matrix whose diagonal elements are
+ $\hat{\pi}_0(1 - \hat{\pi}_0), \, \hat{\pi}_1(1 - \hat{\pi}_2)\dots \hat{\pi}_{n-1}(1 - \hat{\pi}_{n-1})$
+ \par
+
+*/
+
+#include <config.h>
+
+#include <gsl/gsl_blas.h>
+
+#include <gsl/gsl_linalg.h>
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_matrix.h>
+#include <gsl/gsl_vector.h>
+#include <math.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmap.h"
+#include "libpspp/ll.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/categoricals.h"
+#include "math/interaction.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+
+
+#define PRINT_EACH_STEP 0x01
+#define PRINT_SUMMARY 0x02
+#define PRINT_CORR 0x04
+#define PRINT_ITER 0x08
+#define PRINT_GOODFIT 0x10
+#define PRINT_CI 0x20
+
+
+#define PRINT_DEFAULT (PRINT_SUMMARY | PRINT_EACH_STEP)
+
+/*
+ The constant parameters of the procedure.
+ That is, those which are set by the user.
+*/
+struct lr_spec
+{
+ /* The dependent variable */
+ const struct variable *dep_var;
+
+ /* The predictor variables (excluding categorical ones) */
+ const struct variable **predictor_vars;
+ size_t n_predictor_vars;
+
+ /* The categorical predictors */
+ struct interaction **cat_predictors;
+ size_t n_cat_predictors;
+
+
+ /* The union of the categorical and non-categorical variables */
+ const struct variable **indep_vars;
+ size_t n_indep_vars;
+
+
+ /* Which classes of missing vars are to be excluded */
+ enum mv_class exclude;
+
+ /* The weight variable */
+ const struct variable *wv;
+
+ /* The dictionary of the dataset */
+ const struct dictionary *dict;
+
+ /* True iff the constant (intercept) is to be included in the model */
+ bool constant;
+
+ /* Ths maximum number of iterations */
+ int max_iter;
+
+ /* Other iteration limiting conditions */
+ double bcon;
+ double min_epsilon;
+ double lcon;
+
+ /* The confidence interval (in percent) */
+ int confidence;
+
+ /* What results should be presented */
+ unsigned int print;
+
+ /* Inverse logit of the cut point */
+ double ilogit_cut_point;
+};
+
+
+/* The results and intermediate result of the procedure.
+ These are mutated as the procedure runs. Used for
+ temporary variables etc.
+*/
+struct lr_result
+{
+ /* Used to indicate if a pass should flag a warning when
+ invalid (ie negative or missing) weight values are encountered */
+ bool warn_bad_weight;
+
+ /* The two values of the dependent variable. */
+ union value y0;
+ union value y1;
+
+
+ /* The sum of caseweights */
+ double cc;
+
+ /* The number of missing and nonmissing cases */
+ casenumber n_missing;
+ casenumber n_nonmissing;
+
+
+ gsl_matrix *hessian;
+
+ /* The categoricals and their payload. Null if the analysis has no
+ categorical predictors */
+ struct categoricals *cats;
+ struct payload cp;
+
+
+ /* The estimates of the predictor coefficients */
+ gsl_vector *beta_hat;
+
+ /* The predicted classifications:
+ True Negative, True Positive, False Negative, False Positive */
+ double tn, tp, fn, fp;
+};
+
+
+/*
+ Convert INPUT into a dichotomous scalar, according to how the dependent variable's
+ values are mapped.
+ For simple cases, this is a 1:1 mapping
+ The return value is always either 0 or 1
+*/
+static double
+map_dependent_var (const struct lr_spec *cmd, const struct lr_result *res, const union value *input)
+{
+ const int width = var_get_width (cmd->dep_var);
+ if (value_equal (input, &res->y0, width))
+ return 0;
+
+ if (value_equal (input, &res->y1, width))
+ return 1;
+
+ /* This should never happen. If it does, then y0 and/or y1 have probably not been set */
+ NOT_REACHED ();
+
+ return SYSMIS;
+}
+
+static void output_classification_table (const struct lr_spec *cmd, const struct lr_result *res);
+
+static void output_categories (const struct lr_spec *cmd, const struct lr_result *res);
+
+static void output_depvarmap (const struct lr_spec *cmd, const struct lr_result *);
+
+static void output_variables (const struct lr_spec *cmd,
+ const struct lr_result *);
+
+static void output_model_summary (const struct lr_result *,
+ double initial_likelihood, double likelihood);
+
+static void case_processing_summary (const struct lr_result *);
+
+
+/* Return the value of case C corresponding to the INDEX'th entry in the
+ model */
+static double
+predictor_value (const struct ccase *c,
+ const struct variable **x, size_t n_x,
+ const struct categoricals *cats,
+ size_t index)
+{
+ /* Values of the scalar predictor variables */
+ if (index < n_x)
+ return case_num (c, x[index]);
+
+ /* Coded values of categorical predictor variables (or interactions) */
+ if (cats && index - n_x < categoricals_df_total (cats))
+ {
+ double x = categoricals_get_dummy_code_for_case (cats, index - n_x, c);
+ return x;
+ }
+
+ /* The constant term */
+ return 1.0;
+}
+
+
+/*
+ Return the probability beta_hat (that is the estimator logit(y))
+ corresponding to the coefficient estimator for case C
+*/
+static double
+pi_hat (const struct lr_spec *cmd,
+ const struct lr_result *res,
+ const struct variable **x, size_t n_x,
+ const struct ccase *c)
+{
+ int v0;
+ double pi = 0;
+ size_t n_coeffs = res->beta_hat->size;
+
+ if (cmd->constant)
+ {
+ pi += gsl_vector_get (res->beta_hat, res->beta_hat->size - 1);
+ n_coeffs--;
+ }
+
+ for (v0 = 0; v0 < n_coeffs; ++v0)
+ {
+ pi += gsl_vector_get (res->beta_hat, v0) *
+ predictor_value (c, x, n_x, res->cats, v0);
+ }
+
+ pi = 1.0 / (1.0 + exp(-pi));
+
+ return pi;
+}
+
+
+/*
+ Calculates the Hessian matrix X' V X,
+ where: X is the n by N_X matrix comprising the n cases in INPUT
+ V is a diagonal matrix { (pi_hat_0)(1 - pi_hat_0), (pi_hat_1)(1 - pi_hat_1), ... (pi_hat_{N-1})(1 - pi_hat_{N-1})}
+ (the partial derivative of the predicted values)
+
+ If ALL predicted values derivatives are close to zero or one, then CONVERGED
+ will be set to true.
+*/
+static void
+hessian (const struct lr_spec *cmd,
+ struct lr_result *res,
+ struct casereader *input,
+ const struct variable **x, size_t n_x,
+ bool *converged)
+{
+ struct casereader *reader;
+ struct ccase *c;
+
+ double max_w = -DBL_MAX;
+
+ gsl_matrix_set_zero (res->hessian);
+
+ for (reader = casereader_clone (input);
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ int v0, v1;
+ double pi = pi_hat (cmd, res, x, n_x, c);
+
+ double weight = dict_get_case_weight (cmd->dict, c, &res->warn_bad_weight);
+ double w = pi * (1 - pi);
+ if (w > max_w)
+ max_w = w;
+ w *= weight;
+
+ for (v0 = 0; v0 < res->beta_hat->size; ++v0)
+ {
+ double in0 = predictor_value (c, x, n_x, res->cats, v0);
+ for (v1 = 0; v1 < res->beta_hat->size; ++v1)
+ {
+ double in1 = predictor_value (c, x, n_x, res->cats, v1);
+ double *o = gsl_matrix_ptr (res->hessian, v0, v1);
+ *o += in0 * w * in1;
+ }
+ }
+ }
+ casereader_destroy (reader);
+
+ if (max_w < cmd->min_epsilon)
+ {
+ *converged = true;
+ msg (MN, _("All predicted values are either 1 or 0"));
+ }
+}
+
+
+/* Calculates the value X' (y - pi)
+ where X is the design model,
+ y is the vector of observed independent variables
+ pi is the vector of estimates for y
+
+ Side effects:
+ the likelihood is stored in LIKELIHOOD;
+ the predicted values are placed in the respective tn, fn, tp fp values in RES
+*/
+static gsl_vector *
+xt_times_y_pi (const struct lr_spec *cmd,
+ struct lr_result *res,
+ struct casereader *input,
+ const struct variable **x, size_t n_x,
+ const struct variable *y_var,
+ double *llikelihood)
+{
+ struct casereader *reader;
+ struct ccase *c;
+ gsl_vector *output = gsl_vector_calloc (res->beta_hat->size);
+
+ *llikelihood = 0.0;
+ res->tn = res->tp = res->fn = res->fp = 0;
+ for (reader = casereader_clone (input);
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ double pred_y = 0;
+ int v0;
+ double pi = pi_hat (cmd, res, x, n_x, c);
+ double weight = dict_get_case_weight (cmd->dict, c, &res->warn_bad_weight);
+
+
+ double y = map_dependent_var (cmd, res, case_data (c, y_var));
+
+ *llikelihood += (weight * y) * log (pi) + log (1 - pi) * weight * (1 - y);
+
+ for (v0 = 0; v0 < res->beta_hat->size; ++v0)
+ {
+ double in0 = predictor_value (c, x, n_x, res->cats, v0);
+ double *o = gsl_vector_ptr (output, v0);
+ *o += in0 * (y - pi) * weight;
+ pred_y += gsl_vector_get (res->beta_hat, v0) * in0;
+ }
+
+ /* Count the number of cases which would be correctly/incorrectly classified by this
+ estimated model */
+ if (pred_y <= cmd->ilogit_cut_point)
+ {
+ if (y == 0)
+ res->tn += weight;
+ else
+ res->fn += weight;
+ }
+ else
+ {
+ if (y == 0)
+ res->fp += weight;
+ else
+ res->tp += weight;
+ }
+ }
+
+ casereader_destroy (reader);
+
+ return output;
+}
+
+\f
+
+/* "payload" functions for the categoricals.
+ The only function is to accumulate the frequency of each
+ category.
+ */
+
+static void *
+frq_create (const void *aux1 UNUSED, void *aux2 UNUSED)
+{
+ return xzalloc (sizeof (double));
+}
+
+static void
+frq_update (const void *aux1 UNUSED, void *aux2 UNUSED,
+ void *ud, const struct ccase *c UNUSED , double weight)
+{
+ double *freq = ud;
+ *freq += weight;
+}
+
+static void
+frq_destroy (const void *aux1 UNUSED, void *aux2 UNUSED, void *user_data)
+{
+ free (user_data);
+}
+
+\f
+
+/*
+ Makes an initial pass though the data, doing the following:
+
+ * Checks that the dependent variable is dichotomous,
+ * Creates and initialises the categoricals,
+ * Accumulates summary results,
+ * Calculates necessary initial values.
+ * Creates an initial value for \hat\beta the vector of beta_hats of \beta
+
+ Returns true if successful
+*/
+static bool
+initial_pass (const struct lr_spec *cmd, struct lr_result *res, struct casereader *input)
+{
+ const int width = var_get_width (cmd->dep_var);
+
+ struct ccase *c;
+ struct casereader *reader;
+
+ double sum;
+ double sumA = 0.0;
+ double sumB = 0.0;
+
+ bool v0set = false;
+ bool v1set = false;
+
+ size_t n_coefficients = cmd->n_predictor_vars;
+ if (cmd->constant)
+ n_coefficients++;
+
+ /* Create categoricals if appropriate */
+ if (cmd->n_cat_predictors > 0)
+ {
+ res->cp.create = frq_create;
+ res->cp.update = frq_update;
+ res->cp.calculate = NULL;
+ res->cp.destroy = frq_destroy;
+
+ res->cats = categoricals_create (cmd->cat_predictors, cmd->n_cat_predictors,
+ cmd->wv, MV_ANY);
+
+ categoricals_set_payload (res->cats, &res->cp, cmd, res);
+ }
+
+ res->cc = 0;
+ for (reader = casereader_clone (input);
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ int v;
+ bool missing = false;
+ double weight = dict_get_case_weight (cmd->dict, c, &res->warn_bad_weight);
+ const union value *depval = case_data (c, cmd->dep_var);
+
+ if (var_is_value_missing (cmd->dep_var, depval) & cmd->exclude)
+ {
+ missing = true;
+ }
+ else
+ for (v = 0; v < cmd->n_indep_vars; ++v)
+ {
+ const union value *val = case_data (c, cmd->indep_vars[v]);
+ if (var_is_value_missing (cmd->indep_vars[v], val) & cmd->exclude)
+ {
+ missing = true;
+ break;
+ }
+ }
+
+ /* Accumulate the missing and non-missing counts */
+ if (missing)
+ {
+ res->n_missing++;
+ continue;
+ }
+ res->n_nonmissing++;
+
+ /* Find the values of the dependent variable */
+ if (!v0set)
+ {
+ value_clone (&res->y0, depval, width);
+ v0set = true;
+ }
+ else if (!v1set)
+ {
+ if (!value_equal (&res->y0, depval, width))
+ {
+ value_clone (&res->y1, depval, width);
+ v1set = true;
+ }
+ }
+ else
+ {
+ if (!value_equal (&res->y0, depval, width)
+ &&
+ !value_equal (&res->y1, depval, width)
+ )
+ {
+ msg (ME, _("Dependent variable's values are not dichotomous."));
+ case_unref (c);
+ goto error;
+ }
+ }
+
+ if (v0set && value_equal (&res->y0, depval, width))
+ sumA += weight;
+
+ if (v1set && value_equal (&res->y1, depval, width))
+ sumB += weight;
+
+
+ res->cc += weight;
+
+ categoricals_update (res->cats, c);
+ }
+ casereader_destroy (reader);
+
+ categoricals_done (res->cats);
+
+ sum = sumB;
+
+ /* Ensure that Y0 is less than Y1. Otherwise the mapping gets
+ inverted, which is confusing to users */
+ if (var_is_numeric (cmd->dep_var) && value_compare_3way (&res->y0, &res->y1, width) > 0)
+ {
+ union value tmp;
+ value_clone (&tmp, &res->y0, width);
+ value_copy (&res->y0, &res->y1, width);
+ value_copy (&res->y1, &tmp, width);
+ value_destroy (&tmp, width);
+ sum = sumA;
+ }
+
+ n_coefficients += categoricals_df_total (res->cats);
+ res->beta_hat = gsl_vector_calloc (n_coefficients);
+
+ if (cmd->constant)
+ {
+ double mean = sum / res->cc;
+ gsl_vector_set (res->beta_hat, res->beta_hat->size - 1, log (mean / (1 - mean)));
+ }
+
+ return true;
+
+ error:
+ casereader_destroy (reader);
+ return false;
+}
+
+
+
+/* Start of the logistic regression routine proper */
+static bool
+run_lr (const struct lr_spec *cmd, struct casereader *input,
+ const struct dataset *ds UNUSED)
+{
+ int i;
+
+ bool converged = false;
+
+ /* Set the log likelihoods to a sentinel value */
+ double log_likelihood = SYSMIS;
+ double prev_log_likelihood = SYSMIS;
+ double initial_log_likelihood = SYSMIS;
+
+ struct lr_result work;
+ work.n_missing = 0;
+ work.n_nonmissing = 0;
+ work.warn_bad_weight = true;
+ work.cats = NULL;
+ work.beta_hat = NULL;
+ work.hessian = NULL;
+
+ /* Get the initial estimates of \beta and their standard errors.
+ And perform other auxiliary initialisation. */
+ if (!initial_pass (cmd, &work, input))
+ goto error;
+
+ for (i = 0; i < cmd->n_cat_predictors; ++i)
+ {
+ if (1 >= categoricals_n_count (work.cats, i))
+ {
+ struct string str;
+ ds_init_empty (&str);
+
+ interaction_to_string (cmd->cat_predictors[i], &str);
+
+ msg (ME, _("Category %s does not have at least two distinct values. Logistic regression will not be run."),
+ ds_cstr(&str));
+ ds_destroy (&str);
+ goto error;
+ }
+ }
+
+ output_depvarmap (cmd, &work);
+
+ case_processing_summary (&work);
+
+
+ input = casereader_create_filter_missing (input,
+ cmd->indep_vars,
+ cmd->n_indep_vars,
+ cmd->exclude,
+ NULL,
+ NULL);
+
+ input = casereader_create_filter_missing (input,
+ &cmd->dep_var,
+ 1,
+ cmd->exclude,
+ NULL,
+ NULL);
+
+ work.hessian = gsl_matrix_calloc (work.beta_hat->size, work.beta_hat->size);
+
+ /* Start the Newton Raphson iteration process... */
+ for(i = 0; i < cmd->max_iter; ++i)
+ {
+ double min, max;
+ gsl_vector *v;
+
+
+ hessian (cmd, &work, input,
+ cmd->predictor_vars, cmd->n_predictor_vars,
+ &converged);
+
+ gsl_linalg_cholesky_decomp (work.hessian);
+ gsl_linalg_cholesky_invert (work.hessian);
+
+ v = xt_times_y_pi (cmd, &work, input,
+ cmd->predictor_vars, cmd->n_predictor_vars,
+ cmd->dep_var,
+ &log_likelihood);
+
+ {
+ /* delta = M.v */
+ gsl_vector *delta = gsl_vector_alloc (v->size);
+ gsl_blas_dgemv (CblasNoTrans, 1.0, work.hessian, v, 0, delta);
+ gsl_vector_free (v);
+
+
+ gsl_vector_add (work.beta_hat, delta);
+
+ gsl_vector_minmax (delta, &min, &max);
+
+ if (fabs (min) < cmd->bcon && fabs (max) < cmd->bcon)
+ {
+ msg (MN, _("Estimation terminated at iteration number %d because parameter estimates changed by less than %g"),
+ i + 1, cmd->bcon);
+ converged = true;
+ }
+
+ gsl_vector_free (delta);
+ }
+
+ if (i > 0)
+ {
+ if (-log_likelihood > -(1.0 - cmd->lcon) * prev_log_likelihood)
+ {
+ msg (MN, _("Estimation terminated at iteration number %d because Log Likelihood decreased by less than %g%%"), i + 1, 100 * cmd->lcon);
+ converged = true;
+ }
+ }
+ if (i == 0)
+ initial_log_likelihood = log_likelihood;
+ prev_log_likelihood = log_likelihood;
+
+ if (converged)
+ break;
+ }
+
+
+
+ if (!converged)
+ msg (MW, _("Estimation terminated at iteration number %d because maximum iterations has been reached"), i);
+
+
+ output_model_summary (&work, initial_log_likelihood, log_likelihood);
+
+ if (work.cats)
+ output_categories (cmd, &work);
+
+ output_classification_table (cmd, &work);
+ output_variables (cmd, &work);
+
+ casereader_destroy (input);
+ gsl_matrix_free (work.hessian);
+ gsl_vector_free (work.beta_hat);
+ categoricals_destroy (work.cats);
+
+ return true;
+
+ error:
+ casereader_destroy (input);
+ gsl_matrix_free (work.hessian);
+ gsl_vector_free (work.beta_hat);
+ categoricals_destroy (work.cats);
+
+ return false;
+}
+
+struct variable_node
+{
+ struct hmap_node node; /* Node in hash map. */
+ const struct variable *var; /* The variable */
+};
+
+static struct variable_node *
+lookup_variable (const struct hmap *map, const struct variable *var, unsigned int hash)
+{
+ struct variable_node *vn;
+ HMAP_FOR_EACH_WITH_HASH (vn, struct variable_node, node, hash, map)
+ if (vn->var == var)
+ return vn;
+
+ return NULL;
+}
+
+static void
+insert_variable (struct hmap *map, const struct variable *var, unsigned int hash)
+{
+ if (!lookup_variable (map, var, hash))
+ {
+ struct variable_node *vn = xmalloc (sizeof *vn);
+ *vn = (struct variable_node) { .var = var };
+ hmap_insert (map, &vn->node, hash);
+ }
+}
+
+/* Parse the LOGISTIC REGRESSION command syntax */
+int
+cmd_logistic (struct lexer *lexer, struct dataset *ds)
+{
+ /* Temporary location for the predictor variables.
+ These may or may not include the categorical predictors */
+ const struct variable **pred_vars = NULL;
+ size_t n_pred_vars = 0;
+ double cp = 0.5;
+
+ struct dictionary *dict = dataset_dict (ds);
+ struct lr_spec lr = {
+ .dict = dict,
+ .exclude = MV_ANY,
+ .wv = dict_get_weight (dict),
+ .max_iter = 20,
+ .lcon = 0.0000,
+ .bcon = 0.001,
+ .min_epsilon = 0.00000001,
+ .constant = true,
+ .confidence = 95,
+ .print = PRINT_DEFAULT,
+ };
+
+ if (lex_match_id (lexer, "VARIABLES"))
+ lex_match (lexer, T_EQUALS);
+
+ lr.dep_var = parse_variable_const (lexer, lr.dict);
+ if (!lr.dep_var)
+ goto error;
+
+ if (!lex_force_match (lexer, T_WITH))
+ goto error;
+
+ if (!parse_variables_const (lexer, lr.dict, &pred_vars, &n_pred_vars,
+ PV_NO_DUPLICATE))
+ goto error;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ lr.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ lr.exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "ORIGIN"))
+ lr.constant = false;
+ else if (lex_match_id (lexer, "NOORIGIN"))
+ lr.constant = true;
+ else if (lex_match_id (lexer, "NOCONST"))
+ lr.constant = false;
+ else if (lex_match_id (lexer, "EXTERNAL"))
+ {
+ /* This is for compatibility. It does nothing */
+ }
+ else if (lex_match_id (lexer, "CATEGORICAL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ struct variable **cats;
+ size_t n_cats;
+ if (!parse_variables (lexer, lr.dict, &cats, &n_cats, PV_NO_DUPLICATE))
+ goto error;
+
+ lr.cat_predictors = xrealloc (lr.cat_predictors,
+ sizeof *lr.cat_predictors
+ * (n_cats + lr.n_cat_predictors));
+ for (size_t i = 0; i < n_cats; i++)
+ lr.cat_predictors[lr.n_cat_predictors++] = interaction_create (cats[i]);
+ free (cats);
+ }
+ else if (lex_match_id (lexer, "PRINT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "DEFAULT"))
+ lr.print |= PRINT_DEFAULT;
+ else if (lex_match_id (lexer, "SUMMARY"))
+ lr.print |= PRINT_SUMMARY;
+#if 0
+ else if (lex_match_id (lexer, "CORR"))
+ lr.print |= PRINT_CORR;
+ else if (lex_match_id (lexer, "ITER"))
+ lr.print |= PRINT_ITER;
+ else if (lex_match_id (lexer, "GOODFIT"))
+ lr.print |= PRINT_GOODFIT;
+#endif
+ else if (lex_match_id (lexer, "CI"))
+ {
+ lr.print |= PRINT_CI;
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ lr.confidence = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ALL"))
+ lr.print = ~0x0000;
+ else
+ {
+ lex_error_expecting (lexer, "DEFAULT", "SUMMARY",
+#if 0
+ "CORR", "ITER", "GOODFIT",
+#endif
+ "CI", "ALL");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "BCON"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ lr.bcon = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ITERATE"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "ITERATE", 0, INT_MAX))
+ goto error;
+ lr.max_iter = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "LCON"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ lr.lcon = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "EPS"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ lr.min_epsilon = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "CUT"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_closed (lexer, "CUT", 0, 1))
+ goto error;
+
+ cp = lex_number (lexer);
+
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "BCON", "ITERATE", "LCON", "EPS",
+ "CUT");
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MISSING", "ORIGIN", "NOORIGIN",
+ "NOCONST", "EXTERNAL", "CATEGORICAL",
+ "PRINT", "CRITERIA");
+ goto error;
+ }
+ }
+
+ lr.ilogit_cut_point = - log (1/cp - 1);
+
+ /* Copy the predictor variables from the temporary location into the
+ final one, dropping any categorical variables which appear there.
+ FIXME: This is O(NxM).
+ */
+ struct hmap allvars = HMAP_INITIALIZER (allvars);
+ size_t allocated_predictor_vars = 0;
+ for (size_t v = 0; v < n_pred_vars; ++v)
+ {
+ bool drop = false;
+ const struct variable *var = pred_vars[v];
+
+ unsigned int hash = hash_pointer (var, 0);
+ insert_variable (&allvars, var, hash);
+
+ for (size_t cv = 0; cv < lr.n_cat_predictors; ++cv)
+ {
+ const struct interaction *iact = lr.cat_predictors[cv];
+ for (size_t iv = 0; iv < iact->n_vars; ++iv)
+ {
+ const struct variable *ivar = iact->vars[iv];
+ unsigned int hash = hash_pointer (ivar, 0);
+ insert_variable (&allvars, ivar, hash);
+
+ if (var == ivar)
+ drop = true;
+ }
+ }
+
+ if (drop)
+ continue;
+
+ if (lr.n_predictor_vars >= allocated_predictor_vars)
+ lr.predictor_vars = x2nrealloc (lr.predictor_vars,
+ &allocated_predictor_vars,
+ sizeof *lr.predictor_vars);
+ lr.predictor_vars[lr.n_predictor_vars++] = var;
+ }
+
+ lr.n_indep_vars = hmap_count (&allvars);
+ lr.indep_vars = xmalloc (lr.n_indep_vars * sizeof *lr.indep_vars);
+
+ /* Interate over each variable and push it into the array */
+ size_t x = 0;
+ struct variable_node *vn, *next;
+ HMAP_FOR_EACH_SAFE (vn, next, struct variable_node, node, &allvars)
+ {
+ lr.indep_vars[x++] = vn->var;
+ hmap_delete (&allvars, &vn->node);
+ free (vn);
+ }
+ assert (x == lr.n_indep_vars);
+ hmap_destroy (&allvars);
+
+ /* Run logistical regression for each split group. */
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), lr.dict);
+ struct casereader *group;
+ bool ok = true;
+ while (casegrouper_get_next_group (grouper, &group))
+ ok = run_lr (&lr, group, ds) && ok;
+ ok = casegrouper_destroy (grouper) && ok;
+ ok = proc_commit (ds) && ok;
+
+ for (size_t i = 0; i < lr.n_cat_predictors; ++i)
+ interaction_destroy (lr.cat_predictors[i]);
+ free (lr.predictor_vars);
+ free (lr.cat_predictors);
+ free (lr.indep_vars);
+ free (pred_vars);
+
+ return CMD_SUCCESS;
+
+ error:
+ for (size_t i = 0; i < lr.n_cat_predictors; ++i)
+ interaction_destroy (lr.cat_predictors[i]);
+ free (lr.predictor_vars);
+ free (lr.cat_predictors);
+ free (lr.indep_vars);
+ free (pred_vars);
+
+ return CMD_FAILURE;
+}
+
+
+\f
+
+/* Show the Dependent Variable Encoding box.
+ This indicates how the dependent variable
+ is mapped to the internal zero/one values.
+*/
+static void
+output_depvarmap (const struct lr_spec *cmd, const struct lr_result *res)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Dependent Variable Encoding"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Mapping"),
+ N_("Internal Value"));
+
+ struct pivot_dimension *original = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Original Value"));
+ original->root->show_label = true;
+
+ for (int i = 0; i < 2; i++)
+ {
+ const union value *v = i ? &res->y1 : &res->y0;
+ int orig_idx = pivot_category_create_leaf (
+ original->root, pivot_value_new_var_value (cmd->dep_var, v));
+ pivot_table_put2 (table, 0, orig_idx, pivot_value_new_number (
+ map_dependent_var (cmd, res, v)));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+/* Show the Variables in the Equation box */
+static void
+output_variables (const struct lr_spec *cmd,
+ const struct lr_result *res)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Variables in the Equation"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("B"), PIVOT_RC_OTHER,
+ N_("S.E."), PIVOT_RC_OTHER,
+ N_("Wald"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE,
+ N_("Exp(B)"), PIVOT_RC_OTHER);
+ if (cmd->print & PRINT_CI)
+ {
+ struct pivot_category *group = pivot_category_create_group__ (
+ statistics->root,
+ pivot_value_new_text_format (N_("%d%% CI for Exp(B)"),
+ cmd->confidence));
+ pivot_category_create_leaves (group, N_("Lower"), N_("Upper"));
+ }
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+ struct pivot_category *step1 = pivot_category_create_group (
+ variables->root, N_("Step 1"));
+
+ int ivar = 0;
+ int idx_correction = 0;
+ int i = 0;
+
+ int nr = cmd->n_predictor_vars;
+ if (cmd->constant)
+ nr++;
+ if (res->cats)
+ nr += categoricals_df_total (res->cats) + cmd->n_cat_predictors;
+
+ for (int row = 0; row < nr; row++)
+ {
+ const int idx = row - idx_correction;
+
+ int var_idx;
+ if (idx < cmd->n_predictor_vars)
+ var_idx = pivot_category_create_leaf (
+ step1, pivot_value_new_variable (cmd->predictor_vars[idx]));
+ else if (i < cmd->n_cat_predictors)
+ {
+ const struct interaction *cat_predictors = cmd->cat_predictors[i];
+ struct string str = DS_EMPTY_INITIALIZER;
+ interaction_to_string (cat_predictors, &str);
+ if (ivar != 0)
+ ds_put_format (&str, "(%d)", ivar);
+ var_idx = pivot_category_create_leaf (
+ step1, pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
+
+ int df = categoricals_df (res->cats, i);
+ bool summary = ivar == 0;
+ if (summary)
+ {
+ /* Calculate the Wald statistic,
+ which is \beta' C^-1 \beta .
+ where \beta is the vector of the coefficient estimates comprising this
+ categorial variable. and C is the corresponding submatrix of the
+ hessian matrix.
+ */
+ gsl_matrix_const_view mv =
+ gsl_matrix_const_submatrix (res->hessian, idx, idx, df, df);
+ gsl_matrix *subhessian = gsl_matrix_alloc (mv.matrix.size1, mv.matrix.size2);
+ gsl_vector_const_view vv = gsl_vector_const_subvector (res->beta_hat, idx, df);
+ gsl_vector *temp = gsl_vector_alloc (df);
+
+ gsl_matrix_memcpy (subhessian, &mv.matrix);
+ gsl_linalg_cholesky_decomp (subhessian);
+ gsl_linalg_cholesky_invert (subhessian);
+
+ gsl_blas_dgemv (CblasTrans, 1.0, subhessian, &vv.vector, 0, temp);
+ double wald;
+ gsl_blas_ddot (temp, &vv.vector, &wald);
+
+ double entries[] = { wald, df, gsl_cdf_chisq_Q (wald, df) };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j + 2, var_idx,
+ pivot_value_new_number (entries[j]));
+
+ idx_correction++;
+ gsl_matrix_free (subhessian);
+ gsl_vector_free (temp);
+ }
+
+ if (ivar++ == df)
+ {
+ ++i; /* next interaction */
+ ivar = 0;
+ }
+
+ if (summary)
+ continue;
+ }
+ else
+ var_idx = pivot_category_create_leaves (step1, N_("Constant"));
+
+ double b = gsl_vector_get (res->beta_hat, idx);
+ double sigma2 = gsl_matrix_get (res->hessian, idx, idx);
+ double wald = pow2 (b) / sigma2;
+ double df = 1;
+ double wc = (gsl_cdf_ugaussian_Pinv (0.5 + cmd->confidence / 200.0)
+ * sqrt (sigma2));
+ bool show_ci = cmd->print & PRINT_CI && row < nr - cmd->constant;
+
+ double entries[] = {
+ b,
+ sqrt (sigma2),
+ wald,
+ df,
+ gsl_cdf_chisq_Q (wald, df),
+ exp (b),
+ show_ci ? exp (b - wc) : SYSMIS,
+ show_ci ? exp (b + wc) : SYSMIS,
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ if (entries[j] != SYSMIS)
+ pivot_table_put2 (table, j, var_idx,
+ pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+/* Show the model summary box */
+static void
+output_model_summary (const struct lr_result *res,
+ double initial_log_likelihood, double log_likelihood)
+{
+ struct pivot_table *table = pivot_table_create (N_("Model Summary"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("-2 Log likelihood"), PIVOT_RC_OTHER,
+ N_("Cox & Snell R Square"), PIVOT_RC_OTHER,
+ N_("Nagelkerke R Square"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *step = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Step"));
+ step->root->show_label = true;
+ pivot_category_create_leaf (step->root, pivot_value_new_integer (1));
+
+ double cox = (1.0 - exp ((initial_log_likelihood - log_likelihood)
+ * (2 / res->cc)));
+ double entries[] = {
+ -2 * log_likelihood,
+ cox,
+ cox / (1.0 - exp(initial_log_likelihood * (2 / res->cc)))
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, 0, pivot_value_new_number (entries[i]));
+
+ pivot_table_submit (table);
+}
+
+/* Show the case processing summary box */
+static void
+case_processing_summary (const struct lr_result *res)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Case Processing Summary"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Percent"), PIVOT_RC_PERCENT);
+
+ struct pivot_dimension *cases = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Unweighted Cases"),
+ N_("Included in Analysis"), N_("Missing Cases"), N_("Total"));
+ cases->root->show_label = true;
+
+ double total = res->n_nonmissing + res->n_missing;
+ struct entry
+ {
+ int stat_idx;
+ int case_idx;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, res->n_nonmissing },
+ { 0, 1, res->n_missing },
+ { 0, 2, total },
+ { 1, 0, 100.0 * res->n_nonmissing / total },
+ { 1, 1, 100.0 * res->n_missing / total },
+ { 1, 2, 100.0 },
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, entries[i].stat_idx, entries[i].case_idx,
+ pivot_value_new_number (entries[i].x));
+
+ pivot_table_submit (table);
+}
+
+static void
+output_categories (const struct lr_spec *cmd, const struct lr_result *res)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Categorical Variables' Codings"));
+ pivot_table_set_weight_var (table, dict_get_weight (cmd->dict));
+
+ int max_df = 0;
+ int total_cats = 0;
+ for (int i = 0; i < cmd->n_cat_predictors; ++i)
+ {
+ size_t n = categoricals_n_count (res->cats, i);
+ size_t df = categoricals_df (res->cats, i);
+ if (max_df < df)
+ max_df = df;
+ total_cats += n;
+ }
+
+ struct pivot_dimension *codings = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Codings"),
+ N_("Frequency"), PIVOT_RC_COUNT);
+ struct pivot_category *coding_group = pivot_category_create_group (
+ codings->root, N_("Parameter coding"));
+ for (int i = 0; i < max_df; ++i)
+ pivot_category_create_leaf_rc (
+ coding_group,
+ pivot_value_new_user_text_nocopy (xasprintf ("(%d)", i + 1)),
+ PIVOT_RC_INTEGER);
+
+ struct pivot_dimension *categories = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Categories"));
+
+ int cumulative_df = 0;
+ for (int v = 0; v < cmd->n_cat_predictors; ++v)
+ {
+ int cat;
+ const struct interaction *cat_predictors = cmd->cat_predictors[v];
+ int df = categoricals_df (res->cats, v);
+
+ struct string str = DS_EMPTY_INITIALIZER;
+ interaction_to_string (cat_predictors, &str);
+ struct pivot_category *var_group = pivot_category_create_group__ (
+ categories->root,
+ pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
+
+ for (cat = 0; cat < categoricals_n_count (res->cats, v); ++cat)
+ {
+ const struct ccase *c = categoricals_get_case_by_category_real (
+ res->cats, v, cat);
+ struct string label = DS_EMPTY_INITIALIZER;
+ for (int x = 0; x < cat_predictors->n_vars; ++x)
+ {
+ if (!ds_is_empty (&label))
+ ds_put_byte (&label, ' ');
+
+ const union value *val = case_data (c, cat_predictors->vars[x]);
+ var_append_value_name (cat_predictors->vars[x], val, &label);
+ }
+ int cat_idx = pivot_category_create_leaf (
+ var_group,
+ pivot_value_new_user_text_nocopy (ds_steal_cstr (&label)));
+
+ double *freq = categoricals_get_user_data_by_category_real (
+ res->cats, v, cat);
+ pivot_table_put2 (table, 0, cat_idx, pivot_value_new_number (*freq));
+
+ for (int x = 0; x < df; ++x)
+ pivot_table_put2 (table, x + 1, cat_idx,
+ pivot_value_new_number (cat == x));
+ }
+ cumulative_df += df;
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+create_classification_dimension (const struct lr_spec *cmd,
+ const struct lr_result *res,
+ struct pivot_table *table,
+ enum pivot_axis_type axis_type,
+ const char *label, const char *total)
+{
+ struct pivot_dimension *d = pivot_dimension_create (
+ table, axis_type, label);
+ d->root->show_label = true;
+ struct pivot_category *pred_group = pivot_category_create_group__ (
+ d->root, pivot_value_new_variable (cmd->dep_var));
+ for (int i = 0; i < 2; i++)
+ {
+ const union value *y = i ? &res->y1 : &res->y0;
+ pivot_category_create_leaf_rc (
+ pred_group, pivot_value_new_var_value (cmd->dep_var, y),
+ PIVOT_RC_COUNT);
+ }
+ pivot_category_create_leaves (d->root, total, PIVOT_RC_PERCENT);
+}
+
+static void
+output_classification_table (const struct lr_spec *cmd, const struct lr_result *res)
+{
+ struct pivot_table *table = pivot_table_create (N_("Classification Table"));
+ pivot_table_set_weight_var (table, cmd->wv);
+
+ create_classification_dimension (cmd, res, table, PIVOT_AXIS_COLUMN,
+ N_("Predicted"), N_("Percentage Correct"));
+ create_classification_dimension (cmd, res, table, PIVOT_AXIS_ROW,
+ N_("Observed"), N_("Overall Percentage"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Step"), N_("Step 1"));
+
+ struct entry
+ {
+ int pred_idx;
+ int obs_idx;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, res->tn },
+ { 0, 1, res->fn },
+ { 1, 0, res->fp },
+ { 1, 1, res->tp },
+ { 2, 0, 100 * res->tn / (res->tn + res->fp) },
+ { 2, 1, 100 * res->tp / (res->tp + res->fn) },
+ { 2, 2,
+ 100 * (res->tp + res->tn) / (res->tp + res->tn + res->fp + res->fn)},
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put3 (table, e->pred_idx, e->obs_idx, 0,
+ pivot_value_new_number (e->x));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009-2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <limits.h>
+
+#include "data/case.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/settings.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/inpt-pgm.h"
+#include "language/expressions/public.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+struct loop_trns
+ {
+ /* a=a TO b [BY c]. */
+ struct variable *index_var; /* Index variable. */
+ struct expression *first_expr; /* Starting index. */
+ struct expression *by_expr; /* Index increment (or NULL). */
+ struct expression *last_expr; /* Terminal index. */
+
+ /* IF condition for LOOP or END LOOP. */
+ struct expression *loop_condition;
+ struct expression *end_loop_condition;
+
+ /* Inner transformations. */
+ struct trns_chain xforms;
+
+ /* State. */
+ double cur, by, last; /* Index data. */
+ int iteration; /* For MXLOOPS. */
+ size_t resume_idx; /* For resuming after END CASE. */
+ };
+
+static struct trns_class loop_trns_class;
+
+static int in_loop;
+
+static bool parse_if_clause (struct lexer *, struct dataset *,
+ struct expression **);
+static bool parse_index_clause (struct dataset *, struct lexer *,
+ struct loop_trns *);
+\f
+/* LOOP. */
+
+/* Parses LOOP. */
+int
+cmd_loop (struct lexer *lexer, struct dataset *ds)
+{
+ struct loop_trns *loop = xmalloc (sizeof *loop);
+ *loop = (struct loop_trns) { .resume_idx = SIZE_MAX };
+
+ bool ok = true;
+ while (lex_token (lexer) != T_ENDCMD && ok)
+ {
+ if (lex_match_id (lexer, "IF"))
+ ok = parse_if_clause (lexer, ds, &loop->loop_condition);
+ else
+ ok = parse_index_clause (ds, lexer, loop);
+ }
+ if (ok)
+ lex_end_of_command (lexer);
+ lex_discard_rest_of_command (lexer);
+
+ proc_push_transformations (ds);
+ in_loop++;
+ for (;;)
+ {
+ if (lex_token (lexer) == T_STOP)
+ {
+ lex_error_expecting (lexer, "END LOOP");
+ ok = false;
+ break;
+ }
+ else if (lex_match_phrase (lexer, "END LOOP"))
+ {
+ if (lex_match_id (lexer, "IF"))
+ ok = parse_if_clause (lexer, ds, &loop->end_loop_condition) && ok;
+ break;
+ }
+ else
+ cmd_parse_in_state (lexer, ds,
+ (in_input_program ()
+ ? CMD_STATE_NESTED_INPUT_PROGRAM
+ : CMD_STATE_NESTED_DATA));
+ }
+ in_loop--;
+ proc_pop_transformations (ds, &loop->xforms);
+
+ add_transformation (ds, &loop_trns_class, loop);
+
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+}
+
+int
+cmd_inside_loop (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("This command cannot appear outside LOOP...END LOOP."));
+ return CMD_FAILURE;
+}
+
+static enum trns_result
+break_trns_proc (void *aux UNUSED, struct ccase **c UNUSED,
+ casenumber case_num UNUSED)
+{
+ return TRNS_BREAK;
+}
+
+/* Parses BREAK. */
+int
+cmd_break (struct lexer *lexer, struct dataset *ds)
+{
+ if (!in_loop)
+ {
+ cmd_inside_loop (lexer, ds);
+ return CMD_FAILURE;
+ }
+
+ static const struct trns_class trns_class = {
+ .name = "BREAK",
+ .execute = break_trns_proc
+ };
+ add_transformation (ds, &trns_class, NULL);
+
+ return CMD_SUCCESS;
+}
+
+/* Parses an IF clause for LOOP or END LOOP and stores the
+ resulting expression to *CONDITION.
+ Returns true if successful, false on failure. */
+static bool
+parse_if_clause (struct lexer *lexer, struct dataset *ds,
+ struct expression **condition)
+{
+ if (*condition != NULL)
+ {
+ lex_sbc_only_once (lexer, "IF");
+ return false;
+ }
+
+ *condition = expr_parse_bool (lexer, ds);
+ return *condition != NULL;
+}
+
+/* Parses an indexing clause into LOOP. Returns true if successful, false on
+ failure. */
+static bool
+parse_index_clause (struct dataset *ds, struct lexer *lexer,
+ struct loop_trns *loop)
+{
+ if (loop->index_var != NULL)
+ {
+ lex_error (lexer, _("Only one index clause may be specified."));
+ return false;
+ }
+
+ if (!lex_force_id (lexer))
+ return false;
+
+ loop->index_var = dict_lookup_var (dataset_dict (ds), lex_tokcstr (lexer));
+ if (!loop->index_var)
+ loop->index_var = dict_create_var_assert (dataset_dict (ds),
+ lex_tokcstr (lexer), 0);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_EQUALS))
+ return false;
+
+ loop->first_expr = expr_parse (lexer, ds, VAL_NUMERIC);
+ if (loop->first_expr == NULL)
+ return false;
+
+ for (;;)
+ {
+ struct expression **e;
+ if (lex_match (lexer, T_TO))
+ e = &loop->last_expr;
+ else if (lex_match (lexer, T_BY))
+ e = &loop->by_expr;
+ else
+ break;
+
+ if (*e != NULL)
+ {
+ lex_sbc_only_once (lexer, e == &loop->last_expr ? "TO" : "BY");
+ return false;
+ }
+ *e = expr_parse (lexer, ds, VAL_NUMERIC);
+ if (*e == NULL)
+ return false;
+ }
+ if (loop->last_expr == NULL)
+ {
+ lex_sbc_missing (lexer, "TO");
+ return false;
+ }
+
+ return true;
+}
+
+/* Sets up LOOP for the first pass. */
+static enum trns_result
+loop_trns_proc (void *loop_, struct ccase **c, casenumber case_num)
+{
+ struct loop_trns *loop = loop_;
+
+ size_t start_idx = loop->resume_idx;
+ loop->resume_idx = SIZE_MAX;
+ if (start_idx != SIZE_MAX)
+ goto resume;
+
+ if (loop->index_var)
+ {
+ /* Evaluate loop index expressions. */
+ loop->cur = expr_evaluate_num (loop->first_expr, *c, case_num);
+ loop->by = (loop->by_expr
+ ? expr_evaluate_num (loop->by_expr, *c, case_num)
+ : 1.0);
+ loop->last = expr_evaluate_num (loop->last_expr, *c, case_num);
+
+ /* Even if the loop is never entered, set the index
+ variable to the initial value. */
+ *c = case_unshare (*c);
+ *case_num_rw (*c, loop->index_var) = loop->cur;
+
+ /* Throw out pathological cases. */
+ if (!isfinite (loop->cur)
+ || !isfinite (loop->by)
+ || !isfinite (loop->last)
+ || loop->by == 0.0
+ || (loop->by > 0.0 && loop->cur > loop->last)
+ || (loop->by < 0.0 && loop->cur < loop->last))
+ return TRNS_CONTINUE;
+ }
+
+ for (loop->iteration = 0;
+ loop->index_var || loop->iteration < settings_get_mxloops ();
+ loop->iteration++)
+ {
+ if (loop->loop_condition
+ && expr_evaluate_num (loop->loop_condition, *c, case_num) != 1.0)
+ break;
+
+ start_idx = 0;
+ resume:
+ for (size_t i = start_idx; i < loop->xforms.n; i++)
+ {
+ const struct transformation *trns = &loop->xforms.xforms[i];
+ enum trns_result r = trns->class->execute (trns->aux, c, case_num);
+ switch (r)
+ {
+ case TRNS_CONTINUE:
+ break;
+
+ case TRNS_BREAK:
+ return TRNS_CONTINUE;
+
+ case TRNS_END_CASE:
+ loop->resume_idx = i;
+ return TRNS_END_CASE;
+
+ case TRNS_ERROR:
+ case TRNS_END_FILE:
+ return r;
+
+ case TRNS_DROP_CASE:
+ NOT_REACHED ();
+ }
+ }
+
+ if (loop->end_loop_condition != NULL
+ && expr_evaluate_num (loop->end_loop_condition, *c, case_num) != 0.0)
+ break;
+
+ if (loop->index_var)
+ {
+ loop->cur += loop->by;
+ if (loop->by > 0.0 ? loop->cur > loop->last : loop->cur < loop->last)
+ break;
+
+ *c = case_unshare (*c);
+ *case_num_rw (*c, loop->index_var) = loop->cur;
+ }
+ }
+ return TRNS_CONTINUE;
+}
+
+/* Frees LOOP. */
+static bool
+loop_trns_free (void *loop_)
+{
+ struct loop_trns *loop = loop_;
+
+ expr_free (loop->first_expr);
+ expr_free (loop->by_expr);
+ expr_free (loop->last_expr);
+
+ expr_free (loop->loop_condition);
+ expr_free (loop->end_loop_condition);
+
+ trns_chain_uninit (&loop->xforms);
+
+ free (loop);
+ return true;
+}
+
+static struct trns_class loop_trns_class = {
+ .name = "LOOP",
+ .execute = loop_trns_proc,
+ .destroy = loop_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/mann-whitney.h"
+
+#include <gsl/gsl_cdf.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "libpspp/cast.h"
+#include "libpspp/misc.h"
+#include "math/sort.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+/* Calculates the adjustment necessary for tie compensation */
+static void
+distinct_callback (double v UNUSED, casenumber t, double w UNUSED, void *aux)
+{
+ double *tiebreaker = aux;
+
+ *tiebreaker += (pow3 (t) - t) / 12.0;
+}
+
+struct mw
+{
+ double rank_sum[2];
+ double n[2];
+
+ double u; /* The Mann-Whitney U statistic */
+ double w; /* The Wilcoxon Rank Sum W statistic */
+ double z;
+};
+
+static void show_ranks_box (const struct n_sample_test *, const struct mw *);
+static void show_statistics_box (const struct n_sample_test *,
+ const struct mw *);
+
+
+
+static bool
+belongs_to_test (const struct ccase *c, void *aux)
+{
+ const struct n_sample_test *nst = aux;
+
+ const union value *group = case_data (c, nst->indep_var);
+ const size_t group_var_width = var_get_width (nst->indep_var);
+
+ if (value_equal (group, &nst->val1, group_var_width))
+ return true;
+
+ if (value_equal (group, &nst->val2, group_var_width))
+ return true;
+
+ return false;
+}
+
+
+
+void
+mann_whitney_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ int i;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test, parent);
+
+ const struct caseproto *proto = casereader_get_proto (input);
+ size_t rank_idx = caseproto_get_n_widths (proto);
+
+ struct mw *mw = XCALLOC (nst->n_vars, struct mw);
+
+ for (i = 0; i < nst->n_vars; ++i)
+ {
+ double tiebreaker = 0.0;
+ bool warn = true;
+ enum rank_error rerr = 0;
+ struct casereader *rr;
+ struct ccase *c;
+ const struct variable *var = nst->vars[i];
+
+ struct casereader *reader =
+ casereader_create_filter_func (casereader_clone (input),
+ belongs_to_test,
+ NULL,
+ CONST_CAST (struct n_sample_test *, nst),
+ NULL);
+
+ reader = casereader_create_filter_missing (reader, &var, 1,
+ exclude,
+ NULL, NULL);
+
+ reader = sort_execute_1var (reader, var);
+
+ rr = casereader_create_append_rank (reader, var,
+ dict_get_weight (dict),
+ &rerr,
+ distinct_callback, &tiebreaker);
+
+ for (; (c = casereader_read (rr)); case_unref (c))
+ {
+ const union value *group = case_data (c, nst->indep_var);
+ const size_t group_var_width = var_get_width (nst->indep_var);
+ const double rank = case_num_idx (c, rank_idx);
+
+ if (value_equal (group, &nst->val1, group_var_width))
+ {
+ mw[i].rank_sum[0] += rank;
+ mw[i].n[0] += dict_get_case_weight (dict, c, &warn);
+ }
+ else if (value_equal (group, &nst->val2, group_var_width))
+ {
+ mw[i].rank_sum[1] += rank;
+ mw[i].n[1] += dict_get_case_weight (dict, c, &warn);
+ }
+ }
+ casereader_destroy (rr);
+
+ {
+ double n;
+ double denominator;
+ struct mw *mwv = &mw[i];
+
+ mwv->u = mwv->n[0] * mwv->n[1] ;
+ mwv->u += mwv->n[0] * (mwv->n[0] + 1) / 2.0;
+ mwv->u -= mwv->rank_sum[0];
+
+ mwv->w = mwv->rank_sum[1];
+ if (mwv->u > mwv->n[0] * mwv->n[1] / 2.0)
+ {
+ mwv->u = mwv->n[0] * mwv->n[1] - mwv->u;
+ mwv->w = mwv->rank_sum[0];
+ }
+ mwv->z = mwv->u - mwv->n[0] * mwv->n[1] / 2.0;
+ n = mwv->n[0] + mwv->n[1];
+ denominator = pow3(n) - n;
+ denominator /= 12;
+ denominator -= tiebreaker;
+ denominator *= mwv->n[0] * mwv->n[1];
+ denominator /= n * (n - 1);
+
+ mwv->z /= sqrt (denominator);
+ }
+ }
+ casereader_destroy (input);
+
+ show_ranks_box (nst, mw);
+ show_statistics_box (nst, mw);
+
+ free (mw);
+}
+\f
+static void
+show_ranks_box (const struct n_sample_test *nst, const struct mw *mwv)
+{
+ struct pivot_table *table = pivot_table_create (N_("Ranks"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Mean Rank"), PIVOT_RC_OTHER,
+ N_("Sum of Ranks"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *indep = pivot_dimension_create__ (
+ table, PIVOT_AXIS_ROW, pivot_value_new_variable (nst->indep_var));
+ pivot_category_create_leaf (indep->root,
+ pivot_value_new_var_value (nst->indep_var,
+ &nst->val1));
+ pivot_category_create_leaf (indep->root,
+ pivot_value_new_var_value (nst->indep_var,
+ &nst->val2));
+ pivot_category_create_leaves (indep->root, N_("Total"));
+
+ struct pivot_dimension *dep = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ for (size_t i = 0 ; i < nst->n_vars ; ++i)
+ {
+ const struct mw *mw = &mwv[i];
+
+ int dep_idx = pivot_category_create_leaf (
+ dep->root, pivot_value_new_variable (nst->vars[i]));
+
+ struct entry
+ {
+ int stat_idx;
+ int indep_idx;
+ double x;
+ }
+ entries[] = {
+ /* N. */
+ { 0, 0, mw->n[0] },
+ { 0, 1, mw->n[1] },
+ { 0, 2, mw->n[0] + mw->n[1] },
+
+ /* Mean Rank. */
+ { 1, 0, mw->rank_sum[0] / mw->n[0] },
+ { 1, 1, mw->rank_sum[1] / mw->n[1] },
+
+ /* Sum of Ranks. */
+ { 2, 0, mw->rank_sum[0] },
+ { 2, 1, mw->rank_sum[1] },
+ };
+
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ {
+ const struct entry *e = &entries[j];
+ pivot_table_put3 (table, e->stat_idx, e->indep_idx, dep_idx,
+ pivot_value_new_number (e->x));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_statistics_box (const struct n_sample_test *nst, const struct mw *mwv)
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ _("Mann-Whitney U"), PIVOT_RC_OTHER,
+ _("Wilcoxon W"), PIVOT_RC_OTHER,
+ _("Z"), PIVOT_RC_OTHER,
+ _("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0 ; i < nst->n_vars ; ++i)
+ {
+ const struct mw *mw = &mwv[i];
+
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (nst->vars[i]));
+
+ double entries[] = {
+ mw->u,
+ mw->w,
+ mw->z,
+ 2.0 * gsl_cdf_ugaussian_P (mw->z),
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !mann_whitney_h
+#define mann_whitney_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+
+struct mann_whitney_test
+{
+ struct two_sample_test parent;
+};
+
+struct casereader;
+struct dataset;
+
+void mann_whitney_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer
+ );
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2017 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_matrix.h>
+#include <gsl/gsl_vector.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/data-in.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/short-names.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/data-parser.h"
+#include "language/commands/data-reader.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/inpt-pgm.h"
+#include "language/commands/placement-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/i18n.h"
+#include "libpspp/intern.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/c-ctype.h"
+#include "gl/minmax.h"
+#include "gl/xsize.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+\f
+#define ROWTYPES \
+ /* Matrix row types. */ \
+ RT(CORR, 2) \
+ RT(COV, 2) \
+ RT(MAT, 2) \
+ RT(N_MATRIX, 2) \
+ RT(PROX, 2) \
+ \
+ /* Vector row types. */ \
+ RT(COUNT, 1) \
+ RT(DFE, 1) \
+ RT(MEAN, 1) \
+ RT(MSE, 1) \
+ RT(STDDEV, 1) \
+ RT(N, 1) \
+ \
+ /* Scalar row types. */ \
+ RT(N_SCALAR, 0)
+
+enum rowtype
+ {
+#define RT(NAME, DIMS) C_##NAME,
+ ROWTYPES
+#undef RT
+ };
+
+enum
+ {
+#define RT(NAME, DIMS) +1
+ N_ROWTYPES = ROWTYPES
+#undef RT
+ };
+verify (N_ROWTYPES < 32);
+
+/* Returns the number of dimensions in the indexes for row type RT. A matrix
+ has 2 dimensions, a vector has 1, a scalar has 0. */
+static int
+rowtype_dimensions (enum rowtype rt)
+{
+ static const int rowtype_dims[N_ROWTYPES] = {
+#define RT(NAME, DIMS) [C_##NAME] = DIMS,
+ ROWTYPES
+#undef RT
+ };
+ return rowtype_dims[rt];
+}
+
+static struct substring
+rowtype_name (enum rowtype rt)
+{
+ static const struct substring rowtype_names[N_ROWTYPES] = {
+#define RT(NAME, DIMS) [C_##NAME] = SS_LITERAL_INITIALIZER (#NAME),
+ ROWTYPES
+#undef RT
+ };
+
+ return rowtype_names[rt];
+}
+
+static bool
+rowtype_from_string (struct substring token, enum rowtype *rt)
+{
+ ss_trim (&token, ss_cstr (CC_SPACES));
+ for (size_t i = 0; i < N_ROWTYPES; i++)
+ if (lex_id_match (rowtype_name (i), token))
+ {
+ *rt = i;
+ return true;
+ }
+
+ if (lex_id_match (ss_cstr ("N_VECTOR"), token))
+ {
+ *rt = C_N;
+ return true;
+ }
+ else if (lex_id_match (ss_cstr ("SD"), token))
+ {
+ *rt = C_STDDEV;
+ return true;
+ }
+
+ return false;
+}
+
+static bool
+rowtype_parse (struct lexer *lexer, enum rowtype *rt)
+{
+ bool parsed = (lex_token (lexer) == T_ID
+ && rowtype_from_string (lex_tokss (lexer), rt));
+ if (parsed)
+ lex_get (lexer);
+ return parsed;
+}
+\f
+struct matrix_format
+ {
+ bool span;
+ enum triangle
+ {
+ LOWER,
+ UPPER,
+ FULL
+ }
+ triangle;
+ enum diagonal
+ {
+ DIAGONAL,
+ NO_DIAGONAL
+ }
+ diagonal;
+
+ bool input_rowtype;
+ struct variable **input_vars;
+ size_t n_input_vars;
+
+ /* How to read matrices with each possible number of dimensions (0=scalar,
+ 1=vector, 2=matrix). */
+ struct matrix_sched
+ {
+ /* Number of rows and columns in the matrix: (1,1) for a scalar, (1,n) for
+ a vector, (n,n) for a matrix. */
+ int nr, nc;
+
+ /* Rows of data to read and the number of columns in each. Because we
+ often read just a triangle and sometimes omit the diagonal, 'n_rp' can
+ be less than 'nr' and 'rp[i]->y' isn't always 'y'. */
+ struct row_sched
+ {
+ /* The y-value of the row inside the matrix. */
+ int y;
+
+ /* first and last (exclusive) columns to read in this row. */
+ int x0, x1;
+ }
+ *rp;
+ size_t n_rp;
+ }
+ ms[3];
+
+ struct variable *rowtype;
+ struct variable *varname;
+ struct variable **cvars;
+ int n_cvars;
+ struct variable **svars;
+ size_t *svar_indexes;
+ size_t n_svars;
+ struct variable **fvars;
+ size_t *fvar_indexes;
+ size_t n_fvars;
+ int cells;
+ int n;
+
+ unsigned int pooled_rowtype_mask;
+ unsigned int factor_rowtype_mask;
+
+ struct content
+ {
+ bool open;
+ enum rowtype rowtype;
+ bool close;
+ }
+ *contents;
+ size_t n_contents;
+ };
+
+static void
+matrix_format_uninit (struct matrix_format *mf)
+{
+ free (mf->input_vars);
+ for (int i = 0; i < 3; i++)
+ free (mf->ms[i].rp);
+ free (mf->cvars);
+ free (mf->svars);
+ free (mf->svar_indexes);
+ free (mf->fvars);
+ free (mf->fvar_indexes);
+ free (mf->contents);
+}
+
+static void
+set_string (struct ccase *outcase, const struct variable *var,
+ struct substring src)
+{
+ struct substring dst = case_ss (outcase, var);
+ for (size_t i = 0; i < dst.length; i++)
+ dst.string[i] = i < src.length ? src.string[i] : ' ';
+}
+
+static void
+parse_msg (struct dfm_reader *reader, const struct substring *token,
+ char *text, enum msg_severity severity)
+{
+ int first_column = 0;
+ if (token)
+ {
+ struct substring line = dfm_get_record (reader);
+ if (token->string >= line.string && token->string < ss_end (line))
+ first_column = ss_pointer_to_position (line, token->string) + 1;
+ }
+
+ int line_number = dfm_get_line_number (reader);
+ struct msg_location *location = xmalloc (sizeof *location);
+ int last_column = (first_column && token->length
+ ? first_column + token->length - 1
+ : 0);
+ *location = (struct msg_location) {
+ .file_name = intern_new (dfm_get_file_name (reader)),
+ .start = { .line = line_number, .column = first_column },
+ .end = { .line = line_number, .column = last_column },
+ };
+ struct msg *m = xmalloc (sizeof *m);
+ *m = (struct msg) {
+ .category = MSG_C_DATA,
+ .severity = severity,
+ .location = location,
+ .text = text,
+ };
+ msg_emit (m);
+}
+
+static void PRINTF_FORMAT (3, 4)
+parse_warning (struct dfm_reader *reader, const struct substring *token,
+ const char *format, ...)
+{
+ va_list args;
+ va_start (args, format);
+ parse_msg (reader, token, xvasprintf (format, args), MSG_S_WARNING);
+ va_end (args);
+}
+
+static void PRINTF_FORMAT (3, 4)
+parse_error (struct dfm_reader *reader, const struct substring *token,
+ const char *format, ...)
+{
+ va_list args;
+ va_start (args, format);
+ parse_msg (reader, token, xvasprintf (format, args), MSG_S_ERROR);
+ va_end (args);
+}
+
+/* Advance to beginning of next token. */
+static bool
+more_tokens (struct substring *p, struct dfm_reader *r)
+{
+ for (;;)
+ {
+ ss_ltrim (p, ss_cstr (CC_SPACES ","));
+ if (p->length)
+ return true;
+
+ dfm_forward_record (r);
+ if (dfm_eof (r))
+ return false;
+ *p = dfm_get_record (r);
+ }
+}
+
+static bool
+next_token (struct substring *p, struct dfm_reader *r, struct substring *token)
+{
+ if (!more_tokens (p, r))
+ return false;
+
+ /* Collect token. */
+ int c = ss_first (*p);
+ if (c == '\'' || c == '"')
+ {
+ ss_advance (p, 1);
+ ss_get_until (p, c, token);
+ }
+ else
+ {
+ size_t n = 1;
+ for (;;)
+ {
+ c = ss_at (*p, n);
+ if (c == EOF
+ || ss_find_byte (ss_cstr (CC_SPACES ","), c) != SIZE_MAX
+ || ((c == '+' || c == '-')
+ && ss_find_byte (ss_cstr ("dDeE"),
+ ss_at (*p, n - 1)) == SIZE_MAX))
+ break;
+ n++;
+ }
+ ss_get_bytes (p, n, token);
+ }
+ return true;
+}
+
+static bool
+next_number (struct substring *p, struct dfm_reader *r, double *d)
+{
+ struct substring token;
+ if (!next_token (p, r, &token))
+ return false;
+
+ union value v;
+ char *error = data_in (token, dfm_reader_get_encoding (r), FMT_F,
+ settings_get_fmt_settings (), &v, 0, NULL);
+ if (error)
+ {
+ parse_error (r, &token, "%s", error);
+ free (error);
+ }
+ *d = v.f;
+ return true;
+}
+
+static bool
+next_rowtype (struct substring *p, struct dfm_reader *r, enum rowtype *rt)
+{
+ struct substring token;
+ if (!next_token (p, r, &token))
+ return false;
+
+ if (rowtype_from_string (token, rt))
+ return true;
+
+ parse_error (r, &token, _("Unknown row type \"%.*s\"."),
+ (int) token.length, token.string);
+ return false;
+}
+
+struct read_matrix_params
+ {
+ /* Adjustments to first and last row to read. */
+ int dy0, dy1;
+
+ /* Left and right columns to read in first row, inclusive.
+ For x1, INT_MAX is the rightmost column. */
+ int x0, x1;
+
+ /* Adjustment to x0 and x1 for each subsequent row we read. Each of these
+ is 0 to keep it the same or -1 or +1 to adjust it by that much. */
+ int dx0, dx1;
+ };
+
+static const struct read_matrix_params *
+get_read_matrix_params (const struct matrix_format *mf)
+{
+ if (mf->triangle == FULL)
+ {
+ /* 1 2 3 4
+ 2 1 5 6
+ 3 5 1 7
+ 4 6 7 1 */
+ static const struct read_matrix_params rmp = { 0, 0, 0, INT_MAX, 0, 0 };
+ return &rmp;
+ }
+ else if (mf->triangle == LOWER)
+ {
+ if (mf->diagonal == DIAGONAL)
+ {
+ /* 1 . . .
+ 2 1 . .
+ 3 5 1 .
+ 4 6 7 1 */
+ static const struct read_matrix_params rmp = { 0, 0, 0, 0, 0, 1 };
+ return &rmp;
+ }
+ else
+ {
+ /* . . . .
+ 2 . . .
+ 3 5 . .
+ 4 6 7 . */
+ static const struct read_matrix_params rmp = { 1, 0, 0, 0, 0, 1 };
+ return &rmp;
+ }
+ }
+ else if (mf->triangle == UPPER)
+ {
+ if (mf->diagonal == DIAGONAL)
+ {
+ /* 1 2 3 4
+ . 1 5 6
+ . . 1 7
+ . . . 1 */
+ static const struct read_matrix_params rmp = { 0, 0, 0, INT_MAX, 1, 0 };
+ return &rmp;
+ }
+ else
+ {
+ /* . 2 3 4
+ . . 5 6
+ . . . 7
+ . . . . */
+ static const struct read_matrix_params rmp = { 0, -1, 1, INT_MAX, 1, 0 };
+ return &rmp;
+ }
+ }
+ else
+ NOT_REACHED ();
+}
+
+static void
+schedule_matrices (struct matrix_format *mf)
+{
+ struct matrix_sched *ms0 = &mf->ms[0];
+ ms0->nr = 1;
+ ms0->nc = 1;
+ ms0->rp = xmalloc (sizeof *ms0->rp);
+ ms0->rp[0] = (struct row_sched) { .y = 0, .x0 = 0, .x1 = 1 };
+ ms0->n_rp = 1;
+
+ struct matrix_sched *ms1 = &mf->ms[1];
+ ms1->nr = 1;
+ ms1->nc = mf->n_cvars;
+ ms1->rp = xmalloc (sizeof *ms1->rp);
+ ms1->rp[0] = (struct row_sched) { .y = 0, .x0 = 0, .x1 = mf->n_cvars };
+ ms1->n_rp = 1;
+
+ struct matrix_sched *ms2 = &mf->ms[2];
+ ms2->nr = mf->n_cvars;
+ ms2->nc = mf->n_cvars;
+ ms2->rp = xmalloc (mf->n_cvars * sizeof *ms2->rp);
+ ms2->n_rp = 0;
+
+ const struct read_matrix_params *rmp = get_read_matrix_params (mf);
+ int x0 = rmp->x0;
+ int x1 = rmp->x1 < mf->n_cvars ? rmp->x1 : mf->n_cvars - 1;
+ int y0 = rmp->dy0;
+ int y1 = (int) mf->n_cvars + rmp->dy1;
+ for (int y = y0; y < y1; y++)
+ {
+ assert (x0 >= 0 && x0 < mf->n_cvars);
+ assert (x1 >= 0 && x1 < mf->n_cvars);
+ assert (x1 >= x0);
+
+ ms2->rp[ms2->n_rp++] = (struct row_sched) {
+ .y = y, .x0 = x0, .x1 = x1 + 1
+ };
+
+ x0 += rmp->dx0;
+ x1 += rmp->dx1;
+ }
+}
+
+static bool
+read_id_columns (const struct matrix_format *mf,
+ struct substring *p, struct dfm_reader *r,
+ double *d, enum rowtype *rt)
+{
+ for (size_t i = 0; mf->input_vars[i] != mf->cvars[0]; i++)
+ if (!(mf->input_vars[i] == mf->rowtype
+ ? next_rowtype (p, r, rt)
+ : next_number (p, r, &d[i])))
+ return false;
+ return true;
+}
+
+static bool
+equal_id_columns (const struct matrix_format *mf,
+ const double *a, const double *b)
+{
+ for (size_t i = 0; mf->input_vars[i] != mf->cvars[0]; i++)
+ if (mf->input_vars[i] != mf->rowtype && a[i] != b[i])
+ return false;
+ return true;
+}
+
+static bool
+equal_split_columns (const struct matrix_format *mf,
+ const double *a, const double *b)
+{
+ for (size_t i = 0; i < mf->n_svars; i++)
+ {
+ size_t idx = mf->svar_indexes[i];
+ if (a[idx] != b[idx])
+ return false;
+ }
+ return true;
+}
+
+static bool
+is_pooled (const struct matrix_format *mf, const double *d)
+{
+ for (size_t i = 0; i < mf->n_fvars; i++)
+ if (d[mf->fvar_indexes[i]] != SYSMIS)
+ return false;
+ return true;
+}
+
+static void
+matrix_sched_init (const struct matrix_format *mf, enum rowtype rt,
+ gsl_matrix *m)
+{
+ int n_dims = rowtype_dimensions (rt);
+ const struct matrix_sched *ms = &mf->ms[n_dims];
+ double diagonal = n_dims < 2 || rt != C_CORR ? SYSMIS : 1.0;
+ for (size_t y = 0; y < ms->nr; y++)
+ for (size_t x = 0; x < ms->nc; x++)
+ gsl_matrix_set (m, y, x, y == x ? diagonal : SYSMIS);
+}
+
+static void
+matrix_sched_output (const struct matrix_format *mf, enum rowtype rt,
+ gsl_matrix *m, const double *d, int split_num,
+ struct casewriter *w)
+{
+ int n_dims = rowtype_dimensions (rt);
+ const struct matrix_sched *ms = &mf->ms[n_dims];
+
+ if (rt == C_N_SCALAR)
+ {
+ for (size_t x = 1; x < mf->n_cvars; x++)
+ gsl_matrix_set (m, 0, x, gsl_matrix_get (m, 0, 0));
+ rt = C_N;
+ }
+
+ for (int y = 0; y < ms->nr; y++)
+ {
+ struct ccase *c = case_create (casewriter_get_proto (w));
+ for (size_t i = 0; mf->input_vars[i] != mf->cvars[0]; i++)
+ if (mf->input_vars[i] != mf->rowtype)
+ *case_num_rw (c, mf->input_vars[i]) = d[i];
+ if (mf->n_svars && !mf->svar_indexes)
+ *case_num_rw (c, mf->svars[0]) = split_num;
+ set_string (c, mf->rowtype, rowtype_name (rt));
+ const char *varname = n_dims == 2 ? var_get_name (mf->cvars[y]) : "";
+ set_string (c, mf->varname, ss_cstr (varname));
+ for (int x = 0; x < mf->n_cvars; x++)
+ *case_num_rw (c, mf->cvars[x]) = gsl_matrix_get (m, y, x);
+ casewriter_write (w, c);
+ }
+}
+
+static void
+matrix_sched_output_n (const struct matrix_format *mf, double n,
+ gsl_matrix *m, const double *d, int split_num,
+ struct casewriter *w)
+{
+ gsl_matrix_set (m, 0, 0, n);
+ matrix_sched_output (mf, C_N_SCALAR, m, d, split_num, w);
+}
+
+static void
+check_eol (const struct matrix_format *mf, struct substring *p,
+ struct dfm_reader *r)
+{
+ if (!mf->span)
+ {
+ ss_ltrim (p, ss_cstr (CC_SPACES ","));
+ if (p->length)
+ {
+ parse_error (r, p, _("Extraneous data expecting end of line."));
+ p->length = 0;
+ }
+ }
+}
+
+static void
+parse_data_with_rowtype (const struct matrix_format *mf,
+ struct dfm_reader *r, struct casewriter *w)
+{
+ if (dfm_eof (r))
+ return;
+ struct substring p = dfm_get_record (r);
+
+ double *prev = NULL;
+ gsl_matrix *m = gsl_matrix_alloc (mf->n_cvars, mf->n_cvars);
+
+ double *d = xnmalloc (mf->n_input_vars, sizeof *d);
+ enum rowtype rt;
+
+ double *d_next = xnmalloc (mf->n_input_vars, sizeof *d_next);
+
+ if (!read_id_columns (mf, &p, r, d, &rt))
+ goto exit;
+ for (;;)
+ {
+ /* If this has rowtype N but there was an N subcommand, then the
+ subcommand takes precedence, so we will suppress outputting this
+ record. We still need to parse it, though, so we can't skip other
+ work. */
+ bool suppress_output = mf->n >= 0 && (rt == C_N || rt == C_N_SCALAR);
+ if (suppress_output)
+ parse_error (r, NULL, _("N record is not allowed with N subcommand. "
+ "Ignoring N record."));
+
+ /* If there's an N subcommand, and this is a new split, then output an N
+ record. */
+ if (mf->n >= 0 && (!prev || !equal_split_columns (mf, prev, d)))
+ {
+ matrix_sched_output_n (mf, mf->n, m, d, 0, w);
+
+ if (!prev)
+ prev = xnmalloc (mf->n_input_vars, sizeof *prev);
+ memcpy (prev, d, mf->n_input_vars * sizeof *prev);
+ }
+
+ /* Usually users don't provide the CONTENTS subcommand with ROWTYPE_, but
+ if they did then warn if ROWTYPE_ is an unexpected type. */
+ if (mf->factor_rowtype_mask || mf->pooled_rowtype_mask)
+ {
+ const char *name = rowtype_name (rt).string;
+ if (is_pooled (mf, d))
+ {
+ if (!((1u << rt) & mf->pooled_rowtype_mask))
+ parse_warning (r, NULL, _("Data contains pooled row type %s not "
+ "included in CONTENTS."), name);
+ }
+ else
+ {
+ if (!((1u << rt) & mf->factor_rowtype_mask))
+ parse_warning (r, NULL, _("Data contains with-factors row type "
+ "%s not included in CONTENTS."), name);
+ }
+ }
+
+ /* Initialize the matrix to be filled-in. */
+ int n_dims = rowtype_dimensions (rt);
+ const struct matrix_sched *ms = &mf->ms[n_dims];
+ matrix_sched_init (mf, rt, m);
+
+ enum rowtype rt_next;
+ bool eof;
+
+ size_t n_rows;
+ for (n_rows = 1; ; n_rows++)
+ {
+ if (n_rows <= ms->n_rp)
+ {
+ const struct row_sched *rs = &ms->rp[n_rows - 1];
+ size_t y = rs->y;
+ for (size_t x = rs->x0; x < rs->x1; x++)
+ {
+ double e;
+ if (!next_number (&p, r, &e))
+ goto exit;
+ gsl_matrix_set (m, y, x, e);
+ if (n_dims == 2 && mf->triangle != FULL)
+ gsl_matrix_set (m, x, y, e);
+ }
+ check_eol (mf, &p, r);
+ }
+ else
+ {
+ /* Suppress bad input data. We'll issue an error later. */
+ p.length = 0;
+ }
+
+ eof = (!more_tokens (&p, r)
+ || !read_id_columns (mf, &p, r, d_next, &rt_next));
+ if (eof)
+ break;
+
+ if (!equal_id_columns (mf, d, d_next) || rt_next != rt)
+ break;
+ }
+ if (!suppress_output)
+ matrix_sched_output (mf, rt, m, d, 0, w);
+
+ if (n_rows != ms->n_rp)
+ parse_error (r, NULL,
+ _("Matrix %s had %zu rows but %zu rows were expected."),
+ rowtype_name (rt).string, n_rows, ms->n_rp);
+ if (eof)
+ break;
+
+ double *d_tmp = d;
+ d = d_next;
+ d_next = d_tmp;
+
+ rt = rt_next;
+ }
+
+exit:
+ free (prev);
+ gsl_matrix_free (m);
+ free (d);
+ free (d_next);
+}
+
+static void
+parse_matrix_without_rowtype (const struct matrix_format *mf,
+ struct substring *p, struct dfm_reader *r,
+ gsl_matrix *m, enum rowtype rowtype, bool pooled,
+ int split_num, struct casewriter *w)
+{
+ int n_dims = rowtype_dimensions (rowtype);
+ const struct matrix_sched *ms = &mf->ms[n_dims];
+
+ double *d = xnmalloc (mf->n_input_vars, sizeof *d);
+ matrix_sched_init (mf, rowtype, m);
+ for (size_t i = 0; i < ms->n_rp; i++)
+ {
+ int y = ms->rp[i].y;
+ int k = 0;
+ int h = 0;
+ for (size_t j = 0; j < mf->n_input_vars; j++)
+ {
+ const struct variable *iv = mf->input_vars[j];
+ if (k < mf->n_cvars && iv == mf->cvars[k])
+ {
+ if (k < ms->rp[i].x1 - ms->rp[i].x0)
+ {
+ double e;
+ if (!next_number (p, r, &e))
+ goto exit;
+
+ int x = k + ms->rp[i].x0;
+ gsl_matrix_set (m, y, x, e);
+ if (n_dims == 2 && mf->triangle != FULL)
+ gsl_matrix_set (m, x, y, e);
+ }
+ k++;
+ continue;
+ }
+ if (h < mf->n_fvars && iv == mf->fvars[h])
+ {
+ h++;
+ if (pooled)
+ {
+ d[j] = SYSMIS;
+ continue;
+ }
+ }
+
+ double e;
+ if (!next_number (p, r, &e))
+ goto exit;
+ d[j] = e;
+ }
+ check_eol (mf, p, r);
+ }
+
+ matrix_sched_output (mf, rowtype, m, d, split_num, w);
+exit:
+ free (d);
+}
+
+static void
+parse_data_without_rowtype (const struct matrix_format *mf,
+ struct dfm_reader *r, struct casewriter *w)
+{
+ if (dfm_eof (r))
+ return;
+ struct substring p = dfm_get_record (r);
+
+ gsl_matrix *m = gsl_matrix_alloc (mf->n_cvars, mf->n_cvars);
+
+ int split_num = 1;
+ do
+ {
+ for (size_t i = 0; i < mf->n_contents; )
+ {
+ size_t j = i;
+ if (mf->contents[i].open)
+ while (!mf->contents[j].close)
+ j++;
+
+ if (mf->contents[i].open)
+ {
+ for (size_t k = 0; k < mf->cells; k++)
+ for (size_t h = i; h <= j; h++)
+ parse_matrix_without_rowtype (mf, &p, r, m,
+ mf->contents[h].rowtype, false,
+ split_num, w);
+ }
+ else
+ parse_matrix_without_rowtype (mf, &p, r, m, mf->contents[i].rowtype,
+ true, split_num, w);
+ i = j + 1;
+ }
+
+ split_num++;
+ }
+ while (more_tokens (&p, r));
+
+ gsl_matrix_free (m);
+}
+
+/* Parses VARIABLES=varnames for MATRIX DATA and returns a dictionary with the
+ named variables in it. */
+static struct dictionary *
+parse_matrix_data_variables (struct lexer *lexer)
+{
+ if (!lex_force_match_id (lexer, "VARIABLES"))
+ return NULL;
+ lex_match (lexer, T_EQUALS);
+
+ struct dictionary *dict = dict_create (get_default_encoding ());
+
+ size_t n_names = 0;
+ char **names = NULL;
+ int vars_start = lex_ofs (lexer);
+ if (!parse_DATA_LIST_vars (lexer, dict, &names, &n_names, PV_NO_DUPLICATE))
+ {
+ dict_unref (dict);
+ return NULL;
+ }
+ int vars_end = lex_ofs (lexer) - 1;
+
+ for (size_t i = 0; i < n_names; i++)
+ if (!strcasecmp (names[i], "ROWTYPE_"))
+ dict_create_var_assert (dict, "ROWTYPE_", 8);
+ else
+ {
+ struct variable *var = dict_create_var_assert (dict, names[i], 0);
+ var_set_measure (var, MEASURE_SCALE);
+ }
+
+ for (size_t i = 0; i < n_names; ++i)
+ free (names[i]);
+ free (names);
+
+ if (dict_lookup_var (dict, "VARNAME_"))
+ {
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("VARIABLES may not include VARNAME_."));
+ dict_unref (dict);
+ return NULL;
+ }
+ return dict;
+}
+
+static bool
+parse_matrix_data_subvars (struct lexer *lexer, struct dictionary *dict,
+ bool *taken_vars,
+ struct variable ***vars, size_t **indexes,
+ size_t *n_vars)
+{
+ int start_ofs = lex_ofs (lexer);
+ if (!parse_variables (lexer, dict, vars, n_vars, 0))
+ return false;
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ *indexes = xnmalloc (*n_vars, sizeof **indexes);
+ for (size_t i = 0; i < *n_vars; i++)
+ {
+ struct variable *v = (*vars)[i];
+ if (!strcasecmp (var_get_name (v), "ROWTYPE_"))
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("ROWTYPE_ is not allowed on SPLIT or FACTORS."));
+ goto error;
+ }
+ (*indexes)[i] = var_get_dict_index (v);
+
+ bool *tv = &taken_vars[var_get_dict_index (v)];
+ if (*tv)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("%s may not appear on both SPLIT and FACTORS."),
+ var_get_name (v));
+ goto error;
+ }
+ *tv = true;
+
+ var_set_measure (v, MEASURE_NOMINAL);
+ var_set_both_formats (v, &(struct fmt_spec) { .type = FMT_F, .w = 4 });
+ }
+ return true;
+
+error:
+ free (*vars);
+ *vars = NULL;
+ *n_vars = 0;
+ free (*indexes);
+ *indexes = NULL;
+ return false;
+}
+
+int
+cmd_matrix_data (struct lexer *lexer, struct dataset *ds)
+{
+ int input_vars_start = lex_ofs (lexer);
+ struct dictionary *dict = parse_matrix_data_variables (lexer);
+ if (!dict)
+ return CMD_FAILURE;
+ int input_vars_end = lex_ofs (lexer) - 1;
+
+ size_t n_input_vars = dict_get_n_vars (dict);
+ struct variable **input_vars = xnmalloc (n_input_vars, sizeof *input_vars);
+ for (size_t i = 0; i < n_input_vars; i++)
+ input_vars[i] = dict_get_var (dict, i);
+
+ int varname_width = 8;
+ for (size_t i = 0; i < n_input_vars; i++)
+ {
+ int w = strlen (var_get_name (input_vars[i]));
+ varname_width = MAX (w, varname_width);
+ }
+
+ struct variable *rowtype = dict_lookup_var (dict, "ROWTYPE_");
+ bool input_rowtype = rowtype != NULL;
+ if (!rowtype)
+ rowtype = dict_create_var_assert (dict, "ROWTYPE_", 8);
+
+ struct matrix_format mf = {
+ .input_rowtype = input_rowtype,
+ .input_vars = input_vars,
+ .n_input_vars = n_input_vars,
+
+ .rowtype = rowtype,
+ .varname = dict_create_var_assert (dict, "VARNAME_", varname_width),
+
+ .triangle = LOWER,
+ .diagonal = DIAGONAL,
+ .n = -1,
+ .cells = -1,
+ };
+
+ bool *taken_vars = XCALLOC (n_input_vars, bool);
+ if (input_rowtype)
+ taken_vars[var_get_dict_index (rowtype)] = true;
+
+ struct file_handle *fh = NULL;
+ int n_start = 0;
+ int n_end = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+
+ if (lex_match_id (lexer, "N"))
+ {
+ n_start = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_int_range (lexer, "N", 0, INT_MAX))
+ goto error;
+
+ mf.n = lex_integer (lexer);
+ n_end = lex_ofs (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ int start_ofs = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "LIST"))
+ mf.span = false;
+ else if (lex_match_id (lexer, "FREE"))
+ mf.span = true;
+ else if (lex_match_id (lexer, "UPPER"))
+ mf.triangle = UPPER;
+ else if (lex_match_id (lexer, "LOWER"))
+ mf.triangle = LOWER;
+ else if (lex_match_id (lexer, "FULL"))
+ mf.triangle = FULL;
+ else if (lex_match_id (lexer, "DIAGONAL"))
+ mf.diagonal = DIAGONAL;
+ else if (lex_match_id (lexer, "NODIAGONAL"))
+ mf.diagonal = NO_DIAGONAL;
+ else
+ {
+ lex_error_expecting (lexer, "LIST", "FREE",
+ "UPPER", "LOWER", "FULL",
+ "DIAGONAL", "NODIAGONAL");
+ goto error;
+ }
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ if (mf.diagonal == NO_DIAGONAL && mf.triangle == FULL)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("FORMAT=FULL and FORMAT=NODIAGONAL are "
+ "mutually exclusive."));
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "FILE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ fh_unref (fh);
+ fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
+ if (!fh)
+ goto error;
+ }
+ else if (!mf.n_svars && lex_match_id (lexer, "SPLIT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!mf.input_rowtype
+ && lex_token (lexer) == T_ID
+ && !dict_lookup_var (dict, lex_tokcstr (lexer)))
+ {
+ mf.svars = xmalloc (sizeof *mf.svars);
+ mf.svars[0] = dict_create_var_assert (dict, lex_tokcstr (lexer),
+ 0);
+ var_set_measure (mf.svars[0], MEASURE_NOMINAL);
+ var_set_both_formats (
+ mf.svars[0], &(struct fmt_spec) { .type = FMT_F, .w = 4 });
+ mf.n_svars = 1;
+ lex_get (lexer);
+ }
+ else if (!parse_matrix_data_subvars (lexer, dict, taken_vars,
+ &mf.svars, &mf.svar_indexes,
+ &mf.n_svars))
+ goto error;
+ }
+ else if (!mf.n_fvars && lex_match_id (lexer, "FACTORS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!parse_matrix_data_subvars (lexer, dict, taken_vars,
+ &mf.fvars, &mf.fvar_indexes,
+ &mf.n_fvars))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "CELLS"))
+ {
+ if (mf.input_rowtype)
+ lex_next_msg (lexer, SW,
+ -1, -1, _("CELLS is ignored when VARIABLES "
+ "includes ROWTYPE_"));
+
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_int_range (lexer, "CELLS", 0, INT_MAX))
+ goto error;
+
+ mf.cells = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "CONTENTS"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ size_t allocated_contents = mf.n_contents;
+ bool in_parens = false;
+ for (;;)
+ {
+ bool open = !in_parens && lex_match (lexer, T_LPAREN);
+ enum rowtype rt;
+ if (!rowtype_parse (lexer, &rt))
+ {
+ if (open || in_parens || (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH))
+ {
+ const char *rowtypes[] = {
+#define RT(NAME, DIMS) #NAME,
+ ROWTYPES
+#undef RT
+ "N_VECTOR", "SD",
+ };
+ lex_error_expecting_array (
+ lexer, rowtypes, sizeof rowtypes / sizeof *rowtypes);
+ goto error;
+ }
+ break;
+ }
+
+ if (open)
+ in_parens = true;
+
+ if (in_parens)
+ mf.factor_rowtype_mask |= 1u << rt;
+ else
+ mf.pooled_rowtype_mask |= 1u << rt;
+
+ bool close = in_parens && lex_match (lexer, T_RPAREN);
+ if (close)
+ in_parens = false;
+
+ if (mf.n_contents >= allocated_contents)
+ mf.contents = x2nrealloc (mf.contents, &allocated_contents,
+ sizeof *mf.contents);
+ mf.contents[mf.n_contents++] = (struct content) {
+ .open = open, .rowtype = rt, .close = close
+ };
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "N", "FORMAT", "FILE", "SPLIT", "FACTORS",
+ "CELLS", "CONTENTS");
+ goto error;
+ }
+ }
+ if (!mf.input_rowtype)
+ {
+ if (mf.cells < 0)
+ {
+ if (mf.n_fvars)
+ {
+ msg (SE, _("CELLS is required when factor variables are specified "
+ "and VARIABLES does not include ROWTYPE_."));
+ goto error;
+ }
+ mf.cells = 1;
+ }
+
+ if (!mf.n_contents)
+ {
+ msg (SW, _("CONTENTS was not specified and VARIABLES does not "
+ "include ROWTYPE_. Assuming CONTENTS=CORR."));
+
+ mf.n_contents = 1;
+ mf.contents = xmalloc (sizeof *mf.contents);
+ *mf.contents = (struct content) { .rowtype = C_CORR };
+ }
+ }
+ mf.cvars = xmalloc (mf.n_input_vars * sizeof *mf.cvars);
+ for (size_t i = 0; i < mf.n_input_vars; i++)
+ if (!taken_vars[i])
+ {
+ struct variable *v = input_vars[i];
+ mf.cvars[mf.n_cvars++] = v;
+ var_set_both_formats (v, &(struct fmt_spec) { .type = FMT_F, .w = 10,
+ .d = 4 });
+ }
+ if (!mf.n_cvars)
+ {
+ lex_ofs_error (lexer, input_vars_start, input_vars_end,
+ _("At least one continuous variable is required."));
+ goto error;
+ }
+ if (mf.input_rowtype)
+ {
+ for (size_t i = 0; i < mf.n_cvars; i++)
+ if (mf.cvars[i] != input_vars[n_input_vars - mf.n_cvars + i])
+ {
+ lex_ofs_error (lexer, input_vars_start, input_vars_end,
+ _("VARIABLES includes ROWTYPE_ but the continuous "
+ "variables are not the last ones on VARIABLES."));
+ goto error;
+ }
+ }
+ unsigned int rowtype_mask = mf.pooled_rowtype_mask | mf.factor_rowtype_mask;
+ if (rowtype_mask & (1u << C_N) && mf.n >= 0)
+ {
+ lex_ofs_error (lexer, n_start, n_end,
+ _("Cannot specify N on CONTENTS along with the "
+ "N subcommand."));
+ goto error;
+ }
+
+ struct variable **order = xnmalloc (dict_get_n_vars (dict), sizeof *order);
+ size_t n_order = 0;
+ for (size_t i = 0; i < mf.n_svars; i++)
+ order[n_order++] = mf.svars[i];
+ order[n_order++] = mf.rowtype;
+ for (size_t i = 0; i < mf.n_fvars; i++)
+ order[n_order++] = mf.fvars[i];
+ order[n_order++] = mf.varname;
+ for (size_t i = 0; i < mf.n_cvars; i++)
+ order[n_order++] = mf.cvars[i];
+ assert (n_order == dict_get_n_vars (dict));
+ dict_reorder_vars (dict, order, n_order);
+ free (order);
+
+ dict_set_split_vars (dict, mf.svars, mf.n_svars, SPLIT_LAYERED);
+
+ schedule_matrices (&mf);
+
+ if (fh == NULL)
+ fh = fh_inline_file ();
+
+ if (lex_end_of_command (lexer) != CMD_SUCCESS)
+ goto error;
+
+ struct dfm_reader *reader = dfm_open_reader (fh, lexer, NULL);
+ if (reader == NULL)
+ goto error;
+
+ struct casewriter *writer = autopaging_writer_create (dict_get_proto (dict));
+ if (mf.input_rowtype)
+ parse_data_with_rowtype (&mf, reader, writer);
+ else
+ parse_data_without_rowtype (&mf, reader, writer);
+ dfm_close_reader (reader);
+
+ dataset_set_dict (ds, dict);
+ dataset_set_source (ds, casewriter_make_reader (writer));
+
+ matrix_format_uninit (&mf);
+ free (taken_vars);
+ fh_unref (fh);
+
+ return CMD_SUCCESS;
+
+ error:
+ matrix_format_uninit (&mf);
+ free (taken_vars);
+ dict_unref (dict);
+ fh_unref (fh);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2017, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "matrix-reader.h"
+
+#include <stdbool.h>
+#include <math.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+struct lexer;
+
+/*
+This module interprets a "data matrix", typically generated by the command
+MATRIX DATA. The dictionary of such a matrix takes the form:
+
+ s_0, s_1, ... s_m, ROWTYPE_, VARNAME_, v_0, v_1, .... v_n
+
+where s_0, s_1 ... s_m are the variables defining the splits, and
+v_0, v_1 ... v_n are the continuous variables.
+
+m >= 0; n >= 0
+
+The ROWTYPE_ variable is of type A8.
+The VARNAME_ variable is a string type whose width is not predetermined.
+The variables s_x are of type F4.0 (although this reader accepts any type),
+and v_x are of any numeric type.
+
+The values of the ROWTYPE_ variable are in the set {MEAN, STDDEV, N, CORR, COV}
+and determine the purpose of that case.
+The values of the VARNAME_ variable must correspond to the names of the varibles
+in {v_0, v_1 ... v_n} and indicate the rows of the correlation or covariance
+matrices.
+
+
+
+A typical example is as follows:
+
+s_0 ROWTYPE_ VARNAME_ v_0 v_1 v_2
+
+0 MEAN 5.0000 4.0000 3.0000
+0 STDDEV 1.0000 2.0000 3.0000
+0 N 9.0000 9.0000 9.0000
+0 CORR V1 1.0000 .6000 .7000
+0 CORR V2 .6000 1.0000 .8000
+0 CORR V3 .7000 .8000 1.0000
+1 MEAN 9.0000 8.0000 7.0000
+1 STDDEV 5.0000 6.0000 7.0000
+1 N 9.0000 9.0000 9.0000
+1 CORR V1 1.0000 .4000 .3000
+1 CORR V2 .4000 1.0000 .2000
+1 CORR V3 .3000 .2000 1.0000
+
+*/
+
+void
+matrix_material_uninit (struct matrix_material *mm)
+{
+ gsl_matrix_free (mm->corr);
+ gsl_matrix_free (mm->cov);
+ gsl_matrix_free (mm->n);
+ gsl_matrix_free (mm->mean_matrix);
+ gsl_matrix_free (mm->var_matrix);
+}
+\f
+static const struct variable *
+find_matrix_string_var (const struct dictionary *dict, const char *name)
+{
+ const struct variable *var = dict_lookup_var (dict, name);
+ if (var == NULL)
+ {
+ msg (SE, _("Matrix dataset lacks a variable called %s."), name);
+ return NULL;
+ }
+ if (!var_is_alpha (var))
+ {
+ msg (SE, _("Matrix dataset variable %s should be of string type."), name);
+ return NULL;
+ }
+ return var;
+}
+
+struct matrix_reader *
+matrix_reader_create (const struct dictionary *dict,
+ struct casereader *in_reader)
+{
+ const struct variable *varname = find_matrix_string_var (dict, "VARNAME_");
+ const struct variable *rowtype = find_matrix_string_var (dict, "ROWTYPE_");
+ if (!varname || !rowtype)
+ return NULL;
+
+ size_t varname_idx = var_get_dict_index (varname);
+ size_t rowtype_idx = var_get_dict_index (rowtype);
+ if (varname_idx < rowtype_idx)
+ {
+ msg (SE, _("Variable %s must precede %s in matrix file dictionary."),
+ "ROWTYPE_", "VARNAME_");
+ return NULL;
+ }
+
+ for (size_t i = 0; i < dict_get_n_vars (dict); i++)
+ {
+ const struct variable *v = dict_get_var (dict, i);
+ if (!var_is_numeric (v) && v != rowtype && v != varname)
+ {
+ msg (SE, _("Matrix dataset variable %s should be numeric."),
+ var_get_name (v));
+ return NULL;
+ }
+ }
+
+ size_t n_vars;
+ const struct variable **vars;
+ dict_get_vars (dict, &vars, &n_vars, DC_SCRATCH);
+
+ /* Different kinds of variables. */
+ size_t first_svar = 0;
+ size_t n_svars = rowtype_idx;
+ size_t first_fvar = rowtype_idx + 1;
+ size_t n_fvars = varname_idx - rowtype_idx - 1;
+ size_t first_cvar = varname_idx + 1;
+ size_t n_cvars = n_vars - varname_idx - 1;
+ if (!n_cvars)
+ {
+ msg (SE, _("Matrix dataset does not have any continuous variables."));
+ free (vars);
+ return NULL;
+ }
+
+ struct matrix_reader *mr = xmalloc (sizeof *mr);
+ *mr = (struct matrix_reader) {
+ .dict = dict,
+ .grouper = casegrouper_create_vars (in_reader, &vars[first_svar], n_svars),
+ .svars = xmemdup (vars + first_svar, n_svars * sizeof *mr->svars),
+ .n_svars = n_svars,
+ .rowtype = rowtype,
+ .fvars = xmemdup (vars + first_fvar, n_fvars * sizeof *mr->fvars),
+ .n_fvars = n_fvars,
+ .varname = varname,
+ .cvars = xmemdup (vars + first_cvar, n_cvars * sizeof *mr->cvars),
+ .n_cvars = n_cvars,
+ };
+ free (vars);
+
+ return mr;
+}
+
+bool
+matrix_reader_destroy (struct matrix_reader *mr)
+{
+ if (mr == NULL)
+ return false;
+ bool ret = casegrouper_destroy (mr->grouper);
+ free (mr->svars);
+ free (mr->cvars);
+ free (mr->fvars);
+ free (mr);
+ return ret;
+}
+
+
+/*
+ Allocates MATRIX if necessary,
+ and populates row MROW, from the data in C corresponding to
+ variables in VARS. N_VARS is the length of VARS.
+*/
+static void
+matrix_fill_row (gsl_matrix **matrix,
+ const struct ccase *c, int mrow,
+ const struct variable **vars, size_t n_vars)
+{
+ int col;
+ if (*matrix == NULL)
+ {
+ *matrix = gsl_matrix_alloc (n_vars, n_vars);
+ gsl_matrix_set_all (*matrix, SYSMIS);
+ }
+
+ for (col = 0; col < n_vars; ++col)
+ {
+ const struct variable *cv = vars [col];
+ double x = case_num (c, cv);
+ assert (col < (*matrix)->size2);
+ assert (mrow < (*matrix)->size1);
+ gsl_matrix_set (*matrix, mrow, col, x);
+ }
+}
+
+static int
+find_varname (const struct variable **vars, int n_vars,
+ const char *varname)
+{
+ for (int i = 0; i < n_vars; i++)
+ if (!strcasecmp (var_get_name (vars[i]), varname))
+ return i;
+ return -1;
+}
+
+struct substring
+matrix_reader_get_string (const struct ccase *c, const struct variable *var)
+{
+ struct substring s = case_ss (c, var);
+ ss_rtrim (&s, ss_cstr (CC_SPACES));
+ return s;
+}
+
+void
+matrix_reader_set_string (struct ccase *c, const struct variable *var,
+ struct substring src)
+{
+ struct substring dst = case_ss (c, var);
+ for (size_t i = 0; i < dst.length; i++)
+ dst.string[i] = i < src.length ? src.string[i] : ' ';
+}
+
+bool
+matrix_reader_next (struct matrix_material *mm, struct matrix_reader *mr,
+ struct casereader **groupp)
+{
+ struct casereader *group;
+ if (!casegrouper_get_next_group (mr->grouper, &group))
+ {
+ *mm = (struct matrix_material) MATRIX_MATERIAL_INIT;
+ if (groupp)
+ *groupp = NULL;
+ return false;
+ }
+
+ if (groupp)
+ *groupp = casereader_clone (group);
+
+ const struct variable **vars = mr->cvars;
+ size_t n_vars = mr->n_cvars;
+
+ *mm = (struct matrix_material) { .n = NULL };
+
+ struct matrix
+ {
+ const char *name;
+ gsl_matrix **m;
+ size_t good_rows;
+ size_t bad_rows;
+ };
+ struct matrix matrices[] = {
+ { .name = "CORR", .m = &mm->corr },
+ { .name = "COV", .m = &mm->cov },
+ };
+ enum { N_MATRICES = 2 };
+
+ struct ccase *c;
+ for (; (c = casereader_read (group)); case_unref (c))
+ {
+ struct substring rowtype = matrix_reader_get_string (c, mr->rowtype);
+
+ gsl_matrix **v
+ = (ss_equals_case (rowtype, ss_cstr ("N")) ? &mm->n
+ : ss_equals_case (rowtype, ss_cstr ("MEAN")) ? &mm->mean_matrix
+ : ss_equals_case (rowtype, ss_cstr ("STDDEV")) ? &mm->var_matrix
+ : NULL);
+ if (v)
+ {
+ if (!*v)
+ *v = gsl_matrix_calloc (n_vars, n_vars);
+
+ for (int x = 0; x < n_vars; ++x)
+ {
+ double n = case_num (c, vars[x]);
+ if (v == &mm->var_matrix)
+ n *= n;
+ for (int y = 0; y < n_vars; ++y)
+ gsl_matrix_set (*v, y, x, n);
+ }
+ continue;
+ }
+
+ struct matrix *m = NULL;
+ for (size_t i = 0; i < N_MATRICES; i++)
+ if (ss_equals_case (rowtype, ss_cstr (matrices[i].name)))
+ {
+ m = &matrices[i];
+ break;
+ }
+ if (m)
+ {
+ struct substring varname_raw = case_ss (c, mr->varname);
+ struct substring varname = ss_cstr (
+ recode_string (UTF8, dict_get_encoding (mr->dict),
+ varname_raw.string, varname_raw.length));
+ ss_rtrim (&varname, ss_cstr (CC_SPACES));
+ varname.string[varname.length] = '\0';
+
+ int y = find_varname (vars, n_vars, varname.string);
+ if (y >= 0)
+ {
+ m->good_rows++;
+ matrix_fill_row (m->m, c, y, vars, n_vars);
+ }
+ else
+ m->bad_rows++;
+ ss_dealloc (&varname);
+ }
+ }
+ casereader_destroy (group);
+
+ for (size_t i = 0; i < N_MATRICES; i++)
+ if (matrices[i].good_rows && matrices[i].good_rows != n_vars)
+ msg (SW, _("%s matrix has %zu columns but %zu rows named variables "
+ "to be analyzed (and %zu rows named unknown variables)."),
+ matrices[i].name, n_vars, matrices[i].good_rows,
+ matrices[i].bad_rows);
+
+ return true;
+}
+
+int
+cmd_debug_matrix_read (struct lexer *lexer, struct dataset *ds)
+{
+ if (lex_match_id (lexer, "NODATA"))
+ {
+ struct casereader *cr = casewriter_make_reader (
+ mem_writer_create (dict_get_proto (dataset_dict (ds))));
+ struct matrix_reader *mr = matrix_reader_create (dataset_dict (ds), cr);
+ if (!mr)
+ {
+ casereader_destroy (cr);
+ return CMD_FAILURE;
+ }
+ matrix_reader_destroy (mr);
+ return CMD_SUCCESS;
+ }
+
+ struct matrix_reader *mr = matrix_reader_create (dataset_dict (ds),
+ proc_open (ds));
+ if (!mr)
+ return CMD_FAILURE;
+
+ struct pivot_table *pt = pivot_table_create ("Debug Matrix Reader");
+
+ enum mm_stat
+ {
+ MM_CORR,
+ MM_COV,
+ MM_N,
+ MM_MEAN,
+ MM_STDDEV,
+ };
+ const char *mm_stat_names[] = {
+ [MM_CORR] = "Correlation",
+ [MM_COV] = "Covariance",
+ [MM_N] = "N",
+ [MM_MEAN] = "Mean",
+ [MM_STDDEV] = "Standard Deviation",
+ };
+ enum { N_STATS = sizeof mm_stat_names / sizeof *mm_stat_names };
+ for (size_t i = 0; i < 2; i++)
+ {
+ struct pivot_dimension *d = pivot_dimension_create (
+ pt,
+ i ? PIVOT_AXIS_COLUMN : PIVOT_AXIS_ROW,
+ i ? "Column" : "Row");
+ if (!i)
+ pivot_category_create_leaf_rc (d->root, pivot_value_new_text ("Value"),
+ PIVOT_RC_CORRELATION);
+ for (size_t j = 0; j < mr->n_cvars; j++)
+ pivot_category_create_leaf_rc (
+ d->root, pivot_value_new_variable (mr->cvars[j]),
+ PIVOT_RC_CORRELATION);
+ }
+
+ struct pivot_dimension *stat = pivot_dimension_create (pt, PIVOT_AXIS_ROW,
+ "Statistic");
+ for (size_t i = 0; i < N_STATS; i++)
+ pivot_category_create_leaf (stat->root,
+ pivot_value_new_text (mm_stat_names[i]));
+
+ struct pivot_dimension *split = pivot_dimension_create (
+ pt, PIVOT_AXIS_ROW, "Split");
+
+ int split_num = 0;
+
+ struct matrix_material mm = MATRIX_MATERIAL_INIT;
+ while (matrix_reader_next (&mm, mr, NULL))
+ {
+ pivot_category_create_leaf (split->root,
+ pivot_value_new_integer (split_num + 1));
+
+ const gsl_matrix *m[N_STATS] = {
+ [MM_CORR] = mm.corr,
+ [MM_COV] = mm.cov,
+ [MM_N] = mm.n,
+ [MM_MEAN] = mm.mean_matrix,
+ [MM_STDDEV] = mm.var_matrix,
+ };
+
+ for (size_t i = 0; i < N_STATS; i++)
+ if (m[i])
+ {
+ if (i == MM_COV || i == MM_CORR)
+ {
+ for (size_t y = 0; y < mr->n_cvars; y++)
+ for (size_t x = 0; x < mr->n_cvars; x++)
+ pivot_table_put4 (
+ pt, y + 1, x, i, split_num,
+ pivot_value_new_number (gsl_matrix_get (m[i], y, x)));
+ }
+ else
+ for (size_t x = 0; x < mr->n_cvars; x++)
+ {
+ double n = gsl_matrix_get (m[i], 0, x);
+ if (i == MM_STDDEV)
+ n = sqrt (n);
+ pivot_table_put4 (pt, 0, x, i, split_num,
+ pivot_value_new_number (n));
+ }
+ }
+
+ split_num++;
+ matrix_material_uninit (&mm);
+ }
+ pivot_table_submit (pt);
+
+ proc_commit (ds);
+
+ matrix_reader_destroy (mr);
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2017 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef MATRIX_READER_H
+#define MATRIX_READER_H
+
+#include <gsl/gsl_matrix.h>
+#include <stdbool.h>
+
+struct casereader;
+struct ccase;
+struct dictionary;
+struct matrix_reader;
+struct variable;
+
+struct matrix_reader
+ {
+ const struct dictionary *dict;
+ struct casegrouper *grouper;
+
+ /* Variables in 'dict'. */
+ const struct variable **svars; /* Split variables. */
+ size_t n_svars;
+ const struct variable *rowtype; /* ROWTYPE_. */
+ const struct variable **fvars; /* Factor variables. */
+ size_t n_fvars;
+ const struct variable *varname; /* VARNAME_. */
+ const struct variable **cvars; /* Continuous variables. */
+ size_t n_cvars;
+ };
+
+struct matrix_material
+{
+ gsl_matrix *corr; /* The correlation matrix */
+ gsl_matrix *cov; /* The covariance matrix */
+
+ /* Moment matrices */
+ gsl_matrix *n; /* MOMENT 0 */
+ gsl_matrix *mean_matrix; /* MOMENT 1 */
+ gsl_matrix *var_matrix; /* MOMENT 2 */
+};
+
+#define MATRIX_MATERIAL_INIT { .corr = NULL }
+void matrix_material_uninit (struct matrix_material *);
+
+struct matrix_reader *matrix_reader_create (const struct dictionary *,
+ struct casereader *);
+
+bool matrix_reader_destroy (struct matrix_reader *mr);
+
+bool matrix_reader_next (struct matrix_material *mm, struct matrix_reader *mr,
+ struct casereader **groupp);
+
+struct substring matrix_reader_get_string (const struct ccase *,
+ const struct variable *);
+void matrix_reader_set_string (struct ccase *, const struct variable *,
+ struct substring);
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2021 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_blas.h>
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_eigen.h>
+#include <gsl/gsl_linalg.h>
+#include <gsl/gsl_matrix.h>
+#include <gsl/gsl_permutation.h>
+#include <gsl/gsl_randist.h>
+#include <gsl/gsl_vector.h>
+#include <limits.h>
+#include <math.h>
+#include <uniwidth.h>
+
+#include "data/any-reader.h"
+#include "data/any-writer.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/data-in.h"
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/file-handle-def.h"
+#include "language/command.h"
+#include "language/commands/data-reader.h"
+#include "language/commands/data-writer.h"
+#include "language/commands/file-handle.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/hmap.h"
+#include "libpspp/i18n.h"
+#include "libpspp/intern.h"
+#include "libpspp/misc.h"
+#include "libpspp/str.h"
+#include "libpspp/string-array.h"
+#include "libpspp/stringi-set.h"
+#include "libpspp/u8-line.h"
+#include "math/distributions.h"
+#include "math/random.h"
+#include "output/driver.h"
+#include "output/output-item.h"
+#include "output/pivot-table.h"
+
+#include "gl/c-ctype.h"
+#include "gl/c-strcase.h"
+#include "gl/ftoastr.h"
+#include "gl/minmax.h"
+#include "gl/xsize.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+struct matrix_state;
+
+/* A variable in the matrix language. */
+struct matrix_var
+ {
+ struct hmap_node hmap_node; /* In matrix_state's 'vars' hmap. */
+ char *name; /* UTF-8. */
+ gsl_matrix *value; /* NULL, if the variable is uninitialized. */
+ };
+
+/* All the MSAVE commands within a matrix program share common configuration,
+ provided by the first MSAVE command within the program. This structure
+ encapsulates this configuration. */
+struct msave_common
+ {
+ /* Common configuration for all MSAVEs. */
+ struct msg_location *location; /* Range of lines for first MSAVE. */
+ struct file_handle *outfile; /* Output file for all the MSAVEs. */
+ struct msg_location *outfile_location;
+ struct string_array variables; /* VARIABLES subcommand. */
+ struct msg_location *variables_location;
+ struct string_array fnames; /* FNAMES subcommand. */
+ struct msg_location *fnames_location;
+ struct string_array snames; /* SNAMES subcommand. */
+ struct msg_location *snames_location;
+
+ /* Collects and owns factors and splits. The individual msave_command
+ structs point to these but do not own them. (This is because factors
+ and splits can be carried over from one MSAVE to the next, so it's
+ easiest to just take the most recent.) */
+ struct matrix_expr **factors;
+ size_t n_factors, allocated_factors;
+ struct matrix_expr **splits;
+ size_t n_splits, allocated_splits;
+
+ /* Execution state. */
+ struct dictionary *dict;
+ struct casewriter *writer;
+ };
+
+/* A file used by one or more READ commands. */
+struct read_file
+ {
+ /* Parse state. */
+ struct file_handle *file;
+
+ /* Execution state. */
+ struct dfm_reader *reader;
+ char *encoding;
+ };
+
+static struct read_file *read_file_create (struct matrix_state *,
+ struct file_handle *);
+static struct dfm_reader *read_file_open (struct read_file *);
+
+/* A file used by one or more WRITE comamnds. */
+struct write_file
+ {
+ /* Parse state. */
+ struct file_handle *file;
+
+ /* Execution state. */
+ struct dfm_writer *writer;
+ char *encoding;
+ struct u8_line *held; /* Output held by a previous WRITE with /HOLD. */
+ };
+
+static struct write_file *write_file_create (struct matrix_state *,
+ struct file_handle *);
+static struct dfm_writer *write_file_open (struct write_file *);
+static void write_file_destroy (struct write_file *);
+
+/* A file used by one or more SAVE commands. */
+struct save_file
+ {
+ /* Parse state. */
+ struct file_handle *file;
+ struct dataset *dataset;
+ struct string_array variables;
+ struct matrix_expr *names;
+ struct stringi_set strings;
+
+ /* Execution state. */
+ bool error;
+ struct casewriter *writer;
+ struct dictionary *dict;
+ struct msg_location *location;
+ };
+
+/* State of an entire matrix program. */
+struct matrix_state
+ {
+ /* State passed into MATRIX from outside. */
+ struct dataset *dataset;
+ struct session *session;
+ struct lexer *lexer;
+
+ /* Matrix program's own state. */
+ struct hmap vars; /* Dictionary of matrix variables. */
+ bool in_loop; /* True if parsing within a LOOP. */
+
+ /* MSAVE. */
+ struct msave_common *msave_common;
+
+ /* READ. */
+ struct file_handle *prev_read_file;
+ struct read_file **read_files;
+ size_t n_read_files;
+
+ /* WRITE. */
+ struct file_handle *prev_write_file;
+ struct write_file **write_files;
+ size_t n_write_files;
+
+ /* SAVE. */
+ struct file_handle *prev_save_file;
+ struct save_file **save_files;
+ size_t n_save_files;
+ };
+
+/* Finds and returns the variable with the given NAME (case-insensitive) within
+ S, if there is one, or a null pointer if there is not. */
+static struct matrix_var *
+matrix_var_lookup (struct matrix_state *s, struct substring name)
+{
+ struct matrix_var *var;
+
+ HMAP_FOR_EACH_WITH_HASH (var, struct matrix_var, hmap_node,
+ utf8_hash_case_substring (name, 0), &s->vars)
+ if (!utf8_sscasecmp (ss_cstr (var->name), name))
+ return var;
+
+ return NULL;
+}
+
+/* Creates and returns a new variable named NAME within S. There must not
+ already be a variable with the same (case-insensitive) name. The variable
+ is created uninitialized. */
+static struct matrix_var *
+matrix_var_create (struct matrix_state *s, struct substring name)
+{
+ struct matrix_var *var = xmalloc (sizeof *var);
+ *var = (struct matrix_var) { .name = ss_xstrdup (name) };
+ hmap_insert (&s->vars, &var->hmap_node, utf8_hash_case_substring (name, 0));
+ return var;
+}
+
+/* Replaces VAR's value by VALUE. Takes ownership of VALUE. */
+static void
+matrix_var_set (struct matrix_var *var, gsl_matrix *value)
+{
+ gsl_matrix_free (var->value);
+ var->value = value;
+}
+\f
+/* Matrix function catalog. */
+
+/* The third argument to F() is a "prototype". For most prototypes, the first
+ letter (before the _) represents the return type and each other letter
+ (after the _) is an argument type. The types are:
+
+ - "m": A matrix of unrestricted dimensions.
+
+ - "d": A scalar.
+
+ - "v": A row or column vector.
+
+ - "e": Primarily for the first argument, this is a matrix with
+ unrestricted dimensions treated elementwise. Each element in the matrix
+ is passed to the implementation function separately.
+
+ - "n": This gets passed the "const struct matrix_expr *" that represents
+ the expression. This allows the evaluation function to grab the source
+ location of arguments so that it can report accurate error locations.
+ This type doesn't correspond to an argument passed in by the user.
+
+ The fourth argument is an optional constraints string. For this purpose the
+ first argument is named "a", the second "b", and so on. The following kinds
+ of constraints are supported. For matrix arguments, the constraints are
+ applied to each value in the matrix separately:
+
+ - "a(0,1)" or "a[0,1]": 0 < a < 1 or 0 <= a <= 1, respectively. Any
+ integer may substitute for 0 and 1. Half-open constraints (] and [) are
+ also supported.
+
+ - "ai": Restrict a to integer values.
+
+ - "a>0", "a<0", "a>=0", "a<=0", "a!=0".
+
+ - "a<b", "a>b", "a<=b", "a>=b", "b!=0".
+*/
+#define MATRIX_FUNCTIONS \
+ F(ABS, "ABS", m_e, NULL) \
+ F(ALL, "ALL", d_m, NULL) \
+ F(ANY, "ANY", d_m, NULL) \
+ F(ARSIN, "ARSIN", m_e, "a[-1,1]") \
+ F(ARTAN, "ARTAN", m_e, NULL) \
+ F(BLOCK, "BLOCK", m_any, NULL) \
+ F(CHOL, "CHOL", m_mn, NULL) \
+ F(CMIN, "CMIN", m_m, NULL) \
+ F(CMAX, "CMAX", m_m, NULL) \
+ F(COS, "COS", m_e, NULL) \
+ F(CSSQ, "CSSQ", m_m, NULL) \
+ F(CSUM, "CSUM", m_m, NULL) \
+ F(DESIGN, "DESIGN", m_mn, NULL) \
+ F(DET, "DET", d_m, NULL) \
+ F(DIAG, "DIAG", m_m, NULL) \
+ F(EVAL, "EVAL", m_mn, NULL) \
+ F(EXP, "EXP", m_e, NULL) \
+ F(GINV, "GINV", m_m, NULL) \
+ F(GRADE, "GRADE", m_m, NULL) \
+ F(GSCH, "GSCH", m_mn, NULL) \
+ F(IDENT, "IDENT", IDENT, NULL) \
+ F(INV, "INV", m_m, NULL) \
+ F(KRONEKER, "KRONEKER", m_mm, NULL) \
+ F(LG10, "LG10", m_e, "a>0") \
+ F(LN, "LN", m_e, "a>0") \
+ F(MAGIC, "MAGIC", m_d, "ai>=3") \
+ F(MAKE, "MAKE", m_ddd, "ai>=0 bi>=0") \
+ F(MDIAG, "MDIAG", m_v, NULL) \
+ F(MMAX, "MMAX", d_m, NULL) \
+ F(MMIN, "MMIN", d_m, NULL) \
+ F(MOD, "MOD", m_md, "b!=0") \
+ F(MSSQ, "MSSQ", d_m, NULL) \
+ F(MSUM, "MSUM", d_m, NULL) \
+ F(NCOL, "NCOL", d_m, NULL) \
+ F(NROW, "NROW", d_m, NULL) \
+ F(RANK, "RANK", d_m, NULL) \
+ F(RESHAPE, "RESHAPE", m_mddn, NULL) \
+ F(RMAX, "RMAX", m_m, NULL) \
+ F(RMIN, "RMIN", m_m, NULL) \
+ F(RND, "RND", m_e, NULL) \
+ F(RNKORDER, "RNKORDER", m_m, NULL) \
+ F(RSSQ, "RSSQ", m_m, NULL) \
+ F(RSUM, "RSUM", m_m, NULL) \
+ F(SIN, "SIN", m_e, NULL) \
+ F(SOLVE, "SOLVE", m_mmn, NULL) \
+ F(SQRT, "SQRT", m_e, "a>=0") \
+ F(SSCP, "SSCP", m_m, NULL) \
+ F(SVAL, "SVAL", m_m, NULL) \
+ F(SWEEP, "SWEEP", m_mdn, NULL) \
+ F(T, "T", m_m, NULL) \
+ F(TRACE, "TRACE", d_m, NULL) \
+ F(TRANSPOS, "TRANSPOS", m_m, NULL) \
+ F(TRUNC, "TRUNC", m_e, NULL) \
+ F(UNIFORM, "UNIFORM", m_ddn, "ai>=0 bi>=0") \
+ F(PDF_BETA, "PDF.BETA", m_edd, "a[0,1] b>0 c>0") \
+ F(CDF_BETA, "CDF.BETA", m_edd, "a[0,1] b>0 c>0") \
+ F(IDF_BETA, "IDF.BETA", m_edd, "a[0,1] b>0 c>0") \
+ F(RV_BETA, "RV.BETA", d_dd, "a>0 b>0") \
+ F(NCDF_BETA, "NCDF.BETA", m_eddd, "a>=0 b>0 c>0 d>0") \
+ F(NPDF_BETA, "NCDF.BETA", m_eddd, "a>=0 b>0 c>0 d>0") \
+ F(CDF_BVNOR, "CDF.BVNOR", m_eed, "c[-1,1]") \
+ F(PDF_BVNOR, "PDF.BVNOR", m_eed, "c[-1,1]") \
+ F(CDF_CAUCHY, "CDF.CAUCHY", m_edd, "c>0") \
+ F(IDF_CAUCHY, "IDF.CAUCHY", m_edd, "a(0,1) c>0") \
+ F(PDF_CAUCHY, "PDF.CAUCHY", m_edd, "c>0") \
+ F(RV_CAUCHY, "RV.CAUCHY", d_dd, "b>0") \
+ F(CDF_CHISQ, "CDF.CHISQ", m_ed, "a>=0 b>0") \
+ F(CHICDF, "CHICDF", m_ed, "a>=0 b>0") \
+ F(IDF_CHISQ, "IDF.CHISQ", m_ed, "a[0,1) b>0") \
+ F(PDF_CHISQ, "PDF.CHISQ", m_ed, "a>=0 b>0") \
+ F(RV_CHISQ, "RV.CHISQ", d_d, "a>0") \
+ F(SIG_CHISQ, "SIG.CHISQ", m_ed, "a>=0 b>0") \
+ F(CDF_EXP, "CDF.EXP", m_ed, "a>=0 b>=0") \
+ F(IDF_EXP, "IDF.EXP", m_ed, "a[0,1) b>0") \
+ F(PDF_EXP, "PDF.EXP", m_ed, "a>=0 b>0") \
+ F(RV_EXP, "RV.EXP", d_d, "a>0") \
+ F(PDF_XPOWER, "PDF.XPOWER", m_edd, "b>0 c>=0") \
+ F(RV_XPOWER, "RV.XPOWER", d_dd, "a>0 c>=0") \
+ F(CDF_F, "CDF.F", m_edd, "a>=0 b>0 c>0") \
+ F(FCDF, "FCDF", m_edd, "a>=0 b>0 c>0") \
+ F(IDF_F, "IDF.F", m_edd, "a[0,1) b>0 c>0") \
+ F(PDF_F, "PDF.F", m_edd, "a>=0 b>0 c>0") \
+ F(RV_F, "RV.F", d_dd, "a>0 b>0") \
+ F(SIG_F, "SIG.F", m_edd, "a>=0 b>0 c>0") \
+ F(CDF_GAMMA, "CDF.GAMMA", m_edd, "a>=0 b>0 c>0") \
+ F(IDF_GAMMA, "IDF.GAMMA", m_edd, "a[0,1] b>0 c>0") \
+ F(PDF_GAMMA, "PDF.GAMMA", m_edd, "a>=0 b>0 c>0") \
+ F(RV_GAMMA, "RV.GAMMA", d_dd, "a>0 b>0") \
+ F(PDF_LANDAU, "PDF.LANDAU", m_e, NULL) \
+ F(RV_LANDAU, "RV.LANDAU", d_none, NULL) \
+ F(CDF_LAPLACE, "CDF.LAPLACE", m_edd, "c>0") \
+ F(IDF_LAPLACE, "IDF.LAPLACE", m_edd, "a(0,1) c>0") \
+ F(PDF_LAPLACE, "PDF.LAPLACE", m_edd, "c>0") \
+ F(RV_LAPLACE, "RV.LAPLACE", d_dd, "b>0") \
+ F(RV_LEVY, "RV.LEVY", d_dd, "b(0,2]") \
+ F(RV_LVSKEW, "RV.LVSKEW", d_ddd, "b(0,2] c[-1,1]") \
+ F(CDF_LOGISTIC, "CDF.LOGISTIC", m_edd, "c>0") \
+ F(IDF_LOGISTIC, "IDF.LOGISTIC", m_edd, "a(0,1) c>0") \
+ F(PDF_LOGISTIC, "PDF.LOGISTIC", m_edd, "c>0") \
+ F(RV_LOGISTIC, "RV.LOGISTIC", d_dd, "b>0") \
+ F(CDF_LNORMAL, "CDF.LNORMAL", m_edd, "a>=0 b>0 c>0") \
+ F(IDF_LNORMAL, "IDF.LNORMAL", m_edd, "a[0,1) b>0 c>0") \
+ F(PDF_LNORMAL, "PDF.LNORMAL", m_edd, "a>=0 b>0 c>0") \
+ F(RV_LNORMAL, "RV.LNORMAL", d_dd, "a>0 b>0") \
+ F(CDF_NORMAL, "CDF.NORMAL", m_edd, "c>0") \
+ F(IDF_NORMAL, "IDF.NORMAL", m_edd, "a(0,1) c>0") \
+ F(PDF_NORMAL, "PDF.NORMAL", m_edd, "c>0") \
+ F(RV_NORMAL, "RV.NORMAL", d_dd, "b>0") \
+ F(CDFNORM, "CDFNORM", m_e, NULL) \
+ F(PROBIT, "PROBIT", m_e, "a(0,1)") \
+ F(NORMAL, "NORMAL", m_e, "a>0") \
+ F(PDF_NTAIL, "PDF.NTAIL", m_edd, "b>0 c>0") \
+ F(RV_NTAIL, "RV.NTAIL", d_dd, "a>0 b>0") \
+ F(CDF_PARETO, "CDF.PARETO", m_edd, "a>=b b>0 c>0") \
+ F(IDF_PARETO, "IDF.PARETO", m_edd, "a[0,1) b>0 c>0") \
+ F(PDF_PARETO, "PDF.PARETO", m_edd, "a>=b b>0 c>0") \
+ F(RV_PARETO, "RV.PARETO", d_dd, "a>0 b>0") \
+ F(CDF_RAYLEIGH, "CDF.RAYLEIGH", m_ed, "b>0") \
+ F(IDF_RAYLEIGH, "IDF.RAYLEIGH", m_ed, "a[0,1] b>0") \
+ F(PDF_RAYLEIGH, "PDF.RAYLEIGH", m_ed, "b>0") \
+ F(RV_RAYLEIGH, "RV.RAYLEIGH", d_d, "a>0") \
+ F(PDF_RTAIL, "PDF.RTAIL", m_edd, NULL) \
+ F(RV_RTAIL, "RV.RTAIL", d_dd, NULL) \
+ F(CDF_T, "CDF.T", m_ed, "b>0") \
+ F(TCDF, "TCDF", m_ed, "b>0") \
+ F(IDF_T, "IDF.T", m_ed, "a(0,1) b>0") \
+ F(PDF_T, "PDF.T", m_ed, "b>0") \
+ F(RV_T, "RV.T", d_d, "a>0") \
+ F(CDF_T1G, "CDF.T1G", m_edd, NULL) \
+ F(IDF_T1G, "IDF.T1G", m_edd, "a(0,1)") \
+ F(PDF_T1G, "PDF.T1G", m_edd, NULL) \
+ F(RV_T1G, "RV.T1G", d_dd, NULL) \
+ F(CDF_T2G, "CDF.T2G", m_edd, NULL) \
+ F(IDF_T2G, "IDF.T2G", m_edd, "a(0,1)") \
+ F(PDF_T2G, "PDF.T2G", m_edd, NULL) \
+ F(RV_T2G, "RV.T2G", d_dd, NULL) \
+ F(CDF_UNIFORM, "CDF.UNIFORM", m_edd, "a<=c b<=c") \
+ F(IDF_UNIFORM, "IDF.UNIFORM", m_edd, "a[0,1] b<=c") \
+ F(PDF_UNIFORM, "PDF.UNIFORM", m_edd, "a<=c b<=c") \
+ F(RV_UNIFORM, "RV.UNIFORM", d_dd, "a<=b") \
+ F(CDF_WEIBULL, "CDF.WEIBULL", m_edd, "a>=0 b>0 c>0") \
+ F(IDF_WEIBULL, "IDF.WEIBULL", m_edd, "a[0,1) b>0 c>0") \
+ F(PDF_WEIBULL, "PDF.WEIBULL", m_edd, "a>=0 b>0 c>0") \
+ F(RV_WEIBULL, "RV.WEIBULL", d_dd, "a>0 b>0") \
+ F(CDF_BERNOULLI, "CDF.BERNOULLI", m_ed, "ai[0,1] b[0,1]") \
+ F(PDF_BERNOULLI, "PDF.BERNOULLI", m_ed, "ai[0,1] b[0,1]") \
+ F(RV_BERNOULLI, "RV.BERNOULLI", d_d, "a[0,1]") \
+ F(CDF_BINOM, "CDF.BINOM", m_edd, "bi>0 c[0,1]") \
+ F(PDF_BINOM, "PDF.BINOM", m_edd, "ai>=0<=b bi>0 c[0,1]") \
+ F(RV_BINOM, "RV.BINOM", d_dd, "ai>0 b[0,1]") \
+ F(CDF_GEOM, "CDF.GEOM", m_ed, "ai>=1 b[0,1]") \
+ F(PDF_GEOM, "PDF.GEOM", m_ed, "ai>=1 b[0,1]") \
+ F(RV_GEOM, "RV.GEOM", d_d, "a[0,1]") \
+ F(CDF_HYPER, "CDF.HYPER", m_eddd, "ai>=0<=d bi>0 ci>0<=b di>0<=b") \
+ F(PDF_HYPER, "PDF.HYPER", m_eddd, "ai>=0<=d bi>0 ci>0<=b di>0<=b") \
+ F(RV_HYPER, "RV.HYPER", d_ddd, "ai>0 bi>0<=a ci>0<=a") \
+ F(PDF_LOG, "PDF.LOG", m_ed, "a>=1 b(0,1]") \
+ F(RV_LOG, "RV.LOG", d_d, "a(0,1]") \
+ F(CDF_NEGBIN, "CDF.NEGBIN", m_edd, "a>=1 bi c(0,1]") \
+ F(PDF_NEGBIN, "PDF.NEGBIN", m_edd, "a>=1 bi c(0,1]") \
+ F(RV_NEGBIN, "RV.NEGBIN", d_dd, "ai b(0,1]") \
+ F(CDF_POISSON, "CDF.POISSON", m_ed, "ai>=0 b>0") \
+ F(PDF_POISSON, "PDF.POISSON", m_ed, "ai>=0 b>0") \
+ F(RV_POISSON, "RV.POISSON", d_d, "a>0")
+
+/* Properties of a matrix function.
+
+ These come straight from the macro invocations above. */
+struct matrix_function_properties
+ {
+ const char *name;
+ const char *constraints;
+ };
+
+/* Minimum and maximum argument counts for each matrix function prototype. */
+enum { IDENT_MIN_ARGS = 1, IDENT_MAX_ARGS = 2 };
+enum { d_d_MIN_ARGS = 1, d_d_MAX_ARGS = 1 };
+enum { d_dd_MIN_ARGS = 2, d_dd_MAX_ARGS = 2 };
+enum { d_ddd_MIN_ARGS = 3, d_ddd_MAX_ARGS = 3 };
+enum { d_m_MIN_ARGS = 1, d_m_MAX_ARGS = 1 };
+enum { d_none_MIN_ARGS = 0, d_none_MAX_ARGS = 0 };
+enum { m_any_MIN_ARGS = 1, m_any_MAX_ARGS = INT_MAX };
+enum { m_d_MIN_ARGS = 1, m_d_MAX_ARGS = 1 };
+enum { m_ddd_MIN_ARGS = 3, m_ddd_MAX_ARGS = 3 };
+enum { m_ddn_MIN_ARGS = 2, m_ddn_MAX_ARGS = 2 };
+enum { m_e_MIN_ARGS = 1, m_e_MAX_ARGS = 1 };
+enum { m_ed_MIN_ARGS = 2, m_ed_MAX_ARGS = 2 };
+enum { m_edd_MIN_ARGS = 3, m_edd_MAX_ARGS = 3 };
+enum { m_eddd_MIN_ARGS = 4, m_eddd_MAX_ARGS = 4 };
+enum { m_eed_MIN_ARGS = 3, m_eed_MAX_ARGS = 3 };
+enum { m_m_MIN_ARGS = 1, m_m_MAX_ARGS = 1 };
+enum { m_md_MIN_ARGS = 2, m_md_MAX_ARGS = 2 };
+enum { m_mddn_MIN_ARGS = 3, m_mddn_MAX_ARGS = 3 };
+enum { m_mdn_MIN_ARGS = 2, m_mdn_MAX_ARGS = 2 };
+enum { m_mm_MIN_ARGS = 2, m_mm_MAX_ARGS = 2 };
+enum { m_mmn_MIN_ARGS = 2, m_mmn_MAX_ARGS = 2 };
+enum { m_mn_MIN_ARGS = 1, m_mn_MAX_ARGS = 1 };
+enum { m_v_MIN_ARGS = 1, m_v_MAX_ARGS = 1 };
+
+/* C function prototype for each matrix function prototype. */
+typedef double matrix_proto_d_none (void);
+typedef double matrix_proto_d_d (double);
+typedef double matrix_proto_d_dd (double, double);
+typedef double matrix_proto_d_dd (double, double);
+typedef double matrix_proto_d_ddd (double, double, double);
+typedef gsl_matrix *matrix_proto_m_d (double);
+typedef gsl_matrix *matrix_proto_m_ddd (double, double, double);
+typedef gsl_matrix *matrix_proto_m_ddn (double, double,
+ const struct matrix_expr *);
+typedef gsl_matrix *matrix_proto_m_m (gsl_matrix *);
+typedef gsl_matrix *matrix_proto_m_mn (gsl_matrix *,
+ const struct matrix_expr *);
+typedef double matrix_proto_m_e (double);
+typedef gsl_matrix *matrix_proto_m_md (gsl_matrix *, double);
+typedef gsl_matrix *matrix_proto_m_mdn (gsl_matrix *, double,
+ const struct matrix_expr *);
+typedef double matrix_proto_m_ed (double, double);
+typedef gsl_matrix *matrix_proto_m_mddn (gsl_matrix *, double, double,
+ const struct matrix_expr *);
+typedef double matrix_proto_m_edd (double, double, double);
+typedef double matrix_proto_m_eddd (double, double, double, double);
+typedef double matrix_proto_m_eed (double, double, double);
+typedef gsl_matrix *matrix_proto_m_mm (gsl_matrix *, gsl_matrix *);
+typedef gsl_matrix *matrix_proto_m_mmn (gsl_matrix *, gsl_matrix *,
+ const struct matrix_expr *);
+typedef gsl_matrix *matrix_proto_m_v (gsl_vector *);
+typedef double matrix_proto_d_m (gsl_matrix *);
+typedef gsl_matrix *matrix_proto_m_any (gsl_matrix *[], size_t n);
+typedef gsl_matrix *matrix_proto_IDENT (double, double);
+
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
+ static matrix_proto_##PROTO matrix_eval_##ENUM;
+MATRIX_FUNCTIONS
+#undef F
+\f
+/* Matrix expression data structure and parsing. */
+
+/* A node in a matrix expression. */
+struct matrix_expr
+ {
+ enum matrix_op
+ {
+ /* Functions. */
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) MOP_F_##ENUM,
+ MATRIX_FUNCTIONS
+#undef F
+
+ /* Elementwise and scalar arithmetic. */
+ MOP_NEGATE, /* unary - */
+ MOP_ADD_ELEMS, /* + */
+ MOP_SUB_ELEMS, /* - */
+ MOP_MUL_ELEMS, /* &* */
+ MOP_DIV_ELEMS, /* / and &/ */
+ MOP_EXP_ELEMS, /* &** */
+ MOP_SEQ, /* a:b */
+ MOP_SEQ_BY, /* a:b:c */
+
+ /* Matrix arithmetic. */
+ MOP_MUL_MAT, /* * */
+ MOP_EXP_MAT, /* ** */
+
+ /* Relational. */
+ MOP_GT, /* > */
+ MOP_GE, /* >= */
+ MOP_LT, /* < */
+ MOP_LE, /* <= */
+ MOP_EQ, /* = */
+ MOP_NE, /* <> */
+
+ /* Logical. */
+ MOP_NOT, /* NOT */
+ MOP_AND, /* AND */
+ MOP_OR, /* OR */
+ MOP_XOR, /* XOR */
+
+ /* {}. */
+ MOP_PASTE_HORZ, /* a, b, c, ... */
+ MOP_PASTE_VERT, /* a; b; c; ... */
+ MOP_EMPTY, /* {} */
+
+ /* Sub-matrices. */
+ MOP_VEC_INDEX, /* x(y) */
+ MOP_VEC_ALL, /* x(:) */
+ MOP_MAT_INDEX, /* x(y,z) */
+ MOP_ROW_INDEX, /* x(y,:) */
+ MOP_COL_INDEX, /* x(:,z) */
+
+ /* Literals. */
+ MOP_NUMBER,
+ MOP_VARIABLE,
+
+ /* Oddball stuff. */
+ MOP_EOF, /* EOF('file') */
+ }
+ op;
+
+ union
+ {
+ /* Nonterminal expression nodes. */
+ struct
+ {
+ struct matrix_expr **subs;
+ size_t n_subs;
+ };
+
+ /* Terminal expression nodes. */
+ double number; /* MOP_NUMBER. */
+ struct matrix_var *variable; /* MOP_VARIABLE. */
+ struct read_file *eof; /* MOP_EOF. */
+ };
+
+ /* The syntax location corresponding to this expression node, for use in
+ error messages. This is always nonnull for terminal expression nodes.
+ For most others, it is null because it can be computed lazily if and
+ when it is needed.
+
+ Use matrix_expr_location() instead of using this member directly, so
+ that it gets computed lazily if needed. */
+ struct msg_location *location;
+ };
+
+static void
+matrix_expr_location__ (const struct matrix_expr *e,
+ const struct msg_location **minp,
+ const struct msg_location **maxp)
+{
+ struct msg_location *loc = e->location;
+ if (loc)
+ {
+ const struct msg_location *min = *minp;
+ if (loc->start.line
+ && (!min
+ || loc->start.line < min->start.line
+ || (loc->start.line == min->start.line
+ && loc->start.column < min->start.column)))
+ *minp = loc;
+
+ const struct msg_location *max = *maxp;
+ if (loc->end.line
+ && (!max
+ || loc->end.line > max->end.line
+ || (loc->end.line == max->end.line
+ && loc->end.column > max->end.column)))
+ *maxp = loc;
+
+ return;
+ }
+
+ assert (e->op != MOP_NUMBER && e->op != MOP_VARIABLE && e->op != MOP_EOF);
+ for (size_t i = 0; i < e->n_subs; i++)
+ matrix_expr_location__ (e->subs[i], minp, maxp);
+}
+
+/* Returns the source code location corresponding to expression E, computing it
+ lazily if needed. */
+static const struct msg_location *
+matrix_expr_location (const struct matrix_expr *e_)
+{
+ struct matrix_expr *e = CONST_CAST (struct matrix_expr *, e_);
+ if (!e)
+ return NULL;
+
+ if (!e->location)
+ {
+ const struct msg_location *min = NULL;
+ const struct msg_location *max = NULL;
+ matrix_expr_location__ (e, &min, &max);
+ if (min && max)
+ {
+ e->location = msg_location_dup (min);
+ e->location->end = max->end;
+ }
+ }
+ return e->location;
+}
+
+/* Sets e->location to the tokens in S's lexer from offset START_OFS to the
+ token before the current one. Has no effect if E already has a location or
+ if E is null. */
+static void
+matrix_expr_add_location (struct matrix_state *s, int start_ofs,
+ struct matrix_expr *e)
+{
+ if (e && !e->location)
+ e->location = lex_ofs_location (s->lexer, start_ofs,
+ lex_ofs (s->lexer) - 1);
+}
+
+/* Frees E and all the data and sub-expressions that it references. */
+static void
+matrix_expr_destroy (struct matrix_expr *e)
+{
+ if (!e)
+ return;
+
+ switch (e->op)
+ {
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) case MOP_F_##ENUM:
+MATRIX_FUNCTIONS
+#undef F
+ case MOP_NEGATE:
+ case MOP_ADD_ELEMS:
+ case MOP_SUB_ELEMS:
+ case MOP_MUL_ELEMS:
+ case MOP_DIV_ELEMS:
+ case MOP_EXP_ELEMS:
+ case MOP_SEQ:
+ case MOP_SEQ_BY:
+ case MOP_MUL_MAT:
+ case MOP_EXP_MAT:
+ case MOP_GT:
+ case MOP_GE:
+ case MOP_LT:
+ case MOP_LE:
+ case MOP_EQ:
+ case MOP_NE:
+ case MOP_NOT:
+ case MOP_AND:
+ case MOP_OR:
+ case MOP_XOR:
+ case MOP_EMPTY:
+ case MOP_PASTE_HORZ:
+ case MOP_PASTE_VERT:
+ case MOP_VEC_INDEX:
+ case MOP_VEC_ALL:
+ case MOP_MAT_INDEX:
+ case MOP_ROW_INDEX:
+ case MOP_COL_INDEX:
+ for (size_t i = 0; i < e->n_subs; i++)
+ matrix_expr_destroy (e->subs[i]);
+ free (e->subs);
+ break;
+
+ case MOP_NUMBER:
+ case MOP_VARIABLE:
+ case MOP_EOF:
+ break;
+ }
+ msg_location_destroy (e->location);
+ free (e);
+}
+
+/* Creates and returns a new matrix_expr with type OP, which must be a
+ nonterminal type. Initializes the new matrix_expr with the N_SUBS
+ expressions in SUBS as subexpressions. */
+static struct matrix_expr *
+matrix_expr_create_subs (enum matrix_op op, struct matrix_expr **subs,
+ size_t n_subs)
+{
+ struct matrix_expr *e = xmalloc (sizeof *e);
+ *e = (struct matrix_expr) {
+ .op = op,
+ .subs = xmemdup (subs, n_subs * sizeof *subs),
+ .n_subs = n_subs
+ };
+ return e;
+}
+
+static struct matrix_expr *
+matrix_expr_create_0 (enum matrix_op op)
+{
+ struct matrix_expr *sub;
+ return matrix_expr_create_subs (op, &sub, 0);
+}
+
+static struct matrix_expr *
+matrix_expr_create_1 (enum matrix_op op, struct matrix_expr *sub)
+{
+ return matrix_expr_create_subs (op, &sub, 1);
+}
+
+static struct matrix_expr *
+matrix_expr_create_2 (enum matrix_op op,
+ struct matrix_expr *sub0, struct matrix_expr *sub1)
+{
+ struct matrix_expr *subs[] = { sub0, sub1 };
+ return matrix_expr_create_subs (op, subs, sizeof subs / sizeof *subs);
+}
+
+static struct matrix_expr *
+matrix_expr_create_3 (enum matrix_op op, struct matrix_expr *sub0,
+ struct matrix_expr *sub1, struct matrix_expr *sub2)
+{
+ struct matrix_expr *subs[] = { sub0, sub1, sub2 };
+ return matrix_expr_create_subs (op, subs, sizeof subs / sizeof *subs);
+}
+
+/* Creates and returns a new MOP_NUMBER expression node to contain NUMBER. */
+static struct matrix_expr *
+matrix_expr_create_number (double number)
+{
+ struct matrix_expr *e = xmalloc (sizeof *e);
+ *e = (struct matrix_expr) {
+ .op = MOP_NUMBER,
+ .number = number,
+ };
+ return e;
+}
+
+static struct matrix_expr *matrix_expr_parse (struct matrix_state *);
+
+/* A binary operator for matrix_parse_binary_operator(). */
+struct matrix_operator_syntax
+ {
+ /* Exactly one of these specifies the operator syntax. */
+ enum token_type token; /* A token, e.g. T_ASTERISK. */
+ const char *id; /* An identifier, e.g. "XOR". */
+ const char *phrase; /* A token phrase, e.g. "&**". */
+
+ /* The matrix operator corresponding to the syntax. */
+ enum matrix_op op;
+ };
+
+static bool
+matrix_operator_syntax_match (struct lexer *lexer,
+ const struct matrix_operator_syntax *syntax,
+ size_t n_syntax, enum matrix_op *op)
+{
+ const struct matrix_operator_syntax *end = &syntax[n_syntax];
+ for (const struct matrix_operator_syntax *syn = syntax; syn < end; syn++)
+ if (syn->id ? lex_match_id (lexer, syn->id)
+ : syn->phrase ? lex_match_phrase (lexer, syn->phrase)
+ : lex_match (lexer, syn->token))
+ {
+ *op = syn->op;
+ return true;
+ }
+ return false;
+}
+
+/* Parses a binary operator level in the recursive descent parser, returning a
+ matrix expression if successful or a null pointer otherwise. PARSE_NEXT
+ must be the function to parse the next level of precedence. The N_SYNTAX
+ elements of SYNTAX must specify the syntax and matrix_expr node type to
+ parse at this level. */
+static struct matrix_expr *
+matrix_parse_binary_operator (
+ struct matrix_state *s,
+ struct matrix_expr *(*parse_next) (struct matrix_state *),
+ const struct matrix_operator_syntax *syntax, size_t n_syntax)
+{
+ struct matrix_expr *lhs = parse_next (s);
+ if (!lhs)
+ return NULL;
+
+ for (;;)
+ {
+ enum matrix_op op;
+ if (!matrix_operator_syntax_match (s->lexer, syntax, n_syntax, &op))
+ return lhs;
+
+ struct matrix_expr *rhs = parse_next (s);
+ if (!rhs)
+ {
+ matrix_expr_destroy (lhs);
+ return NULL;
+ }
+ lhs = matrix_expr_create_2 (op, lhs, rhs);
+ }
+}
+
+/* Parses a comma-separated list of expressions within {}, transforming them
+ into MOP_PASTE_HORZ operators. Returns the new expression or NULL on
+ error. */
+static struct matrix_expr *
+matrix_parse_curly_comma (struct matrix_state *s)
+{
+ static const struct matrix_operator_syntax op = {
+ .token = T_COMMA, .op = MOP_PASTE_HORZ
+ };
+ return matrix_parse_binary_operator (s, matrix_expr_parse, &op, 1);
+}
+
+/* Parses a semicolon-separated list of expressions within {}, transforming
+ them into MOP_PASTE_VERT operators. Returns the new expression or NULL on
+ error. */
+static struct matrix_expr *
+matrix_parse_curly_semi (struct matrix_state *s)
+{
+ if (lex_token (s->lexer) == T_RCURLY)
+ {
+ /* {} is a special case for a 0×0 matrix. */
+ return matrix_expr_create_0 (MOP_EMPTY);
+ }
+
+ static const struct matrix_operator_syntax op = {
+ .token = T_SEMICOLON, .op = MOP_PASTE_VERT
+ };
+ return matrix_parse_binary_operator (s, matrix_parse_curly_comma, &op, 1);
+}
+
+struct matrix_function
+ {
+ const char *name;
+ enum matrix_op op;
+ size_t min_args, max_args;
+ };
+
+static struct matrix_expr *matrix_expr_parse (struct matrix_state *);
+
+static bool
+word_matches (const char **test, const char **name)
+{
+ size_t test_len = strcspn (*test, ".");
+ size_t name_len = strcspn (*name, ".");
+ if (test_len == name_len)
+ {
+ if (buf_compare_case (*test, *name, test_len))
+ return false;
+ }
+ else if (test_len < 3 || test_len > name_len)
+ return false;
+ else
+ {
+ if (buf_compare_case (*test, *name, test_len))
+ return false;
+ }
+
+ *test += test_len;
+ *name += name_len;
+ if (**test != **name)
+ return false;
+
+ if (**test == '.')
+ {
+ (*test)++;
+ (*name)++;
+ }
+ return true;
+}
+
+/* Returns 0 if TOKEN and FUNC do not match,
+ 1 if TOKEN is an acceptable abbreviation for FUNC,
+ 2 if TOKEN equals FUNC. */
+static int
+compare_function_names (const char *token_, const char *func_)
+{
+ const char *token = token_;
+ const char *func = func_;
+ while (*token || *func)
+ if (!word_matches (&token, &func))
+ return 0;
+ return !c_strcasecmp (token_, func_) ? 2 : 1;
+}
+
+static const struct matrix_function *
+matrix_parse_function_name (const char *token)
+{
+ static const struct matrix_function functions[] =
+ {
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
+ { STRING, MOP_F_##ENUM, PROTO##_MIN_ARGS, PROTO##_MAX_ARGS },
+ MATRIX_FUNCTIONS
+#undef F
+ };
+ enum { N_FUNCTIONS = sizeof functions / sizeof *functions };
+
+ for (size_t i = 0; i < N_FUNCTIONS; i++)
+ {
+ if (compare_function_names (token, functions[i].name) > 0)
+ return &functions[i];
+ }
+ return NULL;
+}
+
+static bool
+matrix_parse_function (struct matrix_state *s, const char *token,
+ struct matrix_expr **exprp)
+{
+ *exprp = NULL;
+ if (lex_next_token (s->lexer, 1) != T_LPAREN)
+ return false;
+
+ int start_ofs = lex_ofs (s->lexer);
+ if (lex_match_id (s->lexer, "EOF"))
+ {
+ lex_get (s->lexer);
+ struct file_handle *fh = fh_parse (s->lexer, FH_REF_FILE, s->session);
+ if (!fh)
+ return true;
+
+ if (!lex_force_match (s->lexer, T_RPAREN))
+ {
+ fh_unref (fh);
+ return true;
+ }
+
+ struct read_file *rf = read_file_create (s, fh);
+
+ struct matrix_expr *e = xmalloc (sizeof *e);
+ *e = (struct matrix_expr) { .op = MOP_EOF, .eof = rf };
+ matrix_expr_add_location (s, start_ofs, e);
+ *exprp = e;
+ return true;
+ }
+
+ const struct matrix_function *f = matrix_parse_function_name (token);
+ if (!f)
+ return false;
+
+ struct matrix_expr *e = xmalloc (sizeof *e);
+ *e = (struct matrix_expr) { .op = f->op };
+
+ lex_get_n (s->lexer, 2);
+ if (lex_token (s->lexer) != T_RPAREN)
+ {
+ size_t allocated_subs = 0;
+ do
+ {
+ struct matrix_expr *sub = matrix_expr_parse (s);
+ if (!sub)
+ goto error;
+
+ if (e->n_subs >= allocated_subs)
+ e->subs = x2nrealloc (e->subs, &allocated_subs, sizeof *e->subs);
+ e->subs[e->n_subs++] = sub;
+ }
+ while (lex_match (s->lexer, T_COMMA));
+ }
+ if (!lex_force_match (s->lexer, T_RPAREN))
+ goto error;
+
+ if (e->n_subs < f->min_args || e->n_subs > f->max_args)
+ {
+ if (f->min_args == f->max_args)
+ msg_at (SE, e->location,
+ ngettext ("Matrix function %s requires %zu argument.",
+ "Matrix function %s requires %zu arguments.",
+ f->min_args),
+ f->name, f->min_args);
+ else if (f->min_args == 1 && f->max_args == 2)
+ msg_at (SE, e->location,
+ ngettext ("Matrix function %s requires 1 or 2 arguments, "
+ "but %zu was provided.",
+ "Matrix function %s requires 1 or 2 arguments, "
+ "but %zu were provided.",
+ e->n_subs),
+ f->name, e->n_subs);
+ else if (f->min_args == 1 && f->max_args == INT_MAX)
+ msg_at (SE, e->location,
+ _("Matrix function %s requires at least one argument."),
+ f->name);
+ else
+ NOT_REACHED ();
+
+ goto error;
+ }
+
+ matrix_expr_add_location (s, start_ofs, e);
+
+ *exprp = e;
+ return true;
+
+error:
+ matrix_expr_destroy (e);
+ return true;
+}
+
+static struct matrix_expr *
+matrix_parse_primary__ (struct matrix_state *s)
+{
+ if (lex_is_number (s->lexer))
+ {
+ double number = lex_number (s->lexer);
+ lex_get (s->lexer);
+
+ return matrix_expr_create_number (number);
+ }
+ else if (lex_is_string (s->lexer))
+ {
+ char string[sizeof (double)];
+ buf_copy_str_rpad (string, sizeof string, lex_tokcstr (s->lexer), ' ');
+ lex_get (s->lexer);
+
+ double number;
+ memcpy (&number, string, sizeof number);
+
+ return matrix_expr_create_number (number);
+ }
+ else if (lex_match (s->lexer, T_LPAREN))
+ {
+ struct matrix_expr *e = matrix_expr_parse (s);
+ if (!e || !lex_force_match (s->lexer, T_RPAREN))
+ {
+ matrix_expr_destroy (e);
+ return NULL;
+ }
+ return e;
+ }
+ else if (lex_match (s->lexer, T_LCURLY))
+ {
+ struct matrix_expr *e = matrix_parse_curly_semi (s);
+ if (!e || !lex_force_match (s->lexer, T_RCURLY))
+ {
+ matrix_expr_destroy (e);
+ return NULL;
+ }
+ return e;
+ }
+ else if (lex_token (s->lexer) == T_ID)
+ {
+ struct matrix_expr *retval;
+ if (matrix_parse_function (s, lex_tokcstr (s->lexer), &retval))
+ return retval;
+
+ struct matrix_var *var = matrix_var_lookup (s, lex_tokss (s->lexer));
+ if (!var)
+ {
+ lex_error (s->lexer, _("Unknown variable %s."),
+ lex_tokcstr (s->lexer));
+ return NULL;
+ }
+ lex_get (s->lexer);
+
+ struct matrix_expr *e = xmalloc (sizeof *e);
+ *e = (struct matrix_expr) { .op = MOP_VARIABLE, .variable = var };
+ return e;
+ }
+ else if (lex_token (s->lexer) == T_ALL)
+ {
+ struct matrix_expr *retval;
+ if (matrix_parse_function (s, "ALL", &retval))
+ return retval;
+ }
+
+ lex_error (s->lexer, _("Syntax error expecting matrix expression."));
+ return NULL;
+}
+
+static struct matrix_expr *
+matrix_parse_primary (struct matrix_state *s)
+{
+ int start_ofs = lex_ofs (s->lexer);
+ struct matrix_expr *e = matrix_parse_primary__ (s);
+ matrix_expr_add_location (s, start_ofs, e);
+ return e;
+}
+
+static struct matrix_expr *matrix_parse_postfix (struct matrix_state *);
+
+static bool
+matrix_parse_index_expr (struct matrix_state *s,
+ struct matrix_expr **indexp,
+ struct msg_location **locationp)
+{
+ if (lex_match (s->lexer, T_COLON))
+ {
+ if (locationp)
+ *locationp = lex_get_location (s->lexer, -1, -1);
+ *indexp = NULL;
+ return true;
+ }
+ else
+ {
+ *indexp = matrix_expr_parse (s);
+ if (locationp && *indexp)
+ *locationp = msg_location_dup (matrix_expr_location (*indexp));
+ return *indexp != NULL;
+ }
+}
+
+static struct matrix_expr *
+matrix_parse_postfix (struct matrix_state *s)
+{
+ struct matrix_expr *lhs = matrix_parse_primary (s);
+ if (!lhs || !lex_match (s->lexer, T_LPAREN))
+ return lhs;
+
+ struct matrix_expr *i0;
+ if (!matrix_parse_index_expr (s, &i0, NULL))
+ {
+ matrix_expr_destroy (lhs);
+ return NULL;
+ }
+ if (lex_match (s->lexer, T_RPAREN))
+ return (i0
+ ? matrix_expr_create_2 (MOP_VEC_INDEX, lhs, i0)
+ : matrix_expr_create_1 (MOP_VEC_ALL, lhs));
+ else if (lex_match (s->lexer, T_COMMA))
+ {
+ struct matrix_expr *i1;
+ if (!matrix_parse_index_expr (s, &i1, NULL)
+ || !lex_force_match (s->lexer, T_RPAREN))
+ {
+ matrix_expr_destroy (lhs);
+ matrix_expr_destroy (i0);
+ matrix_expr_destroy (i1);
+ return NULL;
+ }
+ return (i0 && i1 ? matrix_expr_create_3 (MOP_MAT_INDEX, lhs, i0, i1)
+ : i0 ? matrix_expr_create_2 (MOP_ROW_INDEX, lhs, i0)
+ : i1 ? matrix_expr_create_2 (MOP_COL_INDEX, lhs, i1)
+ : lhs);
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "`)'", "`,'");
+ return NULL;
+ }
+}
+
+static struct matrix_expr *
+matrix_parse_unary (struct matrix_state *s)
+{
+ int start_ofs = lex_ofs (s->lexer);
+
+ struct matrix_expr *e;
+ if (lex_match (s->lexer, T_DASH))
+ {
+ struct matrix_expr *sub = matrix_parse_unary (s);
+ if (!sub)
+ return NULL;
+ e = matrix_expr_create_1 (MOP_NEGATE, sub);
+ }
+ else if (lex_match (s->lexer, T_PLUS))
+ {
+ e = matrix_parse_unary (s);
+ if (!e)
+ return NULL;
+ }
+ else
+ return matrix_parse_postfix (s);
+
+ matrix_expr_add_location (s, start_ofs, e);
+ e->location->start = lex_ofs_start_point (s->lexer, start_ofs);
+ return e;
+}
+
+static struct matrix_expr *
+matrix_parse_seq (struct matrix_state *s)
+{
+ struct matrix_expr *start = matrix_parse_unary (s);
+ if (!start || !lex_match (s->lexer, T_COLON))
+ return start;
+
+ struct matrix_expr *end = matrix_parse_unary (s);
+ if (!end)
+ {
+ matrix_expr_destroy (start);
+ return NULL;
+ }
+
+ if (lex_match (s->lexer, T_COLON))
+ {
+ struct matrix_expr *increment = matrix_parse_unary (s);
+ if (!increment)
+ {
+ matrix_expr_destroy (start);
+ matrix_expr_destroy (end);
+ return NULL;
+ }
+ return matrix_expr_create_3 (MOP_SEQ_BY, start, end, increment);
+ }
+ else
+ return matrix_expr_create_2 (MOP_SEQ, start, end);
+}
+
+static struct matrix_expr *
+matrix_parse_exp (struct matrix_state *s)
+{
+ static const struct matrix_operator_syntax syntax[] = {
+ { .token = T_EXP, .op = MOP_EXP_MAT },
+ { .phrase = "&**", .op = MOP_EXP_ELEMS },
+ };
+ size_t n_syntax = sizeof syntax / sizeof *syntax;
+
+ return matrix_parse_binary_operator (s, matrix_parse_seq, syntax, n_syntax);
+}
+
+static struct matrix_expr *
+matrix_parse_mul_div (struct matrix_state *s)
+{
+ static const struct matrix_operator_syntax syntax[] = {
+ { .token = T_ASTERISK, .op = MOP_MUL_MAT },
+ { .token = T_SLASH, .op = MOP_DIV_ELEMS },
+ { .phrase = "&*", .op = MOP_MUL_ELEMS },
+ { .phrase = "&/", .op = MOP_DIV_ELEMS },
+ };
+ size_t n_syntax = sizeof syntax / sizeof *syntax;
+
+ return matrix_parse_binary_operator (s, matrix_parse_exp, syntax, n_syntax);
+}
+
+static struct matrix_expr *
+matrix_parse_add_sub (struct matrix_state *s)
+{
+ struct matrix_expr *lhs = matrix_parse_mul_div (s);
+ if (!lhs)
+ return NULL;
+
+ for (;;)
+ {
+ enum matrix_op op;
+ if (lex_match (s->lexer, T_PLUS))
+ op = MOP_ADD_ELEMS;
+ else if (lex_match (s->lexer, T_DASH))
+ op = MOP_SUB_ELEMS;
+ else if (lex_token (s->lexer) == T_NEG_NUM)
+ op = MOP_ADD_ELEMS;
+ else
+ return lhs;
+
+ struct matrix_expr *rhs = matrix_parse_mul_div (s);
+ if (!rhs)
+ {
+ matrix_expr_destroy (lhs);
+ return NULL;
+ }
+ lhs = matrix_expr_create_2 (op, lhs, rhs);
+ }
+}
+
+static struct matrix_expr *
+matrix_parse_relational (struct matrix_state *s)
+{
+ static const struct matrix_operator_syntax syntax[] = {
+ { .token = T_GT, .op = MOP_GT },
+ { .token = T_GE, .op = MOP_GE },
+ { .token = T_LT, .op = MOP_LT },
+ { .token = T_LE, .op = MOP_LE },
+ { .token = T_EQUALS, .op = MOP_EQ },
+ { .token = T_EQ, .op = MOP_EQ },
+ { .token = T_NE, .op = MOP_NE },
+ };
+ size_t n_syntax = sizeof syntax / sizeof *syntax;
+
+ return matrix_parse_binary_operator (s, matrix_parse_add_sub,
+ syntax, n_syntax);
+}
+
+static struct matrix_expr *
+matrix_parse_not (struct matrix_state *s)
+{
+ int start_ofs = lex_ofs (s->lexer);
+ if (lex_match (s->lexer, T_NOT))
+ {
+ struct matrix_expr *sub = matrix_parse_not (s);
+ if (!sub)
+ return NULL;
+
+ struct matrix_expr *e = matrix_expr_create_1 (MOP_NOT, sub);
+ matrix_expr_add_location (s, start_ofs, e);
+ e->location->start = lex_ofs_start_point (s->lexer, start_ofs);
+ return e;
+ }
+ else
+ return matrix_parse_relational (s);
+}
+
+static struct matrix_expr *
+matrix_parse_and (struct matrix_state *s)
+{
+ static const struct matrix_operator_syntax op = {
+ .token = T_AND, .op = MOP_AND
+ };
+
+ return matrix_parse_binary_operator (s, matrix_parse_not, &op, 1);
+}
+
+static struct matrix_expr *
+matrix_expr_parse__ (struct matrix_state *s)
+{
+ static const struct matrix_operator_syntax syntax[] = {
+ { .token = T_OR, .op = MOP_OR },
+ { .id = "XOR", .op = MOP_XOR },
+ };
+ size_t n_syntax = sizeof syntax / sizeof *syntax;
+
+ return matrix_parse_binary_operator (s, matrix_parse_and, syntax, n_syntax);
+}
+
+static struct matrix_expr *
+matrix_expr_parse (struct matrix_state *s)
+{
+ int start_ofs = lex_ofs (s->lexer);
+ struct matrix_expr *e = matrix_expr_parse__ (s);
+ matrix_expr_add_location (s, start_ofs, e);
+ return e;
+}
+\f
+/* Matrix expression evaluation. */
+
+/* Iterates over all the elements in matrix M, setting Y and X to the row and
+ column indexes, respectively, and pointing D to the entry at each
+ position. */
+#define MATRIX_FOR_ALL_ELEMENTS(D, Y, X, M) \
+ for (size_t Y = 0; Y < (M)->size1; Y++) \
+ for (size_t X = 0; X < (M)->size2; X++) \
+ for (double *D = gsl_matrix_ptr ((M), Y, X); D; D = NULL)
+
+static bool
+is_vector (const gsl_matrix *m)
+{
+ return m->size1 <= 1 || m->size2 <= 1;
+}
+
+static gsl_vector
+to_vector (gsl_matrix *m)
+{
+ return (m->size1 == 1
+ ? gsl_matrix_row (m, 0).vector
+ : gsl_matrix_column (m, 0).vector);
+}
+
+static double
+matrix_eval_ABS (double d)
+{
+ return fabs (d);
+}
+
+static double
+matrix_eval_ALL (gsl_matrix *m)
+{
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ if (*d == 0.0)
+ return 0.0;
+ return 1.0;
+}
+
+static double
+matrix_eval_ANY (gsl_matrix *m)
+{
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ if (*d != 0.0)
+ return 1.0;
+ return 0.0;
+}
+
+static double
+matrix_eval_ARSIN (double d)
+{
+ return asin (d);
+}
+
+static double
+matrix_eval_ARTAN (double d)
+{
+ return atan (d);
+}
+
+static gsl_matrix *
+matrix_eval_BLOCK (gsl_matrix *m[], size_t n)
+{
+ size_t r = 0;
+ size_t c = 0;
+ for (size_t i = 0; i < n; i++)
+ {
+ r += m[i]->size1;
+ c += m[i]->size2;
+ }
+ gsl_matrix *block = gsl_matrix_calloc (r, c);
+ r = c = 0;
+ for (size_t i = 0; i < n; i++)
+ {
+ for (size_t y = 0; y < m[i]->size1; y++)
+ for (size_t x = 0; x < m[i]->size2; x++)
+ gsl_matrix_set (block, r + y, c + x, gsl_matrix_get (m[i], y, x));
+ r += m[i]->size1;
+ c += m[i]->size2;
+ }
+ return block;
+}
+
+static gsl_matrix *
+matrix_eval_CHOL (gsl_matrix *m, const struct matrix_expr *e)
+{
+ if (!gsl_linalg_cholesky_decomp1 (m))
+ {
+ for (size_t y = 0; y < m->size1; y++)
+ for (size_t x = y + 1; x < m->size2; x++)
+ gsl_matrix_set (m, y, x, gsl_matrix_get (m, x, y));
+
+ for (size_t y = 0; y < m->size1; y++)
+ for (size_t x = 0; x < y; x++)
+ gsl_matrix_set (m, y, x, 0);
+ return m;
+ }
+ else
+ {
+ msg_at (SE, e->subs[0]->location,
+ _("Input to CHOL function is not positive-definite."));
+ return NULL;
+ }
+}
+
+static gsl_matrix *
+matrix_eval_col_extremum (gsl_matrix *m, bool min)
+{
+ if (m->size1 <= 1)
+ return m;
+ else if (!m->size2)
+ return gsl_matrix_alloc (1, 0);
+
+ gsl_matrix *cext = gsl_matrix_alloc (1, m->size2);
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double ext = gsl_matrix_get (m, 0, x);
+ for (size_t y = 1; y < m->size1; y++)
+ {
+ double value = gsl_matrix_get (m, y, x);
+ if (min ? value < ext : value > ext)
+ ext = value;
+ }
+ gsl_matrix_set (cext, 0, x, ext);
+ }
+ return cext;
+}
+
+static gsl_matrix *
+matrix_eval_CMAX (gsl_matrix *m)
+{
+ return matrix_eval_col_extremum (m, false);
+}
+
+static gsl_matrix *
+matrix_eval_CMIN (gsl_matrix *m)
+{
+ return matrix_eval_col_extremum (m, true);
+}
+
+static double
+matrix_eval_COS (double d)
+{
+ return cos (d);
+}
+
+static gsl_matrix *
+matrix_eval_col_sum (gsl_matrix *m, bool square)
+{
+ if (m->size1 == 0)
+ return m;
+ else if (!m->size2)
+ return gsl_matrix_alloc (1, 0);
+
+ gsl_matrix *result = gsl_matrix_alloc (1, m->size2);
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double sum = 0;
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ double d = gsl_matrix_get (m, y, x);
+ sum += square ? pow2 (d) : d;
+ }
+ gsl_matrix_set (result, 0, x, sum);
+ }
+ return result;
+}
+
+static gsl_matrix *
+matrix_eval_CSSQ (gsl_matrix *m)
+{
+ return matrix_eval_col_sum (m, true);
+}
+
+static gsl_matrix *
+matrix_eval_CSUM (gsl_matrix *m)
+{
+ return matrix_eval_col_sum (m, false);
+}
+
+static int
+compare_double_3way (const void *a_, const void *b_)
+{
+ const double *a = a_;
+ const double *b = b_;
+ return *a < *b ? -1 : *a > *b;
+}
+
+static gsl_matrix *
+matrix_eval_DESIGN (gsl_matrix *m, const struct matrix_expr *e)
+{
+ double *tmp = xmalloc (m->size1 * m->size2 * sizeof *tmp);
+ gsl_matrix m2 = gsl_matrix_view_array (tmp, m->size2, m->size1).matrix;
+ gsl_matrix_transpose_memcpy (&m2, m);
+
+ for (size_t y = 0; y < m2.size1; y++)
+ qsort (tmp + y * m2.size2, m2.size2, sizeof *tmp, compare_double_3way);
+
+ size_t *n = xcalloc (m2.size1, sizeof *n);
+ size_t n_total = 0;
+ for (size_t i = 0; i < m2.size1; i++)
+ {
+ double *row = tmp + m2.size2 * i;
+ for (size_t j = 0; j < m2.size2; )
+ {
+ size_t k;
+ for (k = j + 1; k < m2.size2; k++)
+ if (row[j] != row[k])
+ break;
+ row[n[i]++] = row[j];
+ j = k;
+ }
+
+ if (n[i] <= 1)
+ msg_at (MW, e->subs[0]->location,
+ _("Column %zu in DESIGN argument has constant value."), i + 1);
+ else
+ n_total += n[i];
+ }
+
+ gsl_matrix *result = gsl_matrix_alloc (m->size1, n_total);
+ size_t x = 0;
+ for (size_t i = 0; i < m->size2; i++)
+ {
+ if (n[i] <= 1)
+ continue;
+
+ const double *unique = tmp + m2.size2 * i;
+ for (size_t j = 0; j < n[i]; j++, x++)
+ {
+ double value = unique[j];
+ for (size_t y = 0; y < m->size1; y++)
+ gsl_matrix_set (result, y, x, gsl_matrix_get (m, y, i) == value);
+ }
+ }
+
+ free (n);
+ free (tmp);
+
+ return result;
+}
+
+static double
+matrix_eval_DET (gsl_matrix *m)
+{
+ gsl_permutation *p = gsl_permutation_alloc (m->size1);
+ int signum;
+ gsl_linalg_LU_decomp (m, p, &signum);
+ gsl_permutation_free (p);
+ return gsl_linalg_LU_det (m, signum);
+}
+
+static gsl_matrix *
+matrix_eval_DIAG (gsl_matrix *m)
+{
+ gsl_matrix *diag = gsl_matrix_alloc (MIN (m->size1, m->size2), 1);
+ for (size_t i = 0; i < diag->size1; i++)
+ gsl_matrix_set (diag, i, 0, gsl_matrix_get (m, i, i));
+ return diag;
+}
+
+static bool
+is_symmetric (const gsl_matrix *m)
+{
+ if (m->size1 != m->size2)
+ return false;
+
+ for (size_t y = 0; y < m->size1; y++)
+ for (size_t x = 0; x < y; x++)
+ if (gsl_matrix_get (m, y, x) != gsl_matrix_get (m, x, y))
+ return false;
+
+ return true;
+}
+
+static int
+compare_double_desc (const void *a_, const void *b_)
+{
+ const double *a = a_;
+ const double *b = b_;
+ return *a > *b ? -1 : *a < *b;
+}
+
+static gsl_matrix *
+matrix_eval_EVAL (gsl_matrix *m, const struct matrix_expr *e)
+{
+ if (!is_symmetric (m))
+ {
+ msg_at (SE, e->subs[0]->location,
+ _("Argument of EVAL must be symmetric."));
+ return NULL;
+ }
+
+ gsl_eigen_symm_workspace *w = gsl_eigen_symm_alloc (m->size1);
+ gsl_matrix *eval = gsl_matrix_alloc (m->size1, 1);
+ gsl_vector v_eval = to_vector (eval);
+ gsl_eigen_symm (m, &v_eval, w);
+ gsl_eigen_symm_free (w);
+
+ assert (v_eval.stride == 1);
+ qsort (v_eval.data, v_eval.size, sizeof *v_eval.data, compare_double_desc);
+
+ return eval;
+}
+
+static double
+matrix_eval_EXP (double d)
+{
+ return exp (d);
+}
+
+/* From https://gist.github.com/turingbirds/5e99656e08dbe1324c99, where it was
+ marked as:
+
+ Charl Linssen <charl@itfromb.it>
+ Feb 2016
+ PUBLIC DOMAIN */
+static gsl_matrix *
+matrix_eval_GINV (gsl_matrix *A)
+{
+ size_t n = A->size1;
+ size_t m = A->size2;
+ bool swap = m > n;
+ gsl_matrix *tmp_mat = NULL;
+ if (swap)
+ {
+ /* libgsl SVD can only handle the case m <= n, so transpose matrix. */
+ tmp_mat = gsl_matrix_alloc (m, n);
+ gsl_matrix_transpose_memcpy (tmp_mat, A);
+ A = tmp_mat;
+ size_t i = m;
+ m = n;
+ n = i;
+ }
+
+ /* Do SVD. */
+ gsl_matrix *V = gsl_matrix_alloc (m, m);
+ gsl_vector *u = gsl_vector_alloc (m);
+
+ gsl_vector *tmp_vec = gsl_vector_alloc (m);
+ gsl_linalg_SV_decomp (A, V, u, tmp_vec);
+ gsl_vector_free (tmp_vec);
+
+ /* Compute Σ⁻¹. */
+ gsl_matrix *Sigma_pinv = gsl_matrix_alloc (m, n);
+ gsl_matrix_set_zero (Sigma_pinv);
+ double cutoff = 1e-15 * gsl_vector_max (u);
+
+ for (size_t i = 0; i < m; ++i)
+ {
+ double x = gsl_vector_get (u, i);
+ gsl_matrix_set (Sigma_pinv, i, i, x > cutoff ? 1.0 / x : 0);
+ }
+
+ /* libgsl SVD yields "thin" SVD. Pad to full matrix by adding zeros. */
+ gsl_matrix *U = gsl_matrix_calloc (n, n);
+ for (size_t i = 0; i < n; i++)
+ for (size_t j = 0; j < m; j++)
+ gsl_matrix_set (U, i, j, gsl_matrix_get (A, i, j));
+
+ /* Two dot products to obtain pseudoinverse. */
+ gsl_matrix *tmp_mat2 = gsl_matrix_alloc (m, n);
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1., V, Sigma_pinv, 0., tmp_mat2);
+
+ gsl_matrix *A_pinv;
+ if (swap)
+ {
+ A_pinv = gsl_matrix_alloc (n, m);
+ gsl_blas_dgemm (CblasNoTrans, CblasTrans, 1., U, tmp_mat2, 0., A_pinv);
+ }
+ else
+ {
+ A_pinv = gsl_matrix_alloc (m, n);
+ gsl_blas_dgemm (CblasNoTrans, CblasTrans, 1., tmp_mat2, U, 0., A_pinv);
+ }
+
+ gsl_matrix_free (tmp_mat);
+ gsl_matrix_free (tmp_mat2);
+ gsl_matrix_free (U);
+ gsl_matrix_free (Sigma_pinv);
+ gsl_vector_free (u);
+ gsl_matrix_free (V);
+
+ return A_pinv;
+}
+
+struct grade
+ {
+ size_t y, x;
+ double value;
+ };
+
+static int
+grade_compare_3way (const void *a_, const void *b_)
+{
+ const struct grade *a = a_;
+ const struct grade *b = b_;
+
+ return (a->value < b->value ? -1
+ : a->value > b->value ? 1
+ : a->y < b->y ? -1
+ : a->y > b->y ? 1
+ : a->x < b->x ? -1
+ : a->x > b->x);
+}
+
+static gsl_matrix *
+matrix_eval_GRADE (gsl_matrix *m)
+{
+ size_t n = m->size1 * m->size2;
+ struct grade *grades = xmalloc (n * sizeof *grades);
+
+ size_t i = 0;
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ grades[i++] = (struct grade) { .y = y, .x = x, .value = *d };
+ qsort (grades, n, sizeof *grades, grade_compare_3way);
+
+ for (size_t i = 0; i < n; i++)
+ gsl_matrix_set (m, grades[i].y, grades[i].x, i + 1);
+
+ free (grades);
+
+ return m;
+}
+
+static double
+dot (gsl_vector *a, gsl_vector *b)
+{
+ double result = 0.0;
+ for (size_t i = 0; i < a->size; i++)
+ result += gsl_vector_get (a, i) * gsl_vector_get (b, i);
+ return result;
+}
+
+static double
+norm2 (gsl_vector *v)
+{
+ double result = 0.0;
+ for (size_t i = 0; i < v->size; i++)
+ result += pow2 (gsl_vector_get (v, i));
+ return result;
+}
+
+static double
+norm (gsl_vector *v)
+{
+ return sqrt (norm2 (v));
+}
+
+static gsl_matrix *
+matrix_eval_GSCH (gsl_matrix *v, const struct matrix_expr *e)
+{
+ if (v->size2 < v->size1)
+ {
+ msg_at (SE, e->subs[0]->location,
+ _("GSCH requires its argument to have at least as many columns "
+ "as rows, but it has dimensions %zu×%zu."),
+ v->size1, v->size2);
+ return NULL;
+ }
+ if (!v->size1 || !v->size2)
+ return v;
+
+ gsl_matrix *u = gsl_matrix_calloc (v->size1, v->size2);
+ size_t ux = 0;
+ for (size_t vx = 0; vx < v->size2; vx++)
+ {
+ gsl_vector u_i = gsl_matrix_column (u, ux).vector;
+ gsl_vector v_i = gsl_matrix_column (v, vx).vector;
+
+ gsl_vector_memcpy (&u_i, &v_i);
+ for (size_t j = 0; j < ux; j++)
+ {
+ gsl_vector u_j = gsl_matrix_column (u, j).vector;
+ double scale = dot (&u_j, &u_i) / norm2 (&u_j);
+ for (size_t k = 0; k < u_i.size; k++)
+ gsl_vector_set (&u_i, k, (gsl_vector_get (&u_i, k)
+ - scale * gsl_vector_get (&u_j, k)));
+ }
+
+ double len = norm (&u_i);
+ if (len > 1e-15)
+ {
+ gsl_vector_scale (&u_i, 1.0 / len);
+ if (++ux >= v->size1)
+ break;
+ }
+ }
+
+ if (ux < v->size1)
+ {
+ msg_at (SE, e->subs[0]->location,
+ _("%zu×%zu argument to GSCH contains only "
+ "%zu linearly independent columns."),
+ v->size1, v->size2, ux);
+ gsl_matrix_free (u);
+ return NULL;
+ }
+
+ u->size2 = v->size1;
+ return u;
+}
+
+static gsl_matrix *
+matrix_eval_IDENT (double s1, double s2)
+{
+ gsl_matrix *m = gsl_matrix_alloc (s1, s2);
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ *d = x == y;
+ return m;
+}
+
+/* Inverts X, storing the inverse into INVERSE. As a side effect, replaces X
+ by its LU decomposition. */
+static void
+invert_matrix (gsl_matrix *x, gsl_matrix *inverse)
+{
+ gsl_permutation *p = gsl_permutation_alloc (x->size1);
+ int signum;
+ gsl_linalg_LU_decomp (x, p, &signum);
+ gsl_linalg_LU_invert (x, p, inverse);
+ gsl_permutation_free (p);
+}
+
+static gsl_matrix *
+matrix_eval_INV (gsl_matrix *src)
+{
+ gsl_matrix *dst = gsl_matrix_alloc (src->size1, src->size2);
+ invert_matrix (src, dst);
+ return dst;
+}
+
+static gsl_matrix *
+matrix_eval_KRONEKER (gsl_matrix *a, gsl_matrix *b)
+{
+ gsl_matrix *k = gsl_matrix_alloc (a->size1 * b->size1,
+ a->size2 * b->size2);
+ size_t y = 0;
+ for (size_t ar = 0; ar < a->size1; ar++)
+ for (size_t br = 0; br < b->size1; br++, y++)
+ {
+ size_t x = 0;
+ for (size_t ac = 0; ac < a->size2; ac++)
+ for (size_t bc = 0; bc < b->size2; bc++, x++)
+ {
+ double av = gsl_matrix_get (a, ar, ac);
+ double bv = gsl_matrix_get (b, br, bc);
+ gsl_matrix_set (k, y, x, av * bv);
+ }
+ }
+ return k;
+}
+
+static double
+matrix_eval_LG10 (double d)
+{
+ return log10 (d);
+}
+
+static double
+matrix_eval_LN (double d)
+{
+ return log (d);
+}
+
+static void
+matrix_eval_MAGIC_odd (gsl_matrix *m, size_t n)
+{
+ /* Siamese method: https://en.wikipedia.org/wiki/Siamese_method. */
+ size_t y = 0;
+ size_t x = n / 2;
+ for (size_t i = 1; i <= n * n; i++)
+ {
+ gsl_matrix_set (m, y, x, i);
+
+ size_t y1 = !y ? n - 1 : y - 1;
+ size_t x1 = x + 1 >= n ? 0 : x + 1;
+ if (gsl_matrix_get (m, y1, x1) == 0)
+ {
+ y = y1;
+ x = x1;
+ }
+ else
+ y = y + 1 >= n ? 0 : y + 1;
+ }
+}
+
+static void
+magic_exchange (gsl_matrix *m, size_t y1, size_t x1, size_t y2, size_t x2)
+{
+ double a = gsl_matrix_get (m, y1, x1);
+ double b = gsl_matrix_get (m, y2, x2);
+ gsl_matrix_set (m, y1, x1, b);
+ gsl_matrix_set (m, y2, x2, a);
+}
+
+static void
+matrix_eval_MAGIC_doubly_even (gsl_matrix *m, size_t n)
+{
+ size_t x, y;
+
+ /* A. Umar, "On the Construction of Even Order Magic Squares",
+ https://arxiv.org/ftp/arxiv/papers/1202/1202.0948.pdf. */
+ x = y = 0;
+ for (size_t i = 1; i <= n * n / 2; i++)
+ {
+ gsl_matrix_set (m, y, x, i);
+ if (++y >= n)
+ {
+ y = 0;
+ x++;
+ }
+ }
+
+ x = n - 1;
+ y = 0;
+ for (size_t i = n * n; i > n * n / 2; i--)
+ {
+ gsl_matrix_set (m, y, x, i);
+ if (++y >= n)
+ {
+ y = 0;
+ x--;
+ }
+ }
+
+ for (size_t y = 0; y < n; y++)
+ for (size_t x = 0; x < n / 2; x++)
+ {
+ unsigned int d = gsl_matrix_get (m, y, x);
+ if (d % 2 != (y < n / 2))
+ magic_exchange (m, y, x, y, n - x - 1);
+ }
+
+ size_t y1 = n / 2;
+ size_t y2 = n - 1;
+ size_t x1 = n / 2 - 1;
+ size_t x2 = n / 2;
+ magic_exchange (m, y1, x1, y2, x1);
+ magic_exchange (m, y1, x2, y2, x2);
+}
+
+static void
+matrix_eval_MAGIC_singly_even (gsl_matrix *m, size_t n)
+{
+ /* A. Umar, "On the Construction of Even Order Magic Squares",
+ https://arxiv.org/ftp/arxiv/papers/1202/1202.0948.pdf. */
+ size_t x, y;
+
+ x = y = 0;
+ for (size_t i = 1; ; i++)
+ {
+ gsl_matrix_set (m, y, x, i);
+ if (++y == n / 2 - 1)
+ y += 2;
+ else if (y >= n)
+ {
+ y = 0;
+ if (++x >= n / 2)
+ break;
+ }
+ }
+
+ x = n - 1;
+ y = 0;
+ for (size_t i = n * n; ; i--)
+ {
+ gsl_matrix_set (m, y, x, i);
+ if (++y == n / 2 - 1)
+ y += 2;
+ else if (y >= n)
+ {
+ y = 0;
+ if (--x < n / 2)
+ break;
+ }
+ }
+ for (size_t y = 0; y < n; y++)
+ if (y != n / 2 - 1 && y != n / 2)
+ for (size_t x = 0; x < n / 2; x++)
+ {
+ unsigned int d = gsl_matrix_get (m, y, x);
+ if (d % 2 != (y < n / 2))
+ magic_exchange (m, y, x, y, n - x - 1);
+ }
+
+ size_t a0 = (n * n - 2 * n) / 2 + 1;
+ for (size_t i = 0; i < n / 2; i++)
+ {
+ size_t a = a0 + i;
+ gsl_matrix_set (m, n / 2, i, a);
+ gsl_matrix_set (m, n / 2 - 1, i, (n * n + 1) - a);
+ }
+ for (size_t i = 0; i < n / 2; i++)
+ {
+ size_t a = a0 + i + n / 2;
+ gsl_matrix_set (m, n / 2 - 1, n - i - 1, a);
+ gsl_matrix_set (m, n / 2, n - i - 1, (n * n + 1) - a);
+ }
+ for (size_t x = 1; x < n / 2; x += 2)
+ magic_exchange (m, n / 2, x, n / 2 - 1, x);
+ for (size_t x = n / 2 + 2; x <= n - 3; x += 2)
+ magic_exchange (m, n / 2, x, n / 2 - 1, x);
+ size_t x1 = n / 2 - 2;
+ size_t x2 = n / 2 + 1;
+ size_t y1 = n / 2 - 2;
+ size_t y2 = n / 2 + 1;
+ magic_exchange (m, y1, x1, y2, x1);
+ magic_exchange (m, y1, x2, y2, x2);
+}
+
+static gsl_matrix *
+matrix_eval_MAGIC (double n_)
+{
+ size_t n = n_;
+
+ gsl_matrix *m = gsl_matrix_calloc (n, n);
+ if (n % 2)
+ matrix_eval_MAGIC_odd (m, n);
+ else if (n % 4)
+ matrix_eval_MAGIC_singly_even (m, n);
+ else
+ matrix_eval_MAGIC_doubly_even (m, n);
+ return m;
+}
+
+static gsl_matrix *
+matrix_eval_MAKE (double r, double c, double value)
+{
+ gsl_matrix *m = gsl_matrix_alloc (r, c);
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ *d = value;
+ return m;
+}
+
+static gsl_matrix *
+matrix_eval_MDIAG (gsl_vector *v)
+{
+ gsl_matrix *m = gsl_matrix_calloc (v->size, v->size);
+ gsl_vector diagonal = gsl_matrix_diagonal (m).vector;
+ gsl_vector_memcpy (&diagonal, v);
+ return m;
+}
+
+static double
+matrix_eval_MMAX (gsl_matrix *m)
+{
+ return gsl_matrix_max (m);
+}
+
+static double
+matrix_eval_MMIN (gsl_matrix *m)
+{
+ return gsl_matrix_min (m);
+}
+
+static gsl_matrix *
+matrix_eval_MOD (gsl_matrix *m, double divisor)
+{
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ *d = fmod (*d, divisor);
+ return m;
+}
+
+static double
+matrix_eval_MSSQ (gsl_matrix *m)
+{
+ double mssq = 0.0;
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ mssq += *d * *d;
+ return mssq;
+}
+
+static double
+matrix_eval_MSUM (gsl_matrix *m)
+{
+ double msum = 0.0;
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ msum += *d;
+ return msum;
+}
+
+static double
+matrix_eval_NCOL (gsl_matrix *m)
+{
+ return m->size2;
+}
+
+static double
+matrix_eval_NROW (gsl_matrix *m)
+{
+ return m->size1;
+}
+
+static double
+matrix_eval_RANK (gsl_matrix *m)
+{
+ gsl_vector *tau = gsl_vector_alloc (MIN (m->size1, m->size2));
+ gsl_linalg_QR_decomp (m, tau);
+ gsl_vector_free (tau);
+
+ return gsl_linalg_QRPT_rank (m, -1);
+}
+
+static gsl_matrix *
+matrix_eval_RESHAPE (gsl_matrix *m, double r_, double c_,
+ const struct matrix_expr *e)
+{
+ bool r_ok = r_ >= 0 && r_ < SIZE_MAX;
+ bool c_ok = c_ >= 0 && c_ < SIZE_MAX;
+ if (!r_ok || !c_ok)
+ {
+ msg_at (SE,
+ !r_ok ? e->subs[1]->location : e->subs[2]->location,
+ _("Arguments 2 and 3 to RESHAPE must be integers."));
+ return NULL;
+ }
+ size_t r = r_;
+ size_t c = c_;
+ if (size_overflow_p (xtimes (r, xmax (c, 1))) || c * r != m->size1 * m->size2)
+ {
+ struct msg_location *loc = msg_location_dup (e->subs[1]->location);
+ loc->end = e->subs[2]->location->end;
+ msg_at (SE, loc, _("Product of RESHAPE size arguments (%zu×%zu = %zu) "
+ "differs from product of matrix dimensions "
+ "(%zu×%zu = %zu)."),
+ r, c, r * c,
+ m->size1, m->size2, m->size1 * m->size2);
+ msg_location_destroy (loc);
+ return NULL;
+ }
+
+ gsl_matrix *dst = gsl_matrix_alloc (r, c);
+ size_t y1 = 0;
+ size_t x1 = 0;
+ MATRIX_FOR_ALL_ELEMENTS (d, y2, x2, m)
+ {
+ gsl_matrix_set (dst, y1, x1, *d);
+ if (++x1 >= c)
+ {
+ x1 = 0;
+ y1++;
+ }
+ }
+ return dst;
+}
+
+static gsl_matrix *
+matrix_eval_row_extremum (gsl_matrix *m, bool min)
+{
+ if (m->size2 <= 1)
+ return m;
+ else if (!m->size1)
+ return gsl_matrix_alloc (0, 1);
+
+ gsl_matrix *rext = gsl_matrix_alloc (m->size1, 1);
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ double ext = gsl_matrix_get (m, y, 0);
+ for (size_t x = 1; x < m->size2; x++)
+ {
+ double value = gsl_matrix_get (m, y, x);
+ if (min ? value < ext : value > ext)
+ ext = value;
+ }
+ gsl_matrix_set (rext, y, 0, ext);
+ }
+ return rext;
+}
+
+static gsl_matrix *
+matrix_eval_RMAX (gsl_matrix *m)
+{
+ return matrix_eval_row_extremum (m, false);
+}
+
+static gsl_matrix *
+matrix_eval_RMIN (gsl_matrix *m)
+{
+ return matrix_eval_row_extremum (m, true);
+}
+
+static double
+matrix_eval_RND (double d)
+{
+ return rint (d);
+}
+
+struct rank
+ {
+ size_t y, x;
+ double value;
+ };
+
+static int
+rank_compare_3way (const void *a_, const void *b_)
+{
+ const struct rank *a = a_;
+ const struct rank *b = b_;
+
+ return a->value < b->value ? -1 : a->value > b->value;
+}
+
+static gsl_matrix *
+matrix_eval_RNKORDER (gsl_matrix *m)
+{
+ size_t n = m->size1 * m->size2;
+ struct rank *ranks = xmalloc (n * sizeof *ranks);
+ size_t i = 0;
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ ranks[i++] = (struct rank) { .y = y, .x = x, .value = *d };
+ qsort (ranks, n, sizeof *ranks, rank_compare_3way);
+
+ for (size_t i = 0; i < n; )
+ {
+ size_t j;
+ for (j = i + 1; j < n; j++)
+ if (ranks[i].value != ranks[j].value)
+ break;
+
+ double rank = (i + j + 1.0) / 2.0;
+ for (size_t k = i; k < j; k++)
+ gsl_matrix_set (m, ranks[k].y, ranks[k].x, rank);
+
+ i = j;
+ }
+
+ free (ranks);
+
+ return m;
+}
+
+static gsl_matrix *
+matrix_eval_row_sum (gsl_matrix *m, bool square)
+{
+ if (m->size1 == 0)
+ return m;
+ else if (!m->size1)
+ return gsl_matrix_alloc (0, 1);
+
+ gsl_matrix *result = gsl_matrix_alloc (m->size1, 1);
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ double sum = 0;
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double d = gsl_matrix_get (m, y, x);
+ sum += square ? pow2 (d) : d;
+ }
+ gsl_matrix_set (result, y, 0, sum);
+ }
+ return result;
+}
+
+static gsl_matrix *
+matrix_eval_RSSQ (gsl_matrix *m)
+{
+ return matrix_eval_row_sum (m, true);
+}
+
+static gsl_matrix *
+matrix_eval_RSUM (gsl_matrix *m)
+{
+ return matrix_eval_row_sum (m, false);
+}
+
+static double
+matrix_eval_SIN (double d)
+{
+ return sin (d);
+}
+
+static gsl_matrix *
+matrix_eval_SOLVE (gsl_matrix *m1, gsl_matrix *m2, const struct matrix_expr *e)
+{
+ if (m1->size1 != m2->size1)
+ {
+ struct msg_location *loc = msg_location_dup (e->subs[0]->location);
+ loc->end = e->subs[1]->location->end;
+
+ msg_at (SE, e->location,
+ _("SOLVE arguments must have the same number of rows."));
+ msg_at (SN, e->subs[0]->location,
+ _("Argument 1 has dimensions %zu×%zu."), m1->size1, m1->size2);
+ msg_at (SN, e->subs[1]->location,
+ _("Argument 2 has dimensions %zu×%zu."), m2->size1, m2->size2);
+
+ msg_location_destroy (loc);
+ return NULL;
+ }
+
+ gsl_matrix *x = gsl_matrix_alloc (m2->size1, m2->size2);
+ gsl_permutation *p = gsl_permutation_alloc (m1->size1);
+ int signum;
+ gsl_linalg_LU_decomp (m1, p, &signum);
+ for (size_t i = 0; i < m2->size2; i++)
+ {
+ gsl_vector bi = gsl_matrix_column (m2, i).vector;
+ gsl_vector xi = gsl_matrix_column (x, i).vector;
+ gsl_linalg_LU_solve (m1, p, &bi, &xi);
+ }
+ gsl_permutation_free (p);
+ return x;
+}
+
+static double
+matrix_eval_SQRT (double d)
+{
+ return sqrt (d);
+}
+
+static gsl_matrix *
+matrix_eval_SSCP (gsl_matrix *m)
+{
+ gsl_matrix *sscp = gsl_matrix_alloc (m->size2, m->size2);
+ gsl_blas_dgemm (CblasTrans, CblasNoTrans, 1.0, m, m, 0.0, sscp);
+ return sscp;
+}
+
+static gsl_matrix *
+matrix_eval_SVAL (gsl_matrix *m)
+{
+ gsl_matrix *tmp_mat = NULL;
+ if (m->size2 > m->size1)
+ {
+ tmp_mat = gsl_matrix_alloc (m->size2, m->size1);
+ gsl_matrix_transpose_memcpy (tmp_mat, m);
+ m = tmp_mat;
+ }
+
+ /* Do SVD. */
+ gsl_matrix *V = gsl_matrix_alloc (m->size2, m->size2);
+ gsl_vector *S = gsl_vector_alloc (m->size2);
+ gsl_vector *work = gsl_vector_alloc (m->size2);
+ gsl_linalg_SV_decomp (m, V, S, work);
+
+ gsl_matrix *vals = gsl_matrix_alloc (m->size2, 1);
+ for (size_t i = 0; i < m->size2; i++)
+ gsl_matrix_set (vals, i, 0, gsl_vector_get (S, i));
+
+ gsl_matrix_free (V);
+ gsl_vector_free (S);
+ gsl_vector_free (work);
+ gsl_matrix_free (tmp_mat);
+
+ return vals;
+}
+
+static gsl_matrix *
+matrix_eval_SWEEP (gsl_matrix *m, double d, const struct matrix_expr *e)
+{
+ if (d < 1 || d > SIZE_MAX)
+ {
+ msg_at (SE, e->subs[1]->location,
+ _("Scalar argument to SWEEP must be integer."));
+ return NULL;
+ }
+ size_t k = d - 1;
+ if (k >= MIN (m->size1, m->size2))
+ {
+ msg_at (SE, e->subs[1]->location,
+ _("Scalar argument to SWEEP must be integer less than or "
+ "equal to the smaller of the matrix argument's rows and "
+ "columns."));
+ return NULL;
+ }
+
+ double m_kk = gsl_matrix_get (m, k, k);
+ if (fabs (m_kk) > 1e-19)
+ {
+ gsl_matrix *a = gsl_matrix_alloc (m->size1, m->size2);
+ MATRIX_FOR_ALL_ELEMENTS (a_ij, i, j, a)
+ {
+ double m_ij = gsl_matrix_get (m, i, j);
+ double m_ik = gsl_matrix_get (m, i, k);
+ double m_kj = gsl_matrix_get (m, k, j);
+ *a_ij = (i != k && j != k ? m_ij * m_kk - m_ik * m_kj
+ : i != k ? -m_ik
+ : j != k ? m_kj
+ : 1.0) / m_kk;
+ }
+ return a;
+ }
+ else
+ {
+ for (size_t i = 0; i < m->size1; i++)
+ {
+ gsl_matrix_set (m, i, k, 0);
+ gsl_matrix_set (m, k, i, 0);
+ }
+ return m;
+ }
+}
+
+static double
+matrix_eval_TRACE (gsl_matrix *m)
+{
+ double sum = 0;
+ size_t n = MIN (m->size1, m->size2);
+ for (size_t i = 0; i < n; i++)
+ sum += gsl_matrix_get (m, i, i);
+ return sum;
+}
+
+static gsl_matrix *
+matrix_eval_T (gsl_matrix *m)
+{
+ return matrix_eval_TRANSPOS (m);
+}
+
+static gsl_matrix *
+matrix_eval_TRANSPOS (gsl_matrix *m)
+{
+ if (m->size1 == m->size2)
+ {
+ gsl_matrix_transpose (m);
+ return m;
+ }
+ else
+ {
+ gsl_matrix *t = gsl_matrix_alloc (m->size2, m->size1);
+ gsl_matrix_transpose_memcpy (t, m);
+ return t;
+ }
+}
+
+static double
+matrix_eval_TRUNC (double d)
+{
+ return trunc (d);
+}
+
+static gsl_matrix *
+matrix_eval_UNIFORM (double r_, double c_, const struct matrix_expr *e)
+{
+ size_t r = r_;
+ size_t c = c_;
+ if (size_overflow_p (xtimes (r, xmax (c, 1))))
+ {
+ struct msg_location *loc = msg_location_dup (e->subs[0]->location);
+ loc->end = e->subs[1]->location->end;
+
+ msg_at (SE, loc,
+ _("Product of arguments to UNIFORM exceeds memory size."));
+
+ msg_location_destroy (loc);
+ return NULL;
+ }
+
+ gsl_matrix *m = gsl_matrix_alloc (r, c);
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
+ *d = gsl_ran_flat (get_rng (), 0, 1);
+ return m;
+}
+
+static double
+matrix_eval_PDF_BETA (double x, double a, double b)
+{
+ return gsl_ran_beta_pdf (x, a, b);
+}
+
+static double
+matrix_eval_CDF_BETA (double x, double a, double b)
+{
+ return gsl_cdf_beta_P (x, a, b);
+}
+
+static double
+matrix_eval_IDF_BETA (double P, double a, double b)
+{
+ return gsl_cdf_beta_Pinv (P, a, b);
+}
+
+static double
+matrix_eval_RV_BETA (double a, double b)
+{
+ return gsl_ran_beta (get_rng (), a, b);
+}
+
+static double
+matrix_eval_NCDF_BETA (double x, double a, double b, double lambda)
+{
+ return ncdf_beta (x, a, b, lambda);
+}
+
+static double
+matrix_eval_NPDF_BETA (double x, double a, double b, double lambda)
+{
+ return npdf_beta (x, a, b, lambda);
+}
+
+static double
+matrix_eval_CDF_BVNOR (double x0, double x1, double r)
+{
+ return cdf_bvnor (x0, x1, r);
+}
+
+static double
+matrix_eval_PDF_BVNOR (double x0, double x1, double r)
+{
+ return gsl_ran_bivariate_gaussian_pdf (x0, x1, 1, 1, r);
+}
+
+static double
+matrix_eval_CDF_CAUCHY (double x, double a, double b)
+{
+ return gsl_cdf_cauchy_P ((x - a) / b, 1);
+}
+
+static double
+matrix_eval_IDF_CAUCHY (double P, double a, double b)
+{
+ return a + b * gsl_cdf_cauchy_Pinv (P, 1);
+}
+
+static double
+matrix_eval_PDF_CAUCHY (double x, double a, double b)
+{
+ return gsl_ran_cauchy_pdf ((x - a) / b, 1) / b;
+}
+
+static double
+matrix_eval_RV_CAUCHY (double a, double b)
+{
+ return a + b * gsl_ran_cauchy (get_rng (), 1);
+}
+
+static double
+matrix_eval_CDF_CHISQ (double x, double df)
+{
+ return gsl_cdf_chisq_P (x, df);
+}
+
+static double
+matrix_eval_CHICDF (double x, double df)
+{
+ return matrix_eval_CDF_CHISQ (x, df);
+}
+
+static double
+matrix_eval_IDF_CHISQ (double P, double df)
+{
+ return gsl_cdf_chisq_Pinv (P, df);
+}
+
+static double
+matrix_eval_PDF_CHISQ (double x, double df)
+{
+ return gsl_ran_chisq_pdf (x, df);
+}
+
+static double
+matrix_eval_RV_CHISQ (double df)
+{
+ return gsl_ran_chisq (get_rng (), df);
+}
+
+static double
+matrix_eval_SIG_CHISQ (double x, double df)
+{
+ return gsl_cdf_chisq_Q (x, df);
+}
+
+static double
+matrix_eval_CDF_EXP (double x, double a)
+{
+ return gsl_cdf_exponential_P (x, 1. / a);
+}
+
+static double
+matrix_eval_IDF_EXP (double P, double a)
+{
+ return gsl_cdf_exponential_Pinv (P, 1. / a);
+}
+
+static double
+matrix_eval_PDF_EXP (double x, double a)
+{
+ return gsl_ran_exponential_pdf (x, 1. / a);
+}
+
+static double
+matrix_eval_RV_EXP (double a)
+{
+ return gsl_ran_exponential (get_rng (), 1. / a);
+}
+
+static double
+matrix_eval_PDF_XPOWER (double x, double a, double b)
+{
+ return gsl_ran_exppow_pdf (x, a, b);
+}
+
+static double
+matrix_eval_RV_XPOWER (double a, double b)
+{
+ return gsl_ran_exppow (get_rng (), a, b);
+}
+
+static double
+matrix_eval_CDF_F (double x, double df1, double df2)
+{
+ return gsl_cdf_fdist_P (x, df1, df2);
+}
+
+static double
+matrix_eval_FCDF (double x, double df1, double df2)
+{
+ return matrix_eval_CDF_F (x, df1, df2);
+}
+
+static double
+matrix_eval_IDF_F (double P, double df1, double df2)
+{
+ return idf_fdist (P, df1, df2);
+}
+
+static double
+matrix_eval_RV_F (double df1, double df2)
+{
+ return gsl_ran_fdist (get_rng (), df1, df2);
+}
+
+static double
+matrix_eval_PDF_F (double x, double df1, double df2)
+{
+ return gsl_ran_fdist_pdf (x, df1, df2);
+}
+
+static double
+matrix_eval_SIG_F (double x, double df1, double df2)
+{
+ return gsl_cdf_fdist_Q (x, df1, df2);
+}
+
+static double
+matrix_eval_CDF_GAMMA (double x, double a, double b)
+{
+ return gsl_cdf_gamma_P (x, a, 1. / b);
+}
+
+static double
+matrix_eval_IDF_GAMMA (double P, double a, double b)
+{
+ return gsl_cdf_gamma_Pinv (P, a, 1. / b);
+}
+
+static double
+matrix_eval_PDF_GAMMA (double x, double a, double b)
+{
+ return gsl_ran_gamma_pdf (x, a, 1. / b);
+}
+
+static double
+matrix_eval_RV_GAMMA (double a, double b)
+{
+ return gsl_ran_gamma (get_rng (), a, 1. / b);
+}
+
+static double
+matrix_eval_PDF_LANDAU (double x)
+{
+ return gsl_ran_landau_pdf (x);
+}
+
+static double
+matrix_eval_RV_LANDAU (void)
+{
+ return gsl_ran_landau (get_rng ());
+}
+
+static double
+matrix_eval_CDF_LAPLACE (double x, double a, double b)
+{
+ return gsl_cdf_laplace_P ((x - a) / b, 1);
+}
+
+static double
+matrix_eval_IDF_LAPLACE (double P, double a, double b)
+{
+ return a + b * gsl_cdf_laplace_Pinv (P, 1);
+}
+
+static double
+matrix_eval_PDF_LAPLACE (double x, double a, double b)
+{
+ return gsl_ran_laplace_pdf ((x - a) / b, 1);
+}
+
+static double
+matrix_eval_RV_LAPLACE (double a, double b)
+{
+ return a + b * gsl_ran_laplace (get_rng (), 1);
+}
+
+static double
+matrix_eval_RV_LEVY (double c, double alpha)
+{
+ return gsl_ran_levy (get_rng (), c, alpha);
+}
+
+static double
+matrix_eval_RV_LVSKEW (double c, double alpha, double beta)
+{
+ return gsl_ran_levy_skew (get_rng (), c, alpha, beta);
+}
+
+static double
+matrix_eval_CDF_LOGISTIC (double x, double a, double b)
+{
+ return gsl_cdf_logistic_P ((x - a) / b, 1);
+}
+
+static double
+matrix_eval_IDF_LOGISTIC (double P, double a, double b)
+{
+ return a + b * gsl_cdf_logistic_Pinv (P, 1);
+}
+
+static double
+matrix_eval_PDF_LOGISTIC (double x, double a, double b)
+{
+ return gsl_ran_logistic_pdf ((x - a) / b, 1) / b;
+}
+
+static double
+matrix_eval_RV_LOGISTIC (double a, double b)
+{
+ return a + b * gsl_ran_logistic (get_rng (), 1);
+}
+
+static double
+matrix_eval_CDF_LNORMAL (double x, double m, double s)
+{
+ return gsl_cdf_lognormal_P (x, log (m), s);
+}
+
+static double
+matrix_eval_IDF_LNORMAL (double P, double m, double s)
+{
+ return gsl_cdf_lognormal_Pinv (P, log (m), s);;
+}
+
+static double
+matrix_eval_PDF_LNORMAL (double x, double m, double s)
+{
+ return gsl_ran_lognormal_pdf (x, log (m), s);
+}
+
+static double
+matrix_eval_RV_LNORMAL (double m, double s)
+{
+ return gsl_ran_lognormal (get_rng (), log (m), s);
+}
+
+static double
+matrix_eval_CDF_NORMAL (double x, double u, double s)
+{
+ return gsl_cdf_gaussian_P (x - u, s);
+}
+
+static double
+matrix_eval_IDF_NORMAL (double P, double u, double s)
+{
+ return u + gsl_cdf_gaussian_Pinv (P, s);
+}
+
+static double
+matrix_eval_PDF_NORMAL (double x, double u, double s)
+{
+ return gsl_ran_gaussian_pdf ((x - u) / s, 1) / s;
+}
+
+static double
+matrix_eval_RV_NORMAL (double u, double s)
+{
+ return u + gsl_ran_gaussian (get_rng (), s);
+}
+
+static double
+matrix_eval_CDFNORM (double x)
+{
+ return gsl_cdf_ugaussian_P (x);
+}
+
+static double
+matrix_eval_PROBIT (double P)
+{
+ return gsl_cdf_ugaussian_Pinv (P);
+}
+
+static double
+matrix_eval_NORMAL (double s)
+{
+ return gsl_ran_gaussian (get_rng (), s);
+}
+
+static double
+matrix_eval_PDF_NTAIL (double x, double a, double sigma)
+{
+ return gsl_ran_gaussian_tail_pdf (x, a, sigma);;
+}
+
+static double
+matrix_eval_RV_NTAIL (double a, double sigma)
+{
+ return gsl_ran_gaussian_tail (get_rng (), a, sigma);
+}
+
+static double
+matrix_eval_CDF_PARETO (double x, double a, double b)
+{
+ return gsl_cdf_pareto_P (x, b, a);
+}
+
+static double
+matrix_eval_IDF_PARETO (double P, double a, double b)
+{
+ return gsl_cdf_pareto_Pinv (P, b, a);
+}
+
+static double
+matrix_eval_PDF_PARETO (double x, double a, double b)
+{
+ return gsl_ran_pareto_pdf (x, b, a);
+}
+
+static double
+matrix_eval_RV_PARETO (double a, double b)
+{
+ return gsl_ran_pareto (get_rng (), b, a);
+}
+
+static double
+matrix_eval_CDF_RAYLEIGH (double x, double sigma)
+{
+ return gsl_cdf_rayleigh_P (x, sigma);
+}
+
+static double
+matrix_eval_IDF_RAYLEIGH (double P, double sigma)
+{
+ return gsl_cdf_rayleigh_Pinv (P, sigma);
+}
+
+static double
+matrix_eval_PDF_RAYLEIGH (double x, double sigma)
+{
+ return gsl_ran_rayleigh_pdf (x, sigma);
+}
+
+static double
+matrix_eval_RV_RAYLEIGH (double sigma)
+{
+ return gsl_ran_rayleigh (get_rng (), sigma);
+}
+
+static double
+matrix_eval_PDF_RTAIL (double x, double a, double sigma)
+{
+ return gsl_ran_rayleigh_tail_pdf (x, a, sigma);
+}
+
+static double
+matrix_eval_RV_RTAIL (double a, double sigma)
+{
+ return gsl_ran_rayleigh_tail (get_rng (), a, sigma);
+}
+
+static double
+matrix_eval_CDF_T (double x, double df)
+{
+ return gsl_cdf_tdist_P (x, df);
+}
+
+static double
+matrix_eval_TCDF (double x, double df)
+{
+ return matrix_eval_CDF_T (x, df);
+}
+
+static double
+matrix_eval_IDF_T (double P, double df)
+{
+ return gsl_cdf_tdist_Pinv (P, df);
+}
+
+static double
+matrix_eval_PDF_T (double x, double df)
+{
+ return gsl_ran_tdist_pdf (x, df);
+}
+
+static double
+matrix_eval_RV_T (double df)
+{
+ return gsl_ran_tdist (get_rng (), df);
+}
+
+static double
+matrix_eval_CDF_T1G (double x, double a, double b)
+{
+ return gsl_cdf_gumbel1_P (x, a, b);
+}
+
+static double
+matrix_eval_IDF_T1G (double P, double a, double b)
+{
+ return gsl_cdf_gumbel1_Pinv (P, a, b);
+}
+
+static double
+matrix_eval_PDF_T1G (double x, double a, double b)
+{
+ return gsl_ran_gumbel1_pdf (x, a, b);
+}
+
+static double
+matrix_eval_RV_T1G (double a, double b)
+{
+ return gsl_ran_gumbel1 (get_rng (), a, b);
+}
+
+static double
+matrix_eval_CDF_T2G (double x, double a, double b)
+{
+ return gsl_cdf_gumbel1_P (x, a, b);
+}
+
+static double
+matrix_eval_IDF_T2G (double P, double a, double b)
+{
+ return gsl_cdf_gumbel1_Pinv (P, a, b);
+}
+
+static double
+matrix_eval_PDF_T2G (double x, double a, double b)
+{
+ return gsl_ran_gumbel1_pdf (x, a, b);
+}
+
+static double
+matrix_eval_RV_T2G (double a, double b)
+{
+ return gsl_ran_gumbel1 (get_rng (), a, b);
+}
+
+static double
+matrix_eval_CDF_UNIFORM (double x, double a, double b)
+{
+ return gsl_cdf_flat_P (x, a, b);
+}
+
+static double
+matrix_eval_IDF_UNIFORM (double P, double a, double b)
+{
+ return gsl_cdf_flat_Pinv (P, a, b);
+}
+
+static double
+matrix_eval_PDF_UNIFORM (double x, double a, double b)
+{
+ return gsl_ran_flat_pdf (x, a, b);
+}
+
+static double
+matrix_eval_RV_UNIFORM (double a, double b)
+{
+ return gsl_ran_flat (get_rng (), a, b);
+}
+
+static double
+matrix_eval_CDF_WEIBULL (double x, double a, double b)
+{
+ return gsl_cdf_weibull_P (x, a, b);
+}
+
+static double
+matrix_eval_IDF_WEIBULL (double P, double a, double b)
+{
+ return gsl_cdf_weibull_Pinv (P, a, b);
+}
+
+static double
+matrix_eval_PDF_WEIBULL (double x, double a, double b)
+{
+ return gsl_ran_weibull_pdf (x, a, b);
+}
+
+static double
+matrix_eval_RV_WEIBULL (double a, double b)
+{
+ return gsl_ran_weibull (get_rng (), a, b);
+}
+
+static double
+matrix_eval_CDF_BERNOULLI (double k, double p)
+{
+ return k ? 1 : 1 - p;
+}
+
+static double
+matrix_eval_PDF_BERNOULLI (double k, double p)
+{
+ return gsl_ran_bernoulli_pdf (k, p);
+}
+
+static double
+matrix_eval_RV_BERNOULLI (double p)
+{
+ return gsl_ran_bernoulli (get_rng (), p);
+}
+
+static double
+matrix_eval_CDF_BINOM (double k, double n, double p)
+{
+ return gsl_cdf_binomial_P (k, p, n);
+}
+
+static double
+matrix_eval_PDF_BINOM (double k, double n, double p)
+{
+ return gsl_ran_binomial_pdf (k, p, n);
+}
+
+static double
+matrix_eval_RV_BINOM (double n, double p)
+{
+ return gsl_ran_binomial (get_rng (), p, n);
+}
+
+static double
+matrix_eval_CDF_GEOM (double k, double p)
+{
+ return gsl_cdf_geometric_P (k, p);
+}
+
+static double
+matrix_eval_PDF_GEOM (double k, double p)
+{
+ return gsl_ran_geometric_pdf (k, p);
+}
+
+static double
+matrix_eval_RV_GEOM (double p)
+{
+ return gsl_ran_geometric (get_rng (), p);
+}
+
+static double
+matrix_eval_CDF_HYPER (double k, double a, double b, double c)
+{
+ return gsl_cdf_hypergeometric_P (k, c, a - c, b);
+}
+
+static double
+matrix_eval_PDF_HYPER (double k, double a, double b, double c)
+{
+ return gsl_ran_hypergeometric_pdf (k, c, a - c, b);
+}
+
+static double
+matrix_eval_RV_HYPER (double a, double b, double c)
+{
+ return gsl_ran_hypergeometric (get_rng (), c, a - c, b);
+}
+
+static double
+matrix_eval_PDF_LOG (double k, double p)
+{
+ return gsl_ran_logarithmic_pdf (k, p);
+}
+
+static double
+matrix_eval_RV_LOG (double p)
+{
+ return gsl_ran_logarithmic (get_rng (), p);
+}
+
+static double
+matrix_eval_CDF_NEGBIN (double k, double n, double p)
+{
+ return gsl_cdf_negative_binomial_P (k, p, n);
+}
+
+static double
+matrix_eval_PDF_NEGBIN (double k, double n, double p)
+{
+ return gsl_ran_negative_binomial_pdf (k, p, n);
+}
+
+static double
+matrix_eval_RV_NEGBIN (double n, double p)
+{
+ return gsl_ran_negative_binomial (get_rng (), p, n);
+}
+
+static double
+matrix_eval_CDF_POISSON (double k, double mu)
+{
+ return gsl_cdf_poisson_P (k, mu);
+}
+
+static double
+matrix_eval_PDF_POISSON (double k, double mu)
+{
+ return gsl_ran_poisson_pdf (k, mu);
+}
+
+static double
+matrix_eval_RV_POISSON (double mu)
+{
+ return gsl_ran_poisson (get_rng (), mu);
+}
+
+static double
+matrix_op_eval (enum matrix_op op, double a, double b)
+{
+ switch (op)
+ {
+ case MOP_ADD_ELEMS: return a + b;
+ case MOP_SUB_ELEMS: return a - b;
+ case MOP_MUL_ELEMS: return a * b;
+ case MOP_DIV_ELEMS: return a / b;
+ case MOP_EXP_ELEMS: return pow (a, b);
+ case MOP_GT: return a > b;
+ case MOP_GE: return a >= b;
+ case MOP_LT: return a < b;
+ case MOP_LE: return a <= b;
+ case MOP_EQ: return a == b;
+ case MOP_NE: return a != b;
+ case MOP_AND: return (a > 0) && (b > 0);
+ case MOP_OR: return (a > 0) || (b > 0);
+ case MOP_XOR: return (a > 0) != (b > 0);
+
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) case MOP_F_##ENUM:
+ MATRIX_FUNCTIONS
+#undef F
+ case MOP_NEGATE:
+ case MOP_SEQ:
+ case MOP_SEQ_BY:
+ case MOP_MUL_MAT:
+ case MOP_EXP_MAT:
+ case MOP_NOT:
+ case MOP_PASTE_HORZ:
+ case MOP_PASTE_VERT:
+ case MOP_EMPTY:
+ case MOP_VEC_INDEX:
+ case MOP_VEC_ALL:
+ case MOP_MAT_INDEX:
+ case MOP_ROW_INDEX:
+ case MOP_COL_INDEX:
+ case MOP_NUMBER:
+ case MOP_VARIABLE:
+ case MOP_EOF:
+ NOT_REACHED ();
+ }
+ NOT_REACHED ();
+}
+
+static const char *
+matrix_op_name (enum matrix_op op)
+{
+ switch (op)
+ {
+ case MOP_ADD_ELEMS: return "+";
+ case MOP_SUB_ELEMS: return "-";
+ case MOP_MUL_ELEMS: return "&*";
+ case MOP_DIV_ELEMS: return "&/";
+ case MOP_EXP_ELEMS: return "&**";
+ case MOP_GT: return ">";
+ case MOP_GE: return ">=";
+ case MOP_LT: return "<";
+ case MOP_LE: return "<=";
+ case MOP_EQ: return "=";
+ case MOP_NE: return "<>";
+ case MOP_AND: return "AND";
+ case MOP_OR: return "OR";
+ case MOP_XOR: return "XOR";
+
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) case MOP_F_##ENUM:
+ MATRIX_FUNCTIONS
+#undef F
+ case MOP_NEGATE:
+ case MOP_SEQ:
+ case MOP_SEQ_BY:
+ case MOP_MUL_MAT:
+ case MOP_EXP_MAT:
+ case MOP_NOT:
+ case MOP_PASTE_HORZ:
+ case MOP_PASTE_VERT:
+ case MOP_EMPTY:
+ case MOP_VEC_INDEX:
+ case MOP_VEC_ALL:
+ case MOP_MAT_INDEX:
+ case MOP_ROW_INDEX:
+ case MOP_COL_INDEX:
+ case MOP_NUMBER:
+ case MOP_VARIABLE:
+ case MOP_EOF:
+ NOT_REACHED ();
+ }
+ NOT_REACHED ();
+}
+
+static bool
+is_scalar (const gsl_matrix *m)
+{
+ return m->size1 == 1 && m->size2 == 1;
+}
+
+static double
+to_scalar (const gsl_matrix *m)
+{
+ assert (is_scalar (m));
+ return gsl_matrix_get (m, 0, 0);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_elementwise (const struct matrix_expr *e,
+ enum matrix_op op,
+ gsl_matrix *a, gsl_matrix *b)
+{
+ if (is_scalar (b))
+ {
+ double be = to_scalar (b);
+ for (size_t r = 0; r < a->size1; r++)
+ for (size_t c = 0; c < a->size2; c++)
+ {
+ double *ae = gsl_matrix_ptr (a, r, c);
+ *ae = matrix_op_eval (op, *ae, be);
+ }
+ return a;
+ }
+ else if (is_scalar (a))
+ {
+ double ae = to_scalar (a);
+ for (size_t r = 0; r < b->size1; r++)
+ for (size_t c = 0; c < b->size2; c++)
+ {
+ double *be = gsl_matrix_ptr (b, r, c);
+ *be = matrix_op_eval (op, ae, *be);
+ }
+ return b;
+ }
+ else if (a->size1 == b->size1 && a->size2 == b->size2)
+ {
+ for (size_t r = 0; r < a->size1; r++)
+ for (size_t c = 0; c < a->size2; c++)
+ {
+ double *ae = gsl_matrix_ptr (a, r, c);
+ double be = gsl_matrix_get (b, r, c);
+ *ae = matrix_op_eval (op, *ae, be);
+ }
+ return a;
+ }
+ else
+ {
+ msg_at (SE, matrix_expr_location (e),
+ _("The operands of %s must have the same dimensions or one "
+ "must be a scalar."),
+ matrix_op_name (op));
+ msg_at (SN, matrix_expr_location (e->subs[0]),
+ _("The left-hand operand is a %zu×%zu matrix."),
+ a->size1, a->size2);
+ msg_at (SN, matrix_expr_location (e->subs[1]),
+ _("The right-hand operand is a %zu×%zu matrix."),
+ b->size1, b->size2);
+ return NULL;
+ }
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_mul_mat (const struct matrix_expr *e,
+ gsl_matrix *a, gsl_matrix *b)
+{
+ if (is_scalar (a) || is_scalar (b))
+ return matrix_expr_evaluate_elementwise (e, MOP_MUL_ELEMS, a, b);
+
+ if (a->size2 != b->size1)
+ {
+ msg_at (SE, e->location,
+ _("Matrices not conformable for multiplication."));
+ msg_at (SN, matrix_expr_location (e->subs[0]),
+ _("The left-hand operand is a %zu×%zu matrix."),
+ a->size1, a->size2);
+ msg_at (SN, matrix_expr_location (e->subs[1]),
+ _("The right-hand operand is a %zu×%zu matrix."),
+ b->size1, b->size2);
+ return NULL;
+ }
+
+ gsl_matrix *c = gsl_matrix_alloc (a->size1, b->size2);
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, a, b, 0.0, c);
+ return c;
+}
+
+static void
+swap_matrix (gsl_matrix **a, gsl_matrix **b)
+{
+ gsl_matrix *tmp = *a;
+ *a = *b;
+ *b = tmp;
+}
+
+static void
+mul_matrix (gsl_matrix **z, const gsl_matrix *x, const gsl_matrix *y,
+ gsl_matrix **tmp)
+{
+ gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, x, y, 0.0, *tmp);
+ swap_matrix (z, tmp);
+}
+
+static void
+square_matrix (gsl_matrix **x, gsl_matrix **tmp)
+{
+ mul_matrix (x, *x, *x, tmp);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_exp_mat (const struct matrix_expr *e,
+ gsl_matrix *x_, gsl_matrix *b)
+{
+ gsl_matrix *x = x_;
+ if (x->size1 != x->size2)
+ {
+ msg_at (SE, matrix_expr_location (e->subs[0]),
+ _("Matrix exponentation with ** requires a square matrix on "
+ "the left-hand size, not one with dimensions %zu×%zu."),
+ x->size1, x->size2);
+ return NULL;
+ }
+ if (!is_scalar (b))
+ {
+ msg_at (SE, matrix_expr_location (e->subs[1]),
+ _("Matrix exponentiation with ** requires a scalar on the "
+ "right-hand side, not a matrix with dimensions %zu×%zu."),
+ b->size1, b->size2);
+ return NULL;
+ }
+ double bf = to_scalar (b);
+ if (bf != floor (bf) || bf <= LONG_MIN || bf > LONG_MAX)
+ {
+ msg_at (SE, matrix_expr_location (e->subs[1]),
+ _("Exponent %.1f in matrix exponentiation is non-integer "
+ "or outside the valid range."), bf);
+ return NULL;
+ }
+ long int bl = bf;
+
+ gsl_matrix *y_ = gsl_matrix_alloc (x->size1, x->size2);
+ gsl_matrix *y = y_;
+ gsl_matrix_set_identity (y);
+ if (bl == 0)
+ return y;
+
+ gsl_matrix *t_ = gsl_matrix_alloc (x->size1, x->size2);
+ gsl_matrix *t = t_;
+ for (unsigned long int n = labs (bl); n > 1; n /= 2)
+ if (n & 1)
+ {
+ mul_matrix (&y, x, y, &t);
+ square_matrix (&x, &t);
+ }
+ else
+ square_matrix (&x, &t);
+
+ mul_matrix (&y, x, y, &t);
+ if (bf < 0)
+ {
+ invert_matrix (y, x);
+ swap_matrix (&x, &y);
+ }
+
+ /* Garbage collection.
+
+ There are three matrices: 'x_', 'y_', and 't_', and 'x', 'y', and 't' are
+ a permutation of them. We are returning one of them; that one must not be
+ destroyed. We must not destroy 'x_' because the caller owns it. */
+ if (y != y_)
+ gsl_matrix_free (y_);
+ if (y != t_)
+ gsl_matrix_free (t_);
+
+ return y;
+}
+
+static void
+note_operand_size (const gsl_matrix *m, const struct matrix_expr *e)
+{
+ msg_at (SN, matrix_expr_location (e),
+ _("This operand is a %zu×%zu matrix."), m->size1, m->size2);
+}
+
+static void
+note_nonscalar (const gsl_matrix *m, const struct matrix_expr *e)
+{
+ if (!is_scalar (m))
+ note_operand_size (m, e);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_seq (const struct matrix_expr *e,
+ gsl_matrix *start_, gsl_matrix *end_,
+ gsl_matrix *by_)
+{
+ if (!is_scalar (start_) || !is_scalar (end_) || (by_ && !is_scalar (by_)))
+ {
+ msg_at (SE, matrix_expr_location (e),
+ _("All operands of : operator must be scalars."));
+
+ note_nonscalar (start_, e->subs[0]);
+ note_nonscalar (end_, e->subs[1]);
+ if (by_)
+ note_nonscalar (by_, e->subs[2]);
+ return NULL;
+ }
+
+ long int start = to_scalar (start_);
+ long int end = to_scalar (end_);
+ long int by = by_ ? to_scalar (by_) : 1;
+
+ if (!by)
+ {
+ msg_at (SE, matrix_expr_location (e->subs[2]),
+ _("The increment operand to : must be nonzero."));
+ return NULL;
+ }
+
+ long int n = (end >= start && by > 0 ? (end - start + by) / by
+ : end <= start && by < 0 ? (start - end - by) / -by
+ : 0);
+ gsl_matrix *m = gsl_matrix_alloc (1, n);
+ for (long int i = 0; i < n; i++)
+ gsl_matrix_set (m, 0, i, start + i * by);
+ return m;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_not (gsl_matrix *a)
+{
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, a)
+ *d = !(*d > 0);
+ return a;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_paste_horz (const struct matrix_expr *e,
+ gsl_matrix *a, gsl_matrix *b)
+{
+ if (a->size1 != b->size1)
+ {
+ if (!a->size1 || !a->size2)
+ return b;
+ else if (!b->size1 || !b->size2)
+ return a;
+
+ msg_at (SE, matrix_expr_location (e),
+ _("This expression tries to horizontally join matrices with "
+ "differing numbers of rows."));
+ note_operand_size (a, e->subs[0]);
+ note_operand_size (b, e->subs[1]);
+ return NULL;
+ }
+
+ gsl_matrix *c = gsl_matrix_alloc (a->size1, a->size2 + b->size2);
+ for (size_t y = 0; y < a->size1; y++)
+ {
+ for (size_t x = 0; x < a->size2; x++)
+ gsl_matrix_set (c, y, x, gsl_matrix_get (a, y, x));
+ for (size_t x = 0; x < b->size2; x++)
+ gsl_matrix_set (c, y, x + a->size2, gsl_matrix_get (b, y, x));
+ }
+ return c;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_paste_vert (const struct matrix_expr *e,
+ gsl_matrix *a, gsl_matrix *b)
+{
+ if (a->size2 != b->size2)
+ {
+ if (!a->size1 || !a->size2)
+ return b;
+ else if (!b->size1 || !b->size2)
+ return a;
+
+ msg_at (SE, matrix_expr_location (e),
+ _("This expression tries to vertically join matrices with "
+ "differing numbers of columns."));
+ note_operand_size (a, e->subs[0]);
+ note_operand_size (b, e->subs[1]);
+ return NULL;
+ }
+
+ gsl_matrix *c = gsl_matrix_alloc (a->size1 + b->size1, a->size2);
+ for (size_t x = 0; x < a->size2; x++)
+ {
+ for (size_t y = 0; y < a->size1; y++)
+ gsl_matrix_set (c, y, x, gsl_matrix_get (a, y, x));
+ for (size_t y = 0; y < b->size1; y++)
+ gsl_matrix_set (c, y + a->size1, x, gsl_matrix_get (b, y, x));
+ }
+ return c;
+}
+
+static gsl_vector *
+matrix_to_vector (gsl_matrix *m)
+{
+ assert (m->owner);
+ gsl_vector v = to_vector (m);
+ assert (v.block == m->block || !v.block);
+ assert (!v.owner);
+ v.owner = 1;
+ m->owner = 0;
+ gsl_matrix_free (m);
+ return xmemdup (&v, sizeof v);
+}
+
+enum index_type {
+ IV_ROW,
+ IV_COLUMN,
+ IV_VECTOR
+};
+
+struct index_vector
+ {
+ size_t *indexes;
+ size_t n;
+ };
+#define INDEX_VECTOR_INIT (struct index_vector) { .n = 0 }
+
+static void
+index_vector_uninit (struct index_vector *iv)
+{
+ if (iv)
+ free (iv->indexes);
+}
+
+static bool
+matrix_normalize_index_vector (const gsl_matrix *m,
+ const struct matrix_expr *me, size_t size,
+ enum index_type index_type, size_t other_size,
+ struct index_vector *iv)
+{
+ if (m)
+ {
+ if (!is_vector (m))
+ {
+ switch (index_type)
+ {
+ case IV_VECTOR:
+ msg_at (SE, matrix_expr_location (me),
+ _("Vector index must be scalar or vector, not a "
+ "%zu×%zu matrix."),
+ m->size1, m->size2);
+ break;
+
+ case IV_ROW:
+ msg_at (SE, matrix_expr_location (me),
+ _("Matrix row index must be scalar or vector, not a "
+ "%zu×%zu matrix."),
+ m->size1, m->size2);
+ break;
+
+ case IV_COLUMN:
+ msg_at (SE, matrix_expr_location (me),
+ _("Matrix column index must be scalar or vector, not a "
+ "%zu×%zu matrix."),
+ m->size1, m->size2);
+ break;
+ }
+ return false;
+ }
+
+ gsl_vector v = to_vector (CONST_CAST (gsl_matrix *, m));
+ *iv = (struct index_vector) {
+ .indexes = xnmalloc (v.size, sizeof *iv->indexes),
+ .n = v.size,
+ };
+ for (size_t i = 0; i < v.size; i++)
+ {
+ double index = gsl_vector_get (&v, i);
+ if (index < 1 || index >= size + 1)
+ {
+ switch (index_type)
+ {
+ case IV_VECTOR:
+ msg_at (SE, matrix_expr_location (me),
+ _("Index %g is out of range for vector "
+ "with %zu elements."), index, size);
+ break;
+
+ case IV_ROW:
+ msg_at (SE, matrix_expr_location (me),
+ _("%g is not a valid row index for "
+ "a %zu×%zu matrix."),
+ index, size, other_size);
+ break;
+
+ case IV_COLUMN:
+ msg_at (SE, matrix_expr_location (me),
+ _("%g is not a valid column index for "
+ "a %zu×%zu matrix."),
+ index, other_size, size);
+ break;
+ }
+
+ index_vector_uninit (iv);
+ return false;
+ }
+ iv->indexes[i] = index - 1;
+ }
+ return true;
+ }
+ else
+ {
+ *iv = (struct index_vector) {
+ .indexes = xnmalloc (size, sizeof *iv->indexes),
+ .n = size,
+ };
+ for (size_t i = 0; i < size; i++)
+ iv->indexes[i] = i;
+ return true;
+ }
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_vec_all (const struct matrix_expr *e,
+ gsl_matrix *sm)
+{
+ if (!is_vector (sm))
+ {
+ msg_at (SE, matrix_expr_location (e->subs[0]),
+ _("Vector index operator may not be applied to "
+ "a %zu×%zu matrix."),
+ sm->size1, sm->size2);
+ return NULL;
+ }
+
+ return sm;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_vec_index (const struct matrix_expr *e,
+ gsl_matrix *sm, gsl_matrix *im)
+{
+ if (!matrix_expr_evaluate_vec_all (e, sm))
+ return NULL;
+
+ gsl_vector sv = to_vector (sm);
+ struct index_vector iv;
+ if (!matrix_normalize_index_vector (im, e->subs[1],
+ sv.size, IV_VECTOR, 0, &iv))
+ return NULL;
+
+ gsl_matrix *dm = gsl_matrix_alloc (sm->size1 == 1 ? 1 : iv.n,
+ sm->size1 == 1 ? iv.n : 1);
+ gsl_vector dv = to_vector (dm);
+ for (size_t dx = 0; dx < iv.n; dx++)
+ {
+ size_t sx = iv.indexes[dx];
+ gsl_vector_set (&dv, dx, gsl_vector_get (&sv, sx));
+ }
+ index_vector_uninit (&iv);
+
+ return dm;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_mat_index (gsl_matrix *sm,
+ gsl_matrix *im0, const struct matrix_expr *eim0,
+ gsl_matrix *im1, const struct matrix_expr *eim1)
+{
+ struct index_vector iv0;
+ if (!matrix_normalize_index_vector (im0, eim0, sm->size1,
+ IV_ROW, sm->size2, &iv0))
+ return NULL;
+
+ struct index_vector iv1;
+ if (!matrix_normalize_index_vector (im1, eim1, sm->size2,
+ IV_COLUMN, sm->size1, &iv1))
+ {
+ index_vector_uninit (&iv0);
+ return NULL;
+ }
+
+ gsl_matrix *dm = gsl_matrix_alloc (iv0.n, iv1.n);
+ for (size_t dy = 0; dy < iv0.n; dy++)
+ {
+ size_t sy = iv0.indexes[dy];
+
+ for (size_t dx = 0; dx < iv1.n; dx++)
+ {
+ size_t sx = iv1.indexes[dx];
+ gsl_matrix_set (dm, dy, dx, gsl_matrix_get (sm, sy, sx));
+ }
+ }
+ index_vector_uninit (&iv0);
+ index_vector_uninit (&iv1);
+ return dm;
+}
+
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
+ static gsl_matrix *matrix_expr_evaluate_##PROTO ( \
+ const struct matrix_function_properties *, gsl_matrix *[], \
+ const struct matrix_expr *, matrix_proto_##PROTO *);
+MATRIX_FUNCTIONS
+#undef F
+
+static bool
+check_scalar_arg (const char *name, gsl_matrix *subs[],
+ const struct matrix_expr *e, size_t index)
+{
+ if (!is_scalar (subs[index]))
+ {
+ msg_at (SE, matrix_expr_location (e->subs[index]),
+ _("Function %s argument %zu must be a scalar, "
+ "not a %zu×%zu matrix."),
+ name, index + 1, subs[index]->size1, subs[index]->size2);
+ return false;
+ }
+ return true;
+}
+
+static bool
+check_vector_arg (const char *name, gsl_matrix *subs[],
+ const struct matrix_expr *e, size_t index)
+{
+ if (!is_vector (subs[index]))
+ {
+ msg_at (SE, matrix_expr_location (e->subs[index]),
+ _("Function %s argument %zu must be a vector, "
+ "not a %zu×%zu matrix."),
+ name, index + 1, subs[index]->size1, subs[index]->size2);
+ return false;
+ }
+ return true;
+}
+
+static bool
+to_scalar_args (const char *name, gsl_matrix *subs[],
+ const struct matrix_expr *e, double d[])
+{
+ for (size_t i = 0; i < e->n_subs; i++)
+ {
+ if (!check_scalar_arg (name, subs, e, i))
+ return false;
+ d[i] = to_scalar (subs[i]);
+ }
+ return true;
+}
+
+static int
+parse_constraint_value (const char **constraintsp)
+{
+ char *tail;
+ long retval = strtol (*constraintsp, &tail, 10);
+ assert (tail > *constraintsp);
+ *constraintsp = tail;
+ return retval;
+}
+
+enum matrix_argument_relop
+ {
+ MRR_GT, /* > */
+ MRR_GE, /* >= */
+ MRR_LT, /* < */
+ MRR_LE, /* <= */
+ MRR_NE, /* <> */
+ };
+
+static void
+argument_inequality_error (
+ const struct matrix_function_properties *props, const struct matrix_expr *e,
+ size_t ai, gsl_matrix *a, size_t y, size_t x,
+ size_t bi, double b,
+ enum matrix_argument_relop relop)
+{
+ const struct msg_location *loc = matrix_expr_location (e);
+ switch (relop)
+ {
+ case MRR_GE:
+ msg_at (ME, loc, _("Argument %zu to matrix function %s must be greater "
+ "than or equal to argument %zu."),
+ ai + 1, props->name, bi + 1);
+ break;
+
+ case MRR_GT:
+ msg_at (ME, loc, _("Argument %zu to matrix function %s must be greater "
+ "than argument %zu."),
+ ai + 1, props->name, bi + 1);
+ break;
+
+ case MRR_LE:
+ msg_at (ME, loc, _("Argument %zu to matrix function %s must be less than "
+ "or equal to argument %zu."),
+ ai + 1, props->name, bi + 1);
+ break;
+
+ case MRR_LT:
+ msg_at (ME, loc, _("Argument %zu to matrix function %s must be less than "
+ "argument %zu."),
+ ai + 1, props->name, bi + 1);
+ break;
+
+ case MRR_NE:
+ msg_at (ME, loc, _("Argument %zu to matrix function %s must not be equal "
+ "to argument %zu."),
+ ai + 1, props->name, bi + 1);
+ break;
+ }
+
+ const struct msg_location *a_loc = matrix_expr_location (e->subs[ai]);
+ if (is_scalar (a))
+ msg_at (SN, a_loc, _("Argument %zu is %g."),
+ ai + 1, gsl_matrix_get (a, y, x));
+ else
+ msg_at (SN, a_loc, _("Row %zu, column %zu of argument %zu is %g."),
+ y + 1, x + 1, ai + 1, gsl_matrix_get (a, y, x));
+
+ msg_at (SN, matrix_expr_location (e->subs[bi]),
+ _("Argument %zu is %g."), bi + 1, b);
+}
+
+static void
+argument_value_error (
+ const struct matrix_function_properties *props, const struct matrix_expr *e,
+ size_t ai, gsl_matrix *a, size_t y, size_t x,
+ double b,
+ enum matrix_argument_relop relop)
+{
+ const struct msg_location *loc = matrix_expr_location (e);
+ switch (relop)
+ {
+ case MRR_GE:
+ msg_at (SE, loc, _("Argument %zu to matrix function %s must be greater "
+ "than or equal to %g."),
+ ai + 1, props->name, b);
+ break;
+
+ case MRR_GT:
+ msg_at (SE, loc, _("Argument %zu to matrix function %s must be greater "
+ "than %g."),
+ ai + 1, props->name, b);
+ break;
+
+ case MRR_LE:
+ msg_at (SE, loc, _("Argument %zu to matrix function %s must be less than "
+ "or equal to %g."),
+ ai + 1, props->name, b);
+ break;
+
+ case MRR_LT:
+ msg_at (SE, loc, _("Argument %zu to matrix function %s must be less than "
+ "%g."),
+ ai + 1, props->name, b);
+ break;
+
+ case MRR_NE:
+ msg_at (SE, loc, _("Argument %zu to matrix function %s must not be equal "
+ "to %g."),
+ ai + 1, props->name, b);
+ break;
+ }
+
+ const struct msg_location *a_loc = matrix_expr_location (e->subs[ai]);
+ if (is_scalar (a))
+ {
+ if (relop != MRR_NE)
+ msg_at (SN, a_loc, _("Argument %zu is %g."),
+ ai + 1, gsl_matrix_get (a, y, x));
+ }
+ else
+ msg_at (SN, a_loc, _("Row %zu, column %zu of argument %zu is %g."),
+ y + 1, x + 1, ai + 1, gsl_matrix_get (a, y, x));
+}
+
+static bool
+matrix_argument_relop_is_satisfied (double a, double b,
+ enum matrix_argument_relop relop)
+{
+ switch (relop)
+ {
+ case MRR_GE: return a >= b;
+ case MRR_GT: return a > b;
+ case MRR_LE: return a <= b;
+ case MRR_LT: return a < b;
+ case MRR_NE: return a != b;
+ }
+
+ NOT_REACHED ();
+}
+
+static enum matrix_argument_relop
+matrix_argument_relop_flip (enum matrix_argument_relop relop)
+{
+ switch (relop)
+ {
+ case MRR_GE: return MRR_LE;
+ case MRR_GT: return MRR_LT;
+ case MRR_LE: return MRR_GE;
+ case MRR_LT: return MRR_GT;
+ case MRR_NE: return MRR_NE;
+ }
+
+ NOT_REACHED ();
+}
+
+static bool
+check_constraints (const struct matrix_function_properties *props,
+ gsl_matrix *args[], const struct matrix_expr *e)
+{
+ size_t n_args = e->n_subs;
+ const char *constraints = props->constraints;
+ if (!constraints)
+ return true;
+
+ size_t arg_index = SIZE_MAX;
+ while (*constraints)
+ {
+ if (*constraints >= 'a' && *constraints <= 'd')
+ {
+ arg_index = *constraints++ - 'a';
+ assert (arg_index < n_args);
+ }
+ else if (*constraints == '[' || *constraints == '(')
+ {
+ assert (arg_index < n_args);
+ bool open_lower = *constraints++ == '(';
+ int minimum = parse_constraint_value (&constraints);
+ assert (*constraints == ',');
+ constraints++;
+ int maximum = parse_constraint_value (&constraints);
+ assert (*constraints == ']' || *constraints == ')');
+ bool open_upper = *constraints++ == ')';
+
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, args[arg_index])
+ if ((open_lower ? *d <= minimum : *d < minimum)
+ || (open_upper ? *d >= maximum : *d > maximum))
+ {
+ if (!is_scalar (args[arg_index]))
+ msg_at (SE, matrix_expr_location (e->subs[arg_index]),
+ _("Row %zu, column %zu of argument %zu to matrix "
+ "function %s is %g, which is outside "
+ "the valid range %c%d,%d%c."),
+ y + 1, x + 1, arg_index + 1, props->name, *d,
+ open_lower ? '(' : '[',
+ minimum, maximum,
+ open_upper ? ')' : ']');
+ else
+ msg_at (SE, matrix_expr_location (e->subs[arg_index]),
+ _("Argument %zu to matrix function %s is %g, "
+ "which is outside the valid range %c%d,%d%c."),
+ arg_index + 1, props->name, *d,
+ open_lower ? '(' : '[',
+ minimum, maximum,
+ open_upper ? ')' : ']');
+ return false;
+ }
+ }
+ else if (*constraints == 'i')
+ {
+ constraints++;
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, args[arg_index])
+ if (*d != floor (*d))
+ {
+ if (!is_scalar (args[arg_index]))
+ msg_at (SE, matrix_expr_location (e->subs[arg_index]),
+ _("Argument %zu to matrix function %s, which must be "
+ "integer, contains non-integer value %g in "
+ "row %zu, column %zu."),
+ arg_index + 1, props->name, *d, y + 1, x + 1);
+ else
+ msg_at (SE, matrix_expr_location (e->subs[arg_index]),
+ _("Argument %zu to matrix function %s, which must be "
+ "integer, has non-integer value %g."),
+ arg_index + 1, props->name, *d);
+ return false;
+ }
+ }
+ else if (*constraints == '>'
+ || *constraints == '<'
+ || *constraints == '!')
+ {
+ enum matrix_argument_relop relop;
+ switch (*constraints++)
+ {
+ case '>':
+ if (*constraints == '=')
+ {
+ constraints++;
+ relop = MRR_GE;
+ }
+ else
+ relop = MRR_GT;
+ break;
+
+ case '<':
+ if (*constraints == '=')
+ {
+ constraints++;
+ relop = MRR_LE;
+ }
+ else
+ relop = MRR_LT;
+ break;
+
+ case '!':
+ assert (*constraints == '=');
+ constraints++;
+ relop = MRR_NE;
+ break;
+
+ default:
+ NOT_REACHED ();
+ }
+
+ if (*constraints >= 'a' && *constraints <= 'd')
+ {
+ size_t a_index = arg_index;
+ size_t b_index = *constraints - 'a';
+ assert (a_index < n_args);
+ assert (b_index < n_args);
+
+ /* We only support one of the two arguments being non-scalar.
+ It's easier to support only the first one being non-scalar, so
+ flip things around if it's the other way. */
+ if (!is_scalar (args[b_index]))
+ {
+ assert (is_scalar (args[a_index]));
+ size_t tmp_index = a_index;
+ a_index = b_index;
+ b_index = tmp_index;
+ relop = matrix_argument_relop_flip (relop);
+ }
+
+ double b = to_scalar (args[b_index]);
+ MATRIX_FOR_ALL_ELEMENTS (a, y, x, args[a_index])
+ if (!matrix_argument_relop_is_satisfied (*a, b, relop))
+ {
+ argument_inequality_error (
+ props, e,
+ a_index, args[a_index], y, x,
+ b_index, b,
+ relop);
+ return false;
+ }
+ }
+ else
+ {
+ int comparand = parse_constraint_value (&constraints);
+
+ MATRIX_FOR_ALL_ELEMENTS (d, y, x, args[arg_index])
+ if (!matrix_argument_relop_is_satisfied (*d, comparand, relop))
+ {
+ argument_value_error (
+ props, e,
+ arg_index, args[arg_index], y, x,
+ comparand,
+ relop);
+ return false;
+ }
+ }
+ }
+ else
+ {
+ assert (*constraints == ' ');
+ constraints++;
+ arg_index = SIZE_MAX;
+ }
+ }
+ return true;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_d_none (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_d_none *f)
+{
+ assert (e->n_subs == 0);
+
+ if (!check_constraints (props, subs, e))
+ return NULL;
+
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, f ());
+ return m;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_d_d (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_d_d *f)
+{
+ assert (e->n_subs == 1);
+
+ double d;
+ if (!to_scalar_args (props->name, subs, e, &d)
+ || !check_constraints (props, subs, e))
+ return NULL;
+
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, f (d));
+ return m;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_d_dd (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_d_dd *f)
+{
+ assert (e->n_subs == 2);
+
+ double d[2];
+ if (!to_scalar_args (props->name, subs, e, d)
+ && !check_constraints (props, subs, e))
+ return NULL;
+
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, f (d[0], d[1]));
+ return m;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_d_ddd (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_d_ddd *f)
+{
+ assert (e->n_subs == 3);
+
+ double d[3];
+ if (!to_scalar_args (props->name, subs, e, d)
+ || !check_constraints (props, subs, e))
+ return NULL;
+
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, f (d[0], d[1], d[2]));
+ return m;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_d (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_d *f)
+{
+ assert (e->n_subs == 1);
+
+ double d;
+ return (to_scalar_args (props->name, subs, e, &d)
+ && check_constraints (props, subs, e)
+ ? f(d)
+ : NULL);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_ddd (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_ddd *f)
+{
+ assert (e->n_subs == 3);
+
+ double d[3];
+ return (to_scalar_args (props->name, subs, e, d)
+ && check_constraints (props, subs, e)
+ ? f(d[0], d[1], d[2])
+ : NULL);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_ddn (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_ddn *f)
+{
+ assert (e->n_subs == 2);
+
+ double d[2];
+ return (to_scalar_args (props->name, subs, e, d)
+ && check_constraints (props, subs, e)
+ ? f(d[0], d[1], e)
+ : NULL);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_m (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_m *f)
+{
+ assert (e->n_subs == 1);
+ return check_constraints (props, subs, e) ? f (subs[0]) : NULL;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_mn (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_mn *f)
+{
+ assert (e->n_subs == 1);
+ return check_constraints (props, subs, e) ? f (subs[0], e) : NULL;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_e (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_e *f)
+{
+ assert (e->n_subs == 1);
+
+ if (!check_constraints (props, subs, e))
+ return NULL;
+
+ MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
+ *a = f (*a);
+ return subs[0];
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_md (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_md *f)
+{
+ assert (e->n_subs == 2);
+ return (check_scalar_arg (props->name, subs, e, 1)
+ && check_constraints (props, subs, e)
+ ? f (subs[0], to_scalar (subs[1]))
+ : NULL);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_mdn (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_mdn *f)
+{
+ assert (e->n_subs == 2);
+ return (check_scalar_arg (props->name, subs, e, 1)
+ && check_constraints (props, subs, e)
+ ? f (subs[0], to_scalar (subs[1]), e)
+ : NULL);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_ed (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_ed *f)
+{
+ assert (e->n_subs == 2);
+ if (!check_scalar_arg (props->name, subs, e, 1)
+ || !check_constraints (props, subs, e))
+ return NULL;
+
+ double b = to_scalar (subs[1]);
+ MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
+ *a = f (*a, b);
+ return subs[0];
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_mddn (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_mddn *f)
+{
+ assert (e->n_subs == 3);
+ if (!check_scalar_arg (props->name, subs, e, 1)
+ || !check_scalar_arg (props->name, subs, e, 2)
+ || !check_constraints (props, subs, e))
+ return NULL;
+ return f (subs[0], to_scalar (subs[1]), to_scalar (subs[2]), e);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_edd (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_edd *f)
+{
+ assert (e->n_subs == 3);
+ if (!check_scalar_arg (props->name, subs, e, 1)
+ || !check_scalar_arg (props->name, subs, e, 2)
+ || !check_constraints (props, subs, e))
+ return NULL;
+
+ double b = to_scalar (subs[1]);
+ double c = to_scalar (subs[2]);
+ MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
+ *a = f (*a, b, c);
+ return subs[0];
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_eddd (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_eddd *f)
+{
+ assert (e->n_subs == 4);
+ for (size_t i = 1; i < 4; i++)
+ if (!check_scalar_arg (props->name, subs, e, i))
+ return NULL;
+
+ if (!check_constraints (props, subs, e))
+ return NULL;
+
+ double b = to_scalar (subs[1]);
+ double c = to_scalar (subs[2]);
+ double d = to_scalar (subs[3]);
+ MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
+ *a = f (*a, b, c, d);
+ return subs[0];
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_eed (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_eed *f)
+{
+ assert (e->n_subs == 3);
+ if (!check_scalar_arg (props->name, subs, e, 2))
+ return NULL;
+
+ if (!is_scalar (subs[0]) && !is_scalar (subs[1])
+ && (subs[0]->size1 != subs[1]->size1 || subs[0]->size2 != subs[1]->size2))
+ {
+ struct msg_location *loc = msg_location_dup (e->subs[0]->location);
+ loc->end = e->subs[1]->location->end;
+
+ msg_at (ME, loc,
+ _("Arguments 1 and 2 to %s have dimensions %zu×%zu and "
+ "%zu×%zu, but %s requires these arguments either to have "
+ "the same dimensions or for one of them to be a scalar."),
+ props->name,
+ subs[0]->size1, subs[0]->size2,
+ subs[1]->size1, subs[1]->size2,
+ props->name);
+
+ msg_location_destroy (loc);
+ return NULL;
+ }
+
+ if (!check_constraints (props, subs, e))
+ return NULL;
+
+ double c = to_scalar (subs[2]);
+
+ if (is_scalar (subs[0]))
+ {
+ double a = to_scalar (subs[0]);
+ MATRIX_FOR_ALL_ELEMENTS (b, y, x, subs[1])
+ *b = f (a, *b, c);
+ return subs[1];
+ }
+ else
+ {
+ double b = to_scalar (subs[1]);
+ MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
+ *a = f (*a, b, c);
+ return subs[0];
+ }
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_mm (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_mm *f)
+{
+ assert (e->n_subs == 2);
+ return check_constraints (props, subs, e) ? f (subs[0], subs[1]) : NULL;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_mmn (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_mmn *f)
+{
+ assert (e->n_subs == 2);
+ return check_constraints (props, subs, e) ? f (subs[0], subs[1], e) : NULL;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_v (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_v *f)
+{
+ assert (e->n_subs == 1);
+ if (!check_vector_arg (props->name, subs, e, 0)
+ || !check_constraints (props, subs, e))
+ return NULL;
+ gsl_vector v = to_vector (subs[0]);
+ return f (&v);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_d_m (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_d_m *f)
+{
+ assert (e->n_subs == 1);
+
+ if (!check_constraints (props, subs, e))
+ return NULL;
+
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, f (subs[0]));
+ return m;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_m_any (const struct matrix_function_properties *props,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_m_any *f)
+{
+ return check_constraints (props, subs, e) ? f (subs, e->n_subs) : NULL;
+}
+
+static gsl_matrix *
+matrix_expr_evaluate_IDENT (const struct matrix_function_properties *props_ UNUSED,
+ gsl_matrix *subs[], const struct matrix_expr *e,
+ matrix_proto_IDENT *f)
+{
+ static const struct matrix_function_properties p1 = {
+ .name = "IDENT",
+ .constraints = "ai>=0"
+ };
+ static const struct matrix_function_properties p2 = {
+ .name = "IDENT",
+ .constraints = "ai>=0 bi>=0"
+ };
+ const struct matrix_function_properties *props = e->n_subs == 1 ? &p1 : &p2;
+
+ assert (e->n_subs <= 2);
+
+ double d[2];
+ return (to_scalar_args (props->name, subs, e, d)
+ && check_constraints (props, subs, e)
+ ? f (d[0], d[e->n_subs - 1])
+ : NULL);
+}
+
+static gsl_matrix *
+matrix_expr_evaluate (const struct matrix_expr *e)
+{
+ if (e->op == MOP_NUMBER)
+ {
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, e->number);
+ return m;
+ }
+ else if (e->op == MOP_VARIABLE)
+ {
+ const gsl_matrix *src = e->variable->value;
+ if (!src)
+ {
+ msg_at (SE, e->location,
+ _("Uninitialized variable %s used in expression."),
+ e->variable->name);
+ return NULL;
+ }
+
+ gsl_matrix *dst = gsl_matrix_alloc (src->size1, src->size2);
+ gsl_matrix_memcpy (dst, src);
+ return dst;
+ }
+ else if (e->op == MOP_EOF)
+ {
+ struct dfm_reader *reader = read_file_open (e->eof);
+ gsl_matrix *m = gsl_matrix_alloc (1, 1);
+ gsl_matrix_set (m, 0, 0, !reader || dfm_eof (reader));
+ return m;
+ }
+
+ enum { N_LOCAL = 3 };
+ gsl_matrix *local_subs[N_LOCAL];
+ gsl_matrix **subs = (e->n_subs < N_LOCAL
+ ? local_subs
+ : xmalloc (e->n_subs * sizeof *subs));
+
+ for (size_t i = 0; i < e->n_subs; i++)
+ {
+ subs[i] = matrix_expr_evaluate (e->subs[i]);
+ if (!subs[i])
+ {
+ for (size_t j = 0; j < i; j++)
+ gsl_matrix_free (subs[j]);
+ if (subs != local_subs)
+ free (subs);
+ return NULL;
+ }
+ }
+
+ gsl_matrix *result = NULL;
+ switch (e->op)
+ {
+#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
+ case MOP_F_##ENUM: \
+ { \
+ static const struct matrix_function_properties props = { \
+ .name = STRING, \
+ .constraints = CONSTRAINTS, \
+ }; \
+ result = matrix_expr_evaluate_##PROTO (&props, subs, e, \
+ matrix_eval_##ENUM); \
+ } \
+ break;
+ MATRIX_FUNCTIONS
+#undef F
+
+ case MOP_NEGATE:
+ gsl_matrix_scale (subs[0], -1.0);
+ result = subs[0];
+ break;
+
+ case MOP_ADD_ELEMS:
+ case MOP_SUB_ELEMS:
+ case MOP_MUL_ELEMS:
+ case MOP_DIV_ELEMS:
+ case MOP_EXP_ELEMS:
+ case MOP_GT:
+ case MOP_GE:
+ case MOP_LT:
+ case MOP_LE:
+ case MOP_EQ:
+ case MOP_NE:
+ case MOP_AND:
+ case MOP_OR:
+ case MOP_XOR:
+ result = matrix_expr_evaluate_elementwise (e, e->op, subs[0], subs[1]);
+ break;
+
+ case MOP_NOT:
+ result = matrix_expr_evaluate_not (subs[0]);
+ break;
+
+ case MOP_SEQ:
+ result = matrix_expr_evaluate_seq (e, subs[0], subs[1], NULL);
+ break;
+
+ case MOP_SEQ_BY:
+ result = matrix_expr_evaluate_seq (e, subs[0], subs[1], subs[2]);
+ break;
+
+ case MOP_MUL_MAT:
+ result = matrix_expr_evaluate_mul_mat (e, subs[0], subs[1]);
+ break;
+
+ case MOP_EXP_MAT:
+ result = matrix_expr_evaluate_exp_mat (e, subs[0], subs[1]);
+ break;
+
+ case MOP_PASTE_HORZ:
+ result = matrix_expr_evaluate_paste_horz (e, subs[0], subs[1]);
+ break;
+
+ case MOP_PASTE_VERT:
+ result = matrix_expr_evaluate_paste_vert (e, subs[0], subs[1]);
+ break;
+
+ case MOP_EMPTY:
+ result = gsl_matrix_alloc (0, 0);
+ break;
+
+ case MOP_VEC_INDEX:
+ result = matrix_expr_evaluate_vec_index (e, subs[0], subs[1]);
+ break;
+
+ case MOP_VEC_ALL:
+ result = matrix_expr_evaluate_vec_all (e, subs[0]);
+ break;
+
+ case MOP_MAT_INDEX:
+ result = matrix_expr_evaluate_mat_index (subs[0],
+ subs[1], e->subs[1],
+ subs[2], e->subs[2]);
+ break;
+
+ case MOP_ROW_INDEX:
+ result = matrix_expr_evaluate_mat_index (subs[0],
+ subs[1], e->subs[1],
+ NULL, NULL);
+ break;
+
+ case MOP_COL_INDEX:
+ result = matrix_expr_evaluate_mat_index (subs[0],
+ NULL, NULL,
+ subs[1], e->subs[1]);
+ break;
+
+ case MOP_NUMBER:
+ case MOP_VARIABLE:
+ case MOP_EOF:
+ NOT_REACHED ();
+ }
+
+ for (size_t i = 0; i < e->n_subs; i++)
+ if (subs[i] != result)
+ gsl_matrix_free (subs[i]);
+ if (subs != local_subs)
+ free (subs);
+ return result;
+}
+
+static bool
+matrix_expr_evaluate_scalar (const struct matrix_expr *e, const char *context,
+ double *d)
+{
+ gsl_matrix *m = matrix_expr_evaluate (e);
+ if (!m)
+ return false;
+
+ if (!is_scalar (m))
+ {
+ msg_at (SE, matrix_expr_location (e),
+ _("Expression for %s must evaluate to scalar, "
+ "not a %zu×%zu matrix."),
+ context, m->size1, m->size2);
+ gsl_matrix_free (m);
+ return false;
+ }
+
+ *d = to_scalar (m);
+ gsl_matrix_free (m);
+ return true;
+}
+
+static bool
+matrix_expr_evaluate_integer (const struct matrix_expr *e, const char *context,
+ long int *integer)
+{
+ double d;
+ if (!matrix_expr_evaluate_scalar (e, context, &d))
+ return false;
+
+ d = trunc (d);
+ if (d < LONG_MIN || d > LONG_MAX)
+ {
+ msg_at (SE, matrix_expr_location (e),
+ _("Expression for %s is outside the integer range."), context);
+ return false;
+ }
+ *integer = d;
+ return true;
+}
+\f
+/* Matrix lvalues.
+
+ An lvalue is an expression that can appear on the left side of a COMPUTE
+ command and in other contexts that assign values.
+
+ An lvalue is parsed once, with matrix_lvalue_parse(). It can then be
+ evaluated (with matrix_lvalue_evaluate()) and assigned (with
+ matrix_lvalue_assign()).
+
+ There are three kinds of lvalues:
+
+ - A variable name. A variable used as an lvalue need not be initialized,
+ since the assignment will initialize.
+
+ - A subvector, e.g. "var(index0)". The variable must be initialized and
+ must have the form of a vector (it must have 1 column or 1 row).
+
+ - A submatrix, e.g. "var(index0, index1)". The variable must be
+ initialized. */
+struct matrix_lvalue
+ {
+ struct matrix_var *var; /* Destination variable. */
+ struct matrix_expr *indexes[2]; /* Index expressions, if any. */
+ size_t n_indexes; /* Number of indexes. */
+
+ struct msg_location *var_location; /* Variable name. */
+ struct msg_location *full_location; /* Variable name plus indexing. */
+ struct msg_location *index_locations[2]; /* Index expressions. */
+ };
+
+/* Frees LVALUE. */
+static void
+matrix_lvalue_destroy (struct matrix_lvalue *lvalue)
+{
+ if (lvalue)
+ {
+ msg_location_destroy (lvalue->var_location);
+ msg_location_destroy (lvalue->full_location);
+ for (size_t i = 0; i < lvalue->n_indexes; i++)
+ {
+ matrix_expr_destroy (lvalue->indexes[i]);
+ msg_location_destroy (lvalue->index_locations[i]);
+ }
+ free (lvalue);
+ }
+}
+
+/* Parses and returns an lvalue at the current position in S's lexer. Returns
+ null on parse failure. On success, the caller must eventually free the
+ lvalue. */
+static struct matrix_lvalue *
+matrix_lvalue_parse (struct matrix_state *s)
+{
+ if (!lex_force_id (s->lexer))
+ return NULL;
+
+ struct matrix_lvalue *lvalue = xzalloc (sizeof *lvalue);
+ int start_ofs = lex_ofs (s->lexer);
+ lvalue->var_location = lex_get_location (s->lexer, 0, 0);
+ lvalue->var = matrix_var_lookup (s, lex_tokss (s->lexer));
+ if (lex_next_token (s->lexer, 1) == T_LPAREN)
+ {
+ if (!lvalue->var)
+ {
+ lex_error (s->lexer, _("Undefined variable %s."),
+ lex_tokcstr (s->lexer));
+ goto error;
+ }
+
+ lex_get_n (s->lexer, 2);
+
+ if (!matrix_parse_index_expr (s, &lvalue->indexes[0],
+ &lvalue->index_locations[0]))
+ goto error;
+ lvalue->n_indexes++;
+
+ if (lex_match (s->lexer, T_COMMA))
+ {
+ if (!matrix_parse_index_expr (s, &lvalue->indexes[1],
+ &lvalue->index_locations[1]))
+ goto error;
+ lvalue->n_indexes++;
+ }
+ if (!lex_force_match (s->lexer, T_RPAREN))
+ goto error;
+
+ lvalue->full_location = lex_ofs_location (s->lexer, start_ofs,
+ lex_ofs (s->lexer) - 1);
+ }
+ else
+ {
+ if (!lvalue->var)
+ lvalue->var = matrix_var_create (s, lex_tokss (s->lexer));
+ lex_get (s->lexer);
+ }
+ return lvalue;
+
+error:
+ matrix_lvalue_destroy (lvalue);
+ return NULL;
+}
+
+static bool
+matrix_lvalue_evaluate_vector (struct matrix_expr *e, size_t size,
+ enum index_type index_type, size_t other_size,
+ struct index_vector *iv)
+{
+ gsl_matrix *m;
+ if (e)
+ {
+ m = matrix_expr_evaluate (e);
+ if (!m)
+ return false;
+ }
+ else
+ m = NULL;
+
+ bool ok = matrix_normalize_index_vector (m, e, size, index_type,
+ other_size, iv);
+ gsl_matrix_free (m);
+ return ok;
+}
+
+/* Evaluates the indexes in LVALUE into IV0 and IV1, owned by the caller.
+ Returns true if successful, false if evaluating the expressions failed or if
+ LVALUE otherwise can't be used for an assignment.
+
+ On success, the caller retains ownership of the index vectors, which are
+ suitable for passing to matrix_lvalue_assign(). If not used for that
+ purpose then they need to eventually be freed (with
+ index_vector_uninit()). */
+static bool
+matrix_lvalue_evaluate (struct matrix_lvalue *lvalue,
+ struct index_vector *iv0,
+ struct index_vector *iv1)
+{
+ *iv0 = INDEX_VECTOR_INIT;
+ *iv1 = INDEX_VECTOR_INIT;
+ if (!lvalue->n_indexes)
+ return true;
+
+ /* Validate destination matrix exists and has the right shape. */
+ gsl_matrix *dm = lvalue->var->value;
+ if (!dm)
+ {
+ msg_at (SE, lvalue->var_location,
+ _("Undefined variable %s."), lvalue->var->name);
+ return false;
+ }
+ else if (dm->size1 == 0 || dm->size2 == 0)
+ {
+ msg_at (SE, lvalue->full_location, _("Cannot index %zu×%zu matrix %s."),
+ dm->size1, dm->size2, lvalue->var->name);
+ return false;
+ }
+ else if (lvalue->n_indexes == 1)
+ {
+ if (!is_vector (dm))
+ {
+ msg_at (SE, lvalue->full_location,
+ _("Can't use vector indexing on %zu×%zu matrix %s."),
+ dm->size1, dm->size2, lvalue->var->name);
+ return false;
+ }
+ return matrix_lvalue_evaluate_vector (lvalue->indexes[0],
+ MAX (dm->size1, dm->size2),
+ IV_VECTOR, 0, iv0);
+ }
+ else
+ {
+ assert (lvalue->n_indexes == 2);
+ if (!matrix_lvalue_evaluate_vector (lvalue->indexes[0], dm->size1,
+ IV_ROW, dm->size2, iv0))
+ return false;
+
+ if (!matrix_lvalue_evaluate_vector (lvalue->indexes[1], dm->size2,
+ IV_COLUMN, dm->size1, iv1))
+ {
+ index_vector_uninit (iv0);
+ return false;
+ }
+ return true;
+ }
+}
+
+static bool
+matrix_lvalue_assign_vector (struct matrix_lvalue *lvalue,
+ struct index_vector *iv,
+ gsl_matrix *sm, const struct msg_location *lsm)
+{
+ /* Convert source matrix 'sm' to source vector 'sv'. */
+ if (!is_vector (sm))
+ {
+ msg_at (SE, lvalue->full_location,
+ _("Only an %zu-element vector may be assigned to this "
+ "%zu-element subvector of %s."),
+ iv->n, iv->n, lvalue->var->name);
+ msg_at (SE, lsm,
+ _("The source is an %zu×%zu matrix."),
+ sm->size1, sm->size2);
+ return false;
+ }
+ gsl_vector sv = to_vector (sm);
+ if (iv->n != sv.size)
+ {
+ msg_at (SE, lvalue->full_location,
+ _("Only an %zu-element vector may be assigned to this "
+ "%zu-element subvector of %s."),
+ iv->n, iv->n, lvalue->var->name);
+ msg_at (SE, lsm, ngettext ("The source vector has %zu element.",
+ "The source vector has %zu elements.",
+ sv.size),
+ sv.size);
+ return false;
+ }
+
+ /* Assign elements. */
+ gsl_vector dv = to_vector (lvalue->var->value);
+ for (size_t x = 0; x < iv->n; x++)
+ gsl_vector_set (&dv, iv->indexes[x], gsl_vector_get (&sv, x));
+ return true;
+}
+
+static bool
+matrix_lvalue_assign_matrix (struct matrix_lvalue *lvalue,
+ struct index_vector *iv0,
+ struct index_vector *iv1,
+ gsl_matrix *sm, const struct msg_location *lsm)
+{
+ gsl_matrix *dm = lvalue->var->value;
+
+ /* Convert source matrix 'sm' to source vector 'sv'. */
+ bool wrong_rows = iv0->n != sm->size1;
+ bool wrong_columns = iv1->n != sm->size2;
+ if (wrong_rows || wrong_columns)
+ {
+ if (wrong_rows && wrong_columns)
+ msg_at (SE, lvalue->full_location,
+ _("Numbers of indexes for assigning to %s differ from the "
+ "size of the source matrix."),
+ lvalue->var->name);
+ else if (wrong_rows)
+ msg_at (SE, lvalue->full_location,
+ _("Number of row indexes for assigning to %s differs from "
+ "number of rows in source matrix."),
+ lvalue->var->name);
+ else
+ msg_at (SE, lvalue->full_location,
+ _("Number of column indexes for assigning to %s differs from "
+ "number of columns in source matrix."),
+ lvalue->var->name);
+
+ if (wrong_rows)
+ {
+ if (lvalue->indexes[0])
+ msg_at (SN, lvalue->index_locations[0],
+ ngettext ("There is %zu row index.",
+ "There are %zu row indexes.",
+ iv0->n),
+ iv0->n);
+ else
+ msg_at (SN, lvalue->index_locations[0],
+ ngettext ("Destination matrix %s has %zu row.",
+ "Destination matrix %s has %zu rows.",
+ iv0->n),
+ lvalue->var->name, iv0->n);
+ }
+
+ if (wrong_columns)
+ {
+ if (lvalue->indexes[1])
+ msg_at (SN, lvalue->index_locations[1],
+ ngettext ("There is %zu column index.",
+ "There are %zu column indexes.",
+ iv1->n),
+ iv1->n);
+ else
+ msg_at (SN, lvalue->index_locations[1],
+ ngettext ("Destination matrix %s has %zu column.",
+ "Destination matrix %s has %zu columns.",
+ iv1->n),
+ lvalue->var->name, iv1->n);
+ }
+
+ msg_at (SN, lsm, _("The source matrix is %zu×%zu."),
+ sm->size1, sm->size2);
+ return false;
+ }
+
+ /* Assign elements. */
+ for (size_t y = 0; y < iv0->n; y++)
+ {
+ size_t f0 = iv0->indexes[y];
+ for (size_t x = 0; x < iv1->n; x++)
+ {
+ size_t f1 = iv1->indexes[x];
+ gsl_matrix_set (dm, f0, f1, gsl_matrix_get (sm, y, x));
+ }
+ }
+ return true;
+}
+
+/* Assigns rvalue SM to LVALUE, whose index vectors IV0 and IV1 were previously
+ obtained with matrix_lvalue_evaluate(). Returns true if successful, false
+ on error. Always takes ownership of M. LSM should be the source location
+ to use for errors related to SM. */
+static bool
+matrix_lvalue_assign (struct matrix_lvalue *lvalue,
+ struct index_vector *iv0, struct index_vector *iv1,
+ gsl_matrix *sm, const struct msg_location *lsm)
+{
+ if (!lvalue->n_indexes)
+ {
+ gsl_matrix_free (lvalue->var->value);
+ lvalue->var->value = sm;
+ return true;
+ }
+ else
+ {
+ bool ok = (lvalue->n_indexes == 1
+ ? matrix_lvalue_assign_vector (lvalue, iv0, sm, lsm)
+ : matrix_lvalue_assign_matrix (lvalue, iv0, iv1, sm, lsm));
+ index_vector_uninit (iv0);
+ index_vector_uninit (iv1);
+ gsl_matrix_free (sm);
+ return ok;
+ }
+}
+
+/* Evaluates and then assigns SM to LVALUE. Always takes ownership of M. LSM
+ should be the source location to use for errors related to SM.*/
+static bool
+matrix_lvalue_evaluate_and_assign (struct matrix_lvalue *lvalue,
+ gsl_matrix *sm,
+ const struct msg_location *lsm)
+{
+ struct index_vector iv0, iv1;
+ if (!matrix_lvalue_evaluate (lvalue, &iv0, &iv1))
+ {
+ gsl_matrix_free (sm);
+ return false;
+ }
+
+ return matrix_lvalue_assign (lvalue, &iv0, &iv1, sm, lsm);
+}
+\f
+/* Matrix command data structure. */
+
+/* An array of matrix commands. */
+struct matrix_commands
+ {
+ struct matrix_command **commands;
+ size_t n;
+ };
+
+static bool matrix_commands_parse (struct matrix_state *,
+ struct matrix_commands *,
+ const char *command_name,
+ const char *stop1, const char *stop2);
+static void matrix_commands_uninit (struct matrix_commands *);
+
+/* A single matrix command. */
+struct matrix_command
+ {
+ /* The type of command. */
+ enum matrix_command_type
+ {
+ MCMD_COMPUTE,
+ MCMD_PRINT,
+ MCMD_DO_IF,
+ MCMD_LOOP,
+ MCMD_BREAK,
+ MCMD_DISPLAY,
+ MCMD_RELEASE,
+ MCMD_SAVE,
+ MCMD_READ,
+ MCMD_WRITE,
+ MCMD_GET,
+ MCMD_MSAVE,
+ MCMD_MGET,
+ MCMD_EIGEN,
+ MCMD_SETDIAG,
+ MCMD_SVD,
+ }
+ type;
+
+ /* Source lines for this command. */
+ struct msg_location *location;
+
+ union
+ {
+ struct matrix_compute
+ {
+ struct matrix_lvalue *lvalue;
+ struct matrix_expr *rvalue;
+ }
+ compute;
+
+ struct matrix_print
+ {
+ struct matrix_expr *expression;
+ bool use_default_format;
+ struct fmt_spec format;
+ char *title;
+ int space; /* -1 means NEWPAGE. */
+
+ struct print_labels
+ {
+ struct string_array literals; /* CLABELS/RLABELS. */
+ struct matrix_expr *expr; /* CNAMES/RNAMES. */
+ }
+ *rlabels, *clabels;
+ }
+ print;
+
+ struct matrix_do_if
+ {
+ struct do_if_clause
+ {
+ struct matrix_expr *condition;
+ struct matrix_commands commands;
+ }
+ *clauses;
+ size_t n_clauses;
+ }
+ do_if;
+
+ struct matrix_loop
+ {
+ /* Index. */
+ struct matrix_var *var;
+ struct matrix_expr *start;
+ struct matrix_expr *end;
+ struct matrix_expr *increment;
+
+ /* Loop conditions. */
+ struct matrix_expr *top_condition;
+ struct matrix_expr *bottom_condition;
+
+ /* Commands. */
+ struct matrix_commands commands;
+ }
+ loop;
+
+ struct matrix_display
+ {
+ struct matrix_state *state;
+ }
+ display;
+
+ struct matrix_release
+ {
+ struct matrix_var **vars;
+ size_t n_vars;
+ }
+ release;
+
+ struct matrix_save
+ {
+ struct matrix_expr *expression;
+ struct save_file *sf;
+ }
+ save;
+
+ struct matrix_read
+ {
+ struct read_file *rf;
+ struct matrix_lvalue *dst;
+ struct matrix_expr *size;
+ int c1, c2;
+ enum fmt_type format;
+ int w;
+ bool symmetric;
+ bool reread;
+ }
+ read;
+
+ struct matrix_write
+ {
+ struct write_file *wf;
+ struct matrix_expr *expression;
+ int c1, c2;
+
+ /* If this is nonnull, WRITE uses this format.
+
+ If this is NULL, WRITE uses free-field format with as many
+ digits of precision as needed. */
+ struct fmt_spec *format;
+
+ bool triangular;
+ bool hold;
+ }
+ write;
+
+ struct matrix_get
+ {
+ struct lexer *lexer;
+ struct matrix_lvalue *dst;
+ struct dataset *dataset;
+ struct file_handle *file;
+ char *encoding;
+ struct var_syntax *vars;
+ size_t n_vars;
+ struct matrix_var *names;
+
+ /* Treatment of missing values. */
+ struct
+ {
+ enum
+ {
+ MGET_ERROR, /* Flag error on command. */
+ MGET_ACCEPT, /* Accept without change, user-missing only. */
+ MGET_OMIT, /* Drop this case. */
+ MGET_RECODE /* Recode to 'substitute'. */
+ }
+ treatment;
+ double substitute; /* MGET_RECODE only. */
+ }
+ user, system;
+ }
+ get;
+
+ struct matrix_msave
+ {
+ struct msave_common *common;
+ struct matrix_expr *expr;
+ const char *rowtype;
+ const struct matrix_expr *factors;
+ const struct matrix_expr *splits;
+ }
+ msave;
+
+ struct matrix_mget
+ {
+ struct matrix_state *state;
+ struct file_handle *file;
+ char *encoding;
+ struct stringi_set rowtypes;
+ }
+ mget;
+
+ struct matrix_eigen
+ {
+ struct matrix_expr *expr;
+ struct matrix_var *evec;
+ struct matrix_var *eval;
+ }
+ eigen;
+
+ struct matrix_setdiag
+ {
+ struct matrix_var *dst;
+ struct matrix_expr *expr;
+ }
+ setdiag;
+
+ struct matrix_svd
+ {
+ struct matrix_expr *expr;
+ struct matrix_var *u;
+ struct matrix_var *s;
+ struct matrix_var *v;
+ }
+ svd;
+ };
+ };
+
+static struct matrix_command *matrix_command_parse (struct matrix_state *);
+static bool matrix_command_execute (struct matrix_command *);
+static void matrix_command_destroy (struct matrix_command *);
+\f
+/* COMPUTE. */
+
+static struct matrix_command *
+matrix_compute_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_COMPUTE,
+ .compute = { .lvalue = NULL },
+ };
+
+ struct matrix_compute *compute = &cmd->compute;
+ compute->lvalue = matrix_lvalue_parse (s);
+ if (!compute->lvalue)
+ goto error;
+
+ if (!lex_force_match (s->lexer, T_EQUALS))
+ goto error;
+
+ compute->rvalue = matrix_expr_parse (s);
+ if (!compute->rvalue)
+ goto error;
+
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_compute_execute (struct matrix_command *cmd)
+{
+ struct matrix_compute *compute = &cmd->compute;
+ gsl_matrix *value = matrix_expr_evaluate (compute->rvalue);
+ if (!value)
+ return;
+
+ matrix_lvalue_evaluate_and_assign (compute->lvalue, value,
+ matrix_expr_location (compute->rvalue));
+}
+\f
+/* PRINT. */
+
+static void
+print_labels_destroy (struct print_labels *labels)
+{
+ if (labels)
+ {
+ string_array_destroy (&labels->literals);
+ matrix_expr_destroy (labels->expr);
+ free (labels);
+ }
+}
+
+static void
+parse_literal_print_labels (struct matrix_state *s,
+ struct print_labels **labelsp)
+{
+ lex_match (s->lexer, T_EQUALS);
+ print_labels_destroy (*labelsp);
+ *labelsp = xzalloc (sizeof **labelsp);
+ while (lex_token (s->lexer) != T_SLASH
+ && lex_token (s->lexer) != T_ENDCMD
+ && lex_token (s->lexer) != T_STOP)
+ {
+ struct string label = DS_EMPTY_INITIALIZER;
+ while (lex_token (s->lexer) != T_COMMA
+ && lex_token (s->lexer) != T_SLASH
+ && lex_token (s->lexer) != T_ENDCMD
+ && lex_token (s->lexer) != T_STOP)
+ {
+ if (!ds_is_empty (&label))
+ ds_put_byte (&label, ' ');
+
+ if (lex_token (s->lexer) == T_STRING)
+ ds_put_cstr (&label, lex_tokcstr (s->lexer));
+ else
+ {
+ char *rep = lex_next_representation (s->lexer, 0, 0);
+ ds_put_cstr (&label, rep);
+ free (rep);
+ }
+ lex_get (s->lexer);
+ }
+ string_array_append_nocopy (&(*labelsp)->literals,
+ ds_steal_cstr (&label));
+
+ if (!lex_match (s->lexer, T_COMMA))
+ break;
+ }
+}
+
+static bool
+parse_expr_print_labels (struct matrix_state *s, struct print_labels **labelsp)
+{
+ lex_match (s->lexer, T_EQUALS);
+ struct matrix_expr *e = matrix_parse_exp (s);
+ if (!e)
+ return false;
+
+ print_labels_destroy (*labelsp);
+ *labelsp = xzalloc (sizeof **labelsp);
+ (*labelsp)->expr = e;
+ return true;
+}
+
+static struct matrix_command *
+matrix_print_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_PRINT,
+ .print = {
+ .use_default_format = true,
+ }
+ };
+
+ if (lex_token (s->lexer) != T_SLASH && lex_token (s->lexer) != T_ENDCMD)
+ {
+ int start_ofs = lex_ofs (s->lexer);
+ cmd->print.expression = matrix_parse_exp (s);
+ if (!cmd->print.expression)
+ goto error;
+ cmd->print.title = lex_ofs_representation (s->lexer, start_ofs,
+ lex_ofs (s->lexer) - 1);
+ }
+
+ while (lex_match (s->lexer, T_SLASH))
+ {
+ if (lex_match_id (s->lexer, "FORMAT"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!parse_format_specifier (s->lexer, &cmd->print.format))
+ goto error;
+ cmd->print.use_default_format = false;
+ }
+ else if (lex_match_id (s->lexer, "TITLE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!lex_force_string (s->lexer))
+ goto error;
+ free (cmd->print.title);
+ cmd->print.title = ss_xstrdup (lex_tokss (s->lexer));
+ lex_get (s->lexer);
+ }
+ else if (lex_match_id (s->lexer, "SPACE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (lex_match_id (s->lexer, "NEWPAGE"))
+ cmd->print.space = -1;
+ else if (lex_force_int_range (s->lexer, "SPACE", 1, INT_MAX))
+ {
+ cmd->print.space = lex_integer (s->lexer);
+ lex_get (s->lexer);
+ }
+ else
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "RLABELS"))
+ parse_literal_print_labels (s, &cmd->print.rlabels);
+ else if (lex_match_id (s->lexer, "CLABELS"))
+ parse_literal_print_labels (s, &cmd->print.clabels);
+ else if (lex_match_id (s->lexer, "RNAMES"))
+ {
+ if (!parse_expr_print_labels (s, &cmd->print.rlabels))
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "CNAMES"))
+ {
+ if (!parse_expr_print_labels (s, &cmd->print.clabels))
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "FORMAT", "TITLE", "SPACE",
+ "RLABELS", "CLABELS", "RNAMES", "CNAMES");
+ goto error;
+ }
+
+ }
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static bool
+matrix_is_integer (const gsl_matrix *m)
+{
+ for (size_t y = 0; y < m->size1; y++)
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double d = gsl_matrix_get (m, y, x);
+ if (d != floor (d))
+ return false;
+ }
+ return true;
+}
+
+static double
+matrix_max_magnitude (const gsl_matrix *m)
+{
+ double max = 0.0;
+ for (size_t y = 0; y < m->size1; y++)
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double d = fabs (gsl_matrix_get (m, y, x));
+ if (d > max)
+ max = d;
+ }
+ return max;
+}
+
+static bool
+format_fits (struct fmt_spec format, double x)
+{
+ char *s = data_out (&(union value) { .f = x }, NULL,
+ &format, settings_get_fmt_settings ());
+ bool fits = *s != '*' && !strchr (s, 'E');
+ free (s);
+ return fits;
+}
+
+static struct fmt_spec
+default_format (const gsl_matrix *m, int *log_scale)
+{
+ *log_scale = 0;
+
+ double max = matrix_max_magnitude (m);
+
+ if (matrix_is_integer (m))
+ for (int w = 1; w <= 12; w++)
+ {
+ struct fmt_spec format = { .type = FMT_F, .w = w };
+ if (format_fits (format, -max))
+ return format;
+ }
+
+ if (max >= 1e9 || max <= 1e-4)
+ {
+ char s[64];
+ snprintf (s, sizeof s, "%.1e", max);
+
+ const char *e = strchr (s, 'e');
+ if (e)
+ *log_scale = atoi (e + 1);
+ }
+
+ return (struct fmt_spec) { .type = FMT_F, .w = 13, .d = 10 };
+}
+
+static char *
+trimmed_string (double d)
+{
+ struct substring s = ss_buffer ((char *) &d, sizeof d);
+ ss_rtrim (&s, ss_cstr (" "));
+ return ss_xstrdup (s);
+}
+
+static struct string_array *
+print_labels_get (const struct print_labels *labels, size_t n,
+ const char *prefix, bool truncate)
+{
+ if (!labels)
+ return NULL;
+
+ struct string_array *out = xzalloc (sizeof *out);
+ if (labels->literals.n)
+ string_array_clone (out, &labels->literals);
+ else if (labels->expr)
+ {
+ gsl_matrix *m = matrix_expr_evaluate (labels->expr);
+ if (m && is_vector (m))
+ {
+ gsl_vector v = to_vector (m);
+ for (size_t i = 0; i < v.size; i++)
+ string_array_append_nocopy (out, trimmed_string (
+ gsl_vector_get (&v, i)));
+ }
+ gsl_matrix_free (m);
+ }
+
+ while (out->n < n)
+ {
+ if (labels->expr)
+ string_array_append_nocopy (out, xasprintf ("%s%zu", prefix,
+ out->n + 1));
+ else
+ string_array_append (out, "");
+ }
+ while (out->n > n)
+ string_array_delete (out, out->n - 1);
+
+ if (truncate)
+ for (size_t i = 0; i < out->n; i++)
+ {
+ char *s = out->strings[i];
+ s[strnlen (s, 8)] = '\0';
+ }
+
+ return out;
+}
+
+static void
+matrix_print_space (int space)
+{
+ if (space < 0)
+ output_item_submit (page_break_item_create ());
+ for (int i = 0; i < space; i++)
+ output_log ("%s", "");
+}
+
+static void
+matrix_print_text (const struct matrix_print *print, const gsl_matrix *m,
+ struct fmt_spec format, int log_scale)
+{
+ matrix_print_space (print->space);
+ if (print->title)
+ output_log ("%s", print->title);
+ if (log_scale != 0)
+ output_log (" 10 ** %d X", log_scale);
+
+ struct string_array *clabels = print_labels_get (print->clabels,
+ m->size2, "col", true);
+ if (clabels && format.w < 8)
+ format.w = 8;
+ struct string_array *rlabels = print_labels_get (print->rlabels,
+ m->size1, "row", true);
+
+ if (clabels)
+ {
+ struct string line = DS_EMPTY_INITIALIZER;
+ if (rlabels)
+ ds_put_byte_multiple (&line, ' ', 8);
+ for (size_t x = 0; x < m->size2; x++)
+ ds_put_format (&line, " %*s", format.w, clabels->strings[x]);
+ output_log_nocopy (ds_steal_cstr (&line));
+ }
+
+ double scale = pow (10.0, log_scale);
+ bool numeric = fmt_is_numeric (format.type);
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ struct string line = DS_EMPTY_INITIALIZER;
+ if (rlabels)
+ ds_put_format (&line, "%-8s", rlabels->strings[y]);
+
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double f = gsl_matrix_get (m, y, x);
+ char *s = (numeric
+ ? data_out (&(union value) { .f = f / scale}, NULL,
+ &format, settings_get_fmt_settings ())
+ : trimmed_string (f));
+ ds_put_format (&line, " %s", s);
+ free (s);
+ }
+ output_log_nocopy (ds_steal_cstr (&line));
+ }
+
+ string_array_destroy (rlabels);
+ free (rlabels);
+ string_array_destroy (clabels);
+ free (clabels);
+}
+
+static void
+create_print_dimension (struct pivot_table *table, enum pivot_axis_type axis,
+ const struct print_labels *print_labels, size_t n,
+ const char *name, const char *prefix)
+{
+ struct string_array *labels = print_labels_get (print_labels, n, prefix,
+ false);
+ struct pivot_dimension *d = pivot_dimension_create (table, axis, name);
+ for (size_t i = 0; i < n; i++)
+ pivot_category_create_leaf (
+ d->root, (labels
+ ? pivot_value_new_user_text (labels->strings[i], SIZE_MAX)
+ : pivot_value_new_integer (i + 1)));
+ if (!labels)
+ d->hide_all_labels = true;
+ string_array_destroy (labels);
+ free (labels);
+}
+
+static void
+matrix_print_tables (const struct matrix_print *print, const gsl_matrix *m,
+ struct fmt_spec format, int log_scale)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_user_text (print->title ? print->title : "", SIZE_MAX),
+ "Matrix Print");
+
+ create_print_dimension (table, PIVOT_AXIS_ROW, print->rlabels, m->size1,
+ N_("Rows"), "row");
+ create_print_dimension (table, PIVOT_AXIS_COLUMN, print->clabels, m->size2,
+ N_("Columns"), "col");
+
+ struct pivot_footnote *footnote = NULL;
+ if (log_scale != 0)
+ {
+ char *s = xasprintf ("× 10**%d", log_scale);
+ footnote = pivot_table_create_footnote (
+ table, pivot_value_new_user_text_nocopy (s));
+ }
+
+ double scale = pow (10.0, log_scale);
+ bool numeric = fmt_is_numeric (format.type);
+ for (size_t y = 0; y < m->size1; y++)
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double f = gsl_matrix_get (m, y, x);
+ struct pivot_value *v;
+ if (numeric)
+ {
+ v = pivot_value_new_number (f / scale);
+ v->numeric.format = format;
+ }
+ else
+ v = pivot_value_new_user_text_nocopy (trimmed_string (f));
+ if (footnote)
+ pivot_value_add_footnote (v, footnote);
+ pivot_table_put2 (table, y, x, v);
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+matrix_print_execute (const struct matrix_print *print)
+{
+ if (print->expression)
+ {
+ gsl_matrix *m = matrix_expr_evaluate (print->expression);
+ if (!m)
+ return;
+
+ int log_scale = 0;
+ struct fmt_spec format = (print->use_default_format
+ ? default_format (m, &log_scale)
+ : print->format);
+
+ if (settings_get_mdisplay () == SETTINGS_MDISPLAY_TEXT)
+ matrix_print_text (print, m, format, log_scale);
+ else
+ matrix_print_tables (print, m, format, log_scale);
+
+ gsl_matrix_free (m);
+ }
+ else
+ {
+ matrix_print_space (print->space);
+ if (print->title)
+ output_log ("%s", print->title);
+ }
+}
+\f
+/* DO IF. */
+
+static bool
+matrix_do_if_clause_parse (struct matrix_state *s,
+ struct matrix_do_if *ifc,
+ bool parse_condition,
+ size_t *allocated_clauses)
+{
+ if (ifc->n_clauses >= *allocated_clauses)
+ ifc->clauses = x2nrealloc (ifc->clauses, allocated_clauses,
+ sizeof *ifc->clauses);
+ struct do_if_clause *c = &ifc->clauses[ifc->n_clauses++];
+ *c = (struct do_if_clause) { .condition = NULL };
+
+ if (parse_condition)
+ {
+ c->condition = matrix_expr_parse (s);
+ if (!c->condition)
+ return false;
+ }
+
+ return matrix_commands_parse (s, &c->commands, "DO IF", "ELSE", "END IF");
+}
+
+static struct matrix_command *
+matrix_do_if_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_DO_IF,
+ .do_if = { .n_clauses = 0 }
+ };
+
+ size_t allocated_clauses = 0;
+ do
+ {
+ if (!matrix_do_if_clause_parse (s, &cmd->do_if, true, &allocated_clauses))
+ goto error;
+ }
+ while (lex_match_phrase (s->lexer, "ELSE IF"));
+
+ if (lex_match_id (s->lexer, "ELSE")
+ && !matrix_do_if_clause_parse (s, &cmd->do_if, false, &allocated_clauses))
+ goto error;
+
+ if (!lex_match_phrase (s->lexer, "END IF"))
+ NOT_REACHED ();
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static bool
+matrix_do_if_execute (struct matrix_do_if *cmd)
+{
+ for (size_t i = 0; i < cmd->n_clauses; i++)
+ {
+ struct do_if_clause *c = &cmd->clauses[i];
+ if (c->condition)
+ {
+ double d;
+ if (!matrix_expr_evaluate_scalar (c->condition,
+ i ? "ELSE IF" : "DO IF",
+ &d) || d <= 0)
+ continue;
+ }
+
+ for (size_t j = 0; j < c->commands.n; j++)
+ if (!matrix_command_execute (c->commands.commands[j]))
+ return false;
+ return true;
+ }
+ return true;
+}
+\f
+/* LOOP. */
+
+static struct matrix_command *
+matrix_loop_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) { .type = MCMD_LOOP, .loop = { .var = NULL } };
+
+ struct matrix_loop *loop = &cmd->loop;
+ if (lex_token (s->lexer) == T_ID && lex_next_token (s->lexer, 1) == T_EQUALS)
+ {
+ struct substring name = lex_tokss (s->lexer);
+ loop->var = matrix_var_lookup (s, name);
+ if (!loop->var)
+ loop->var = matrix_var_create (s, name);
+
+ lex_get (s->lexer);
+ lex_get (s->lexer);
+
+ loop->start = matrix_expr_parse (s);
+ if (!loop->start || !lex_force_match (s->lexer, T_TO))
+ goto error;
+
+ loop->end = matrix_expr_parse (s);
+ if (!loop->end)
+ goto error;
+
+ if (lex_match (s->lexer, T_BY))
+ {
+ loop->increment = matrix_expr_parse (s);
+ if (!loop->increment)
+ goto error;
+ }
+ }
+
+ if (lex_match_id (s->lexer, "IF"))
+ {
+ loop->top_condition = matrix_expr_parse (s);
+ if (!loop->top_condition)
+ goto error;
+ }
+
+ bool was_in_loop = s->in_loop;
+ s->in_loop = true;
+ bool ok = matrix_commands_parse (s, &loop->commands, "LOOP",
+ "END LOOP", NULL);
+ s->in_loop = was_in_loop;
+ if (!ok)
+ goto error;
+
+ if (!lex_match_phrase (s->lexer, "END LOOP"))
+ NOT_REACHED ();
+
+ if (lex_match_id (s->lexer, "IF"))
+ {
+ loop->bottom_condition = matrix_expr_parse (s);
+ if (!loop->bottom_condition)
+ goto error;
+ }
+
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_loop_execute (struct matrix_loop *cmd)
+{
+ long int value = 0;
+ long int end = 0;
+ long int increment = 1;
+ if (cmd->var)
+ {
+ if (!matrix_expr_evaluate_integer (cmd->start, "LOOP", &value)
+ || !matrix_expr_evaluate_integer (cmd->end, "TO", &end)
+ || (cmd->increment
+ && !matrix_expr_evaluate_integer (cmd->increment, "BY",
+ &increment)))
+ return;
+
+ if (increment > 0 ? value > end
+ : increment < 0 ? value < end
+ : true)
+ return;
+ }
+
+ int mxloops = settings_get_mxloops ();
+ for (int i = 0; i < mxloops; i++)
+ {
+ if (cmd->var)
+ {
+ if (cmd->var->value
+ && (cmd->var->value->size1 != 1 || cmd->var->value->size2 != 1))
+ {
+ gsl_matrix_free (cmd->var->value);
+ cmd->var->value = NULL;
+ }
+ if (!cmd->var->value)
+ cmd->var->value = gsl_matrix_alloc (1, 1);
+
+ gsl_matrix_set (cmd->var->value, 0, 0, value);
+ }
+
+ if (cmd->top_condition)
+ {
+ double d;
+ if (!matrix_expr_evaluate_scalar (cmd->top_condition, "LOOP IF",
+ &d) || d <= 0)
+ return;
+ }
+
+ for (size_t j = 0; j < cmd->commands.n; j++)
+ if (!matrix_command_execute (cmd->commands.commands[j]))
+ return;
+
+ if (cmd->bottom_condition)
+ {
+ double d;
+ if (!matrix_expr_evaluate_scalar (cmd->bottom_condition,
+ "END LOOP IF", &d) || d > 0)
+ return;
+ }
+
+ if (cmd->var)
+ {
+ value += increment;
+ if (increment > 0 ? value > end : value < end)
+ return;
+ }
+ }
+}
+\f
+/* BREAK. */
+
+static struct matrix_command *
+matrix_break_parse (struct matrix_state *s)
+{
+ if (!s->in_loop)
+ {
+ lex_next_error (s->lexer, -1, -1, _("BREAK not inside LOOP."));
+ return NULL;
+ }
+
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) { .type = MCMD_BREAK };
+ return cmd;
+}
+\f
+/* DISPLAY. */
+
+static struct matrix_command *
+matrix_display_parse (struct matrix_state *s)
+{
+ while (lex_token (s->lexer) != T_ENDCMD)
+ {
+ if (!lex_match_id (s->lexer, "DICTIONARY")
+ && !lex_match_id (s->lexer, "STATUS"))
+ {
+ lex_error_expecting (s->lexer, "DICTIONARY", "STATUS");
+ return NULL;
+ }
+ }
+
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) { .type = MCMD_DISPLAY, .display = { s } };
+ return cmd;
+}
+
+static int
+compare_matrix_var_pointers (const void *a_, const void *b_)
+{
+ const struct matrix_var *const *ap = a_;
+ const struct matrix_var *const *bp = b_;
+ const struct matrix_var *a = *ap;
+ const struct matrix_var *b = *bp;
+ return strcmp (a->name, b->name);
+}
+
+static void
+matrix_display_execute (struct matrix_display *cmd)
+{
+ const struct matrix_state *s = cmd->state;
+
+ struct pivot_table *table = pivot_table_create (N_("Matrix Variables"));
+ struct pivot_dimension *attr_dimension
+ = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Attribute"));
+ pivot_category_create_group (attr_dimension->root, N_("Dimension"),
+ N_("Rows"), N_("Columns"));
+ pivot_category_create_leaves (attr_dimension->root, N_("Size (kB)"));
+
+ const struct matrix_var **vars = xmalloc (hmap_count (&s->vars) * sizeof *vars);
+ size_t n_vars = 0;
+
+ const struct matrix_var *var;
+ HMAP_FOR_EACH (var, struct matrix_var, hmap_node, &s->vars)
+ if (var->value)
+ vars[n_vars++] = var;
+ qsort (vars, n_vars, sizeof *vars, compare_matrix_var_pointers);
+
+ struct pivot_dimension *rows = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ const struct matrix_var *var = vars[i];
+ pivot_category_create_leaf (
+ rows->root, pivot_value_new_user_text (var->name, SIZE_MAX));
+
+ size_t r = var->value->size1;
+ size_t c = var->value->size2;
+ double values[] = { r, c, r * c * sizeof (double) / 1024 };
+ for (size_t j = 0; j < sizeof values / sizeof *values; j++)
+ pivot_table_put2 (table, j, i, pivot_value_new_integer (values[j]));
+ }
+ free (vars);
+ pivot_table_submit (table);
+}
+\f
+/* RELEASE. */
+
+static struct matrix_command *
+matrix_release_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) { .type = MCMD_RELEASE };
+
+ size_t allocated_vars = 0;
+ while (lex_token (s->lexer) == T_ID)
+ {
+ struct matrix_var *var = matrix_var_lookup (s, lex_tokss (s->lexer));
+ if (var)
+ {
+ if (cmd->release.n_vars >= allocated_vars)
+ cmd->release.vars = x2nrealloc (cmd->release.vars, &allocated_vars,
+ sizeof *cmd->release.vars);
+ cmd->release.vars[cmd->release.n_vars++] = var;
+ }
+ else
+ lex_error (s->lexer, _("Syntax error expecting variable name."));
+ lex_get (s->lexer);
+
+ if (!lex_match (s->lexer, T_COMMA))
+ break;
+ }
+
+ return cmd;
+}
+
+static void
+matrix_release_execute (struct matrix_release *cmd)
+{
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ {
+ struct matrix_var *var = cmd->vars[i];
+ gsl_matrix_free (var->value);
+ var->value = NULL;
+ }
+}
+\f
+/* SAVE. */
+
+static struct save_file *
+save_file_create (struct matrix_state *s, struct file_handle *fh,
+ struct string_array *variables,
+ struct matrix_expr *names,
+ struct stringi_set *strings)
+{
+ for (size_t i = 0; i < s->n_save_files; i++)
+ {
+ struct save_file *sf = s->save_files[i];
+ if (fh_equal (sf->file, fh))
+ {
+ fh_unref (fh);
+
+ string_array_destroy (variables);
+ matrix_expr_destroy (names);
+ stringi_set_destroy (strings);
+
+ return sf;
+ }
+ }
+
+ struct save_file *sf = xmalloc (sizeof *sf);
+ *sf = (struct save_file) {
+ .file = fh,
+ .dataset = s->dataset,
+ .variables = *variables,
+ .names = names,
+ .strings = STRINGI_SET_INITIALIZER (sf->strings),
+ };
+
+ stringi_set_swap (&sf->strings, strings);
+ stringi_set_destroy (strings);
+
+ s->save_files = xrealloc (s->save_files,
+ (s->n_save_files + 1) * sizeof *s->save_files);
+ s->save_files[s->n_save_files++] = sf;
+ return sf;
+}
+
+static struct casewriter *
+save_file_open (struct save_file *sf, gsl_matrix *m,
+ const struct msg_location *save_location)
+{
+ if (sf->writer || sf->error)
+ {
+ if (sf->writer)
+ {
+ size_t n_variables = caseproto_get_n_widths (
+ casewriter_get_proto (sf->writer));
+ if (m->size2 != n_variables)
+ {
+ const char *file_name = (sf->file == fh_inline_file () ? "*"
+ : fh_get_name (sf->file));
+ msg_at (SE, save_location,
+ _("Cannot save %zu×%zu matrix to %s because the "
+ "first SAVE to %s in this matrix program wrote a "
+ "%zu-column matrix."),
+ m->size1, m->size2, file_name, file_name, n_variables);
+ msg_at (SE, sf->location,
+ _("This is the location of the first SAVE to %s."),
+ file_name);
+ return NULL;
+ }
+ }
+ return sf->writer;
+ }
+
+ bool ok = true;
+ struct dictionary *dict = dict_create (get_default_encoding ());
+
+ /* Fill 'names' with user-specified names if there were any, either from
+ sf->variables or sf->names. */
+ struct string_array names = { .n = 0 };
+ if (sf->variables.n)
+ string_array_clone (&names, &sf->variables);
+ else if (sf->names)
+ {
+ gsl_matrix *nm = matrix_expr_evaluate (sf->names);
+ if (nm && is_vector (nm))
+ {
+ gsl_vector nv = to_vector (nm);
+ for (size_t i = 0; i < nv.size; i++)
+ {
+ char *name = trimmed_string (gsl_vector_get (&nv, i));
+ char *error = dict_id_is_valid__ (dict, name);
+ if (!error)
+ string_array_append_nocopy (&names, name);
+ else
+ {
+ msg_at (SE, save_location, "%s", error);
+ free (error);
+ ok = false;
+ }
+ }
+ }
+ gsl_matrix_free (nm);
+ }
+
+ struct stringi_set strings;
+ stringi_set_clone (&strings, &sf->strings);
+
+ for (size_t i = 0; dict_get_n_vars (dict) < m->size2; i++)
+ {
+ char tmp_name[64];
+ const char *name;
+ if (i < names.n)
+ name = names.strings[i];
+ else
+ {
+ snprintf (tmp_name, sizeof tmp_name, "COL%zu", i + 1);
+ name = tmp_name;
+ }
+
+ int width = stringi_set_delete (&strings, name) ? 8 : 0;
+ struct variable *var = dict_create_var (dict, name, width);
+ if (!var)
+ {
+ msg_at (ME, save_location,
+ _("Duplicate variable name %s in SAVE statement."), name);
+ ok = false;
+ }
+ }
+
+ if (!stringi_set_is_empty (&strings))
+ {
+ size_t count = stringi_set_count (&strings);
+ const char *example = stringi_set_node_get_string (
+ stringi_set_first (&strings));
+
+ if (count == 1)
+ msg_at (ME, save_location,
+ _("The SAVE command STRINGS subcommand specifies an unknown "
+ "variable %s."), example);
+ else
+ msg_at (ME, save_location,
+ ngettext ("The SAVE command STRINGS subcommand specifies %zu "
+ "unknown variable: %s.",
+ "The SAVE command STRINGS subcommand specifies %zu "
+ "unknown variables, including %s.",
+ count),
+ count, example);
+ ok = false;
+ }
+ stringi_set_destroy (&strings);
+ string_array_destroy (&names);
+
+ if (!ok)
+ {
+ dict_unref (dict);
+ sf->error = true;
+ return NULL;
+ }
+
+ if (sf->file == fh_inline_file ())
+ sf->writer = autopaging_writer_create (dict_get_proto (dict));
+ else
+ sf->writer = any_writer_open (sf->file, dict);
+ if (sf->writer)
+ {
+ sf->dict = dict;
+ sf->location = msg_location_dup (save_location);
+ }
+ else
+ {
+ dict_unref (dict);
+ sf->error = true;
+ }
+ return sf->writer;
+}
+
+static void
+save_file_destroy (struct save_file *sf)
+{
+ if (sf)
+ {
+ if (sf->file == fh_inline_file () && sf->writer && sf->dict)
+ {
+ dataset_set_dict (sf->dataset, sf->dict);
+ dataset_set_source (sf->dataset, casewriter_make_reader (sf->writer));
+ }
+ else
+ {
+ casewriter_destroy (sf->writer);
+ dict_unref (sf->dict);
+ }
+ fh_unref (sf->file);
+ string_array_destroy (&sf->variables);
+ matrix_expr_destroy (sf->names);
+ stringi_set_destroy (&sf->strings);
+ msg_location_destroy (sf->location);
+ free (sf);
+ }
+}
+
+static struct matrix_command *
+matrix_save_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_SAVE,
+ .save = { .expression = NULL },
+ };
+
+ struct file_handle *fh = NULL;
+ struct matrix_save *save = &cmd->save;
+
+ struct string_array variables = STRING_ARRAY_INITIALIZER;
+ struct matrix_expr *names = NULL;
+ struct stringi_set strings = STRINGI_SET_INITIALIZER (strings);
+
+ save->expression = matrix_parse_exp (s);
+ if (!save->expression)
+ goto error;
+
+ int names_start = 0;
+ int names_end = 0;
+ while (lex_match (s->lexer, T_SLASH))
+ {
+ if (lex_match_id (s->lexer, "OUTFILE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ fh_unref (fh);
+ fh = (lex_match (s->lexer, T_ASTERISK)
+ ? fh_inline_file ()
+ : fh_parse (s->lexer, FH_REF_FILE, s->session));
+ if (!fh)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "VARIABLES"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ char **names;
+ size_t n;
+ struct dictionary *d = dict_create (get_default_encoding ());
+ bool ok = parse_DATA_LIST_vars (s->lexer, d, &names, &n,
+ PV_NO_SCRATCH | PV_NO_DUPLICATE);
+ dict_unref (d);
+ if (!ok)
+ goto error;
+
+ string_array_clear (&variables);
+ variables = (struct string_array) {
+ .strings = names,
+ .n = n,
+ .allocated = n,
+ };
+ }
+ else if (lex_match_id (s->lexer, "NAMES"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ matrix_expr_destroy (names);
+ names_start = lex_ofs (s->lexer);
+ names = matrix_parse_exp (s);
+ names_end = lex_ofs (s->lexer) - 1;
+ if (!names)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "STRINGS"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ while (lex_token (s->lexer) == T_ID)
+ {
+ stringi_set_insert (&strings, lex_tokcstr (s->lexer));
+ lex_get (s->lexer);
+ if (!lex_match (s->lexer, T_COMMA))
+ break;
+ }
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "OUTFILE", "VARIABLES", "NAMES",
+ "STRINGS");
+ goto error;
+ }
+ }
+
+ if (!fh)
+ {
+ if (s->prev_save_file)
+ fh = fh_ref (s->prev_save_file);
+ else
+ {
+ lex_sbc_missing (s->lexer, "OUTFILE");
+ goto error;
+ }
+ }
+ fh_unref (s->prev_save_file);
+ s->prev_save_file = fh_ref (fh);
+
+ if (variables.n && names)
+ {
+ lex_ofs_msg (s->lexer, SW, names_start, names_end,
+ _("Ignoring NAMES because VARIABLES was also specified."));
+ matrix_expr_destroy (names);
+ names = NULL;
+ }
+
+ save->sf = save_file_create (s, fh, &variables, names, &strings);
+ return cmd;
+
+error:
+ string_array_destroy (&variables);
+ matrix_expr_destroy (names);
+ stringi_set_destroy (&strings);
+ fh_unref (fh);
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_save_execute (const struct matrix_command *cmd)
+{
+ const struct matrix_save *save = &cmd->save;
+
+ gsl_matrix *m = matrix_expr_evaluate (save->expression);
+ if (!m)
+ return;
+
+ struct casewriter *writer = save_file_open (save->sf, m, cmd->location);
+ if (!writer)
+ {
+ gsl_matrix_free (m);
+ return;
+ }
+
+ const struct caseproto *proto = casewriter_get_proto (writer);
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ struct ccase *c = case_create (proto);
+ for (size_t x = 0; x < m->size2; x++)
+ {
+ double d = gsl_matrix_get (m, y, x);
+ int width = caseproto_get_width (proto, x);
+ union value *value = case_data_rw_idx (c, x);
+ if (width == 0)
+ value->f = d;
+ else
+ memcpy (value->s, &d, width);
+ }
+ casewriter_write (writer, c);
+ }
+ gsl_matrix_free (m);
+}
+\f
+/* READ. */
+
+static struct read_file *
+read_file_create (struct matrix_state *s, struct file_handle *fh)
+{
+ for (size_t i = 0; i < s->n_read_files; i++)
+ {
+ struct read_file *rf = s->read_files[i];
+ if (rf->file == fh)
+ {
+ fh_unref (fh);
+ return rf;
+ }
+ }
+
+ struct read_file *rf = xmalloc (sizeof *rf);
+ *rf = (struct read_file) { .file = fh };
+
+ s->read_files = xrealloc (s->read_files,
+ (s->n_read_files + 1) * sizeof *s->read_files);
+ s->read_files[s->n_read_files++] = rf;
+ return rf;
+}
+
+static struct dfm_reader *
+read_file_open (struct read_file *rf)
+{
+ if (!rf->reader)
+ rf->reader = dfm_open_reader (rf->file, NULL, rf->encoding);
+ return rf->reader;
+}
+
+static void
+read_file_destroy (struct read_file *rf)
+{
+ if (rf)
+ {
+ fh_unref (rf->file);
+ dfm_close_reader (rf->reader);
+ free (rf->encoding);
+ free (rf);
+ }
+}
+
+static struct matrix_command *
+matrix_read_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_READ,
+ .read = { .format = FMT_F },
+ };
+
+ struct file_handle *fh = NULL;
+ char *encoding = NULL;
+ struct matrix_read *read = &cmd->read;
+ read->dst = matrix_lvalue_parse (s);
+ if (!read->dst)
+ goto error;
+
+ int by_ofs = 0;
+ int format_ofs = 0;
+ int record_width_start = 0, record_width_end = 0;
+
+ int by = 0;
+ int repetitions = 0;
+ int record_width = 0;
+ bool seen_format = false;
+ while (lex_match (s->lexer, T_SLASH))
+ {
+ if (lex_match_id (s->lexer, "FILE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ fh_unref (fh);
+ fh = fh_parse (s->lexer, FH_REF_FILE, NULL);
+ if (!fh)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "ENCODING"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!lex_force_string (s->lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (s->lexer));
+
+ lex_get (s->lexer);
+ }
+ else if (lex_match_id (s->lexer, "FIELD"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ record_width_start = lex_ofs (s->lexer);
+ if (!lex_force_int_range (s->lexer, "FIELD", 1, INT_MAX))
+ goto error;
+ read->c1 = lex_integer (s->lexer);
+ lex_get (s->lexer);
+ if (!lex_force_match (s->lexer, T_TO)
+ || !lex_force_int_range (s->lexer, "TO", read->c1, INT_MAX))
+ goto error;
+ read->c2 = lex_integer (s->lexer) + 1;
+ record_width_end = lex_ofs (s->lexer);
+ lex_get (s->lexer);
+
+ record_width = read->c2 - read->c1;
+ if (lex_match (s->lexer, T_BY))
+ {
+ if (!lex_force_int_range (s->lexer, "BY", 1,
+ read->c2 - read->c1))
+ goto error;
+ by = lex_integer (s->lexer);
+ by_ofs = lex_ofs (s->lexer);
+ int field_end = lex_ofs (s->lexer);
+ lex_get (s->lexer);
+
+ if (record_width % by)
+ {
+ lex_ofs_error (
+ s->lexer, record_width_start, field_end,
+ _("Field width %d does not evenly divide record width %d."),
+ by, record_width);
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates the record width."));
+ lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
+ _("This syntax specifies the field width."));
+ goto error;
+ }
+ }
+ else
+ by = 0;
+ }
+ else if (lex_match_id (s->lexer, "SIZE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ matrix_expr_destroy (read->size);
+ read->size = matrix_parse_exp (s);
+ if (!read->size)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "MODE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (lex_match_id (s->lexer, "RECTANGULAR"))
+ read->symmetric = false;
+ else if (lex_match_id (s->lexer, "SYMMETRIC"))
+ read->symmetric = true;
+ else
+ {
+ lex_error_expecting (s->lexer, "RECTANGULAR", "SYMMETRIC");
+ goto error;
+ }
+ }
+ else if (lex_match_id (s->lexer, "REREAD"))
+ read->reread = true;
+ else if (lex_match_id (s->lexer, "FORMAT"))
+ {
+ if (seen_format)
+ {
+ lex_sbc_only_once (s->lexer, "FORMAT");
+ goto error;
+ }
+ seen_format = true;
+
+ lex_match (s->lexer, T_EQUALS);
+
+ if (lex_token (s->lexer) != T_STRING && !lex_force_id (s->lexer))
+ goto error;
+
+ format_ofs = lex_ofs (s->lexer);
+ const char *p = lex_tokcstr (s->lexer);
+ if (c_isdigit (p[0]))
+ {
+ repetitions = atoi (p);
+ p += strspn (p, "0123456789");
+ if (!fmt_from_name (p, &read->format))
+ {
+ lex_error (s->lexer, _("Unknown format %s."), p);
+ goto error;
+ }
+ lex_get (s->lexer);
+ }
+ else if (fmt_from_name (p, &read->format))
+ lex_get (s->lexer);
+ else
+ {
+ struct fmt_spec format;
+ if (!parse_format_specifier (s->lexer, &format))
+ goto error;
+ read->format = format.type;
+ read->w = format.w;
+ }
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "FILE", "FIELD", "MODE",
+ "REREAD", "FORMAT");
+ goto error;
+ }
+ }
+
+ if (!read->c1)
+ {
+ lex_sbc_missing (s->lexer, "FIELD");
+ goto error;
+ }
+
+ if (!read->dst->n_indexes && !read->size)
+ {
+ msg (SE, _("SIZE is required for reading data into a full matrix "
+ "(as opposed to a submatrix)."));
+ msg_at (SN, read->dst->var_location,
+ _("This expression designates a full matrix."));
+ goto error;
+ }
+
+ if (!fh)
+ {
+ if (s->prev_read_file)
+ fh = fh_ref (s->prev_read_file);
+ else
+ {
+ lex_sbc_missing (s->lexer, "FILE");
+ goto error;
+ }
+ }
+ fh_unref (s->prev_read_file);
+ s->prev_read_file = fh_ref (fh);
+
+ read->rf = read_file_create (s, fh);
+ fh = NULL;
+ if (encoding)
+ {
+ free (read->rf->encoding);
+ read->rf->encoding = encoding;
+ encoding = NULL;
+ }
+
+ /* Field width may be specified in multiple ways:
+
+ 1. BY on FIELD.
+ 2. The format on FORMAT.
+ 3. The repetition factor on FORMAT.
+
+ (2) and (3) are mutually exclusive.
+
+ If more than one of these is present, they must agree. If none of them is
+ present, then free-field format is used.
+ */
+ if (repetitions > record_width)
+ {
+ msg (SE, _("%d repetitions cannot fit in record width %d."),
+ repetitions, record_width);
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ _("This syntax designates the number of repetitions."));
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates the record width."));
+ goto error;
+ }
+ int w = (repetitions ? record_width / repetitions
+ : read->w ? read->w
+ : by);
+ if (by && w != by)
+ {
+ msg (SE, _("This command specifies two different field widths."));
+ if (repetitions)
+ {
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ ngettext ("This syntax specifies %d repetition.",
+ "This syntax specifies %d repetitions.",
+ repetitions),
+ repetitions);
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates record width %d, "
+ "which divided by %d repetitions implies "
+ "field width %d."),
+ record_width, repetitions, w);
+ }
+ else
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ _("This syntax specifies field width %d."), w);
+
+ lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
+ _("This syntax specifies field width %d."), by);
+ goto error;
+ }
+ read->w = w;
+ return cmd;
+
+error:
+ fh_unref (fh);
+ matrix_command_destroy (cmd);
+ free (encoding);
+ return NULL;
+}
+
+static void
+parse_error (const struct dfm_reader *reader, enum fmt_type format,
+ struct substring data, size_t y, size_t x,
+ int first_column, int last_column, char *error)
+{
+ int line_number = dfm_get_line_number (reader);
+ struct msg_location location = {
+ .file_name = intern_new (dfm_get_file_name (reader)),
+ .start = { .line = line_number, .column = first_column },
+ .end = { .line = line_number, .column = last_column },
+ };
+ msg_at (DW, &location, _("Error reading \"%.*s\" as format %s "
+ "for matrix row %zu, column %zu: %s"),
+ (int) data.length, data.string, fmt_name (format),
+ y + 1, x + 1, error);
+ msg_location_uninit (&location);
+ free (error);
+}
+
+static void
+matrix_read_set_field (struct matrix_read *read, struct dfm_reader *reader,
+ gsl_matrix *m, struct substring p, size_t y, size_t x,
+ const char *line_start)
+{
+ const char *input_encoding = dfm_reader_get_encoding (reader);
+ char *error;
+ double f;
+ if (fmt_is_numeric (read->format))
+ {
+ union value v;
+ error = data_in (p, input_encoding, read->format,
+ settings_get_fmt_settings (), &v, 0, NULL);
+ if (!error && v.f == SYSMIS)
+ error = xstrdup (_("Matrix data may not contain missing value."));
+ f = v.f;
+ }
+ else
+ {
+ uint8_t s[sizeof (double)];
+ union value v = { .s = s };
+ error = data_in (p, input_encoding, read->format,
+ settings_get_fmt_settings (), &v, sizeof s, "UTF-8");
+ memcpy (&f, s, sizeof f);
+ }
+
+ if (error)
+ {
+ int c1 = utf8_count_columns (line_start, p.string - line_start) + 1;
+ int nc = ss_utf8_count_columns (p);
+ int c2 = c1 + MAX (1, nc) - 1;
+ parse_error (reader, read->format, p, y, x, c1, c2, error);
+ }
+ else
+ {
+ gsl_matrix_set (m, y, x, f);
+ if (read->symmetric && x != y)
+ gsl_matrix_set (m, x, y, f);
+ }
+}
+
+static bool
+matrix_read_line (struct matrix_command *cmd, struct dfm_reader *reader,
+ struct substring *line, const char **startp)
+{
+ struct matrix_read *read = &cmd->read;
+ if (dfm_eof (reader))
+ {
+ msg_at (SE, cmd->location,
+ _("Unexpected end of file reading matrix data."));
+ return false;
+ }
+ dfm_expand_tabs (reader);
+ struct substring record = dfm_get_record (reader);
+ /* XXX need to recode record into UTF-8 */
+ *startp = record.string;
+ *line = ss_utf8_columns (record, read->c1 - 1, read->c2 - read->c1);
+ return true;
+}
+
+static void
+matrix_read (struct matrix_command *cmd, struct dfm_reader *reader,
+ gsl_matrix *m)
+{
+ struct matrix_read *read = &cmd->read;
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ size_t nx = read->symmetric ? y + 1 : m->size2;
+
+ struct substring line = ss_empty ();
+ const char *line_start = line.string;
+ for (size_t x = 0; x < nx; x++)
+ {
+ struct substring p;
+ if (!read->w)
+ {
+ for (;;)
+ {
+ ss_ltrim (&line, ss_cstr (" ,"));
+ if (!ss_is_empty (line))
+ break;
+ if (!matrix_read_line (cmd, reader, &line, &line_start))
+ return;
+ dfm_forward_record (reader);
+ }
+
+ ss_get_bytes (&line, ss_cspan (line, ss_cstr (" ,")), &p);
+ }
+ else
+ {
+ if (!matrix_read_line (cmd, reader, &line, &line_start))
+ return;
+ size_t fields_per_line = (read->c2 - read->c1) / read->w;
+ int f = x % fields_per_line;
+ if (f == fields_per_line - 1)
+ dfm_forward_record (reader);
+
+ p = ss_substr (line, read->w * f, read->w);
+ }
+
+ matrix_read_set_field (read, reader, m, p, y, x, line_start);
+ }
+
+ if (read->w)
+ dfm_forward_record (reader);
+ else
+ {
+ ss_ltrim (&line, ss_cstr (" ,"));
+ if (!ss_is_empty (line))
+ {
+ int line_number = dfm_get_line_number (reader);
+ int c1 = utf8_count_columns (line_start,
+ line.string - line_start) + 1;
+ int c2 = c1 + ss_utf8_count_columns (line) - 1;
+ struct msg_location location = {
+ .file_name = intern_new (dfm_get_file_name (reader)),
+ .start = { .line = line_number, .column = c1 },
+ .end = { .line = line_number, .column = c2 },
+ };
+ msg_at (DW, &location,
+ _("Trailing garbage following data for matrix row %zu."),
+ y + 1);
+ msg_location_uninit (&location);
+ }
+ }
+ }
+}
+
+static void
+matrix_read_execute (struct matrix_command *cmd)
+{
+ struct matrix_read *read = &cmd->read;
+ struct index_vector iv0, iv1;
+ if (!matrix_lvalue_evaluate (read->dst, &iv0, &iv1))
+ return;
+
+ size_t size[2] = { SIZE_MAX, SIZE_MAX };
+ if (read->size)
+ {
+ gsl_matrix *m = matrix_expr_evaluate (read->size);
+ if (!m)
+ return;
+
+ if (!is_vector (m))
+ {
+ msg_at (SE, matrix_expr_location (read->size),
+ _("SIZE must evaluate to a scalar or a 2-element vector, "
+ "not a %zu×%zu matrix."), m->size1, m->size2);
+ gsl_matrix_free (m);
+ index_vector_uninit (&iv0);
+ index_vector_uninit (&iv1);
+ return;
+ }
+
+ gsl_vector v = to_vector (m);
+ double d[2];
+ if (v.size == 1)
+ {
+ d[0] = gsl_vector_get (&v, 0);
+ d[1] = 1;
+ }
+ else if (v.size == 2)
+ {
+ d[0] = gsl_vector_get (&v, 0);
+ d[1] = gsl_vector_get (&v, 1);
+ }
+ else
+ {
+ msg_at (SE, matrix_expr_location (read->size),
+ _("SIZE must evaluate to a scalar or a 2-element vector, "
+ "not a %zu×%zu matrix."),
+ m->size1, m->size2),
+ gsl_matrix_free (m);
+ index_vector_uninit (&iv0);
+ index_vector_uninit (&iv1);
+ return;
+ }
+ gsl_matrix_free (m);
+
+ if (d[0] < 0 || d[0] > SIZE_MAX || d[1] < 0 || d[1] > SIZE_MAX)
+ {
+ msg_at (SE, matrix_expr_location (read->size),
+ _("Matrix dimensions %g×%g specified on SIZE "
+ "are outside valid range."),
+ d[0], d[1]);
+ index_vector_uninit (&iv0);
+ index_vector_uninit (&iv1);
+ return;
+ }
+
+ size[0] = d[0];
+ size[1] = d[1];
+ }
+
+ if (read->dst->n_indexes)
+ {
+ size_t submatrix_size[2];
+ if (read->dst->n_indexes == 2)
+ {
+ submatrix_size[0] = iv0.n;
+ submatrix_size[1] = iv1.n;
+ }
+ else if (read->dst->var->value->size1 == 1)
+ {
+ submatrix_size[0] = 1;
+ submatrix_size[1] = iv0.n;
+ }
+ else
+ {
+ submatrix_size[0] = iv0.n;
+ submatrix_size[1] = 1;
+ }
+
+ if (read->size)
+ {
+ if (size[0] != submatrix_size[0] || size[1] != submatrix_size[1])
+ {
+ msg_at (SE, cmd->location,
+ _("Dimensions specified on SIZE differ from dimensions "
+ "of destination submatrix."));
+ msg_at (SN, matrix_expr_location (read->size),
+ _("SIZE specifies dimensions %zu×%zu."),
+ size[0], size[1]);
+ msg_at (SN, read->dst->full_location,
+ _("Destination submatrix has dimensions %zu×%zu."),
+ submatrix_size[0], submatrix_size[1]);
+ index_vector_uninit (&iv0);
+ index_vector_uninit (&iv1);
+ return;
+ }
+ }
+ else
+ {
+ size[0] = submatrix_size[0];
+ size[1] = submatrix_size[1];
+ }
+ }
+
+ struct dfm_reader *reader = read_file_open (read->rf);
+ if (read->reread)
+ dfm_reread_record (reader, 1);
+
+ if (read->symmetric && size[0] != size[1])
+ {
+ msg_at (SE, cmd->location,
+ _("Cannot read non-square %zu×%zu matrix "
+ "using READ with MODE=SYMMETRIC."),
+ size[0], size[1]);
+ index_vector_uninit (&iv0);
+ index_vector_uninit (&iv1);
+ return;
+ }
+ gsl_matrix *tmp = gsl_matrix_calloc (size[0], size[1]);
+ matrix_read (cmd, reader, tmp);
+ matrix_lvalue_assign (read->dst, &iv0, &iv1, tmp, cmd->location);
+}
+\f
+/* WRITE. */
+
+static struct write_file *
+write_file_create (struct matrix_state *s, struct file_handle *fh)
+{
+ for (size_t i = 0; i < s->n_write_files; i++)
+ {
+ struct write_file *wf = s->write_files[i];
+ if (wf->file == fh)
+ {
+ fh_unref (fh);
+ return wf;
+ }
+ }
+
+ struct write_file *wf = xmalloc (sizeof *wf);
+ *wf = (struct write_file) { .file = fh };
+
+ s->write_files = xrealloc (s->write_files,
+ (s->n_write_files + 1) * sizeof *s->write_files);
+ s->write_files[s->n_write_files++] = wf;
+ return wf;
+}
+
+static struct dfm_writer *
+write_file_open (struct write_file *wf)
+{
+ if (!wf->writer)
+ wf->writer = dfm_open_writer (wf->file, wf->encoding);
+ return wf->writer;
+}
+
+static void
+write_file_destroy (struct write_file *wf)
+{
+ if (wf)
+ {
+ if (wf->held)
+ {
+ dfm_put_record_utf8 (wf->writer, wf->held->s.ss.string,
+ wf->held->s.ss.length);
+ u8_line_destroy (wf->held);
+ free (wf->held);
+ }
+
+ fh_unref (wf->file);
+ dfm_close_writer (wf->writer);
+ free (wf->encoding);
+ free (wf);
+ }
+}
+
+static struct matrix_command *
+matrix_write_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_WRITE,
+ };
+
+ struct file_handle *fh = NULL;
+ char *encoding = NULL;
+ struct matrix_write *write = &cmd->write;
+ write->expression = matrix_parse_exp (s);
+ if (!write->expression)
+ goto error;
+
+ int by_ofs = 0;
+ int format_ofs = 0;
+ int record_width_start = 0, record_width_end = 0;
+
+ int by = 0;
+ int repetitions = 0;
+ int record_width = 0;
+ enum fmt_type format = FMT_F;
+ bool has_format = false;
+ while (lex_match (s->lexer, T_SLASH))
+ {
+ if (lex_match_id (s->lexer, "OUTFILE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ fh_unref (fh);
+ fh = fh_parse (s->lexer, FH_REF_FILE, NULL);
+ if (!fh)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "ENCODING"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!lex_force_string (s->lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (s->lexer));
+
+ lex_get (s->lexer);
+ }
+ else if (lex_match_id (s->lexer, "FIELD"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ record_width_start = lex_ofs (s->lexer);
+
+ if (!lex_force_int_range (s->lexer, "FIELD", 1, INT_MAX))
+ goto error;
+ write->c1 = lex_integer (s->lexer);
+ lex_get (s->lexer);
+ if (!lex_force_match (s->lexer, T_TO)
+ || !lex_force_int_range (s->lexer, "TO", write->c1, INT_MAX))
+ goto error;
+ write->c2 = lex_integer (s->lexer) + 1;
+ record_width_end = lex_ofs (s->lexer);
+ lex_get (s->lexer);
+
+ record_width = write->c2 - write->c1;
+ if (lex_match (s->lexer, T_BY))
+ {
+ if (!lex_force_int_range (s->lexer, "BY", 1,
+ write->c2 - write->c1))
+ goto error;
+ by_ofs = lex_ofs (s->lexer);
+ int field_end = lex_ofs (s->lexer);
+ by = lex_integer (s->lexer);
+ lex_get (s->lexer);
+
+ if (record_width % by)
+ {
+ lex_ofs_error (
+ s->lexer, record_width_start, field_end,
+ _("Field width %d does not evenly divide record width %d."),
+ by, record_width);
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates the record width."));
+ lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
+ _("This syntax specifies the field width."));
+ goto error;
+ }
+ }
+ else
+ by = 0;
+ }
+ else if (lex_match_id (s->lexer, "MODE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (lex_match_id (s->lexer, "RECTANGULAR"))
+ write->triangular = false;
+ else if (lex_match_id (s->lexer, "TRIANGULAR"))
+ write->triangular = true;
+ else
+ {
+ lex_error_expecting (s->lexer, "RECTANGULAR", "TRIANGULAR");
+ goto error;
+ }
+ }
+ else if (lex_match_id (s->lexer, "HOLD"))
+ write->hold = true;
+ else if (lex_match_id (s->lexer, "FORMAT"))
+ {
+ if (has_format || write->format)
+ {
+ lex_sbc_only_once (s->lexer, "FORMAT");
+ goto error;
+ }
+
+ lex_match (s->lexer, T_EQUALS);
+
+ if (lex_token (s->lexer) != T_STRING && !lex_force_id (s->lexer))
+ goto error;
+
+ format_ofs = lex_ofs (s->lexer);
+ const char *p = lex_tokcstr (s->lexer);
+ if (c_isdigit (p[0]))
+ {
+ repetitions = atoi (p);
+ p += strspn (p, "0123456789");
+ if (!fmt_from_name (p, &format))
+ {
+ lex_error (s->lexer, _("Unknown format %s."), p);
+ goto error;
+ }
+ has_format = true;
+ lex_get (s->lexer);
+ }
+ else if (fmt_from_name (p, &format))
+ {
+ has_format = true;
+ lex_get (s->lexer);
+ }
+ else
+ {
+ struct fmt_spec spec;
+ if (!parse_format_specifier (s->lexer, &spec))
+ goto error;
+ write->format = xmemdup (&spec, sizeof spec);
+ }
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "OUTFILE", "FIELD", "MODE",
+ "HOLD", "FORMAT");
+ goto error;
+ }
+ }
+
+ if (!write->c1)
+ {
+ lex_sbc_missing (s->lexer, "FIELD");
+ goto error;
+ }
+
+ if (!fh)
+ {
+ if (s->prev_write_file)
+ fh = fh_ref (s->prev_write_file);
+ else
+ {
+ lex_sbc_missing (s->lexer, "OUTFILE");
+ goto error;
+ }
+ }
+ fh_unref (s->prev_write_file);
+ s->prev_write_file = fh_ref (fh);
+
+ write->wf = write_file_create (s, fh);
+ fh = NULL;
+ if (encoding)
+ {
+ free (write->wf->encoding);
+ write->wf->encoding = encoding;
+ encoding = NULL;
+ }
+
+ /* Field width may be specified in multiple ways:
+
+ 1. BY on FIELD.
+ 2. The format on FORMAT.
+ 3. The repetition factor on FORMAT.
+
+ (2) and (3) are mutually exclusive.
+
+ If more than one of these is present, they must agree. If none of them is
+ present, then free-field format is used.
+ */
+ if (repetitions > record_width)
+ {
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ _("This syntax designates the number of repetitions."));
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates the record width."));
+ goto error;
+ }
+ int w = (repetitions ? record_width / repetitions
+ : write->format ? write->format->w
+ : by);
+ if (by && w != by)
+ {
+ msg (SE, _("This command specifies two different field widths."));
+ if (repetitions)
+ {
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ ngettext ("This syntax specifies %d repetition.",
+ "This syntax specifies %d repetitions.",
+ repetitions),
+ repetitions);
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates record width %d, "
+ "which divided by %d repetitions implies "
+ "field width %d."),
+ record_width, repetitions, w);
+ }
+ else
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ _("This syntax specifies field width %d."), w);
+
+ lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
+ _("This syntax specifies field width %d."), by);
+ goto error;
+ }
+ if (w && !write->format)
+ {
+ write->format = xmalloc (sizeof *write->format);
+ *write->format = (struct fmt_spec) { .type = format, .w = w };
+
+ char *error = fmt_check_output__ (write->format);
+ if (error)
+ {
+ msg (SE, "%s", error);
+ free (error);
+
+ if (has_format)
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ _("This syntax specifies format %s."),
+ fmt_name (format));
+
+ if (repetitions)
+ {
+ lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
+ ngettext ("This syntax specifies %d repetition.",
+ "This syntax specifies %d repetitions.",
+ repetitions),
+ repetitions);
+ lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
+ _("This syntax designates record width %d, "
+ "which divided by %d repetitions implies "
+ "field width %d."),
+ record_width, repetitions, w);
+ }
+
+ if (by)
+ lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
+ _("This syntax specifies field width %d."), by);
+
+ goto error;
+ }
+ }
+
+ if (write->format && fmt_var_width (write->format) > sizeof (double))
+ {
+ char fs[FMT_STRING_LEN_MAX + 1];
+ fmt_to_string (write->format, fs);
+ lex_ofs_error (s->lexer, format_ofs, format_ofs,
+ _("Format %s is too wide for %zu-byte matrix elements."),
+ fs, sizeof (double));
+ goto error;
+ }
+
+ return cmd;
+
+error:
+ fh_unref (fh);
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_write_execute (struct matrix_write *write)
+{
+ gsl_matrix *m = matrix_expr_evaluate (write->expression);
+ if (!m)
+ return;
+
+ if (write->triangular && m->size1 != m->size2)
+ {
+ msg_at (SE, matrix_expr_location (write->expression),
+ _("WRITE with MODE=TRIANGULAR requires a square matrix but "
+ "the matrix to be written has dimensions %zu×%zu."),
+ m->size1, m->size2);
+ gsl_matrix_free (m);
+ return;
+ }
+
+ struct dfm_writer *writer = write_file_open (write->wf);
+ if (!writer || !m->size1)
+ {
+ gsl_matrix_free (m);
+ return;
+ }
+
+ const struct fmt_settings *settings = settings_get_fmt_settings ();
+ struct u8_line *line = write->wf->held;
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ if (!line)
+ {
+ line = xmalloc (sizeof *line);
+ u8_line_init (line);
+ }
+ size_t nx = write->triangular ? y + 1 : m->size2;
+ int x0 = write->c1;
+ for (size_t x = 0; x < nx; x++)
+ {
+ char *s;
+ double f = gsl_matrix_get (m, y, x);
+ if (write->format)
+ {
+ union value v;
+ if (fmt_is_numeric (write->format->type))
+ v.f = f;
+ else
+ v.s = (uint8_t *) &f;
+ s = data_out (&v, NULL, write->format, settings);
+ }
+ else
+ {
+ s = xmalloc (DBL_BUFSIZE_BOUND);
+ if (c_dtoastr (s, DBL_BUFSIZE_BOUND, FTOASTR_UPPER_E, 0, f)
+ >= DBL_BUFSIZE_BOUND)
+ abort ();
+ }
+ size_t len = strlen (s);
+ int width = u8_width (CHAR_CAST (const uint8_t *, s), len, UTF8);
+ if (width + x0 > write->c2)
+ {
+ dfm_put_record_utf8 (writer, line->s.ss.string,
+ line->s.ss.length);
+ u8_line_clear (line);
+ x0 = write->c1;
+ }
+ u8_line_put (line, x0, x0 + width, s, len);
+ free (s);
+
+ x0 += write->format ? write->format->w : width + 1;
+ }
+
+ if (y + 1 >= m->size1 && write->hold)
+ break;
+ dfm_put_record_utf8 (writer, line->s.ss.string, line->s.ss.length);
+ u8_line_clear (line);
+ }
+ if (!write->hold)
+ {
+ u8_line_destroy (line);
+ free (line);
+ line = NULL;
+ }
+ write->wf->held = line;
+
+ gsl_matrix_free (m);
+}
+\f
+/* GET. */
+
+static struct matrix_command *
+matrix_get_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_GET,
+ .get = {
+ .lexer = s->lexer,
+ .dataset = s->dataset,
+ .user = { .treatment = MGET_ERROR },
+ .system = { .treatment = MGET_ERROR },
+ }
+ };
+
+ struct matrix_get *get = &cmd->get;
+ get->dst = matrix_lvalue_parse (s);
+ if (!get->dst)
+ goto error;
+
+ while (lex_match (s->lexer, T_SLASH))
+ {
+ if (lex_match_id (s->lexer, "FILE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ fh_unref (get->file);
+ if (lex_match (s->lexer, T_ASTERISK))
+ get->file = NULL;
+ else
+ {
+ get->file = fh_parse (s->lexer, FH_REF_FILE, s->session);
+ if (!get->file)
+ goto error;
+ }
+ }
+ else if (lex_match_id (s->lexer, "ENCODING"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!lex_force_string (s->lexer))
+ goto error;
+
+ free (get->encoding);
+ get->encoding = ss_xstrdup (lex_tokss (s->lexer));
+
+ lex_get (s->lexer);
+ }
+ else if (lex_match_id (s->lexer, "VARIABLES"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ if (get->n_vars)
+ {
+ lex_sbc_only_once (s->lexer, "VARIABLES");
+ goto error;
+ }
+
+ if (!var_syntax_parse (s->lexer, &get->vars, &get->n_vars))
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "NAMES"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!lex_force_id (s->lexer))
+ goto error;
+
+ struct substring name = lex_tokss (s->lexer);
+ get->names = matrix_var_lookup (s, name);
+ if (!get->names)
+ get->names = matrix_var_create (s, name);
+ lex_get (s->lexer);
+ }
+ else if (lex_match_id (s->lexer, "MISSING"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (lex_match_id (s->lexer, "ACCEPT"))
+ get->user.treatment = MGET_ACCEPT;
+ else if (lex_match_id (s->lexer, "OMIT"))
+ get->user.treatment = MGET_OMIT;
+ else if (lex_is_number (s->lexer))
+ {
+ get->user.treatment = MGET_RECODE;
+ get->user.substitute = lex_number (s->lexer);
+ lex_get (s->lexer);
+ }
+ else
+ {
+ lex_error (s->lexer, _("Syntax error expecting ACCEPT or OMIT or "
+ "a number for MISSING."));
+ goto error;
+ }
+ }
+ else if (lex_match_id (s->lexer, "SYSMIS"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (lex_match_id (s->lexer, "OMIT"))
+ get->system.treatment = MGET_OMIT;
+ else if (lex_is_number (s->lexer))
+ {
+ get->system.treatment = MGET_RECODE;
+ get->system.substitute = lex_number (s->lexer);
+ lex_get (s->lexer);
+ }
+ else
+ {
+ lex_error (s->lexer, _("Syntax error expecting OMIT or a number "
+ "for SYSMIS."));
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "FILE", "VARIABLES", "NAMES",
+ "MISSING", "SYSMIS");
+ goto error;
+ }
+ }
+
+ if (get->user.treatment != MGET_ACCEPT)
+ get->system.treatment = MGET_ERROR;
+
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_get_execute__ (struct matrix_command *cmd, struct casereader *reader,
+ const struct dictionary *dict)
+{
+ struct matrix_get *get = &cmd->get;
+ struct variable **vars;
+ size_t n_vars = 0;
+
+ if (get->n_vars)
+ {
+ if (!var_syntax_evaluate (get->lexer, get->vars, get->n_vars, dict,
+ &vars, &n_vars, PV_NUMERIC))
+ return;
+ }
+ else
+ {
+ n_vars = dict_get_n_vars (dict);
+ vars = xnmalloc (n_vars, sizeof *vars);
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ struct variable *var = dict_get_var (dict, i);
+ if (!var_is_numeric (var))
+ {
+ msg_at (SE, cmd->location, _("Variable %s is not numeric."),
+ var_get_name (var));
+ free (vars);
+ return;
+ }
+ vars[i] = var;
+ }
+ }
+
+ if (get->names)
+ {
+ gsl_matrix *names = gsl_matrix_alloc (n_vars, 1);
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ char s[sizeof (double)];
+ double f;
+ buf_copy_str_rpad (s, sizeof s, var_get_name (vars[i]), ' ');
+ memcpy (&f, s, sizeof f);
+ gsl_matrix_set (names, i, 0, f);
+ }
+
+ gsl_matrix_free (get->names->value);
+ get->names->value = names;
+ }
+
+ size_t n_rows = 0;
+ gsl_matrix *m = gsl_matrix_alloc (4, n_vars);
+ long long int casenum = 1;
+ bool error = false;
+ for (struct ccase *c = casereader_read (reader); c;
+ c = casereader_read (reader), casenum++)
+ {
+ if (n_rows >= m->size1)
+ {
+ gsl_matrix *p = gsl_matrix_alloc (m->size1 * 2, n_vars);
+ for (size_t y = 0; y < n_rows; y++)
+ for (size_t x = 0; x < n_vars; x++)
+ gsl_matrix_set (p, y, x, gsl_matrix_get (m, y, x));
+ gsl_matrix_free (m);
+ m = p;
+ }
+
+ bool keep = true;
+ for (size_t x = 0; x < n_vars; x++)
+ {
+ const struct variable *var = vars[x];
+ double d = case_num (c, var);
+ if (d == SYSMIS)
+ {
+ if (get->system.treatment == MGET_RECODE)
+ d = get->system.substitute;
+ else if (get->system.treatment == MGET_OMIT)
+ keep = false;
+ else
+ {
+ msg_at (SE, cmd->location, _("Variable %s in case %lld "
+ "is system-missing."),
+ var_get_name (var), casenum);
+ error = true;
+ }
+ }
+ else if (var_is_num_missing (var, d) == MV_USER)
+ {
+ if (get->user.treatment == MGET_RECODE)
+ d = get->user.substitute;
+ else if (get->user.treatment == MGET_OMIT)
+ keep = false;
+ else if (get->user.treatment != MGET_ACCEPT)
+ {
+ msg_at (SE, cmd->location,
+ _("Variable %s in case %lld has user-missing "
+ "value %g."),
+ var_get_name (var), casenum, d);
+ error = true;
+ }
+ }
+ gsl_matrix_set (m, n_rows, x, d);
+ }
+ case_unref (c);
+ if (error)
+ break;
+ if (keep)
+ n_rows++;
+ }
+ if (!error)
+ {
+ m->size1 = n_rows;
+ matrix_lvalue_evaluate_and_assign (get->dst, m, cmd->location);
+ }
+ else
+ gsl_matrix_free (m);
+ free (vars);
+}
+
+static bool
+matrix_open_casereader (const struct matrix_command *cmd,
+ const char *command_name, struct file_handle *file,
+ const char *encoding, struct dataset *dataset,
+ struct casereader **readerp, struct dictionary **dictp)
+{
+ if (file)
+ {
+ *readerp = any_reader_open_and_decode (file, encoding, dictp, NULL);
+ return *readerp != NULL;
+ }
+ else
+ {
+ if (dict_get_n_vars (dataset_dict (dataset)) == 0)
+ {
+ msg_at (SE, cmd->location,
+ _("The %s command cannot read an empty active file."),
+ command_name);
+ return false;
+ }
+ *readerp = proc_open (dataset);
+ *dictp = dict_ref (dataset_dict (dataset));
+ return true;
+ }
+}
+
+static void
+matrix_close_casereader (struct file_handle *file, struct dataset *dataset,
+ struct casereader *reader, struct dictionary *dict)
+{
+ dict_unref (dict);
+ casereader_destroy (reader);
+ if (!file)
+ proc_commit (dataset);
+}
+
+static void
+matrix_get_execute (struct matrix_command *cmd)
+{
+ struct matrix_get *get = &cmd->get;
+ struct casereader *r;
+ struct dictionary *d;
+ if (matrix_open_casereader (cmd, "GET", get->file, get->encoding,
+ get->dataset, &r, &d))
+ {
+ matrix_get_execute__ (cmd, r, d);
+ matrix_close_casereader (get->file, get->dataset, r, d);
+ }
+}
+\f
+/* MSAVE. */
+
+static bool
+variables_changed (const char *keyword,
+ const struct string_array *new_vars,
+ const struct msg_location *new_vars_location,
+ const struct msg_location *new_location,
+ const struct string_array *old_vars,
+ const struct msg_location *old_vars_location,
+ const struct msg_location *old_location)
+{
+ if (new_vars->n)
+ {
+ if (!old_vars->n)
+ {
+ msg_at (SE, new_location,
+ _("%s may only be specified on MSAVE if it was specified "
+ "on the first MSAVE within MATRIX."), keyword);
+ msg_at (SN, old_location,
+ _("The first MSAVE in MATRIX did not specify %s."),
+ keyword);
+ msg_at (SN, new_vars_location,
+ _("This is the specification of %s on a later MSAVE."),
+ keyword);
+ return true;
+ }
+ if (!string_array_equal_case (old_vars, new_vars))
+ {
+ msg_at (SE, new_location,
+ _("%s must specify the same variables on each MSAVE "
+ "within a given MATRIX."), keyword);
+ msg_at (SE, old_vars_location,
+ _("This is the specification of %s on the first MSAVE."),
+ keyword);
+ msg_at (SE, new_vars_location,
+ _("This is a different specification of %s on a later MSAVE."),
+ keyword);
+ return true;
+ }
+ }
+ return false;
+}
+
+static bool
+msave_common_changed (const struct msave_common *old,
+ const struct msave_common *new)
+{
+ if (new->outfile && !fh_equal (old->outfile, new->outfile))
+ {
+ msg (SE, _("OUTFILE must name the same file on each MSAVE "
+ "within a single MATRIX command."));
+ msg_at (SN, old->outfile_location,
+ _("This is the OUTFILE on the first MSAVE command."));
+ msg_at (SN, new->outfile_location,
+ _("This is the OUTFILE on a later MSAVE command."));
+ return false;
+ }
+
+ if (!variables_changed ("VARIABLES",
+ &new->variables, new->variables_location, new->location,
+ &old->variables, old->variables_location, old->location)
+ && !variables_changed ("FNAMES",
+ &new->fnames, new->fnames_location, new->location,
+ &old->fnames, old->fnames_location, old->location)
+ && !variables_changed ("SNAMES",
+ &new->snames, new->snames_location, new->location,
+ &old->snames, old->snames_location, old->location))
+ return false;
+
+ return true;
+}
+
+static void
+msave_common_destroy (struct msave_common *common)
+{
+ if (common)
+ {
+ msg_location_destroy (common->location);
+ fh_unref (common->outfile);
+ msg_location_destroy (common->outfile_location);
+ string_array_destroy (&common->variables);
+ msg_location_destroy (common->variables_location);
+ string_array_destroy (&common->fnames);
+ msg_location_destroy (common->fnames_location);
+ string_array_destroy (&common->snames);
+ msg_location_destroy (common->snames_location);
+
+ for (size_t i = 0; i < common->n_factors; i++)
+ matrix_expr_destroy (common->factors[i]);
+ free (common->factors);
+
+ for (size_t i = 0; i < common->n_splits; i++)
+ matrix_expr_destroy (common->splits[i]);
+ free (common->splits);
+
+ dict_unref (common->dict);
+ casewriter_destroy (common->writer);
+
+ free (common);
+ }
+}
+
+static const char *
+match_rowtype (struct lexer *lexer)
+{
+ static const char *rowtypes[] = {
+ "COV", "CORR", "MEAN", "STDDEV", "N", "COUNT"
+ };
+ size_t n_rowtypes = sizeof rowtypes / sizeof *rowtypes;
+
+ for (size_t i = 0; i < n_rowtypes; i++)
+ if (lex_match_id (lexer, rowtypes[i]))
+ return rowtypes[i];
+
+ lex_error_expecting_array (lexer, rowtypes, n_rowtypes);
+ return NULL;
+}
+
+static bool
+parse_var_names (struct lexer *lexer, struct string_array *sa,
+ struct msg_location **locationp)
+{
+ lex_match (lexer, T_EQUALS);
+
+ string_array_clear (sa);
+ msg_location_destroy (*locationp);
+ *locationp = NULL;
+
+ struct dictionary *dict = dict_create (get_default_encoding ());
+ char **names;
+ size_t n_names;
+ int start_ofs = lex_ofs (lexer);
+ bool ok = parse_DATA_LIST_vars (lexer, dict, &names, &n_names,
+ PV_NO_DUPLICATE | PV_NO_SCRATCH);
+ int end_ofs = lex_ofs (lexer) - 1;
+ dict_unref (dict);
+
+ if (ok)
+ {
+ for (size_t i = 0; i < n_names; i++)
+ if (ss_equals_case (ss_cstr (names[i]), ss_cstr ("ROWTYPE_"))
+ || ss_equals_case (ss_cstr (names[i]), ss_cstr ("VARNAME_")))
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("Variable name %s is reserved."), names[i]);
+ for (size_t j = 0; j < n_names; j++)
+ free (names[i]);
+ free (names);
+ return false;
+ }
+
+ sa->strings = names;
+ sa->n = sa->allocated = n_names;
+ *locationp = lex_ofs_location (lexer, start_ofs, end_ofs);
+ }
+ return ok;
+}
+
+static struct matrix_command *
+matrix_msave_parse (struct matrix_state *s)
+{
+ int start_ofs = lex_ofs (s->lexer);
+
+ struct msave_common *common = xmalloc (sizeof *common);
+ *common = (struct msave_common) { .outfile = NULL };
+
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) { .type = MCMD_MSAVE, .msave = { .expr = NULL } };
+
+ struct matrix_expr *splits = NULL;
+ struct matrix_expr *factors = NULL;
+
+ struct matrix_msave *msave = &cmd->msave;
+ msave->expr = matrix_parse_exp (s);
+ if (!msave->expr)
+ goto error;
+
+ while (lex_match (s->lexer, T_SLASH))
+ {
+ if (lex_match_id (s->lexer, "TYPE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ msave->rowtype = match_rowtype (s->lexer);
+ if (!msave->rowtype)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "OUTFILE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ fh_unref (common->outfile);
+ int start_ofs = lex_ofs (s->lexer);
+ common->outfile = fh_parse (s->lexer, FH_REF_FILE, NULL);
+ if (!common->outfile)
+ goto error;
+ msg_location_destroy (common->outfile_location);
+ common->outfile_location = lex_ofs_location (s->lexer, start_ofs,
+ lex_ofs (s->lexer) - 1);
+ }
+ else if (lex_match_id (s->lexer, "VARIABLES"))
+ {
+ if (!parse_var_names (s->lexer, &common->variables,
+ &common->variables_location))
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "FNAMES"))
+ {
+ if (!parse_var_names (s->lexer, &common->fnames,
+ &common->fnames_location))
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "SNAMES"))
+ {
+ if (!parse_var_names (s->lexer, &common->snames,
+ &common->snames_location))
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "SPLIT"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ matrix_expr_destroy (splits);
+ splits = matrix_parse_exp (s);
+ if (!splits)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "FACTOR"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ matrix_expr_destroy (factors);
+ factors = matrix_parse_exp (s);
+ if (!factors)
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "TYPE", "OUTFILE", "VARIABLES",
+ "FNAMES", "SNAMES", "SPLIT", "FACTOR");
+ goto error;
+ }
+ }
+ if (!msave->rowtype)
+ {
+ lex_sbc_missing (s->lexer, "TYPE");
+ goto error;
+ }
+
+ if (!s->msave_common)
+ {
+ if (common->fnames.n && !factors)
+ {
+ msg_at (SE, common->fnames_location, _("FNAMES requires FACTOR."));
+ goto error;
+ }
+ if (common->snames.n && !splits)
+ {
+ msg_at (SE, common->snames_location, _("SNAMES requires SPLIT."));
+ goto error;
+ }
+ if (!common->outfile)
+ {
+ lex_sbc_missing (s->lexer, "OUTFILE");
+ goto error;
+ }
+ common->location = lex_ofs_location (s->lexer, start_ofs,
+ lex_ofs (s->lexer));
+ msg_location_remove_columns (common->location);
+ s->msave_common = common;
+ }
+ else
+ {
+ if (msave_common_changed (s->msave_common, common))
+ goto error;
+ msave_common_destroy (common);
+ }
+ msave->common = s->msave_common;
+
+ struct msave_common *c = s->msave_common;
+ if (factors)
+ {
+ if (c->n_factors >= c->allocated_factors)
+ c->factors = x2nrealloc (c->factors, &c->allocated_factors,
+ sizeof *c->factors);
+ c->factors[c->n_factors++] = factors;
+ }
+ if (c->n_factors > 0)
+ msave->factors = c->factors[c->n_factors - 1];
+
+ if (splits)
+ {
+ if (c->n_splits >= c->allocated_splits)
+ c->splits = x2nrealloc (c->splits, &c->allocated_splits,
+ sizeof *c->splits);
+ c->splits[c->n_splits++] = splits;
+ }
+ if (c->n_splits > 0)
+ msave->splits = c->splits[c->n_splits - 1];
+
+ return cmd;
+
+error:
+ matrix_expr_destroy (splits);
+ matrix_expr_destroy (factors);
+ msave_common_destroy (common);
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static gsl_vector *
+matrix_expr_evaluate_vector (const struct matrix_expr *e, const char *name)
+{
+ gsl_matrix *m = matrix_expr_evaluate (e);
+ if (!m)
+ return NULL;
+
+ if (!is_vector (m))
+ {
+ msg_at (SE, matrix_expr_location (e),
+ _("%s expression must evaluate to vector, "
+ "not a %zu×%zu matrix."),
+ name, m->size1, m->size2);
+ gsl_matrix_free (m);
+ return NULL;
+ }
+
+ return matrix_to_vector (m);
+}
+
+static const char *
+msave_add_vars (struct dictionary *d, const struct string_array *vars)
+{
+ for (size_t i = 0; i < vars->n; i++)
+ if (!dict_create_var (d, vars->strings[i], 0))
+ return vars->strings[i];
+ return NULL;
+}
+
+static struct dictionary *
+msave_create_dict (const struct msave_common *common)
+{
+ struct dictionary *dict = dict_create (get_default_encoding ());
+
+ const char *dup_split = msave_add_vars (dict, &common->snames);
+ if (dup_split)
+ {
+ /* Should not be possible because the parser ensures that the names are
+ unique. */
+ NOT_REACHED ();
+ }
+
+ dict_create_var_assert (dict, "ROWTYPE_", 8);
+
+ const char *dup_factor = msave_add_vars (dict, &common->fnames);
+ if (dup_factor)
+ {
+ msg_at (SE, common->fnames_location,
+ _("Duplicate or invalid FACTOR variable name %s."),
+ dup_factor);
+ goto error;
+ }
+
+ dict_create_var_assert (dict, "VARNAME_", 8);
+
+ const char *dup_var = msave_add_vars (dict, &common->variables);
+ if (dup_var)
+ {
+ msg_at (SE, common->variables_location,
+ _("Duplicate or invalid variable name %s."),
+ dup_var);
+ goto error;
+ }
+
+ return dict;
+
+error:
+ dict_unref (dict);
+ return NULL;
+}
+
+static void
+matrix_msave_execute (struct matrix_command *cmd)
+{
+ struct matrix_msave *msave = &cmd->msave;
+ struct msave_common *common = msave->common;
+ gsl_matrix *m = NULL;
+ gsl_vector *factors = NULL;
+ gsl_vector *splits = NULL;
+
+ m = matrix_expr_evaluate (msave->expr);
+ if (!m)
+ goto error;
+
+ if (!common->variables.n)
+ for (size_t i = 0; i < m->size2; i++)
+ string_array_append_nocopy (&common->variables,
+ xasprintf ("COL%zu", i + 1));
+ else if (m->size2 != common->variables.n)
+ {
+ msg_at (SE, matrix_expr_location (msave->expr),
+ _("Matrix on MSAVE has %zu columns but there are %zu variables."),
+ m->size2, common->variables.n);
+ goto error;
+ }
+
+ if (msave->factors)
+ {
+ factors = matrix_expr_evaluate_vector (msave->factors, "FACTOR");
+ if (!factors)
+ goto error;
+
+ if (!common->fnames.n)
+ for (size_t i = 0; i < factors->size; i++)
+ string_array_append_nocopy (&common->fnames,
+ xasprintf ("FAC%zu", i + 1));
+ else if (factors->size != common->fnames.n)
+ {
+ msg_at (SE, matrix_expr_location (msave->factors),
+ _("There are %zu factor variables, "
+ "but %zu factor values were supplied."),
+ common->fnames.n, factors->size);
+ goto error;
+ }
+ }
+
+ if (msave->splits)
+ {
+ splits = matrix_expr_evaluate_vector (msave->splits, "SPLIT");
+ if (!splits)
+ goto error;
+
+ if (!common->snames.n)
+ for (size_t i = 0; i < splits->size; i++)
+ string_array_append_nocopy (&common->snames,
+ xasprintf ("SPL%zu", i + 1));
+ else if (splits->size != common->snames.n)
+ {
+ msg_at (SE, matrix_expr_location (msave->splits),
+ _("There are %zu split variables, "
+ "but %zu split values were supplied."),
+ common->snames.n, splits->size);
+ goto error;
+ }
+ }
+
+ if (!common->writer)
+ {
+ struct dictionary *dict = msave_create_dict (common);
+ if (!dict)
+ goto error;
+
+ common->writer = any_writer_open (common->outfile, dict);
+ if (!common->writer)
+ {
+ dict_unref (dict);
+ goto error;
+ }
+
+ common->dict = dict;
+ }
+
+ bool matrix = (!strcmp (msave->rowtype, "COV")
+ || !strcmp (msave->rowtype, "CORR"));
+ for (size_t y = 0; y < m->size1; y++)
+ {
+ struct ccase *c = case_create (dict_get_proto (common->dict));
+ size_t idx = 0;
+
+ /* Split variables */
+ if (splits)
+ for (size_t i = 0; i < splits->size; i++)
+ case_data_rw_idx (c, idx++)->f = gsl_vector_get (splits, i);
+
+ /* ROWTYPE_. */
+ buf_copy_str_rpad (CHAR_CAST (char *, case_data_rw_idx (c, idx++)->s), 8,
+ msave->rowtype, ' ');
+
+ /* Factors. */
+ if (factors)
+ for (size_t i = 0; i < factors->size; i++)
+ *case_num_rw_idx (c, idx++) = gsl_vector_get (factors, i);
+
+ /* VARNAME_. */
+ const char *varname_ = (matrix && y < common->variables.n
+ ? common->variables.strings[y]
+ : "");
+ buf_copy_str_rpad (CHAR_CAST (char *, case_data_rw_idx (c, idx++)->s), 8,
+ varname_, ' ');
+
+ /* Continuous variables. */
+ for (size_t x = 0; x < m->size2; x++)
+ case_data_rw_idx (c, idx++)->f = gsl_matrix_get (m, y, x);
+ casewriter_write (common->writer, c);
+ }
+
+error:
+ gsl_matrix_free (m);
+ gsl_vector_free (factors);
+ gsl_vector_free (splits);
+}
+\f
+/* MGET. */
+
+static struct matrix_command *
+matrix_mget_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_MGET,
+ .mget = {
+ .state = s,
+ .rowtypes = STRINGI_SET_INITIALIZER (cmd->mget.rowtypes),
+ },
+ };
+
+ struct matrix_mget *mget = &cmd->mget;
+
+ lex_match (s->lexer, T_SLASH);
+ while (lex_token (s->lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (s->lexer, "FILE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+
+ fh_unref (mget->file);
+ mget->file = fh_parse (s->lexer, FH_REF_FILE, s->session);
+ if (!mget->file)
+ goto error;
+ }
+ else if (lex_match_id (s->lexer, "ENCODING"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ if (!lex_force_string (s->lexer))
+ goto error;
+
+ free (mget->encoding);
+ mget->encoding = ss_xstrdup (lex_tokss (s->lexer));
+
+ lex_get (s->lexer);
+ }
+ else if (lex_match_id (s->lexer, "TYPE"))
+ {
+ lex_match (s->lexer, T_EQUALS);
+ while (lex_token (s->lexer) != T_SLASH
+ && lex_token (s->lexer) != T_ENDCMD)
+ {
+ const char *rowtype = match_rowtype (s->lexer);
+ if (!rowtype)
+ goto error;
+
+ stringi_set_insert (&mget->rowtypes, rowtype);
+ }
+ }
+ else
+ {
+ lex_error_expecting (s->lexer, "FILE", "TYPE");
+ goto error;
+ }
+ lex_match (s->lexer, T_SLASH);
+ }
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static const struct variable *
+get_a8_var (const struct msg_location *loc,
+ const struct dictionary *d, const char *name)
+{
+ const struct variable *v = dict_lookup_var (d, name);
+ if (!v)
+ {
+ msg_at (SE, loc, _("Matrix data file lacks %s variable."), name);
+ return NULL;
+ }
+ if (var_get_width (v) != 8)
+ {
+ msg_at (SE, loc, _("%s variable in matrix data file must be "
+ "8-byte string, but it has width %d."),
+ name, var_get_width (v));
+ return NULL;
+ }
+ return v;
+}
+
+static bool
+var_changed (const struct ccase *ca, const struct ccase *cb,
+ const struct variable *var)
+{
+ return (ca && cb
+ ? !value_equal (case_data (ca, var), case_data (cb, var),
+ var_get_width (var))
+ : ca || cb);
+}
+
+static bool
+vars_changed (const struct ccase *ca, const struct ccase *cb,
+ const struct dictionary *d,
+ size_t first_var, size_t n_vars)
+{
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ const struct variable *v = dict_get_var (d, first_var + i);
+ if (var_changed (ca, cb, v))
+ return true;
+ }
+ return false;
+}
+
+static bool
+vars_all_missing (const struct ccase *c, const struct dictionary *d,
+ size_t first_var, size_t n_vars)
+{
+ for (size_t i = 0; i < n_vars; i++)
+ if (case_num (c, dict_get_var (d, first_var + i)) != SYSMIS)
+ return false;
+ return true;
+}
+
+static void
+matrix_mget_commit_var (struct ccase **rows, size_t n_rows,
+ const struct dictionary *d,
+ const struct variable *rowtype_var,
+ const struct stringi_set *accepted_rowtypes,
+ struct matrix_state *s,
+ size_t ss, size_t sn, size_t si,
+ size_t fs, size_t fn, size_t fi,
+ size_t cs, size_t cn,
+ struct pivot_table *pt,
+ struct pivot_dimension *var_dimension)
+{
+ if (!n_rows)
+ goto exit;
+
+ /* Is this a matrix for pooled data, either where there are no factor
+ variables or the factor variables are missing? */
+ bool pooled = !fn || vars_all_missing (rows[0], d, fs, fn);
+
+ struct substring rowtype = case_ss (rows[0], rowtype_var);
+ ss_rtrim (&rowtype, ss_cstr (" "));
+ if (!stringi_set_is_empty (accepted_rowtypes)
+ && !stringi_set_contains_len (accepted_rowtypes,
+ rowtype.string, rowtype.length))
+ goto exit;
+
+ const char *prefix = (ss_equals_case (rowtype, ss_cstr ("COV")) ? "CV"
+ : ss_equals_case (rowtype, ss_cstr ("CORR")) ? "CR"
+ : ss_equals_case (rowtype, ss_cstr ("MEAN")) ? "MN"
+ : ss_equals_case (rowtype, ss_cstr ("STDDEV")) ? "SD"
+ : ss_equals_case (rowtype, ss_cstr ("N")) ? "NC"
+ : ss_equals_case (rowtype, ss_cstr ("COUNT")) ? "CN"
+ : NULL);
+ if (!prefix)
+ {
+ msg (SE, _("Matrix data file contains unknown ROWTYPE_ \"%.*s\"."),
+ (int) rowtype.length, rowtype.string);
+ goto exit;
+ }
+
+ struct string name = DS_EMPTY_INITIALIZER;
+ ds_put_cstr (&name, prefix);
+ if (!pooled)
+ ds_put_format (&name, "F%zu", fi);
+ if (si > 0)
+ ds_put_format (&name, "S%zu", si);
+
+ struct matrix_var *mv = matrix_var_lookup (s, ds_ss (&name));
+ if (!mv)
+ mv = matrix_var_create (s, ds_ss (&name));
+ else if (mv->value)
+ {
+ msg (SW, _("Matrix data file contains variable with existing name %s."),
+ ds_cstr (&name));
+ goto exit_free_name;
+ }
+
+ gsl_matrix *m = gsl_matrix_alloc (n_rows, cn);
+ size_t n_missing = 0;
+ for (size_t y = 0; y < n_rows; y++)
+ {
+ for (size_t x = 0; x < cn; x++)
+ {
+ struct variable *var = dict_get_var (d, cs + x);
+ double value = case_num (rows[y], var);
+ if (var_is_num_missing (var, value))
+ {
+ n_missing++;
+ value = 0.0;
+ }
+ gsl_matrix_set (m, y, x, value);
+ }
+ }
+
+ int var_index = pivot_category_create_leaf (
+ var_dimension->root, pivot_value_new_user_text (ds_cstr (&name), SIZE_MAX));
+ double values[] = { n_rows, cn };
+ for (size_t j = 0; j < sn; j++)
+ {
+ struct variable *var = dict_get_var (d, ss + j);
+ const union value *value = case_data (rows[0], var);
+ pivot_table_put2 (pt, j, var_index,
+ pivot_value_new_var_value (var, value));
+ }
+ for (size_t j = 0; j < fn; j++)
+ {
+ struct variable *var = dict_get_var (d, fs + j);
+ const union value sysmis = { .f = SYSMIS };
+ const union value *value = pooled ? &sysmis : case_data (rows[0], var);
+ pivot_table_put2 (pt, j + sn, var_index,
+ pivot_value_new_var_value (var, value));
+ }
+ for (size_t j = 0; j < sizeof values / sizeof *values; j++)
+ pivot_table_put2 (pt, j + sn + fn, var_index,
+ pivot_value_new_integer (values[j]));
+
+ if (n_missing)
+ msg (SE, ngettext ("Matrix data file variable %s contains a missing "
+ "value, which was treated as zero.",
+ "Matrix data file variable %s contains %zu missing "
+ "values, which were treated as zero.", n_missing),
+ ds_cstr (&name), n_missing);
+ mv->value = m;
+
+exit_free_name:
+ ds_destroy (&name);
+
+exit:
+ for (size_t y = 0; y < n_rows; y++)
+ case_unref (rows[y]);
+}
+
+static void
+matrix_mget_execute__ (struct matrix_command *cmd, struct casereader *r,
+ const struct dictionary *d)
+{
+ struct matrix_mget *mget = &cmd->mget;
+ const struct msg_location *loc = cmd->location;
+ const struct variable *rowtype_ = get_a8_var (loc, d, "ROWTYPE_");
+ const struct variable *varname_ = get_a8_var (loc, d, "VARNAME_");
+ if (!rowtype_ || !varname_)
+ return;
+
+ if (var_get_dict_index (rowtype_) >= var_get_dict_index (varname_))
+ {
+ msg_at (SE, loc,
+ _("ROWTYPE_ must precede VARNAME_ in matrix data file."));
+ return;
+ }
+ if (var_get_dict_index (varname_) + 1 >= dict_get_n_vars (d))
+ {
+ msg_at (SE, loc, _("Matrix data file contains no continuous variables."));
+ return;
+ }
+
+ for (size_t i = 0; i < dict_get_n_vars (d); i++)
+ {
+ const struct variable *v = dict_get_var (d, i);
+ if (v != rowtype_ && v != varname_ && var_get_width (v) != 0)
+ {
+ msg_at (SE, loc,
+ _("Matrix data file contains unexpected string variable %s."),
+ var_get_name (v));
+ return;
+ }
+ }
+
+ /* SPLIT variables. */
+ size_t ss = 0;
+ size_t sn = var_get_dict_index (rowtype_);
+ struct ccase *sc = NULL;
+ size_t si = 0;
+
+ /* FACTOR variables. */
+ size_t fs = var_get_dict_index (rowtype_) + 1;
+ size_t fn = var_get_dict_index (varname_) - var_get_dict_index (rowtype_) - 1;
+ struct ccase *fc = NULL;
+ size_t fi = 0;
+
+ /* Continuous variables. */
+ size_t cs = var_get_dict_index (varname_) + 1;
+ size_t cn = dict_get_n_vars (d) - cs;
+ struct ccase *cc = NULL;
+
+ /* Pivot table. */
+ struct pivot_table *pt = pivot_table_create (
+ N_("Matrix Variables Created by MGET"));
+ struct pivot_dimension *attr_dimension = pivot_dimension_create (
+ pt, PIVOT_AXIS_COLUMN, N_("Attribute"));
+ struct pivot_dimension *var_dimension = pivot_dimension_create (
+ pt, PIVOT_AXIS_ROW, N_("Variable"));
+ if (sn > 0)
+ {
+ struct pivot_category *splits = pivot_category_create_group (
+ attr_dimension->root, N_("Split Values"));
+ for (size_t i = 0; i < sn; i++)
+ pivot_category_create_leaf (splits, pivot_value_new_variable (
+ dict_get_var (d, ss + i)));
+ }
+ if (fn > 0)
+ {
+ struct pivot_category *factors = pivot_category_create_group (
+ attr_dimension->root, N_("Factors"));
+ for (size_t i = 0; i < fn; i++)
+ pivot_category_create_leaf (factors, pivot_value_new_variable (
+ dict_get_var (d, fs + i)));
+ }
+ pivot_category_create_group (attr_dimension->root, N_("Dimensions"),
+ N_("Rows"), N_("Columns"));
+
+ /* Matrix. */
+ struct ccase **rows = NULL;
+ size_t allocated_rows = 0;
+ size_t n_rows = 0;
+
+ struct ccase *c;
+ while ((c = casereader_read (r)) != NULL)
+ {
+ bool row_has_factors = fn && !vars_all_missing (c, d, fs, fn);
+
+ enum
+ {
+ SPLITS_CHANGED,
+ FACTORS_CHANGED,
+ ROWTYPE_CHANGED,
+ NOTHING_CHANGED
+ }
+ change
+ = (sn && (!sc || vars_changed (sc, c, d, ss, sn)) ? SPLITS_CHANGED
+ : fn && (!fc || vars_changed (fc, c, d, fs, fn)) ? FACTORS_CHANGED
+ : !cc || var_changed (cc, c, rowtype_) ? ROWTYPE_CHANGED
+ : NOTHING_CHANGED);
+
+ if (change != NOTHING_CHANGED)
+ {
+ matrix_mget_commit_var (rows, n_rows, d,
+ rowtype_, &mget->rowtypes,
+ mget->state,
+ ss, sn, si,
+ fs, fn, fi,
+ cs, cn,
+ pt, var_dimension);
+ n_rows = 0;
+ case_unref (cc);
+ cc = case_ref (c);
+ }
+
+ if (n_rows >= allocated_rows)
+ rows = x2nrealloc (rows, &allocated_rows, sizeof *rows);
+ rows[n_rows++] = c;
+
+ if (change == SPLITS_CHANGED)
+ {
+ si++;
+ case_unref (sc);
+ sc = case_ref (c);
+
+ /* Reset the factor number, if there are factors. */
+ if (fn)
+ {
+ fi = 0;
+ if (row_has_factors)
+ fi++;
+ case_unref (fc);
+ fc = case_ref (c);
+ }
+ }
+ else if (change == FACTORS_CHANGED)
+ {
+ if (row_has_factors)
+ fi++;
+ case_unref (fc);
+ fc = case_ref (c);
+ }
+ }
+ matrix_mget_commit_var (rows, n_rows, d,
+ rowtype_, &mget->rowtypes,
+ mget->state,
+ ss, sn, si,
+ fs, fn, fi,
+ cs, cn,
+ pt, var_dimension);
+ free (rows);
+
+ case_unref (sc);
+ case_unref (fc);
+ case_unref (cc);
+
+ if (var_dimension->n_leaves)
+ pivot_table_submit (pt);
+ else
+ pivot_table_unref (pt);
+}
+
+static void
+matrix_mget_execute (struct matrix_command *cmd)
+{
+ struct matrix_mget *mget = &cmd->mget;
+ struct casereader *r;
+ struct dictionary *d;
+ if (matrix_open_casereader (cmd, "MGET", mget->file, mget->encoding,
+ mget->state->dataset, &r, &d))
+ {
+ matrix_mget_execute__ (cmd, r, d);
+ matrix_close_casereader (mget->file, mget->state->dataset, r, d);
+ }
+}
+\f
+/* CALL EIGEN. */
+
+static bool
+matrix_parse_dst_var (struct matrix_state *s, struct matrix_var **varp)
+{
+ if (!lex_force_id (s->lexer))
+ return false;
+
+ *varp = matrix_var_lookup (s, lex_tokss (s->lexer));
+ if (!*varp)
+ *varp = matrix_var_create (s, lex_tokss (s->lexer));
+ lex_get (s->lexer);
+ return true;
+}
+
+static struct matrix_command *
+matrix_eigen_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_EIGEN,
+ .eigen = { .expr = NULL }
+ };
+
+ struct matrix_eigen *eigen = &cmd->eigen;
+ if (!lex_force_match (s->lexer, T_LPAREN))
+ goto error;
+ eigen->expr = matrix_expr_parse (s);
+ if (!eigen->expr
+ || !lex_force_match (s->lexer, T_COMMA)
+ || !matrix_parse_dst_var (s, &eigen->evec)
+ || !lex_force_match (s->lexer, T_COMMA)
+ || !matrix_parse_dst_var (s, &eigen->eval)
+ || !lex_force_match (s->lexer, T_RPAREN))
+ goto error;
+
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_eigen_execute (struct matrix_command *cmd)
+{
+ struct matrix_eigen *eigen = &cmd->eigen;
+ gsl_matrix *A = matrix_expr_evaluate (eigen->expr);
+ if (!A)
+ return;
+ if (!is_symmetric (A))
+ {
+ msg_at (SE, cmd->location, _("Argument of EIGEN must be symmetric."));
+ gsl_matrix_free (A);
+ return;
+ }
+
+ gsl_eigen_symmv_workspace *w = gsl_eigen_symmv_alloc (A->size1);
+ gsl_matrix *eval = gsl_matrix_alloc (A->size1, 1);
+ gsl_vector v_eval = to_vector (eval);
+ gsl_matrix *evec = gsl_matrix_alloc (A->size1, A->size2);
+ gsl_eigen_symmv (A, &v_eval, evec, w);
+ gsl_eigen_symmv_free (w);
+
+ gsl_eigen_symmv_sort (&v_eval, evec, GSL_EIGEN_SORT_VAL_DESC);
+
+ gsl_matrix_free (A);
+
+ gsl_matrix_free (eigen->eval->value);
+ eigen->eval->value = eval;
+
+ gsl_matrix_free (eigen->evec->value);
+ eigen->evec->value = evec;
+}
+\f
+/* CALL SETDIAG. */
+
+static struct matrix_command *
+matrix_setdiag_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_SETDIAG,
+ .setdiag = { .dst = NULL }
+ };
+
+ struct matrix_setdiag *setdiag = &cmd->setdiag;
+ if (!lex_force_match (s->lexer, T_LPAREN) || !lex_force_id (s->lexer))
+ goto error;
+
+ setdiag->dst = matrix_var_lookup (s, lex_tokss (s->lexer));
+ if (!setdiag->dst)
+ {
+ lex_error (s->lexer, _("Unknown variable %s."), lex_tokcstr (s->lexer));
+ goto error;
+ }
+ lex_get (s->lexer);
+
+ if (!lex_force_match (s->lexer, T_COMMA))
+ goto error;
+
+ setdiag->expr = matrix_expr_parse (s);
+ if (!setdiag->expr)
+ goto error;
+
+ if (!lex_force_match (s->lexer, T_RPAREN))
+ goto error;
+
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_setdiag_execute (struct matrix_command *cmd)
+{
+ struct matrix_setdiag *setdiag = &cmd->setdiag;
+ gsl_matrix *dst = setdiag->dst->value;
+ if (!dst)
+ {
+ msg_at (SE, cmd->location,
+ _("SETDIAG destination matrix %s is uninitialized."),
+ setdiag->dst->name);
+ return;
+ }
+
+ gsl_matrix *src = matrix_expr_evaluate (setdiag->expr);
+ if (!src)
+ return;
+
+ size_t n = MIN (dst->size1, dst->size2);
+ if (is_scalar (src))
+ {
+ double d = to_scalar (src);
+ for (size_t i = 0; i < n; i++)
+ gsl_matrix_set (dst, i, i, d);
+ }
+ else if (is_vector (src))
+ {
+ gsl_vector v = to_vector (src);
+ for (size_t i = 0; i < n && i < v.size; i++)
+ gsl_matrix_set (dst, i, i, gsl_vector_get (&v, i));
+ }
+ else
+ msg_at (SE, matrix_expr_location (setdiag->expr),
+ _("SETDIAG argument 2 must be a scalar or a vector, "
+ "not a %zu×%zu matrix."),
+ src->size1, src->size2);
+ gsl_matrix_free (src);
+}
+\f
+/* CALL SVD. */
+
+static struct matrix_command *
+matrix_svd_parse (struct matrix_state *s)
+{
+ struct matrix_command *cmd = xmalloc (sizeof *cmd);
+ *cmd = (struct matrix_command) {
+ .type = MCMD_SVD,
+ .svd = { .expr = NULL }
+ };
+
+ struct matrix_svd *svd = &cmd->svd;
+ if (!lex_force_match (s->lexer, T_LPAREN))
+ goto error;
+ svd->expr = matrix_expr_parse (s);
+ if (!svd->expr
+ || !lex_force_match (s->lexer, T_COMMA)
+ || !matrix_parse_dst_var (s, &svd->u)
+ || !lex_force_match (s->lexer, T_COMMA)
+ || !matrix_parse_dst_var (s, &svd->s)
+ || !lex_force_match (s->lexer, T_COMMA)
+ || !matrix_parse_dst_var (s, &svd->v)
+ || !lex_force_match (s->lexer, T_RPAREN))
+ goto error;
+
+ return cmd;
+
+error:
+ matrix_command_destroy (cmd);
+ return NULL;
+}
+
+static void
+matrix_svd_execute (struct matrix_svd *svd)
+{
+ gsl_matrix *m = matrix_expr_evaluate (svd->expr);
+ if (!m)
+ return;
+
+ if (m->size1 >= m->size2)
+ {
+ gsl_matrix *A = m;
+ gsl_matrix *V = gsl_matrix_alloc (A->size2, A->size2);
+ gsl_matrix *S = gsl_matrix_calloc (A->size2, A->size2);
+ gsl_vector Sv = gsl_matrix_diagonal (S).vector;
+ gsl_vector *work = gsl_vector_alloc (A->size2);
+ gsl_linalg_SV_decomp (A, V, &Sv, work);
+ gsl_vector_free (work);
+
+ matrix_var_set (svd->u, A);
+ matrix_var_set (svd->s, S);
+ matrix_var_set (svd->v, V);
+ }
+ else
+ {
+ gsl_matrix *At = gsl_matrix_alloc (m->size2, m->size1);
+ gsl_matrix_transpose_memcpy (At, m);
+ gsl_matrix_free (m);
+
+ gsl_matrix *Vt = gsl_matrix_alloc (At->size2, At->size2);
+ gsl_matrix *St = gsl_matrix_calloc (At->size2, At->size2);
+ gsl_vector Stv = gsl_matrix_diagonal (St).vector;
+ gsl_vector *work = gsl_vector_alloc (At->size2);
+ gsl_linalg_SV_decomp (At, Vt, &Stv, work);
+ gsl_vector_free (work);
+
+ matrix_var_set (svd->v, At);
+ matrix_var_set (svd->s, St);
+ matrix_var_set (svd->u, Vt);
+ }
+}
+\f
+/* The main MATRIX command logic. */
+
+static bool
+matrix_command_execute (struct matrix_command *cmd)
+{
+ switch (cmd->type)
+ {
+ case MCMD_COMPUTE:
+ matrix_compute_execute (cmd);
+ break;
+
+ case MCMD_PRINT:
+ matrix_print_execute (&cmd->print);
+ break;
+
+ case MCMD_DO_IF:
+ return matrix_do_if_execute (&cmd->do_if);
+
+ case MCMD_LOOP:
+ matrix_loop_execute (&cmd->loop);
+ break;
+
+ case MCMD_BREAK:
+ return false;
+
+ case MCMD_DISPLAY:
+ matrix_display_execute (&cmd->display);
+ break;
+
+ case MCMD_RELEASE:
+ matrix_release_execute (&cmd->release);
+ break;
+
+ case MCMD_SAVE:
+ matrix_save_execute (cmd);
+ break;
+
+ case MCMD_READ:
+ matrix_read_execute (cmd);
+ break;
+
+ case MCMD_WRITE:
+ matrix_write_execute (&cmd->write);
+ break;
+
+ case MCMD_GET:
+ matrix_get_execute (cmd);
+ break;
+
+ case MCMD_MSAVE:
+ matrix_msave_execute (cmd);
+ break;
+
+ case MCMD_MGET:
+ matrix_mget_execute (cmd);
+ break;
+
+ case MCMD_EIGEN:
+ matrix_eigen_execute (cmd);
+ break;
+
+ case MCMD_SETDIAG:
+ matrix_setdiag_execute (cmd);
+ break;
+
+ case MCMD_SVD:
+ matrix_svd_execute (&cmd->svd);
+ break;
+ }
+
+ return true;
+}
+
+static void
+matrix_command_destroy (struct matrix_command *cmd)
+{
+ if (!cmd)
+ return;
+
+ msg_location_destroy (cmd->location);
+
+ switch (cmd->type)
+ {
+ case MCMD_COMPUTE:
+ matrix_lvalue_destroy (cmd->compute.lvalue);
+ matrix_expr_destroy (cmd->compute.rvalue);
+ break;
+
+ case MCMD_PRINT:
+ matrix_expr_destroy (cmd->print.expression);
+ free (cmd->print.title);
+ print_labels_destroy (cmd->print.rlabels);
+ print_labels_destroy (cmd->print.clabels);
+ break;
+
+ case MCMD_DO_IF:
+ for (size_t i = 0; i < cmd->do_if.n_clauses; i++)
+ {
+ matrix_expr_destroy (cmd->do_if.clauses[i].condition);
+ matrix_commands_uninit (&cmd->do_if.clauses[i].commands);
+ }
+ free (cmd->do_if.clauses);
+ break;
+
+ case MCMD_LOOP:
+ matrix_expr_destroy (cmd->loop.start);
+ matrix_expr_destroy (cmd->loop.end);
+ matrix_expr_destroy (cmd->loop.increment);
+ matrix_expr_destroy (cmd->loop.top_condition);
+ matrix_expr_destroy (cmd->loop.bottom_condition);
+ matrix_commands_uninit (&cmd->loop.commands);
+ break;
+
+ case MCMD_BREAK:
+ break;
+
+ case MCMD_DISPLAY:
+ break;
+
+ case MCMD_RELEASE:
+ free (cmd->release.vars);
+ break;
+
+ case MCMD_SAVE:
+ matrix_expr_destroy (cmd->save.expression);
+ break;
+
+ case MCMD_READ:
+ matrix_lvalue_destroy (cmd->read.dst);
+ matrix_expr_destroy (cmd->read.size);
+ break;
+
+ case MCMD_WRITE:
+ matrix_expr_destroy (cmd->write.expression);
+ free (cmd->write.format);
+ break;
+
+ case MCMD_GET:
+ matrix_lvalue_destroy (cmd->get.dst);
+ fh_unref (cmd->get.file);
+ free (cmd->get.encoding);
+ var_syntax_destroy (cmd->get.vars, cmd->get.n_vars);
+ break;
+
+ case MCMD_MSAVE:
+ matrix_expr_destroy (cmd->msave.expr);
+ break;
+
+ case MCMD_MGET:
+ fh_unref (cmd->mget.file);
+ stringi_set_destroy (&cmd->mget.rowtypes);
+ break;
+
+ case MCMD_EIGEN:
+ matrix_expr_destroy (cmd->eigen.expr);
+ break;
+
+ case MCMD_SETDIAG:
+ matrix_expr_destroy (cmd->setdiag.expr);
+ break;
+
+ case MCMD_SVD:
+ matrix_expr_destroy (cmd->svd.expr);
+ break;
+ }
+ free (cmd);
+}
+
+static bool
+matrix_commands_parse (struct matrix_state *s, struct matrix_commands *c,
+ const char *command_name,
+ const char *stop1, const char *stop2)
+{
+ lex_end_of_command (s->lexer);
+ lex_discard_rest_of_command (s->lexer);
+
+ size_t allocated = 0;
+ for (;;)
+ {
+ while (lex_token (s->lexer) == T_ENDCMD)
+ lex_get (s->lexer);
+
+ if (lex_at_phrase (s->lexer, stop1)
+ || (stop2 && lex_at_phrase (s->lexer, stop2)))
+ return true;
+
+ if (lex_at_phrase (s->lexer, "END MATRIX"))
+ {
+ lex_next_error (s->lexer, 0, 1,
+ _("Premature END MATRIX within %s."), command_name);
+ return false;
+ }
+
+ struct matrix_command *cmd = matrix_command_parse (s);
+ if (!cmd)
+ return false;
+
+ if (c->n >= allocated)
+ c->commands = x2nrealloc (c->commands, &allocated, sizeof *c->commands);
+ c->commands[c->n++] = cmd;
+ }
+}
+
+static void
+matrix_commands_uninit (struct matrix_commands *cmds)
+{
+ for (size_t i = 0; i < cmds->n; i++)
+ matrix_command_destroy (cmds->commands[i]);
+ free (cmds->commands);
+}
+
+struct matrix_command_name
+ {
+ const char *name;
+ struct matrix_command *(*parse) (struct matrix_state *);
+ };
+
+static const struct matrix_command_name *
+matrix_command_name_parse (struct lexer *lexer)
+{
+ static const struct matrix_command_name commands[] = {
+ { "COMPUTE", matrix_compute_parse },
+ { "CALL EIGEN", matrix_eigen_parse },
+ { "CALL SETDIAG", matrix_setdiag_parse },
+ { "CALL SVD", matrix_svd_parse },
+ { "PRINT", matrix_print_parse },
+ { "DO IF", matrix_do_if_parse },
+ { "LOOP", matrix_loop_parse },
+ { "BREAK", matrix_break_parse },
+ { "READ", matrix_read_parse },
+ { "WRITE", matrix_write_parse },
+ { "GET", matrix_get_parse },
+ { "SAVE", matrix_save_parse },
+ { "MGET", matrix_mget_parse },
+ { "MSAVE", matrix_msave_parse },
+ { "DISPLAY", matrix_display_parse },
+ { "RELEASE", matrix_release_parse },
+ };
+ static size_t n = sizeof commands / sizeof *commands;
+
+ for (const struct matrix_command_name *c = commands; c < &commands[n]; c++)
+ if (lex_match_phrase (lexer, c->name))
+ return c;
+ return NULL;
+}
+
+static struct matrix_command *
+matrix_command_parse (struct matrix_state *s)
+{
+ int start_ofs = lex_ofs (s->lexer);
+ size_t nesting_level = SIZE_MAX;
+
+ struct matrix_command *c = NULL;
+ const struct matrix_command_name *cmd = matrix_command_name_parse (s->lexer);
+ if (!cmd)
+ lex_error (s->lexer, _("Unknown matrix command."));
+ else if (!cmd->parse)
+ lex_error (s->lexer, _("Matrix command %s is not yet implemented."),
+ cmd->name);
+ else
+ {
+ nesting_level = output_open_group (
+ group_item_create_nocopy (utf8_to_title (cmd->name),
+ utf8_to_title (cmd->name)));
+ c = cmd->parse (s);
+ }
+
+ if (c)
+ {
+ c->location = lex_ofs_location (s->lexer, start_ofs, lex_ofs (s->lexer));
+ msg_location_remove_columns (c->location);
+ lex_end_of_command (s->lexer);
+ }
+ lex_discard_rest_of_command (s->lexer);
+ if (nesting_level != SIZE_MAX)
+ output_close_groups (nesting_level);
+
+ return c;
+}
+
+int
+cmd_matrix (struct lexer *lexer, struct dataset *ds)
+{
+ if (!lex_force_match (lexer, T_ENDCMD))
+ return CMD_FAILURE;
+
+ struct matrix_state state = {
+ .dataset = ds,
+ .session = dataset_session (ds),
+ .lexer = lexer,
+ .vars = HMAP_INITIALIZER (state.vars),
+ };
+
+ for (;;)
+ {
+ while (lex_match (lexer, T_ENDCMD))
+ continue;
+ if (lex_token (lexer) == T_STOP)
+ {
+ msg (SE, _("Unexpected end of input expecting matrix command."));
+ break;
+ }
+
+ if (lex_match_phrase (lexer, "END MATRIX"))
+ break;
+
+ struct matrix_command *c = matrix_command_parse (&state);
+ if (c)
+ {
+ matrix_command_execute (c);
+ matrix_command_destroy (c);
+ }
+ }
+
+ struct matrix_var *var, *next;
+ HMAP_FOR_EACH_SAFE (var, next, struct matrix_var, hmap_node, &state.vars)
+ {
+ free (var->name);
+ gsl_matrix_free (var->value);
+ hmap_delete (&state.vars, &var->hmap_node);
+ free (var);
+ }
+ hmap_destroy (&state.vars);
+ msave_common_destroy (state.msave_common);
+ fh_unref (state.prev_read_file);
+ for (size_t i = 0; i < state.n_read_files; i++)
+ read_file_destroy (state.read_files[i]);
+ free (state.read_files);
+ fh_unref (state.prev_write_file);
+ for (size_t i = 0; i < state.n_write_files; i++)
+ write_file_destroy (state.write_files[i]);
+ free (state.write_files);
+ fh_unref (state.prev_save_file);
+ for (size_t i = 0; i < state.n_save_files; i++)
+ save_file_destroy (state.save_files[i]);
+ free (state.save_files);
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "mcnemar.h"
+
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_randist.h>
+
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "data/variable.h"
+#include "language/commands/npar.h"
+#include "libpspp/str.h"
+#include "output/pivot-table.h"
+#include "libpspp/message.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "data/value.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct mcnemar
+{
+ union value val0;
+ union value val1;
+
+ double n00;
+ double n01;
+ double n10;
+ double n11;
+};
+
+static void
+output_freq_table (variable_pair *vp,
+ const struct mcnemar *param,
+ const struct dictionary *dict);
+
+
+static void
+output_statistics_table (const struct two_sample_test *t2s,
+ const struct mcnemar *param,
+ const struct dictionary *dict);
+
+
+void
+mcnemar_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ int i;
+ bool warn = true;
+
+ const struct dictionary *dict = dataset_dict (ds);
+
+ const struct two_sample_test *t2s = UP_CAST (test, const struct two_sample_test, parent);
+ struct ccase *c;
+
+ struct casereader *r = input;
+
+ struct mcnemar *mc = XCALLOC (t2s->n_pairs, struct mcnemar);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ mc[i].val0.f = mc[i].val1.f = SYSMIS;
+ }
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ const double weight = dict_get_case_weight (dict, c, &warn);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+ const union value *value0 = case_data (c, (*vp)[0]);
+ const union value *value1 = case_data (c, (*vp)[1]);
+
+ if (var_is_value_missing ((*vp)[0], value0) & exclude)
+ continue;
+
+ if (var_is_value_missing ((*vp)[1], value1) & exclude)
+ continue;
+
+
+ if (mc[i].val0.f == SYSMIS)
+ {
+ if (mc[i].val1.f != value0->f)
+ mc[i].val0.f = value0->f;
+ else if (mc[i].val1.f != value1->f)
+ mc[i].val0.f = value1->f;
+ }
+
+ if (mc[i].val1.f == SYSMIS)
+ {
+ if (mc[i].val0.f != value1->f)
+ mc[i].val1.f = value1->f;
+ else if (mc[i].val0.f != value0->f)
+ mc[i].val1.f = value0->f;
+ }
+
+ if (mc[i].val0.f == value0->f && mc[i].val0.f == value1->f)
+ {
+ mc[i].n00 += weight;
+ }
+ else if (mc[i].val0.f == value0->f && mc[i].val1.f == value1->f)
+ {
+ mc[i].n10 += weight;
+ }
+ else if (mc[i].val1.f == value0->f && mc[i].val0.f == value1->f)
+ {
+ mc[i].n01 += weight;
+ }
+ else if (mc[i].val1.f == value0->f && mc[i].val1.f == value1->f)
+ {
+ mc[i].n11 += weight;
+ }
+ else
+ {
+ msg (ME, _("The McNemar test is appropriate only for dichotomous variables"));
+ }
+ }
+ }
+
+ casereader_destroy (r);
+
+ for (i = 0 ; i < t2s->n_pairs ; ++i)
+ output_freq_table (&t2s->pairs[i], mc + i, dict);
+
+ output_statistics_table (t2s, mc, dict);
+
+ free (mc);
+}
+
+static char *
+make_pair_name (const variable_pair *pair)
+{
+ return xasprintf ("%s & %s", var_to_string ((*pair)[0]),
+ var_to_string ((*pair)[1]));
+}
+
+static void
+output_freq_table (variable_pair *vp,
+ const struct mcnemar *param,
+ const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_user_text_nocopy (make_pair_name (vp)), "Frequencies");
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ struct pivot_dimension *vars[2];
+ for (int i = 0; i < 2; i++)
+ {
+ vars[i] = pivot_dimension_create__ (
+ table, i ? PIVOT_AXIS_COLUMN : PIVOT_AXIS_ROW,
+ pivot_value_new_variable ((*vp)[i]));
+ vars[i]->root->show_label = true;
+
+ for (int j = 0; j < 2; j++)
+ {
+ const union value *val = j ? ¶m->val1 : ¶m->val0;
+ pivot_category_create_leaf_rc (
+ vars[i]->root, pivot_value_new_var_value ((*vp)[0], val),
+ PIVOT_RC_COUNT);
+ }
+ }
+
+ struct entry
+ {
+ int idx0;
+ int idx1;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, param->n00 },
+ { 1, 0, param->n01 },
+ { 0, 1, param->n10 },
+ { 1, 1, param->n11 },
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put2 (table, e->idx0, e->idx1,
+ pivot_value_new_number (e->x));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+output_statistics_table (const struct two_sample_test *t2s,
+ const struct mcnemar *mc,
+ const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Point Probability"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Pairs"));
+
+ for (size_t i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+ int pair_idx = pivot_category_create_leaf (
+ pairs->root, pivot_value_new_user_text_nocopy (make_pair_name (vp)));
+
+ double n = mc[i].n00 + mc[i].n01 + mc[i].n10 + mc[i].n11;
+ double sig = gsl_cdf_binomial_P ((mc[i].n01 > mc[i].n10) ? mc[i].n10: mc[i].n01,
+ 0.5, mc[i].n01 + mc[i].n10);
+
+ double point = gsl_ran_binomial_pdf (mc[i].n01, 0.5,
+ mc[i].n01 + mc[i].n10);
+ double entries[] = { n, 2.0 * sig, sig, point };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, pair_idx,
+ pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !mcnemar_h
+#define mcnemar_h 1
+
+
+#include <stdbool.h>
+#include "data/missing-values.h"
+
+struct casereader;
+struct dataset;
+struct npar_test;
+
+void mcnemar_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer);
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2021 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <math.h>
+
+#include "data/any-reader.h"
+#include "data/any-writer.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/matrix-reader.h"
+#include "language/lexer/lexer.h"
+#include "language/command.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_mconvert (struct lexer *lexer, struct dataset *ds)
+{
+ bool append = false;
+ struct file_handle *in = NULL;
+ struct file_handle *out = NULL;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "APPEND"))
+ append = true;
+ else if (lex_match_id (lexer, "REPLACE"))
+ append = false;
+ else
+ {
+ if (lex_match_id (lexer, "MATRIX"))
+ lex_match (lexer, T_EQUALS);
+
+ struct file_handle **fhp = (lex_match_id (lexer, "IN") ? &in
+ : lex_match_id (lexer, "OUT") ? &out
+ : NULL);
+ if (!fhp)
+ {
+ lex_error_expecting (lexer, "IN", "OUT", "APPEND", "REPLACE");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ fh_unref (*fhp);
+ if (lex_match (lexer, T_ASTERISK))
+ *fhp = NULL;
+ else
+ {
+ *fhp = fh_parse (lexer, FH_REF_FILE, dataset_session (ds));
+ if (!*fhp)
+ goto error;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+
+ if (!in && !dataset_has_source (ds))
+ {
+ msg (SE, _("No active file is defined and no external file is "
+ "specified on MATRIX=IN."));
+ goto error;
+ }
+
+ struct dictionary *d;
+ struct casereader *cr;
+ if (in)
+ {
+ cr = any_reader_open_and_decode (in, NULL, &d, NULL);
+ if (!cr)
+ goto error;
+ }
+ else
+ {
+ d = dict_clone (dataset_dict (ds));
+ cr = proc_open (ds);
+ }
+
+ struct matrix_reader *mr = matrix_reader_create (d, cr);
+ if (!mr)
+ {
+ casereader_destroy (cr);
+ dict_unref (d);
+ if (!in)
+ proc_commit (ds);
+ goto error;
+ }
+
+ struct casewriter *cw;
+ if (out)
+ {
+ cw = any_writer_open (out, d);
+ if (!cw)
+ {
+ matrix_reader_destroy (mr);
+ casereader_destroy (cr);
+ dict_unref (d);
+ if (!in)
+ proc_commit (ds);
+ goto error;
+ }
+ }
+ else
+ cw = autopaging_writer_create (dict_get_proto (d));
+
+ for (;;)
+ {
+ struct matrix_material mm;
+ struct casereader *group;
+ if (!matrix_reader_next (&mm, mr, &group))
+ break;
+
+ bool add_corr = mm.cov && !mm.corr;
+ bool add_cov = mm.corr && !mm.cov && mm.var_matrix;
+ bool add_stddev = add_corr && !mm.var_matrix;
+ bool remove_corr = add_cov && !append;
+ bool remove_cov = add_corr && !append;
+
+ struct ccase *model = casereader_peek (group, 0);
+ for (size_t i = 0; i < mr->n_fvars; i++)
+ *case_num_rw (model, mr->fvars[i]) = SYSMIS;
+
+ for (;;)
+ {
+ struct ccase *c = casereader_read (group);
+ if (!c)
+ break;
+
+ struct substring rowtype = matrix_reader_get_string (c, mr->rowtype);
+ if ((remove_cov && ss_equals_case (rowtype, ss_cstr ("COV")))
+ || (remove_corr && ss_equals_case (rowtype, ss_cstr ("CORR"))))
+ case_unref (c);
+ else
+ casewriter_write (cw, c);
+ }
+ casereader_destroy (group);
+
+ if (add_corr)
+ {
+ for (size_t y = 0; y < mr->n_cvars; y++)
+ {
+ struct ccase *c = case_clone (model);
+ for (size_t x = 0; x < mr->n_cvars; x++)
+ {
+ double d1 = gsl_matrix_get (mm.cov, x, x);
+ double d2 = gsl_matrix_get (mm.cov, y, y);
+ double cov = gsl_matrix_get (mm.cov, y, x);
+ *case_num_rw (c, mr->cvars[x]) = cov / sqrt (d1 * d2);
+ }
+ matrix_reader_set_string (c, mr->rowtype, ss_cstr ("CORR"));
+ matrix_reader_set_string (c, mr->varname,
+ ss_cstr (var_get_name (mr->cvars[y])));
+ casewriter_write (cw, c);
+ }
+ }
+
+ if (add_stddev)
+ {
+ struct ccase *c = case_clone (model);
+ for (size_t x = 0; x < mr->n_cvars; x++)
+ {
+ double variance = gsl_matrix_get (mm.cov, x, x);
+ *case_num_rw (c, mr->cvars[x]) = sqrt (variance);
+ }
+ matrix_reader_set_string (c, mr->rowtype, ss_cstr ("STDDEV"));
+ matrix_reader_set_string (c, mr->varname, ss_empty ());
+ casewriter_write (cw, c);
+ }
+
+ if (add_cov)
+ {
+ for (size_t y = 0; y < mr->n_cvars; y++)
+ {
+ struct ccase *c = case_clone (model);
+ for (size_t x = 0; x < mr->n_cvars; x++)
+ {
+ double d1 = gsl_matrix_get (mm.var_matrix, x, x);
+ double d2 = gsl_matrix_get (mm.var_matrix, y, y);
+ double corr = gsl_matrix_get (mm.corr, y, x);
+ *case_num_rw (c, mr->cvars[x]) = corr * sqrt (d1 * d2);
+ }
+ matrix_reader_set_string (c, mr->rowtype, ss_cstr ("COV"));
+ matrix_reader_set_string (c, mr->varname,
+ ss_cstr (var_get_name (mr->cvars[y])));
+ casewriter_write (cw, c);
+ }
+ }
+
+ case_unref (model);
+ matrix_material_uninit (&mm);
+ }
+
+ matrix_reader_destroy (mr);
+ if (!in)
+ proc_commit (ds);
+ if (out)
+ casewriter_destroy (cw);
+ else
+ {
+ dataset_set_dict (ds, dict_ref (d));
+ dataset_set_source (ds, casewriter_make_reader (cw));
+ }
+
+ fh_unref (in);
+ fh_unref (out);
+ dict_unref (d);
+ return CMD_SUCCESS;
+
+error:
+ fh_unref (in);
+ fh_unref (out);
+ return CMD_FAILURE;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011, 2012, 2013, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "data/case.h"
+#include "data/format.h"
+#include "data/variable.h"
+
+#include "libpspp/bt.h"
+#include "libpspp/hmap.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+
+#include "math/moments.h"
+#include "output/pivot-table.h"
+
+#include <math.h>
+
+#include "means.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+/* A base struct for all statistics. */
+struct statistic
+{
+};
+
+/* Statistics which accumulate a single value. */
+struct statistic_simple
+{
+ struct statistic parent;
+ double acc;
+};
+
+/* Statistics based on moments. */
+struct statistic_moment
+{
+ struct statistic parent;
+ struct moments1 *mom;
+};
+
+
+static struct statistic *
+default_create (struct pool *pool)
+{
+ struct statistic_moment *pvd = pool_alloc (pool, sizeof *pvd);
+
+ pvd->mom = moments1_create (MOMENT_KURTOSIS);
+
+ return (struct statistic *) pvd;
+}
+
+static void
+default_update (struct statistic *stat, double w, double x)
+{
+ struct statistic_moment *pvd = (struct statistic_moment *) stat;
+
+ moments1_add (pvd->mom, x, w);
+}
+
+static void
+default_destroy (struct statistic *stat)
+{
+ struct statistic_moment *pvd = (struct statistic_moment *) stat;
+ moments1_destroy (pvd->mom);
+}
+
+
+/* Simple statistics have nothing to destroy. */
+static void
+simple_destroy (struct statistic *stat UNUSED)
+{
+}
+
+\f
+
+/* HARMONIC MEAN: The reciprocal of the sum of the reciprocals:
+ 1 / (1/(x_0) + 1/(x_1) + ... + 1/(x_{n-1})) */
+
+struct harmonic_mean
+{
+ struct statistic parent;
+ double rsum;
+ double n;
+};
+
+static struct statistic *
+harmonic_create (struct pool *pool)
+{
+ struct harmonic_mean *hm = pool_alloc (pool, sizeof *hm);
+
+ hm->rsum = 0;
+ hm->n = 0;
+
+ return (struct statistic *) hm;
+}
+
+
+static void
+harmonic_update (struct statistic *stat, double w, double x)
+{
+ struct harmonic_mean *hm = (struct harmonic_mean *) stat;
+ hm->rsum += w / x;
+ hm->n += w;
+}
+
+
+static double
+harmonic_get (const struct statistic *pvd)
+{
+ const struct harmonic_mean *hm = (const struct harmonic_mean *) pvd;
+
+ return hm->n / hm->rsum;
+}
+
+\f
+
+/* GEOMETRIC MEAN: The nth root of the product of all n observations
+ pow ((x_0 * x_1 * ... x_{n - 1}), 1/n) */
+struct geometric_mean
+{
+ struct statistic parent;
+ double prod;
+ double n;
+};
+
+static struct statistic *
+geometric_create (struct pool *pool)
+{
+ struct geometric_mean *gm = pool_alloc (pool, sizeof *gm);
+
+ gm->prod = 1.0;
+ gm->n = 0;
+
+ return (struct statistic *) gm;
+}
+
+static void
+geometric_update (struct statistic *pvd, double w, double x)
+{
+ struct geometric_mean *gm = (struct geometric_mean *)pvd;
+ gm->prod *= pow (x, w);
+ gm->n += w;
+}
+
+
+static double
+geometric_get (const struct statistic *pvd)
+{
+ const struct geometric_mean *gm = (const struct geometric_mean *)pvd;
+ return pow (gm->prod, 1.0 / gm->n);
+}
+
+\f
+
+/* The getters for moment based statistics simply calculate the
+ moment. The only exception is Std Dev. which needs to call
+ sqrt as well. */
+
+static double
+sum_get (const struct statistic *pvd)
+{
+ double n, mean;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, &mean, 0, 0, 0);
+
+ return mean * n;
+}
+
+
+static double
+n_get (const struct statistic *pvd)
+{
+ double n;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, 0, 0, 0, 0);
+
+ return n;
+}
+
+static double
+arithmean_get (const struct statistic *pvd)
+{
+ double n, mean;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, &mean, 0, 0, 0);
+
+ return mean;
+}
+
+static double
+variance_get (const struct statistic *pvd)
+{
+ double n, mean, variance;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, &mean, &variance, 0, 0);
+
+ return variance;
+}
+
+
+static double
+stddev_get (const struct statistic *pvd)
+{
+ return sqrt (variance_get (pvd));
+}
+
+static double
+skew_get (const struct statistic *pvd)
+{
+ double skew;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, NULL, NULL, NULL, &skew, 0);
+
+ return skew;
+}
+
+static double
+sekurt_get (const struct statistic *pvd)
+{
+ double n;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, NULL, NULL, NULL, NULL);
+
+ return calc_sekurt (n);
+}
+
+static double
+seskew_get (const struct statistic *pvd)
+{
+ double n;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, NULL, NULL, NULL, NULL);
+
+ return calc_seskew (n);
+}
+
+static double
+kurt_get (const struct statistic *pvd)
+{
+ double kurt;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, NULL, NULL, NULL, NULL, &kurt);
+
+ return kurt;
+}
+
+static double
+semean_get (const struct statistic *pvd)
+{
+ double n, var;
+
+ moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, NULL, &var, NULL, NULL);
+
+ return sqrt (var / n);
+}
+
+\f
+
+/* MIN: The smallest (closest to minus infinity) value. */
+
+static struct statistic *
+min_create (struct pool *pool)
+{
+ struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
+
+ pvd->acc = DBL_MAX;
+
+ return (struct statistic *) pvd;
+}
+
+static void
+min_update (struct statistic *pvd, double w UNUSED, double x)
+{
+ double *r = &((struct statistic_simple *)pvd)->acc;
+
+ if (x < *r)
+ *r = x;
+}
+
+static double
+min_get (const struct statistic *pvd)
+{
+ double *r = &((struct statistic_simple *)pvd)->acc;
+
+ return *r;
+}
+
+/* MAX: The largest (closest to plus infinity) value. */
+
+static struct statistic *
+max_create (struct pool *pool)
+{
+ struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
+
+ pvd->acc = -DBL_MAX;
+
+ return (struct statistic *) pvd;
+}
+
+static void
+max_update (struct statistic *pvd, double w UNUSED, double x)
+{
+ double *r = &((struct statistic_simple *)pvd)->acc;
+
+ if (x > *r)
+ *r = x;
+}
+
+static double
+max_get (const struct statistic *pvd)
+{
+ double *r = &((struct statistic_simple *)pvd)->acc;
+
+ return *r;
+}
+
+\f
+
+struct range
+{
+ struct statistic parent;
+ double min;
+ double max;
+};
+
+/* Initially min and max are set to their most (inverted) extreme possible
+ values. */
+static struct statistic *
+range_create (struct pool *pool)
+{
+ struct range *r = pool_alloc (pool, sizeof *r);
+
+ r->min = DBL_MAX;
+ r->max = -DBL_MAX;
+
+ return (struct statistic *) r;
+}
+
+/* On each update, set min and max to X or leave unchanged,
+ as appropriate. */
+static void
+range_update (struct statistic *pvd, double w UNUSED, double x)
+{
+ struct range *r = (struct range *) pvd;
+
+ if (x > r->max)
+ r->max = x;
+
+ if (x < r->min)
+ r->min = x;
+}
+
+/* Get the difference between min and max. */
+static double
+range_get (const struct statistic *pvd)
+{
+ const struct range *r = (struct range *) pvd;
+
+ return r->max - r->min;
+}
+
+\f
+
+/* LAST: The last value (the one closest to the end of the file). */
+
+static struct statistic *
+last_create (struct pool *pool)
+{
+ struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
+
+ return (struct statistic *) pvd;
+}
+
+static void
+last_update (struct statistic *pvd, double w UNUSED, double x)
+{
+ struct statistic_simple *stat = (struct statistic_simple *) pvd;
+
+ stat->acc = x;
+}
+
+static double
+last_get (const struct statistic *pvd)
+{
+ const struct statistic_simple *stat = (struct statistic_simple *) pvd;
+
+ return stat->acc;
+}
+
+/* FIRST: The first value (the one closest to the start of the file). */
+
+static struct statistic *
+first_create (struct pool *pool)
+{
+ struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
+
+ pvd->acc = SYSMIS;
+
+ return (struct statistic *) pvd;
+}
+
+static void
+first_update (struct statistic *pvd, double w UNUSED, double x)
+{
+ struct statistic_simple *stat = (struct statistic_simple *) pvd;
+
+ if (stat->acc == SYSMIS)
+ stat->acc = x;
+}
+
+static double
+first_get (const struct statistic *pvd)
+{
+ const struct statistic_simple *stat = (struct statistic_simple *) pvd;
+
+ return stat->acc;
+}
+
+/* Table of cell_specs */
+const struct cell_spec cell_spec[n_MEANS_STATISTICS] = {
+ {N_("Mean"), "MEAN", NULL , default_create, default_update, arithmean_get, default_destroy},
+ {N_("N"), "COUNT", PIVOT_RC_COUNT, default_create, default_update, n_get, default_destroy},
+ {N_("Std. Deviation"), "STDDEV", NULL , default_create, default_update, stddev_get, default_destroy},
+#if 0
+ {N_("Median"), "MEDIAN", NULL , default_create, default_update, NULL, default_destroy},
+ {N_("Group Median"), "GMEDIAN", NULL , default_create, default_update, NULL, default_destroy},
+#endif
+ {N_("S.E. Mean"), "SEMEAN", NULL , default_create, default_update, semean_get, default_destroy},
+ {N_("Sum"), "SUM", NULL , default_create, default_update, sum_get, default_destroy},
+ {N_("Minimum"), "MIN", NULL , min_create, min_update, min_get, simple_destroy},
+ {N_("Maximum"), "MAX", NULL , max_create, max_update, max_get, simple_destroy},
+ {N_("Range"), "RANGE", NULL , range_create, range_update, range_get, simple_destroy},
+ {N_("Variance"), "VARIANCE", PIVOT_RC_OTHER, default_create, default_update, variance_get, default_destroy},
+ {N_("Kurtosis"), "KURT", PIVOT_RC_OTHER, default_create, default_update, kurt_get, default_destroy},
+ {N_("S.E. Kurt"), "SEKURT", PIVOT_RC_OTHER, default_create, default_update, sekurt_get, default_destroy},
+ {N_("Skewness"), "SKEW", PIVOT_RC_OTHER, default_create, default_update, skew_get, default_destroy},
+ {N_("S.E. Skew"), "SESKEW", PIVOT_RC_OTHER, default_create, default_update, seskew_get, default_destroy},
+ {N_("First"), "FIRST", NULL , first_create, first_update, first_get, simple_destroy},
+ {N_("Last"), "LAST", NULL , last_create, last_update, last_get, simple_destroy},
+#if 0
+ {N_("Percent N"), "NPCT", PIVOT_RC_PERCENT, default_create, default_update, NULL, default_destroy},
+ {N_("Percent Sum"), "SPCT", PIVOT_RC_PERCENT, default_create, default_update, NULL, default_destroy},
+#endif
+ {N_("Harmonic Mean"), "HARMONIC", NULL , harmonic_create, harmonic_update, harmonic_get, simple_destroy},
+ {N_("Geom. Mean"), "GEOMETRIC", NULL , geometric_create, geometric_update, geometric_get, simple_destroy}
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011, 2012, 2013, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+
+#include "libpspp/pool.h"
+
+#include "means.h"
+
+/* Parse the /TABLES stanza of the command. */
+static bool
+parse_means_table_syntax (struct lexer *lexer, const struct means *cmd,
+ struct mtable *table)
+{
+ memset (table, 0, sizeof *table);
+
+ /* Dependent variable (s) */
+ if (!parse_variables_const_pool (lexer, cmd->pool, cmd->dict,
+ &table->dep_vars, &table->n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ return false;
+
+ /* Factor variable (s) */
+ while (lex_match (lexer, T_BY))
+ {
+ struct layer *layer = pool_zalloc (cmd->pool, sizeof *layer);
+
+ table->layers =
+ pool_nrealloc (cmd->pool, table->layers, table->n_layers + 1,
+ sizeof *table->layers);
+ table->layers[table->n_layers] = layer;
+ table->n_layers++;
+
+ if (!parse_variables_const_pool
+ (lexer, cmd->pool, cmd->dict,
+ &layer->factor_vars,
+ &layer->n_factor_vars,
+ PV_NO_DUPLICATE))
+ return false;
+ }
+
+ return true;
+}
+
+/* Match a variable.
+ If the match succeeds, the variable will be placed in VAR.
+ Returns true if successful */
+static bool
+lex_is_variable (struct lexer *lexer, const struct dictionary *dict,
+ int n)
+{
+ if (lex_next_token (lexer, n) != T_ID)
+ return false;
+
+ const char *tstr = lex_next_tokcstr (lexer, n);
+ return dict_lookup_var (dict, tstr) != NULL;
+}
+
+static const struct cell_spec *
+match_cell (struct lexer *lexer)
+{
+ for (size_t i = 0; i < n_MEANS_STATISTICS; ++i)
+ {
+ const struct cell_spec *cs = &cell_spec[i];
+ if (lex_match_id (lexer, cs->keyword))
+ return cs;
+ }
+ return NULL;
+}
+
+static void
+add_statistic (struct means *means, int statistic)
+{
+ if (means->n_statistics >= means->allocated_statistics)
+ means->statistics = pool_2nrealloc (means->pool, means->statistics,
+ &means->allocated_statistics,
+ sizeof *means->statistics);
+ means->statistics[means->n_statistics++] = statistic;
+}
+
+void
+means_set_default_statistics (struct means *means)
+{
+ means->n_statistics = 0;
+ add_statistic (means, MEANS_MEAN);
+ add_statistic (means, MEANS_N);
+ add_statistic (means, MEANS_STDDEV);
+}
+
+bool
+means_parse (struct lexer *lexer, struct means *means)
+{
+ /* Optional TABLES=. */
+ if (lex_match_id (lexer, "TABLES") && !lex_force_match (lexer, T_EQUALS))
+ return false;
+
+ /* Parse the "tables" */
+ for (;;)
+ {
+ means->table = pool_realloc (means->pool, means->table,
+ (means->n_tables + 1) * sizeof *means->table);
+
+ if (!parse_means_table_syntax (lexer, means,
+ &means->table[means->n_tables]))
+ return false;
+ means->n_tables++;
+
+ /* Look ahead to see if there are more tables to be parsed */
+ if (lex_next_token (lexer, 0) != T_SLASH
+ || !lex_is_variable (lexer, means->dict, 1))
+ break;
+ lex_match (lexer, T_SLASH);
+ }
+
+ /* /MISSING subcommand */
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ /* If no MISSING subcommand is specified, each combination of a
+ dependent variable and categorical variables is handled
+ separately. */
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "INCLUDE"))
+ {
+ /* Use the subcommand "/MISSING=INCLUDE" to include user-missing
+ values in the analysis. */
+
+ means->ctrl_exclude = MV_SYSTEM;
+ means->dep_exclude = MV_SYSTEM;
+ }
+ else if (lex_match_id (lexer, "DEPENDENT"))
+ /* Use the command "/MISSING=DEPENDENT" to include user-missing
+ values for the categorical variables, while excluding them for
+ the dependent variables.
+
+ Cases are dropped only when user-missing values appear in
+ dependent variables. User-missing values for categorical
+ variables are treated according to their face value.
+
+ Cases are ALWAYS dropped when System Missing values appear in
+ the categorical variables. */
+ {
+ means->dep_exclude = MV_ANY;
+ means->ctrl_exclude = MV_SYSTEM;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "DEPENDENT");
+ return false;
+ }
+ }
+ else if (lex_match_id (lexer, "CELLS"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ /* The default values become overwritten */
+ means->n_statistics = 0;
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match (lexer, T_ALL))
+ {
+ means->n_statistics = 0;
+ for (int i = 0; i < n_MEANS_STATISTICS; ++i)
+ add_statistic (means, i);
+ }
+ else if (lex_match_id (lexer, "NONE"))
+ means->n_statistics = 0;
+ else if (lex_match_id (lexer, "DEFAULT"))
+ means_set_default_statistics (means);
+ else
+ {
+ const struct cell_spec *cs = match_cell (lexer);
+ if (cs)
+ add_statistic (means, cs - cell_spec);
+ else
+ {
+ const char *keywords[n_MEANS_STATISTICS];
+ for (int i = 0; i < n_MEANS_STATISTICS; ++i)
+ keywords[i] = cell_spec[i].keyword;
+ lex_error_expecting_array (lexer, keywords,
+ n_MEANS_STATISTICS);
+ return false;
+ }
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MISSING", "CELLS");
+ return false;
+ }
+ }
+ return true;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+
+#include "libpspp/hmap.h"
+#include "libpspp/bt.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+
+#include "count-one-bits.h"
+#include "count-leading-zeros.h"
+
+#include "output/pivot-table.h"
+
+#include "means.h"
+
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+
+/* A "cell" in this procedure represents a distinct value of the
+ procedure's categorical variables, and a set of summary statistics
+ of all cases which whose categorical variables have that set of
+ values. For example, the dataset
+
+ v1 v2 cat1 cat2
+ 100 202 0 1
+ 100 202 0 2
+ 100 202 1 0
+ 100 202 0 1
+
+
+ has three cells in layer 0 and two cells in layer 1 in addition
+ to a "grand summary" cell to which all (non-missing) cases
+ contribute.
+
+ The cells form a n-ary tree structure with the "grand summary"
+ cell at the root.
+*/
+struct cell
+{
+ struct hmap_node hmap_node; /* Element in hash table. */
+ struct bt_node bt_node; /* Element in binary tree */
+
+ int n_children;
+ struct cell_container *children;
+
+ /* The statistics to be calculated for the cell. */
+ struct statistic **stat;
+
+ /* The parent of this cell, or NULL if this is the root cell. */
+ const struct cell *parent_cell;
+
+ /* A bit-field variable which indicates which control variables
+ are allocated a fixed value (for this cell), and which are
+ "wildcards".
+
+ A one indicates a fixed value. A zero indicates a wildcard.
+ Wildcard values are used to calculate totals and sub-totals.
+ */
+ unsigned int not_wild;
+
+ /* The value(s). */
+ union value *values;
+
+ /* The variables corresponding to the above values. */
+ const struct variable **vars;
+};
+
+/* A structure used to find the union of all values used
+ within a layer, and to sort those values. */
+struct instance
+{
+ struct hmap_node hmap_node; /* Element in hash table. */
+ struct bt_node bt_node; /* Element in binary tree */
+
+ /* A unique, consecutive, zero based index identifying this
+ instance. */
+ int index;
+
+ /* The top level value of this instance. */
+ union value value;
+ const struct variable *var;
+};
+
+
+static void
+destroy_workspace (const struct mtable *mt, struct workspace *ws)
+{
+ for (int l = 0; l < mt->n_layers; ++l)
+ {
+ struct cell_container *instances = ws->instances + l;
+ struct instance *inst;
+ struct instance *next;
+ HMAP_FOR_EACH_SAFE (inst, next, struct instance, hmap_node,
+ &instances->map)
+ {
+ int width = var_get_width (inst->var);
+ value_destroy (&inst->value, width);
+ free (inst);
+ }
+ hmap_destroy (&instances->map);
+ }
+ free (ws->control_idx);
+ free (ws->instances);
+}
+
+/* Destroy CELL. */
+static void
+destroy_cell (const struct means *means,
+ const struct mtable *mt, struct cell *cell)
+{
+ int idx = 0;
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ if (0 == ((cell->not_wild >> i) & 0x1))
+ continue;
+
+ const struct layer *layer = mt->layers[i];
+ for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
+ {
+ struct workspace *ws = mt->ws + cmb;
+ const struct variable *var
+ = layer->factor_vars[ws->control_idx[i]];
+
+ int width = var_get_width (var);
+ value_destroy (&cell->values[idx++], width);
+ }
+ }
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *container = cell->children + i;
+ hmap_destroy (&container->map);
+ }
+
+ for (int v = 0; v < mt->n_dep_vars; ++v)
+ {
+ for (int s = 0; s < means->n_statistics; ++s)
+ {
+ stat_destroy *des = cell_spec[means->statistics[s]].sf;
+ des (cell->stat[s + v * means->n_statistics]);
+ }
+ }
+ free (cell->stat);
+
+ free (cell->children);
+ free (cell->values);
+ free (cell->vars);
+ free (cell);
+}
+
+
+/* Walk the tree in postorder starting from CELL and destroy all the
+ cells. */
+static void
+means_destroy_cells (const struct means *means, struct cell *cell,
+ const struct mtable *table)
+{
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *container = cell->children + i;
+ struct cell *sub_cell;
+ struct cell *next;
+ HMAP_FOR_EACH_SAFE (sub_cell, next, struct cell, hmap_node,
+ &container->map)
+ {
+ means_destroy_cells (means, sub_cell, table);
+ }
+ }
+
+ destroy_cell (means, table, cell);
+}
+
+#if 0
+
+static void
+dump_cell (const struct cell *cell, const struct mtable *mt, int level)
+{
+ for (int l = 0; l < level; ++l)
+ putchar (' ');
+ printf ("%p: ", cell);
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ putchar (((cell->not_wild >> i) & 0x1) ? 'w' : '.');
+ }
+ printf (" - ");
+ int x = 0;
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ if ((cell->not_wild >> i) & 0x1)
+ {
+ printf ("%s: ", var_get_name (cell->vars[x]));
+ printf ("%g ", cell->values[x++].f);
+ }
+ else
+ printf ("x ");
+ }
+ stat_get *sg = cell_spec[MEANS_N].sd;
+ printf ("--- S1: %g", sg (cell->stat[0]));
+
+ printf ("--- N Children: %d", cell->n_children);
+ // printf ("--- Level: %d", level);
+ printf ("--- Parent: %p", cell->parent_cell);
+ printf ("\n");
+}
+
+static void
+dump_indeces (const size_t *indexes, int n)
+{
+ for (int i = 0; i < n; ++i)
+ {
+ printf ("%ld; ", indexes[i]);
+ }
+ printf ("\n");
+}
+
+/* Dump the tree in pre-order. */
+static void
+dump_tree (const struct cell *cell, const struct mtable *table,
+ int level, const struct cell *parent)
+{
+ assert (cell->parent_cell == parent);
+ dump_cell (cell, table, level);
+
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *container = cell->children + i;
+ struct cell *sub_cell;
+ BT_FOR_EACH (sub_cell, struct cell, bt_node, &container->bt)
+ {
+ dump_tree (sub_cell, table, level + 1, cell);
+ }
+ }
+}
+
+#endif
+
+/* Generate a hash based on the values of the N variables in
+ the array VARS which are taken from the case C. */
+static unsigned int
+generate_hash (const struct mtable *mt,
+ const struct ccase *c,
+ unsigned int not_wild,
+ const struct workspace *ws)
+{
+ unsigned int hash = 0;
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ if (0 == ((not_wild >> i) & 0x1))
+ continue;
+
+ const struct layer *layer = mt->layers[i];
+ const struct variable *var = layer->factor_vars[ws->control_idx[i]];
+ const union value *vv = case_data (c, var);
+ int width = var_get_width (var);
+ hash = hash_int (i, hash);
+ hash = value_hash (vv, width, hash);
+ }
+
+ return hash;
+}
+
+/* Create a cell based on the N variables in the array VARS,
+ which are indeces into the case C.
+ The caller is responsible for destroying this cell when
+ no longer needed. */
+static struct cell *
+generate_cell (const struct means *means,
+ const struct mtable *mt,
+ const struct ccase *c,
+ unsigned int not_wild,
+ const struct cell *pcell,
+ const struct workspace *ws)
+{
+ int n_vars = count_one_bits (not_wild);
+ struct cell *cell = XZALLOC (struct cell);
+ cell->values = xcalloc (n_vars, sizeof *cell->values);
+ cell->vars = xcalloc (n_vars, sizeof *cell->vars);
+ cell->not_wild = not_wild;
+
+ cell->parent_cell = pcell;
+ cell->n_children = mt->n_layers -
+ (sizeof (cell->not_wild) * CHAR_BIT) +
+ count_leading_zeros (cell->not_wild);
+
+ int idx = 0;
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ if (0 == ((not_wild >> i) & 0x1))
+ continue;
+
+ const struct layer *layer = mt->layers[i];
+ const struct variable *var = layer->factor_vars[ws->control_idx[i]];
+ const union value *vv = case_data (c, var);
+ int width = var_get_width (var);
+ cell->vars[idx] = var;
+ value_clone (&cell->values[idx++], vv, width);
+ }
+ assert (idx == n_vars);
+
+ cell->children = xcalloc (cell->n_children, sizeof *cell->children);
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *container = cell->children + i;
+ hmap_init (&container->map);
+ }
+
+ cell->stat = xcalloc (means->n_statistics * mt->n_dep_vars, sizeof *cell->stat);
+ for (int v = 0; v < mt->n_dep_vars; ++v)
+ {
+ for (int stat = 0; stat < means->n_statistics; ++stat)
+ {
+ stat_create *sc = cell_spec[means->statistics[stat]].sc;
+
+ cell->stat[stat + v * means->n_statistics] = sc (means->pool);
+ }
+ }
+ return cell;
+}
+
+
+/* If a cell based on the N variables in the array VARS,
+ which are indeces into the case C and whose hash is HASH,
+ exists in HMAP, then return that cell.
+ Otherwise, return NULL. */
+static struct cell *
+lookup_cell (const struct mtable *mt,
+ struct hmap *hmap, unsigned int hash,
+ const struct ccase *c,
+ unsigned int not_wild,
+ const struct workspace *ws)
+{
+ struct cell *cell = NULL;
+ HMAP_FOR_EACH_WITH_HASH (cell, struct cell, hmap_node, hash, hmap)
+ {
+ bool match = true;
+ int idx = 0;
+ if (cell->not_wild != not_wild)
+ continue;
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ if (0 == ((cell->not_wild >> i) & 0x1))
+ continue;
+
+ const struct layer *layer = mt->layers[i];
+ const struct variable *var = layer->factor_vars[ws->control_idx[i]];
+ const union value *vv = case_data (c, var);
+ int width = var_get_width (var);
+ assert (var == cell->vars[idx]);
+ if (!value_equal (vv, &cell->values[idx++], width))
+ {
+ match = false;
+ break;
+ }
+ }
+ if (match)
+ return cell;
+ }
+ return NULL;
+}
+
+
+/* A comparison function used to sort cells in a binary tree.
+ Only the innermost value needs to be compared, because no
+ two cells with similar outer values will appear in the same
+ tree/map. */
+static int
+cell_compare_3way (const struct bt_node *a,
+ const struct bt_node *b,
+ const void *aux UNUSED)
+{
+ const struct cell *fa = BT_DATA (a, struct cell, bt_node);
+ const struct cell *fb = BT_DATA (b, struct cell, bt_node);
+
+ assert (fa->not_wild == fb->not_wild);
+ int vidx = count_one_bits (fa->not_wild) - 1;
+ assert (fa->vars[vidx] == fb->vars[vidx]);
+
+ return value_compare_3way (&fa->values[vidx],
+ &fb->values[vidx],
+ var_get_width (fa->vars[vidx]));
+}
+
+/* A comparison function used to sort cells in a binary tree. */
+static int
+compare_instance_3way (const struct bt_node *a,
+ const struct bt_node *b,
+ const void *aux UNUSED)
+{
+ const struct instance *fa = BT_DATA (a, struct instance, bt_node);
+ const struct instance *fb = BT_DATA (b, struct instance, bt_node);
+
+ assert (fa->var == fb->var);
+
+ return value_compare_3way (&fa->value,
+ &fb->value,
+ var_get_width (fa->var));
+}
+
+
+static void arrange_cells (struct workspace *ws,
+ struct cell *cell, const struct mtable *table);
+
+
+/* Iterate CONTAINER's map inserting a copy of its elements into
+ CONTAINER's binary tree. Also, for each layer in TABLE, create
+ an instance container, containing the union of all elements in
+ CONTAINER. */
+static void
+arrange_cell (struct workspace *ws, struct cell_container *container,
+ const struct mtable *mt)
+{
+ struct bt *bt = &container->bt;
+ struct hmap *map = &container->map;
+ bt_init (bt, cell_compare_3way, NULL);
+
+ struct cell *cell;
+ HMAP_FOR_EACH (cell, struct cell, hmap_node, map)
+ {
+ bt_insert (bt, &cell->bt_node);
+
+ int idx = 0;
+ for (int i = 0; i < mt->n_layers; ++i)
+ {
+ if (0 == ((cell->not_wild >> i) & 0x1))
+ continue;
+
+ struct cell_container *instances = ws->instances + i;
+ const struct variable *var = cell->vars[idx];
+ int width = var_get_width (var);
+ unsigned int hash
+ = value_hash (&cell->values[idx], width, 0);
+
+ struct instance *inst = NULL;
+ struct instance *next = NULL;
+ HMAP_FOR_EACH_WITH_HASH_SAFE (inst, next, struct instance,
+ hmap_node,
+ hash, &instances->map)
+ {
+ assert (cell->vars[idx] == var);
+ if (value_equal (&inst->value,
+ &cell->values[idx],
+ width))
+ {
+ break;
+ }
+ }
+
+ if (!inst)
+ {
+ inst = xzalloc (sizeof *inst);
+ inst->index = -1;
+ inst->var = var;
+ value_clone (&inst->value, &cell->values[idx],
+ width);
+ hmap_insert (&instances->map, &inst->hmap_node, hash);
+ }
+
+ idx++;
+ }
+
+ arrange_cells (ws, cell, mt);
+ }
+}
+
+/* Arrange the children and then all the subtotals. */
+static void
+arrange_cells (struct workspace *ws, struct cell *cell,
+ const struct mtable *table)
+{
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *container = cell->children + i;
+ arrange_cell (ws, container, table);
+ }
+}
+
+
+\f
+
+/* If the top level value in CELL, has an instance in the L_IDX'th layer,
+ then return that instance. Otherwise return NULL. */
+static const struct instance *
+lookup_instance (const struct mtable *mt, const struct workspace *ws,
+ int l_idx, const struct cell *cell)
+{
+ const struct layer *layer = mt->layers[l_idx];
+ int n_vals = count_one_bits (cell->not_wild);
+ const struct variable *var = layer->factor_vars[ws->control_idx[l_idx]];
+ const union value *val = cell->values + n_vals - 1;
+ int width = var_get_width (var);
+ unsigned int hash = value_hash (val, width, 0);
+ const struct cell_container *instances = ws->instances + l_idx;
+ struct instance *inst = NULL;
+ struct instance *next;
+ HMAP_FOR_EACH_WITH_HASH_SAFE (inst, next,
+ struct instance, hmap_node,
+ hash, &instances->map)
+ {
+ if (value_equal (val, &inst->value, width))
+ break;
+ }
+ return inst;
+}
+
+/* Enter the values into PT. */
+static void
+populate_table (const struct means *means, const struct mtable *mt,
+ const struct workspace *ws,
+ const struct cell *cell,
+ struct pivot_table *pt)
+{
+ size_t *indexes = XCALLOC (pt->n_dimensions, size_t);
+ for (int v = 0; v < mt->n_dep_vars; ++v)
+ {
+ for (int s = 0; s < means->n_statistics; ++s)
+ {
+ int i = 0;
+ if (mt->n_dep_vars > 1)
+ indexes[i++] = v;
+ indexes[i++] = s;
+ int stat = means->statistics[s];
+ stat_get *sg = cell_spec[stat].sd;
+ {
+ const struct cell *pc = cell;
+ for (; i < pt->n_dimensions; ++i)
+ {
+ int l_idx = pt->n_dimensions - i - 1;
+ const struct cell_container *instances = ws->instances + l_idx;
+ if (0 == (cell->not_wild >> l_idx & 0x1U))
+ {
+ indexes [i] = hmap_count (&instances->map);
+ }
+ else
+ {
+ assert (pc);
+ const struct instance *inst
+ = lookup_instance (mt, ws, l_idx, pc);
+ assert (inst);
+ indexes [i] = inst->index;
+ pc = pc->parent_cell;
+ }
+ }
+ }
+
+ int idx = s + v * means->n_statistics;
+ struct pivot_value *pv
+ = pivot_value_new_number (sg (cell->stat[idx]));
+ if (NULL == cell_spec[stat].rc)
+ {
+ const struct variable *dv = mt->dep_vars[v];
+ pv->numeric.format = * var_get_print_format (dv);
+ }
+ pivot_table_put (pt, indexes, pt->n_dimensions, pv);
+ }
+ }
+ free (indexes);
+
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *container = cell->children + i;
+ struct cell *child = NULL;
+ BT_FOR_EACH (child, struct cell, bt_node, &container->bt)
+ {
+ populate_table (means, mt, ws, child, pt);
+ }
+ }
+}
+
+static void
+create_table_structure (const struct mtable *mt, struct pivot_table *pt,
+ const struct workspace *ws)
+{
+ int * lindexes = ws->control_idx;
+ /* The inner layers are situated rightmost in the table.
+ So this iteration is in reverse order. */
+ for (int l = mt->n_layers - 1; l >= 0; --l)
+ {
+ const struct layer *layer = mt->layers[l];
+ const struct cell_container *instances = ws->instances + l;
+ const struct variable *var = layer->factor_vars[lindexes[l]];
+ struct pivot_dimension *dim_layer
+ = pivot_dimension_create (pt, PIVOT_AXIS_ROW,
+ var_to_string (var));
+ dim_layer->root->show_label = true;
+
+ /* Place the values of the control variables as table headings. */
+ {
+ struct instance *inst = NULL;
+ BT_FOR_EACH (inst, struct instance, bt_node, &instances->bt)
+ {
+ struct substring space = SS_LITERAL_INITIALIZER ("\t ");
+ struct string str;
+ ds_init_empty (&str);
+ var_append_value_name (var,
+ &inst->value,
+ &str);
+
+ ds_ltrim (&str, space);
+
+ pivot_category_create_leaf (dim_layer->root,
+ pivot_value_new_text (ds_cstr (&str)));
+
+ ds_destroy (&str);
+ }
+ }
+
+ pivot_category_create_leaf (dim_layer->root,
+ pivot_value_new_text ("Total"));
+ }
+}
+
+/* Initialise C_DES with a string describing the control variable
+ relating to MT, LINDEXES. */
+static void
+layers_to_string (const struct mtable *mt, const int *lindexes,
+ struct string *c_des)
+{
+ for (int l = 0; l < mt->n_layers; ++l)
+ {
+ const struct layer *layer = mt->layers[l];
+ const struct variable *ctrl_var = layer->factor_vars[lindexes[l]];
+ if (l > 0)
+ ds_put_cstr (c_des, " * ");
+ ds_put_cstr (c_des, var_get_name (ctrl_var));
+ }
+}
+
+static void
+populate_case_processing_summary (struct pivot_category *pc,
+ const struct mtable *mt,
+ const int *lindexes)
+{
+ struct string ds;
+ ds_init_empty (&ds);
+ int l = 0;
+ for (l = 0; l < mt->n_layers; ++l)
+ {
+ const struct layer *layer = mt->layers[l];
+ const struct variable *ctrl_var = layer->factor_vars[lindexes[l]];
+ if (l > 0)
+ ds_put_cstr (&ds, " * ");
+ ds_put_cstr (&ds, var_get_name (ctrl_var));
+ }
+ for (int dv = 0; dv < mt->n_dep_vars; ++dv)
+ {
+ struct string dss;
+ ds_init_empty (&dss);
+ ds_put_cstr (&dss, var_get_name (mt->dep_vars[dv]));
+ if (mt->n_layers > 0)
+ {
+ ds_put_cstr (&dss, " * ");
+ ds_put_substring (&dss, ds.ss);
+ }
+ pivot_category_create_leaf (pc,
+ pivot_value_new_text (ds_cstr (&dss)));
+ ds_destroy (&dss);
+ }
+
+ ds_destroy (&ds);
+}
+
+/* Create the "Case Processing Summary" table. */
+static void
+means_case_processing_summary (const struct mtable *mt)
+{
+ struct pivot_table *pt = pivot_table_create (N_("Case Processing Summary"));
+
+ struct pivot_dimension *dim_cases =
+ pivot_dimension_create (pt, PIVOT_AXIS_COLUMN, N_("Cases"));
+ dim_cases->root->show_label = true;
+
+ struct pivot_category *cats[3];
+ cats[0] = pivot_category_create_group (dim_cases->root,
+ N_("Included"), NULL);
+ cats[1] = pivot_category_create_group (dim_cases->root,
+ N_("Excluded"), NULL);
+ cats[2] = pivot_category_create_group (dim_cases->root,
+ N_("Total"), NULL);
+ for (int i = 0; i < 3; ++i)
+ {
+ pivot_category_create_leaf_rc (cats[i],
+ pivot_value_new_text (N_("N")),
+ PIVOT_RC_COUNT);
+ pivot_category_create_leaf_rc (cats[i],
+ pivot_value_new_text (N_("Percent")),
+ PIVOT_RC_PERCENT);
+ }
+
+ struct pivot_dimension *rows =
+ pivot_dimension_create (pt, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
+ {
+ const struct workspace *ws = mt->ws + cmb;
+ populate_case_processing_summary (rows->root, mt, ws->control_idx);
+ for (int dv = 0; dv < mt->n_dep_vars; ++dv)
+ {
+ int idx = cmb * mt->n_dep_vars + dv;
+ const struct summary *summ = mt->summ + idx;
+ double n_included = summ->n_total - summ->n_missing;
+ pivot_table_put2 (pt, 5, idx,
+ pivot_value_new_number (100.0 * summ->n_total / summ->n_total));
+ pivot_table_put2 (pt, 4, idx,
+ pivot_value_new_number (summ->n_total));
+
+ pivot_table_put2 (pt, 3, idx,
+ pivot_value_new_number (100.0 * summ->n_missing / summ->n_total));
+ pivot_table_put2 (pt, 2, idx,
+ pivot_value_new_number (summ->n_missing));
+
+ pivot_table_put2 (pt, 1, idx,
+ pivot_value_new_number (100.0 * n_included / summ->n_total));
+ pivot_table_put2 (pt, 0, idx,
+ pivot_value_new_number (n_included));
+ }
+ }
+
+ pivot_table_submit (pt);
+}
+
+static void
+means_shipout_single (const struct mtable *mt, const struct means *means,
+ const struct workspace *ws)
+{
+ struct pivot_table *pt = pivot_table_create (N_("Report"));
+
+ struct pivot_dimension *dim_cells =
+ pivot_dimension_create (pt, PIVOT_AXIS_COLUMN, N_("Statistics"));
+
+ /* Set the statistics headings, eg "Mean", "Std. Dev" etc. */
+ for (int i = 0; i < means->n_statistics; ++i)
+ {
+ const struct cell_spec *cs = cell_spec + means->statistics[i];
+ pivot_category_create_leaf_rc
+ (dim_cells->root,
+ pivot_value_new_text (gettext (cs->title)), cs->rc);
+ }
+
+ create_table_structure (mt, pt, ws);
+ populate_table (means, mt, ws, ws->root_cell, pt);
+ pivot_table_submit (pt);
+}
+
+
+static void
+means_shipout_multivar (const struct mtable *mt, const struct means *means,
+ const struct workspace *ws)
+{
+ struct string dss;
+ ds_init_empty (&dss);
+ for (int dv = 0; dv < mt->n_dep_vars; ++dv)
+ {
+ if (dv > 0)
+ ds_put_cstr (&dss, " * ");
+ ds_put_cstr (&dss, var_get_name (mt->dep_vars[dv]));
+ }
+
+ for (int l = 0; l < mt->n_layers; ++l)
+ {
+ ds_put_cstr (&dss, " * ");
+ const struct layer *layer = mt->layers[l];
+ const struct variable *var = layer->factor_vars[ws->control_idx[l]];
+ ds_put_cstr (&dss, var_get_name (var));
+ }
+
+ struct pivot_table *pt = pivot_table_create (ds_cstr (&dss));
+ ds_destroy (&dss);
+
+ struct pivot_dimension *dim_cells =
+ pivot_dimension_create (pt, PIVOT_AXIS_COLUMN, N_("Variables"));
+
+ for (int i = 0; i < mt->n_dep_vars; ++i)
+ {
+ pivot_category_create_leaf
+ (dim_cells->root,
+ pivot_value_new_variable (mt->dep_vars[i]));
+ }
+
+ struct pivot_dimension *dim_stats
+ = pivot_dimension_create (pt, PIVOT_AXIS_ROW,
+ N_ ("Statistics"));
+ dim_stats->root->show_label = false;
+
+ for (int i = 0; i < means->n_statistics; ++i)
+ {
+ const struct cell_spec *cs = cell_spec + means->statistics[i];
+ pivot_category_create_leaf_rc
+ (dim_stats->root,
+ pivot_value_new_text (gettext (cs->title)), cs->rc);
+ }
+
+ create_table_structure (mt, pt, ws);
+ populate_table (means, mt, ws, ws->root_cell, pt);
+ pivot_table_submit (pt);
+}
+
+static void
+means_shipout (const struct mtable *mt, const struct means *means)
+{
+ for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
+ {
+ const struct workspace *ws = mt->ws + cmb;
+ if (ws->root_cell == NULL)
+ {
+ struct string des;
+ ds_init_empty (&des);
+ layers_to_string (mt, ws->control_idx, &des);
+ msg (MW, _("The table \"%s\" has no non-empty control variables."
+ " No result for this table will be displayed."),
+ ds_cstr (&des));
+ ds_destroy (&des);
+ continue;
+ }
+ if (mt->n_dep_vars > 1)
+ means_shipout_multivar (mt, means, ws);
+ else
+ means_shipout_single (mt, means, ws);
+ }
+}
+
+
+\f
+
+static bool
+control_var_missing (const struct means *means,
+ const struct mtable *mt,
+ unsigned int not_wild UNUSED,
+ const struct ccase *c,
+ const struct workspace *ws)
+{
+ bool miss = false;
+ for (int l = 0; l < mt->n_layers; ++l)
+ {
+ /* if (0 == ((not_wild >> l) & 0x1)) */
+ /* { */
+ /* continue; */
+ /* } */
+
+ const struct layer *layer = mt->layers[l];
+ const struct variable *var = layer->factor_vars[ws->control_idx[l]];
+ const union value *vv = case_data (c, var);
+
+ miss = (var_is_value_missing (var, vv) & means->ctrl_exclude) != 0;
+ if (miss)
+ break;
+ }
+
+ return miss;
+}
+
+/* Lookup the set of control variables described by MT, C and NOT_WILD,
+ in the hash table MAP. If there is no such entry, then create a
+ cell with these paremeters and add is to MAP.
+ If the generated cell has childen, repeat for all the children.
+ Returns the root cell.
+*/
+static struct cell *
+service_cell_map (const struct means *means, const struct mtable *mt,
+ const struct ccase *c,
+ unsigned int not_wild,
+ struct hmap *map,
+ const struct cell *pcell,
+ int level,
+ const struct workspace *ws)
+{
+ struct cell *cell = NULL;
+ if (map)
+ {
+ if (!control_var_missing (means, mt, not_wild, c, ws))
+ {
+ /* Lookup this set of values in the cell's hash table. */
+ unsigned int hash = generate_hash (mt, c, not_wild, ws);
+ cell = lookup_cell (mt, map, hash, c, not_wild, ws);
+
+ /* If it has not been seen before, then create a new
+ subcell, with this set of values, and insert it
+ into the table. */
+ if (cell == NULL)
+ {
+ cell = generate_cell (means, mt, c, not_wild, pcell, ws);
+ hmap_insert (map, &cell->hmap_node, hash);
+ }
+ }
+ }
+ else
+ {
+ /* This condition should only happen in the root node case. */
+ cell = ws->root_cell;
+ if (cell == NULL &&
+ !control_var_missing (means, mt, not_wild, c, ws))
+ cell = generate_cell (means, mt, c, not_wild, pcell, ws);
+ }
+
+ if (cell)
+ {
+ /* Here is where the business really happens! After
+ testing for missing values, the cell's statistics
+ are accumulated. */
+ if (!control_var_missing (means, mt, not_wild, c, ws))
+ {
+ for (int v = 0; v < mt->n_dep_vars; ++v)
+ {
+ const struct variable *dep_var = mt->dep_vars[v];
+ const union value *vv = case_data (c, dep_var);
+ if (var_is_value_missing (dep_var, vv) & means->dep_exclude)
+ continue;
+
+ for (int stat = 0; stat < means->n_statistics; ++stat)
+ {
+ const double weight = dict_get_case_weight (means->dict, c,
+ NULL);
+ stat_update *su = cell_spec[means->statistics[stat]].su;
+ su (cell->stat[stat + v * means->n_statistics], weight,
+ case_num (c, dep_var));
+ }
+ }
+ }
+
+ /* Recurse into all the children (if there are any). */
+ for (int i = 0; i < cell->n_children; ++i)
+ {
+ struct cell_container *cc = cell->children + i;
+ service_cell_map (means, mt, c,
+ not_wild | (0x1U << (i + level)),
+ &cc->map, cell, level + i + 1, ws);
+ }
+ }
+
+ return cell;
+}
+
+/* Do all the necessary preparation and pre-calculation that
+ needs to be done before iterating the data. */
+static void
+prepare_means (struct means *cmd)
+{
+ for (int t = 0; t < cmd->n_tables; ++t)
+ {
+ struct mtable *mt = cmd->table + t;
+
+ for (int i = 0; i < mt->n_combinations; ++i)
+ {
+ struct workspace *ws = mt->ws + i;
+ ws->root_cell = NULL;
+ ws->control_idx = xcalloc (mt->n_layers, sizeof *ws->control_idx);
+ ws->instances = xcalloc (mt->n_layers, sizeof *ws->instances);
+ int cmb = i;
+ for (int l = mt->n_layers - 1; l >= 0; --l)
+ {
+ struct cell_container *instances = ws->instances + l;
+ const struct layer *layer = mt->layers[l];
+ ws->control_idx[l] = cmb % layer->n_factor_vars;
+ cmb /= layer->n_factor_vars;
+ hmap_init (&instances->map);
+ }
+ }
+ }
+}
+
+
+/* Do all the necessary calculations that occur AFTER iterating
+ the data. */
+static void
+post_means (struct means *cmd)
+{
+ for (int t = 0; t < cmd->n_tables; ++t)
+ {
+ struct mtable *mt = cmd->table + t;
+ for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
+ {
+ struct workspace *ws = mt->ws + cmb;
+ if (ws->root_cell == NULL)
+ continue;
+ arrange_cells (ws, ws->root_cell, mt);
+ /* The root cell should have no parent. */
+ assert (ws->root_cell->parent_cell == 0);
+
+ for (int l = 0; l < mt->n_layers; ++l)
+ {
+ struct cell_container *instances = ws->instances + l;
+ bt_init (&instances->bt, compare_instance_3way, NULL);
+
+ /* Iterate the instance hash table, and insert each instance
+ into the binary tree BT. */
+ struct instance *inst;
+ HMAP_FOR_EACH (inst, struct instance, hmap_node,
+ &instances->map)
+ {
+ bt_insert (&instances->bt, &inst->bt_node);
+ }
+
+ /* Iterate the binary tree (in order) and assign the index
+ member accordingly. */
+ int index = 0;
+ BT_FOR_EACH (inst, struct instance, bt_node, &instances->bt)
+ {
+ inst->index = index++;
+ }
+ }
+ }
+ }
+}
+
+
+/* Update the summary information (the missings and the totals). */
+static void
+update_summaries (const struct means *means, struct mtable *mt,
+ const struct ccase *c, double weight)
+{
+ for (int dv = 0; dv < mt->n_dep_vars; ++dv)
+ {
+ for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
+ {
+ struct workspace *ws = mt->ws + cmb;
+ struct summary *summ = mt->summ
+ + cmb * mt->n_dep_vars + dv;
+
+ summ->n_total += weight;
+ const struct variable *var = mt->dep_vars[dv];
+ const union value *vv = case_data (c, var);
+ /* First check if the dependent variable is missing. */
+ if (var_is_value_missing (var, vv) & means->dep_exclude)
+ summ->n_missing += weight;
+ /* If the dep var is not missing, then check each
+ control variable. */
+ else
+ for (int l = 0; l < mt->n_layers; ++l)
+ {
+ const struct layer *layer = mt->layers [l];
+ const struct variable *var
+ = layer->factor_vars[ws->control_idx[l]];
+ const union value *vv = case_data (c, var);
+ if (var_is_value_missing (var, vv) & means->ctrl_exclude)
+ {
+ summ->n_missing += weight;
+ break;
+ }
+ }
+ }
+ }
+}
+
+
+void
+run_means (struct means *cmd, struct casereader *input,
+ const struct dataset *ds UNUSED)
+{
+ struct ccase *c = NULL;
+ struct casereader *reader;
+
+ prepare_means (cmd);
+
+ for (reader = input;
+ (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ const double weight
+ = dict_get_case_weight (cmd->dict, c, NULL);
+ for (int t = 0; t < cmd->n_tables; ++t)
+ {
+ struct mtable *mt = cmd->table + t;
+ update_summaries (cmd, mt, c, weight);
+
+ for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
+ {
+ struct workspace *ws = mt->ws + cmb;
+
+ ws->root_cell = service_cell_map (cmd, mt, c,
+ 0U, NULL, NULL, 0, ws);
+ }
+ }
+ }
+ casereader_destroy (reader);
+
+ post_means (cmd);
+}
+
+int
+cmd_means (struct lexer *lexer, struct dataset *ds)
+{
+ struct means means = {
+ .pool = pool_create (),
+ .ctrl_exclude = MV_ANY,
+ .dep_exclude = MV_ANY,
+ .dict = dataset_dict (ds),
+ };
+ means_set_default_statistics (&means);
+
+ if (!means_parse (lexer, &means))
+ goto error;
+
+ /* Calculate some constant data for each table. */
+ for (int t = 0; t < means.n_tables; ++t)
+ {
+ struct mtable *mt = means.table + t;
+ mt->n_combinations = 1;
+ for (int l = 0; l < mt->n_layers; ++l)
+ mt->n_combinations *= mt->layers[l]->n_factor_vars;
+ }
+
+ struct casegrouper *grouper
+ = casegrouper_create_splits (proc_open (ds), means.dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ /* Allocate the workspaces. */
+ for (int t = 0; t < means.n_tables; ++t)
+ {
+ struct mtable *mt = means.table + t;
+ mt->summ = xcalloc (mt->n_combinations * mt->n_dep_vars,
+ sizeof *mt->summ);
+ mt->ws = xcalloc (mt->n_combinations, sizeof *mt->ws);
+ }
+ run_means (&means, group, ds);
+ for (int t = 0; t < means.n_tables; ++t)
+ {
+ const struct mtable *mt = means.table + t;
+
+ means_case_processing_summary (mt);
+ means_shipout (mt, &means);
+
+ for (int i = 0; i < mt->n_combinations; ++i)
+ {
+ struct workspace *ws = mt->ws + i;
+ if (ws->root_cell)
+ means_destroy_cells (&means, ws->root_cell, mt);
+ }
+ }
+
+ /* Destroy the workspaces. */
+ for (int t = 0; t < means.n_tables; ++t)
+ {
+ struct mtable *mt = means.table + t;
+ free (mt->summ);
+ for (int i = 0; i < mt->n_combinations; ++i)
+ {
+ struct workspace *ws = mt->ws + i;
+ destroy_workspace (mt, ws);
+ }
+ free (mt->ws);
+ }
+ }
+
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+ if (!ok)
+ goto error;
+
+ pool_destroy (means.pool);
+ return CMD_SUCCESS;
+
+ error:
+ pool_destroy (means.pool);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011, 2012, 2013, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef MEANS_H
+#define MEANS_H
+
+#include "libpspp/hmap.h"
+#include "libpspp/bt.h"
+#include "libpspp/compiler.h"
+
+struct casereader;
+struct dataset;
+struct lexer;
+
+struct cell_container
+{
+ /* A hash table containing the cells. The table is indexed by a hash
+ based on the cell's categorical value. */
+ struct hmap map;
+
+ /* A binary tree containing the cells. This is
+ used to sort the elements in order of their categorical
+ values. */
+ struct bt bt;
+};
+
+
+
+struct layer
+{
+ size_t n_factor_vars;
+ const struct variable **factor_vars;
+};
+
+
+struct statistic;
+
+typedef struct statistic *stat_create (struct pool *pool);
+typedef void stat_update (struct statistic *stat, double w, double x);
+typedef double stat_get (const struct statistic *);
+typedef void stat_destroy (struct statistic *);
+
+
+struct cell_spec
+{
+ /* Printable title for output */
+ const char *title;
+
+ /* Keyword for syntax */
+ const char *keyword;
+
+ /* The result class for the datum. */
+ const char *rc;
+
+ stat_create *sc;
+ stat_update *su;
+ stat_get *sd;
+ stat_destroy *sf;
+};
+
+struct summary
+{
+ double n_total;
+ double n_missing;
+};
+
+/* Intermediate data per table. */
+struct workspace
+{
+ /* An array of n_layers integers which are used
+ to permute access into the factor_vars of each layer. */
+ int *control_idx;
+
+ /* An array of n_layers cell_containers which hold the union
+ of instances used respectively by each layer. */
+ struct cell_container *instances;
+
+ struct cell *root_cell;
+};
+
+/* The thing parsed after TABLES= */
+struct mtable
+{
+ size_t n_dep_vars;
+ const struct variable **dep_vars;
+
+ struct layer **layers;
+ int n_layers;
+
+ int n_combinations;
+
+ /* An array of n_combinations workspaces. */
+ struct workspace *ws;
+
+ /* An array of n_combinations * n_dep_vars summaries.
+ These are displayed in the Case Processing
+ Summary box. */
+ struct summary *summ;
+};
+
+/* A structure created by the parser. Contains the definition of the
+ what the procedure should calculate. */
+struct means
+{
+ const struct dictionary *dict;
+
+ /* The "tables" (ie, a definition of how the data should
+ be broken down). */
+ struct mtable *table;
+ size_t n_tables;
+
+ /* Missing value class for categorical variables. */
+ enum mv_class ctrl_exclude;
+
+ /* Missing value class for dependent variables */
+ enum mv_class dep_exclude;
+
+ /* The statistics to be calculated for each cell. */
+ int *statistics;
+ int n_statistics;
+ size_t allocated_statistics;
+
+ /* Pool on which cell functions may allocate data. */
+ struct pool *pool;
+};
+
+
+
+#define n_MEANS_STATISTICS 17
+extern const struct cell_spec cell_spec[n_MEANS_STATISTICS];
+
+/* This enum must be consistent with the array cell_spec (in means-calc.c).
+ A bitfield instead of enums would in my opinion be
+ more elegent. However we want the order of the specified
+ statistics to be retained in the output. */
+enum
+ {
+ MEANS_MEAN = 0,
+ MEANS_N,
+ MEANS_STDDEV
+ };
+
+void run_means (struct means *, struct casereader *, const struct dataset *);
+bool means_parse (struct lexer *, struct means *);
+void means_set_default_statistics (struct means *);
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <config.h>
+#include "median.h"
+
+#include <gsl/gsl_cdf.h>
+
+#include "data/format.h"
+
+
+#include "data/variable.h"
+#include "data/case.h"
+#include "data/dictionary.h"
+#include "data/dataset.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/subcase.h"
+#include "data/value.h"
+
+#include "math/percentiles.h"
+#include "math/sort.h"
+
+#include "libpspp/cast.h"
+#include "libpspp/hmap.h"
+#include "libpspp/array.h"
+#include "libpspp/str.h"
+#include "libpspp/misc.h"
+
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct val_node
+{
+ struct hmap_node node;
+ union value val;
+ casenumber le;
+ casenumber gt;
+};
+
+struct results
+{
+ const struct variable *var;
+ struct val_node **sorted_array;
+ double n;
+ double median;
+ double chisq;
+};
+
+
+
+static int
+val_node_cmp_3way (const void *a_, const void *b_, const void *aux)
+{
+ const struct variable *indep_var = aux;
+ const struct val_node *const *a = a_;
+ const struct val_node *const *b = b_;
+
+ return value_compare_3way (&(*a)->val, &(*b)->val, var_get_width (indep_var));
+}
+
+static void
+show_frequencies (const struct n_sample_test *nst, const struct results *results, int n_vals, const struct dictionary *);
+
+static void
+show_test_statistics (const struct n_sample_test *nst, const struct results *results, int, const struct dictionary *);
+
+
+static struct val_node *
+find_value (const struct hmap *map, const union value *val,
+ const struct variable *var)
+{
+ struct val_node *foo = NULL;
+ size_t hash = value_hash (val, var_get_width (var), 0);
+ HMAP_FOR_EACH_WITH_HASH (foo, struct val_node, node, hash, map)
+ if (value_equal (val, &foo->val, var_get_width (var)))
+ break;
+
+ return foo;
+}
+
+void
+median_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct variable *wvar = dict_get_weight (dict);
+ bool warn = true;
+ int v;
+ const struct median_test *mt = UP_CAST (test, const struct median_test,
+ parent.parent);
+
+ const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test,
+ parent);
+
+ const bool n_sample_test = (value_compare_3way (&nst->val2, &nst->val1,
+ var_get_width (nst->indep_var)) > 0);
+
+ struct results *results = XCALLOC (nst->n_vars, struct results);
+ int n_vals = 0;
+ for (v = 0; v < nst->n_vars; ++v)
+ {
+ double count = 0;
+ double cc = 0;
+ double median = mt->median;
+ const struct variable *var = nst->vars[v];
+ struct ccase *c;
+ struct hmap map = HMAP_INITIALIZER (map);
+ struct casereader *r = casereader_clone (input);
+
+
+
+ if (n_sample_test == false)
+ {
+ struct val_node *vn = XZALLOC (struct val_node);
+ value_clone (&vn->val, &nst->val1, var_get_width (nst->indep_var));
+ hmap_insert (&map, &vn->node, value_hash (&nst->val1,
+ var_get_width (nst->indep_var), 0));
+
+ vn = xzalloc (sizeof *vn);
+ value_clone (&vn->val, &nst->val2, var_get_width (nst->indep_var));
+ hmap_insert (&map, &vn->node, value_hash (&nst->val2,
+ var_get_width (nst->indep_var), 0));
+ }
+
+ if (median == SYSMIS)
+ {
+ struct percentile *ptl;
+ struct order_stats *os;
+
+ struct casereader *rr;
+ struct subcase sc;
+ struct casewriter *writer;
+ subcase_init_var (&sc, var, SC_ASCEND);
+ rr = casereader_clone (r);
+ writer = sort_create_writer (&sc, casereader_get_proto (rr));
+
+ for (; (c = casereader_read (rr)) != NULL;)
+ {
+ if (var_is_value_missing (var, case_data (c, var)) & exclude)
+ {
+ case_unref (c);
+ continue;
+ }
+
+ cc += dict_get_case_weight (dict, c, &warn);
+ casewriter_write (writer, c);
+ }
+ subcase_uninit (&sc);
+ casereader_destroy (rr);
+
+ rr = casewriter_make_reader (writer);
+
+ ptl = percentile_create (0.5, cc);
+ os = &ptl->parent;
+
+ order_stats_accumulate (&os, 1,
+ rr,
+ wvar,
+ var,
+ exclude);
+
+ median = percentile_calculate (ptl, PC_HAVERAGE);
+ statistic_destroy (&ptl->parent.parent);
+ }
+
+ results[v].median = median;
+
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ struct val_node *vn ;
+ const double weight = dict_get_case_weight (dict, c, &warn);
+ const union value *val = case_data (c, var);
+ const union value *indep_val = case_data (c, nst->indep_var);
+
+ if (var_is_value_missing (var, case_data (c, var)) & exclude)
+ {
+ continue;
+ }
+
+ if (n_sample_test)
+ {
+ int width = var_get_width (nst->indep_var);
+ /* Ignore out of range values */
+ if (
+ value_compare_3way (indep_val, &nst->val1, width) < 0
+ ||
+ value_compare_3way (indep_val, &nst->val2, width) > 0
+ )
+ {
+ continue;
+ }
+ }
+
+ vn = find_value (&map, indep_val, nst->indep_var);
+ if (vn == NULL)
+ {
+ if (n_sample_test == true)
+ {
+ int width = var_get_width (nst->indep_var);
+ vn = xzalloc (sizeof *vn);
+ value_clone (&vn->val, indep_val, width);
+
+ hmap_insert (&map, &vn->node, value_hash (indep_val, width, 0));
+ }
+ else
+ {
+ continue;
+ }
+ }
+
+ if (val->f <= median)
+ vn->le += weight;
+ else
+ vn->gt += weight;
+
+ count += weight;
+ }
+ casereader_destroy (r);
+
+ {
+ int x = 0;
+ struct val_node *vn = NULL;
+ double r_0 = 0;
+ double r_1 = 0;
+ HMAP_FOR_EACH (vn, struct val_node, node, &map)
+ {
+ r_0 += vn->le;
+ r_1 += vn->gt;
+ }
+
+ results[v].n = count;
+ results[v].sorted_array = XCALLOC (hmap_count (&map), struct val_node *);
+ results[v].var = var;
+
+ HMAP_FOR_EACH (vn, struct val_node, node, &map)
+ {
+ double e_0j = r_0 * (vn->le + vn->gt) / count;
+ double e_1j = r_1 * (vn->le + vn->gt) / count;
+
+ results[v].chisq += pow2 (vn->le - e_0j) / e_0j;
+ results[v].chisq += pow2 (vn->gt - e_1j) / e_1j;
+
+ results[v].sorted_array[x++] = vn;
+ }
+
+ n_vals = x;
+ hmap_destroy (&map);
+
+ sort (results[v].sorted_array, x, sizeof (*results[v].sorted_array),
+ val_node_cmp_3way, nst->indep_var);
+
+ }
+ }
+
+ casereader_destroy (input);
+
+ show_frequencies (nst, results, n_vals, dict);
+ show_test_statistics (nst, results, n_vals, dict);
+
+ for (v = 0; v < nst->n_vars; ++v)
+ {
+ int i;
+ const struct results *rs = results + v;
+
+ for (i = 0; i < n_vals; ++i)
+ {
+ struct val_node *vn = rs->sorted_array[i];
+ value_destroy (&vn->val, var_get_width (nst->indep_var));
+ free (vn);
+ }
+ free (rs->sorted_array);
+ }
+ free (results);
+}
+
+
+
+static void
+show_frequencies (const struct n_sample_test *nst, const struct results *results, int n_vals, const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (N_("Frequencies"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ struct pivot_dimension *indep = pivot_dimension_create__ (
+ table, PIVOT_AXIS_COLUMN, pivot_value_new_variable (nst->indep_var));
+ indep->root->show_label = true;
+ for (int i = 0; i < n_vals; ++i)
+ pivot_category_create_leaf_rc (
+ indep->root, pivot_value_new_var_value (
+ nst->indep_var, &results->sorted_array[i]->val), PIVOT_RC_COUNT);
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("> Median"), N_("≤ Median"));
+
+ struct pivot_dimension *dep = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ for (int v = 0; v < nst->n_vars; ++v)
+ {
+ const struct results *rs = &results[v];
+
+ int dep_idx = pivot_category_create_leaf (
+ dep->root, pivot_value_new_variable (rs->var));
+
+ for (int indep_idx = 0; indep_idx < n_vals; indep_idx++)
+ {
+ const struct val_node *vn = rs->sorted_array[indep_idx];
+ pivot_table_put3 (table, indep_idx, 0, dep_idx,
+ pivot_value_new_number (vn->gt));
+ pivot_table_put3 (table, indep_idx, 1, dep_idx,
+ pivot_value_new_number (vn->le));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_test_statistics (const struct n_sample_test *nst,
+ const struct results *results,
+ int n_vals,
+ const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Median"),
+ N_("Chi-Square"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_COUNT,
+ N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (int v = 0; v < nst->n_vars; ++v)
+ {
+ double df = n_vals - 1;
+ const struct results *rs = &results[v];
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (rs->var));
+
+ double entries[] = {
+ rs->n,
+ rs->median,
+ rs->chisq,
+ df,
+ gsl_cdf_chisq_Q (rs->chisq, df),
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ struct pivot_value *value
+ = pivot_value_new_number (entries[i]);
+ if (i == 1)
+ value->numeric.format = *var_get_print_format (rs->var);
+ pivot_table_put2 (table, i, var_idx, value);
+ }
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !median_h
+#define median_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+struct median_test
+{
+ struct n_sample_test parent;
+ double median;
+};
+
+struct casereader;
+struct dataset;
+
+void median_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer
+ );
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2013, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/data-in.h"
+#include "data/dictionary.h"
+#include "data/dataset.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "data/value.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/token.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_missing_values (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ struct missing_values mv = MV_INIT_EMPTY_NUMERIC;
+ struct variable **v = NULL;
+ size_t nv;
+
+ if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
+ goto error;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ int values_start = lex_ofs (lexer);
+ int values_end;
+ for (values_end = values_start; ; values_end++)
+ {
+ enum token_type next = lex_ofs_token (lexer, values_end + 1)->type;
+ if (next == T_RPAREN || next == T_ENDCMD || next == T_STOP)
+ break;
+ }
+
+ if (!lex_match (lexer, T_RPAREN))
+ {
+ if (var_is_numeric (v[0]))
+ {
+ while (!lex_match (lexer, T_RPAREN))
+ {
+ enum fmt_type type = var_get_print_format (v[0])->type;
+ double x, y;
+
+ if (!parse_num_range (lexer, &x, &y, &type))
+ goto error;
+
+ if (!(x == y
+ ? mv_add_num (&mv, x)
+ : mv_add_range (&mv, x, y)))
+ {
+ lex_ofs_error (lexer, values_start, values_end,
+ _("Too many numeric missing values. At "
+ "most three individual values or one "
+ "value and one range are allowed."));
+ goto error;
+ }
+
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else
+ {
+ const char *encoding = dict_get_encoding (dict);
+
+ mv_init (&mv, MV_MAX_STRING);
+ while (!lex_match (lexer, T_RPAREN))
+ {
+ if (!lex_force_string (lexer))
+ goto error;
+
+ /* Truncate the string to fit in 8 bytes in the dictionary
+ encoding. */
+ const char *utf8_s = lex_tokcstr (lexer);
+ size_t utf8_len = ss_length (lex_tokss (lexer));
+ size_t utf8_trunc_len = utf8_encoding_trunc_len (
+ utf8_s, encoding, MV_MAX_STRING);
+ if (utf8_trunc_len < utf8_len)
+ lex_error (lexer, _("Truncating missing value to maximum "
+ "acceptable length (%d bytes)."),
+ MV_MAX_STRING);
+
+ /* Recode to dictionary encoding and add. */
+ char *raw_s = recode_string (encoding, "UTF-8",
+ utf8_s, utf8_trunc_len);
+ bool ok = mv_add_str (&mv, CHAR_CAST (const uint8_t *, raw_s),
+ strlen (raw_s));
+ free (raw_s);
+ if (!ok)
+ {
+ lex_ofs_error (lexer, values_start, values_end,
+ _("Too many string missing values. "
+ "At most three individual values "
+ "are allowed."));
+ goto error;
+ }
+
+ lex_get (lexer);
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ }
+ lex_match (lexer, T_SLASH);
+
+ bool ok = true;
+ for (size_t i = 0; i < nv; i++)
+ {
+ int var_width = var_get_width (v[i]);
+
+ if (mv_is_resizable (&mv, var_width))
+ var_set_missing_values (v[i], &mv);
+ else
+ {
+ ok = false;
+ if (!var_width)
+ lex_ofs_error (lexer, values_start, values_end,
+ _("Cannot assign string missing values to "
+ "numeric variable %s."), var_get_name (v[i]));
+ else
+ lex_ofs_error (lexer, values_start, values_end,
+ _("Missing values are too long to assign "
+ "to variable %s with width %d."),
+ var_get_name (v[i]), var_get_width (v[i]));
+ }
+ }
+ mv_destroy (&mv);
+ free (v);
+ if (!ok)
+ return CMD_FAILURE;
+ continue;
+
+ error:
+ mv_destroy (&mv);
+ free (v);
+ return CMD_FAILURE;
+ }
+
+ return CMD_SUCCESS;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/mrset.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/hmap.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "libpspp/stringi-map.h"
+#include "libpspp/stringi-set.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+static bool parse_group (struct lexer *, struct dictionary *, enum mrset_type);
+static bool parse_delete (struct lexer *, struct dictionary *);
+static bool parse_display (struct lexer *, struct dictionary *);
+
+int
+cmd_mrsets (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+
+ while (lex_match (lexer, T_SLASH))
+ {
+ bool ok;
+
+ if (lex_match_id (lexer, "MDGROUP"))
+ ok = parse_group (lexer, dict, MRSET_MD);
+ else if (lex_match_id (lexer, "MCGROUP"))
+ ok = parse_group (lexer, dict, MRSET_MC);
+ else if (lex_match_id (lexer, "DELETE"))
+ ok = parse_delete (lexer, dict);
+ else if (lex_match_id (lexer, "DISPLAY"))
+ ok = parse_display (lexer, dict);
+ else
+ {
+ ok = false;
+ lex_error_expecting (lexer, "MDGROUP", "MCGROUP",
+ "DELETE", "DISPLAY");
+ }
+
+ if (!ok)
+ return CMD_FAILURE;
+ }
+
+ return CMD_SUCCESS;
+}
+
+static bool
+parse_group (struct lexer *lexer, struct dictionary *dict,
+ enum mrset_type type)
+{
+ const char *subcommand_name = type == MRSET_MD ? "MDGROUP" : "MCGROUP";
+
+ struct mrset *mrset = XZALLOC (struct mrset);
+ mrset->type = type;
+ mrset->cat_source = MRSET_VARLABELS;
+
+ bool labelsource_varlabel = false;
+ bool has_value = false;
+
+ int vars_start = 0;
+ int vars_end = 0;
+ int value_ofs = 0;
+ int labelsource_start = 0;
+ int labelsource_end = 0;
+ int label_start = 0;
+ int label_end = 0;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "NAME"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS) || !lex_force_id (lexer))
+ goto error;
+ char *error = mrset_is_valid_name__ (lex_tokcstr (lexer),
+ dict_get_encoding (dict));
+ if (error)
+ {
+ lex_error (lexer, "%s", error);
+ free (error);
+ goto error;
+ }
+
+ free (mrset->name);
+ mrset->name = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "VARIABLES"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ free (mrset->vars);
+ vars_start = lex_ofs (lexer);
+ if (!parse_variables (lexer, dict, &mrset->vars, &mrset->n_vars,
+ PV_SAME_TYPE | PV_NO_SCRATCH))
+ goto error;
+ vars_end = lex_ofs (lexer) - 1;
+
+ if (mrset->n_vars < 2)
+ {
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("At least two variables are required."));
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "LABEL"))
+ {
+ label_start = lex_ofs (lexer) - 1;
+ if (!lex_force_match (lexer, T_EQUALS) || !lex_force_string (lexer))
+ goto error;
+ label_end = lex_ofs (lexer);
+
+ free (mrset->label);
+ mrset->label = ss_xstrdup (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (type == MRSET_MD && lex_match_id (lexer, "LABELSOURCE"))
+ {
+ if (!lex_force_match_phrase (lexer, "=VARLABEL"))
+ goto error;
+
+ labelsource_varlabel = true;
+ labelsource_start = lex_ofs (lexer) - 3;
+ labelsource_end = lex_ofs (lexer) - 1;
+ }
+ else if (type == MRSET_MD && lex_match_id (lexer, "VALUE"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ has_value = true;
+ value_ofs = lex_ofs (lexer);
+ if (lex_is_number (lexer))
+ {
+ if (!lex_is_integer (lexer))
+ {
+ lex_error (lexer, _("Numeric VALUE must be an integer."));
+ goto error;
+ }
+ value_destroy (&mrset->counted, mrset->width);
+ mrset->counted.f = lex_integer (lexer);
+ mrset->width = 0;
+ }
+ else if (lex_is_string (lexer))
+ {
+ size_t width;
+ char *s;
+
+ s = recode_string (dict_get_encoding (dict), "UTF-8",
+ lex_tokcstr (lexer), -1);
+ width = strlen (s);
+
+ /* Trim off trailing spaces, but don't trim the string until
+ it's empty because a width of 0 is a numeric type. */
+ while (width > 1 && s[width - 1] == ' ')
+ width--;
+
+ value_destroy (&mrset->counted, mrset->width);
+ value_init (&mrset->counted, width);
+ memcpy (mrset->counted.s, s, width);
+ mrset->width = width;
+
+ free (s);
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting integer or string."));
+ goto error;
+ }
+ lex_get (lexer);
+ }
+ else if (type == MRSET_MD && lex_match_id (lexer, "CATEGORYLABELS"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ if (lex_match_id (lexer, "VARLABELS"))
+ mrset->cat_source = MRSET_VARLABELS;
+ else if (lex_match_id (lexer, "COUNTEDVALUES"))
+ mrset->cat_source = MRSET_COUNTEDVALUES;
+ else
+ {
+ lex_error_expecting (lexer, "VARLABELS", "COUNTEDVALUES");
+ goto error;
+ }
+ }
+ else
+ {
+ if (type == MRSET_MD)
+ lex_error_expecting (lexer, "NAME", "VARIABLES", "LABEL",
+ "LABELSOURCE", "VALUE", "CATEGORYLABELS");
+ else
+ lex_error_expecting (lexer, "NAME", "VARIABLES", "LABEL");
+ goto error;
+ }
+ }
+
+ if (mrset->name == NULL)
+ {
+ lex_spec_missing (lexer, subcommand_name, "NAME");
+ goto error;
+ }
+ else if (mrset->n_vars == 0)
+ {
+ lex_spec_missing (lexer, subcommand_name, "VARIABLES");
+ goto error;
+ }
+
+ if (type == MRSET_MD)
+ {
+ /* Check that VALUE is specified and is valid for the VARIABLES. */
+ if (!has_value)
+ {
+ lex_spec_missing (lexer, subcommand_name, "VALUE");
+ goto error;
+ }
+
+ if (var_is_alpha (mrset->vars[0]) != (mrset->width > 0))
+ {
+ msg (SE, _("VARIABLES and VALUE must have the same type."));
+ if (var_is_alpha (mrset->vars[0]))
+ lex_ofs_msg (lexer, SN, vars_start, vars_end,
+ _("These are string variables."));
+ else
+ lex_ofs_msg (lexer, SN, vars_start, vars_end,
+ _("These are numeric variables."));
+ if (mrset->width > 0)
+ lex_ofs_msg (lexer, SN, value_ofs, value_ofs,
+ _("This is a string value."));
+ else
+ lex_ofs_msg (lexer, SN, value_ofs, value_ofs,
+ _("This is a numeric value."));
+ goto error;
+ }
+ if (var_is_alpha (mrset->vars[0]))
+ {
+ const struct variable *shortest_var = NULL;
+ int min_width = INT_MAX;
+
+ for (size_t i = 0; i < mrset->n_vars; i++)
+ {
+ int width = var_get_width (mrset->vars[i]);
+ if (width < min_width)
+ {
+ shortest_var = mrset->vars[i];
+ min_width = width;
+ }
+ }
+ if (mrset->width > min_width)
+ {
+ msg (SE, _("The VALUE string must be no longer than the "
+ "narrowest variable in the group."));
+ lex_ofs_msg (lexer, SN, value_ofs, value_ofs,
+ _("The VALUE string is %d bytes long."),
+ mrset->width);
+ lex_ofs_msg (lexer, SN, vars_start, vars_end,
+ _("Variable %s has a width of %d bytes."),
+ var_get_name (shortest_var), min_width);
+ goto error;
+ }
+ }
+
+ /* Implement LABELSOURCE=VARLABEL. */
+ if (labelsource_varlabel)
+ {
+ if (mrset->cat_source != MRSET_COUNTEDVALUES)
+ lex_ofs_msg (lexer, SW, labelsource_start, labelsource_end,
+ _("MDGROUP subcommand for group %s specifies "
+ "LABELSOURCE=VARLABEL but not "
+ "CATEGORYLABELS=COUNTEDVALUES. "
+ "Ignoring LABELSOURCE."),
+ mrset->name);
+ else if (mrset->label)
+ {
+ msg (SW, _("MDGROUP subcommand for group %s specifies both "
+ "LABEL and LABELSOURCE, but only one of these "
+ "subcommands may be used at a time. "
+ "Ignoring LABELSOURCE."),
+ mrset->name);
+ lex_ofs_msg (lexer, SN, label_start, label_end,
+ _("Here is the %s setting."), "LABEL");
+ lex_ofs_msg (lexer, SN, labelsource_start, labelsource_end,
+ _("Here is the %s setting."), "LABELSOURCE");
+ }
+ else
+ {
+ mrset->label_from_var_label = true;
+ for (size_t i = 0; mrset->label == NULL && i < mrset->n_vars; i++)
+ {
+ const char *label = var_get_label (mrset->vars[i]);
+ if (label != NULL)
+ {
+ mrset->label = xstrdup (label);
+ break;
+ }
+ }
+ }
+ }
+
+ /* Warn if categories cannot be distinguished in output. */
+ if (mrset->cat_source == MRSET_VARLABELS)
+ {
+ struct stringi_map seen;
+ size_t i;
+
+ stringi_map_init (&seen);
+ for (i = 0; i < mrset->n_vars; i++)
+ {
+ const struct variable *var = mrset->vars[i];
+ const char *name = var_get_name (var);
+ const char *label = var_get_label (var);
+ if (label != NULL)
+ {
+ const char *other_name = stringi_map_find (&seen, label);
+
+ if (other_name == NULL)
+ stringi_map_insert (&seen, label, name);
+ else
+ lex_ofs_msg (lexer, SW, vars_start, vars_end,
+ _("Variables %s and %s specified as part of "
+ "multiple dichotomy group %s have the same "
+ "variable label. Categories represented by "
+ "these variables will not be distinguishable "
+ "in output."),
+ other_name, name, mrset->name);
+ }
+ }
+ stringi_map_destroy (&seen);
+ }
+ else
+ {
+ struct stringi_map seen = STRINGI_MAP_INITIALIZER (seen);
+ for (size_t i = 0; i < mrset->n_vars; i++)
+ {
+ const struct variable *var = mrset->vars[i];
+ const char *name = var_get_name (var);
+
+ union value value;
+ value_clone (&value, &mrset->counted, mrset->width);
+ value_resize (&value, mrset->width, var_get_width (var));
+
+ const struct val_labs *val_labs = var_get_value_labels (var);
+ const char *label = val_labs_find (val_labs, &value);
+ if (label == NULL)
+ lex_ofs_msg (lexer, SW, vars_start, vars_end,
+ _("Variable %s specified as part of multiple "
+ "dichotomy group %s (which has "
+ "CATEGORYLABELS=COUNTEDVALUES) has no value "
+ "label for its counted value. This category "
+ "will not be distinguishable in output."),
+ name, mrset->name);
+ else
+ {
+ const char *other_name = stringi_map_find (&seen, label);
+
+ if (other_name == NULL)
+ stringi_map_insert (&seen, label, name);
+ else
+ lex_ofs_msg (lexer, SW, vars_start, vars_end,
+ _("Variables %s and %s specified as part of "
+ "multiple dichotomy group %s (which has "
+ "CATEGORYLABELS=COUNTEDVALUES) have the same "
+ "value label for the group's counted "
+ "value. These categories will not be "
+ "distinguishable in output."),
+ other_name, name, mrset->name);
+ }
+
+ value_destroy (&value, var_get_width (var));
+ }
+ stringi_map_destroy (&seen);
+ }
+ }
+ else /* MCGROUP. */
+ {
+ /* Warn if categories cannot be distinguished in output. */
+ struct category
+ {
+ struct hmap_node hmap_node;
+ union value value;
+ int width;
+ const char *label;
+ const char *var_name;
+ bool warned;
+ };
+
+ struct hmap categories = HMAP_INITIALIZER (categories);
+ for (size_t i = 0; i < mrset->n_vars; i++)
+ {
+ const struct variable *var = mrset->vars[i];
+ const char *name = var_get_name (var);
+ int width = var_get_width (var);
+ const struct val_labs *val_labs = var_get_value_labels (var);
+
+ const struct val_lab *vl;
+ for (vl = val_labs_first (val_labs); vl != NULL;
+ vl = val_labs_next (val_labs, vl))
+ {
+ const union value *value = val_lab_get_value (vl);
+ const char *label = val_lab_get_label (vl);
+ unsigned int hash = value_hash (value, width, 0);
+
+ struct category *c;
+ HMAP_FOR_EACH_WITH_HASH (c, struct category, hmap_node,
+ hash, &categories)
+ {
+ if (width == c->width
+ && value_equal (value, &c->value, width))
+ {
+ if (!c->warned && utf8_strcasecmp (c->label, label))
+ {
+ char *s = data_out (value, var_get_encoding (var),
+ var_get_print_format (var),
+ settings_get_fmt_settings ());
+ c->warned = true;
+ lex_ofs_msg (lexer, SW, vars_start, vars_end,
+ _("Variables specified on MCGROUP should "
+ "have the same categories, but %s and "
+ "%s (and possibly others) in multiple "
+ "category group %s have different "
+ "value labels for value %s."),
+ c->var_name, name, mrset->name, s);
+ free (s);
+ }
+ goto found;
+ }
+ }
+
+ c = xmalloc (sizeof *c);
+ *c = (struct category) {
+ .width = width,
+ .label = label,
+ .var_name = name,
+ .warned = false,
+ };
+ value_clone (&c->value, value, width);
+ hmap_insert (&categories, &c->hmap_node, hash);
+
+ found: ;
+ }
+ }
+
+ struct category *c, *next;
+ HMAP_FOR_EACH_SAFE (c, next, struct category, hmap_node, &categories)
+ {
+ value_destroy (&c->value, c->width);
+ hmap_delete (&categories, &c->hmap_node);
+ free (c);
+ }
+ hmap_destroy (&categories);
+ }
+
+ dict_add_mrset (dict, mrset);
+ return true;
+
+error:
+ mrset_destroy (mrset);
+ return false;
+}
+
+static bool
+parse_mrset_names (struct lexer *lexer, struct dictionary *dict,
+ struct stringi_set *mrset_names)
+{
+ if (!lex_force_match_phrase (lexer, "NAME="))
+ return false;
+
+ stringi_set_init (mrset_names);
+ if (lex_match (lexer, T_LBRACK))
+ {
+ while (!lex_match (lexer, T_RBRACK))
+ {
+ if (!lex_force_id (lexer))
+ return false;
+ if (dict_lookup_mrset (dict, lex_tokcstr (lexer)) == NULL)
+ {
+ lex_error (lexer, _("No multiple response set named %s."),
+ lex_tokcstr (lexer));
+ stringi_set_destroy (mrset_names);
+ return false;
+ }
+ stringi_set_insert (mrset_names, lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ }
+ else if (lex_match (lexer, T_ALL))
+ {
+ size_t n_sets = dict_get_n_mrsets (dict);
+ size_t i;
+
+ for (i = 0; i < n_sets; i++)
+ stringi_set_insert (mrset_names, dict_get_mrset (dict, i)->name);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "`['", "ALL");
+ return false;
+ }
+
+ return true;
+}
+
+static bool
+parse_delete (struct lexer *lexer, struct dictionary *dict)
+{
+ struct stringi_set mrset_names;
+ const char *name;
+ if (!parse_mrset_names (lexer, dict, &mrset_names))
+ return false;
+
+ const struct stringi_set_node *node;
+ STRINGI_SET_FOR_EACH (name, node, &mrset_names)
+ dict_delete_mrset (dict, name);
+ stringi_set_destroy (&mrset_names);
+
+ return true;
+}
+
+static bool
+parse_display (struct lexer *lexer, struct dictionary *dict)
+{
+ struct stringi_set mrset_names_set;
+ if (!parse_mrset_names (lexer, dict, &mrset_names_set))
+ return false;
+
+ size_t n = stringi_set_count (&mrset_names_set);
+ if (n == 0)
+ {
+ if (dict_get_n_mrsets (dict) == 0)
+ lex_next_msg (lexer, SN, -1, -1,
+ _("The active dataset dictionary does not contain any "
+ "multiple response sets."));
+ stringi_set_destroy (&mrset_names_set);
+ return true;
+ }
+
+ struct pivot_table *table = pivot_table_create (
+ N_("Multiple Response Sets"));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Attributes"),
+ N_("Label"), N_("Encoding"), N_("Counted Value"), N_("Member Variables"));
+
+ struct pivot_dimension *mrsets = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Name"));
+ mrsets->root->show_label = true;
+
+ char **mrset_names = stringi_set_get_sorted_array (&mrset_names_set);
+ for (size_t i = 0; i < n; i++)
+ {
+ const struct mrset *mrset = dict_lookup_mrset (dict, mrset_names[i]);
+
+ int row = pivot_category_create_leaf (
+ mrsets->root, pivot_value_new_user_text (mrset->name, -1));
+
+ if (mrset->label != NULL)
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_user_text (mrset->label, -1));
+
+ pivot_table_put2 (table, 1, row,
+ pivot_value_new_text (mrset->type == MRSET_MD
+ ? _("Dichotomies")
+ : _("Categories")));
+
+ if (mrset->type == MRSET_MD)
+ pivot_table_put2 (table, 2, row,
+ pivot_value_new_value (
+ &mrset->counted, mrset->width,
+ &F_8_0, dict_get_encoding (dict)));
+
+ /* Variable names. */
+ struct string var_names = DS_EMPTY_INITIALIZER;
+ for (size_t j = 0; j < mrset->n_vars; j++)
+ ds_put_format (&var_names, "%s\n", var_get_name (mrset->vars[j]));
+ ds_chomp_byte (&var_names, '\n');
+ pivot_table_put2 (table, 3, row,
+ pivot_value_new_user_text_nocopy (
+ ds_steal_cstr (&var_names)));
+ }
+ free (mrset_names);
+ stringi_set_destroy (&mrset_names_set);
+
+ pivot_table_submit (table);
+
+ return true;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/npar-summary.h"
+
+#include <math.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "math/moments.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+void
+npar_summary_calc_descriptives (struct descriptives *desc,
+ struct casereader *input,
+ const struct dictionary *dict,
+ const struct variable *const *vv,
+ int n_vars,
+ enum mv_class filter)
+{
+ int i = 0;
+ for (i = 0 ; i < n_vars; ++i)
+ {
+ double minimum = DBL_MAX;
+ double maximum = -DBL_MAX;
+ double var;
+ struct moments1 *moments = moments1_create (MOMENT_VARIANCE);
+ struct ccase *c;
+ const struct variable *v = vv[i];
+ struct casereader *pass;
+
+ pass = casereader_clone (input);
+ pass = casereader_create_filter_missing (pass,
+ &v, 1,
+ filter, NULL, NULL);
+ pass = casereader_create_filter_weight (pass, dict, NULL, NULL);
+ while ((c = casereader_read (pass)) != NULL)
+ {
+ double val = case_num (c, v);
+ double w = dict_get_case_weight (dict, c, NULL);
+ minimum = MIN (minimum, val);
+ maximum = MAX (maximum, val);
+ moments1_add (moments, val, w);
+ case_unref (c);
+ }
+ casereader_destroy (pass);
+
+ moments1_calculate (moments,
+ &desc[i].n,
+ &desc[i].mean,
+ &var,
+ NULL, NULL);
+
+ desc[i].std_dev = sqrt (var);
+
+ moments1_destroy (moments);
+
+ desc[i].min = minimum;
+ desc[i].max = maximum;
+ }
+
+ casereader_destroy (input);
+}
+
+
+
+void
+do_summary_box (const struct descriptives *desc,
+ const struct variable *const *vv,
+ int n_vars,
+ const struct fmt_spec *wfmt)
+{
+ if (!desc)
+ return;
+
+ struct pivot_table *table = pivot_table_create (
+ N_("Descriptive Statistics"));
+ pivot_table_set_weight_format (table, wfmt);
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("Minimum"), N_("Maximum"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (int v = 0; v < n_vars; ++v)
+ {
+ const struct variable *var = vv[v];
+
+ int row = pivot_category_create_leaf (variables->root,
+ pivot_value_new_variable (var));
+
+ double entries[] = { desc[v].n, desc[v].mean, desc[v].std_dev };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, row, pivot_value_new_number (entries[j]));
+
+ union value extrema[2] = { { .f = desc[v].min }, { .f = desc[v].max } };
+ for (size_t j = 0; j < 2; j++)
+ pivot_table_put2 (table, 3 + j, row,
+ pivot_value_new_var_value (var, &extrema[j]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !n_par_summary_h
+#define n_par_summary_h 1
+
+#include "data/missing-values.h"
+
+struct variable ;
+struct casereader ;
+struct dictionary;
+struct fmt_spec;
+
+struct descriptives
+{
+ double n;
+ double mean;
+ double std_dev;
+ double min;
+ double max;
+};
+
+void
+do_summary_box (const struct descriptives *desc,
+ const struct variable *const *vv,
+ int n_vars,
+ const struct fmt_spec *wfmt);
+
+void npar_summary_calc_descriptives (struct descriptives *desc,
+ struct casereader *input,
+ const struct dictionary *dict,
+ const struct variable *const *vv,
+ int n_vars,
+ enum mv_class filter);
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis. -*-c-*-
+ Copyright (C) 2006, 2008, 2009, 2010, 2011, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/npar.h"
+
+#include <stdlib.h>
+#include <math.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/settings.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/binomial.h"
+#include "language/commands/chisquare.h"
+#include "language/commands/ks-one-sample.h"
+#include "language/commands/cochran.h"
+#include "language/commands/friedman.h"
+#include "language/commands/jonckheere-terpstra.h"
+#include "language/commands/kruskal-wallis.h"
+#include "language/commands/mann-whitney.h"
+#include "language/commands/mcnemar.h"
+#include "language/commands/median.h"
+#include "language/commands/npar-summary.h"
+#include "language/commands/runs.h"
+#include "language/commands/sign.h"
+#include "language/commands/wilcoxon.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmapx.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+#include "libpspp/taint.h"
+#include "math/moments.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* NPAR TESTS structure. */
+struct npar_specs
+{
+ struct pool *pool;
+ struct npar_test **test;
+ size_t n_tests;
+
+ const struct variable **vv; /* Compendium of all variables
+ (those mentioned on ANY subcommand */
+ int n_vars; /* Number of variables in vv */
+
+ enum mv_class filter; /* Missing values to filter. */
+ bool listwise_missing;
+
+ bool descriptives; /* Descriptive statistics should be calculated */
+ bool quartiles; /* Quartiles should be calculated */
+
+ bool exact; /* Whether exact calculations have been requested */
+ double timer; /* Maximum time (in minutes) to wait for exact calculations */
+};
+
+
+/* Prototype for custom subcommands of NPAR TESTS. */
+static bool npar_chisquare (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_binomial (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_ks_one_sample (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_runs (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_friedman (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_kendall (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_cochran (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_wilcoxon (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_sign (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_kruskal_wallis (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_jonckheere_terpstra (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_mann_whitney (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_mcnemar (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_median (struct lexer *, struct dataset *, struct npar_specs *);
+static bool npar_method (struct lexer *, struct npar_specs *);
+
+/* Command parsing functions. */
+
+static int
+parse_npar_tests (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *nps)
+{
+ bool seen_missing = false;
+ bool seen_method = false;
+ lex_match (lexer, T_SLASH);
+ do
+ {
+ if (lex_match_id (lexer, "COCHRAN"))
+ {
+ if (!npar_cochran (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "FRIEDMAN"))
+ {
+ if (!npar_friedman (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "KENDALL"))
+ {
+ if (!npar_kendall (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "RUNS"))
+ {
+ if (!npar_runs (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "CHISQUARE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_chisquare (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "BINOMIAL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_binomial (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_phrase (lexer, "K-S") ||
+ lex_match_phrase (lexer, "KOLMOGOROV-SMIRNOV"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_ks_one_sample (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_phrase (lexer, "J-T") ||
+ lex_match_phrase (lexer, "JONCKHEERE-TERPSTRA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_jonckheere_terpstra (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_phrase (lexer, "K-W") ||
+ lex_match_phrase (lexer, "KRUSKAL-WALLIS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_kruskal_wallis (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_phrase (lexer, "MCNEMAR"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_mcnemar (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_phrase (lexer, "M-W") ||
+ lex_match_phrase (lexer, "MANN-WHITNEY"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_mann_whitney (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_phrase (lexer, "MEDIAN"))
+ {
+ if (!npar_median (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "WILCOXON"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_wilcoxon (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "SIGN"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!npar_sign (lexer, ds, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (seen_missing)
+ {
+ lex_sbc_only_once (lexer, "MISSING");
+ return false;
+ }
+ seen_missing = true;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "ANALYSIS"))
+ nps->listwise_missing = false;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ nps->listwise_missing = true;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ nps->filter = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ nps->filter = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "ANALYSIS", "LISTWISE",
+ "INCLUDE", "EXCLUDE");
+ return false;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "METHOD"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (seen_method)
+ {
+ lex_sbc_only_once (lexer, "METHOD");
+ return false;
+ }
+ seen_method = true;
+ if (!npar_method (lexer, nps))
+ return false;
+ }
+ else if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "DESCRIPTIVES"))
+ nps->descriptives = true;
+ else if (lex_match_id (lexer, "QUARTILES"))
+ nps->quartiles = true;
+ else if (lex_match (lexer, T_ALL))
+ nps->descriptives = nps->quartiles = true;
+ else
+ {
+ lex_error_expecting (lexer, "DESCRIPTIVES", "QUARTILES",
+ "ALL");
+ return false;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "ALGORITHM"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "COMPATIBLE"))
+ settings_set_cmd_algorithm (COMPATIBLE);
+ else if (lex_match_id (lexer, "ENHANCED"))
+ settings_set_cmd_algorithm (ENHANCED);
+ else
+ {
+ lex_error_expecting (lexer, "COMPATIBLE", "ENHANCED");
+ return false;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "COCHRAN", "FRIEDMAN", "KENDALL", "RUNS",
+ "CHISQUARE", "BINOMIAL", "K-S", "J-T", "K-W",
+ "MCNEMAR", "M-W", "MEDIAN", "WILCOXON",
+ "SIGN", "MISSING", "METHOD", "STATISTICS",
+ "ALGORITHM");
+ return false;
+ }
+ }
+ while (lex_match (lexer, T_SLASH));
+
+ return true;
+}
+
+static void one_sample_insert_variables (const struct npar_test *test,
+ struct hmapx *);
+
+static void two_sample_insert_variables (const struct npar_test *test,
+ struct hmapx *);
+
+static void n_sample_insert_variables (const struct npar_test *test,
+ struct hmapx *);
+
+static void
+npar_execute (struct casereader *input,
+ const struct npar_specs *specs,
+ const struct dataset *ds)
+{
+ struct descriptives *summary_descriptives = NULL;
+
+ for (size_t t = 0; t < specs->n_tests; ++t)
+ {
+ const struct npar_test *test = specs->test[t];
+ test->execute (ds, casereader_clone (input), specs->filter,
+ test, specs->exact, specs->timer);
+ }
+
+ if (specs->descriptives && specs->n_vars > 0)
+ {
+ summary_descriptives = xnmalloc (sizeof (*summary_descriptives),
+ specs->n_vars);
+
+ npar_summary_calc_descriptives (summary_descriptives,
+ casereader_clone (input),
+ dataset_dict (ds),
+ specs->vv, specs->n_vars,
+ specs->filter);
+ }
+
+ if ((specs->descriptives || specs->quartiles)
+ && !taint_has_tainted_successor (casereader_get_taint (input)))
+ do_summary_box (summary_descriptives, specs->vv, specs->n_vars,
+ dict_get_weight_format (dataset_dict (ds)));
+
+ free (summary_descriptives);
+ casereader_destroy (input);
+}
+
+int
+cmd_npar_tests (struct lexer *lexer, struct dataset *ds)
+{
+ struct npar_specs npar_specs = {
+ .pool = pool_create (),
+ .filter = MV_ANY,
+ .listwise_missing = false,
+ };
+
+ if (!parse_npar_tests (lexer, ds, &npar_specs))
+ {
+ pool_destroy (npar_specs.pool);
+ return CMD_FAILURE;
+ }
+
+ struct hmapx var_map = HMAPX_INITIALIZER (var_map);
+ for (size_t i = 0; i < npar_specs.n_tests; ++i)
+ {
+ const struct npar_test *test = npar_specs.test[i];
+ test->insert_variables (test, &var_map);
+ }
+
+ struct hmapx_node *node;
+ struct variable *var;
+ npar_specs.vv = pool_alloc (npar_specs.pool,
+ hmapx_count (&var_map) * sizeof *npar_specs.vv);
+ HMAPX_FOR_EACH (var, node, &var_map)
+ npar_specs.vv[npar_specs.n_vars++] = var;
+ assert (npar_specs.n_vars == hmapx_count (&var_map));
+
+ sort (npar_specs.vv, npar_specs.n_vars, sizeof *npar_specs.vv,
+ compare_var_ptrs_by_name, NULL);
+
+ struct casereader *input = proc_open (ds);
+ if (npar_specs.listwise_missing)
+ input = casereader_create_filter_missing (input,
+ npar_specs.vv,
+ npar_specs.n_vars,
+ npar_specs.filter,
+ NULL, NULL);
+
+ struct casegrouper *grouper = casegrouper_create_splits (input, dataset_dict (ds));
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ npar_execute (group, &npar_specs, ds);
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ pool_destroy (npar_specs.pool);
+ hmapx_destroy (&var_map);
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+}
+
+static void
+add_test (struct npar_specs *specs, struct npar_test *nt)
+{
+ specs->test = pool_realloc (specs->pool, specs->test,
+ (specs->n_tests + 1) * sizeof *specs->test);
+
+ specs->test[specs->n_tests++] = nt;
+}
+
+static bool
+npar_runs (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct runs_test *rt = pool_alloc (specs->pool, sizeof (*rt));
+ struct one_sample_test *tp = &rt->parent;
+ struct npar_test *nt = &tp->parent;
+
+ nt->execute = runs_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ return false;
+
+ if (lex_match_id (lexer, "MEAN"))
+ rt->cp_mode = CP_MEAN;
+ else if (lex_match_id (lexer, "MEDIAN"))
+ rt->cp_mode = CP_MEDIAN;
+ else if (lex_match_id (lexer, "MODE"))
+ rt->cp_mode = CP_MODE;
+ else if (lex_is_number (lexer))
+ {
+ rt->cutpoint = lex_number (lexer);
+ rt->cp_mode = CP_CUSTOM;
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting %s, %s, %s or a number."),
+ "MEAN", "MEDIAN", "MODE");
+ return false;
+ }
+
+ if (!lex_force_match_phrase (lexer, ")="))
+ return false;
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &tp->vars, &tp->n_vars,
+ PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_friedman (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct friedman_test *ft = pool_alloc (specs->pool, sizeof (*ft));
+ struct one_sample_test *ost = &ft->parent;
+ struct npar_test *nt = &ost->parent;
+
+ ft->kendalls_w = false;
+ nt->execute = friedman_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ lex_match (lexer, T_EQUALS);
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &ost->vars, &ost->n_vars,
+ PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_kendall (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct friedman_test *kt = pool_alloc (specs->pool, sizeof (*kt));
+ struct one_sample_test *ost = &kt->parent;
+ struct npar_test *nt = &ost->parent;
+
+ kt->kendalls_w = true;
+ nt->execute = friedman_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ lex_match (lexer, T_EQUALS);
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &ost->vars, &ost->n_vars,
+ PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+
+static bool
+npar_cochran (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct one_sample_test *ft = pool_alloc (specs->pool, sizeof (*ft));
+ struct npar_test *nt = &ft->parent;
+
+ nt->execute = cochran_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ lex_match (lexer, T_EQUALS);
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &ft->vars, &ft->n_vars,
+ PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_chisquare (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct chisquare_test *cstp = pool_alloc (specs->pool, sizeof (*cstp));
+ struct one_sample_test *tp = &cstp->parent;
+ struct npar_test *nt = &tp->parent;
+
+ nt->execute = chisquare_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &tp->vars, &tp->n_vars,
+ PV_NO_SCRATCH | PV_NO_DUPLICATE))
+ return false;
+
+ cstp->ranged = false;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ cstp->ranged = true;
+ if (!lex_force_num (lexer))
+ return false;
+ cstp->lo = lex_number (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_COMMA))
+ return false;
+ if (!lex_force_num_range_open (lexer, "HI", cstp->lo, DBL_MAX))
+ return false;
+ cstp->hi = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+
+ cstp->n_expected = 0;
+ cstp->expected = NULL;
+ int expected_start = 0;
+ int expected_end = 0;
+ if (lex_match_phrase (lexer, "/EXPECTED"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ return false;
+
+ if (!lex_match_id (lexer, "EQUAL"))
+ {
+ expected_start = lex_ofs (lexer);
+ while (lex_is_number (lexer))
+ {
+ int n = 1;
+ double f = lex_number (lexer);
+ lex_get (lexer);
+ if (lex_match (lexer, T_ASTERISK))
+ {
+ n = f;
+ if (!lex_force_num (lexer))
+ return false;
+ f = lex_number (lexer);
+ lex_get (lexer);
+ }
+ lex_match (lexer, T_COMMA);
+
+ cstp->n_expected += n;
+ cstp->expected = pool_realloc (specs->pool,
+ cstp->expected,
+ sizeof (double) * cstp->n_expected);
+ for (int i = cstp->n_expected - n; i < cstp->n_expected; ++i)
+ cstp->expected[i] = f;
+ }
+ expected_end = lex_ofs (lexer) - 1;
+ }
+ }
+
+ if (cstp->ranged && cstp->n_expected > 0 &&
+ cstp->n_expected != cstp->hi - cstp->lo + 1)
+ {
+ lex_ofs_error (lexer, expected_start, expected_end,
+ _("%d expected values were given, but the specified "
+ "range (%d-%d) requires exactly %d values."),
+ cstp->n_expected, cstp->lo, cstp->hi,
+ cstp->hi - cstp->lo +1);
+ return false;
+ }
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_binomial (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct binomial_test *btp = pool_alloc (specs->pool, sizeof (*btp));
+ struct one_sample_test *tp = &btp->parent;
+ struct npar_test *nt = &tp->parent;
+
+ nt->execute = binomial_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ btp->category1 = btp->category2 = btp->cutpoint = SYSMIS;
+
+ btp->p = 0.5;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ btp->p = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ if (!lex_force_match (lexer, T_EQUALS))
+ return false;
+ }
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &tp->vars, &tp->n_vars,
+ PV_NUMERIC | PV_NO_SCRATCH | PV_NO_DUPLICATE))
+ return false;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ btp->category1 = lex_number (lexer);
+ lex_get (lexer);
+ if (lex_match (lexer, T_COMMA))
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ btp->category2 = lex_number (lexer);
+ lex_get (lexer);
+ }
+ else
+ btp->cutpoint = btp->category1;
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+
+ add_test (specs, nt);
+ return true;
+}
+
+static void
+ks_one_sample_parse_params (struct lexer *lexer, struct ks_one_sample_test *kst, int params)
+{
+ assert (params == 1 || params == 2);
+
+ if (lex_is_number (lexer))
+ {
+ kst->p[0] = lex_number (lexer);
+
+ lex_get (lexer);
+ if (params == 2)
+ {
+ lex_match (lexer, T_COMMA);
+ if (lex_force_num (lexer))
+ {
+ kst->p[1] = lex_number (lexer);
+ lex_get (lexer);
+ }
+ }
+ }
+}
+
+static bool
+npar_ks_one_sample (struct lexer *lexer, struct dataset *ds, struct npar_specs *specs)
+{
+ struct ks_one_sample_test *kst = pool_alloc (specs->pool, sizeof (*kst));
+ struct one_sample_test *tp = &kst->parent;
+ struct npar_test *nt = &tp->parent;
+
+ nt->execute = ks_one_sample_execute;
+ nt->insert_variables = one_sample_insert_variables;
+
+ kst->p[0] = kst->p[1] = SYSMIS;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ return false;
+
+ if (lex_match_id (lexer, "NORMAL"))
+ {
+ kst->dist = KS_NORMAL;
+ ks_one_sample_parse_params (lexer, kst, 2);
+ }
+ else if (lex_match_id (lexer, "POISSON"))
+ {
+ kst->dist = KS_POISSON;
+ ks_one_sample_parse_params (lexer, kst, 1);
+ }
+ else if (lex_match_id (lexer, "UNIFORM"))
+ {
+ kst->dist = KS_UNIFORM;
+ ks_one_sample_parse_params (lexer, kst, 2);
+ }
+ else if (lex_match_id (lexer, "EXPONENTIAL"))
+ {
+ kst->dist = KS_EXPONENTIAL;
+ ks_one_sample_parse_params (lexer, kst, 1);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "NORMAL", "POISSON", "UNIFORM",
+ "EXPONENTIAL");
+ return false;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+
+ lex_match (lexer, T_EQUALS);
+
+ if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
+ &tp->vars, &tp->n_vars,
+ PV_NUMERIC | PV_NO_SCRATCH | PV_NO_DUPLICATE))
+ return false;
+
+ add_test (specs, nt);
+
+ return true;
+}
+
+static bool
+parse_two_sample_related_test (struct lexer *lexer,
+ const struct dictionary *dict,
+ struct two_sample_test *tp,
+ struct pool *pool)
+{
+ tp->parent.insert_variables = two_sample_insert_variables;
+
+ const struct variable **v1;
+ size_t n1;
+ int vars_start = lex_ofs (lexer);
+ if (!parse_variables_const_pool (lexer, pool, dict, &v1, &n1,
+ PV_NUMERIC | PV_NO_SCRATCH | PV_DUPLICATE))
+ return false;
+
+ bool with = false;
+ bool paired = false;
+ const struct variable **v2 = NULL;
+ size_t n2 = 0;
+ if (lex_match (lexer, T_WITH))
+ {
+ with = true;
+ if (!parse_variables_const_pool (lexer, pool, dict, &v2, &n2,
+ PV_NUMERIC | PV_NO_SCRATCH | PV_DUPLICATE))
+ return false;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_match_phrase (lexer, "PAIRED)"))
+ return false;
+ paired = true;
+
+ if (n1 != n2)
+ {
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("PAIRED was specified, but the number of "
+ "variables preceding WITH (%zu) does not match "
+ "the number following (%zu)."),
+ n1, n2);
+ return false;
+ }
+ }
+ }
+
+ tp->n_pairs = (paired ? n1
+ : with ? n1 * n2
+ : (n1 * (n1 - 1)) / 2);
+ tp->pairs = pool_alloc (pool, sizeof (variable_pair) * tp->n_pairs);
+
+ size_t n = 0;
+ if (!with)
+ for (size_t i = 0; i < n1 - 1; ++i)
+ for (size_t j = i + 1; j < n1; ++j)
+ {
+ assert (n < tp->n_pairs);
+ tp->pairs[n][0] = v1[i];
+ tp->pairs[n][1] = v1[j];
+ n++;
+ }
+ else if (paired)
+ {
+ assert (n1 == n2);
+ for (size_t i = 0; i < n1; ++i)
+ {
+ tp->pairs[n][0] = v1[i];
+ tp->pairs[n][1] = v2[i];
+ n++;
+ }
+ }
+ else
+ {
+ for (size_t i = 0; i < n1; ++i)
+ for (size_t j = 0; j < n2; ++j)
+ {
+ tp->pairs[n][0] = v1[i];
+ tp->pairs[n][1] = v2[j];
+ n++;
+ }
+ }
+ assert (n == tp->n_pairs);
+
+ return true;
+}
+
+static bool
+parse_n_sample_related_test (struct lexer *lexer, const struct dictionary *dict,
+ struct n_sample_test *nst, struct pool *pool)
+{
+ if (!parse_variables_const_pool (lexer, pool, dict, &nst->vars, &nst->n_vars,
+ PV_NUMERIC | PV_NO_SCRATCH | PV_NO_DUPLICATE))
+ return false;
+
+ if (!lex_force_match (lexer, T_BY))
+ return false;
+
+ nst->indep_var = parse_variable_const (lexer, dict);
+ if (!nst->indep_var)
+ return false;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ return false;
+
+ value_init (&nst->val1, var_get_width (nst->indep_var));
+ if (!parse_value (lexer, &nst->val1, nst->indep_var))
+ {
+ value_destroy (&nst->val1, var_get_width (nst->indep_var));
+ return false;
+ }
+
+ lex_match (lexer, T_COMMA);
+
+ value_init (&nst->val2, var_get_width (nst->indep_var));
+ if (!parse_value (lexer, &nst->val2, nst->indep_var))
+ {
+ value_destroy (&nst->val2, var_get_width (nst->indep_var));
+ return false;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+
+ return true;
+}
+
+static bool
+npar_wilcoxon (struct lexer *lexer,
+ struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct two_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
+ struct npar_test *nt = &tp->parent;
+ nt->execute = wilcoxon_execute;
+
+ if (!parse_two_sample_related_test (lexer, dataset_dict (ds),
+ tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_mann_whitney (struct lexer *lexer,
+ struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct n_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
+ struct npar_test *nt = &tp->parent;
+
+ nt->insert_variables = n_sample_insert_variables;
+ nt->execute = mann_whitney_execute;
+
+ if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_median (struct lexer *lexer,
+ struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct median_test *mt = pool_alloc (specs->pool, sizeof (*mt));
+ struct n_sample_test *tp = &mt->parent;
+ struct npar_test *nt = &tp->parent;
+
+ mt->median = SYSMIS;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ mt->median = lex_number (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ nt->insert_variables = n_sample_insert_variables;
+ nt->execute = median_execute;
+
+ if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_sign (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct two_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
+ struct npar_test *nt = &tp->parent;
+
+ nt->execute = sign_execute;
+
+ if (!parse_two_sample_related_test (lexer, dataset_dict (ds),
+ tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_mcnemar (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct two_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
+ struct npar_test *nt = &tp->parent;
+
+ nt->execute = mcnemar_execute;
+
+ if (!parse_two_sample_related_test (lexer, dataset_dict (ds),
+ tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+
+static bool
+npar_jonckheere_terpstra (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct n_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
+ struct npar_test *nt = &tp->parent;
+
+ nt->insert_variables = n_sample_insert_variables;
+ nt->execute = jonckheere_terpstra_execute;
+
+ if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static bool
+npar_kruskal_wallis (struct lexer *lexer, struct dataset *ds,
+ struct npar_specs *specs)
+{
+ struct n_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
+ struct npar_test *nt = &tp->parent;
+
+ nt->insert_variables = n_sample_insert_variables;
+
+ nt->execute = kruskal_wallis_execute;
+
+ if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
+ return false;
+
+ add_test (specs, nt);
+ return true;
+}
+
+static void
+insert_variable_into_map (struct hmapx *var_map, const struct variable *var)
+{
+ size_t hash = hash_pointer (var, 0);
+ struct hmapx_node *node;
+ const struct variable *v = NULL;
+
+ HMAPX_FOR_EACH_WITH_HASH (v, node, hash, var_map)
+ if (v == var)
+ return;
+
+ hmapx_insert (var_map, CONST_CAST (struct variable *, var), hash);
+}
+
+/* Insert the variables for TEST into VAR_MAP */
+static void
+one_sample_insert_variables (const struct npar_test *test,
+ struct hmapx *var_map)
+{
+ const struct one_sample_test *ost = UP_CAST (test, const struct one_sample_test, parent);
+
+ for (size_t i = 0; i < ost->n_vars; ++i)
+ insert_variable_into_map (var_map, ost->vars[i]);
+}
+
+
+static void
+two_sample_insert_variables (const struct npar_test *test,
+ struct hmapx *var_map)
+{
+ const struct two_sample_test *tst = UP_CAST (test, const struct two_sample_test, parent);
+
+ for (size_t i = 0; i < tst->n_pairs; ++i)
+ {
+ variable_pair *pair = &tst->pairs[i];
+
+ insert_variable_into_map (var_map, (*pair)[0]);
+ insert_variable_into_map (var_map, (*pair)[1]);
+ }
+}
+
+static void
+n_sample_insert_variables (const struct npar_test *test,
+ struct hmapx *var_map)
+{
+ const struct n_sample_test *tst = UP_CAST (test, const struct n_sample_test, parent);
+
+ for (size_t i = 0; i < tst->n_vars; ++i)
+ insert_variable_into_map (var_map, tst->vars[i]);
+
+ insert_variable_into_map (var_map, tst->indep_var);
+}
+
+static bool
+npar_method (struct lexer *lexer, struct npar_specs *specs)
+{
+ if (lex_match_id (lexer, "EXACT"))
+ {
+ specs->exact = true;
+ specs->timer = 0.0;
+ if (lex_match_id (lexer, "TIMER"))
+ {
+ specs->timer = 5.0;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ specs->timer = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !npar_h
+#define npar_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "data/missing-values.h"
+#include "data/value.h"
+
+typedef const struct variable *variable_pair[2];
+
+struct hmapx;
+struct casefilter;
+struct casereader;
+struct dataset;
+
+
+struct npar_test
+{
+ void (*execute) (const struct dataset *,
+ struct casereader *,
+ enum mv_class exclude,
+ const struct npar_test *,
+ bool,
+ double);
+
+ void (*insert_variables) (const struct npar_test *,
+ struct hmapx *);
+};
+
+
+struct one_sample_test
+{
+ struct npar_test parent;
+ const struct variable **vars;
+ size_t n_vars;
+};
+
+struct two_sample_test
+{
+ struct npar_test parent;
+ variable_pair *pairs;
+ size_t n_pairs;
+};
+
+
+struct n_sample_test
+{
+ struct npar_test parent;
+ const struct variable **vars;
+ size_t n_vars;
+
+ union value val1, val2;
+ const struct variable *indep_var;
+};
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2010, 2011, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "data/format.h"
+#include "language/command.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Parses the NUMERIC command. */
+int
+cmd_numeric (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ char **v;
+ size_t nv;
+ int vars_start = lex_ofs (lexer);
+ if (!parse_DATA_LIST_vars (lexer, dataset_dict (ds),
+ &v, &nv, PV_NO_DUPLICATE))
+ return CMD_FAILURE;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ bool ok = false;
+
+ /* Get the optional format specification. */
+ struct fmt_spec f = var_default_formats (0);
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!parse_format_specifier (lexer, &f))
+ goto done;
+
+ char *error = fmt_check_output__ (&f);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ goto done;
+ }
+
+ if (fmt_is_string (f.type))
+ {
+ char str[FMT_STRING_LEN_MAX + 1];
+ lex_next_error (lexer, -1, -1,
+ _("Format type %s may not be used with a numeric "
+ "variable."), fmt_to_string (&f, str));
+ goto done;
+ }
+
+ if (!lex_match (lexer, T_RPAREN))
+ {
+ lex_error_expecting (lexer, "`)'");
+ goto done;
+ }
+ }
+
+ /* Create each variable. */
+ for (size_t i = 0; i < nv; i++)
+ {
+ struct variable *new_var = dict_create_var (dataset_dict (ds),
+ v[i], 0);
+ if (!new_var)
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("There is already a variable named %s."), v[i]);
+ else
+ var_set_both_formats (new_var, &f);
+ }
+ ok = true;
+
+ done:
+ for (size_t i = 0; i < nv; i++)
+ free (v[i]);
+ free (v);
+ if (!ok)
+ return CMD_FAILURE;
+ }
+ while (lex_match (lexer, T_SLASH));
+
+ return CMD_SUCCESS;
+}
+
+/* Parses the STRING command. */
+int
+cmd_string (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ char **v;
+ size_t nv;
+ int vars_start = lex_ofs (lexer);
+ if (!parse_DATA_LIST_vars (lexer, dataset_dict (ds),
+ &v, &nv, PV_NO_DUPLICATE))
+ return CMD_FAILURE;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ bool ok = false;
+
+ struct fmt_spec f;
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !parse_format_specifier (lexer, &f))
+ goto done;
+
+ char *error = fmt_check_type_compat__ (&f, NULL, VAL_STRING);
+ if (!error)
+ error = fmt_check_output__ (&f);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ goto done;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto done;
+
+ /* Create each variable. */
+ int width = fmt_var_width (&f);
+ for (size_t i = 0; i < nv; i++)
+ {
+ struct variable *new_var = dict_create_var (dataset_dict (ds), v[i],
+ width);
+ if (!new_var)
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("There is already a variable named %s."), v[i]);
+ else
+ var_set_both_formats (new_var, &f);
+ }
+ ok = true;
+
+ done:
+ for (size_t i = 0; i < nv; i++)
+ free (v[i]);
+ free (v);
+ if (!ok)
+ return CMD_FAILURE;
+ }
+ while (lex_match (lexer, T_SLASH));
+
+ return CMD_SUCCESS;
+}
+
+/* Parses the LEAVE command. */
+int
+cmd_leave (struct lexer *lexer, struct dataset *ds)
+{
+ struct variable **v;
+ size_t nv;
+
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
+ return CMD_CASCADING_FAILURE;
+ for (size_t i = 0; i < nv; i++)
+ var_set_leave (v[i], true);
+ free (v);
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2007, 2009, 2010, 2011, 2012, 2013, 2014,
+ 2020 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_matrix.h>
+#include <math.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/ll.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/taint.h"
+#include "linreg/sweep.h"
+#include "tukey/tukey.h"
+#include "math/categoricals.h"
+#include "math/interaction.h"
+#include "math/covariance.h"
+#include "math/levene.h"
+#include "math/moments.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+/* Workspace variable for each dependent variable */
+struct per_var_ws
+ {
+ struct interaction *iact;
+ struct categoricals *cat;
+ struct covariance *cov;
+ struct levene *nl;
+
+ double n;
+
+ double sst;
+ double sse;
+ double ssa;
+
+ int n_groups;
+
+ double mse;
+ };
+
+/* Per category data */
+struct descriptive_data
+ {
+ const struct variable *var;
+ struct moments1 *mom;
+
+ double minimum;
+ double maximum;
+ };
+
+enum missing_type
+ {
+ MISS_LISTWISE,
+ MISS_ANALYSIS,
+ };
+
+struct coeff_node
+{
+ struct ll ll;
+ double coeff;
+};
+
+
+struct contrasts_node
+{
+ struct ll ll;
+ struct ll_list coefficient_list;
+};
+
+
+struct oneway_spec;
+
+typedef double df_func (const struct per_var_ws *pvw, const struct moments1 *mom_i, const struct moments1 *mom_j);
+typedef double ts_func (int k, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err);
+typedef double p1tail_func (double ts, double df1, double df2);
+
+typedef double pinv_func (double std_err, double alpha, double df, int k, const struct moments1 *mom_i, const struct moments1 *mom_j);
+
+
+struct posthoc
+{
+ const char *syntax;
+ const char *label;
+
+ df_func *dff;
+ ts_func *tsf;
+ p1tail_func *p1f;
+
+ pinv_func *pinv;
+};
+
+struct oneway_spec
+{
+ size_t n_vars;
+ const struct variable **vars;
+
+ const struct variable *indep_var;
+
+ bool descriptive_stats;
+ bool homogeneity_stats;
+
+ enum missing_type missing_type;
+ enum mv_class exclude;
+
+ /* List of contrasts */
+ struct ll_list contrast_list;
+
+ /* The weight variable */
+ const struct variable *wv;
+ const struct fmt_spec *wfmt;
+
+ /* The confidence level for multiple comparisons */
+ double alpha;
+
+ int *posthoc;
+ int n_posthoc;
+};
+
+static double
+df_common (const struct per_var_ws *pvw, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
+{
+ return pvw->n - pvw->n_groups;
+}
+
+static double
+df_individual (const struct per_var_ws *pvw UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j)
+{
+ double n_i, var_i;
+ double n_j, var_j;
+ double nom,denom;
+
+ moments1_calculate (mom_i, &n_i, NULL, &var_i, 0, 0);
+ moments1_calculate (mom_j, &n_j, NULL, &var_j, 0, 0);
+
+ if (n_i <= 1.0 || n_j <= 1.0)
+ return SYSMIS;
+
+ nom = pow2 (var_i/n_i + var_j/n_j);
+ denom = pow2 (var_i/n_i) / (n_i - 1) + pow2 (var_j/n_j) / (n_j - 1);
+
+ return nom / denom;
+}
+
+static double lsd_pinv (double std_err, double alpha, double df, int k UNUSED, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
+{
+ return std_err * gsl_cdf_tdist_Pinv (1.0 - alpha / 2.0, df);
+}
+
+static double bonferroni_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
+{
+ const int m = k * (k - 1) / 2;
+ return std_err * gsl_cdf_tdist_Pinv (1.0 - alpha / (2.0 * m), df);
+}
+
+static double sidak_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
+{
+ const double m = k * (k - 1) / 2;
+ double lp = 1.0 - exp (log (1.0 - alpha) / m);
+ return std_err * gsl_cdf_tdist_Pinv (1.0 - lp / 2.0, df);
+}
+
+static double tukey_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
+{
+ if (k < 2 || df < 2)
+ return SYSMIS;
+
+ return std_err / sqrt (2.0) * qtukey (1 - alpha, 1.0, k, df, 1, 0);
+}
+
+static double scheffe_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
+{
+ double x = (k - 1) * gsl_cdf_fdist_Pinv (1.0 - alpha, k - 1, df);
+ return std_err * sqrt (x);
+}
+
+static double gh_pinv (double std_err UNUSED, double alpha, double df, int k, const struct moments1 *mom_i, const struct moments1 *mom_j)
+{
+ double n_i, mean_i, var_i;
+ double n_j, mean_j, var_j;
+ double m;
+
+ moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
+ moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
+
+ m = sqrt ((var_i/n_i + var_j/n_j) / 2.0);
+
+ if (k < 2 || df < 2)
+ return SYSMIS;
+
+ return m * qtukey (1 - alpha, 1.0, k, df, 1, 0);
+}
+
+
+static double
+multiple_comparison_sig (double std_err,
+ const struct per_var_ws *pvw,
+ const struct descriptive_data *dd_i, const struct descriptive_data *dd_j,
+ const struct posthoc *ph)
+{
+ int k = pvw->n_groups;
+ double df = ph->dff (pvw, dd_i->mom, dd_j->mom);
+ double ts = ph->tsf (k, dd_i->mom, dd_j->mom, std_err);
+ if (df == SYSMIS)
+ return SYSMIS;
+ return ph->p1f (ts, k - 1, df);
+}
+
+static double
+mc_half_range (const struct oneway_spec *cmd, const struct per_var_ws *pvw, double std_err, const struct descriptive_data *dd_i, const struct descriptive_data *dd_j, const struct posthoc *ph)
+{
+ int k = pvw->n_groups;
+ double df = ph->dff (pvw, dd_i->mom, dd_j->mom);
+ if (df == SYSMIS)
+ return SYSMIS;
+
+ return ph->pinv (std_err, cmd->alpha, df, k, dd_i->mom, dd_j->mom);
+}
+
+static double tukey_1tailsig (double ts, double df1, double df2)
+{
+ double twotailedsig;
+
+ if (df2 < 2 || df1 < 1)
+ return SYSMIS;
+
+ twotailedsig = 1.0 - ptukey (ts, 1.0, df1 + 1, df2, 1, 0);
+
+ return twotailedsig / 2.0;
+}
+
+static double lsd_1tailsig (double ts, double df1 UNUSED, double df2)
+{
+ return ts < 0 ? gsl_cdf_tdist_P (ts, df2) : gsl_cdf_tdist_Q (ts, df2);
+}
+
+static double sidak_1tailsig (double ts, double df1, double df2)
+{
+ double ex = (df1 + 1.0) * df1 / 2.0;
+ double lsd_sig = 2 * lsd_1tailsig (ts, df1, df2);
+
+ return 0.5 * (1.0 - pow (1.0 - lsd_sig, ex));
+}
+
+static double bonferroni_1tailsig (double ts, double df1, double df2)
+{
+ const int m = (df1 + 1) * df1 / 2;
+
+ double p = ts < 0 ? gsl_cdf_tdist_P (ts, df2) : gsl_cdf_tdist_Q (ts, df2);
+ p *= m;
+
+ return p > 0.5 ? 0.5 : p;
+}
+
+static double scheffe_1tailsig (double ts, double df1, double df2)
+{
+ return 0.5 * gsl_cdf_fdist_Q (ts, df1, df2);
+}
+
+
+static double tukey_test_stat (int k UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err)
+{
+ double ts;
+ double n_i, mean_i, var_i;
+ double n_j, mean_j, var_j;
+
+ moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
+ moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
+
+ ts = (mean_i - mean_j) / std_err;
+ ts = fabs (ts) * sqrt (2.0);
+
+ return ts;
+}
+
+static double lsd_test_stat (int k UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err)
+{
+ double n_i, mean_i, var_i;
+ double n_j, mean_j, var_j;
+
+ moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
+ moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
+
+ return (mean_i - mean_j) / std_err;
+}
+
+static double scheffe_test_stat (int k, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err)
+{
+ double t;
+ double n_i, mean_i, var_i;
+ double n_j, mean_j, var_j;
+
+ moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
+ moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
+
+ t = (mean_i - mean_j) / std_err;
+ t = pow2 (t);
+ t /= k - 1;
+
+ return t;
+}
+
+static double gh_test_stat (int k UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err UNUSED)
+{
+ double ts;
+ double thing;
+ double n_i, mean_i, var_i;
+ double n_j, mean_j, var_j;
+
+ moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
+ moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
+
+ thing = var_i / n_i + var_j / n_j;
+ thing /= 2.0;
+ thing = sqrt (thing);
+
+ ts = (mean_i - mean_j) / thing;
+
+ return fabs (ts);
+}
+
+
+
+static const struct posthoc ph_tests [] =
+ {
+ { "LSD", N_("LSD"), df_common, lsd_test_stat, lsd_1tailsig, lsd_pinv},
+ { "TUKEY", N_("Tukey HSD"), df_common, tukey_test_stat, tukey_1tailsig, tukey_pinv},
+ { "BONFERRONI", N_("Bonferroni"), df_common, lsd_test_stat, bonferroni_1tailsig, bonferroni_pinv},
+ { "SCHEFFE", N_("Scheffé"), df_common, scheffe_test_stat, scheffe_1tailsig, scheffe_pinv},
+ { "GH", N_("Games-Howell"), df_individual, gh_test_stat, tukey_1tailsig, gh_pinv},
+ { "SIDAK", N_("Šidák"), df_common, lsd_test_stat, sidak_1tailsig, sidak_pinv}
+ };
+
+
+struct oneway_workspace
+{
+ /* The number of distinct values of the independent variable, when all
+ missing values are disregarded */
+ int actual_number_of_groups;
+
+ struct per_var_ws *vws;
+
+ /* An array of descriptive data. One for each dependent variable */
+ struct descriptive_data **dd_total;
+};
+
+/* Routines to show the output tables */
+static void show_anova_table (const struct oneway_spec *, const struct oneway_workspace *);
+static void show_descriptives (const struct oneway_spec *, const struct oneway_workspace *);
+static void show_homogeneity (const struct oneway_spec *, const struct oneway_workspace *);
+
+static void output_oneway (const struct oneway_spec *, struct oneway_workspace *ws);
+static void run_oneway (const struct oneway_spec *cmd, struct casereader *input, const struct dataset *ds);
+
+
+static void
+destroy_coeff_list (struct contrasts_node *coeff_list)
+{
+ struct coeff_node *cn = NULL;
+ struct coeff_node *cnx = NULL;
+ struct ll_list *cl = &coeff_list->coefficient_list;
+
+ ll_for_each_safe (cn, cnx, struct coeff_node, ll, cl)
+ {
+ free (cn);
+ }
+
+ free (coeff_list);
+}
+
+static void
+oneway_cleanup (struct oneway_spec *cmd)
+{
+ struct contrasts_node *coeff_list = NULL;
+ struct contrasts_node *coeff_next = NULL;
+ ll_for_each_safe (coeff_list, coeff_next, struct contrasts_node, ll, &cmd->contrast_list)
+ {
+ destroy_coeff_list (coeff_list);
+ }
+
+ free (cmd->posthoc);
+}
+
+
+
+int
+cmd_oneway (struct lexer *lexer, struct dataset *ds)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ struct oneway_spec oneway = {
+ .missing_type = MISS_ANALYSIS,
+ .exclude = MV_ANY,
+ .wv = dict_get_weight (dict),
+ .wfmt = dict_get_weight_format (dict),
+ .alpha = 0.05,
+ };
+
+ ll_init (&oneway.contrast_list);
+ if (lex_match (lexer, T_SLASH))
+ {
+ if (!lex_force_match_id (lexer, "VARIABLES"))
+ goto error;
+ lex_match (lexer, T_EQUALS);
+ }
+
+ if (!parse_variables_const (lexer, dict,
+ &oneway.vars, &oneway.n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+
+ if (!lex_force_match (lexer, T_BY))
+ goto error;
+
+ oneway.indep_var = parse_variable_const (lexer, dict);
+ if (oneway.indep_var == NULL)
+ goto error;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "STATISTICS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "DESCRIPTIVES"))
+ oneway.descriptive_stats = true;
+ else if (lex_match_id (lexer, "HOMOGENEITY"))
+ oneway.homogeneity_stats = true;
+ else
+ {
+ lex_error_expecting (lexer, "DESCRIPTIVES", "HOMOGENEITY");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "POSTHOC"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ bool method = false;
+ for (size_t p = 0; p < sizeof ph_tests / sizeof *ph_tests; ++p)
+ if (lex_match_id (lexer, ph_tests[p].syntax))
+ {
+ oneway.n_posthoc++;
+ oneway.posthoc = xrealloc (oneway.posthoc, sizeof (*oneway.posthoc) * oneway.n_posthoc);
+ oneway.posthoc[oneway.n_posthoc - 1] = p;
+ method = true;
+ break;
+ }
+ if (method == false)
+ {
+ if (lex_match_id (lexer, "ALPHA"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num (lexer))
+ goto error;
+ oneway.alpha = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else
+ {
+ lex_error (lexer, _("Unknown post hoc analysis method."));
+ goto error;
+ }
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "CONTRAST"))
+ {
+ struct contrasts_node *cl = XZALLOC (struct contrasts_node);
+
+ struct ll_list *coefficient_list = &cl->coefficient_list;
+ lex_match (lexer, T_EQUALS);
+
+ ll_init (coefficient_list);
+
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (!lex_force_num (lexer))
+ {
+ destroy_coeff_list (cl);
+ goto error;
+ }
+
+ struct coeff_node *cc = xmalloc (sizeof *cc);
+ cc->coeff = lex_number (lexer);
+
+ ll_push_tail (coefficient_list, &cc->ll);
+ lex_get (lexer);
+ }
+
+ if (ll_count (coefficient_list) <= 0)
+ {
+ destroy_coeff_list (cl);
+ goto error;
+ }
+
+ ll_push_tail (&oneway.contrast_list, &cl->ll);
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ oneway.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ oneway.exclude = MV_ANY;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ oneway.missing_type = MISS_LISTWISE;
+ else if (lex_match_id (lexer, "ANALYSIS"))
+ oneway.missing_type = MISS_ANALYSIS;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE",
+ "LISTWISE", "ANALYSIS");
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "STATISTICS", "POSTHOC", "CONTRAST",
+ "MISSING");
+ goto error;
+ }
+ }
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ run_oneway (&oneway, group, ds);
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ oneway_cleanup (&oneway);
+ free (oneway.vars);
+ return CMD_SUCCESS;
+
+ error:
+ oneway_cleanup (&oneway);
+ free (oneway.vars);
+ return CMD_FAILURE;
+}
+\f
+static struct descriptive_data *
+dd_create (const struct variable *var)
+{
+ struct descriptive_data *dd = xmalloc (sizeof *dd);
+
+ dd->mom = moments1_create (MOMENT_VARIANCE);
+ dd->minimum = DBL_MAX;
+ dd->maximum = -DBL_MAX;
+ dd->var = var;
+
+ return dd;
+}
+
+static void
+dd_destroy (struct descriptive_data *dd)
+{
+ moments1_destroy (dd->mom);
+ free (dd);
+}
+
+static void *
+makeit (const void *aux1, void *aux2 UNUSED)
+{
+ const struct variable *var = aux1;
+
+ struct descriptive_data *dd = dd_create (var);
+
+ return dd;
+}
+
+static void
+killit (const void *aux1 UNUSED, void *aux2 UNUSED, void *user_data)
+{
+ struct descriptive_data *dd = user_data;
+
+ dd_destroy (dd);
+}
+
+
+static void
+updateit (const void *aux1, void *aux2, void *user_data,
+ const struct ccase *c, double weight)
+{
+ struct descriptive_data *dd = user_data;
+
+ const struct variable *varp = aux1;
+
+ const union value *valx = case_data (c, varp);
+
+ struct descriptive_data *dd_total = aux2;
+
+ moments1_add (dd->mom, valx->f, weight);
+ if (valx->f < dd->minimum)
+ dd->minimum = valx->f;
+
+ if (valx->f > dd->maximum)
+ dd->maximum = valx->f;
+
+ {
+ const struct variable *var = dd_total->var;
+ const union value *val = case_data (c, var);
+
+ moments1_add (dd_total->mom,
+ val->f,
+ weight);
+
+ if (val->f < dd_total->minimum)
+ dd_total->minimum = val->f;
+
+ if (val->f > dd_total->maximum)
+ dd_total->maximum = val->f;
+ }
+}
+
+static void
+run_oneway (const struct oneway_spec *cmd, struct casereader *input,
+ const struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct oneway_workspace ws = {
+ .vws = xcalloc (cmd->n_vars, sizeof *ws.vws),
+ .dd_total = XCALLOC (cmd->n_vars, struct descriptive_data *),
+ };
+
+ for (size_t v = 0; v < cmd->n_vars; ++v)
+ ws.dd_total[v] = dd_create (cmd->vars[v]);
+
+ for (size_t v = 0; v < cmd->n_vars; ++v)
+ {
+ static const struct payload payload =
+ {
+ .create = makeit,
+ .update = updateit,
+ .destroy = killit,
+ };
+
+ ws.vws[v].iact = interaction_create (cmd->indep_var);
+ ws.vws[v].cat = categoricals_create (&ws.vws[v].iact, 1, cmd->wv,
+ cmd->exclude);
+
+ categoricals_set_payload (ws.vws[v].cat, &payload,
+ CONST_CAST (struct variable *, cmd->vars[v]),
+ ws.dd_total[v]);
+
+
+ ws.vws[v].cov = covariance_2pass_create (1, &cmd->vars[v],
+ ws.vws[v].cat,
+ cmd->wv, cmd->exclude, true);
+ ws.vws[v].nl = levene_create (var_get_width (cmd->indep_var), NULL);
+ }
+
+ output_split_file_values_peek (ds, input);
+
+ struct taint *taint = taint_clone (casereader_get_taint (input));
+ input = casereader_create_filter_missing (input, &cmd->indep_var, 1,
+ cmd->exclude, NULL, NULL);
+ if (cmd->missing_type == MISS_LISTWISE)
+ input = casereader_create_filter_missing (input, cmd->vars, cmd->n_vars,
+ cmd->exclude, NULL, NULL);
+ input = casereader_create_filter_weight (input, dict, NULL, NULL);
+
+ struct casereader *reader = casereader_clone (input);
+ struct ccase *c;
+ for (; (c = casereader_read (reader)) != NULL; case_unref (c))
+ {
+ double w = dict_get_case_weight (dict, c, NULL);
+
+ for (size_t i = 0; i < cmd->n_vars; ++i)
+ {
+ struct per_var_ws *pvw = &ws.vws[i];
+ const struct variable *v = cmd->vars[i];
+ const union value *val = case_data (c, v);
+
+ if (MISS_ANALYSIS == cmd->missing_type)
+ {
+ if (var_is_value_missing (v, val) & cmd->exclude)
+ continue;
+ }
+
+ covariance_accumulate_pass1 (pvw->cov, c);
+ levene_pass_one (pvw->nl, val->f, w, case_data (c, cmd->indep_var));
+ }
+ }
+ casereader_destroy (reader);
+
+ reader = casereader_clone (input);
+ for (; (c = casereader_read (reader)); case_unref (c))
+ {
+ double w = dict_get_case_weight (dict, c, NULL);
+ for (size_t i = 0; i < cmd->n_vars; ++i)
+ {
+ struct per_var_ws *pvw = &ws.vws[i];
+ const struct variable *v = cmd->vars[i];
+ const union value *val = case_data (c, v);
+
+ if (MISS_ANALYSIS == cmd->missing_type)
+ {
+ if (var_is_value_missing (v, val) & cmd->exclude)
+ continue;
+ }
+
+ covariance_accumulate_pass2 (pvw->cov, c);
+ levene_pass_two (pvw->nl, val->f, w, case_data (c, cmd->indep_var));
+ }
+ }
+ casereader_destroy (reader);
+
+ reader = casereader_clone (input);
+ for (; (c = casereader_read (reader)); case_unref (c))
+ {
+ double w = dict_get_case_weight (dict, c, NULL);
+
+ for (size_t i = 0; i < cmd->n_vars; ++i)
+ {
+ struct per_var_ws *pvw = &ws.vws[i];
+ const struct variable *v = cmd->vars[i];
+ const union value *val = case_data (c, v);
+
+ if (MISS_ANALYSIS == cmd->missing_type)
+ {
+ if (var_is_value_missing (v, val) & cmd->exclude)
+ continue;
+ }
+
+ levene_pass_three (pvw->nl, val->f, w, case_data (c, cmd->indep_var));
+ }
+ }
+ casereader_destroy (reader);
+
+ for (size_t v = 0; v < cmd->n_vars; ++v)
+ {
+ struct per_var_ws *pvw = &ws.vws[v];
+
+ const struct categoricals *cats = covariance_get_categoricals (pvw->cov);
+ if (!categoricals_sane (cats))
+ {
+ msg (MW, _("Dependent variable %s has no non-missing values. "
+ "No analysis for this variable will be done."),
+ var_get_name (cmd->vars[v]));
+ continue;
+ }
+
+ const gsl_matrix *ucm = covariance_calculate_unnormalized (pvw->cov);
+
+ gsl_matrix *cm = gsl_matrix_alloc (ucm->size1, ucm->size2);
+ gsl_matrix_memcpy (cm, ucm);
+
+ moments1_calculate (ws.dd_total[v]->mom, &pvw->n, NULL, NULL, NULL, NULL);
+
+ pvw->sst = gsl_matrix_get (cm, 0, 0);
+
+ reg_sweep (cm, 0);
+
+ pvw->sse = gsl_matrix_get (cm, 0, 0);
+ gsl_matrix_free (cm);
+
+ pvw->ssa = pvw->sst - pvw->sse;
+
+ pvw->n_groups = categoricals_n_total (cats);
+
+ pvw->mse = (pvw->sst - pvw->ssa) / (pvw->n - pvw->n_groups);
+ }
+
+ for (size_t v = 0; v < cmd->n_vars; ++v)
+ {
+ const struct categoricals *cats = covariance_get_categoricals (ws.vws[v].cov);
+ if (categoricals_is_complete (cats))
+ {
+ if (categoricals_n_total (cats) > ws.actual_number_of_groups)
+ ws.actual_number_of_groups = categoricals_n_total (cats);
+ }
+ }
+ casereader_destroy (input);
+
+ if (!taint_has_tainted_successor (taint))
+ output_oneway (cmd, &ws);
+
+ taint_destroy (taint);
+
+ for (size_t v = 0; v < cmd->n_vars; ++v)
+ {
+ covariance_destroy (ws.vws[v].cov);
+ levene_destroy (ws.vws[v].nl);
+ dd_destroy (ws.dd_total[v]);
+ interaction_destroy (ws.vws[v].iact);
+ }
+
+ free (ws.vws);
+ free (ws.dd_total);
+}
+
+static void show_contrast_coeffs (const struct oneway_spec *, const struct oneway_workspace *);
+static void show_contrast_tests (const struct oneway_spec *, const struct oneway_workspace *);
+static void show_comparisons (const struct oneway_spec *, const struct oneway_workspace *, int depvar);
+
+static void
+output_oneway (const struct oneway_spec *cmd, struct oneway_workspace *ws)
+{
+ size_t list_idx = 0;
+
+ /* Check the sanity of the given contrast values */
+ struct contrasts_node *coeff_list = NULL;
+ struct contrasts_node *coeff_next = NULL;
+ ll_for_each_safe (coeff_list, coeff_next, struct contrasts_node, ll, &cmd->contrast_list)
+ {
+ struct coeff_node *cn = NULL;
+ double sum = 0;
+ struct ll_list *cl = &coeff_list->coefficient_list;
+ ++list_idx;
+
+ if (ll_count (cl) != ws->actual_number_of_groups)
+ {
+ msg (SW,
+ _("In contrast list %zu, the number of coefficients (%zu) does not equal the number of groups (%d). This contrast list will be ignored."),
+ list_idx, ll_count (cl), ws->actual_number_of_groups);
+
+ ll_remove (&coeff_list->ll);
+ destroy_coeff_list (coeff_list);
+ continue;
+ }
+
+ ll_for_each (cn, struct coeff_node, ll, cl)
+ sum += cn->coeff;
+
+ if (sum != 0.0)
+ msg (SW, _("Coefficients for contrast %zu do not total zero"),
+ list_idx);
+ }
+
+ if (cmd->descriptive_stats)
+ show_descriptives (cmd, ws);
+
+ if (cmd->homogeneity_stats)
+ show_homogeneity (cmd, ws);
+
+ show_anova_table (cmd, ws);
+
+ if (ll_count (&cmd->contrast_list) > 0)
+ {
+ show_contrast_coeffs (cmd, ws);
+ show_contrast_tests (cmd, ws);
+ }
+
+ if (cmd->posthoc)
+ for (size_t v = 0; v < cmd->n_vars; ++v)
+ {
+ const struct categoricals *cats = covariance_get_categoricals (ws->vws[v].cov);
+
+ if (categoricals_is_complete (cats))
+ show_comparisons (cmd, ws, v);
+ }
+}
+
+
+/* Show the ANOVA table */
+static void
+show_anova_table (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
+{
+ struct pivot_table *table = pivot_table_create (N_("ANOVA"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Sum of Squares"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Mean Square"), PIVOT_RC_OTHER,
+ N_("F"), PIVOT_RC_OTHER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Type"),
+ N_("Between Groups"), N_("Within Groups"),
+ N_("Total"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0; i < cmd->n_vars; ++i)
+ {
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (cmd->vars[i]));
+
+ const struct per_var_ws *pvw = &ws->vws[i];
+
+ double n;
+ moments1_calculate (ws->dd_total[i]->mom, &n, NULL, NULL, NULL, NULL);
+
+ double df1 = pvw->n_groups - 1;
+ double df2 = n - pvw->n_groups;
+ double msa = pvw->ssa / df1;
+ double F = msa / pvw->mse;
+
+ struct entry
+ {
+ int stat_idx;
+ int type_idx;
+ double x;
+ }
+ entries[] = {
+ /* Sums of Squares. */
+ { 0, 0, pvw->ssa },
+ { 0, 1, pvw->sse },
+ { 0, 2, pvw->sst },
+ /* Degrees of Freedom. */
+ { 1, 0, df1 },
+ { 1, 1, df2 },
+ { 1, 2, n - 1 },
+ /* Mean Squares. */
+ { 2, 0, msa },
+ { 2, 1, pvw->mse },
+ /* F. */
+ { 3, 0, F },
+ /* Significance. */
+ { 4, 0, gsl_cdf_fdist_Q (F, df1, df2) },
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ {
+ const struct entry *e = &entries[j];
+ pivot_table_put3 (table, e->stat_idx, e->type_idx, var_idx,
+ pivot_value_new_number (e->x));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+/* Show the descriptives table */
+static void
+show_descriptives (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
+{
+ if (!cmd->n_vars)
+ return;
+ const struct categoricals *cats = covariance_get_categoricals (
+ ws->vws[0].cov);
+
+ struct pivot_table *table = pivot_table_create (N_("Descriptives"));
+ pivot_table_set_weight_format (table, cmd->wfmt);
+
+ const double confidence = 0.95;
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT, N_("Mean"), N_("Std. Deviation"),
+ N_("Std. Error"));
+ struct pivot_category *interval = pivot_category_create_group__ (
+ statistics->root,
+ pivot_value_new_text_format (N_("%g%% Confidence Interval for Mean"),
+ confidence * 100.0));
+ pivot_category_create_leaves (interval, N_("Lower Bound"),
+ N_("Upper Bound"));
+ pivot_category_create_leaves (statistics->root,
+ N_("Minimum"), N_("Maximum"));
+
+ struct pivot_dimension *indep_var = pivot_dimension_create__ (
+ table, PIVOT_AXIS_ROW, pivot_value_new_variable (cmd->indep_var));
+ indep_var->root->show_label = true;
+ size_t n;
+ union value *values = categoricals_get_var_values (cats, cmd->indep_var, &n);
+ for (size_t j = 0; j < n; j++)
+ pivot_category_create_leaf (
+ indep_var->root, pivot_value_new_var_value (cmd->indep_var, &values[j]));
+ pivot_category_create_leaf (
+ indep_var->root, pivot_value_new_text_format (N_("Total")));
+
+ struct pivot_dimension *dep_var = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variable"));
+
+ const double q = (1.0 - confidence) / 2.0;
+ for (int v = 0; v < cmd->n_vars; ++v)
+ {
+ int dep_var_idx = pivot_category_create_leaf (
+ dep_var->root, pivot_value_new_variable (cmd->vars[v]));
+
+ struct per_var_ws *pvw = &ws->vws[v];
+ const struct categoricals *cats = covariance_get_categoricals (pvw->cov);
+
+ int count;
+ for (count = 0; count < categoricals_n_total (cats); ++count)
+ {
+ const struct descriptive_data *dd
+ = categoricals_get_user_data_by_category (cats, count);
+
+ double n, mean, variance;
+ moments1_calculate (dd->mom, &n, &mean, &variance, NULL, NULL);
+
+ double std_dev = sqrt (variance);
+ double std_error = std_dev / sqrt (n);
+ double T = gsl_cdf_tdist_Qinv (q, n - 1);
+
+ double entries[] = {
+ n,
+ mean,
+ std_dev,
+ std_error,
+ mean - T * std_error,
+ mean + T * std_error,
+ dd->minimum,
+ dd->maximum,
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put3 (table, i, count, dep_var_idx,
+ pivot_value_new_number (entries[i]));
+ }
+
+ if (categoricals_is_complete (cats))
+ {
+ double n, mean, variance;
+ moments1_calculate (ws->dd_total[v]->mom, &n, &mean, &variance,
+ NULL, NULL);
+
+ double std_dev = sqrt (variance);
+ double std_error = std_dev / sqrt (n);
+ double T = gsl_cdf_tdist_Qinv (q, n - 1);
+
+ double entries[] = {
+ n,
+ mean,
+ std_dev,
+ std_error,
+ mean - T * std_error,
+ mean + T * std_error,
+ ws->dd_total[v]->minimum,
+ ws->dd_total[v]->maximum,
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put3 (table, i, count, dep_var_idx,
+ pivot_value_new_number (entries[i]));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+/* Show the homogeneity table */
+static void
+show_homogeneity (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Test of Homogeneity of Variances"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Levene Statistic"), PIVOT_RC_OTHER,
+ N_("df1"), PIVOT_RC_INTEGER,
+ N_("df2"), PIVOT_RC_INTEGER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (int v = 0; v < cmd->n_vars; ++v)
+ {
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (cmd->vars[v]));
+
+ double n;
+ moments1_calculate (ws->dd_total[v]->mom, &n, NULL, NULL, NULL, NULL);
+
+ const struct per_var_ws *pvw = &ws->vws[v];
+ double df1 = pvw->n_groups - 1;
+ double df2 = n - pvw->n_groups;
+ double F = levene_calculate (pvw->nl);
+
+ double entries[] =
+ {
+ F,
+ df1,
+ df2,
+ gsl_cdf_fdist_Q (F, df1, df2),
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, var_idx,
+ pivot_value_new_number (entries[i]));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+/* Show the contrast coefficients table */
+static void
+show_contrast_coeffs (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
+{
+ struct pivot_table *table = pivot_table_create (N_("Contrast Coefficients"));
+
+ struct pivot_dimension *indep_var = pivot_dimension_create__ (
+ table, PIVOT_AXIS_COLUMN, pivot_value_new_variable (cmd->indep_var));
+ indep_var->root->show_label = true;
+
+ struct pivot_dimension *contrast = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Contrast"));
+ contrast->root->show_label = true;
+
+ const struct covariance *cov = ws->vws[0].cov;
+
+ const struct contrasts_node *cn;
+ int c_num = 1;
+ ll_for_each (cn, struct contrasts_node, ll, &cmd->contrast_list)
+ {
+ int contrast_idx = pivot_category_create_leaf (
+ contrast->root, pivot_value_new_integer (c_num++));
+
+ const struct coeff_node *coeffn;
+ int indep_idx = 0;
+ ll_for_each (coeffn, struct coeff_node, ll, &cn->coefficient_list)
+ {
+ const struct categoricals *cats = covariance_get_categoricals (cov);
+ const struct ccase *gcc = categoricals_get_case_by_category (
+ cats, indep_idx);
+
+ if (!contrast_idx)
+ pivot_category_create_leaf (
+ indep_var->root, pivot_value_new_var_value (
+ cmd->indep_var, case_data (gcc, cmd->indep_var)));
+
+ pivot_table_put2 (table, indep_idx++, contrast_idx,
+ pivot_value_new_integer (coeffn->coeff));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+/* Show the results of the contrast tests */
+static void
+show_contrast_tests (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
+{
+ struct pivot_table *table = pivot_table_create (N_("Contrast Tests"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Value of Contrast"), PIVOT_RC_OTHER,
+ N_("Std. Error"), PIVOT_RC_OTHER,
+ N_("t"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_OTHER,
+ N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *contrasts = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Contrast"));
+ contrasts->root->show_label = true;
+ int n_contrasts = ll_count (&cmd->contrast_list);
+ for (int i = 1; i <= n_contrasts; i++)
+ pivot_category_create_leaf (contrasts->root, pivot_value_new_integer (i));
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Assumption"),
+ N_("Assume equal variances"),
+ N_("Does not assume equal variances"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variable"));
+
+ for (int v = 0; v < cmd->n_vars; ++v)
+ {
+ const struct per_var_ws *pvw = &ws->vws[v];
+ const struct categoricals *cats = covariance_get_categoricals (pvw->cov);
+ if (!categoricals_is_complete (cats))
+ continue;
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (cmd->vars[v]));
+
+ struct contrasts_node *cn;
+ int contrast_idx = 0;
+ ll_for_each (cn, struct contrasts_node, ll, &cmd->contrast_list)
+ {
+
+ /* Note: The calculation of the degrees of freedom in the
+ "variances not equal" case is painfull!!
+ The following formula may help to understand it:
+ \frac{\left (\sum_{i=1}^k{c_i^2\frac{s_i^2}{n_i}}\right)^2}
+ {
+ \sum_{i=1}^k\left (
+ \frac{\left (c_i^2\frac{s_i^2}{n_i}\right)^2} {n_i-1}
+ \right)
+ }
+ */
+
+ double grand_n;
+ moments1_calculate (ws->dd_total[v]->mom, &grand_n, NULL, NULL,
+ NULL, NULL);
+ double df = grand_n - pvw->n_groups;
+
+ double contrast_value = 0.0;
+ double coef_msq = 0.0;
+ double sec_vneq = 0.0;
+ double df_denominator = 0.0;
+ double df_numerator = 0.0;
+ struct coeff_node *coeffn;
+ int ci = 0;
+ ll_for_each (coeffn, struct coeff_node, ll, &cn->coefficient_list)
+ {
+ const struct descriptive_data *dd
+ = categoricals_get_user_data_by_category (cats, ci);
+ const double coef = coeffn->coeff;
+
+ double n, mean, variance;
+ moments1_calculate (dd->mom, &n, &mean, &variance, NULL, NULL);
+
+ double winv = variance / n;
+ contrast_value += coef * mean;
+ coef_msq += pow2 (coef) / n;
+ sec_vneq += pow2 (coef) * variance / n;
+ df_numerator += pow2 (coef) * winv;
+ df_denominator += pow2(pow2 (coef) * winv) / (n - 1);
+
+ ci++;
+ }
+ sec_vneq = sqrt (sec_vneq);
+ df_numerator = pow2 (df_numerator);
+
+ double std_error_contrast = sqrt (pvw->mse * coef_msq);
+ double T = contrast_value / std_error_contrast;
+ double T_ne = contrast_value / sec_vneq;
+ double df_ne = df_numerator / df_denominator;
+
+ struct entry
+ {
+ int stat_idx;
+ int assumption_idx;
+ double x;
+ }
+ entries[] =
+ {
+ /* Assume equal. */
+ { 0, 0, contrast_value },
+ { 1, 0, std_error_contrast },
+ { 2, 0, T },
+ { 3, 0, df },
+ { 4, 0, 2 * gsl_cdf_tdist_Q (fabs(T), df) },
+ /* Do not assume equal. */
+ { 0, 1, contrast_value },
+ { 1, 1, sec_vneq },
+ { 2, 1, T_ne },
+ { 3, 1, df_ne },
+ { 4, 1, 2 * gsl_cdf_tdist_Q (fabs(T_ne), df_ne) },
+ };
+
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put4 (
+ table, e->stat_idx, contrast_idx, e->assumption_idx, var_idx,
+ pivot_value_new_number (e->x));
+ }
+
+ contrast_idx++;
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_comparisons (const struct oneway_spec *cmd, const struct oneway_workspace *ws, int v)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_user_text_nocopy (xasprintf (
+ _("Multiple Comparisons (%s)"),
+ var_to_string (cmd->vars[v]))),
+ "Multiple Comparisons");
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Mean Difference (I - J)"), PIVOT_RC_OTHER,
+ N_("Std. Error"), PIVOT_RC_OTHER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+ struct pivot_category *interval = pivot_category_create_group__ (
+ statistics->root,
+ pivot_value_new_text_format (N_("%g%% Confidence Interval"),
+ (1 - cmd->alpha) * 100.0));
+ pivot_category_create_leaves (interval,
+ N_("Lower Bound"), PIVOT_RC_OTHER,
+ N_("Upper Bound"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *j_family = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("(J) Family"));
+ j_family->root->show_label = true;
+
+ struct pivot_dimension *i_family = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("(J) Family"));
+ i_family->root->show_label = true;
+
+ const struct per_var_ws *pvw = &ws->vws[v];
+ const struct categoricals *cat = pvw->cat;
+ for (int i = 0; i < pvw->n_groups; i++)
+ {
+ const struct ccase *gcc = categoricals_get_case_by_category (cat, i);
+ for (int j = 0; j < 2; j++)
+ pivot_category_create_leaf (
+ j ? j_family->root : i_family->root,
+ pivot_value_new_var_value (cmd->indep_var,
+ case_data (gcc, cmd->indep_var)));
+ }
+
+ struct pivot_dimension *test = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Test"));
+
+ for (int p = 0; p < cmd->n_posthoc; ++p)
+ {
+ const struct posthoc *ph = &ph_tests[cmd->posthoc[p]];
+
+ int test_idx = pivot_category_create_leaf (
+ test->root, pivot_value_new_text (ph->label));
+
+ for (int i = 0; i < pvw->n_groups; ++i)
+ {
+ struct descriptive_data *dd_i
+ = categoricals_get_user_data_by_category (cat, i);
+ double weight_i, mean_i, var_i;
+ moments1_calculate (dd_i->mom, &weight_i, &mean_i, &var_i, 0, 0);
+
+ for (int j = 0; j < pvw->n_groups; ++j)
+ {
+ if (j == i)
+ continue;
+
+ struct descriptive_data *dd_j
+ = categoricals_get_user_data_by_category (cat, j);
+ double weight_j, mean_j, var_j;
+ moments1_calculate (dd_j->mom, &weight_j, &mean_j, &var_j, 0, 0);
+
+ double std_err = pvw->mse;
+ std_err *= weight_i + weight_j;
+ std_err /= weight_i * weight_j;
+ std_err = sqrt (std_err);
+
+ double sig = 2 * multiple_comparison_sig (std_err, pvw,
+ dd_i, dd_j, ph);
+ double half_range = mc_half_range (cmd, pvw, std_err,
+ dd_i, dd_j, ph);
+ double entries[] = {
+ mean_i - mean_j,
+ std_err,
+ sig,
+ (mean_i - mean_j) - half_range,
+ (mean_i - mean_j) + half_range,
+ };
+ for (size_t k = 0; k < sizeof entries / sizeof *entries; k++)
+ pivot_table_put4 (table, k, j, i, test_idx,
+ pivot_value_new_number (entries[k]));
+ }
+ }
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/settings.h"
+#include "data/format.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/format-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/string-set.h"
+#include "libpspp/version.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_output_modify (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ struct string_set rc_names = STRING_SET_INITIALIZER (rc_names);
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "SELECT"))
+ {
+ if (!lex_force_match_id (lexer, "TABLES"))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "TABLECELLS"))
+ {
+ string_set_clear (&rc_names);
+ struct fmt_spec fmt = { .type = 0 };
+
+ while (lex_token (lexer) != T_SLASH &&
+ lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "SELECT"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ if (!lex_force_match (lexer, T_LBRACK))
+ goto error;
+
+ while (lex_token (lexer) == T_ID)
+ {
+ string_set_insert (&rc_names, lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+
+ if (!lex_force_match (lexer, T_RBRACK))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ char type[FMT_TYPE_LEN_MAX + 1];
+ uint16_t width;
+ uint8_t decimals;
+
+ if (!lex_force_match (lexer, T_EQUALS)
+ || !parse_abstract_format_specifier (lexer, type,
+ &width, &decimals))
+ goto error;
+
+ if (width <= 0)
+ {
+ const struct fmt_spec *dflt = settings_get_format ();
+ width = dflt->w;
+ }
+
+ if (!fmt_from_name (type, &fmt.type))
+ {
+ lex_error (lexer, _("Unknown format type `%s'."), type);
+ goto error;
+ }
+
+ fmt.w = width;
+ fmt.d = decimals;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "SELECT", "FORMAT");
+ goto error;
+ }
+ }
+
+ if (fmt.w)
+ {
+ const struct string_set_node *node;
+ const char *s;
+ STRING_SET_FOR_EACH (s, node, &rc_names)
+ if (!pivot_result_class_change (s, &fmt))
+ lex_error (lexer, _("Unknown cell class %s."), s);
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "SELECT", "TABLECELLS");
+ goto error;
+ }
+ }
+
+ string_set_destroy (&rc_names);
+ return CMD_SUCCESS;
+
+ error:
+ string_set_destroy (&rc_names);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2004, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <errno.h>
+#include <stdlib.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "data/settings.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/cast.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/str.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+enum PER {PER_RO, PER_RW};
+
+int change_permissions(const char *file_name, enum PER per);
+
+
+/* Parses the PERMISSIONS command. */
+int
+cmd_permissions (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ if (settings_get_safer_mode ())
+ {
+ lex_next_error (lexer, -1, -1,
+ _("This command not allowed when the %s option is set."),
+ "SAFER");
+ return 0;
+ }
+
+ char *fn = NULL;
+ const char *str = NULL;
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "FILE"))
+ lex_match (lexer, T_EQUALS);
+
+ str = lex_tokcstr (lexer);
+ if (str)
+ fn = strdup (str);
+
+ if (!lex_force_match (lexer, T_STRING) || str == NULL)
+ goto error;
+
+ lex_match (lexer, T_SLASH);
+
+ if (! lex_match_id (lexer, "PERMISSIONS"))
+ goto error;
+
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_match_id (lexer, "READONLY"))
+ {
+ if (! change_permissions (fn, PER_RO))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "WRITEABLE"))
+ {
+ if (! change_permissions (fn, PER_RW))
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "WRITEABLE", "READONLY");
+ goto error;
+ }
+
+ free (fn);
+
+ return CMD_SUCCESS;
+
+ error:
+
+ free(fn);
+
+ return CMD_FAILURE;
+}
+
+
+
+int
+change_permissions (const char *file_name, enum PER per)
+{
+ char *locale_file_name;
+ struct stat buf;
+ mode_t mode;
+
+ locale_file_name = utf8_to_filename (file_name);
+ if (-1 == stat(locale_file_name, &buf))
+ {
+ const int errnum = errno;
+ msg (SE, _("Cannot stat %s: %s"), file_name, strerror(errnum));
+ free (locale_file_name);
+ return 0;
+ }
+
+ if (per == PER_RW)
+ mode = buf.st_mode | 0200;
+ else
+ mode = buf.st_mode & ~0222;
+
+ if (-1 == chmod(locale_file_name, mode))
+
+ {
+ const int errnum = errno;
+ msg (SE, _("Cannot change mode of %s: %s"), file_name, strerror(errnum));
+ free (locale_file_name);
+ return 0;
+ }
+
+ free (locale_file_name);
+
+ return 1;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2010, 2011, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/placement-parser.h"
+
+#include <assert.h>
+
+#include "data/format.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+
+#include "gl/c-strcase.h"
+#include "gl/xalloc.h"
+#include "gl/xsize.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Extensions to the format specifiers used only for
+ placement. */
+enum
+ {
+ PRS_TYPE_T = SCHAR_MAX - 3, /* Tab to absolute column. */
+ PRS_TYPE_X, /* Skip columns. */
+ PRS_TYPE_NEW_REC /* Next record. */
+ };
+
+static bool fixed_parse_columns (struct lexer *, struct pool *, size_t n_vars,
+ enum fmt_use, struct fmt_spec **, size_t *);
+static bool fixed_parse_fortran (struct lexer *l, struct pool *, enum fmt_use,
+ struct fmt_spec **, size_t *);
+
+/* Parses Fortran-like or column-based specifications for placing
+ variable data in fixed positions in columns and rows, that is,
+ formats like those parsed by DATA LIST or PRINT. Returns true
+ only if successful.
+
+ The formats parsed are either input or output formats, according
+ to USE.
+
+ If USE is FMT_FOR_INPUT, then T, X, and / "formats" are parsed,
+ in addition to regular formats. If USE is FMT_FOR_OUTPUT, then
+ T and X "formats" are parsed but not /.
+
+ If successful, formats for N_VARS variables are stored in
+ *FORMATS, and the number of formats required is stored in
+ *FORMAT_CNT. *FORMAT_CNT may be greater than N_VARS because
+ of T, X, and / "formats", but success guarantees that exactly
+ N_VARS variables will be placed by the output formats. The
+ caller should call execute_placement_format to process those
+ "formats" in interpreting the output.
+
+ Uses POOL for allocation. When the caller is finished
+ interpreting *FORMATS, POOL may be destroyed. */
+bool
+parse_var_placements (struct lexer *lexer, struct pool *pool, size_t n_vars,
+ enum fmt_use use,
+ struct fmt_spec **formats, size_t *n_formats)
+{
+ assert (n_vars > 0);
+ if (lex_is_number (lexer))
+ return fixed_parse_columns (lexer, pool, n_vars, use,
+ formats, n_formats);
+ else if (lex_match (lexer, T_LPAREN))
+ {
+ int start_ofs = lex_ofs (lexer);
+ if (!fixed_parse_fortran (lexer, pool, use, formats, n_formats))
+ return false;
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ size_t n_assignments = 0;
+ for (size_t i = 0; i < *n_formats; i++)
+ n_assignments += (*formats)[i].type < FMT_NUMBER_OF_FORMATS;
+
+ if (n_assignments != n_vars)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("Number of variables specified (%zu) "
+ "differs from number of variable formats (%zu)."),
+ n_vars, n_assignments);
+ return false;
+ }
+
+ return true;
+ }
+ else
+ {
+ lex_error (lexer, _("SPSS-like or Fortran-like format "
+ "specification expected after variable names."));
+ return false;
+ }
+}
+
+/* Implements parse_var_placements for column-based formats. */
+static bool
+fixed_parse_columns (struct lexer *lexer, struct pool *pool, size_t n_vars,
+ enum fmt_use use,
+ struct fmt_spec **formats, size_t *n_formats)
+{
+ int start_ofs = lex_ofs (lexer);
+
+ int fc, lc;
+ if (!parse_column_range (lexer, 1, &fc, &lc, NULL))
+ return false;
+
+ /* Divide columns evenly. */
+ int w = (lc - fc + 1) / n_vars;
+ if ((lc - fc + 1) % n_vars)
+ {
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("The %d columns %d-%d "
+ "can't be evenly divided into %zu fields."),
+ lc - fc + 1, fc, lc, n_vars);
+ return false;
+ }
+
+ /* Format specifier. */
+ enum fmt_type type;
+ int d;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ /* Get format type. */
+ if (lex_token (lexer) == T_ID)
+ {
+ if (!parse_format_specifier_name (lexer, &type))
+ return false;
+ lex_match (lexer, T_COMMA);
+ }
+ else
+ type = FMT_F;
+
+ /* Get decimal places. */
+ if (lex_is_integer (lexer))
+ {
+ d = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ d = 0;
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ else
+ {
+ type = FMT_F;
+ d = 0;
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ struct fmt_spec format = { .type = type, .w = w, .d = d };
+ char *error = fmt_check__ (&format, use);
+ if (error)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs, "%s", error);
+ free (error);
+ return false;
+ }
+
+ *formats = pool_nalloc (pool, n_vars + 1, sizeof **formats);
+ *n_formats = n_vars + 1;
+ (*formats)[0].type = (enum fmt_type) PRS_TYPE_T;
+ (*formats)[0].w = fc;
+ for (size_t i = 1; i <= n_vars; i++)
+ (*formats)[i] = format;
+ return true;
+}
+
+/* Implements parse_var_placements for Fortran-like formats. */
+static bool
+fixed_parse_fortran (struct lexer *lexer, struct pool *pool, enum fmt_use use,
+ struct fmt_spec **formats, size_t *n_formats)
+{
+ size_t formats_allocated = 0;
+ size_t formats_used = 0;
+
+ *formats = NULL;
+ while (!lex_match (lexer, T_RPAREN))
+ {
+ struct fmt_spec f;
+ struct fmt_spec *new_formats;
+ size_t n_new_formats;
+ size_t count;
+ size_t formats_needed;
+
+ /* Parse count. */
+ if (lex_is_integer (lexer))
+ {
+ count = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ count = 1;
+
+ /* Parse format specifier. */
+ if (lex_match (lexer, T_LPAREN))
+ {
+ /* Call ourselves recursively to handle parentheses. */
+ if (!fixed_parse_fortran (lexer, pool, use,
+ &new_formats, &n_new_formats))
+ return false;
+ }
+ else
+ {
+ new_formats = &f;
+ n_new_formats = 1;
+ if (use == FMT_FOR_INPUT && lex_match (lexer, T_SLASH))
+ f.type = (enum fmt_type) PRS_TYPE_NEW_REC;
+ else
+ {
+ int ofs = lex_ofs (lexer);
+ char type[FMT_TYPE_LEN_MAX + 1];
+ if (!parse_abstract_format_specifier (lexer, type, &f.w, &f.d))
+ return false;
+
+ if (!c_strcasecmp (type, "T"))
+ f.type = (enum fmt_type) PRS_TYPE_T;
+ else if (!c_strcasecmp (type, "X"))
+ {
+ f.type = (enum fmt_type) PRS_TYPE_X;
+ f.w = count;
+ count = 1;
+ }
+ else
+ {
+ if (!fmt_from_name (type, &f.type))
+ {
+ lex_ofs_error (lexer, ofs, ofs,
+ _("Unknown format type `%s'."), type);
+ return false;
+ }
+ char *error = fmt_check__ (&f, use);
+ if (error)
+ {
+ lex_ofs_error (lexer, ofs, ofs, "%s", error);
+ free (error);
+ return false;
+ }
+ }
+ }
+ }
+
+ /* Add COUNT copies of the NEW_FORMAT_CNT formats in
+ NEW_FORMATS to FORMATS. */
+ if (n_new_formats != 0
+ && size_overflow_p (xtimes (xsum (formats_used,
+ xtimes (count, n_new_formats)),
+ sizeof *formats)))
+ xalloc_die ();
+ formats_needed = count * n_new_formats;
+ if (formats_used + formats_needed > formats_allocated)
+ {
+ formats_allocated = formats_used + formats_needed;
+ *formats = pool_2nrealloc (pool, *formats, &formats_allocated,
+ sizeof **formats);
+ }
+ for (; count > 0; count--)
+ {
+ memcpy (&(*formats)[formats_used], new_formats,
+ sizeof **formats * n_new_formats);
+ formats_used += n_new_formats;
+ }
+
+ lex_match (lexer, T_COMMA);
+ }
+
+ *n_formats = formats_used;
+ return true;
+}
+
+/* Checks whether FORMAT represents one of the special "formats"
+ for T, X, or /. If so, updates *RECORD or *COLUMN (or both)
+ as appropriate, and returns true. Otherwise, returns false
+ without any side effects. */
+bool
+execute_placement_format (const struct fmt_spec *format,
+ int *record, int *column)
+{
+ switch ((int) format->type)
+ {
+ case PRS_TYPE_X:
+ *column += format->w;
+ return true;
+
+ case PRS_TYPE_T:
+ *column = format->w;
+ return true;
+
+ case PRS_TYPE_NEW_REC:
+ (*record)++;
+ *column = 1;
+ return true;
+
+ default:
+ assert (format->type < FMT_NUMBER_OF_FORMATS);
+ return false;
+ }
+}
+
+static bool
+parse_column__ (struct lexer *lexer, bool negative, int base, int *column)
+{
+ assert (base == 0 || base == 1);
+
+ if (!lex_force_int (lexer))
+ return false;
+ long int value = lex_integer (lexer);
+ if (negative)
+ value = -value;
+ lex_get (lexer);
+
+ *column = value - base + 1;
+ if (*column < 1)
+ {
+ if (base == 1)
+ lex_next_error (lexer, -1, -1,
+ _("Column positions for fields must be positive."));
+ else
+ lex_next_error (lexer, -1, -1,
+ _("Column positions for fields must not be negative."));
+ return false;
+ }
+ return true;
+}
+
+/* Parses a BASE-based column using LEXER. Returns true and
+ stores a 1-based column number into *COLUMN if successful,
+ otherwise emits an error message and returns false.
+
+ If BASE is 0, zero-based column numbers are parsed; if BASE is
+ 1, 1-based column numbers are parsed. Regardless of BASE, the
+ values stored in *FIRST_COLUMN and *LAST_COLUMN are
+ 1-based. */
+bool
+parse_column (struct lexer *lexer, int base, int *column)
+{
+ return parse_column__ (lexer, false, base, column);
+}
+
+/* Parse a column or a range of columns, specified as a single
+ integer or two integers delimited by a dash. Stores the range
+ in *FIRST_COLUMN and *LAST_COLUMN. (If only a single integer
+ is given, it is stored in both.) If RANGE_SPECIFIED is
+ non-null, then *RANGE_SPECIFIED is set to true if the syntax
+ contained a dash, false otherwise. Returns true if
+ successful, false if the syntax was invalid or the values
+ specified did not make sense.
+
+ If BASE is 0, zero-based column numbers are parsed; if BASE is
+ 1, 1-based column numbers are parsed. Regardless of BASE, the
+ values stored in *FIRST_COLUMN and *LAST_COLUMN are
+ 1-based. */
+bool
+parse_column_range (struct lexer *lexer, int base,
+ int *first_column, int *last_column,
+ bool *range_specified)
+{
+ int start_ofs = lex_ofs (lexer);
+
+ /* First column. */
+ if (!parse_column__ (lexer, false, base, first_column))
+ return false;
+
+ /* Last column. */
+ if (lex_is_integer (lexer) && lex_integer (lexer) < 0)
+ {
+ if (!parse_column__ (lexer, true, base, last_column))
+ return false;
+
+ if (*last_column < *first_column)
+ {
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("The ending column for a field must be "
+ "greater than the starting column."));
+ return false;
+ }
+
+ if (range_specified)
+ *range_specified = true;
+ }
+ else
+ {
+ *last_column = *first_column;
+ if (range_specified)
+ *range_specified = false;
+ }
+
+ return true;
+}
+
+/* Parses a (possibly empty) sequence of slashes, each of which
+ may be followed by an integer. A slash on its own increases
+ *RECORD by 1 and sets *COLUMN to 1. A slash followed by an
+ integer sets *RECORD to the integer, as long as that increases
+ *RECORD, and sets *COLUMN to 1.
+
+ Returns true if successful, false on syntax error. */
+bool
+parse_record_placement (struct lexer *lexer, int *record, int *column)
+{
+ while (lex_match (lexer, T_SLASH))
+ {
+ if (lex_is_number (lexer))
+ {
+ if (!lex_force_int_range (lexer, NULL, *record + 1, INT_MAX))
+ return false;
+ *record = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else
+ (*record)++;
+ *column = 1;
+ }
+ assert (*record >= 1);
+
+ return true;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef LANGUAGE_DATA_IO_PLACEMENT_PARSER_H
+#define LANGUAGE_DATA_IO_PLACEMENT_PARSER_H 1
+
+#include <stdbool.h>
+#include <stddef.h>
+#include "data/format.h"
+
+struct pool;
+struct lexer;
+
+bool parse_record_placement (struct lexer *, int *record, int *column);
+bool parse_var_placements (struct lexer *, struct pool *, size_t n_vars,
+ enum fmt_use,
+ struct fmt_spec **, size_t *n_formats);
+bool execute_placement_format (const struct fmt_spec *,
+ int *record, int *column);
+bool parse_column (struct lexer *lexer, int base, int *column);
+bool parse_column_range (struct lexer *, int base,
+ int *first_column, int *last_column,
+ bool *range_specified);
+
+#endif /* language/commands/placement-parser.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <limits.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/value.h"
+#include "language/command.h"
+#include "language/commands/data-writer.h"
+#include "language/commands/file-handle.h"
+#include "language/expressions/public.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "output/driver.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* PRINT SPACE transformation. */
+struct print_space_trns
+ {
+ struct dfm_writer *writer; /* Output data file. */
+ struct expression *expr; /* Number of lines; NULL means 1. */
+ struct msg_location *expr_location;
+ };
+
+static const struct trns_class print_space_class;
+
+int
+cmd_print_space (struct lexer *lexer, struct dataset *ds)
+{
+ struct file_handle *handle = NULL;
+ struct expression *expr = NULL;
+ struct msg_location *expr_location = NULL;
+ char *encoding = NULL;
+
+ if (lex_match_id (lexer, "OUTFILE"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ handle = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (handle == NULL)
+ return CMD_FAILURE;
+
+ if (lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ }
+ }
+ else
+ handle = NULL;
+
+ if (lex_token (lexer) != T_ENDCMD)
+ {
+ int start_ofs = lex_ofs (lexer);
+ expr = expr_parse (lexer, ds, VAL_NUMERIC);
+ int end_ofs = lex_ofs (lexer) - 1;
+ expr_location = lex_ofs_location (lexer, start_ofs, end_ofs);
+ if (!expr)
+ goto error;
+
+ if (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_error (lexer, _("Syntax error expecting end of command."));
+ goto error;
+ }
+ }
+ else
+ expr = NULL;
+
+ struct dfm_writer *writer = NULL;
+ if (handle != NULL)
+ {
+ writer = dfm_open_writer (handle, encoding);
+ if (writer == NULL)
+ goto error;
+ }
+
+ struct print_space_trns *trns = xmalloc (sizeof *trns);
+ *trns = (struct print_space_trns) {
+ .writer = writer,
+ .expr = expr,
+ .expr_location = expr_location,
+ };
+
+ add_transformation (ds, &print_space_class, trns);
+ fh_unref (handle);
+ free (encoding);
+ return CMD_SUCCESS;
+
+error:
+ msg_location_destroy (expr_location);
+ fh_unref (handle);
+ expr_free (expr);
+ free (encoding);
+ return CMD_FAILURE;
+}
+
+/* Executes a PRINT SPACE transformation. */
+static enum trns_result
+print_space_trns_proc (void *t_, struct ccase **c,
+ casenumber case_num UNUSED)
+{
+ struct print_space_trns *trns = t_;
+ int n;
+
+ n = 1;
+ if (trns->expr)
+ {
+ double f = expr_evaluate_num (trns->expr, *c, case_num);
+ if (f == SYSMIS)
+ msg_at (SW, trns->expr_location,
+ _("The expression on %s evaluated to the "
+ "system-missing value."), "PRINT SPACE");
+ else if (f < 0 || f > INT_MAX)
+ msg_at (SW, trns->expr_location,
+ _("The expression on %s evaluated to %g."), "PRINT SPACE", f);
+ else
+ n = f;
+ }
+
+ while (n--)
+ if (trns->writer == NULL)
+ output_log ("%s", "");
+ else
+ dfm_put_record (trns->writer, " ", 1); /* XXX */
+
+ if (trns->writer != NULL && dfm_write_error (trns->writer))
+ return TRNS_ERROR;
+ return TRNS_CONTINUE;
+}
+
+/* Frees a PRINT SPACE transformation.
+ Returns true if successful, false if an I/O error occurred. */
+static bool
+print_space_trns_free (void *trns_)
+{
+ struct print_space_trns *trns = trns_;
+ bool ok = dfm_close_writer (trns->writer);
+ expr_free (trns->expr);
+ msg_location_destroy (trns->expr_location);
+ free (trns);
+ return ok;
+}
+
+static const struct trns_class print_space_class = {
+ .name = "PRINT SPACE",
+ .execute = print_space_trns_proc,
+ .destroy = print_space_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+#include <uniwidth.h>
+
+#include "data/case.h"
+#include "data/dataset.h"
+#include "data/data-out.h"
+#include "data/format.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/data-writer.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/placement-parser.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/u8-line.h"
+#include "output/driver.h"
+#include "output/pivot-table.h"
+#include "output/output-item.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+/* Describes what to do when an output field is encountered. */
+enum field_type
+ {
+ PRT_LITERAL, /* Literal string. */
+ PRT_VAR /* Variable. */
+ };
+
+/* Describes how to output one field. */
+struct prt_out_spec
+ {
+ /* All fields. */
+ enum field_type type; /* What type of field this is. */
+ int record; /* 1-based record number. */
+ int first_column; /* 0-based first column. */
+ int start_ofs, end_ofs;
+
+ /* PRT_VAR only. */
+ const struct variable *var; /* Associated variable. */
+ struct fmt_spec format; /* Output spec. */
+ bool add_space; /* Add trailing space? */
+ bool sysmis_as_spaces; /* Output SYSMIS as spaces? */
+
+ /* PRT_LITERAL only. */
+ struct substring string; /* String to output. */
+ int width; /* Width of 'string', in display columns. */
+ };
+
+/* PRINT, PRINT EJECT, WRITE private data structure. */
+struct print_trns
+ {
+ struct pool *pool; /* Stores related data. */
+ bool eject; /* Eject page before printing? */
+ bool include_prefix; /* Prefix lines with space? */
+ const char *encoding; /* Encoding to use for output. */
+ struct dfm_writer *writer; /* Output file, NULL=listing file. */
+ struct prt_out_spec *specs;
+ size_t n_specs;
+ size_t n_records; /* Number of records to write. */
+ };
+
+enum which_formats
+ {
+ PRINT,
+ WRITE
+ };
+
+static const struct trns_class print_binary_trns_class;
+static const struct trns_class print_text_trns_class;
+
+static int cmd_print__ (struct lexer *, struct dataset *,
+ enum which_formats, bool eject);
+static bool parse_specs (struct lexer *, struct pool *tmp_pool,
+ struct print_trns *, int records_ofs,
+ struct dictionary *, enum which_formats);
+static void dump_table (struct print_trns *);
+
+static bool print_trns_free (void *trns_);
+
+static const struct prt_out_spec *find_binary_spec (const struct print_trns *);
+\f
+/* Basic parsing. */
+
+/* Parses PRINT command. */
+int
+cmd_print (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_print__ (lexer, ds, PRINT, false);
+}
+
+/* Parses PRINT EJECT command. */
+int
+cmd_print_eject (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_print__ (lexer, ds, PRINT, true);
+}
+
+/* Parses WRITE command. */
+int
+cmd_write (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_print__ (lexer, ds, WRITE, false);
+}
+
+/* Parses the output commands. */
+static int
+cmd_print__ (struct lexer *lexer, struct dataset *ds,
+ enum which_formats which_formats, bool eject)
+{
+ bool print_table = false;
+ struct file_handle *fh = NULL;
+ char *encoding = NULL;
+
+ /* Fill in prt to facilitate error-handling. */
+ struct pool *pool = pool_create ();
+ struct print_trns *trns = pool_alloc (pool, sizeof *trns);
+ *trns = (struct print_trns) { .pool = pool, .eject = eject };
+ struct pool *tmp_pool = pool_create_subpool (trns->pool);
+
+ /* Parse the command options. */
+ int records_ofs = 0;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ if (lex_match_id (lexer, "OUTFILE"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ fh = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (fh == NULL)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "RECORDS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ lex_match (lexer, T_LPAREN);
+ if (!lex_force_int_range (lexer, "RECORDS", 0, INT_MAX))
+ goto error;
+ trns->n_records = lex_integer (lexer);
+ records_ofs = lex_ofs (lexer);
+ lex_get (lexer);
+ lex_match (lexer, T_RPAREN);
+ }
+ else if (lex_match_id (lexer, "TABLE"))
+ print_table = true;
+ else if (lex_match_id (lexer, "NOTABLE"))
+ print_table = false;
+ else
+ {
+ lex_error_expecting (lexer, "OUTFILE", "ENCODING", "RECORDS",
+ "TABLE", "NOTABLE");
+ goto error;
+ }
+ }
+
+ /* When PRINT or PRINT EJECT writes to an external file, we
+ prefix each line with a space for compatibility. */
+ trns->include_prefix = which_formats == PRINT && fh != NULL;
+
+ /* Parse variables and strings. */
+ if (!parse_specs (lexer, tmp_pool, trns, records_ofs,
+ dataset_dict (ds), which_formats))
+ goto error;
+
+ /* Are there any binary formats?
+
+ There are real difficulties figuring out what to do when both binary
+ formats and nontrivial encodings enter the picture. So when binary
+ formats are present we fall back to much simpler handling. */
+ const struct prt_out_spec *binary_spec = find_binary_spec (trns);
+ if (binary_spec && !fh)
+ {
+ lex_ofs_error (lexer, binary_spec->start_ofs, binary_spec->end_ofs,
+ _("%s is required when binary formats are specified."),
+ "OUTFILE");
+ goto error;
+ }
+
+ if (lex_end_of_command (lexer) != CMD_SUCCESS)
+ goto error;
+
+ if (fh != NULL)
+ {
+ trns->writer = dfm_open_writer (fh, encoding);
+ if (trns->writer == NULL)
+ goto error;
+ trns->encoding = dfm_writer_get_encoding (trns->writer);
+ }
+ else
+ trns->encoding = UTF8;
+
+ /* Output the variable table if requested. */
+ if (print_table)
+ dump_table (trns);
+
+ /* Put the transformation in the queue. */
+ add_transformation (ds, (binary_spec
+ ? &print_binary_trns_class
+ : &print_text_trns_class), trns);
+
+ pool_destroy (tmp_pool);
+ fh_unref (fh);
+
+ return CMD_SUCCESS;
+
+ error:
+ print_trns_free (trns);
+ fh_unref (fh);
+ return CMD_FAILURE;
+}
+\f
+static bool parse_string_argument (struct lexer *, struct print_trns *,
+ size_t *allocated_specs,
+ int record, int *column);
+static bool parse_variable_argument (struct lexer *, const struct dictionary *,
+ struct print_trns *,
+ size_t *allocated_specs,
+ struct pool *tmp_pool,
+ int *record, int *column,
+ enum which_formats);
+
+/* Parses all the variable and string specifications on a single
+ PRINT, PRINT EJECT, or WRITE command into the prt structure.
+ Returns success. */
+static bool
+parse_specs (struct lexer *lexer, struct pool *tmp_pool,
+ struct print_trns *trns, int records_ofs, struct dictionary *dict,
+ enum which_formats which_formats)
+{
+ int record = 0;
+ int column = 1;
+
+ if (lex_token (lexer) == T_ENDCMD)
+ {
+ trns->n_records = 1;
+ return true;
+ }
+
+ size_t allocated_specs = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (!parse_record_placement (lexer, &record, &column))
+ return false;
+
+ bool ok = (lex_is_string (lexer)
+ ? parse_string_argument (lexer, trns, &allocated_specs,
+ record, &column)
+ : parse_variable_argument (lexer, dict, trns, &allocated_specs,
+ tmp_pool, &record, &column,
+ which_formats));
+ if (!ok)
+ return 0;
+
+ lex_match (lexer, T_COMMA);
+ }
+
+ if (trns->n_records != 0 && trns->n_records != record)
+ lex_ofs_error (lexer, records_ofs, records_ofs,
+ _("Output calls for %d records but %zu specified on RECORDS "
+ "subcommand."),
+ record, trns->n_records);
+ trns->n_records = record;
+
+ return true;
+}
+
+static struct prt_out_spec *
+add_spec (struct print_trns *trns, size_t *allocated_specs)
+{
+ if (trns->n_specs >= *allocated_specs)
+ trns->specs = pool_2nrealloc (trns->pool, trns->specs, allocated_specs,
+ sizeof *trns->specs);
+ return &trns->specs[trns->n_specs++];
+}
+
+/* Parses a string argument to the PRINT commands. Returns success. */
+static bool
+parse_string_argument (struct lexer *lexer, struct print_trns *trns,
+ size_t *allocated_specs, int record, int *column)
+{
+ struct prt_out_spec *spec = add_spec (trns, allocated_specs);
+ *spec = (struct prt_out_spec) {
+ .type = PRT_LITERAL,
+ .record = record,
+ .first_column = *column,
+ .string = ss_clone_pool (lex_tokss (lexer), trns->pool),
+ .start_ofs = lex_ofs (lexer),
+ };
+ lex_get (lexer);
+
+ /* Parse the included column range. */
+ if (lex_is_number (lexer))
+ {
+ int first_column, last_column;
+ bool range_specified;
+
+ if (!parse_column_range (lexer, 1,
+ &first_column, &last_column, &range_specified))
+ return false;
+
+ spec->first_column = first_column;
+ if (range_specified)
+ {
+ struct string s;
+ ds_init_substring (&s, spec->string);
+ ds_set_length (&s, last_column - first_column + 1, ' ');
+ spec->string = ss_clone_pool (s.ss, trns->pool);
+ ds_destroy (&s);
+ }
+ }
+ spec->end_ofs = lex_ofs (lexer) - 1;
+
+ spec->width = u8_width (CHAR_CAST (const uint8_t *, spec->string.string),
+ spec->string.length, UTF8);
+ *column = spec->first_column + spec->width;
+
+ return true;
+}
+
+/* Parses a variable argument to the PRINT commands by passing it off
+ to fixed_parse_compatible() or fixed_parse_fortran() as appropriate.
+ Returns success. */
+static bool
+parse_variable_argument (struct lexer *lexer, const struct dictionary *dict,
+ struct print_trns *trns, size_t *allocated_specs,
+ struct pool *tmp_pool, int *record, int *column,
+ enum which_formats which_formats)
+{
+ const struct variable **vars;
+ size_t n_vars;
+ if (!parse_variables_const_pool (lexer, tmp_pool, dict,
+ &vars, &n_vars, PV_DUPLICATE))
+ return false;
+
+ struct fmt_spec *formats, *f;
+ size_t n_formats;
+ bool add_space;
+ int formats_start = lex_ofs (lexer);
+ if (lex_is_number (lexer) || lex_token (lexer) == T_LPAREN)
+ {
+ if (!parse_var_placements (lexer, tmp_pool, n_vars, FMT_FOR_OUTPUT,
+ &formats, &n_formats))
+ return false;
+ add_space = false;
+ }
+ else
+ {
+ lex_match (lexer, T_ASTERISK);
+
+ formats = pool_nmalloc (tmp_pool, n_vars, sizeof *formats);
+ n_formats = n_vars;
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ const struct variable *v = vars[i];
+ formats[i] = (which_formats == PRINT
+ ? *var_get_print_format (v)
+ : *var_get_write_format (v));
+ }
+ add_space = which_formats == PRINT;
+ }
+ int formats_end = lex_ofs (lexer) - 1;
+
+ size_t var_idx = 0;
+ for (f = formats; f < &formats[n_formats]; f++)
+ if (!execute_placement_format (f, record, column))
+ {
+ const struct variable *var = vars[var_idx++];
+ char *error = fmt_check_width_compat__ (f, var_get_name (var),
+ var_get_width (var));
+ if (error)
+ {
+ lex_ofs_error (lexer, formats_start, formats_end, "%s", error);
+ free (error);
+ return false;
+ }
+
+ struct prt_out_spec *spec = add_spec (trns, allocated_specs);
+ *spec = (struct prt_out_spec) {
+ .type = PRT_VAR,
+ .record = *record,
+ .first_column = *column,
+ .var = var,
+ .format = *f,
+ .add_space = add_space,
+
+ /* This is a completely bizarre twist for compatibility: WRITE
+ outputs the system-missing value as a field filled with spaces,
+ instead of using the normal format that usually contains a
+ period. */
+ .sysmis_as_spaces = (which_formats == WRITE
+ && var_is_numeric (var)
+ && (fmt_get_category (f->type)
+ != FMT_CAT_BINARY)),
+ };
+
+ *column += f->w + add_space;
+ }
+ assert (var_idx == n_vars);
+
+ return true;
+}
+
+/* Prints the table produced by the TABLE subcommand to the listing
+ file. */
+static void
+dump_table (struct print_trns *trns)
+{
+ struct pivot_table *table = pivot_table_create (N_("Print Summary"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Attributes"),
+ N_("Record"), N_("Columns"), N_("Format"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t i = 0; i < trns->n_specs; i++)
+ {
+ const struct prt_out_spec *spec = &trns->specs[i];
+ if (spec->type != PRT_VAR)
+ continue;
+
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (spec->var));
+
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_integer (spec->record));
+ int last_column = spec->first_column + spec->format.w - 1;
+ pivot_table_put2 (table, 1, row, pivot_value_new_user_text_nocopy (
+ xasprintf ("%d-%d",
+ spec->first_column, last_column)));
+
+ char fmt_string[FMT_STRING_LEN_MAX + 1];
+ pivot_table_put2 (table, 2, row, pivot_value_new_user_text (
+ fmt_to_string (&spec->format, fmt_string), -1));
+ }
+
+ int row = pivot_category_create_leaf (
+ variables->root, pivot_value_new_text (N_("N of Records")));
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_integer (trns->n_records));
+
+ pivot_table_submit (table);
+}
+
+static const struct prt_out_spec *
+find_binary_spec (const struct print_trns *trns)
+{
+ for (size_t i = 0; i < trns->n_specs; i++)
+ {
+ const struct prt_out_spec *spec = &trns->specs[i];
+ if (spec->type == PRT_VAR
+ && fmt_get_category (spec->format.type) == FMT_CAT_BINARY)
+ return spec;
+ }
+ return NULL;
+}
+\f
+/* Transformation, for all-text output. */
+
+static void print_text_flush_records (struct print_trns *, struct u8_line *,
+ int target_record,
+ bool *eject, int *record);
+
+/* Performs the transformation inside print_trns T on case C. */
+static enum trns_result
+print_text_trns_proc (void *trns_, struct ccase **c,
+ casenumber case_num UNUSED)
+{
+ struct print_trns *trns = trns_;
+ struct u8_line line;
+
+ bool eject = trns->eject;
+ int record = 1;
+
+ u8_line_init (&line);
+ for (size_t i = 0; i < trns->n_specs; i++)
+ {
+ const struct prt_out_spec *spec = &trns->specs[i];
+ int x0 = spec->first_column;
+
+ print_text_flush_records (trns, &line, spec->record, &eject, &record);
+
+ u8_line_set_length (&line, spec->first_column);
+ if (spec->type == PRT_VAR)
+ {
+ const union value *input = case_data (*c, spec->var);
+ int x1;
+
+ if (!spec->sysmis_as_spaces || input->f != SYSMIS)
+ {
+ size_t len;
+ int width;
+ char *s;
+
+ s = data_out (input, var_get_encoding (spec->var),
+ &spec->format, settings_get_fmt_settings ());
+ len = strlen (s);
+ width = u8_width (CHAR_CAST (const uint8_t *, s), len, UTF8);
+ x1 = x0 + width;
+ u8_line_put (&line, x0, x1, s, len);
+ free (s);
+ }
+ else
+ {
+ int n = spec->format.w;
+
+ x1 = x0 + n;
+ memset (u8_line_reserve (&line, x0, x1, n), ' ', n);
+ }
+
+ if (spec->add_space)
+ *u8_line_reserve (&line, x1, x1 + 1, 1) = ' ';
+ }
+ else
+ {
+ const struct substring *s = &spec->string;
+
+ u8_line_put (&line, x0, x0 + spec->width, s->string, s->length);
+ }
+ }
+ print_text_flush_records (trns, &line, trns->n_records + 1,
+ &eject, &record);
+ u8_line_destroy (&line);
+
+ if (trns->writer != NULL && dfm_write_error (trns->writer))
+ return TRNS_ERROR;
+ return TRNS_CONTINUE;
+}
+
+/* Advance from *RECORD to TARGET_RECORD, outputting records
+ along the way. If *EJECT is true, then the first record
+ output is preceded by ejecting the page (and *EJECT is set
+ false). */
+static void
+print_text_flush_records (struct print_trns *trns, struct u8_line *line,
+ int target_record, bool *eject, int *record)
+{
+ for (; target_record > *record; (*record)++)
+ {
+ char leader = ' ';
+
+ if (*eject)
+ {
+ *eject = false;
+ if (trns->writer == NULL)
+ output_item_submit (page_break_item_create ());
+ else
+ leader = '1';
+ }
+ *u8_line_reserve (line, 0, 1, 1) = leader;
+
+ if (trns->writer == NULL)
+ output_log ("%s", ds_cstr (&line->s) + 1);
+ else
+ {
+ size_t len = ds_length (&line->s);
+ char *s = ds_cstr (&line->s);
+
+ if (!trns->include_prefix)
+ {
+ s++;
+ len--;
+ }
+
+ dfm_put_record_utf8 (trns->writer, s, len);
+ }
+ }
+}
+\f
+/* Transformation, for output involving binary. */
+
+static void print_binary_flush_records (struct print_trns *,
+ struct string *line, int target_record,
+ bool *eject, int *record);
+
+/* Performs the transformation inside print_trns T on case C. */
+static enum trns_result
+print_binary_trns_proc (void *trns_, struct ccase **c,
+ casenumber case_num UNUSED)
+{
+ struct print_trns *trns = trns_;
+ bool eject = trns->eject;
+ char encoded_space = recode_byte (trns->encoding, C_ENCODING, ' ');
+ int record = 1;
+ struct string line = DS_EMPTY_INITIALIZER;
+
+ ds_put_byte (&line, ' ');
+ for (size_t i = 0; i < trns->n_specs; i++)
+ {
+ const struct prt_out_spec *spec = &trns->specs[i];
+ print_binary_flush_records (trns, &line, spec->record, &eject, &record);
+
+ ds_set_length (&line, spec->first_column, encoded_space);
+ if (spec->type == PRT_VAR)
+ {
+ const union value *input = case_data (*c, spec->var);
+ if (!spec->sysmis_as_spaces || input->f != SYSMIS)
+ data_out_recode (input, var_get_encoding (spec->var),
+ &spec->format, settings_get_fmt_settings (),
+ &line, trns->encoding);
+ else
+ ds_put_byte_multiple (&line, encoded_space, spec->format.w);
+ if (spec->add_space)
+ ds_put_byte (&line, encoded_space);
+ }
+ else
+ {
+ ds_put_substring (&line, spec->string);
+ if (0 != strcmp (trns->encoding, UTF8))
+ {
+ size_t length = spec->string.length;
+ char *data = ss_data (ds_tail (&line, length));
+ char *s = recode_string (trns->encoding, UTF8, data, length);
+ memcpy (data, s, length);
+ free (s);
+ }
+ }
+ }
+ print_binary_flush_records (trns, &line, trns->n_records + 1,
+ &eject, &record);
+ ds_destroy (&line);
+
+ if (trns->writer != NULL && dfm_write_error (trns->writer))
+ return TRNS_ERROR;
+ return TRNS_CONTINUE;
+}
+
+/* Advance from *RECORD to TARGET_RECORD, outputting records
+ along the way. If *EJECT is true, then the first record
+ output is preceded by ejecting the page (and *EJECT is set
+ false). */
+static void
+print_binary_flush_records (struct print_trns *trns, struct string *line,
+ int target_record, bool *eject, int *record)
+{
+ for (; target_record > *record; (*record)++)
+ {
+ char *s = ds_cstr (line);
+ size_t length = ds_length (line);
+ char leader = ' ';
+
+ if (*eject)
+ {
+ *eject = false;
+ leader = '1';
+ }
+ s[0] = recode_byte (trns->encoding, C_ENCODING, leader);
+
+ if (!trns->include_prefix)
+ {
+ s++;
+ length--;
+ }
+ dfm_put_record (trns->writer, s, length);
+
+ ds_truncate (line, 1);
+ }
+}
+\f
+/* Frees TRNS. */
+static bool
+print_trns_free (void *trns_)
+{
+ struct print_trns *trns = trns_;
+ bool ok = true;
+
+ if (trns->writer != NULL)
+ ok = dfm_close_writer (trns->writer);
+ pool_destroy (trns->pool);
+
+ return ok;
+}
+
+static const struct trns_class print_binary_trns_class = {
+ .name = "PRINT",
+ .execute = print_binary_trns_proc,
+ .destroy = print_trns_free,
+};
+
+static const struct trns_class print_text_trns_class = {
+ .name = "PRINT",
+ .execute = print_text_trns_proc,
+ .destroy = print_trns_free,
+};
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011, 2012, 2015, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_matrix.h>
+#include <gsl/gsl_permutation.h>
+#include <gsl/gsl_sort_vector.h>
+#include <gsl/gsl_statistics.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/assertion.h"
+#include "libpspp/str.h"
+#include "math/random.h"
+#include "output/pivot-table.h"
+#include "output/output-item.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+enum missing_type
+ {
+ MISS_LISTWISE,
+ MISS_PAIRWISE,
+ };
+
+
+struct save_trans_data
+ {
+ /* A writer which contains the values (if any) to be appended to
+ each case in the active dataset */
+ struct casewriter *writer;
+
+ /* A reader created from the writer above. */
+ struct casereader *appending_reader;
+
+ /* The indices to be used to access values in the above,
+ reader/writer */
+ int membership_case_idx;
+ int distance_case_idx;
+
+ /* The variables created to hold the values appended to the dataset */
+ struct variable *membership;
+ struct variable *distance;
+ };
+
+
+struct qc
+ {
+ struct dataset *dataset;
+ struct dictionary *dict;
+
+ const struct variable **vars;
+ size_t n_vars;
+
+ double epsilon; /* The convergence criterion */
+
+ int ngroups; /* Number of group. (Given by the user) */
+ int maxiter; /* Maximum iterations (Given by the user) */
+ bool print_cluster_membership; /* true => print membership */
+ bool print_initial_clusters; /* true => print initial cluster */
+ bool initial; /* false => simplified initial cluster selection */
+ bool update; /* false => do not iterate */
+
+ const struct variable *wv; /* Weighting variable. */
+
+ enum missing_type missing_type;
+ enum mv_class exclude;
+
+ /* Which values are to be saved? */
+ bool save_membership;
+ bool save_distance;
+
+ /* The name of the new variable to contain the cluster of each case. */
+ char *var_membership;
+
+ /* The name of the new variable to contain the distance of each case
+ from its cluster centre. */
+ char *var_distance;
+
+ struct save_trans_data *save_trans_data;
+ };
+
+/* Holds all of the information for the functions. int n, holds the number of
+ observation and its default value is -1. We set it in
+ kmeans_recalculate_centers in first invocation. */
+struct Kmeans
+ {
+ gsl_matrix *centers; /* Centers for groups. */
+ gsl_matrix *updated_centers;
+ casenumber n;
+
+ gsl_vector_long *num_elements_groups;
+
+ gsl_matrix *initial_centers; /* Initial random centers. */
+ double convergence_criteria;
+ gsl_permutation *group_order; /* Group order for reporting. */
+ };
+
+static struct Kmeans *kmeans_create (const struct qc *);
+
+static void kmeans_get_nearest_group (const struct Kmeans *,
+ struct ccase *, const struct qc *,
+ int *, double *, int *, double *);
+
+static void kmeans_order_groups (struct Kmeans *, const struct qc *);
+
+static void kmeans_cluster (struct Kmeans *, struct casereader *,
+ const struct qc *);
+
+static void quick_cluster_show_centers (struct Kmeans *, bool initial,
+ const struct qc *);
+
+static void quick_cluster_show_membership (struct Kmeans *,
+ const struct casereader *,
+ struct qc *);
+
+static void quick_cluster_show_number_cases (struct Kmeans *,
+ const struct qc *);
+
+static void quick_cluster_show_results (struct Kmeans *,
+ const struct casereader *,
+ struct qc *);
+
+int cmd_quick_cluster (struct lexer *, struct dataset *);
+
+static void kmeans_destroy (struct Kmeans *);
+
+/* Creates and returns a struct of Kmeans with given casereader 'cs', parsed
+ variables 'variables', number of cases 'n', number of variables 'm', number
+ of clusters and amount of maximum iterations. */
+static struct Kmeans *
+kmeans_create (const struct qc *qc)
+{
+ struct Kmeans *kmeans = xmalloc (sizeof *kmeans);
+ *kmeans = (struct Kmeans) {
+ .centers = gsl_matrix_alloc (qc->ngroups, qc->n_vars),
+ .updated_centers = gsl_matrix_alloc (qc->ngroups, qc->n_vars),
+ .num_elements_groups = gsl_vector_long_alloc (qc->ngroups),
+ .group_order = gsl_permutation_alloc (qc->ngroups),
+ };
+ return kmeans;
+}
+
+static void
+kmeans_destroy (struct Kmeans *kmeans)
+{
+ gsl_matrix_free (kmeans->centers);
+ gsl_matrix_free (kmeans->updated_centers);
+ gsl_matrix_free (kmeans->initial_centers);
+
+ gsl_vector_long_free (kmeans->num_elements_groups);
+
+ gsl_permutation_free (kmeans->group_order);
+
+ free (kmeans);
+}
+
+static double
+diff_matrix (const gsl_matrix *m1, const gsl_matrix *m2)
+{
+ double max_diff = -INFINITY;
+ for (size_t i = 0; i < m1->size1; ++i)
+ {
+ double diff = 0;
+ for (size_t j = 0; j < m1->size2; ++j)
+ diff += pow2 (gsl_matrix_get (m1,i,j) - gsl_matrix_get (m2,i,j));
+ if (diff > max_diff)
+ max_diff = diff;
+ }
+
+ return max_diff;
+}
+
+
+
+static double
+matrix_mindist (const gsl_matrix *m, int *mn, int *mm)
+{
+ double mindist = INFINITY;
+ for (size_t i = 0; i + 1 < m->size1; ++i)
+ for (size_t j = i + 1; j < m->size1; ++j)
+ {
+ double diff_sq = 0;
+ for (size_t k = 0; k < m->size2; ++k)
+ diff_sq += pow2 (gsl_matrix_get (m, j, k) - gsl_matrix_get (m, i, k));
+ if (diff_sq < mindist)
+ {
+ mindist = diff_sq;
+ if (mn)
+ *mn = i;
+ if (mm)
+ *mm = j;
+ }
+ }
+ return mindist;
+}
+
+/* Return the distance of C from the group whose index is WHICH */
+static double
+dist_from_case (const struct Kmeans *kmeans, const struct ccase *c,
+ const struct qc *qc, int which)
+{
+ double dist = 0;
+ for (size_t j = 0; j < qc->n_vars; j++)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ assert (!(var_is_value_missing (qc->vars[j], val) & qc->exclude));
+ dist += pow2 (gsl_matrix_get (kmeans->centers, which, j) - val->f);
+ }
+
+ return dist;
+}
+
+/* Return the minimum distance of the group WHICH and all other groups */
+static double
+min_dist_from (const struct Kmeans *kmeans, const struct qc *qc, int which)
+{
+ double mindist = INFINITY;
+ for (size_t i = 0; i < qc->ngroups; i++)
+ {
+ if (i == which)
+ continue;
+
+ double dist = 0;
+ for (size_t j = 0; j < qc->n_vars; j++)
+ dist += pow2 (gsl_matrix_get (kmeans->centers, i, j)
+ - gsl_matrix_get (kmeans->centers, which, j));
+
+ if (dist < mindist)
+ mindist = dist;
+ }
+
+ return mindist;
+}
+
+/* Calculate the initial cluster centers. */
+static void
+kmeans_initial_centers (struct Kmeans *kmeans,
+ const struct casereader *reader,
+ const struct qc *qc)
+{
+ int nc = 0;
+
+ struct casereader *cs = casereader_clone (reader);
+ struct ccase *c;
+ for (; (c = casereader_read (cs)) != NULL; case_unref (c))
+ {
+ bool missing = false;
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
+ {
+ missing = true;
+ break;
+ }
+
+ if (nc < qc->ngroups)
+ gsl_matrix_set (kmeans->centers, nc, j, val->f);
+ }
+ if (missing)
+ continue;
+
+ if (nc++ < qc->ngroups)
+ continue;
+
+ if (qc->initial)
+ {
+ int mn, mm;
+ double m = matrix_mindist (kmeans->centers, &mn, &mm);
+
+ int mq, mp;
+ double delta;
+ kmeans_get_nearest_group (kmeans, c, qc, &mq, &delta, &mp, NULL);
+ if (delta > m)
+ /* If the distance between C and the nearest group, is greater than the distance
+ between the two groups which are clostest to each
+ other, then one group must be replaced. */
+ {
+ /* Out of mn and mm, which is the clostest of the two groups to C ? */
+ int which = (dist_from_case (kmeans, c, qc, mn)
+ > dist_from_case (kmeans, c, qc, mm)) ? mm : mn;
+
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ gsl_matrix_set (kmeans->centers, which, j, val->f);
+ }
+ }
+ else if (dist_from_case (kmeans, c, qc, mp) > min_dist_from (kmeans, qc, mq))
+ /* If the distance between C and the second nearest group
+ (MP) is greater than the smallest distance between the
+ nearest group (MQ) and any other group, then replace
+ MQ with C. */
+ {
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ gsl_matrix_set (kmeans->centers, mq, j, val->f);
+ }
+ }
+ }
+ }
+
+ casereader_destroy (cs);
+
+ kmeans->convergence_criteria = qc->epsilon * matrix_mindist (kmeans->centers, NULL, NULL);
+
+ /* As it is the first iteration, the variable kmeans->initial_centers is NULL
+ and it is created once for reporting issues. */
+ kmeans->initial_centers = gsl_matrix_alloc (qc->ngroups, qc->n_vars);
+ gsl_matrix_memcpy (kmeans->initial_centers, kmeans->centers);
+}
+
+/* Return the index of the group which is nearest to the case C */
+static void
+kmeans_get_nearest_group (const struct Kmeans *kmeans, struct ccase *c,
+ const struct qc *qc, int *g_q, double *delta_q,
+ int *g_p, double *delta_p)
+{
+ int result0 = -1;
+ int result1 = -1;
+ double mindist0 = INFINITY;
+ double mindist1 = INFINITY;
+ for (size_t i = 0; i < qc->ngroups; i++)
+ {
+ double dist = 0;
+ for (size_t j = 0; j < qc->n_vars; j++)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
+ continue;
+
+ dist += pow2 (gsl_matrix_get (kmeans->centers, i, j) - val->f);
+ }
+
+ if (dist < mindist0)
+ {
+ mindist1 = mindist0;
+ result1 = result0;
+
+ mindist0 = dist;
+ result0 = i;
+ }
+ else if (dist < mindist1)
+ {
+ mindist1 = dist;
+ result1 = i;
+ }
+ }
+
+ if (delta_q)
+ *delta_q = mindist0;
+
+ if (g_q)
+ *g_q = result0;
+
+ if (delta_p)
+ *delta_p = mindist1;
+
+ if (g_p)
+ *g_p = result1;
+}
+
+static void
+kmeans_order_groups (struct Kmeans *kmeans, const struct qc *qc)
+{
+ gsl_vector *v = gsl_vector_alloc (qc->ngroups);
+ gsl_matrix_get_col (v, kmeans->centers, 0);
+ gsl_sort_vector_index (kmeans->group_order, v);
+ gsl_vector_free (v);
+}
+
+/* Main algorithm.
+ Does iterations, checks convergency. */
+static void
+kmeans_cluster (struct Kmeans *kmeans, struct casereader *reader,
+ const struct qc *qc)
+{
+ kmeans_initial_centers (kmeans, reader, qc);
+
+ gsl_matrix_memcpy (kmeans->updated_centers, kmeans->centers);
+ for (int xx = 0; xx < qc->maxiter; ++xx)
+ {
+ gsl_vector_long_set_all (kmeans->num_elements_groups, 0.0);
+
+ kmeans->n = 0;
+ if (qc->update)
+ {
+ struct casereader *r = casereader_clone (reader);
+ struct ccase *c;
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ bool missing = false;
+ for (size_t j = 0; j < qc->n_vars; j++)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
+ missing = true;
+ }
+ if (missing)
+ continue;
+
+ double mindist = INFINITY;
+ int group = -1;
+ for (size_t g = 0; g < qc->ngroups; ++g)
+ {
+ double d = dist_from_case (kmeans, c, qc, g);
+
+ if (d < mindist)
+ {
+ mindist = d;
+ group = g;
+ }
+ }
+
+ long *n = gsl_vector_long_ptr (kmeans->num_elements_groups, group);
+ *n += qc->wv ? case_num (c, qc->wv) : 1.0;
+ kmeans->n++;
+
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
+ continue;
+ double *x = gsl_matrix_ptr (kmeans->updated_centers, group, j);
+ *x += val->f * (qc->wv ? case_num (c, qc->wv) : 1.0);
+ }
+ }
+
+ casereader_destroy (r);
+ }
+
+ /* Divide the cluster sums by the number of items in each cluster */
+ for (size_t g = 0; g < qc->ngroups; ++g)
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ long n = gsl_vector_long_get (kmeans->num_elements_groups, g);
+ double *x = gsl_matrix_ptr (kmeans->updated_centers, g, j);
+ *x /= n + 1; // Plus 1 for the initial centers
+ }
+ gsl_matrix_memcpy (kmeans->centers, kmeans->updated_centers);
+
+ kmeans->n = 0;
+ /* Step 3 */
+ gsl_vector_long_set_all (kmeans->num_elements_groups, 0.0);
+ gsl_matrix_set_all (kmeans->updated_centers, 0.0);
+ struct ccase *c;
+ struct casereader *cs = casereader_clone (reader);
+ for (; (c = casereader_read (cs)) != NULL; case_unref (c))
+ {
+ int group = -1;
+ kmeans_get_nearest_group (kmeans, c, qc, &group, NULL, NULL, NULL);
+
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ const union value *val = case_data (c, qc->vars[j]);
+ if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
+ continue;
+
+ double *x = gsl_matrix_ptr (kmeans->updated_centers, group, j);
+ *x += val->f;
+ }
+
+ long *n = gsl_vector_long_ptr (kmeans->num_elements_groups, group);
+ *n += qc->wv ? case_num (c, qc->wv) : 1.0;
+ kmeans->n++;
+ }
+ casereader_destroy (cs);
+
+ /* Divide the cluster sums by the number of items in each cluster */
+ for (size_t g = 0; g < qc->ngroups; ++g)
+ for (size_t j = 0; j < qc->n_vars; ++j)
+ {
+ long n = gsl_vector_long_get (kmeans->num_elements_groups, g);
+ double *x = gsl_matrix_ptr (kmeans->updated_centers, g, j);
+ *x /= n;
+ }
+
+ double d = diff_matrix (kmeans->updated_centers, kmeans->centers);
+ if (d < kmeans->convergence_criteria)
+ break;
+
+ if (!qc->update)
+ break;
+ }
+}
+
+/* Reports centers of clusters.
+ Initial parameter is optional for future use.
+ If initial is true, initial cluster centers are reported. Otherwise,
+ resulted centers are reported. */
+static void
+quick_cluster_show_centers (struct Kmeans *kmeans, bool initial, const struct qc *qc)
+{
+ struct pivot_table *table
+ = pivot_table_create (initial
+ ? N_("Initial Cluster Centers")
+ : N_("Final Cluster Centers"));
+
+ struct pivot_dimension *clusters
+ = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Cluster"));
+
+ clusters->root->show_label = true;
+ for (size_t i = 0; i < qc->ngroups; i++)
+ pivot_category_create_leaf (clusters->root,
+ pivot_value_new_integer (i + 1));
+
+ struct pivot_dimension *variables
+ = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Variable"));
+
+ for (size_t i = 0; i < qc->n_vars; i++)
+ pivot_category_create_leaf (variables->root,
+ pivot_value_new_variable (qc->vars[i]));
+
+ const gsl_matrix *matrix = (initial
+ ? kmeans->initial_centers
+ : kmeans->centers);
+ for (size_t i = 0; i < qc->ngroups; i++)
+ for (size_t j = 0; j < qc->n_vars; j++)
+ {
+ double x = gsl_matrix_get (matrix, kmeans->group_order->data[i], j);
+ union value v = { .f = x };
+ pivot_table_put2 (table, i, j,
+ pivot_value_new_var_value (qc->vars[j], &v));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+/* A transformation function which juxtaposes the dataset with the
+ (pre-prepared) dataset containing membership and/or distance
+ values. */
+static enum trns_result
+save_trans_func (void *aux, struct ccase **c, casenumber x UNUSED)
+{
+ const struct save_trans_data *std = aux;
+ struct ccase *ca = casereader_read (std->appending_reader);
+ if (ca == NULL)
+ return TRNS_CONTINUE;
+
+ *c = case_unshare (*c);
+
+ if (std->membership_case_idx >= 0)
+ *case_num_rw (*c, std->membership) = case_num_idx (ca, std->membership_case_idx);
+
+ if (std->distance_case_idx >= 0)
+ *case_num_rw (*c, std->distance) = case_num_idx (ca, std->distance_case_idx);
+
+ case_unref (ca);
+
+ return TRNS_CONTINUE;
+}
+
+/* Free the resources of the transformation. */
+static bool
+save_trans_destroy (void *aux)
+{
+ struct save_trans_data *std = aux;
+ casereader_destroy (std->appending_reader);
+ free (std);
+ return true;
+}
+
+/* Reports cluster membership for each case, and is requested saves the
+ membership and the distance of the case from the cluster centre. */
+static void
+quick_cluster_show_membership (struct Kmeans *kmeans,
+ const struct casereader *reader,
+ struct qc *qc)
+{
+ struct pivot_table *table = NULL;
+ struct pivot_dimension *cases = NULL;
+ if (qc->print_cluster_membership)
+ {
+ table = pivot_table_create (N_("Cluster Membership"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Cluster"),
+ N_("Cluster"));
+
+ cases
+ = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Case Number"));
+
+ cases->root->show_label = true;
+ }
+
+ gsl_permutation *ip = gsl_permutation_alloc (qc->ngroups);
+ gsl_permutation_inverse (ip, kmeans->group_order);
+
+ struct caseproto *proto = caseproto_create ();
+ if (qc->save_membership || qc->save_distance)
+ {
+ /* Prepare data which may potentially be used in a
+ transformation appending new variables to the active
+ dataset. */
+ int idx = 0;
+ int membership_case_idx = -1;
+ if (qc->save_membership)
+ {
+ proto = caseproto_add_width (proto, 0);
+ membership_case_idx = idx++;
+ }
+
+ int distance_case_idx = -1;
+ if (qc->save_distance)
+ {
+ proto = caseproto_add_width (proto, 0);
+ distance_case_idx = idx++;
+ }
+
+ qc->save_trans_data = xmalloc (sizeof *qc->save_trans_data);
+ *qc->save_trans_data = (struct save_trans_data) {
+ .membership_case_idx = membership_case_idx,
+ .distance_case_idx = distance_case_idx,
+ .writer = autopaging_writer_create (proto),
+ };
+ }
+
+ struct casereader *cs = casereader_clone (reader);
+ struct ccase *c;
+ for (int i = 0; (c = casereader_read (cs)) != NULL; i++, case_unref (c))
+ {
+ assert (i < kmeans->n);
+ int clust;
+ kmeans_get_nearest_group (kmeans, c, qc, &clust, NULL, NULL, NULL);
+ int cluster = ip->data[clust];
+
+ if (qc->save_trans_data)
+ {
+ /* Calculate the membership and distance values. */
+ struct ccase *outc = case_create (proto);
+ if (qc->save_membership)
+ *case_num_rw_idx (outc, qc->save_trans_data->membership_case_idx) = cluster + 1;
+
+ if (qc->save_distance)
+ *case_num_rw_idx (outc, qc->save_trans_data->distance_case_idx)
+ = sqrt (dist_from_case (kmeans, c, qc, clust));
+
+ casewriter_write (qc->save_trans_data->writer, outc);
+ }
+
+ if (qc->print_cluster_membership)
+ {
+ /* Print the cluster membership to the table. */
+ int case_idx = pivot_category_create_leaf (cases->root,
+ pivot_value_new_integer (i + 1));
+ pivot_table_put2 (table, 0, case_idx,
+ pivot_value_new_integer (cluster + 1));
+ }
+ }
+
+ caseproto_unref (proto);
+ gsl_permutation_free (ip);
+
+ if (qc->print_cluster_membership)
+ pivot_table_submit (table);
+ casereader_destroy (cs);
+}
+
+
+/* Reports number of cases of each single cluster. */
+static void
+quick_cluster_show_number_cases (struct Kmeans *kmeans, const struct qc *qc)
+{
+ struct pivot_table *table
+ = pivot_table_create (N_("Number of Cases in each Cluster"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Count"));
+
+ struct pivot_dimension *clusters
+ = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Clusters"));
+
+ struct pivot_category *group
+ = pivot_category_create_group (clusters->root, N_("Cluster"));
+
+ long int total = 0;
+ for (int i = 0; i < qc->ngroups; i++)
+ {
+ int cluster_idx
+ = pivot_category_create_leaf (group, pivot_value_new_integer (i + 1));
+ int count = kmeans->num_elements_groups->data [kmeans->group_order->data[i]];
+ pivot_table_put2 (table, 0, cluster_idx, pivot_value_new_integer (count));
+ total += count;
+ }
+
+ int cluster_idx = pivot_category_create_leaf (clusters->root,
+ pivot_value_new_text (N_("Valid")));
+ pivot_table_put2 (table, 0, cluster_idx, pivot_value_new_integer (total));
+ pivot_table_submit (table);
+}
+
+/* Reports. */
+static void
+quick_cluster_show_results (struct Kmeans *kmeans, const struct casereader *reader,
+ struct qc *qc)
+{
+ kmeans_order_groups (kmeans, qc); /* what does this do? */
+
+ if (qc->print_initial_clusters)
+ quick_cluster_show_centers (kmeans, true, qc);
+ quick_cluster_show_centers (kmeans, false, qc);
+ quick_cluster_show_number_cases (kmeans, qc);
+
+ quick_cluster_show_membership (kmeans, reader, qc);
+}
+
+/* Parse the QUICK CLUSTER command and populate QC accordingly.
+ Returns false on error. */
+static bool
+quick_cluster_parse (struct lexer *lexer, struct qc *qc)
+{
+ if (!parse_variables_const (lexer, qc->dict, &qc->vars, &qc->n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ return false;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "LISTWISE")
+ || lex_match_id (lexer, "DEFAULT"))
+ qc->missing_type = MISS_LISTWISE;
+ else if (lex_match_id (lexer, "PAIRWISE"))
+ qc->missing_type = MISS_PAIRWISE;
+ else if (lex_match_id (lexer, "INCLUDE"))
+ qc->exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ qc->exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "LISTWISE", "DEFAULT",
+ "PAIRWISE", "INCLUDE", "EXCLUDE");
+ return false;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "PRINT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "CLUSTER"))
+ qc->print_cluster_membership = true;
+ else if (lex_match_id (lexer, "INITIAL"))
+ qc->print_initial_clusters = true;
+ else
+ {
+ lex_error_expecting (lexer, "CLUSTER", "INITIAL");
+ return false;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "SAVE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "CLUSTER"))
+ {
+ qc->save_membership = true;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_id (lexer))
+ return false;
+
+ free (qc->var_membership);
+ qc->var_membership = xstrdup (lex_tokcstr (lexer));
+ if (NULL != dict_lookup_var (qc->dict, qc->var_membership))
+ {
+ lex_error (lexer,
+ _("A variable called `%s' already exists."),
+ qc->var_membership);
+ free (qc->var_membership);
+ qc->var_membership = NULL;
+ return false;
+ }
+
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ }
+ else if (lex_match_id (lexer, "DISTANCE"))
+ {
+ qc->save_distance = true;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_id (lexer))
+ return false;
+
+ free (qc->var_distance);
+ qc->var_distance = xstrdup (lex_tokcstr (lexer));
+ if (NULL != dict_lookup_var (qc->dict, qc->var_distance))
+ {
+ lex_error (lexer,
+ _("A variable called `%s' already exists."),
+ qc->var_distance);
+ free (qc->var_distance);
+ qc->var_distance = NULL;
+ return false;
+ }
+
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "CLUSTER", "DISTANCE");
+ return false;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "CLUSTERS"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "CLUSTERS", 1, INT_MAX))
+ return false;
+ qc->ngroups = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ else if (lex_match_id (lexer, "CONVERGE"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_num_range_open (lexer, "CONVERGE",
+ 0, DBL_MAX))
+ return false;
+ qc->epsilon = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ else if (lex_match_id (lexer, "MXITER"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "MXITER", 1, INT_MAX))
+ return false;
+ qc->maxiter = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ else if (lex_match_id (lexer, "NOINITIAL"))
+ qc->initial = false;
+ else if (lex_match_id (lexer, "NOUPDATE"))
+ qc->update = false;
+ else
+ {
+ lex_error_expecting (lexer, "CLUSTERS", "CONVERGE", "MXITER",
+ "NOINITIAL", "NOUPDATE");
+ return false;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MISSING", "PRINT", "SAVE", "CRITERIA");
+ return false;
+ }
+ }
+ return true;
+}
+
+int
+cmd_quick_cluster (struct lexer *lexer, struct dataset *ds)
+{
+ struct qc qc = {
+ .dataset = ds,
+ .dict = dataset_dict (ds),
+ .ngroups = 2,
+ .maxiter = 10,
+ .epsilon = DBL_EPSILON,
+ .missing_type = MISS_LISTWISE,
+ .exclude = MV_ANY,
+ .initial = true,
+ .update = true,
+ };
+
+ if (!quick_cluster_parse (lexer, &qc))
+ goto error;
+
+ qc.wv = dict_get_weight (qc.dict);
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), qc.dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ if (qc.missing_type == MISS_LISTWISE)
+ group = casereader_create_filter_missing (group, qc.vars, qc.n_vars,
+ qc.exclude, NULL, NULL);
+
+ struct Kmeans *kmeans = kmeans_create (&qc);
+ kmeans_cluster (kmeans, group, &qc);
+ quick_cluster_show_results (kmeans, group, &qc);
+ kmeans_destroy (kmeans);
+ casereader_destroy (group);
+ }
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ /* If requested, set a transformation to append the cluster and
+ distance values to the current dataset. */
+ if (qc.save_trans_data)
+ {
+ struct save_trans_data *std = qc.save_trans_data;
+
+ std->appending_reader = casewriter_make_reader (std->writer);
+
+ if (qc.save_membership)
+ {
+ /* Invent a variable name if necessary. */
+ int idx = 0;
+ struct string name;
+ ds_init_empty (&name);
+ while (qc.var_membership == NULL)
+ {
+ ds_clear (&name);
+ ds_put_format (&name, "QCL_%d", idx++);
+
+ if (!dict_lookup_var (qc.dict, ds_cstr (&name)))
+ {
+ qc.var_membership = strdup (ds_cstr (&name));
+ break;
+ }
+ }
+ ds_destroy (&name);
+
+ std->membership = dict_create_var_assert (qc.dict, qc.var_membership, 0);
+ }
+
+ if (qc.save_distance)
+ {
+ /* Invent a variable name if necessary. */
+ int idx = 0;
+ struct string name;
+ ds_init_empty (&name);
+ while (qc.var_distance == NULL)
+ {
+ ds_clear (&name);
+ ds_put_format (&name, "QCL_%d", idx++);
+
+ if (!dict_lookup_var (qc.dict, ds_cstr (&name)))
+ {
+ qc.var_distance = strdup (ds_cstr (&name));
+ break;
+ }
+ }
+ ds_destroy (&name);
+
+ std->distance = dict_create_var_assert (qc.dict, qc.var_distance, 0);
+ }
+
+ static const struct trns_class trns_class = {
+ .name = "QUICK CLUSTER",
+ .execute = save_trans_func,
+ .destroy = save_trans_destroy,
+ };
+ add_transformation (qc.dataset, &trns_class, std);
+ }
+
+ free (qc.var_distance);
+ free (qc.var_membership);
+ free (qc.vars);
+ return ok;
+
+ error:
+ free (qc.var_distance);
+ free (qc.var_membership);
+ free (qc.vars);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2005, 2006, 2007, 2009, 2010, 2011, 2012, 2013, 2014, 2016 Free Software Foundation, Inc
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <math.h>
+#include <gsl/gsl_cdf.h>
+
+#include "data/case.h"
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "data/subcase.h"
+#include "data/casewriter.h"
+#include "data/short-names.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "language/commands/sort-criteria.h"
+#include "math/sort.h"
+#include "libpspp/assertion.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/stringi-set.h"
+#include "libpspp/taint.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+struct rank;
+
+typedef double (*rank_function_t) (const struct rank*, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_proportion (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_normal (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_percent (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_rfraction (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_rank (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_n (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_savage (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+static double rank_ntiles (const struct rank *, double c, double cc, double cc_1,
+ int i, double w);
+
+
+enum rank_func
+ {
+ RANK,
+ NORMAL,
+ PERCENT,
+ RFRACTION,
+ PROPORTION,
+ N,
+ NTILES,
+ SAVAGE,
+ n_RANK_FUNCS
+ };
+
+static const struct fmt_spec dest_format[n_RANK_FUNCS] = {
+ [RANK] = { .type = FMT_F, .w = 9, .d = 3 },
+ [NORMAL] = { .type = FMT_F, .w = 6, .d = 4 },
+ [PERCENT] = { .type = FMT_F, .w = 6, .d = 2 },
+ [RFRACTION] = { .type = FMT_F, .w = 6, .d = 4 },
+ [PROPORTION] = { .type = FMT_F, .w = 6, .d = 4 },
+ [N] = { .type = FMT_F, .w = 6, .d = 0 },
+ [NTILES] = { .type = FMT_F, .w = 3, .d = 0 },
+ [SAVAGE] = { .type = FMT_F, .w = 8, .d = 4 }
+};
+
+static const char * const function_name[n_RANK_FUNCS] = {
+ "RANK",
+ "NORMAL",
+ "PERCENT",
+ "RFRACTION",
+ "PROPORTION",
+ "N",
+ "NTILES",
+ "SAVAGE"
+};
+
+static const rank_function_t rank_func[n_RANK_FUNCS] = {
+ rank_rank,
+ rank_normal,
+ rank_percent,
+ rank_rfraction,
+ rank_proportion,
+ rank_n,
+ rank_ntiles,
+ rank_savage
+};
+
+static enum measure rank_measures[n_RANK_FUNCS] = {
+ [RANK] = MEASURE_ORDINAL,
+ [NORMAL] = MEASURE_ORDINAL,
+ [PERCENT] = MEASURE_ORDINAL,
+ [RFRACTION] = MEASURE_ORDINAL,
+ [PROPORTION] = MEASURE_ORDINAL,
+ [N] = MEASURE_SCALE,
+ [NTILES] = MEASURE_ORDINAL,
+ [SAVAGE] = MEASURE_ORDINAL,
+};
+
+enum ties
+ {
+ TIES_LOW,
+ TIES_HIGH,
+ TIES_MEAN,
+ TIES_CONDENSE
+ };
+
+enum fraction
+ {
+ FRAC_BLOM,
+ FRAC_RANKIT,
+ FRAC_TUKEY,
+ FRAC_VW
+ };
+
+struct rank_spec
+{
+ enum rank_func rfunc;
+ const char **dest_names;
+ const char **dest_labels;
+};
+
+/* If NEW_NAME exists in DICT or NEW_NAMES, returns NULL without changing
+ anything. Otherwise, inserts NEW_NAME in NEW_NAMES and returns the copy of
+ NEW_NAME now in NEW_NAMES. In any case, frees NEW_NAME. */
+static const char *
+try_new_name (char *new_name,
+ const struct dictionary *dict, struct stringi_set *new_names)
+{
+ const char *retval = (!dict_lookup_var (dict, new_name)
+ && stringi_set_insert (new_names, new_name)
+ ? stringi_set_find_node (new_names, new_name)->string
+ : NULL);
+ free (new_name);
+ return retval;
+}
+
+/* Returns a variable name for storing ranks of a variable named SRC_NAME
+ according to the rank function F. The name chosen will not be one already in
+ DICT or NEW_NAMES.
+
+ If successful, adds the new name to NEW_NAMES and returns the name added.
+ If no name can be generated, returns NULL. */
+static const char *
+rank_choose_dest_name (struct dictionary *dict, struct stringi_set *new_names,
+ enum rank_func f, const char *src_name)
+{
+ /* Try the first character of the ranking function followed by the first 7
+ bytes of the srcinal variable name. */
+ char *src_name_7 = utf8_encoding_trunc (src_name, dict_get_encoding (dict),
+ 7);
+ const char *s = try_new_name (
+ xasprintf ("%c%s", function_name[f][0], src_name_7), dict, new_names);
+ free (src_name_7);
+ if (s)
+ return s;
+
+ /* Try "fun###". */
+ for (int i = 1; i <= 999; i++)
+ {
+ s = try_new_name (xasprintf ("%.3s%03d", function_name[f], i),
+ dict, new_names);
+ if (s)
+ return s;
+ }
+
+ /* Try "RNKfn##". */
+ for (int i = 1; i <= 99; i++)
+ {
+ s = try_new_name (xasprintf ("RNK%.2s%02d", function_name[f], i),
+ dict, new_names);
+ if (s)
+ return s;
+ }
+
+ msg (ME, _("Cannot generate variable name for ranking %s with %s. "
+ "All candidates in use."),
+ src_name, function_name[f]);
+ return NULL;
+}
+
+struct rank
+{
+ struct dictionary *dict;
+
+ struct subcase sc;
+
+ const struct variable **vars;
+ size_t n_vars;
+
+ const struct variable **group_vars;
+ size_t n_group_vars;
+
+
+ enum mv_class exclude;
+
+ struct rank_spec *rs;
+ size_t n_rs;
+
+ enum ties ties;
+
+ enum fraction fraction;
+ int k_ntiles;
+
+ bool print;
+
+ /* Pool on which cell functions may allocate data */
+ struct pool *pool;
+};
+
+
+static void
+destroy_rank (struct rank *rank)
+{
+ free (rank->vars);
+ free (rank->group_vars);
+ subcase_uninit (&rank->sc);
+ pool_destroy (rank->pool);
+}
+
+static bool
+parse_into (struct lexer *lexer, struct rank *cmd,
+ struct stringi_set *new_names)
+{
+ enum rank_func rfunc;
+ if (lex_match_id (lexer, "RANK"))
+ rfunc = RANK;
+ else if (lex_match_id (lexer, "NORMAL"))
+ rfunc = NORMAL;
+ else if (lex_match_id (lexer, "RFRACTION"))
+ rfunc = RFRACTION;
+ else if (lex_match_id (lexer, "N"))
+ rfunc = N;
+ else if (lex_match_id (lexer, "SAVAGE"))
+ rfunc = SAVAGE;
+ else if (lex_match_id (lexer, "PERCENT"))
+ rfunc = PERCENT;
+ else if (lex_match_id (lexer, "PROPORTION"))
+ rfunc = PROPORTION;
+ else if (lex_match_id (lexer, "NTILES"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, "NTILES", 1, INT_MAX))
+ return false;
+
+ cmd->k_ntiles = lex_integer (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+
+ rfunc = NTILES;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "RANK", "NORMAL", "RFRACTION", "N",
+ "SAVAGE", "PERCENT", "PROPORTION", "NTILES");
+ return false;
+ }
+
+ cmd->rs = pool_realloc (cmd->pool, cmd->rs, sizeof (*cmd->rs) * (cmd->n_rs + 1));
+ struct rank_spec *rs = &cmd->rs[cmd->n_rs++];
+ *rs = (struct rank_spec) {
+ .rfunc = rfunc,
+ .dest_names = pool_calloc (cmd->pool, cmd->n_vars,
+ sizeof *rs->dest_names),
+ };
+
+ if (lex_match_id (lexer, "INTO"))
+ {
+ int vars_start = lex_ofs (lexer);
+ size_t var_count = 0;
+ while (lex_token (lexer) == T_ID)
+ {
+ const char *name = lex_tokcstr (lexer);
+
+ if (var_count >= subcase_get_n_fields (&cmd->sc))
+ lex_ofs_error (lexer, vars_start, lex_ofs (lexer),
+ _("Too many variables in %s clause."), "INTO");
+ else if (dict_lookup_var (cmd->dict, name) != NULL)
+ lex_error (lexer, _("Variable %s already exists."), name);
+ else if (stringi_set_contains (new_names, name))
+ lex_error (lexer, _("Duplicate variable name %s."), name);
+ else
+ {
+ stringi_set_insert (new_names, name);
+ rs->dest_names[var_count++] = pool_strdup (cmd->pool, name);
+ lex_get (lexer);
+ continue;
+ }
+
+ /* Error path. */
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/* Hardly a rank function. */
+static double
+rank_n (const struct rank *cmd UNUSED, double c UNUSED, double cc UNUSED, double cc_1 UNUSED,
+ int i UNUSED, double w)
+{
+ return w;
+}
+
+static double
+rank_rank (const struct rank *cmd, double c, double cc, double cc_1,
+ int i, double w UNUSED)
+{
+ double rank;
+
+ if (c >= 1.0)
+ {
+ switch (cmd->ties)
+ {
+ case TIES_LOW:
+ rank = cc_1 + 1;
+ break;
+ case TIES_HIGH:
+ rank = cc;
+ break;
+ case TIES_MEAN:
+ rank = cc_1 + (c + 1.0)/ 2.0;
+ break;
+ case TIES_CONDENSE:
+ rank = i;
+ break;
+ default:
+ NOT_REACHED ();
+ }
+ }
+ else
+ {
+ switch (cmd->ties)
+ {
+ case TIES_LOW:
+ rank = cc_1;
+ break;
+ case TIES_HIGH:
+ rank = cc;
+ break;
+ case TIES_MEAN:
+ rank = cc_1 + c / 2.0;
+ break;
+ case TIES_CONDENSE:
+ rank = i;
+ break;
+ default:
+ NOT_REACHED ();
+ }
+ }
+
+ return rank;
+}
+
+
+static double
+rank_rfraction (const struct rank *cmd, double c, double cc, double cc_1,
+ int i, double w)
+{
+ return rank_rank (cmd, c, cc, cc_1, i, w) / w;
+}
+
+
+static double
+rank_percent (const struct rank *cmd, double c, double cc, double cc_1,
+ int i, double w)
+{
+ return rank_rank (cmd, c, cc, cc_1, i, w) * 100.0 / w;
+}
+
+
+static double
+rank_proportion (const struct rank *cmd, double c, double cc, double cc_1,
+ int i, double w)
+{
+ const double r = rank_rank (cmd, c, cc, cc_1, i, w);
+
+ double f;
+
+ switch (cmd->fraction)
+ {
+ case FRAC_BLOM:
+ f = (r - 3.0/8.0) / (w + 0.25);
+ break;
+ case FRAC_RANKIT:
+ f = (r - 0.5) / w;
+ break;
+ case FRAC_TUKEY:
+ f = (r - 1.0/3.0) / (w + 1.0/3.0);
+ break;
+ case FRAC_VW:
+ f = r / (w + 1.0);
+ break;
+ default:
+ NOT_REACHED ();
+ }
+
+
+ return (f > 0) ? f : SYSMIS;
+}
+
+static double
+rank_normal (const struct rank *cmd, double c, double cc, double cc_1,
+ int i, double w)
+{
+ double f = rank_proportion (cmd, c, cc, cc_1, i, w);
+
+ return gsl_cdf_ugaussian_Pinv (f);
+}
+
+static double
+rank_ntiles (const struct rank *cmd, double c, double cc, double cc_1,
+ int i, double w)
+{
+ double r = rank_rank (cmd, c, cc, cc_1, i, w);
+
+
+ return (floor ((r * cmd->k_ntiles) / (w + 1)) + 1);
+}
+
+/* Expected value of the order statistics from an exponential distribution */
+static double
+ee (int j, double w_star)
+{
+ double sum = 0.0;
+
+ for (int k = 1; k <= j; k++)
+ sum += 1.0 / (w_star + 1 - k);
+
+ return sum;
+}
+
+
+static double
+rank_savage (const struct rank *cmd UNUSED, double c, double cc, double cc_1,
+ int i UNUSED, double w)
+{
+ double int_part;
+ const int i_1 = floor (cc_1);
+ const int i_2 = floor (cc);
+
+ const double w_star = (modf (w, &int_part) == 0) ? w : floor (w) + 1;
+
+ const double g_1 = cc_1 - i_1;
+ const double g_2 = cc - i_2;
+
+ /* The second factor is infinite, when the first is zero.
+ Therefore, evaluate the second, only when the first is non-zero */
+ const double expr1 = (1 - g_1) ? (1 - g_1) * ee(i_1+1, w_star) : (1 - g_1);
+ const double expr2 = g_2 ? g_2 * ee (i_2+1, w_star) : g_2;
+
+ if (i_1 == i_2)
+ return ee (i_1 + 1, w_star) - 1;
+
+ if (i_1 + 1 == i_2)
+ return ((expr1 + expr2)/c) - 1;
+
+ if (i_1 + 2 <= i_2)
+ {
+ double sigma = 0.0;
+ for (int j = i_1 + 2; j <= i_2; ++j)
+ sigma += ee (j, w_star);
+ return ((expr1 + expr2 + sigma) / c) -1;
+ }
+
+ NOT_REACHED ();
+}
+
+static double
+sum_weights (const struct casereader *input, int weight_idx)
+{
+ if (weight_idx == -1)
+ return casereader_count_cases (input);
+
+ double w = 0.0;
+
+ struct casereader *pass = casereader_clone (input);
+ struct ccase *c;
+ for (; (c = casereader_read (pass)) != NULL; case_unref (c))
+ w += case_num_idx (c, weight_idx);
+ casereader_destroy (pass);
+
+ return w;
+}
+
+static void
+rank_sorted_file (struct casereader *input,
+ struct casewriter *output,
+ int weight_idx,
+ const struct rank *cmd)
+{
+ int tie_group = 1;
+ double cc = 0.0;
+
+ /* Get total group weight. */
+ double w = sum_weights (input, weight_idx);
+
+ /* Do ranking. */
+ struct subcase input_var = SUBCASE_EMPTY_INITIALIZER;
+ subcase_add (&input_var, 0, 0, SC_ASCEND);
+ struct casegrouper *tie_grouper = casegrouper_create_subcase (input, &input_var);
+ subcase_uninit (&input_var);
+
+ struct casereader *tied_cases;
+ for (; casegrouper_get_next_group (tie_grouper, &tied_cases);
+ casereader_destroy (tied_cases))
+ {
+ double tw = sum_weights (tied_cases, weight_idx);
+ double cc_1 = cc;
+ cc += tw;
+
+ taint_propagate (casereader_get_taint (tied_cases),
+ casewriter_get_taint (output));
+
+ /* Rank tied cases. */
+ struct ccase *c;
+ for (; (c = casereader_read (tied_cases)) != NULL; case_unref (c))
+ {
+ struct ccase *out_case = case_create (casewriter_get_proto (output));
+ *case_num_rw_idx (out_case, 0) = case_num_idx (c, 1);
+ for (size_t i = 0; i < cmd->n_rs; ++i)
+ {
+ rank_function_t func = rank_func[cmd->rs[i].rfunc];
+ double rank = func (cmd, tw, cc, cc_1, tie_group, w);
+ *case_num_rw_idx (out_case, i + 1) = rank;
+ }
+
+ casewriter_write (output, out_case);
+ }
+ tie_group++;
+ }
+ casegrouper_destroy (tie_grouper);
+}
+
+
+static bool
+rank_cmd (struct dataset *ds, const struct rank *cmd);
+
+static const char *
+fraction_name (const struct rank *cmd)
+{
+ switch (cmd->fraction)
+ {
+ case FRAC_BLOM: return "BLOM";
+ case FRAC_RANKIT: return "RANKIT";
+ case FRAC_TUKEY: return "TUKEY";
+ case FRAC_VW: return "VW";
+ default: NOT_REACHED ();
+ }
+}
+
+/* Returns a label for a variable derived from SRC_VAR with function F. */
+static const char *
+create_var_label (struct rank *cmd, const struct variable *src_var,
+ enum rank_func f)
+{
+ if (cmd->n_group_vars > 0)
+ {
+ struct string group_var_str = DS_EMPTY_INITIALIZER;
+ for (size_t g = 0; g < cmd->n_group_vars; ++g)
+ {
+ if (g > 0)
+ ds_put_cstr (&group_var_str, " ");
+ ds_put_cstr (&group_var_str, var_get_name (cmd->group_vars[g]));
+ }
+
+ const char *label = pool_asprintf (
+ cmd->pool, _("%s of %s by %s"), function_name[f],
+ var_get_name (src_var), ds_cstr (&group_var_str));
+ ds_destroy (&group_var_str);
+ return label;
+ }
+ else
+ return pool_asprintf (cmd->pool, _("%s of %s"),
+ function_name[f], var_get_name (src_var));
+}
+
+int
+cmd_rank (struct lexer *lexer, struct dataset *ds)
+{
+ struct stringi_set new_names = STRINGI_SET_INITIALIZER (new_names);
+ struct rank rank = {
+ .sc = SUBCASE_EMPTY_INITIALIZER,
+ .exclude = MV_ANY,
+ .dict = dataset_dict (ds),
+ .ties = TIES_MEAN,
+ .fraction = FRAC_BLOM,
+ .print = true,
+ .pool = pool_create (),
+ };
+
+ if (lex_match_id (lexer, "VARIABLES") && !lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ if (!parse_sort_criteria (lexer, rank.dict, &rank.sc, &rank.vars, NULL))
+ goto error;
+ rank.n_vars = rank.sc.n_fields;
+
+ if (lex_match (lexer, T_BY)
+ && !parse_variables_const (lexer, rank.dict,
+ &rank.group_vars, &rank.n_group_vars,
+ PV_NO_DUPLICATE | PV_NO_SCRATCH))
+ goto error;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+ if (lex_match_id (lexer, "TIES"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+ if (lex_match_id (lexer, "MEAN"))
+ rank.ties = TIES_MEAN;
+ else if (lex_match_id (lexer, "LOW"))
+ rank.ties = TIES_LOW;
+ else if (lex_match_id (lexer, "HIGH"))
+ rank.ties = TIES_HIGH;
+ else if (lex_match_id (lexer, "CONDENSE"))
+ rank.ties = TIES_CONDENSE;
+ else
+ {
+ lex_error_expecting (lexer, "MEAN", "LOW", "HIGH", "CONDENSE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "FRACTION"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+ if (lex_match_id (lexer, "BLOM"))
+ rank.fraction = FRAC_BLOM;
+ else if (lex_match_id (lexer, "TUKEY"))
+ rank.fraction = FRAC_TUKEY;
+ else if (lex_match_id (lexer, "VW"))
+ rank.fraction = FRAC_VW;
+ else if (lex_match_id (lexer, "RANKIT"))
+ rank.fraction = FRAC_RANKIT;
+ else
+ {
+ lex_error_expecting (lexer, "BLOM", "TUKEY", "VW", "RANKIT");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "PRINT"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+ if (lex_match_id (lexer, "YES"))
+ rank.print = true;
+ else if (lex_match_id (lexer, "NO"))
+ rank.print = false;
+ else
+ {
+ lex_error_expecting (lexer, "YES", "NO");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+ if (lex_match_id (lexer, "INCLUDE"))
+ rank.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ rank.exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ else if (!parse_into (lexer, &rank, &new_names))
+ goto error;
+ }
+
+
+ /* If no rank specs are given, then apply a default */
+ if (rank.n_rs == 0)
+ {
+ struct rank_spec *rs = pool_malloc (rank.pool, sizeof *rs);
+ *rs = (struct rank_spec) {
+ .rfunc = RANK,
+ .dest_names = pool_calloc (rank.pool, rank.n_vars,
+ sizeof *rs->dest_names),
+ };
+
+ rank.rs = rs;
+ rank.n_rs = 1;
+ }
+
+ /* Choose variable names for all rank destinations which haven't already been
+ created with INTO. */
+ for (struct rank_spec *rs = rank.rs; rs < &rank.rs[rank.n_rs]; rs++)
+ {
+ rs->dest_labels = pool_calloc (rank.pool, rank.n_vars,
+ sizeof *rs->dest_labels);
+ for (int v = 0; v < rank.n_vars; v ++)
+ {
+ const char **dst_name = &rs->dest_names[v];
+ if (*dst_name == NULL)
+ {
+ *dst_name = rank_choose_dest_name (rank.dict, &new_names,
+ rs->rfunc,
+ var_get_name (rank.vars[v]));
+ if (*dst_name == NULL)
+ goto error;
+ }
+
+ rs->dest_labels[v] = create_var_label (&rank, rank.vars[v],
+ rs->rfunc);
+ }
+ }
+
+ if (rank.print)
+ {
+ struct pivot_table *table = pivot_table_create (
+ N_("Variables Created by RANK"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("New Variable"),
+ N_("New Variable"), N_("Function"),
+ N_("Fraction"), N_("Grouping Variables"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Existing Variable"),
+ N_("Existing Variable"));
+ variables->root->show_label = true;
+
+ for (size_t i = 0; i < rank.n_rs; ++i)
+ {
+ for (size_t v = 0; v < rank.n_vars; v ++)
+ {
+ int row_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (rank.vars[v]));
+
+ struct string group_vars = DS_EMPTY_INITIALIZER;
+ for (int g = 0; g < rank.n_group_vars; ++g)
+ {
+ if (g)
+ ds_put_byte (&group_vars, ' ');
+ ds_put_cstr (&group_vars, var_get_name (rank.group_vars[g]));
+ }
+
+ enum rank_func rfunc = rank.rs[i].rfunc;
+ bool has_fraction = rfunc == NORMAL || rfunc == PROPORTION;
+ const char *entries[] =
+ {
+ rank.rs[i].dest_names[v],
+ function_name[rank.rs[i].rfunc],
+ has_fraction ? fraction_name (&rank) : NULL,
+ rank.n_group_vars ? ds_cstr (&group_vars) : NULL,
+ };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ {
+ const char *entry = entries[j];
+ if (entry)
+ pivot_table_put2 (table, j, row_idx,
+ pivot_value_new_user_text (entry, -1));
+ }
+ ds_destroy (&group_vars);
+ }
+ }
+
+ pivot_table_submit (table);
+ }
+
+ /* Do the ranking */
+ rank_cmd (ds, &rank);
+
+ destroy_rank (&rank);
+ stringi_set_destroy (&new_names);
+ return CMD_SUCCESS;
+
+ error:
+ destroy_rank (&rank);
+ stringi_set_destroy (&new_names);
+ return CMD_FAILURE;
+}
+
+/* RANK transformation. */
+struct rank_trns
+ {
+ int order_case_idx;
+
+ struct rank_trns_input_var *input_vars;
+ size_t n_input_vars;
+
+ size_t n_funcs;
+ };
+
+struct rank_trns_input_var
+ {
+ struct casereader *input;
+ struct ccase *current;
+
+ struct variable **output_vars;
+ };
+
+static void
+advance_ranking (struct rank_trns_input_var *iv)
+{
+ case_unref (iv->current);
+ iv->current = casereader_read (iv->input);
+}
+
+static enum trns_result
+rank_trns_proc (void *trns_, struct ccase **c, casenumber case_idx UNUSED)
+{
+ struct rank_trns *trns = trns_;
+ double order = case_num_idx (*c, trns->order_case_idx);
+ struct rank_trns_input_var *iv;
+
+ *c = case_unshare (*c);
+ for (iv = trns->input_vars; iv < &trns->input_vars[trns->n_input_vars]; iv++)
+ while (iv->current != NULL)
+ {
+ double iv_order = case_num_idx (iv->current, 0);
+ if (iv_order == order)
+ {
+ size_t i;
+
+ for (i = 0; i < trns->n_funcs; i++)
+ *case_num_rw (*c, iv->output_vars[i])
+ = case_num_idx (iv->current, i + 1);
+ advance_ranking (iv);
+ break;
+ }
+ else if (iv_order > order)
+ break;
+ else
+ advance_ranking (iv);
+ }
+ return TRNS_CONTINUE;
+}
+
+static bool
+rank_trns_free (void *trns_)
+{
+ struct rank_trns *trns = trns_;
+ struct rank_trns_input_var *iv;
+
+ for (iv = trns->input_vars; iv < &trns->input_vars[trns->n_input_vars]; iv++)
+ {
+ casereader_destroy (iv->input);
+ case_unref (iv->current);
+
+ free (iv->output_vars);
+ }
+ free (trns->input_vars);
+ free (trns);
+
+ return true;
+}
+
+static const struct trns_class rank_trns_class = {
+ .name = "RANK",
+ .execute = rank_trns_proc,
+ .destroy = rank_trns_free,
+};
+
+static bool
+rank_cmd (struct dataset *ds, const struct rank *cmd)
+{
+ struct dictionary *d = dataset_dict (ds);
+ struct variable *weight_var = dict_get_weight (d);
+ bool ok = true;
+
+ struct variable *order_var = add_permanent_ordering_transformation (ds);
+
+ /* Create output files. */
+ struct caseproto *output_proto = caseproto_create ();
+ for (size_t i = 0; i < cmd->n_rs + 1; i++)
+ output_proto = caseproto_add_width (output_proto, 0);
+
+ struct subcase by_order;
+ subcase_init (&by_order, 0, 0, SC_ASCEND);
+
+ struct casewriter **outputs = xnmalloc (cmd->n_vars, sizeof *outputs);
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ outputs[i] = sort_create_writer (&by_order, output_proto);
+
+ subcase_uninit (&by_order);
+ caseproto_unref (output_proto);
+
+ /* Open the active file and make one pass per input variable. */
+ struct casereader *input = proc_open (ds);
+ input = casereader_create_filter_weight (input, d, NULL, NULL);
+ for (size_t i = 0; i < cmd->n_vars; ++i)
+ {
+ const struct variable *input_var = cmd->vars[i];
+
+ /* Discard cases that have missing values of input variable. */
+ struct casereader *input_pass
+ = i == cmd->n_vars - 1 ? input : casereader_clone (input);
+ input_pass = casereader_create_filter_missing (input_pass, &input_var, 1,
+ cmd->exclude, NULL, NULL);
+
+ /* Keep only the columns we really need, to save time and space when we
+ sort them just below.
+
+ After this projection, the input_pass case indexes look like:
+
+ - 0: input_var.
+ - 1: order_var.
+ - 2 and up: cmd->n_group_vars group variables
+ - 2 + cmd->n_group_vars and up: split variables
+ - 2 + cmd->n_group_vars + n_split_vars: weight var
+ */
+ struct subcase projection = SUBCASE_EMPTY_INITIALIZER;
+ subcase_add_var_always (&projection, input_var, SC_ASCEND);
+ subcase_add_var_always (&projection, order_var, SC_ASCEND);
+ subcase_add_vars_always (&projection,
+ cmd->group_vars, cmd->n_group_vars);
+ subcase_add_vars_always (&projection, dict_get_split_vars (d),
+ dict_get_n_splits (d));
+ int weight_idx;
+ if (weight_var != NULL)
+ {
+ subcase_add_var_always (&projection, weight_var, SC_ASCEND);
+ weight_idx = 2 + cmd->n_group_vars + dict_get_n_splits (d);
+ }
+ else
+ weight_idx = -1;
+ input_pass = casereader_project (input_pass, &projection);
+ subcase_uninit (&projection);
+
+ /* Prepare 'group_vars' as the set of grouping variables. */
+ struct subcase group_vars = SUBCASE_EMPTY_INITIALIZER;
+ for (size_t j = 0; j < cmd->n_group_vars; j++)
+ subcase_add_always (&group_vars,
+ j + 2, var_get_width (cmd->group_vars[j]),
+ SC_ASCEND);
+
+ /* Prepare 'rank_ordering' for sorting with the group variables as
+ primary key and the input variable as secondary key. */
+ struct subcase rank_ordering;
+ subcase_clone (&rank_ordering, &group_vars);
+ subcase_add (&rank_ordering, 0, 0, subcase_get_direction (&cmd->sc, i));
+
+ /* Group by split variables */
+ struct subcase split_vars = SUBCASE_EMPTY_INITIALIZER;
+ for (size_t j = 0; j < dict_get_n_splits (d); j++)
+ subcase_add_always (&split_vars, 2 + j + cmd->n_group_vars,
+ var_get_width (dict_get_split_vars (d)[j]),
+ SC_ASCEND);
+
+ struct casegrouper *split_grouper
+ = casegrouper_create_subcase (input_pass, &split_vars);
+ subcase_uninit (&split_vars);
+
+ struct casereader *split_group;
+ while (casegrouper_get_next_group (split_grouper, &split_group))
+ {
+ struct casereader *ordered;
+ struct casegrouper *by_grouper;
+ struct casereader *by_group;
+
+ ordered = sort_execute (split_group, &rank_ordering);
+ by_grouper = casegrouper_create_subcase (ordered, &group_vars);
+ while (casegrouper_get_next_group (by_grouper, &by_group))
+ rank_sorted_file (by_group, outputs[i], weight_idx, cmd);
+ ok = casegrouper_destroy (by_grouper) && ok;
+ }
+ subcase_uninit (&group_vars);
+ subcase_uninit (&rank_ordering);
+
+ ok = casegrouper_destroy (split_grouper) && ok;
+ }
+ ok = proc_commit (ds) && ok;
+
+ /* Re-fetch the dictionary and order variable, because if TEMPORARY was in
+ effect then there's a new dictionary. */
+ d = dataset_dict (ds);
+ order_var = dict_lookup_var_assert (d, "$ORDER");
+
+ /* Merge the original data set with the ranks (which we already sorted on
+ $ORDER). */
+ struct rank_trns *trns = xmalloc (sizeof *trns);
+ trns->order_case_idx = var_get_case_index (order_var);
+ trns->input_vars = xnmalloc (cmd->n_vars, sizeof *trns->input_vars);
+ trns->n_input_vars = cmd->n_vars;
+ trns->n_funcs = cmd->n_rs;
+ for (size_t i = 0; i < trns->n_input_vars; i++)
+ {
+ struct rank_trns_input_var *iv = &trns->input_vars[i];
+
+ iv->input = casewriter_make_reader (outputs[i]);
+ iv->current = casereader_read (iv->input);
+ iv->output_vars = xnmalloc (trns->n_funcs, sizeof *iv->output_vars);
+ for (size_t j = 0; j < trns->n_funcs; j++)
+ {
+ struct rank_spec *rs = &cmd->rs[j];
+ struct variable *var;
+
+ var = dict_create_var_assert (d, rs->dest_names[i], 0);
+ var_set_both_formats (var, &dest_format[rs->rfunc]);
+ var_set_label (var, rs->dest_labels[i]);
+ var_set_measure (var, rank_measures[rs->rfunc]);
+
+ iv->output_vars[j] = var;
+ }
+ }
+ free (outputs);
+
+ add_transformation (ds, &rank_trns_class, trns);
+
+ /* Delete our sort key, which we don't need anymore. */
+ dict_delete_var (d, order_var);
+
+ return ok;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <math.h>
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/data-in.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/compiler.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+\f
+/* Definitions. */
+
+/* Type of source value for RECODE. */
+enum map_in_type
+ {
+ MAP_SINGLE, /* Specific value. */
+ MAP_RANGE, /* Range of values. */
+ MAP_SYSMIS, /* System missing value. */
+ MAP_MISSING, /* Any missing value. */
+ MAP_ELSE, /* Any value. */
+ MAP_CONVERT /* "123" => 123. */
+ };
+
+/* Describes input values to be mapped. */
+struct map_in
+ {
+ enum map_in_type type; /* One of MAP_*. */
+ union value x, y; /* Source values. */
+ };
+
+/* Describes the value used as output from a mapping. */
+struct map_out
+ {
+ bool copy_input; /* If true, copy input to output. */
+ union value value; /* If copy_input false, recoded value. */
+ int width; /* If copy_input false, output value width. */
+ int ofs; /* Lexical location. */
+ };
+
+/* Describes how to recode a single value or range of values into a
+ single value. */
+struct mapping
+ {
+ struct map_in in; /* Input values. */
+ struct map_out out; /* Output value. */
+ };
+
+/* RECODE transformation. */
+struct recode_trns
+ {
+ struct pool *pool;
+
+ /* Variable types, for convenience. */
+ enum val_type src_type; /* src_vars[*] type. */
+ enum val_type dst_type; /* dst_vars[*] type. */
+
+ /* Variables. */
+ const struct variable **src_vars; /* Source variables. */
+ const struct variable **dst_vars; /* Destination variables. */
+ const struct dictionary *dst_dict; /* Dictionary of dst_vars */
+ char **dst_names; /* Name of dest variables, if they're new. */
+ size_t n_vars; /* Number of variables. */
+
+ /* Mappings. */
+ struct mapping *mappings; /* Value mappings. */
+ size_t n_maps; /* Number of mappings. */
+ int max_src_width; /* Maximum width of src_vars[*]. */
+ int max_dst_width; /* Maximum width of any map_out in mappings. */
+ };
+
+static bool parse_src_vars (struct lexer *, struct recode_trns *, const struct dictionary *dict);
+static bool parse_mappings (struct lexer *, struct recode_trns *,
+ const char *dict_encoding);
+static bool parse_dst_vars (struct lexer *, struct recode_trns *,
+ const struct dictionary *,
+ int src_start, int src_end,
+ int mappings_start, int mappings_end);
+
+static void add_mapping (struct recode_trns *,
+ size_t *map_allocated, const struct map_in *);
+
+static bool parse_map_in (struct lexer *lexer, struct map_in *, struct pool *,
+ enum val_type src_type, size_t max_src_width,
+ const char *dict_encoding);
+static void set_map_in_str (struct map_in *, struct pool *,
+ struct substring, size_t width,
+ const char *dict_encoding);
+
+static bool parse_map_out (struct lexer *lexer, struct pool *, struct map_out *);
+static void set_map_out_str (struct map_out *, struct pool *,
+ struct substring);
+
+static bool enlarge_dst_widths (struct lexer *, struct recode_trns *,
+ int dst_start, int dst_end);
+static void create_dst_vars (struct recode_trns *, struct dictionary *);
+
+static bool recode_trns_free (void *trns_);
+
+static const struct trns_class recode_trns_class;
+\f
+/* Parser. */
+
+static bool
+parse_one_recoding (struct lexer *lexer, struct dataset *ds,
+ struct recode_trns *trns)
+{
+ struct dictionary *dict = dataset_dict (ds);
+
+ /* Parse source variable names,
+ then input to output mappings,
+ then destination variable names. */
+ int src_start = lex_ofs (lexer);
+ if (!parse_src_vars (lexer, trns, dict))
+ return false;
+ int src_end = lex_ofs (lexer) - 1;
+
+ int mappings_start = lex_ofs (lexer);
+ if (!parse_mappings (lexer, trns, dict_get_encoding (dict)))
+ return false;
+ int mappings_end = lex_ofs (lexer) - 1;
+
+ int dst_start = lex_ofs (lexer);
+ if (!parse_dst_vars (lexer, trns, dict,
+ src_start, src_end, mappings_start, mappings_end))
+ return false;
+ int dst_end = lex_ofs (lexer) - 1;
+ if (dst_end < dst_start)
+ {
+ /* There was no target variable syntax, so the target variables are the
+ same as the source variables. */
+ dst_start = src_start;
+ dst_end = src_end;
+ }
+
+ /* Ensure that all the output strings are at least as wide
+ as the widest destination variable. */
+ if (trns->dst_type == VAL_STRING
+ && !enlarge_dst_widths (lexer, trns, dst_start, dst_end))
+ return false;
+
+ /* Create destination variables, if needed.
+ This must be the final step; otherwise we'd have to
+ delete destination variables on failure. */
+ trns->dst_dict = dict;
+ if (trns->src_vars != trns->dst_vars)
+ create_dst_vars (trns, dict);
+
+ /* Done. */
+ add_transformation (ds, &recode_trns_class, trns);
+ return true;
+}
+
+/* Parses the RECODE transformation. */
+int
+cmd_recode (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ struct pool *pool = pool_create ();
+ struct recode_trns *trns = pool_alloc (pool, sizeof *trns);
+ *trns = (struct recode_trns) { .pool = pool };
+
+ if (!parse_one_recoding (lexer, ds, trns))
+ {
+ recode_trns_free (trns);
+ return CMD_FAILURE;
+ }
+ }
+ while (lex_match (lexer, T_SLASH));
+
+ return CMD_SUCCESS;
+}
+
+/* Parses a set of variables to recode into TRNS->src_vars and
+ TRNS->n_vars. Sets TRNS->src_type. Returns true if
+ successful, false on parse error. */
+static bool
+parse_src_vars (struct lexer *lexer,
+ struct recode_trns *trns, const struct dictionary *dict)
+{
+ if (!parse_variables_const (lexer, dict, &trns->src_vars, &trns->n_vars,
+ PV_SAME_TYPE))
+ return false;
+ pool_register (trns->pool, free, trns->src_vars);
+ trns->src_type = var_get_type (trns->src_vars[0]);
+ return true;
+}
+
+/* Parses a set of mappings, which take the form (input=output),
+ into TRNS->mappings and TRNS->n_maps. Sets TRNS->dst_type.
+ Returns true if successful, false on parse error. */
+static bool
+parse_mappings (struct lexer *lexer, struct recode_trns *trns,
+ const char *dict_encoding)
+{
+ /* Find length of longest source variable. */
+ trns->max_src_width = var_get_width (trns->src_vars[0]);
+ for (size_t i = 1; i < trns->n_vars; i++)
+ {
+ size_t var_width = var_get_width (trns->src_vars[i]);
+ if (var_width > trns->max_src_width)
+ trns->max_src_width = var_width;
+ }
+
+ /* Parse the mappings in parentheses. */
+ size_t map_allocated = 0;
+ bool have_dst_type = false;
+ if (!lex_force_match (lexer, T_LPAREN))
+ return false;
+ do
+ {
+ enum val_type dst_type;
+
+ if (!lex_match_id (lexer, "CONVERT"))
+ {
+ size_t first_map_idx = trns->n_maps;
+
+ /* Parse source specifications. */
+ do
+ {
+ struct map_in in;
+
+ if (!parse_map_in (lexer, &in, trns->pool,
+ trns->src_type, trns->max_src_width,
+ dict_encoding))
+ return false;
+ add_mapping (trns, &map_allocated, &in);
+ lex_match (lexer, T_COMMA);
+ }
+ while (!lex_match (lexer, T_EQUALS));
+
+ struct map_out out;
+ if (!parse_map_out (lexer, trns->pool, &out))
+ return false;
+
+ dst_type = (out.copy_input
+ ? trns->src_type
+ : val_type_from_width (out.width));
+ for (size_t i = first_map_idx; i < trns->n_maps; i++)
+ trns->mappings[i].out = out;
+ }
+ else
+ {
+ /* Parse CONVERT as a special case. */
+ struct map_in in = { .type = MAP_CONVERT };
+ add_mapping (trns, &map_allocated, &in);
+
+ int ofs = lex_ofs (lexer) - 1;
+ trns->mappings[trns->n_maps - 1].out = (struct map_out) {
+ .ofs = ofs,
+ };
+
+ dst_type = VAL_NUMERIC;
+ if (trns->src_type != VAL_STRING)
+ {
+ lex_ofs_error (lexer, ofs, ofs,
+ _("CONVERT requires string input values."));
+ return false;
+ }
+ }
+ if (have_dst_type && dst_type != trns->dst_type)
+ {
+ msg (SE, _("Output values must be all numeric or all string."));
+
+ assert (trns->n_maps > 1);
+ const struct map_out *numeric = &trns->mappings[trns->n_maps - 2].out;
+ const struct map_out *string = &trns->mappings[trns->n_maps - 1].out;
+
+ if (trns->dst_type == VAL_STRING)
+ {
+ const struct map_out *tmp = numeric;
+ numeric = string;
+ string = tmp;
+ }
+
+ lex_ofs_msg (lexer, SN, numeric->ofs, numeric->ofs,
+ _("This output value is numeric."));
+ lex_ofs_msg (lexer, SN, string->ofs, string->ofs,
+ _("This output value is string."));
+ return false;
+ }
+ trns->dst_type = dst_type;
+ have_dst_type = true;
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ return false;
+ }
+ while (lex_match (lexer, T_LPAREN));
+
+ return true;
+}
+
+/* Parses a mapping input value into IN, allocating memory from
+ POOL. The source value type must be provided as SRC_TYPE and,
+ if string, the maximum width of a string source variable must
+ be provided in MAX_SRC_WIDTH. Returns true if successful,
+ false on parse error. */
+static bool
+parse_map_in (struct lexer *lexer, struct map_in *in, struct pool *pool,
+ enum val_type src_type, size_t max_src_width,
+ const char *dict_encoding)
+{
+
+ if (lex_match_id (lexer, "ELSE"))
+ *in = (struct map_in) { .type = MAP_ELSE };
+ else if (src_type == VAL_NUMERIC)
+ {
+ if (lex_match_id (lexer, "MISSING"))
+ *in = (struct map_in) { .type = MAP_MISSING };
+ else if (lex_match_id (lexer, "SYSMIS"))
+ *in = (struct map_in) { .type = MAP_SYSMIS };
+ else
+ {
+ double x, y;
+ if (!parse_num_range (lexer, &x, &y, NULL))
+ return false;
+ *in = (struct map_in) {
+ .type = x == y ? MAP_SINGLE : MAP_RANGE,
+ .x = { .f = x },
+ .y = { .f = y },
+ };
+ }
+ }
+ else
+ {
+ if (lex_match_id (lexer, "MISSING"))
+ *in = (struct map_in) { .type = MAP_MISSING };
+ else if (!lex_force_string (lexer))
+ return false;
+ else
+ {
+ set_map_in_str (in, pool, lex_tokss (lexer), max_src_width,
+ dict_encoding);
+ lex_get (lexer);
+ if (lex_match_id (lexer, "THRU"))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s is not allowed with string variables."),
+ "THRU");
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/* Adds IN to the list of mappings in TRNS.
+ MAP_ALLOCATED is the current number of allocated mappings,
+ which is updated as needed. */
+static void
+add_mapping (struct recode_trns *trns,
+ size_t *map_allocated, const struct map_in *in)
+{
+ struct mapping *m;
+ if (trns->n_maps >= *map_allocated)
+ trns->mappings = pool_2nrealloc (trns->pool, trns->mappings,
+ map_allocated,
+ sizeof *trns->mappings);
+ m = &trns->mappings[trns->n_maps++];
+ m->in = *in;
+}
+
+/* Sets IN as a string mapping, with STRING as the string,
+ allocated from POOL. The string is padded with spaces on the
+ right to WIDTH characters long. */
+static void
+set_map_in_str (struct map_in *in, struct pool *pool,
+ struct substring string, size_t width,
+ const char *dict_encoding)
+{
+ *in = (struct map_in) { .type = MAP_SINGLE };
+
+ char *s = recode_string (dict_encoding, "UTF-8",
+ ss_data (string), ss_length (string));
+ value_init_pool (pool, &in->x, width);
+ value_copy_buf_rpad (&in->x, width,
+ CHAR_CAST (uint8_t *, s), strlen (s), ' ');
+ free (s);
+}
+
+/* Parses a mapping output value into OUT, allocating memory from
+ POOL. Returns true if successful, false on parse error. */
+static bool
+parse_map_out (struct lexer *lexer, struct pool *pool, struct map_out *out)
+{
+ if (lex_is_number (lexer))
+ {
+ *out = (struct map_out) { .value = { .f = lex_number (lexer) } };
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "SYSMIS"))
+ *out = (struct map_out) { .value = { .f = SYSMIS } };
+ else if (lex_is_string (lexer))
+ {
+ set_map_out_str (out, pool, lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "COPY"))
+ *out = (struct map_out) { .copy_input = true };
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting output value."));
+ return false;
+ }
+ out->ofs = lex_ofs (lexer) - 1;
+ return true;
+}
+
+/* Sets OUT as a string mapping output with the given VALUE. */
+static void
+set_map_out_str (struct map_out *out, struct pool *pool,
+ const struct substring value)
+{
+ const char *string = ss_data (value);
+ size_t length = ss_length (value);
+
+ if (length == 0)
+ {
+ /* A length of 0 will yield a numeric value, which is not
+ what we want. */
+ string = " ";
+ length = 1;
+ }
+
+ *out = (struct map_out) { .width = length };
+ value_init_pool (pool, &out->value, length);
+ memcpy (out->value.s, string, length);
+}
+
+/* Parses a set of target variables into TRNS->dst_vars and
+ TRNS->dst_names. */
+static bool
+parse_dst_vars (struct lexer *lexer, struct recode_trns *trns,
+ const struct dictionary *dict, int src_start, int src_end,
+ int mappings_start, int mappings_end)
+{
+ int dst_start, dst_end;
+ if (lex_match_id (lexer, "INTO"))
+ {
+ dst_start = lex_ofs (lexer);
+ size_t n_names;
+ if (!parse_mixed_vars_pool (lexer, dict, trns->pool,
+ &trns->dst_names, &n_names,
+ PV_NONE))
+ return false;
+ dst_end = lex_ofs (lexer) - 1;
+
+ if (n_names != trns->n_vars)
+ {
+ msg (SE, _("Source and target variable counts must match."));
+ lex_ofs_msg (lexer, SN, src_start, src_end,
+ ngettext ("There is %zu source variable.",
+ "There are %zu source variables.",
+ trns->n_vars),
+ trns->n_vars);
+ lex_ofs_msg (lexer, SN, dst_start, dst_end,
+ ngettext ("There is %zu target variable.",
+ "There are %zu target variables.",
+ n_names),
+ n_names);
+ return false;
+ }
+
+ trns->dst_vars = pool_nalloc (trns->pool,
+ trns->n_vars, sizeof *trns->dst_vars);
+ for (size_t i = 0; i < trns->n_vars; i++)
+ {
+ const struct variable *v;
+ v = trns->dst_vars[i] = dict_lookup_var (dict, trns->dst_names[i]);
+ if (v == NULL && trns->dst_type == VAL_STRING)
+ {
+ msg (SE, _("All string variables specified on INTO must already "
+ "exist. (Use the STRING command to create a string "
+ "variable.)"));
+ lex_ofs_msg (lexer, SN, dst_start, dst_end,
+ _("There is no variable named %s."),
+ trns->dst_names[i]);
+ return false;
+ }
+ }
+ }
+ else
+ {
+ dst_start = src_start;
+ dst_end = src_end;
+
+ trns->dst_vars = trns->src_vars;
+ if (trns->src_type != trns->dst_type)
+ {
+ if (trns->src_type == VAL_NUMERIC)
+ lex_ofs_error (lexer, mappings_start, mappings_end,
+ _("INTO is required with numeric input values "
+ "and string output values."));
+ else
+ lex_ofs_error (lexer, mappings_start, mappings_end,
+ _("INTO is required with string input values "
+ "and numeric output values."));
+ return false;
+ }
+ }
+
+ for (size_t i = 0; i < trns->n_vars; i++)
+ {
+ const struct variable *v = trns->dst_vars[i];
+ if (v && var_get_type (v) != trns->dst_type)
+ {
+ if (trns->dst_type == VAL_STRING)
+ lex_ofs_error (lexer, dst_start, dst_end,
+ _("Type mismatch: cannot store string data in "
+ "numeric variable %s."), var_get_name (v));
+ else
+ lex_ofs_error (lexer, dst_start, dst_end,
+ _("Type mismatch: cannot store numeric data in "
+ "string variable %s."), var_get_name (v));
+ return false;
+ }
+ }
+
+ return true;
+}
+
+/* Ensures that all the output values in TRNS are as wide as the
+ widest destination variable. */
+static bool
+enlarge_dst_widths (struct lexer *lexer, struct recode_trns *trns,
+ int dst_start, int dst_end)
+{
+ const struct variable *narrow_var = NULL;
+ int min_dst_width = INT_MAX;
+ trns->max_dst_width = 0;
+
+ for (size_t i = 0; i < trns->n_vars; i++)
+ {
+ const struct variable *v = trns->dst_vars[i];
+ if (var_get_width (v) > trns->max_dst_width)
+ trns->max_dst_width = var_get_width (v);
+
+ if (var_get_width (v) < min_dst_width)
+ {
+ min_dst_width = var_get_width (v);
+ narrow_var = v;
+ }
+ }
+
+ for (size_t i = 0; i < trns->n_maps; i++)
+ {
+ struct map_out *out = &trns->mappings[i].out;
+ if (!out->copy_input)
+ {
+ if (out->width > min_dst_width)
+ {
+ msg (SE, _("At least one target variable is too narrow for "
+ "the output values."));
+ lex_ofs_msg (lexer, SN, out->ofs, out->ofs,
+ _("This recoding output value has width %d."),
+ out->width);
+ lex_ofs_msg (lexer, SN, dst_start, dst_end,
+ _("Target variable %s only has width %d."),
+ var_get_name (narrow_var),
+ var_get_width (narrow_var));
+ return false;
+ }
+
+ value_resize_pool (trns->pool, &out->value,
+ out->width, trns->max_dst_width);
+ }
+ }
+
+ return true;
+}
+
+/* Creates destination variables that don't already exist. */
+static void
+create_dst_vars (struct recode_trns *trns, struct dictionary *dict)
+{
+ for (size_t i = 0; i < trns->n_vars; i++)
+ {
+ const struct variable **var = &trns->dst_vars[i];
+ const char *name = trns->dst_names[i];
+
+ *var = dict_lookup_var (dict, name);
+ if (*var == NULL)
+ *var = dict_create_var_assert (dict, name, 0);
+ assert (var_get_type (*var) == trns->dst_type);
+ }
+}
+\f
+/* Data transformation. */
+
+/* Returns the output mapping in TRNS for an input of VALUE on
+ variable V, or a null pointer if there is no mapping. */
+static const struct map_out *
+find_src_numeric (struct recode_trns *trns, double value, const struct variable *v)
+{
+ for (struct mapping *m = trns->mappings; m < trns->mappings + trns->n_maps;
+ m++)
+ {
+ const struct map_in *in = &m->in;
+ const struct map_out *out = &m->out;
+ bool match;
+
+ switch (in->type)
+ {
+ case MAP_SINGLE:
+ match = value == in->x.f;
+ break;
+ case MAP_MISSING:
+ match = var_is_num_missing (v, value) != 0;
+ break;
+ case MAP_RANGE:
+ match = value >= in->x.f && value <= in->y.f;
+ break;
+ case MAP_SYSMIS:
+ match = value == SYSMIS;
+ break;
+ case MAP_ELSE:
+ match = true;
+ break;
+ default:
+ NOT_REACHED ();
+ }
+
+ if (match)
+ return out;
+ }
+
+ return NULL;
+}
+
+/* Returns the output mapping in TRNS for an input of VALUE with
+ the given WIDTH, or a null pointer if there is no mapping. */
+static const struct map_out *
+find_src_string (struct recode_trns *trns, const uint8_t *value,
+ const struct variable *src_var)
+{
+ const char *encoding = dict_get_encoding (trns->dst_dict);
+ int width = var_get_width (src_var);
+ for (struct mapping *m = trns->mappings; m < trns->mappings + trns->n_maps;
+ m++)
+ {
+ const struct map_in *in = &m->in;
+ struct map_out *out = &m->out;
+ bool match;
+
+ switch (in->type)
+ {
+ case MAP_SINGLE:
+ match = !memcmp (value, in->x.s, width);
+ break;
+ case MAP_ELSE:
+ match = true;
+ break;
+ case MAP_CONVERT:
+ {
+ union value uv;
+ char *error;
+
+ error = data_in (ss_buffer (CHAR_CAST_BUG (char *, value), width),
+ C_ENCODING, FMT_F, settings_get_fmt_settings (),
+ &uv, 0, encoding);
+ match = error == NULL;
+ free (error);
+
+ out->value.f = uv.f;
+ break;
+ }
+ case MAP_MISSING:
+ match = var_is_str_missing (src_var, value) != 0;
+ break;
+ default:
+ NOT_REACHED ();
+ }
+
+ if (match)
+ return out;
+ }
+
+ return NULL;
+}
+
+/* Performs RECODE transformation. */
+static enum trns_result
+recode_trns_proc (void *trns_, struct ccase **c, casenumber case_idx UNUSED)
+{
+ struct recode_trns *trns = trns_;
+
+ *c = case_unshare (*c);
+ for (size_t i = 0; i < trns->n_vars; i++)
+ {
+ const struct variable *src_var = trns->src_vars[i];
+ const struct variable *dst_var = trns->dst_vars[i];
+ const struct map_out *out;
+
+ if (trns->src_type == VAL_NUMERIC)
+ out = find_src_numeric (trns, case_num (*c, src_var), src_var);
+ else
+ out = find_src_string (trns, case_str (*c, src_var), src_var);
+
+ if (trns->dst_type == VAL_NUMERIC)
+ {
+ double *dst = case_num_rw (*c, dst_var);
+ if (out != NULL)
+ *dst = !out->copy_input ? out->value.f : case_num (*c, src_var);
+ else if (trns->src_vars != trns->dst_vars)
+ *dst = SYSMIS;
+ }
+ else
+ {
+ char *dst = CHAR_CAST_BUG (char *, case_str_rw (*c, dst_var));
+ if (out != NULL)
+ {
+ if (!out->copy_input)
+ memcpy (dst, out->value.s, var_get_width (dst_var));
+ else if (trns->src_vars != trns->dst_vars)
+ {
+ union value *dst_data = case_data_rw (*c, dst_var);
+ const union value *src_data = case_data (*c, src_var);
+ value_copy_rpad (dst_data, var_get_width (dst_var),
+ src_data, var_get_width (src_var), ' ');
+ }
+ }
+ else if (trns->src_vars != trns->dst_vars)
+ memset (dst, ' ', var_get_width (dst_var));
+ }
+ }
+
+ return TRNS_CONTINUE;
+}
+
+/* Frees a RECODE transformation. */
+static bool
+recode_trns_free (void *trns_)
+{
+ struct recode_trns *trns = trns_;
+ pool_destroy (trns->pool);
+ return true;
+}
+
+static const struct trns_class recode_trns_class = {
+ .name = "RECODE",
+ .execute = recode_trns_proc,
+ .destroy = recode_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2005, 2009, 2010, 2011, 2012, 2013, 2014,
+ 2016, 2017, 2019 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <stdbool.h>
+
+#include <gsl/gsl_math.h>
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_matrix.h>
+
+#include <data/dataset.h>
+#include <data/casewriter.h>
+
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dictionary.h"
+
+#include "math/covariance.h"
+#include "math/linreg.h"
+#include "math/moments.h"
+
+#include "libpspp/message.h"
+#include "libpspp/taint.h"
+
+#include "output/pivot-table.h"
+
+#include "gl/intprops.h"
+#include "gl/minmax.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+#define STATS_R (1 << 0)
+#define STATS_COEFF (1 << 1)
+#define STATS_ANOVA (1 << 2)
+#define STATS_OUTS (1 << 3)
+#define STATS_CI (1 << 4)
+#define STATS_BCOV (1 << 5)
+#define STATS_TOL (1 << 6)
+
+#define STATS_DEFAULT (STATS_R | STATS_COEFF | STATS_ANOVA | STATS_OUTS)
+
+
+struct regression
+ {
+ struct dataset *ds;
+
+ const struct variable **vars;
+ size_t n_vars;
+
+ const struct variable **dep_vars;
+ size_t n_dep_vars;
+
+ unsigned int stats;
+ double ci;
+
+ bool resid;
+ bool pred;
+
+ bool origin;
+ };
+
+struct regression_workspace
+{
+ /* The new variables which will be introduced by /SAVE */
+ const struct variable **predvars;
+ const struct variable **residvars;
+
+ /* A reader/writer pair to temporarily hold the
+ values of the new variables */
+ struct casewriter *writer;
+ struct casereader *reader;
+
+ /* Indeces of the new values in the reader/writer (-1 if not applicable) */
+ int res_idx;
+ int pred_idx;
+
+ /* 0, 1 or 2 depending on what new variables are to be created */
+ int extras;
+};
+
+static void run_regression (const struct regression *cmd,
+ struct regression_workspace *ws,
+ struct casereader *input);
+
+
+/* Return a string based on PREFIX which may be used as the name
+ of a new variable in DICT */
+static char *
+reg_get_name (const struct dictionary *dict, const char *prefix)
+{
+ for (size_t i = 1; ; i++)
+ {
+ char *name = xasprintf ("%s%zu", prefix, i);
+ if (!dict_lookup_var (dict, name))
+ return name;
+ free (name);
+ }
+}
+
+static const struct variable *
+create_aux_var (struct dataset *ds, const char *prefix)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ char *name = reg_get_name (dict, prefix);
+ struct variable *var = dict_create_var_assert (dict, name, 0);
+ free (name);
+ return var;
+}
+
+/* Auxiliary data for transformation when /SAVE is entered */
+struct save_trans_data
+ {
+ int n_dep_vars;
+ struct regression_workspace *ws;
+ };
+
+static bool
+save_trans_free (void *aux)
+{
+ struct save_trans_data *save_trans_data = aux;
+ free (save_trans_data->ws->predvars);
+ free (save_trans_data->ws->residvars);
+
+ casereader_destroy (save_trans_data->ws->reader);
+ free (save_trans_data->ws);
+ free (save_trans_data);
+ return true;
+}
+
+static enum trns_result
+save_trans_func (void *aux, struct ccase **c, casenumber x UNUSED)
+{
+ struct save_trans_data *save_trans_data = aux;
+ struct regression_workspace *ws = save_trans_data->ws;
+ struct ccase *in = casereader_read (ws->reader);
+
+ if (in)
+ {
+ *c = case_unshare (*c);
+
+ for (size_t k = 0; k < save_trans_data->n_dep_vars; ++k)
+ {
+ if (ws->pred_idx != -1)
+ {
+ double pred = case_num_idx (in, ws->extras * k + ws->pred_idx);
+ *case_num_rw (*c, ws->predvars[k]) = pred;
+ }
+
+ if (ws->res_idx != -1)
+ {
+ double resid = case_num_idx (in, ws->extras * k + ws->res_idx);
+ *case_num_rw (*c, ws->residvars[k]) = resid;
+ }
+ }
+ case_unref (in);
+ }
+
+ return TRNS_CONTINUE;
+}
+
+int
+cmd_regression (struct lexer *lexer, struct dataset *ds)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+
+ struct regression regression = {
+ .ci = 0.95,
+ .stats = STATS_DEFAULT,
+ .pred = false,
+ .resid = false,
+ .ds = ds,
+ .origin = false,
+ };
+
+ bool variables_seen = false;
+ bool method_seen = false;
+ bool dependent_seen = false;
+ int save_start = 0;
+ int save_end = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "VARIABLES"))
+ {
+ if (method_seen)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("VARIABLES may not appear after %s"), "METHOD");
+ goto error;
+ }
+ if (dependent_seen)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("VARIABLES may not appear after %s"), "DEPENDENT");
+ goto error;
+ }
+ variables_seen = true;
+ lex_match (lexer, T_EQUALS);
+
+ if (!parse_variables_const (lexer, dict,
+ ®ression.vars, ®ression.n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "DEPENDENT"))
+ {
+ dependent_seen = true;
+ lex_match (lexer, T_EQUALS);
+
+ free (regression.dep_vars);
+ regression.n_dep_vars = 0;
+
+ if (!parse_variables_const (lexer, dict,
+ ®ression.dep_vars,
+ ®ression.n_dep_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ORIGIN"))
+ regression.origin = true;
+ else if (lex_match_id (lexer, "NOORIGIN"))
+ regression.origin = false;
+ else if (lex_match_id (lexer, "METHOD"))
+ {
+ method_seen = true;
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_match_id (lexer, "ENTER"))
+ goto error;
+
+ if (!variables_seen)
+ {
+ if (!parse_variables_const (lexer, dict,
+ ®ression.vars, ®ression.n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "STATISTICS"))
+ {
+ unsigned long statistics = 0;
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match (lexer, T_ALL))
+ statistics = ~0;
+ else if (lex_match_id (lexer, "DEFAULTS"))
+ statistics |= STATS_DEFAULT;
+ else if (lex_match_id (lexer, "R"))
+ statistics |= STATS_R;
+ else if (lex_match_id (lexer, "COEFF"))
+ statistics |= STATS_COEFF;
+ else if (lex_match_id (lexer, "ANOVA"))
+ statistics |= STATS_ANOVA;
+ else if (lex_match_id (lexer, "BCOV"))
+ statistics |= STATS_BCOV;
+ else if (lex_match_id (lexer, "TOL"))
+ statistics |= STATS_TOL;
+ else if (lex_match_id (lexer, "CI"))
+ {
+ statistics |= STATS_CI;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num (lexer))
+ goto error;
+ regression.ci = lex_number (lexer) / 100.0;
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "ALL", "DEFAULTS", "R", "COEFF",
+ "ANOVA", "BCOV", "TOL", "CI");
+ goto error;
+ }
+ }
+
+ if (statistics)
+ regression.stats = statistics;
+ }
+ else if (lex_match_id (lexer, "SAVE"))
+ {
+ save_start = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+
+ while (lex_token (lexer) != T_ENDCMD
+ && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "PRED"))
+ regression.pred = true;
+ else if (lex_match_id (lexer, "RESID"))
+ regression.resid = true;
+ else
+ {
+ lex_error_expecting (lexer, "PRED", "RESID");
+ goto error;
+ }
+ }
+ save_end = lex_ofs (lexer) - 1;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "VARIABLES", "DEPENDENT", "ORIGIN",
+ "NOORIGIN", "METHOD", "STATISTICS", "SAVE");
+ goto error;
+ }
+ }
+
+ if (!regression.vars)
+ dict_get_vars (dict, ®ression.vars, ®ression.n_vars, 0);
+
+ struct regression_workspace workspace = {
+ .res_idx = -1,
+ .pred_idx = -1,
+ };
+
+ bool save = regression.pred || regression.resid;
+ if (save)
+ {
+ struct caseproto *proto = caseproto_create ();
+
+ if (regression.resid)
+ {
+ workspace.res_idx = workspace.extras ++;
+ workspace.residvars = xcalloc (regression.n_dep_vars, sizeof (*workspace.residvars));
+
+ for (size_t i = 0; i < regression.n_dep_vars; ++i)
+ {
+ workspace.residvars[i] = create_aux_var (ds, "RES");
+ proto = caseproto_add_width (proto, 0);
+ }
+ }
+
+ if (regression.pred)
+ {
+ workspace.pred_idx = workspace.extras ++;
+ workspace.predvars = xcalloc (regression.n_dep_vars, sizeof (*workspace.predvars));
+
+ for (size_t i = 0; i < regression.n_dep_vars; ++i)
+ {
+ workspace.predvars[i] = create_aux_var (ds, "PRED");
+ proto = caseproto_add_width (proto, 0);
+ }
+ }
+
+ if (proc_make_temporary_transformations_permanent (ds))
+ lex_ofs_msg (lexer, SW, save_start, save_end,
+ _("REGRESSION with SAVE ignores TEMPORARY. "
+ "Temporary transformations will be made permanent."));
+
+ if (dict_get_filter (dict))
+ lex_ofs_msg (lexer, SW, save_start, save_end,
+ _("REGRESSION with SAVE ignores FILTER. "
+ "All cases will be processed."));
+
+ workspace.writer = autopaging_writer_create (proto);
+ caseproto_unref (proto);
+ }
+
+ struct casegrouper *grouper = casegrouper_create_splits (
+ proc_open_filtering (ds, !save), dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ run_regression (®ression,
+ &workspace,
+ group);
+
+ }
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+ if (workspace.writer)
+ {
+ struct save_trans_data *save_trans_data = xmalloc (sizeof *save_trans_data);
+ struct casereader *r = casewriter_make_reader (workspace.writer);
+ workspace.writer = NULL;
+ workspace.reader = r;
+ save_trans_data->ws = xmalloc (sizeof (workspace));
+ memcpy (save_trans_data->ws, &workspace, sizeof (workspace));
+ save_trans_data->n_dep_vars = regression.n_dep_vars;
+
+ static const struct trns_class trns_class = {
+ .name = "REGRESSION",
+ .execute = save_trans_func,
+ .destroy = save_trans_free,
+ };
+ add_transformation (ds, &trns_class, save_trans_data);
+ }
+
+ free (regression.vars);
+ free (regression.dep_vars);
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+
+error:
+ free (regression.vars);
+ free (regression.dep_vars);
+ return CMD_FAILURE;
+}
+
+/* Return the size of the union of dependent and independent variables */
+static size_t
+get_n_all_vars (const struct regression *cmd)
+{
+ size_t result = cmd->n_vars + cmd->n_dep_vars;
+ for (size_t i = 0; i < cmd->n_dep_vars; i++)
+ for (size_t j = 0; j < cmd->n_vars; j++)
+ if (cmd->vars[j] == cmd->dep_vars[i])
+ result--;
+ return result;
+}
+
+/* Fill VARS with the union of dependent and independent variables */
+static void
+fill_all_vars (const struct variable **vars, const struct regression *cmd)
+{
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ vars[i] = cmd->vars[i];
+
+ size_t x = 0;
+ for (size_t i = 0; i < cmd->n_dep_vars; i++)
+ {
+ bool absent = true;
+ for (size_t j = 0; j < cmd->n_vars; j++)
+ if (cmd->dep_vars[i] == cmd->vars[j])
+ {
+ absent = false;
+ break;
+ }
+ if (absent)
+ vars[cmd->n_vars + x++] = cmd->dep_vars[i];
+ }
+}
+
+
+/* Fill the array VARS, with all the predictor variables from CMD, except
+ variable X */
+static void
+fill_predictor_x (const struct variable **vars, const struct variable *x, const struct regression *cmd)
+{
+ size_t n = 0;
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ if (cmd->vars[i] != x)
+ vars[n++] = cmd->vars[i];
+}
+
+/*
+ Is variable k the dependent variable?
+*/
+static bool
+is_depvar (const struct regression *cmd, size_t k, const struct variable *v)
+{
+ return v == cmd->vars[k];
+}
+
+/* Identify the explanatory variables in v_variables. Returns
+ the number of independent variables. */
+static int
+identify_indep_vars (const struct regression *cmd,
+ const struct variable **indep_vars,
+ const struct variable *depvar)
+{
+ int n_indep_vars = 0;
+
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ if (!is_depvar (cmd, i, depvar))
+ indep_vars[n_indep_vars++] = cmd->vars[i];
+ if (n_indep_vars < 1 && is_depvar (cmd, 0, depvar))
+ {
+ /*
+ There is only one independent variable, and it is the same
+ as the dependent variable. Print a warning and continue.
+ */
+ msg (SW,
+ _("The dependent variable is equal to the independent variable. "
+ "The least squares line is therefore Y=X. "
+ "Standard errors and related statistics may be meaningless."));
+ n_indep_vars = 1;
+ indep_vars[0] = cmd->vars[0];
+ }
+ return n_indep_vars;
+}
+
+static double
+fill_covariance (gsl_matrix * cov, struct covariance *all_cov,
+ const struct variable **vars,
+ size_t n_vars, const struct variable *dep_var,
+ const struct variable **all_vars, size_t n_all_vars,
+ double *means)
+{
+ const gsl_matrix *cm = covariance_calculate_unnormalized (all_cov);
+ if (!cm)
+ return 0;
+
+ size_t *rows = xnmalloc (cov->size1 - 1, sizeof (*rows));
+
+ size_t dep_subscript = SIZE_MAX;
+ for (size_t i = 0; i < n_all_vars; i++)
+ {
+ for (size_t j = 0; j < n_vars; j++)
+ if (vars[j] == all_vars[i])
+ rows[j] = i;
+ if (all_vars[i] == dep_var)
+ dep_subscript = i;
+ }
+ assert (dep_subscript != SIZE_MAX);
+
+ const gsl_matrix *mean_matrix = covariance_moments (all_cov, MOMENT_MEAN);
+ const gsl_matrix *ssize_matrix = covariance_moments (all_cov, MOMENT_NONE);
+ for (size_t i = 0; i < cov->size1 - 1; i++)
+ {
+ means[i] = gsl_matrix_get (mean_matrix, rows[i], 0)
+ / gsl_matrix_get (ssize_matrix, rows[i], 0);
+ for (size_t j = 0; j < cov->size2 - 1; j++)
+ {
+ gsl_matrix_set (cov, i, j, gsl_matrix_get (cm, rows[i], rows[j]));
+ gsl_matrix_set (cov, j, i, gsl_matrix_get (cm, rows[j], rows[i]));
+ }
+ }
+ means[cov->size1 - 1] = gsl_matrix_get (mean_matrix, dep_subscript, 0)
+ / gsl_matrix_get (ssize_matrix, dep_subscript, 0);
+ const gsl_matrix *ssizes = covariance_moments (all_cov, MOMENT_NONE);
+ double result = gsl_matrix_get (ssizes, dep_subscript, rows[0]);
+ for (size_t i = 0; i < cov->size1 - 1; i++)
+ {
+ gsl_matrix_set (cov, i, cov->size1 - 1,
+ gsl_matrix_get (cm, rows[i], dep_subscript));
+ gsl_matrix_set (cov, cov->size1 - 1, i,
+ gsl_matrix_get (cm, rows[i], dep_subscript));
+ if (result > gsl_matrix_get (ssizes, rows[i], dep_subscript))
+ result = gsl_matrix_get (ssizes, rows[i], dep_subscript);
+ }
+ gsl_matrix_set (cov, cov->size1 - 1, cov->size1 - 1,
+ gsl_matrix_get (cm, dep_subscript, dep_subscript));
+ free (rows);
+ return result;
+}
+
+\f
+
+struct model_container
+{
+ struct linreg **models;
+};
+
+/*
+ STATISTICS subcommand output functions.
+*/
+static void reg_stats_r (const struct linreg *, const struct variable *);
+static void reg_stats_coeff (const struct regression *, const struct linreg *,
+ const struct model_container *, const gsl_matrix *,
+ const struct variable *);
+static void reg_stats_anova (const struct linreg *, const struct variable *);
+static void reg_stats_bcov (const struct linreg *, const struct variable *);
+
+
+static struct linreg **
+run_regression_get_models (const struct regression *cmd,
+ struct casereader *input,
+ bool output)
+{
+ struct model_container *model_container = XCALLOC (cmd->n_vars, struct model_container);
+
+ struct ccase *c;
+ struct covariance *cov;
+ struct casereader *reader;
+
+ if (cmd->stats & STATS_TOL)
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ {
+ struct regression subreg = {
+ .origin = cmd->origin,
+ .ds = cmd->ds,
+ .n_vars = cmd->n_vars - 1,
+ .n_dep_vars = 1,
+ .vars = xmalloc ((cmd->n_vars - 1) * sizeof *subreg.vars),
+ .dep_vars = &cmd->vars[i],
+ .stats = STATS_R,
+ .ci = 0,
+ .resid = false,
+ .pred = false,
+ };
+ fill_predictor_x (subreg.vars, cmd->vars[i], cmd);
+
+ model_container[i].models =
+ run_regression_get_models (&subreg, input, false);
+ free (subreg.vars);
+ }
+
+ size_t n_all_vars = get_n_all_vars (cmd);
+ const struct variable **all_vars = xnmalloc (n_all_vars, sizeof (*all_vars));
+
+ /* In the (rather pointless) case where the dependent variable is
+ the independent variable, n_all_vars == 1.
+ However this would result in a buffer overflow so we must
+ over-allocate the space required in this malloc call.
+ See bug #58599 */
+ double *means = xnmalloc (MAX (2, n_all_vars), sizeof *means);
+ fill_all_vars (all_vars, cmd);
+ cov = covariance_1pass_create (n_all_vars, all_vars,
+ dict_get_weight (dataset_dict (cmd->ds)),
+ MV_ANY, cmd->origin == false);
+
+ reader = casereader_clone (input);
+ reader = casereader_create_filter_missing (reader, all_vars, n_all_vars,
+ MV_ANY, NULL, NULL);
+
+ struct casereader *r = casereader_clone (reader);
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ covariance_accumulate (cov, c);
+ casereader_destroy (r);
+
+ struct linreg **models = XCALLOC (cmd->n_dep_vars, struct linreg*);
+ for (size_t k = 0; k < cmd->n_dep_vars; k++)
+ {
+ const struct variable **vars = xnmalloc (cmd->n_vars, sizeof *vars);
+ const struct variable *dep_var = cmd->dep_vars[k];
+ int n_indep = identify_indep_vars (cmd, vars, dep_var);
+ gsl_matrix *cov_matrix = gsl_matrix_alloc (n_indep + 1, n_indep + 1);
+ double n_data = fill_covariance (cov_matrix, cov, vars, n_indep,
+ dep_var, all_vars, n_all_vars, means);
+ models[k] = linreg_alloc (dep_var, vars, n_data, n_indep, cmd->origin);
+ for (size_t i = 0; i < n_indep; i++)
+ linreg_set_indep_variable_mean (models[k], i, means[i]);
+ linreg_set_depvar_mean (models[k], means[n_indep]);
+ if (n_data > 0)
+ {
+ linreg_fit (cov_matrix, models[k]);
+
+ if (output
+ && !taint_has_tainted_successor (casereader_get_taint (input)))
+ {
+ /*
+ Find the least-squares estimates and other statistics.
+ */
+ if (cmd->stats & STATS_R)
+ reg_stats_r (models[k], dep_var);
+
+ if (cmd->stats & STATS_ANOVA)
+ reg_stats_anova (models[k], dep_var);
+
+ if (cmd->stats & STATS_COEFF)
+ reg_stats_coeff (cmd, models[k],
+ model_container,
+ cov_matrix, dep_var);
+
+ if (cmd->stats & STATS_BCOV)
+ reg_stats_bcov (models[k], dep_var);
+ }
+ }
+ else
+ msg (SE, _("No valid data found. This command was skipped."));
+ free (vars);
+ gsl_matrix_free (cov_matrix);
+ }
+
+ casereader_destroy (reader);
+
+ for (size_t i = 0; i < cmd->n_vars; i++)
+ {
+ if (model_container[i].models)
+ linreg_unref (model_container[i].models[0]);
+ free (model_container[i].models);
+ }
+ free (model_container);
+
+ free (all_vars);
+ free (means);
+ covariance_destroy (cov);
+ return models;
+}
+
+static void
+run_regression (const struct regression *cmd,
+ struct regression_workspace *ws,
+ struct casereader *input)
+{
+ struct linreg **models = run_regression_get_models (cmd, input, true);
+
+ if (ws->extras > 0)
+ {
+ struct ccase *c;
+ struct casereader *r = casereader_clone (input);
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ struct ccase *outc = case_create (casewriter_get_proto (ws->writer));
+ for (int k = 0; k < cmd->n_dep_vars; k++)
+ {
+ const struct variable **vars = xnmalloc (cmd->n_vars, sizeof (*vars));
+ const struct variable *dep_var = cmd->dep_vars[k];
+ int n_indep = identify_indep_vars (cmd, vars, dep_var);
+ double *vals = xnmalloc (n_indep, sizeof (*vals));
+ for (int i = 0; i < n_indep; i++)
+ {
+ const union value *tmp = case_data (c, vars[i]);
+ vals[i] = tmp->f;
+ }
+
+ if (cmd->pred)
+ {
+ double pred = linreg_predict (models[k], vals, n_indep);
+ *case_num_rw_idx (outc, k * ws->extras + ws->pred_idx) = pred;
+ }
+
+ if (cmd->resid)
+ {
+ double obs = case_num (c, linreg_dep_var (models[k]));
+ double res = linreg_residual (models[k], obs, vals, n_indep);
+ *case_num_rw_idx (outc, k * ws->extras + ws->res_idx) = res;
+ }
+ free (vals);
+ free (vars);
+ }
+ casewriter_write (ws->writer, outc);
+ }
+ casereader_destroy (r);
+ }
+
+ for (size_t k = 0; k < cmd->n_dep_vars; k++)
+ linreg_unref (models[k]);
+
+ free (models);
+ casereader_destroy (input);
+}
+
+\f
+
+
+static void
+reg_stats_r (const struct linreg * c, const struct variable *var)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("Model Summary (%s)"),
+ var_to_string (var)),
+ "Model Summary");
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("R"), N_("R Square"), N_("Adjusted R Square"),
+ N_("Std. Error of the Estimate"));
+
+ double rsq = linreg_ssreg (c) / linreg_sst (c);
+ double adjrsq = (rsq -
+ (1.0 - rsq) * linreg_n_coeffs (c)
+ / (linreg_n_obs (c) - linreg_n_coeffs (c) - 1));
+ double std_error = sqrt (linreg_mse (c));
+
+ double entries[] = {
+ sqrt (rsq), rsq, adjrsq, std_error
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
+
+ pivot_table_submit (table);
+}
+
+/*
+ Table showing estimated regression coefficients.
+*/
+static void
+reg_stats_coeff (const struct regression *cmd, const struct linreg *c,
+ const struct model_container *mc, const gsl_matrix *cov,
+ const struct variable *var)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("Coefficients (%s)"), var_to_string (var)),
+ "Coefficients");
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ pivot_category_create_group (statistics->root,
+ N_("Unstandardized Coefficients"),
+ N_("B"), N_("Std. Error"));
+ pivot_category_create_group (statistics->root,
+ N_("Standardized Coefficients"), N_("Beta"));
+ pivot_category_create_leaves (statistics->root, N_("t"),
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+ if (cmd->stats & STATS_CI)
+ {
+ struct pivot_category *interval = pivot_category_create_group__ (
+ statistics->root, pivot_value_new_text_format (
+ N_("%g%% Confidence Interval for B"),
+ cmd->ci * 100.0));
+ pivot_category_create_leaves (interval, N_("Lower Bound"),
+ N_("Upper Bound"));
+ }
+
+ if (cmd->stats & STATS_TOL)
+ pivot_category_create_group (statistics->root,
+ N_("Collinearity Statistics"),
+ N_("Tolerance"), N_("VIF"));
+
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ double df = linreg_n_obs (c) - linreg_n_coeffs (c) - 1;
+ double q = (1 - cmd->ci) / 2.0; /* 2-tailed test */
+ double tval = gsl_cdf_tdist_Qinv (q, df);
+
+ if (!cmd->origin)
+ {
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_text (N_("(Constant)")));
+
+ double std_err = sqrt (gsl_matrix_get (linreg_cov (c), 0, 0));
+ double t_stat = linreg_intercept (c) / std_err;
+ double base_entries[] = {
+ linreg_intercept (c),
+ std_err,
+ 0.0,
+ t_stat,
+ 2.0 * gsl_cdf_tdist_Q (fabs (t_stat),
+ linreg_n_obs (c) - linreg_n_coeffs (c)),
+ };
+
+ size_t col = 0;
+ for (size_t i = 0; i < sizeof base_entries / sizeof *base_entries; i++)
+ pivot_table_put2 (table, col++, var_idx,
+ pivot_value_new_number (base_entries[i]));
+
+ if (cmd->stats & STATS_CI)
+ {
+ double interval_entries[] = {
+ linreg_intercept (c) - tval * std_err,
+ linreg_intercept (c) + tval * std_err,
+ };
+
+ for (size_t i = 0; i < sizeof interval_entries / sizeof *interval_entries; i++)
+ pivot_table_put2 (table, col++, var_idx,
+ pivot_value_new_number (interval_entries[i]));
+ }
+ }
+
+ for (size_t j = 0; j < linreg_n_coeffs (c); j++)
+ {
+ const struct variable *v = linreg_indep_var (c, j);
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (v));
+
+ double std_err = sqrt (gsl_matrix_get (linreg_cov (c), j + 1, j + 1));
+ double t_stat = linreg_coeff (c, j) / std_err;
+ double base_entries[] = {
+ linreg_coeff (c, j),
+ sqrt (gsl_matrix_get (linreg_cov (c), j + 1, j + 1)),
+ (sqrt (gsl_matrix_get (cov, j, j)) * linreg_coeff (c, j) /
+ sqrt (gsl_matrix_get (cov, cov->size1 - 1, cov->size2 - 1))),
+ t_stat,
+ 2 * gsl_cdf_tdist_Q (fabs (t_stat), df)
+ };
+
+ size_t col = 0;
+ for (size_t i = 0; i < sizeof base_entries / sizeof *base_entries; i++)
+ pivot_table_put2 (table, col++, var_idx,
+ pivot_value_new_number (base_entries[i]));
+
+ if (cmd->stats & STATS_CI)
+ {
+ double interval_entries[] = {
+ linreg_coeff (c, j) - tval * std_err,
+ linreg_coeff (c, j) + tval * std_err,
+ };
+
+
+ for (size_t i = 0; i < sizeof interval_entries / sizeof *interval_entries; i++)
+ pivot_table_put2 (table, col++, var_idx,
+ pivot_value_new_number (interval_entries[i]));
+ }
+
+ if (cmd->stats & STATS_TOL)
+ {
+ {
+ struct linreg *m = mc[j].models[0];
+ double rsq = linreg_ssreg (m) / linreg_sst (m);
+ pivot_table_put2 (table, col++, var_idx, pivot_value_new_number (1.0 - rsq));
+ pivot_table_put2 (table, col++, var_idx, pivot_value_new_number (1.0 / (1.0 - rsq)));
+ }
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+/*
+ Display the ANOVA table.
+*/
+static void
+reg_stats_anova (const struct linreg * c, const struct variable *var)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("ANOVA (%s)"), var_to_string (var)),
+ "ANOVA");
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Sum of Squares"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_INTEGER,
+ N_("Mean Square"), PIVOT_RC_OTHER,
+ N_("F"), PIVOT_RC_OTHER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Source"),
+ N_("Regression"), N_("Residual"), N_("Total"));
+
+ double msm = linreg_ssreg (c) / linreg_dfmodel (c);
+ double mse = linreg_mse (c);
+ double F = msm / mse;
+
+ struct entry
+ {
+ int stat_idx;
+ int source_idx;
+ double x;
+ }
+ entries[] = {
+ /* Sums of Squares. */
+ { 0, 0, linreg_ssreg (c) },
+ { 0, 1, linreg_sse (c) },
+ { 0, 2, linreg_sst (c) },
+ /* Degrees of freedom. */
+ { 1, 0, linreg_dfmodel (c) },
+ { 1, 1, linreg_dferror (c) },
+ { 1, 2, linreg_dftotal (c) },
+ /* Mean Squares. */
+ { 2, 0, msm },
+ { 2, 1, mse },
+ /* F */
+ { 3, 0, F },
+ /* Significance. */
+ { 4, 0, gsl_cdf_fdist_Q (F, linreg_dfmodel (c), linreg_dferror (c)) },
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put2 (table, e->stat_idx, e->source_idx,
+ pivot_value_new_number (e->x));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+reg_stats_bcov (const struct linreg * c, const struct variable *var)
+{
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("Coefficient Correlations (%s)"),
+ var_to_string (var)),
+ "Coefficient Correlations");
+
+ for (size_t i = 0; i < 2; i++)
+ {
+ struct pivot_dimension *models = pivot_dimension_create (
+ table, i ? PIVOT_AXIS_ROW : PIVOT_AXIS_COLUMN, N_("Models"));
+ for (size_t j = 0; j < linreg_n_coeffs (c); j++)
+ pivot_category_create_leaf (
+ models->root, pivot_value_new_variable (
+ linreg_indep_var (c, j)));
+ }
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Covariances"));
+
+ for (size_t i = 0; i < linreg_n_coeffs (c); i++)
+ for (size_t k = 0; k < linreg_n_coeffs (c); k++)
+ {
+ double cov = gsl_matrix_get (linreg_cov (c), MIN (i, k), MAX (i, k));
+ pivot_table_put3 (table, k, i, 0, pivot_value_new_number (cov));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009, 2010, 2011, 2013, 2015, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <math.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/str.h"
+#include "math/moments.h"
+#include "output/pivot-table.h"
+#include "output/output-item.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+struct cronbach
+ {
+ const struct variable **items;
+ size_t n_items;
+ double alpha;
+ double sum_of_variances;
+ double variance_of_sums;
+ int totals_idx; /* Casereader index into the totals */
+
+ struct moments1 **m; /* Moments of the items */
+ struct moments1 *total; /* Moments of the totals */
+ };
+
+#if 0
+static void
+dump_cronbach (const struct cronbach *s)
+{
+ int i;
+ printf ("N items %d\n", s->n_items);
+ for (i = 0; i < s->n_items; ++i)
+ {
+ printf ("%s\n", var_get_name (s->items[i]));
+ }
+
+ printf ("Totals idx %d\n", s->totals_idx);
+
+ printf ("scale variance %g\n", s->variance_of_sums);
+ printf ("alpha %g\n", s->alpha);
+ putchar ('\n');
+}
+#endif
+
+enum model
+ {
+ MODEL_ALPHA,
+ MODEL_SPLIT
+ };
+
+
+struct reliability
+{
+ const struct variable **vars;
+ size_t n_vars;
+ enum mv_class exclude;
+
+ struct cronbach *sc;
+ int n_sc;
+
+ int total_start;
+
+ char *scale_name;
+
+ enum model model;
+ int split_point;
+
+ bool summary_total;
+
+ const struct variable *wv;
+};
+
+
+static bool run_reliability (struct dataset *ds, const struct reliability *reliability);
+
+static void
+reliability_destroy (struct reliability *rel)
+{
+ int j;
+ free (rel->scale_name);
+ if (rel->sc)
+ for (j = 0; j < rel->n_sc; ++j)
+ {
+ int x;
+ free (rel->sc[j].items);
+ moments1_destroy (rel->sc[j].total);
+ if (rel->sc[j].m)
+ for (x = 0; x < rel->sc[j].n_items; ++x)
+ free (rel->sc[j].m[x]);
+ free (rel->sc[j].m);
+ }
+
+ free (rel->sc);
+ free (rel->vars);
+}
+
+int
+cmd_reliability (struct lexer *lexer, struct dataset *ds)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+
+ struct reliability r = {
+ .model = MODEL_ALPHA,
+ .exclude = MV_ANY,
+ .wv = dict_get_weight (dict),
+ .scale_name = xstrdup ("ANY"),
+ };
+
+ lex_match (lexer, T_SLASH);
+
+ if (!lex_force_match_id (lexer, "VARIABLES"))
+ goto error;
+
+ lex_match (lexer, T_EQUALS);
+
+ int vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, dict, &r.vars, &r.n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ if (r.n_vars < 2)
+ lex_ofs_msg (lexer, SW, vars_start, vars_end,
+ _("Reliability on a single variable is not useful."));
+
+ /* Create a default scale. */
+ r.n_sc = 1;
+ r.sc = xcalloc (r.n_sc, sizeof (struct cronbach));
+
+ struct cronbach *c = &r.sc[0];
+ c->n_items = r.n_vars;
+ c->items = xcalloc (c->n_items, sizeof (struct variable*));
+ for (size_t i = 0; i < c->n_items; ++i)
+ c->items[i] = r.vars[i];
+
+ int split_ofs = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "SCALE"))
+ {
+ struct const_var_set *vs;
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ if (!lex_force_string (lexer))
+ goto error;
+ free (r.scale_name);
+ r.scale_name = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ lex_match (lexer, T_EQUALS);
+
+ vs = const_var_set_create_from_array (r.vars, r.n_vars);
+
+ free (r.sc->items);
+ if (!parse_const_var_set_vars (lexer, vs, &r.sc->items, &r.sc->n_items, 0))
+ {
+ const_var_set_destroy (vs);
+ goto error;
+ }
+
+ const_var_set_destroy (vs);
+ }
+ else if (lex_match_id (lexer, "MODEL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "ALPHA"))
+ r.model = MODEL_ALPHA;
+ else if (lex_match_id (lexer, "SPLIT"))
+ {
+ r.model = MODEL_SPLIT;
+ r.split_point = -1;
+
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!lex_force_num (lexer))
+ goto error;
+ split_ofs = lex_ofs (lexer);
+ r.split_point = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "ALPHA", "SPLIT");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "SUMMARY"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "TOTAL") || lex_match (lexer, T_ALL))
+ r.summary_total = true;
+ else
+ {
+ lex_error_expecting (lexer, "TOTAL", "ALL");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ r.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ r.exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "STATISTICS"))
+ {
+ int statistics_start = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+ while (lex_match (lexer, T_ID))
+ continue;
+ int statistics_end = lex_ofs (lexer) - 1;
+
+ lex_ofs_msg (lexer, SW, statistics_start, statistics_end,
+ _("The STATISTICS subcommand is not yet implemented. "
+ "No statistics will be produced."));
+ }
+ else
+ {
+ lex_error_expecting (lexer, "SCALE", "MODEL", "SUMMARY", "MISSING",
+ "STATISTICS");
+ goto error;
+ }
+ }
+
+ if (r.model == MODEL_SPLIT)
+ {
+ if (r.split_point >= r.n_vars)
+ {
+ lex_ofs_error (lexer, split_ofs, split_ofs,
+ _("The split point must be less than the "
+ "number of variables."));
+ lex_ofs_msg (lexer, SN, vars_start, vars_end,
+ ngettext ("There is %zu variable.",
+ "There are %zu variables.", r.n_vars),
+ r.n_vars);
+ goto error;
+ }
+
+ r.n_sc += 2;
+ r.sc = xrealloc (r.sc, sizeof (struct cronbach) * r.n_sc);
+
+ const struct cronbach *s = &r.sc[0];
+
+ r.sc[1].n_items = r.split_point == -1 ? s->n_items / 2 : r.split_point;
+
+ r.sc[2].n_items = s->n_items - r.sc[1].n_items;
+ r.sc[1].items = XCALLOC (r.sc[1].n_items, const struct variable *);
+ r.sc[2].items = XCALLOC (r.sc[2].n_items, const struct variable *);
+
+ size_t i = 0;
+ while (i < r.sc[1].n_items)
+ {
+ r.sc[1].items[i] = s->items[i];
+ i++;
+ }
+ while (i < s->n_items)
+ {
+ r.sc[2].items[i - r.sc[1].n_items] = s->items[i];
+ i++;
+ }
+ }
+
+ if (r.summary_total)
+ {
+ const int base_sc = r.n_sc;
+
+ r.total_start = base_sc;
+
+ r.n_sc += r.sc[0].n_items;
+ r.sc = xrealloc (r.sc, sizeof (struct cronbach) * r.n_sc);
+
+ for (size_t i = 0; i < r.sc[0].n_items; ++i)
+ {
+ struct cronbach *s = &r.sc[i + base_sc];
+
+ s->n_items = r.sc[0].n_items - 1;
+ s->items = xcalloc (s->n_items, sizeof (struct variable *));
+
+ size_t v_dest = 0;
+ for (size_t v_src = 0; v_src < r.sc[0].n_items; ++v_src)
+ if (v_src != i)
+ s->items[v_dest++] = r.sc[0].items[v_src];
+ }
+ }
+
+ if (!run_reliability (ds, &r))
+ goto error;
+
+ reliability_destroy (&r);
+ return CMD_SUCCESS;
+
+ error:
+ reliability_destroy (&r);
+ return CMD_FAILURE;
+}
+
+
+static void
+do_reliability (struct casereader *group, struct dataset *ds,
+ const struct reliability *rel);
+
+
+static void reliability_summary_total (const struct reliability *rel);
+
+static void reliability_statistics (const struct reliability *rel);
+
+
+static bool
+run_reliability (struct dataset *ds, const struct reliability *reliability)
+{
+ for (size_t si = 0; si < reliability->n_sc; ++si)
+ {
+ struct cronbach *s = &reliability->sc[si];
+
+ s->m = xcalloc (s->n_items, sizeof *s->m);
+ s->total = moments1_create (MOMENT_VARIANCE);
+
+ for (size_t i = 0; i < s->n_items; ++i)
+ s->m[i] = moments1_create (MOMENT_VARIANCE);
+ }
+
+ struct dictionary *dict = dataset_dict (ds);
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
+ struct casereader *group;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ do_reliability (group, ds, reliability);
+
+ reliability_statistics (reliability);
+
+ if (reliability->summary_total)
+ reliability_summary_total (reliability);
+ }
+
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+ return ok;
+}
+\f
+/* Return the sum of all the item variables in S */
+static double
+append_sum (const struct ccase *c, casenumber n UNUSED, void *aux)
+{
+ double sum = 0;
+ const struct cronbach *s = aux;
+
+ for (int v = 0; v < s->n_items; ++v)
+ sum += case_num (c, s->items[v]);
+
+ return sum;
+}
+
+static void
+case_processing_summary (casenumber n_valid, casenumber n_missing,
+ const struct dictionary *);
+
+static double
+alpha (int k, double sum_of_variances, double variance_of_sums)
+{
+ return k / (k - 1.0) * (1 - sum_of_variances / variance_of_sums);
+}
+
+static void
+do_reliability (struct casereader *input, struct dataset *ds,
+ const struct reliability *rel)
+{
+ for (size_t si = 0; si < rel->n_sc; ++si)
+ {
+ struct cronbach *s = &rel->sc[si];
+
+ moments1_clear (s->total);
+ for (size_t i = 0; i < s->n_items; ++i)
+ moments1_clear (s->m[i]);
+ }
+
+ casenumber n_missing;
+ input = casereader_create_filter_missing (input,
+ rel->vars,
+ rel->n_vars,
+ rel->exclude,
+ &n_missing,
+ NULL);
+
+ for (size_t si = 0; si < rel->n_sc; ++si)
+ {
+ struct cronbach *s = &rel->sc[si];
+ s->totals_idx = caseproto_get_n_widths (casereader_get_proto (input));
+ input = casereader_create_append_numeric (input, append_sum, s, NULL);
+ }
+
+ struct ccase *c;
+ casenumber n_valid = 0;
+ for (; (c = casereader_read (input)) != NULL; case_unref (c))
+ {
+ double weight = 1.0;
+ n_valid++;
+
+ for (size_t si = 0; si < rel->n_sc; ++si)
+ {
+ struct cronbach *s = &rel->sc[si];
+
+ for (size_t i = 0; i < s->n_items; ++i)
+ moments1_add (s->m[i], case_num (c, s->items[i]), weight);
+ moments1_add (s->total, case_num_idx (c, s->totals_idx), weight);
+ }
+ }
+ casereader_destroy (input);
+
+ for (size_t si = 0; si < rel->n_sc; ++si)
+ {
+ struct cronbach *s = &rel->sc[si];
+
+ s->sum_of_variances = 0;
+ for (size_t i = 0; i < s->n_items; ++i)
+ {
+ double weight, mean, variance;
+ moments1_calculate (s->m[i], &weight, &mean, &variance, NULL, NULL);
+
+ s->sum_of_variances += variance;
+ }
+
+ moments1_calculate (s->total, NULL, NULL, &s->variance_of_sums,
+ NULL, NULL);
+
+ s->alpha = alpha (s->n_items, s->sum_of_variances, s->variance_of_sums);
+ }
+
+ output_item_submit (text_item_create_nocopy (
+ TEXT_ITEM_TITLE,
+ xasprintf (_("Scale: %s"), rel->scale_name),
+ NULL));
+
+ case_processing_summary (n_valid, n_missing, dataset_dict (ds));
+}
+
+static void
+case_processing_summary (casenumber n_valid, casenumber n_missing,
+ const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Case Processing Summary"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Percent"), PIVOT_RC_PERCENT);
+
+ struct pivot_dimension *cases = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Cases"), N_("Valid"), N_("Excluded"),
+ N_("Total"));
+ cases->root->show_label = true;
+
+ casenumber total = n_missing + n_valid;
+
+ struct entry
+ {
+ int stat_idx;
+ int case_idx;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, n_valid },
+ { 0, 1, n_missing },
+ { 0, 2, total },
+ { 1, 0, 100.0 * n_valid / total },
+ { 1, 1, 100.0 * n_missing / total },
+ { 1, 2, 100.0 }
+ };
+
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put2 (table, e->stat_idx, e->case_idx,
+ pivot_value_new_number (e->x));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+reliability_summary_total (const struct reliability *rel)
+{
+ struct pivot_table *table = pivot_table_create (N_("Item-Total Statistics"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Scale Mean if Item Deleted"),
+ N_("Scale Variance if Item Deleted"),
+ N_("Corrected Item-Total Correlation"),
+ N_("Cronbach's Alpha if Item Deleted"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0; i < rel->sc[0].n_items; ++i)
+ {
+ const struct cronbach *s = &rel->sc[rel->total_start + i];
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (rel->sc[0].items[i]));
+
+ double mean;
+ moments1_calculate (s->total, NULL, &mean, NULL, NULL, NULL);
+
+ double var;
+ moments1_calculate (rel->sc[0].m[i], NULL, NULL, &var, NULL, NULL);
+ double cov
+ = (rel->sc[0].variance_of_sums + var - s->variance_of_sums) / 2.0;
+
+ double entries[] = {
+ mean,
+ s->variance_of_sums,
+ (cov - var) / sqrt (var * s->variance_of_sums),
+ s->alpha,
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, var_idx,
+ pivot_value_new_number (entries[i]));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+reliability_statistics (const struct reliability *rel)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Reliability Statistics"));
+ pivot_table_set_weight_var (table, rel->wv);
+
+ if (rel->model == MODEL_ALPHA)
+ {
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN,
+ N_("Statistics"),
+ N_("Cronbach's Alpha"), PIVOT_RC_OTHER,
+ N_("N of Items"), PIVOT_RC_COUNT);
+
+ const struct cronbach *s = &rel->sc[0];
+ pivot_table_put1 (table, 0, pivot_value_new_number (s->alpha));
+ pivot_table_put1 (table, 1, pivot_value_new_number (s->n_items));
+ }
+ else
+ {
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"));
+ struct pivot_category *alpha = pivot_category_create_group (
+ statistics->root, N_("Cronbach's Alpha"));
+ pivot_category_create_group (alpha, N_("Part 1"),
+ N_("Value"), PIVOT_RC_OTHER,
+ N_("N of Items"), PIVOT_RC_COUNT);
+ pivot_category_create_group (alpha, N_("Part 2"),
+ N_("Value"), PIVOT_RC_OTHER,
+ N_("N of Items"), PIVOT_RC_COUNT);
+ pivot_category_create_leaves (alpha,
+ N_("Total N of Items"), PIVOT_RC_COUNT);
+ pivot_category_create_leaves (statistics->root,
+ N_("Correlation Between Forms"),
+ PIVOT_RC_OTHER);
+ pivot_category_create_group (statistics->root,
+ N_("Spearman-Brown Coefficient"),
+ N_("Equal Length"), PIVOT_RC_OTHER,
+ N_("Unequal Length"), PIVOT_RC_OTHER);
+ pivot_category_create_leaves (statistics->root,
+ N_("Guttman Split-Half Coefficient"),
+ PIVOT_RC_OTHER);
+
+ /* R is the correlation between the two parts */
+ double r0 = rel->sc[0].variance_of_sums -
+ rel->sc[1].variance_of_sums -
+ rel->sc[2].variance_of_sums;
+ double r1 = (r0 / sqrt (rel->sc[1].variance_of_sums)
+ / sqrt (rel->sc[2].variance_of_sums)
+ / 2.0);
+
+ /* Guttman Split Half Coefficient */
+ double g = 2 * r0 / rel->sc[0].variance_of_sums;
+
+ double tmp = (1.0 - r1*r1) * rel->sc[1].n_items * rel->sc[2].n_items /
+ pow2 (rel->sc[0].n_items);
+
+ double entries[] = {
+ rel->sc[1].alpha,
+ rel->sc[1].n_items,
+ rel->sc[2].alpha,
+ rel->sc[2].n_items,
+ rel->sc[1].n_items + rel->sc[2].n_items,
+ r1,
+ 2 * r1 / (1.0 + r1),
+ (sqrt (pow4 (r1) + 4 * pow2 (r1) * tmp) - pow2 (r1)) / (2 * tmp),
+ g,
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011, 2015, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_rename_variables (struct lexer *lexer, struct dataset *ds)
+{
+ struct variable **vars_to_be_renamed = NULL;
+ size_t n_vars_to_be_renamed = 0;
+
+ char **new_names = NULL;
+ size_t n_new_names = 0;
+
+ char *err_name;
+
+ int status = CMD_CASCADING_FAILURE;
+
+ if (proc_make_temporary_transformations_permanent (ds))
+ lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
+ _("%s may not be used after %s. "
+ "Temporary transformations will be made permanent."),
+ "RENAME VARS", "TEMPORARY");
+
+ do
+ {
+ int opts = PV_APPEND | PV_NO_DUPLICATE;
+
+ if (!lex_match (lexer, T_LPAREN))
+ opts |= PV_SINGLE;
+ int start_ofs = lex_ofs (lexer);
+ if (!parse_variables (lexer, dataset_dict (ds),
+ &vars_to_be_renamed, &n_vars_to_be_renamed, opts))
+ {
+ goto lossage;
+ }
+ if (!lex_force_match (lexer, T_EQUALS))
+ {
+ goto lossage;
+ }
+ if (!parse_DATA_LIST_vars (lexer, dataset_dict (ds),
+ &new_names, &n_new_names, opts))
+ {
+ goto lossage;
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+ if (n_new_names != n_vars_to_be_renamed)
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("Differing number of variables in old name list "
+ "(%zu) and in new name list (%zu)."),
+ n_vars_to_be_renamed, n_new_names);
+ goto lossage;
+ }
+ if (!(opts & PV_SINGLE) && !lex_force_match (lexer, T_RPAREN))
+ {
+ goto lossage;
+ }
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ if (!dict_rename_vars (dataset_dict (ds),
+ vars_to_be_renamed, new_names, n_new_names,
+ &err_name))
+ {
+ lex_ofs_error (lexer, 2, lex_ofs (lexer) - 1,
+ _("Renaming would duplicate variable name %s."), err_name);
+ goto lossage;
+ }
+
+ status = CMD_SUCCESS;
+
+ lossage:
+ free (vars_to_be_renamed);
+ if (new_names != NULL)
+ {
+ size_t i;
+ for (i = 0; i < n_new_names; ++i)
+ free (new_names[i]);
+ free (new_names);
+ }
+ return status;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2007, 2009-2012 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/settings.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/segment.h"
+#include "language/lexer/token.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/cast.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmap.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "libpspp/misc.h"
+#include "output/output-item.h"
+
+#include "gl/ftoastr.h"
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+#include "gl/xmemdup0.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+struct dummy_var
+ {
+ struct hmap_node hmap_node;
+ struct substring name;
+ char **values;
+ size_t n_values;
+ int start_ofs, end_ofs;
+ };
+
+static bool parse_specification (struct lexer *, struct dictionary *,
+ struct hmap *dummies);
+static bool parse_commands (struct lexer *, struct hmap *dummies);
+static void destroy_dummies (struct hmap *dummies);
+
+static bool parse_ids (struct lexer *, const struct dictionary *,
+ struct dummy_var *);
+static bool parse_numbers (struct lexer *, struct dummy_var *);
+static bool parse_strings (struct lexer *, struct dummy_var *);
+
+int
+cmd_do_repeat (struct lexer *lexer, struct dataset *ds)
+{
+ struct hmap dummies = HMAP_INITIALIZER (dummies);
+ bool ok = parse_specification (lexer, dataset_dict (ds), &dummies);
+ ok = parse_commands (lexer, &dummies) && ok;
+ destroy_dummies (&dummies);
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+}
+
+static const struct dummy_var *
+find_dummy_var (struct hmap *hmap, struct substring name)
+{
+ const struct dummy_var *dv;
+
+ HMAP_FOR_EACH_WITH_HASH (dv, struct dummy_var, hmap_node,
+ utf8_hash_case_substring (name, 0), hmap)
+ if (!utf8_sscasecmp (dv->name, name))
+ return dv;
+
+ return NULL;
+}
+
+/* Parses the whole DO REPEAT command specification.
+ Returns success. */
+static bool
+parse_specification (struct lexer *lexer, struct dictionary *dict,
+ struct hmap *dummies)
+{
+ struct dummy_var *first_dv = NULL;
+
+ do
+ {
+ int start_ofs = lex_ofs (lexer);
+
+ /* Get a stand-in variable name and make sure it's unique. */
+ if (!lex_force_id (lexer))
+ goto error;
+ struct substring name = lex_tokss (lexer);
+ if (dict_lookup_var (dict, name.string))
+ lex_msg (lexer, SW,
+ _("Dummy variable name `%s' hides dictionary variable `%s'."),
+ name.string, name.string);
+ if (find_dummy_var (dummies, name))
+ {
+ lex_error (lexer, _("Dummy variable name `%s' is given twice."),
+ name.string);
+ goto error;
+ }
+
+ /* Make a new macro. */
+ struct dummy_var *dv = xmalloc (sizeof *dv);
+ *dv = (struct dummy_var) {
+ .name = ss_clone (name),
+ .start_ofs = start_ofs,
+ };
+ hmap_insert (dummies, &dv->hmap_node, utf8_hash_case_substring (name, 0));
+
+ /* Skip equals sign. */
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto error;
+
+ /* Get the details of the variable's possible values. */
+ bool ok;
+ if (lex_token (lexer) == T_ID || lex_token (lexer) == T_ALL)
+ ok = parse_ids (lexer, dict, dv);
+ else if (lex_is_number (lexer))
+ ok = parse_numbers (lexer, dv);
+ else if (lex_is_string (lexer))
+ ok = parse_strings (lexer, dv);
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting substitution values."));
+ goto error;
+ }
+ if (!ok)
+ goto error;
+ assert (dv->n_values > 0);
+ if (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ lex_error (lexer, _("Syntax error expecting `/' or end of command."));
+ goto error;
+ }
+ dv->end_ofs = lex_ofs (lexer) - 1;
+
+ /* If this is the first variable then it defines how many replacements
+ there must be; otherwise enforce this number of replacements. */
+ if (first_dv == NULL)
+ first_dv = dv;
+ else if (first_dv->n_values != dv->n_values)
+ {
+ msg (SE, _("Each dummy variable must have the same number of "
+ "substitutions."));
+
+ lex_ofs_msg (lexer, SN, first_dv->start_ofs, first_dv->end_ofs,
+ ngettext ("Dummy variable %s had %zu substitution.",
+ "Dummy variable %s had %zu substitutions.",
+ first_dv->n_values),
+ first_dv->name.string, first_dv->n_values);
+ lex_ofs_msg (lexer, SN, dv->start_ofs, dv->end_ofs,
+ ngettext ("Dummy variable %s had %zu substitution.",
+ "Dummy variable %s had %zu substitutions.",
+ dv->n_values),
+ dv->name.string, dv->n_values);
+ goto error;
+ }
+
+ lex_match (lexer, T_SLASH);
+ }
+ while (!lex_match (lexer, T_ENDCMD));
+
+ while (lex_match (lexer, T_ENDCMD))
+ continue;
+
+ return true;
+
+error:
+ lex_discard_rest_of_command (lexer);
+ while (lex_match (lexer, T_ENDCMD))
+ continue;
+ destroy_dummies (dummies);
+ hmap_init (dummies);
+ return false;
+}
+
+static size_t
+count_values (struct hmap *dummies)
+{
+ if (hmap_is_empty (dummies))
+ return 0;
+ const struct dummy_var *dv = HMAP_FIRST (struct dummy_var, hmap_node, dummies);
+ return dv->n_values;
+}
+
+static void
+do_parse_commands (struct substring s, enum segmenter_mode mode,
+ struct hmap *dummies,
+ struct string *outputs, size_t n_outputs)
+{
+ struct segmenter segmenter = segmenter_init (mode, false);
+ while (!ss_is_empty (s))
+ {
+ enum segment_type type;
+ int n = segmenter_push (&segmenter, s.string, s.length, true, &type);
+ assert (n >= 0);
+
+ if (type == SEG_DO_REPEAT_COMMAND)
+ {
+ for (;;)
+ {
+ int k = segmenter_push (&segmenter, s.string + n, s.length - n,
+ true, &type);
+ if (type != SEG_NEWLINE && type != SEG_DO_REPEAT_COMMAND)
+ break;
+
+ n += k;
+ }
+
+ do_parse_commands (ss_head (s, n), mode, dummies,
+ outputs, n_outputs);
+ }
+ else if (type != SEG_END)
+ {
+ const struct dummy_var *dv
+ = (type == SEG_IDENTIFIER ? find_dummy_var (dummies, ss_head (s, n))
+ : NULL);
+ for (size_t i = 0; i < n_outputs; i++)
+ if (dv != NULL)
+ ds_put_cstr (&outputs[i], dv->values[i]);
+ else
+ ds_put_substring (&outputs[i], ss_head (s, n));
+ }
+
+ ss_advance (&s, n);
+ }
+}
+
+static bool
+parse_commands (struct lexer *lexer, struct hmap *dummies)
+{
+ char *file_name = xstrdup_if_nonnull (lex_get_file_name (lexer));
+ int line_number = lex_ofs_start_point (lexer, lex_ofs (lexer)).line;
+
+ struct string input = DS_EMPTY_INITIALIZER;
+ while (lex_is_string (lexer))
+ {
+ ds_put_substring (&input, lex_tokss (lexer));
+ ds_put_byte (&input, '\n');
+ lex_get (lexer);
+ }
+
+ size_t n_values = count_values (dummies);
+ struct string *outputs = xmalloc (n_values * sizeof *outputs);
+ for (size_t i = 0; i < n_values; i++)
+ ds_init_empty (&outputs[i]);
+
+ do_parse_commands (ds_ss (&input), lex_get_syntax_mode (lexer),
+ dummies, outputs, n_values);
+
+ ds_destroy (&input);
+
+ while (lex_match (lexer, T_ENDCMD))
+ continue;
+
+ bool ok = lex_match_phrase (lexer, "END REPEAT");
+ if (!ok)
+ lex_error (lexer, _("Syntax error expecting END REPEAT."));
+ bool print = ok && lex_match_id (lexer, "PRINT");
+ lex_discard_rest_of_command (lexer);
+
+ for (size_t i = 0; i < n_values; i++)
+ {
+ struct string *output = &outputs[n_values - i - 1];
+ if (print)
+ {
+ struct substring s = output->ss;
+ ss_chomp_byte (&s, '\n');
+ char *label = xasprintf (_("Expansion %zu of %zu"), i + 1, n_values);
+ output_item_submit (
+ text_item_create_nocopy (TEXT_ITEM_LOG, ss_xstrdup (s), label));
+ }
+ const char *encoding = lex_get_encoding (lexer);
+ struct lex_reader *reader = lex_reader_for_substring_nocopy (ds_ss (output), encoding);
+ lex_reader_set_file_name (reader, file_name);
+ reader->line_number = line_number;
+ lex_include (lexer, reader);
+ }
+ free (file_name);
+ free (outputs);
+
+ return ok;
+}
+
+static void
+destroy_dummies (struct hmap *dummies)
+{
+ struct dummy_var *dv, *next;
+
+ HMAP_FOR_EACH_SAFE (dv, next, struct dummy_var, hmap_node, dummies)
+ {
+ hmap_delete (dummies, &dv->hmap_node);
+
+ ss_dealloc (&dv->name);
+ for (size_t i = 0; i < dv->n_values; i++)
+ free (dv->values[i]);
+ free (dv->values);
+ free (dv);
+ }
+ hmap_destroy (dummies);
+}
+
+/* Parses a set of ids for DO REPEAT. */
+static bool
+parse_ids (struct lexer *lexer, const struct dictionary *dict,
+ struct dummy_var *dv)
+{
+ return parse_mixed_vars (lexer, dict, &dv->values, &dv->n_values, PV_NONE);
+}
+
+/* Adds REPLACEMENT to MACRO's list of replacements, which has
+ *USED elements and has room for *ALLOCATED. Allocates memory
+ from POOL. */
+static void
+add_replacement (struct dummy_var *dv, char *value, size_t *allocated)
+{
+ if (dv->n_values == *allocated)
+ dv->values = x2nrealloc (dv->values, allocated, sizeof *dv->values);
+ dv->values[dv->n_values++] = value;
+}
+
+/* Parses a list or range of numbers for DO REPEAT. */
+static bool
+parse_numbers (struct lexer *lexer, struct dummy_var *dv)
+{
+ size_t allocated = 0;
+
+ do
+ {
+ if (!lex_force_num (lexer))
+ return false;
+
+ if (lex_next_token (lexer, 1) == T_TO)
+ {
+ if (!lex_is_integer (lexer))
+ {
+ lex_error (lexer, _("Ranges may only have integer bounds."));
+ return false;
+ }
+
+ long a = lex_integer (lexer);
+ lex_get (lexer);
+ lex_get (lexer);
+
+ if (!lex_force_int_range (lexer, NULL, a, LONG_MAX))
+ return false;
+
+ long b = lex_integer (lexer);
+ if (b < a)
+ {
+ lex_next_error (lexer, -2, 0,
+ _("%ld TO %ld is an invalid range."), a, b);
+ return false;
+ }
+ lex_get (lexer);
+
+ for (long i = a; i <= b; i++)
+ add_replacement (dv, xasprintf ("%ld", i), &allocated);
+ }
+ else
+ {
+ char s[DBL_BUFSIZE_BOUND];
+
+ c_dtoastr (s, sizeof s, 0, 0, lex_number (lexer));
+ add_replacement (dv, xstrdup (s), &allocated);
+ lex_get (lexer);
+ }
+
+ lex_match (lexer, T_COMMA);
+ }
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
+
+ return true;
+}
+
+/* Parses a list of strings for DO REPEAT. */
+static bool
+parse_strings (struct lexer *lexer, struct dummy_var *dv)
+{
+ size_t allocated = 0;
+
+ do
+ {
+ if (!lex_force_string (lexer))
+ return false;
+
+ add_replacement (dv, token_to_string (lex_next (lexer, 0)), &allocated);
+
+ lex_get (lexer);
+ lex_match (lexer, T_COMMA);
+ }
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
+
+ return true;
+}
+\f
+int
+cmd_end_repeat (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ lex_ofs_error (lexer, 0, 1, _("No matching %s."), "DO REPEAT");
+ return CMD_CASCADING_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/roc.h"
+
+#include <gsl/gsl_cdf.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/misc.h"
+#include "math/sort.h"
+#include "output/charts/roc-chart.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) msgid
+
+struct cmd_roc
+{
+ size_t n_vars;
+ const struct variable **vars;
+ const struct dictionary *dict;
+
+ const struct variable *state_var;
+ union value state_value;
+ size_t state_var_width;
+
+ /* Plot the roc curve */
+ bool curve;
+ /* Plot the reference line */
+ bool reference;
+
+ double ci;
+
+ bool print_coords;
+ bool print_se;
+ bool bi_neg_exp; /* True iff the bi-negative exponential critieria
+ should be used */
+ enum mv_class exclude;
+
+ bool invert; /* True iff a smaller test result variable indicates
+ a positive result */
+
+ double pos;
+ double neg;
+ double pos_weighted;
+ double neg_weighted;
+};
+
+static int run_roc (struct dataset *, struct cmd_roc *);
+static void do_roc (struct cmd_roc *, struct casereader *, struct dictionary *);
+
+
+int
+cmd_roc (struct lexer *lexer, struct dataset *ds)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+
+ struct cmd_roc roc = {
+ .exclude = MV_ANY,
+ .curve = true,
+ .ci = 95,
+ .dict = dict,
+ .state_var_width = -1,
+ };
+
+ lex_match (lexer, T_SLASH);
+ if (!parse_variables_const (lexer, dict, &roc.vars, &roc.n_vars,
+ PV_APPEND | PV_NO_DUPLICATE | PV_NUMERIC))
+ goto error;
+
+ if (!lex_force_match (lexer, T_BY))
+ goto error;
+
+ roc.state_var = parse_variable (lexer, dict);
+ if (!roc.state_var)
+ goto error;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ roc.state_var_width = var_get_width (roc.state_var);
+ value_init (&roc.state_value, roc.state_var_width);
+ if (!parse_value (lexer, &roc.state_value, roc.state_var)
+ || !lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ roc.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ roc.exclude = MV_ANY;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "PLOT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "CURVE"))
+ {
+ roc.curve = true;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ roc.reference = true;
+ if (!lex_force_match_id (lexer, "REFERENCE")
+ || !lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "NONE"))
+ roc.curve = false;
+ else
+ {
+ lex_error_expecting (lexer, "CURVE", "NONE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "PRINT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "SE"))
+ roc.print_se = true;
+ else if (lex_match_id (lexer, "COORDINATES"))
+ roc.print_coords = true;
+ else
+ {
+ lex_error_expecting (lexer, "SE", "COORDINATES");
+ goto error;
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "CUTOFF"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+ if (lex_match_id (lexer, "INCLUDE"))
+ roc.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ roc.exclude = MV_USER | MV_SYSTEM;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "TESTPOS"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+ if (lex_match_id (lexer, "LARGE"))
+ roc.invert = false;
+ else if (lex_match_id (lexer, "SMALL"))
+ roc.invert = true;
+ else
+ {
+ lex_error_expecting (lexer, "LARGE", "SMALL");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "CI"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+ if (!lex_force_num (lexer))
+ goto error;
+ roc.ci = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else if (lex_match_id (lexer, "DISTRIBUTION"))
+ {
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+ if (lex_match_id (lexer, "FREE"))
+ roc.bi_neg_exp = false;
+ else if (lex_match_id (lexer, "NEGEXPO"))
+ roc.bi_neg_exp = true;
+ else
+ {
+ lex_error_expecting (lexer, "FREE", "NEGEXPO");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "CUTOFF", "TESTPOS", "CI",
+ "DISTRIBUTION");
+ goto error;
+ }
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "MISSING", "PLOT", "PRINT", "CRITERIA");
+ goto error;
+ }
+ }
+
+ if (!run_roc (ds, &roc))
+ goto error;
+
+ if (roc.state_var)
+ value_destroy (&roc.state_value, roc.state_var_width);
+ free (roc.vars);
+ return CMD_SUCCESS;
+
+ error:
+ if (roc.state_var)
+ value_destroy (&roc.state_value, roc.state_var_width);
+ free (roc.vars);
+ return CMD_FAILURE;
+}
+
+static int
+run_roc (struct dataset *ds, struct cmd_roc *roc)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct casereader *group;
+
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
+ while (casegrouper_get_next_group (grouper, &group))
+ do_roc (roc, group, dataset_dict (ds));
+
+ bool ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+ return ok;
+}
+
+#if 0
+static void
+dump_casereader (struct casereader *reader)
+{
+ struct ccase *c;
+ struct casereader *r = casereader_clone (reader);
+
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ for (size_t i = 0; i < case_get_n_values (c); ++i)
+ printf ("%g ", case_num_idx (c, i));
+ printf ("\n");
+ }
+
+ casereader_destroy (r);
+}
+#endif
+
+
+/*
+ Return true iff the state variable indicates that C has positive actual state.
+
+ As a side effect, this function also accumulates the roc->{pos,neg} and
+ roc->{pos,neg}_weighted counts.
+ */
+static bool
+match_positives (const struct ccase *c, void *aux)
+{
+ struct cmd_roc *roc = aux;
+ const struct variable *wv = dict_get_weight (roc->dict);
+ const double weight = wv ? case_num (c, wv) : 1.0;
+
+ const bool positive =
+ (0 == value_compare_3way (case_data (c, roc->state_var), &roc->state_value,
+ var_get_width (roc->state_var)));
+
+ if (positive)
+ {
+ roc->pos++;
+ roc->pos_weighted += weight;
+ }
+ else
+ {
+ roc->neg++;
+ roc->neg_weighted += weight;
+ }
+
+ return positive;
+}
+
+
+#define VALUE 0
+#define N_EQ 1
+#define N_PRED 2
+
+/* Some intermediate state for calculating the cutpoints and the
+ standard error values */
+struct roc_state
+{
+ double auc; /* Area under the curve */
+
+ double n1; /* total weight of positives */
+ double n2; /* total weight of negatives */
+
+ /* intermediates for standard error */
+ double q1hat;
+ double q2hat;
+
+ /* intermediates for cutpoints */
+ struct casewriter *cutpoint_wtr;
+ struct casereader *cutpoint_rdr;
+ double prev_result;
+ double min;
+ double max;
+};
+
+/*
+ Return a new casereader based upon CUTPOINT_RDR.
+ The number of "positive" cases are placed into
+ the position TRUE_INDEX, and the number of "negative" cases
+ into FALSE_INDEX.
+ POS_COND and RESULT determine the semantics of what is
+ "positive".
+ WEIGHT is the value of a single count.
+ */
+static struct casereader *
+accumulate_counts (struct casereader *input,
+ double result, double weight,
+ bool (*pos_cond) (double, double),
+ int true_index, int false_index)
+{
+ const struct caseproto *proto = casereader_get_proto (input);
+ struct casewriter *w =
+ autopaging_writer_create (proto);
+ struct ccase *cpc;
+ double prev_cp = SYSMIS;
+
+ for (; (cpc = casereader_read (input)); case_unref (cpc))
+ {
+ struct ccase *new_case;
+ const double cp = case_num_idx (cpc, ROC_CUTPOINT);
+
+ assert (cp != SYSMIS);
+
+ /* We don't want duplicates here */
+ if (cp == prev_cp)
+ continue;
+
+ new_case = case_clone (cpc);
+
+ int index = pos_cond (result, cp) ? true_index : false_index;
+ *case_num_rw_idx (new_case, index) += weight;
+
+ prev_cp = cp;
+
+ casewriter_write (w, new_case);
+ }
+ casereader_destroy (input);
+
+ return casewriter_make_reader (w);
+}
+
+
+
+static void output_roc (struct roc_state *rs, const struct cmd_roc *roc);
+
+/*
+ This function does 3 things:
+
+ 1. Counts the number of cases which are equal to every other case in READER,
+ and those cases for which the relationship between it and every other case
+ satifies PRED (normally either > or <). VAR is variable defining a case's value
+ for this purpose.
+
+ 2. Counts the number of true and false cases in reader, and populates
+ CUTPOINT_RDR accordingly. TRUE_INDEX and FALSE_INDEX are the indices
+ which receive these values. POS_COND is the condition defining true
+ and false.
+
+ 3. CC is filled with the cumulative weight of all cases of READER.
+*/
+static struct casereader *
+process_group (const struct variable *var, struct casereader *reader,
+ bool (*pred) (double, double),
+ const struct dictionary *dict,
+ double *cc,
+ struct casereader **cutpoint_rdr,
+ bool (*pos_cond) (double, double),
+ int true_index,
+ int false_index)
+{
+ const struct variable *w = dict_get_weight (dict);
+
+ struct casereader *r1 =
+ casereader_create_distinct (sort_execute_1var (reader, var), var, w);
+
+ const int weight_idx = w ? var_get_case_index (w) :
+ caseproto_get_n_widths (casereader_get_proto (r1)) - 1;
+
+ struct ccase *c1;
+
+ struct casereader *rclone = casereader_clone (r1);
+ struct casewriter *wtr;
+ struct caseproto *proto = caseproto_create ();
+
+ proto = caseproto_add_width (proto, 0);
+ proto = caseproto_add_width (proto, 0);
+ proto = caseproto_add_width (proto, 0);
+
+ wtr = autopaging_writer_create (proto);
+
+ *cc = 0;
+
+ for (; (c1 = casereader_read (r1)); case_unref (c1))
+ {
+ struct ccase *new_case = case_create (proto);
+ struct ccase *c2;
+ struct casereader *r2 = casereader_clone (rclone);
+
+ const double weight1 = case_num_idx (c1, weight_idx);
+ const double d1 = case_num (c1, var);
+ double n_eq = 0.0;
+ double n_pred = 0.0;
+
+ *cutpoint_rdr = accumulate_counts (*cutpoint_rdr, d1, weight1,
+ pos_cond,
+ true_index, false_index);
+
+ *cc += weight1;
+
+ for (; (c2 = casereader_read (r2)); case_unref (c2))
+ {
+ const double d2 = case_num (c2, var);
+ const double weight2 = case_num_idx (c2, weight_idx);
+
+ if (d1 == d2)
+ {
+ n_eq += weight2;
+ continue;
+ }
+ else if (pred (d2, d1))
+ {
+ n_pred += weight2;
+ }
+ }
+
+ *case_num_rw_idx (new_case, VALUE) = d1;
+ *case_num_rw_idx (new_case, N_EQ) = n_eq;
+ *case_num_rw_idx (new_case, N_PRED) = n_pred;
+
+ casewriter_write (wtr, new_case);
+
+ casereader_destroy (r2);
+ }
+
+ casereader_destroy (r1);
+ casereader_destroy (rclone);
+
+ caseproto_unref (proto);
+
+ return casewriter_make_reader (wtr);
+}
+
+/* Some more indeces into case data */
+#define N_POS_EQ 1 /* number of positive cases with values equal to n */
+#define N_POS_GT 2 /* number of positive cases with values greater than n */
+#define N_NEG_EQ 3 /* number of negative cases with values equal to n */
+#define N_NEG_LT 4 /* number of negative cases with values less than n */
+
+static bool
+gt (double d1, double d2)
+{
+ return d1 > d2;
+}
+
+
+static bool
+ge (double d1, double d2)
+{
+ return d1 > d2;
+}
+
+static bool
+lt (double d1, double d2)
+{
+ return d1 < d2;
+}
+
+
+/*
+ Return a casereader with width 3,
+ populated with cases based upon READER.
+ The cases will have the values:
+ (N, number of cases equal to N, number of cases greater than N)
+ As a side effect, update RS->n1 with the number of positive cases.
+*/
+static struct casereader *
+process_positive_group (const struct variable *var, struct casereader *reader,
+ const struct dictionary *dict,
+ struct roc_state *rs)
+{
+ return process_group (var, reader, gt, dict, &rs->n1,
+ &rs->cutpoint_rdr,
+ ge,
+ ROC_TP, ROC_FN);
+}
+
+/*
+ Return a casereader with width 3,
+ populated with cases based upon READER.
+ The cases will have the values:
+ (N, number of cases equal to N, number of cases less than N)
+ As a side effect, update RS->n2 with the number of negative cases.
+*/
+static struct casereader *
+process_negative_group (const struct variable *var, struct casereader *reader,
+ const struct dictionary *dict,
+ struct roc_state *rs)
+{
+ return process_group (var, reader, lt, dict, &rs->n2,
+ &rs->cutpoint_rdr,
+ lt,
+ ROC_TN, ROC_FP);
+}
+
+
+
+
+static void
+append_cutpoint (struct casewriter *writer, double cutpoint)
+{
+ struct ccase *cc = case_create (casewriter_get_proto (writer));
+
+ *case_num_rw_idx (cc, ROC_CUTPOINT) = cutpoint;
+ *case_num_rw_idx (cc, ROC_TP) = 0;
+ *case_num_rw_idx (cc, ROC_FN) = 0;
+ *case_num_rw_idx (cc, ROC_TN) = 0;
+ *case_num_rw_idx (cc, ROC_FP) = 0;
+
+ casewriter_write (writer, cc);
+}
+
+/*
+ Create and initialise the rs[x].cutpoint_rdr casereaders. That is, the
+ readers will be created with width 5, ready to take the values (cutpoint,
+ ROC_TP, ROC_FN, ROC_TN, ROC_FP), and the reader will be populated with its
+ final number of cases. However on exit from this function, only
+ ROC_CUTPOINT entries will be set to their final value. The other entries
+ will be initialised to zero.
+*/
+static struct roc_state *
+prepare_cutpoints (struct cmd_roc *roc, struct casereader *input)
+{
+ struct casereader *r = casereader_clone (input);
+ struct ccase *c;
+
+ struct subcase ordering;
+ subcase_init (&ordering, ROC_CUTPOINT, 0, SC_ASCEND);
+
+ struct caseproto *proto = caseproto_create ();
+ proto = caseproto_add_width (proto, 0); /* cutpoint */
+ proto = caseproto_add_width (proto, 0); /* ROC_TP */
+ proto = caseproto_add_width (proto, 0); /* ROC_FN */
+ proto = caseproto_add_width (proto, 0); /* ROC_TN */
+ proto = caseproto_add_width (proto, 0); /* ROC_FP */
+
+ struct roc_state *rs = xnmalloc (roc->n_vars, sizeof *rs);
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ rs[i] = (struct roc_state) {
+ .cutpoint_wtr = sort_create_writer (&ordering, proto),
+ .prev_result = SYSMIS,
+ .max = -DBL_MAX,
+ .min = DBL_MAX,
+ };
+
+ caseproto_unref (proto);
+ subcase_uninit (&ordering);
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ {
+ const union value *v = case_data (c, roc->vars[i]);
+ const double result = v->f;
+
+ if (mv_is_value_missing (var_get_missing_values (roc->vars[i]), v)
+ & roc->exclude)
+ continue;
+
+ minimize (&rs[i].min, result);
+ maximize (&rs[i].max, result);
+
+ if (rs[i].prev_result != SYSMIS && rs[i].prev_result != result)
+ {
+ const double mean = (result + rs[i].prev_result) / 2.0;
+ append_cutpoint (rs[i].cutpoint_wtr, mean);
+ }
+
+ rs[i].prev_result = result;
+ }
+ casereader_destroy (r);
+
+ /* Append the min and max cutpoints */
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ {
+ append_cutpoint (rs[i].cutpoint_wtr, rs[i].min - 1);
+ append_cutpoint (rs[i].cutpoint_wtr, rs[i].max + 1);
+
+ rs[i].cutpoint_rdr = casewriter_make_reader (rs[i].cutpoint_wtr);
+ }
+
+ return rs;
+}
+
+static void
+do_roc (struct cmd_roc *roc, struct casereader *reader, struct dictionary *dict)
+{
+ struct casereader *input = casereader_create_filter_missing (
+ reader, roc->vars, roc->n_vars, roc->exclude, NULL, NULL);
+ input = casereader_create_filter_missing (
+ input, &roc->state_var, 1, roc->exclude, NULL, NULL);
+
+ struct roc_state *rs = prepare_cutpoints (roc, input);
+
+ /* Separate the positive actual state cases from the negative ones */
+ struct casewriter *neg_wtr
+ = autopaging_writer_create (casereader_get_proto (input));
+ struct casereader *positives = casereader_create_filter_func (
+ input, match_positives, NULL, roc, neg_wtr);
+
+ struct caseproto *n_proto = caseproto_create ();
+ for (size_t i = 0; i < 5; i++)
+ n_proto = caseproto_add_width (n_proto, 0);
+
+ struct subcase up_ordering;
+ struct subcase down_ordering;
+ subcase_init (&up_ordering, VALUE, 0, SC_ASCEND);
+ subcase_init (&down_ordering, VALUE, 0, SC_DESCEND);
+
+ struct casereader *negatives = NULL;
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ {
+ const struct variable *var = roc->vars[i];
+
+ struct casereader *pos = casereader_clone (positives);
+
+ struct casereader *n_pos_reader =
+ process_positive_group (var, pos, dict, &rs[i]);
+
+ if (!negatives)
+ negatives = casewriter_make_reader (neg_wtr);
+
+ struct casereader *neg = casereader_clone (negatives);
+ struct casereader *n_neg_reader
+ = process_negative_group (var, neg, dict, &rs[i]);
+
+ /* Merge the n_pos and n_neg casereaders */
+ struct casewriter *w = sort_create_writer (&up_ordering, n_proto);
+ struct ccase *cpos;
+ for (; (cpos = casereader_read (n_pos_reader)); case_unref (cpos))
+ {
+ struct ccase *pos_case = case_create (n_proto);
+ const double jpos = case_num_idx (cpos, VALUE);
+
+ struct ccase *cneg;
+ while ((cneg = casereader_read (n_neg_reader)))
+ {
+ struct ccase *nc = case_create (n_proto);
+
+ const double jneg = case_num_idx (cneg, VALUE);
+
+ *case_num_rw_idx (nc, VALUE) = jneg;
+ *case_num_rw_idx (nc, N_POS_EQ) = 0;
+
+ *case_num_rw_idx (nc, N_POS_GT) = SYSMIS;
+
+ *case_data_rw_idx (nc, N_NEG_EQ) = *case_data_idx (cneg, N_EQ);
+ *case_data_rw_idx (nc, N_NEG_LT) = *case_data_idx (cneg, N_PRED);
+
+ casewriter_write (w, nc);
+
+ case_unref (cneg);
+ if (jneg > jpos)
+ break;
+ }
+
+ *case_num_rw_idx (pos_case, VALUE) = jpos;
+ *case_data_rw_idx (pos_case, N_POS_EQ) = *case_data_idx (cpos, N_EQ);
+ *case_data_rw_idx (pos_case, N_POS_GT) = *case_data_idx (cpos, N_PRED);
+ *case_num_rw_idx (pos_case, N_NEG_EQ) = 0;
+ *case_num_rw_idx (pos_case, N_NEG_LT) = SYSMIS;
+
+ casewriter_write (w, pos_case);
+ }
+
+ casereader_destroy (n_pos_reader);
+ casereader_destroy (n_neg_reader);
+
+ struct casereader *r = casewriter_make_reader (w);
+
+ /* Propagate the N_POS_GT values from the positive cases
+ to the negative ones */
+ double prev_pos_gt = rs[i].n1;
+ w = sort_create_writer (&down_ordering, n_proto);
+
+ struct ccase *c;
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double n_pos_gt = case_num_idx (c, N_POS_GT);
+ struct ccase *nc = case_clone (c);
+
+ if (n_pos_gt == SYSMIS)
+ {
+ n_pos_gt = prev_pos_gt;
+ *case_num_rw_idx (nc, N_POS_GT) = n_pos_gt;
+ }
+
+ casewriter_write (w, nc);
+ prev_pos_gt = n_pos_gt;
+ }
+ casereader_destroy (r);
+ r = casewriter_make_reader (w);
+
+ /* Propagate the N_NEG_LT values from the negative cases
+ to the positive ones */
+ double prev_neg_lt = rs[i].n2;
+ w = sort_create_writer (&up_ordering, n_proto);
+
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double n_neg_lt = case_num_idx (c, N_NEG_LT);
+ struct ccase *nc = case_clone (c);
+
+ if (n_neg_lt == SYSMIS)
+ {
+ n_neg_lt = prev_neg_lt;
+ *case_num_rw_idx (nc, N_NEG_LT) = n_neg_lt;
+ }
+
+ casewriter_write (w, nc);
+ prev_neg_lt = n_neg_lt;
+ }
+
+ casereader_destroy (r);
+ r = casewriter_make_reader (w);
+
+ struct ccase *prev_case = NULL;
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ struct ccase *next_case = casereader_peek (r, 0);
+
+ const double j = case_num_idx (c, VALUE);
+ double n_pos_eq = case_num_idx (c, N_POS_EQ);
+ double n_pos_gt = case_num_idx (c, N_POS_GT);
+ double n_neg_eq = case_num_idx (c, N_NEG_EQ);
+ double n_neg_lt = case_num_idx (c, N_NEG_LT);
+
+ if (prev_case && j == case_num_idx (prev_case, VALUE))
+ {
+ if (0 == case_num_idx (c, N_POS_EQ))
+ {
+ n_pos_eq = case_num_idx (prev_case, N_POS_EQ);
+ n_pos_gt = case_num_idx (prev_case, N_POS_GT);
+ }
+
+ if (0 == case_num_idx (c, N_NEG_EQ))
+ {
+ n_neg_eq = case_num_idx (prev_case, N_NEG_EQ);
+ n_neg_lt = case_num_idx (prev_case, N_NEG_LT);
+ }
+ }
+
+ if (NULL == next_case || j != case_num_idx (next_case, VALUE))
+ {
+ rs[i].auc += n_pos_gt * n_neg_eq + (n_pos_eq * n_neg_eq) / 2.0;
+
+ rs[i].q1hat +=
+ n_neg_eq * (pow2 (n_pos_gt) + n_pos_gt * n_pos_eq + pow2 (n_pos_eq) / 3.0);
+ rs[i].q2hat +=
+ n_pos_eq * (pow2 (n_neg_lt) + n_neg_lt * n_neg_eq + pow2 (n_neg_eq) / 3.0);
+
+ }
+
+ case_unref (next_case);
+ case_unref (prev_case);
+ prev_case = case_clone (c);
+ }
+ casereader_destroy (r);
+ case_unref (prev_case);
+
+ rs[i].auc /= rs[i].n1 * rs[i].n2;
+ if (roc->invert)
+ rs[i].auc = 1 - rs[i].auc;
+
+ if (roc->bi_neg_exp)
+ {
+ rs[i].q1hat = rs[i].auc / (2 - rs[i].auc);
+ rs[i].q2hat = 2 * pow2 (rs[i].auc) / (1 + rs[i].auc);
+ }
+ else
+ {
+ rs[i].q1hat /= rs[i].n2 * pow2 (rs[i].n1);
+ rs[i].q2hat /= rs[i].n1 * pow2 (rs[i].n2);
+ }
+ }
+
+ casereader_destroy (positives);
+ casereader_destroy (negatives);
+
+ caseproto_unref (n_proto);
+ subcase_uninit (&up_ordering);
+ subcase_uninit (&down_ordering);
+
+ output_roc (rs, roc);
+
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ casereader_destroy (rs[i].cutpoint_rdr);
+
+ free (rs);
+}
+
+static void
+show_auc (struct roc_state *rs, const struct cmd_roc *roc)
+{
+ struct pivot_table *table = pivot_table_create (N_("Area Under the Curve"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Area"), PIVOT_RC_OTHER);
+ if (roc->print_se)
+ {
+ pivot_category_create_leaves (
+ statistics->root,
+ N_("Std. Error"), PIVOT_RC_OTHER,
+ N_("Asymptotic Sig."), PIVOT_RC_SIGNIFICANCE);
+ struct pivot_category *interval = pivot_category_create_group__ (
+ statistics->root,
+ pivot_value_new_text_format (N_("Asymp. %g%% Confidence Interval"),
+ roc->ci));
+ pivot_category_create_leaves (interval,
+ N_("Lower Bound"), PIVOT_RC_OTHER,
+ N_("Upper Bound"), PIVOT_RC_OTHER);
+ }
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable under test"));
+ variables->root->show_label = true;
+
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ {
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (roc->vars[i]));
+
+ pivot_table_put2 (table, 0, var_idx, pivot_value_new_number (rs[i].auc));
+
+ if (roc->print_se)
+ {
+ double se = (rs[i].auc * (1 - rs[i].auc)
+ + (rs[i].n1 - 1) * (rs[i].q1hat - pow2 (rs[i].auc))
+ + (rs[i].n2 - 1) * (rs[i].q2hat - pow2 (rs[i].auc)));
+ se /= rs[i].n1 * rs[i].n2;
+ se = sqrt (se);
+
+ double ci = 1 - roc->ci / 100.0;
+ double yy = gsl_cdf_gaussian_Qinv (ci, se);
+
+ double sd_0_5 = sqrt ((rs[i].n1 + rs[i].n2 + 1) /
+ (12 * rs[i].n1 * rs[i].n2));
+ double sig = 2.0 * gsl_cdf_ugaussian_Q (fabs ((rs[i].auc - 0.5)
+ / sd_0_5));
+ double entries[] = { se, sig, rs[i].auc - yy, rs[i].auc + yy };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i + 1, var_idx,
+ pivot_value_new_number (entries[i]));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+show_summary (const struct cmd_roc *roc)
+{
+ struct pivot_table *table = pivot_table_create (N_("Case Summary"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Valid N (listwise)"),
+ N_("Unweighted"), PIVOT_RC_INTEGER,
+ N_("Weighted"), PIVOT_RC_OTHER);
+ statistics->root->show_label = true;
+
+ struct pivot_dimension *cases = pivot_dimension_create__ (
+ table, PIVOT_AXIS_ROW, pivot_value_new_variable (roc->state_var));
+ cases->root->show_label = true;
+ pivot_category_create_leaves (cases->root, N_("Positive"), N_("Negative"));
+
+ struct entry
+ {
+ int stat_idx;
+ int case_idx;
+ double x;
+ }
+ entries[] = {
+ { 0, 0, roc->pos },
+ { 0, 1, roc->neg },
+ { 1, 0, roc->pos_weighted },
+ { 1, 1, roc->neg_weighted },
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put2 (table, e->stat_idx, e->case_idx,
+ pivot_value_new_number (e->x));
+ }
+ pivot_table_submit (table);
+}
+
+static void
+show_coords (struct roc_state *rs, const struct cmd_roc *roc)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Coordinates of the Curve"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("Positive if greater than or equal to"),
+ N_("Sensitivity"), N_("1 - Specificity"));
+
+ struct pivot_dimension *coordinates = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Coordinates"));
+ coordinates->hide_all_labels = true;
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Test variable"));
+ variables->root->show_label = true;
+
+
+ int n_coords = 0;
+ for (size_t i = 0; i < roc->n_vars; ++i)
+ {
+ struct casereader *r = casereader_clone (rs[i].cutpoint_rdr);
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (roc->vars[i]));
+
+ struct ccase *cc;
+ int coord_idx = 0;
+ for (; (cc = casereader_read (r)) != NULL; case_unref (cc))
+ {
+ const double se = case_num_idx (cc, ROC_TP) /
+ (case_num_idx (cc, ROC_TP) + case_num_idx (cc, ROC_FN));
+
+ const double sp = case_num_idx (cc, ROC_TN) /
+ (case_num_idx (cc, ROC_TN) + case_num_idx (cc, ROC_FP));
+
+ if (coord_idx >= n_coords)
+ {
+ assert (coord_idx == n_coords);
+ pivot_category_create_leaf (
+ coordinates->root, pivot_value_new_integer (++n_coords));
+ }
+
+ pivot_table_put3 (
+ table, 0, coord_idx, var_idx,
+ pivot_value_new_var_value (roc->vars[i],
+ case_data_idx (cc, ROC_CUTPOINT)));
+
+ pivot_table_put3 (table, 1, coord_idx, var_idx,
+ pivot_value_new_number (se));
+ pivot_table_put3 (table, 2, coord_idx, var_idx,
+ pivot_value_new_number (1 - sp));
+ coord_idx++;
+ }
+
+ casereader_destroy (r);
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+output_roc (struct roc_state *rs, const struct cmd_roc *roc)
+{
+ show_summary (roc);
+
+ if (roc->curve)
+ {
+ struct roc_chart *rc = roc_chart_create (roc->reference);
+ for (size_t i = 0; i < roc->n_vars; i++)
+ roc_chart_add_var (rc, var_get_name (roc->vars[i]),
+ rs[i].cutpoint_rdr);
+ roc_chart_submit (rc);
+ }
+
+ show_auc (rs, roc);
+
+ if (roc->print_coords)
+ show_coords (rs, roc);
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef LANGUAGE_STATS_ROC_H
+#define LANGUAGE_STATS_ROC_H 1
+
+/* These are case indexes into the cutpoint case readers for ROC
+ output, used by roc.c and roc-chart.c. */
+#define ROC_CUTPOINT 0
+#define ROC_TP 1
+#define ROC_FN 2
+#define ROC_TN 3
+#define ROC_FP 4
+
+#endif /* language/commands/roc.h */
--- /dev/null
+/* PSPP - a program for statistical analysis. -*-c-*-
+ Copyright (C) 2010, 2011, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <config.h>
+
+#include "language/commands/runs.h"
+
+#include <float.h>
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/percentiles.h"
+#include "math/sort.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct run_state
+{
+ /* The value used to dichotimise the data */
+ double cutpoint;
+
+ /* The number of cases not less than cutpoint */
+ double np;
+
+ /* The number of cases less than cutpoint */
+ double nn;
+
+ /* The sum of np and nn */
+ double n;
+
+ /* The number of runs */
+ long runs;
+
+ /* The sign of the last case seen */
+ short last_sign;
+};
+
+
+
+/* Return the Z statistic representing the assympototic
+ distribution of the number of runs */
+static double
+runs_statistic (const struct run_state *rs)
+{
+ double z;
+ double sigma;
+ double mu = 2 * rs->np * rs->nn;
+ mu /= rs->np + rs->nn;
+ mu += 1.0;
+
+ z = rs->runs - mu;
+
+ if (rs->n < 50)
+ {
+ if (z <= -0.5)
+ z += 0.5;
+ else if (z >= 0.5)
+ z -= 0.5;
+ else
+ return 0;
+ }
+
+ sigma = 2 * rs->np * rs->nn;
+ sigma *= 2 * rs->np * rs->nn - rs->nn - rs->np;
+ sigma /= pow2 (rs->np + rs->nn);
+ sigma /= rs->np + rs->nn - 1.0;
+ sigma = sqrt (sigma);
+
+ z /= sigma;
+
+ return z;
+}
+
+static void show_runs_result (const struct runs_test *, const struct run_state *, const struct dictionary *);
+
+void
+runs_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ int v;
+ struct ccase *c;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct variable *weight = dict_get_weight (dict);
+
+ struct one_sample_test *otp = UP_CAST (test, struct one_sample_test, parent);
+ struct runs_test *rt = UP_CAST (otp, struct runs_test, parent);
+ struct run_state *rs = XCALLOC (otp->n_vars, struct run_state);
+
+ switch (rt->cp_mode)
+ {
+ case CP_MODE:
+ {
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ bool multimodal = false;
+ struct run_state *run = &rs[v];
+ double last_cc;
+ struct casereader *group = NULL;
+ struct casegrouper *grouper;
+ struct casereader *reader = casereader_clone (input);
+ const struct variable *var = otp->vars[v];
+
+ reader = sort_execute_1var (reader, var);
+
+ grouper = casegrouper_create_vars (reader, &var, 1);
+ last_cc = SYSMIS;
+ while (casegrouper_get_next_group (grouper, &group))
+ {
+ double x = SYSMIS;
+ double cc = 0.0;
+ struct ccase *c;
+ for (; (c = casereader_read (group)); case_unref (c))
+ {
+ const double w = weight ? case_num (c, weight) : 1.0;
+ const union value *val = case_data (c, var);
+ if (var_is_value_missing (var, val) & exclude)
+ continue;
+ x = val->f;
+ cc += w;
+ }
+
+ if (cc > last_cc)
+ {
+ run->cutpoint = x;
+ }
+ else if (cc == last_cc)
+ {
+ multimodal = true;
+ if (x > run->cutpoint)
+ run->cutpoint = x;
+ }
+ last_cc = cc;
+ casereader_destroy (group);
+ }
+ casegrouper_destroy (grouper);
+ if (multimodal)
+ msg (MW, _("Multiple modes exist for variable `%s'. "
+ "Using %.*g as the threshold value."),
+ var_get_name (var), DBL_DIG + 1, run->cutpoint);
+ }
+ }
+ break;
+ case CP_MEDIAN:
+ {
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ double cc = 0.0;
+ struct ccase *c;
+ struct run_state *run = &rs[v];
+ struct casereader *reader = casereader_clone (input);
+ const struct variable *var = otp->vars[v];
+ struct casewriter *writer;
+ struct percentile *median;
+ struct order_stats *os;
+ struct subcase sc;
+ subcase_init_var (&sc, var, SC_ASCEND);
+ writer = sort_create_writer (&sc, casereader_get_proto (reader));
+
+ for (; (c = casereader_read (reader));)
+ {
+ const union value *val = case_data (c, var);
+ const double w = weight ? case_num (c, weight) : 1.0;
+ if (var_is_value_missing (var, val) & exclude)
+ {
+ case_unref (c);
+ continue;
+ }
+
+ cc += w;
+ casewriter_write (writer, c);
+ }
+ subcase_uninit (&sc);
+ casereader_destroy (reader);
+ reader = casewriter_make_reader (writer);
+
+ median = percentile_create (0.5, cc);
+ os = &median->parent;
+
+ order_stats_accumulate (&os, 1,
+ reader,
+ weight,
+ var,
+ exclude);
+
+ run->cutpoint = percentile_calculate (median, PC_HAVERAGE);
+ statistic_destroy (&median->parent.parent);
+ }
+ }
+ break;
+ case CP_MEAN:
+ {
+ struct casereader *reader = casereader_clone (input);
+ for (; (c = casereader_read (reader)); case_unref (c))
+ {
+ const double w = weight ? case_num (c, weight) : 1.0;
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ const struct variable *var = otp->vars[v];
+ const union value *val = case_data (c, var);
+ const double x = val->f;
+ struct run_state *run = &rs[v];
+
+ if (var_is_value_missing (var, val) & exclude)
+ continue;
+
+ run->cutpoint += x * w;
+ run->n += w;
+ }
+ }
+ casereader_destroy (reader);
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ struct run_state *run = &rs[v];
+ run->cutpoint /= run->n;
+ }
+ }
+ break;
+ case CP_CUSTOM:
+ {
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ struct run_state *run = &rs[v];
+ run->cutpoint = rt->cutpoint;
+ }
+ }
+ break;
+ }
+
+ for (; (c = casereader_read (input)); case_unref (c))
+ {
+ const double w = weight ? case_num (c, weight) : 1.0;
+
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ struct run_state *run = &rs[v];
+ const struct variable *var = otp->vars[v];
+ const union value *val = case_data (c, var);
+ double x = val->f;
+ double d = x - run->cutpoint;
+ short sign = 0;
+
+ if (var_is_value_missing (var, val) & exclude)
+ continue;
+
+ if (d >= 0)
+ {
+ sign = +1;
+ run->np += w;
+ }
+ else
+ {
+ sign = -1;
+ run->nn += w;
+ }
+
+ if (sign != run->last_sign)
+ run->runs++;
+
+ run->last_sign = sign;
+ }
+ }
+ casereader_destroy (input);
+
+ for (v = 0; v < otp->n_vars; ++v)
+ {
+ struct run_state *run = &rs[v];
+ run->n = run->np + run->nn;
+ }
+
+ show_runs_result (rt, rs, dict);
+
+ free (rs);
+}
+
+\f
+
+static void
+show_runs_result (const struct runs_test *rt, const struct run_state *rs, const struct dictionary *dict)
+{
+ const struct one_sample_test *otp = &rt->parent;
+
+ struct pivot_table *table = pivot_table_create (N_("Runs Test"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ (rt->cp_mode == CP_CUSTOM ? N_("Test Value")
+ : rt->cp_mode == CP_MODE ? N_("Test Value (mode)")
+ : rt->cp_mode == CP_MEAN ? N_("Test Value (mean)")
+ : N_("Test Value (median)")), PIVOT_RC_OTHER,
+ N_("Cases < Test Value"), PIVOT_RC_COUNT,
+ N_("Cases ≥ Test Value"), PIVOT_RC_COUNT,
+ N_("Total Cases"), PIVOT_RC_COUNT,
+ N_("Number of Runs"), PIVOT_RC_INTEGER,
+ N_("Z"), PIVOT_RC_OTHER,
+ N_("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Variable"));
+
+ for (size_t i = 0 ; i < otp->n_vars; ++i)
+ {
+ const struct run_state *run = &rs[i];
+
+ int col = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (otp->vars[i]));
+
+ double z = runs_statistic (run);
+
+ double rows[] = {
+ run->cutpoint,
+ run->nn,
+ run->np,
+ run->n,
+ run->runs,
+ z,
+ 2.0 * (1.0 - gsl_cdf_ugaussian_P (fabs (z))),
+ };
+
+ for (int row = 0; row < sizeof rows / sizeof *rows; row++)
+ pivot_table_put2 (table, row, col, pivot_value_new_number (rows[row]));
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !runs_h
+#define runs_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+#include "language/commands/npar.h"
+
+enum cp_mode
+ {
+ CP_MEAN,
+ CP_MEDIAN,
+ CP_MODE,
+ CP_CUSTOM
+ };
+
+
+struct runs_test
+{
+ struct one_sample_test parent;
+
+ double cutpoint;
+
+ enum cp_mode cp_mode;
+};
+
+
+void runs_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool,
+ double);
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009-2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <gsl/gsl_rng.h>
+#include <limits.h>
+#include <stdio.h>
+#include <math.h>
+
+#include "data/dataset.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/compiler.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "math/random.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* The two different types of samples. */
+enum
+ {
+ TYPE_A_FROM_B, /* 5 FROM 10 */
+ TYPE_FRACTION /* 0.5 */
+ };
+
+/* SAMPLE transformation. */
+struct sample_trns
+ {
+ int type; /* One of TYPE_*. */
+ int n, N; /* TYPE_A_FROM_B: n from N. */
+ int m, t; /* TYPE_A_FROM_B: # picked so far; # so far. */
+ unsigned frac; /* TYPE_FRACTION: a fraction of UINT_MAX. */
+ };
+
+static const struct trns_class sample_trns_class;
+
+int
+cmd_sample (struct lexer *lexer, struct dataset *ds)
+{
+ struct sample_trns *trns;
+
+ int type;
+ int a, b;
+ unsigned frac;
+
+ if (!lex_force_num (lexer))
+ return CMD_FAILURE;
+ if (!lex_is_integer (lexer))
+ {
+ unsigned long min = gsl_rng_min (get_rng ());
+ unsigned long max = gsl_rng_max (get_rng ());
+
+ type = TYPE_FRACTION;
+ if (!lex_force_num_range_open (lexer, "SAMPLE", 0, 1))
+ return CMD_FAILURE;
+
+ frac = lex_tokval (lexer) * (max - min) + min;
+ a = b = 0;
+ }
+ else
+ {
+ type = TYPE_A_FROM_B;
+ a = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match_id (lexer, "FROM"))
+ return CMD_FAILURE;
+ if (!lex_force_int_range (lexer, "FROM", a + 1, INT_MAX))
+ return CMD_FAILURE;
+ b = lex_integer (lexer);
+ frac = 0;
+ }
+ lex_get (lexer);
+
+ trns = xmalloc (sizeof *trns);
+ trns->type = type;
+ trns->n = a;
+ trns->N = b;
+ trns->m = trns->t = 0;
+ trns->frac = frac;
+ add_transformation (ds, &sample_trns_class, trns);
+
+ return CMD_SUCCESS;
+}
+
+/* Executes a SAMPLE transformation. */
+static enum trns_result
+sample_trns_proc (void *t_, struct ccase **c UNUSED,
+ casenumber case_num UNUSED)
+{
+ struct sample_trns *t = t_;
+ double U;
+
+ if (t->type == TYPE_FRACTION)
+ {
+ if (gsl_rng_get (get_rng ()) <= t->frac)
+ return TRNS_CONTINUE;
+ else
+ return TRNS_DROP_CASE;
+ }
+
+ if (t->m >= t->n)
+ return TRNS_DROP_CASE;
+
+ U = gsl_rng_uniform (get_rng ());
+ if ((t->N - t->t) * U >= t->n - t->m)
+ {
+ t->t++;
+ return TRNS_DROP_CASE;
+ }
+ else
+ {
+ t->m++;
+ t->t++;
+ return TRNS_CONTINUE;
+ }
+}
+
+static bool
+sample_trns_free (void *t_)
+{
+ struct sample_trns *t = t_;
+ free (t);
+ return true;
+}
+
+static const struct trns_class sample_trns_class = {
+ .name = "SAMPLE",
+ .execute = sample_trns_proc,
+ .destroy = sample_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2010, 2011, 2013, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/case-map.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/csv-file-writer.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/file-name.h"
+#include "data/format.h"
+#include "data/settings.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/trim.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+int
+cmd_save_translate (struct lexer *lexer, struct dataset *ds)
+{
+ enum { CSV_FILE = 1, TAB_FILE } type = 0;
+
+ struct dictionary *dict = dict_clone (dataset_dict (ds));
+ dict_set_names_must_be_ids (dict, false);
+
+ struct case_map_stage *stage = case_map_stage_create (dict);
+ dict_delete_scratch_vars (dict);
+
+ struct file_handle *handle = NULL;
+
+ bool replace = false;
+
+ bool retain_unselected = true;
+ bool recode_user_missing = false;
+ bool include_var_names = false;
+ bool use_value_labels = false;
+ bool use_print_formats = false;
+ char decimal = settings_get_fmt_settings ()->decimal;
+ char delimiter = 0;
+ char qualifier = '"';
+
+ int outfile_start = 0;
+ int outfile_end = 0;
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ if (!lex_force_match (lexer, T_SLASH))
+ goto error;
+
+ if (lex_match_id (lexer, "OUTFILE"))
+ {
+ outfile_start = lex_ofs (lexer) - 1;
+ if (handle != NULL)
+ {
+ lex_sbc_only_once (lexer, "OUTFILE");
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ handle = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (handle == NULL)
+ goto error;
+ outfile_end = lex_ofs (lexer) - 1;
+ }
+ else if (lex_match_id (lexer, "TYPE"))
+ {
+ if (type != 0)
+ {
+ lex_sbc_only_once (lexer, "TYPE");
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "CSV"))
+ type = CSV_FILE;
+ else if (lex_match_id (lexer, "TAB"))
+ type = TAB_FILE;
+ else
+ {
+ lex_error_expecting (lexer, "CSV", "TAB");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "REPLACE"))
+ replace = true;
+ else if (lex_match_id (lexer, "FIELDNAMES"))
+ include_var_names = true;
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "IGNORE"))
+ recode_user_missing = false;
+ else if (lex_match_id (lexer, "RECODE"))
+ recode_user_missing = true;
+ else
+ {
+ lex_error_expecting (lexer, "IGNORE", "RECODE");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "CELLS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "VALUES"))
+ use_value_labels = false;
+ else if (lex_match_id (lexer, "LABELS"))
+ use_value_labels = true;
+ else
+ {
+ lex_error_expecting (lexer, "VALUES", "LABELS");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "TEXTOPTIONS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ for (;;)
+ {
+ if (lex_match_id (lexer, "DELIMITER"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+ /* XXX should support multibyte UTF-8 delimiters */
+ if (ss_length (lex_tokss (lexer)) != 1)
+ {
+ lex_error (lexer, _("The %s string must contain exactly "
+ "one character."), "DELIMITER");
+ goto error;
+ }
+ delimiter = ss_first (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "QUALIFIER"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_string (lexer))
+ goto error;
+ /* XXX should support multibyte UTF-8 qualifiers */
+ if (ss_length (lex_tokss (lexer)) != 1)
+ {
+ lex_error (lexer, _("The %s string must contain exactly "
+ "one character."), "QUALIFIER");
+ goto error;
+ }
+ qualifier = ss_first (lex_tokss (lexer));
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "DECIMAL"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "DOT"))
+ decimal = '.';
+ else if (lex_match_id (lexer, "COMMA"))
+ decimal = ',';
+ else
+ {
+ lex_error_expecting (lexer, "DOT", "COMMA");
+ goto error;
+ }
+ }
+ else if (lex_match_id (lexer, "FORMAT"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "PLAIN"))
+ use_print_formats = false;
+ else if (lex_match_id (lexer, "VARIABLE"))
+ use_print_formats = true;
+ else
+ {
+ lex_error_expecting (lexer, "PLAIN", "VARIABLE");
+ goto error;
+ }
+ }
+ else
+ break;
+ }
+ }
+ else if (lex_match_id (lexer, "UNSELECTED"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "RETAIN"))
+ retain_unselected = true;
+ else if (lex_match_id (lexer, "DELETE"))
+ retain_unselected = false;
+ else
+ {
+ lex_error_expecting (lexer, "RETAIN", "DELETE");
+ goto error;
+ }
+ }
+ else if (!parse_dict_trim (lexer, dict))
+ goto error;
+ }
+
+ if (type == 0)
+ {
+ lex_sbc_missing (lexer, "TYPE");
+ goto error;
+ }
+ else if (handle == NULL)
+ {
+ lex_sbc_missing (lexer, "OUTFILE");
+ goto error;
+ }
+ else if (!replace && fn_exists (handle))
+ {
+ lex_ofs_error (lexer, outfile_start, outfile_end,
+ _("Output file `%s' exists but %s was not specified."),
+ fh_get_file_name (handle), "REPLACE");
+ goto error;
+ }
+
+ dict_delete_scratch_vars (dict);
+ dict_compact_values (dict);
+
+ struct csv_writer_options csv_opts = {
+ .recode_user_missing = recode_user_missing,
+ .include_var_names = include_var_names,
+ .use_value_labels = use_value_labels,
+ .use_print_formats = use_print_formats,
+ .decimal = decimal,
+ .delimiter = (delimiter ? delimiter
+ : type == TAB_FILE ? '\t'
+ : decimal == '.' ? ','
+ : ';'),
+ .qualifier = qualifier,
+ };
+ struct casewriter *writer = csv_writer_open (handle, dict, &csv_opts);
+ if (writer == NULL)
+ goto error;
+ fh_unref (handle);
+
+ struct case_map *map = case_map_stage_get_case_map (stage);
+ case_map_stage_destroy (stage);
+ if (map != NULL)
+ writer = case_map_create_output_translator (map, writer);
+ dict_unref (dict);
+
+ casereader_transfer (proc_open_filtering (ds, !retain_unselected), writer);
+ bool ok = casewriter_destroy (writer);
+ ok = proc_commit (ds) && ok;
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+
+error:
+ case_map_stage_destroy (stage);
+ fh_unref (handle);
+ dict_unref (dict);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/any-writer.h"
+#include "data/case-map.h"
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/mdd-writer.h"
+#include "data/por-file-writer.h"
+#include "data/sys-file-writer.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/commands/trim.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/message.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Writing system and portable files. */
+
+/* Type of output file. */
+enum writer_type
+ {
+ SYSFILE_WRITER, /* System file. */
+ PORFILE_WRITER /* Portable file. */
+ };
+
+/* Type of a command. */
+enum command_type
+ {
+ XFORM_CMD, /* Transformation. */
+ PROC_CMD /* Procedure. */
+ };
+
+static int parse_output_proc (struct lexer *, struct dataset *,
+ enum writer_type);
+static int parse_output_trns (struct lexer *, struct dataset *,
+ enum writer_type);
+
+int
+cmd_save (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_output_proc (lexer, ds, SYSFILE_WRITER);
+}
+
+int
+cmd_save_data_collection (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_output_proc (lexer, ds, SYSFILE_WRITER);
+}
+
+int
+cmd_export (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_output_proc (lexer, ds, PORFILE_WRITER);
+}
+
+int
+cmd_xsave (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_output_trns (lexer, ds, SYSFILE_WRITER);
+}
+
+int
+cmd_xexport (struct lexer *lexer, struct dataset *ds)
+{
+ return parse_output_trns (lexer, ds, PORFILE_WRITER);
+}
+\f
+struct output_trns
+ {
+ struct casewriter *writer; /* Writer. */
+ };
+
+static const struct trns_class output_trns_class;
+static struct casewriter *parse_write_command (struct lexer *,
+ struct dataset *,
+ enum writer_type,
+ enum command_type,
+ bool *retain_unselected);
+
+/* Parses and performs the SAVE or EXPORT procedure. */
+static int
+parse_output_proc (struct lexer *lexer, struct dataset *ds,
+ enum writer_type writer_type)
+{
+ bool retain_unselected;
+ struct casewriter *output;
+ bool ok;
+
+ output = parse_write_command (lexer, ds, writer_type, PROC_CMD,
+ &retain_unselected);
+ if (output == NULL)
+ return CMD_CASCADING_FAILURE;
+
+ casereader_transfer (proc_open_filtering (ds, !retain_unselected), output);
+ ok = casewriter_destroy (output);
+ ok = proc_commit (ds) && ok;
+
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+}
+
+/* Parses the XSAVE or XEXPORT transformation command. */
+static int
+parse_output_trns (struct lexer *lexer, struct dataset *ds, enum writer_type writer_type)
+{
+ struct output_trns *t = xmalloc (sizeof *t);
+ t->writer = parse_write_command (lexer, ds, writer_type, XFORM_CMD, NULL);
+ if (t->writer == NULL)
+ {
+ free (t);
+ return CMD_CASCADING_FAILURE;
+ }
+
+ add_transformation (ds, &output_trns_class, t);
+ return CMD_SUCCESS;
+}
+
+/* Parses SAVE or XSAVE or EXPORT or XEXPORT command.
+ WRITER_TYPE identifies the type of file to write,
+ and COMMAND_TYPE identifies the type of command.
+
+ On success, returns a writer.
+ For procedures only, sets *RETAIN_UNSELECTED to true if cases
+ that would otherwise be excluded by FILTER or USE should be
+ included.
+
+ On failure, returns a null pointer. */
+static struct casewriter *
+parse_write_command (struct lexer *lexer, struct dataset *ds,
+ enum writer_type writer_type,
+ enum command_type command_type,
+ bool *retain_unselected)
+{
+ /* Common data. */
+ struct file_handle *handle; /* Output file. */
+ struct file_handle *metadata; /* MDD output file. */
+ struct dictionary *dict; /* Dictionary for output file. */
+ struct casewriter *writer; /* Writer. */
+ struct case_map_stage *stage; /* Preparation for 'map'. */
+ struct case_map *map; /* Map from input data to data for writer. */
+ const char *sav_name = "";
+
+ /* Common options. */
+ struct sfm_write_options sysfile_opts;
+ struct pfm_write_options porfile_opts;
+
+ assert (writer_type == SYSFILE_WRITER || writer_type == PORFILE_WRITER);
+ assert (command_type == XFORM_CMD || command_type == PROC_CMD);
+ assert ((retain_unselected != NULL) == (command_type == PROC_CMD));
+
+ if (command_type == PROC_CMD)
+ *retain_unselected = true;
+
+ handle = NULL;
+ metadata = NULL;
+ dict = dict_clone (dataset_dict (ds));
+ writer = NULL;
+ stage = NULL;
+ map = NULL;
+ sysfile_opts = sfm_writer_default_options ();
+ porfile_opts = pfm_writer_default_options ();
+
+ stage = case_map_stage_create (dict);
+ dict_delete_scratch_vars (dict);
+
+ lex_match (lexer, T_SLASH);
+ for (;;)
+ {
+ if (lex_match_id (lexer, "OUTFILE"))
+ {
+ if (handle != NULL)
+ {
+ lex_sbc_only_once (lexer, "OUTFILE");
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ handle = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (handle == NULL)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "METADATA"))
+ {
+ if (metadata != NULL)
+ {
+ lex_sbc_only_once (lexer, "METADATA");
+ goto error;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ metadata = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (metadata == NULL)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "NAMES"))
+ {
+ /* Not yet implemented. */
+ }
+ else if (lex_match_id (lexer, "PERMISSIONS"))
+ {
+ bool cw;
+
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "READONLY"))
+ cw = false;
+ else if (lex_match_id (lexer, "WRITEABLE"))
+ cw = true;
+ else
+ {
+ lex_error_expecting (lexer, "READONLY", "WRITEABLE");
+ goto error;
+ }
+ sysfile_opts.create_writeable = porfile_opts.create_writeable = cw;
+ }
+ else if (command_type == PROC_CMD && lex_match_id (lexer, "UNSELECTED"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "RETAIN"))
+ *retain_unselected = true;
+ else if (lex_match_id (lexer, "DELETE"))
+ *retain_unselected = false;
+ else
+ {
+ lex_error_expecting (lexer, "RETAIN", "DELETE");
+ goto error;
+ }
+ }
+ else if (writer_type == SYSFILE_WRITER
+ && lex_match_id (lexer, "COMPRESSED"))
+ sysfile_opts.compression = ANY_COMP_SIMPLE;
+ else if (writer_type == SYSFILE_WRITER
+ && lex_match_id (lexer, "UNCOMPRESSED"))
+ sysfile_opts.compression = ANY_COMP_NONE;
+ else if (writer_type == SYSFILE_WRITER
+ && lex_match_id (lexer, "ZCOMPRESSED"))
+ sysfile_opts.compression = ANY_COMP_ZLIB;
+ else if (writer_type == SYSFILE_WRITER
+ && lex_match_id (lexer, "VERSION"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "VERSION", 2, 3))
+ goto error;
+ sysfile_opts.version = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (writer_type == PORFILE_WRITER && lex_match_id (lexer, "TYPE"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (lex_match_id (lexer, "COMMUNICATIONS"))
+ porfile_opts.type = PFM_COMM;
+ else if (lex_match_id (lexer, "TAPE"))
+ porfile_opts.type = PFM_TAPE;
+ else
+ {
+ lex_error_expecting (lexer, "COMM", "TAPE");
+ goto error;
+ }
+ }
+ else if (writer_type == PORFILE_WRITER && lex_match_id (lexer, "DIGITS"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "DIGITS", 1, INT_MAX))
+ goto error;
+ porfile_opts.digits = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (!parse_dict_trim (lexer, dict))
+ goto error;
+
+ if (!lex_match (lexer, T_SLASH))
+ break;
+ }
+ if (lex_end_of_command (lexer) != CMD_SUCCESS)
+ goto error;
+
+ if (!handle && !metadata)
+ {
+ msg (SE, _("The OUTFILE or METADATA subcommand is required."));
+ goto error;
+ }
+
+ dict_delete_scratch_vars (dict);
+ dict_compact_values (dict);
+
+ if (handle)
+ {
+ if (metadata)
+ sav_name = (fh_get_referent (handle) == FH_REF_FILE
+ ? fh_get_file_name (handle)
+ : fh_get_name (handle));
+ if (fh_get_referent (handle) == FH_REF_FILE)
+ {
+ switch (writer_type)
+ {
+ case SYSFILE_WRITER:
+ writer = sfm_open_writer (handle, dict, sysfile_opts);
+ break;
+ case PORFILE_WRITER:
+ writer = pfm_open_writer (handle, dict, porfile_opts);
+ break;
+ }
+ }
+ else
+ writer = any_writer_open (handle, dict);
+ if (writer == NULL)
+ goto error;
+ }
+
+ if (metadata)
+ {
+ if (!mdd_write (metadata, dict, sav_name))
+ goto error;
+ }
+
+ map = case_map_stage_get_case_map (stage);
+ case_map_stage_destroy (stage);
+ if (map != NULL)
+ writer = case_map_create_output_translator (map, writer);
+ dict_unref (dict);
+
+ fh_unref (handle);
+ fh_unref (metadata);
+ return writer;
+
+ error:
+ case_map_stage_destroy (stage);
+ fh_unref (handle);
+ fh_unref (metadata);
+ casewriter_destroy (writer);
+ dict_unref (dict);
+ case_map_destroy (map);
+ return NULL;
+}
+
+/* Writes case *C to the system file specified on XSAVE or XEXPORT. */
+static enum trns_result
+output_trns_proc (void *trns_, struct ccase **c, casenumber case_num UNUSED)
+{
+ struct output_trns *t = trns_;
+ casewriter_write (t->writer, case_ref (*c));
+ return TRNS_CONTINUE;
+}
+
+/* Frees an XSAVE or XEXPORT transformation.
+ Returns true if successful, false if an I/O error occurred. */
+static bool
+output_trns_free (void *trns_)
+{
+ struct output_trns *t = trns_;
+ bool ok = casewriter_destroy (t->writer);
+ free (t);
+ return ok;
+}
+
+static const struct trns_class output_trns_class = {
+ .name = "XSAVE/XEXPORT",
+ .execute = output_trns_proc,
+ .destroy = output_trns_free,
+};
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/transformations.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/expressions/public.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* SELECT IF transformation. */
+struct select_if_trns
+ {
+ struct expression *e; /* Test expression. */
+ };
+
+static const struct trns_class select_if_trns_class;
+
+/* Parses the SELECT IF transformation. */
+int
+cmd_select_if (struct lexer *lexer, struct dataset *ds)
+{
+ struct expression *e;
+ struct select_if_trns *t;
+
+ e = expr_parse_bool (lexer, ds);
+ if (!e)
+ return CMD_CASCADING_FAILURE;
+
+ if (lex_token (lexer) != T_ENDCMD)
+ {
+ expr_free (e);
+ lex_error (lexer, _("Syntax error expecting end of command."));
+ return CMD_CASCADING_FAILURE;
+ }
+
+ t = xmalloc (sizeof *t);
+ t->e = e;
+ add_transformation (ds, &select_if_trns_class, t);
+
+ return CMD_SUCCESS;
+}
+
+/* Performs the SELECT IF transformation T on case C. */
+static enum trns_result
+select_if_proc (void *t_, struct ccase **c,
+ casenumber case_num)
+{
+ struct select_if_trns *t = t_;
+ return (expr_evaluate_num (t->e, *c, case_num) == 1.0
+ ? TRNS_CONTINUE : TRNS_DROP_CASE);
+}
+
+/* Frees SELECT IF transformation T. */
+static bool
+select_if_free (void *t_)
+{
+ struct select_if_trns *t = t_;
+ expr_free (t->e);
+ free (t);
+ return true;
+}
+
+static const struct trns_class select_if_trns_class = {
+ .name = "SELECT IF",
+ .execute = select_if_proc,
+ .destroy = select_if_free,
+};
+
+/* Parses the FILTER command. */
+int
+cmd_filter (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ if (lex_match_id (lexer, "OFF"))
+ dict_set_filter (dict, NULL);
+ else if (lex_match (lexer, T_BY))
+ {
+ struct variable *v = parse_variable (lexer, dict);
+ if (!v)
+ return CMD_FAILURE;
+
+ if (var_is_alpha (v))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("The filter variable must be numeric."));
+ return CMD_FAILURE;
+ }
+
+ if (dict_class_from_id (var_get_name (v)) == DC_SCRATCH)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("The filter variable may not be scratch."));
+ return CMD_FAILURE;
+ }
+
+ dict_set_filter (dict, v);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "OFF", "BY");
+ return CMD_FAILURE;
+ }
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <float.h>
+#include <stdio.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "data/casereader.h"
+#include "data/data-in.h"
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/settings.h"
+#include "data/value.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/token.h"
+#include "libpspp/assertion.h"
+#include "libpspp/compiler.h"
+#include "libpspp/copyleft.h"
+#include "libpspp/temp-file.h"
+#include "libpspp/version.h"
+#include "libpspp/float-format.h"
+#include "libpspp/i18n.h"
+#include "libpspp/integer-format.h"
+#include "libpspp/message.h"
+#include "math/random.h"
+#include "output/driver.h"
+#include "output/journal.h"
+#include "output/pivot-table.h"
+
+#include "gl/ftoastr.h"
+#include "gl/minmax.h"
+#include "gl/relocatable.h"
+#include "gl/vasnprintf.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+struct setting
+ {
+ const char *name;
+ bool (*set) (struct lexer *);
+ char *(*show) (const struct dataset *);
+ };
+
+static bool
+match_subcommand (struct lexer *lexer, const char *name)
+{
+ if (lex_match_id (lexer, name))
+ {
+ lex_match (lexer, T_EQUALS);
+ return true;
+ }
+ else
+ return false;
+}
+
+static int
+subcommand_start_ofs (struct lexer *lexer)
+{
+ int ofs = lex_ofs (lexer) - 1;
+ return lex_ofs_token (lexer, ofs)->type == T_EQUALS ? ofs - 1 : ofs;
+}
+
+static int
+parse_enum_valist (struct lexer *lexer, va_list args)
+{
+ for (;;)
+ {
+ const char *name = va_arg (args, char *);
+ if (!name)
+ return -1;
+ int value = va_arg (args, int);
+
+ if (lex_match_id (lexer, name))
+ return value;
+ }
+}
+
+#define parse_enum(...) parse_enum (__VA_ARGS__, NULL_SENTINEL)
+static int SENTINEL(0)
+(parse_enum) (struct lexer *lexer, ...)
+{
+ va_list args;
+
+ va_start (args, lexer);
+ int retval = parse_enum_valist (lexer, args);
+ va_end (args);
+
+ return retval;
+}
+
+#define force_parse_enum(...) force_parse_enum (__VA_ARGS__, NULL_SENTINEL)
+static int SENTINEL(0)
+(force_parse_enum) (struct lexer *lexer, ...)
+{
+ va_list args;
+
+ va_start (args, lexer);
+ int retval = parse_enum_valist (lexer, args);
+ va_end (args);
+
+ if (retval == -1)
+ {
+ enum { MAX_OPTIONS = 9 };
+ const char *options[MAX_OPTIONS];
+ int n = 0;
+
+ va_start (args, lexer);
+ while (n < MAX_OPTIONS)
+ {
+ const char *name = va_arg (args, char *);
+ if (!name)
+ break;
+ va_arg (args, int);
+
+ options[n++] = name;
+ }
+ va_end (args);
+
+ lex_error_expecting_array (lexer, options, n);
+ }
+
+ return retval;
+}
+
+static int
+parse_bool (struct lexer *lexer)
+{
+ return parse_enum (lexer,
+ "ON", true, "YES", true,
+ "OFF", false, "NO", false);
+}
+
+static int
+force_parse_bool (struct lexer *lexer)
+{
+ return force_parse_enum (lexer,
+ "ON", true, "YES", true,
+ "OFF", false, "NO", false);
+}
+
+static bool
+parse_output_routing (struct lexer *lexer, enum settings_output_type type)
+{
+ enum settings_output_devices devices;
+ if (lex_match_id (lexer, "ON") || lex_match_id (lexer, "BOTH"))
+ devices = SETTINGS_DEVICE_LISTING | SETTINGS_DEVICE_TERMINAL;
+ else if (lex_match_id (lexer, "TERMINAL"))
+ devices = SETTINGS_DEVICE_TERMINAL;
+ else if (lex_match_id (lexer, "LISTING"))
+ devices = SETTINGS_DEVICE_LISTING;
+ else if (lex_match_id (lexer, "OFF") || lex_match_id (lexer, "NONE"))
+ devices = 0;
+ else
+ {
+ lex_error_expecting (lexer, "ON", "BOTH", "TERMINAL", "LISTING",
+ "OFF", "NONE");
+ return false;
+ }
+
+ settings_set_output_routing (type, devices);
+
+ return true;
+}
+
+static char *
+show_output_routing (enum settings_output_type type)
+{
+ enum settings_output_devices devices;
+ const char *s;
+
+ devices = settings_get_output_routing (type);
+ if (devices & SETTINGS_DEVICE_LISTING)
+ s = devices & SETTINGS_DEVICE_TERMINAL ? "BOTH" : "LISTING";
+ else if (devices & SETTINGS_DEVICE_TERMINAL)
+ s = "TERMINAL";
+ else
+ s = "NONE";
+
+ return xstrdup (s);
+}
+
+static bool
+parse_integer_format (struct lexer *lexer,
+ void (*set_format) (enum integer_format))
+{
+ int value = force_parse_enum (lexer,
+ "MSBFIRST", INTEGER_MSB_FIRST,
+ "LSBFIRST", INTEGER_LSB_FIRST,
+ "VAX", INTEGER_VAX,
+ "NATIVE", INTEGER_NATIVE);
+ if (value >= 0)
+ set_format (value);
+ return value >= 0;
+}
+
+/* Returns a name for the given INTEGER_FORMAT value. */
+static char *
+show_integer_format (enum integer_format integer_format)
+{
+ return xasprintf ("%s (%s)",
+ (integer_format == INTEGER_MSB_FIRST ? "MSBFIRST"
+ : integer_format == INTEGER_LSB_FIRST ? "LSBFIRST"
+ : "VAX"),
+ integer_format == INTEGER_NATIVE ? "NATIVE" : "nonnative");
+}
+
+static bool
+parse_real_format (struct lexer *lexer,
+ void (*set_format) (enum float_format))
+{
+ int value = force_parse_enum (lexer,
+ "NATIVE", FLOAT_NATIVE_DOUBLE,
+ "ISL", FLOAT_IEEE_SINGLE_LE,
+ "ISB", FLOAT_IEEE_SINGLE_BE,
+ "IDL", FLOAT_IEEE_DOUBLE_LE,
+ "IDB", FLOAT_IEEE_DOUBLE_BE,
+ "VF", FLOAT_VAX_F,
+ "VD", FLOAT_VAX_D,
+ "VG", FLOAT_VAX_G,
+ "ZS", FLOAT_Z_SHORT,
+ "ZL", FLOAT_Z_LONG);
+ if (value >= 0)
+ set_format (value);
+ return value >= 0;
+}
+
+/* Returns a name for the given FLOAT_FORMAT value. */
+static char *
+show_real_format (enum float_format float_format)
+{
+ const char *format_name = "";
+
+ switch (float_format)
+ {
+ case FLOAT_IEEE_SINGLE_LE:
+ format_name = _("ISL (32-bit IEEE 754 single, little-endian)");
+ break;
+ case FLOAT_IEEE_SINGLE_BE:
+ format_name = _("ISB (32-bit IEEE 754 single, big-endian)");
+ break;
+ case FLOAT_IEEE_DOUBLE_LE:
+ format_name = _("IDL (64-bit IEEE 754 double, little-endian)");
+ break;
+ case FLOAT_IEEE_DOUBLE_BE:
+ format_name = _("IDB (64-bit IEEE 754 double, big-endian)");
+ break;
+
+ case FLOAT_VAX_F:
+ format_name = _("VF (32-bit VAX F, VAX-endian)");
+ break;
+ case FLOAT_VAX_D:
+ format_name = _("VD (64-bit VAX D, VAX-endian)");
+ break;
+ case FLOAT_VAX_G:
+ format_name = _("VG (64-bit VAX G, VAX-endian)");
+ break;
+
+ case FLOAT_Z_SHORT:
+ format_name = _("ZS (32-bit IBM Z hexadecimal short, big-endian)");
+ break;
+ case FLOAT_Z_LONG:
+ format_name = _("ZL (64-bit IBM Z hexadecimal long, big-endian)");
+ break;
+
+ case FLOAT_FP:
+ case FLOAT_HEX:
+ NOT_REACHED ();
+ }
+
+ return xasprintf ("%s (%s)", format_name,
+ (float_format == FLOAT_NATIVE_DOUBLE
+ ? "NATIVE" : "nonnative"));
+}
+
+static bool
+parse_unimplemented (struct lexer *lexer, const char *name)
+{
+ int start = subcommand_start_ofs (lexer);
+ if (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ lex_get (lexer);
+ int end = lex_ofs (lexer) - 1;
+
+ lex_ofs_msg (lexer, SW, start, end, _("%s is not yet implemented."), name);
+ return true;
+}
+
+static bool
+parse_ccx (struct lexer *lexer, enum fmt_type ccx)
+{
+ if (!lex_force_string (lexer))
+ return false;
+
+ char *error = settings_set_cc (lex_tokcstr (lexer), ccx);
+ if (error)
+ {
+ lex_error (lexer, "%s", error);
+ free (error);
+ return false;
+ }
+
+ lex_get (lexer);
+ return true;
+}
+\f
+static bool
+parse_BASETEXTDIRECTION (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "BASETEXTDIRECTION");
+}
+
+static bool
+parse_BLANKS (struct lexer *lexer)
+{
+ if (lex_match_id (lexer, "SYSMIS"))
+ settings_set_blanks (SYSMIS);
+ else
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ settings_set_blanks (lex_number (lexer));
+ lex_get (lexer);
+ }
+ return true;
+}
+
+static char *
+show_BLANKS (const struct dataset *ds UNUSED)
+{
+ return (settings_get_blanks () == SYSMIS
+ ? xstrdup ("SYSMIS")
+ : xasprintf ("%.*g", DBL_DIG + 1, settings_get_blanks ()));
+}
+
+static bool
+parse_BLOCK (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "BLOCK");
+}
+
+static bool
+parse_BOX (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "BOX");
+}
+
+static bool
+parse_CACHE (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "CACHE");
+}
+
+static bool
+parse_CCA (struct lexer *lexer)
+{
+ return parse_ccx (lexer, FMT_CCA);
+}
+
+static bool
+parse_CCB (struct lexer *lexer)
+{
+ return parse_ccx (lexer, FMT_CCB);
+}
+
+static bool
+parse_CCC (struct lexer *lexer)
+{
+ return parse_ccx (lexer, FMT_CCC);
+}
+
+static bool
+parse_CCD (struct lexer *lexer)
+{
+ return parse_ccx (lexer, FMT_CCD);
+}
+
+static bool
+parse_CCE (struct lexer *lexer)
+{
+ return parse_ccx (lexer, FMT_CCE);
+}
+
+static char *
+show_cc (enum fmt_type type)
+{
+ return fmt_number_style_to_string (fmt_settings_get_style (
+ settings_get_fmt_settings (), type));
+}
+
+static char *
+show_CCA (const struct dataset *ds UNUSED)
+{
+ return show_cc (FMT_CCA);
+}
+
+static char *
+show_CCB (const struct dataset *ds UNUSED)
+{
+ return show_cc (FMT_CCB);
+}
+
+static char *
+show_CCC (const struct dataset *ds UNUSED)
+{
+ return show_cc (FMT_CCC);
+}
+
+static char *
+show_CCD (const struct dataset *ds UNUSED)
+{
+ return show_cc (FMT_CCD);
+}
+
+static char *
+show_CCE (const struct dataset *ds UNUSED)
+{
+ return show_cc (FMT_CCE);
+}
+
+static bool
+parse_CELLSBREAK (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "CELLSBREAK");
+}
+
+static bool
+parse_CMPTRANS (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "CMPTRANS");
+}
+
+static bool
+parse_COMPRESSION (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "COMPRESSION");
+}
+
+static bool
+parse_CTEMPLATE (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "CTEMPLATE");
+}
+
+static bool
+parse_DECIMAL (struct lexer *lexer)
+{
+ int decimal_char = force_parse_enum (lexer,
+ "DOT", '.',
+ "COMMA", ',');
+ if (decimal_char != -1)
+ settings_set_decimal_char (decimal_char);
+ return decimal_char != -1;
+}
+
+static char *
+show_DECIMAL (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("`%c'", settings_get_fmt_settings ()->decimal);
+}
+
+static bool
+parse_EPOCH (struct lexer *lexer)
+{
+ if (lex_match_id (lexer, "AUTOMATIC"))
+ settings_set_epoch (-1);
+ else if (lex_is_integer (lexer))
+ {
+ if (!lex_force_int_range (lexer, "EPOCH", 1500, INT_MAX))
+ return false;
+ settings_set_epoch (lex_integer (lexer));
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting %s or year."), "AUTOMATIC");
+ return false;
+ }
+
+ return true;
+}
+
+static char *
+show_EPOCH (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_epoch ());
+}
+
+static bool
+parse_ERRORS (struct lexer *lexer)
+{
+ return parse_output_routing (lexer, SETTINGS_OUTPUT_ERROR);
+}
+
+static char *
+show_ERRORS (const struct dataset *ds UNUSED)
+{
+ return show_output_routing (SETTINGS_OUTPUT_ERROR);
+}
+
+static bool
+parse_FORMAT (struct lexer *lexer)
+{
+ int start = subcommand_start_ofs (lexer);
+ struct fmt_spec fmt;
+
+ if (!parse_format_specifier (lexer, &fmt))
+ return false;
+
+ char *error = fmt_check_output__ (&fmt);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ return false;
+ }
+
+ int end = lex_ofs (lexer) - 1;
+ if (fmt_is_string (fmt.type))
+ {
+ char str[FMT_STRING_LEN_MAX + 1];
+ lex_ofs_error (lexer, start, end,
+ _("%s requires numeric output format as an argument. "
+ "Specified format %s is of type string."),
+ "FORMAT", fmt_to_string (&fmt, str));
+ return false;
+ }
+
+ settings_set_format (&fmt);
+ return true;
+}
+
+static char *
+show_FORMAT (const struct dataset *ds UNUSED)
+{
+ char str[FMT_STRING_LEN_MAX + 1];
+ return xstrdup (fmt_to_string (settings_get_format (), str));
+}
+
+static bool
+parse_FUZZBITS (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "FUZZBITS", 0, 20))
+ return false;
+ settings_set_fuzzbits (lex_integer (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_FUZZBITS (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_fuzzbits ());
+}
+
+static bool
+parse_HEADER (struct lexer *lexer)
+{
+ return parse_unimplemented (lexer, "HEADER");
+}
+
+static bool
+parse_INCLUDE (struct lexer *lexer)
+{
+ int include = force_parse_bool (lexer);
+ if (include != -1)
+ settings_set_include (include);
+ return include != -1;
+}
+
+static char *
+show_INCLUDE (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_include () ? "ON" : "OFF");
+}
+
+static bool
+parse_JOURNAL (struct lexer *lexer)
+{
+ int b = parse_bool (lexer);
+ if (b == true)
+ journal_enable ();
+ else if (b == false)
+ journal_disable ();
+ else if (lex_is_string (lexer) || lex_token (lexer) == T_ID)
+ {
+ char *filename = utf8_to_filename (lex_tokcstr (lexer));
+ journal_set_file_name (filename);
+ free (filename);
+
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting ON or OFF or a file name."));
+ return false;
+ }
+ return true;
+}
+
+static char *
+show_JOURNAL (const struct dataset *ds UNUSED)
+{
+ const char *enabled = journal_is_enabled () ? "ON" : "OFF";
+ const char *file_name = journal_get_file_name ();
+ return (file_name
+ ? xasprintf ("%s (%s)", enabled, file_name)
+ : xstrdup (enabled));
+}
+
+static bool
+parse_LEADZERO (struct lexer *lexer)
+{
+ int leadzero = force_parse_bool (lexer);
+ if (leadzero != -1)
+ settings_set_include_leading_zero (leadzero);
+ return leadzero != -1;
+}
+
+static char *
+show_LEADZERO (const struct dataset *ds UNUSED)
+{
+ bool leadzero = settings_get_fmt_settings ()->include_leading_zero;
+ return xstrdup (leadzero ? "ON" : "OFF");
+}
+
+static bool
+parse_LENGTH (struct lexer *lexer)
+{
+ int page_length;
+
+ if (lex_match_id (lexer, "NONE"))
+ page_length = -1;
+ else
+ {
+ if (!lex_force_int_range (lexer, "LENGTH", 1, INT_MAX))
+ return false;
+ page_length = lex_integer (lexer);
+ lex_get (lexer);
+ }
+
+ if (page_length != -1)
+ settings_set_viewlength (page_length);
+
+ return true;
+}
+
+static char *
+show_LENGTH (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_viewlength ());
+}
+
+static bool
+parse_LOCALE (struct lexer *lexer)
+{
+ if (!lex_force_string (lexer))
+ return false;
+
+ /* Try the argument as an encoding name, then as a locale name or alias. */
+ const char *s = lex_tokcstr (lexer);
+ if (valid_encoding (s))
+ set_default_encoding (s);
+ else if (!set_encoding_from_locale (s))
+ {
+ lex_error (lexer, _("%s is not a recognized encoding or locale name"), s);
+ return false;
+ }
+
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_LOCALE (const struct dataset *ds UNUSED)
+{
+ return xstrdup (get_default_encoding ());
+}
+
+static bool
+parse_MDISPLAY (struct lexer *lexer)
+{
+ int mdisplay = force_parse_enum (lexer,
+ "TEXT", SETTINGS_MDISPLAY_TEXT,
+ "TABLES", SETTINGS_MDISPLAY_TABLES);
+ if (mdisplay >= 0)
+ settings_set_mdisplay (mdisplay);
+ return mdisplay >= 0;
+}
+
+static char *
+show_MDISPLAY (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_mdisplay () == SETTINGS_MDISPLAY_TEXT
+ ? "TEXT" : "TABLES");
+}
+
+static bool
+parse_MESSAGES (struct lexer *lexer)
+{
+ return parse_output_routing (lexer, SETTINGS_OUTPUT_NOTE);
+}
+
+static char *
+show_MESSAGES (const struct dataset *ds UNUSED)
+{
+ return show_output_routing (SETTINGS_OUTPUT_NOTE);
+}
+
+static bool
+parse_MEXPAND (struct lexer *lexer)
+{
+ int mexpand = force_parse_bool (lexer);
+ if (mexpand != -1)
+ settings_set_mexpand (mexpand);
+ return mexpand != -1;
+}
+
+static char *
+show_MEXPAND (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_mexpand () ? "ON" : "OFF");
+}
+
+static bool
+parse_MITERATE (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "MITERATE", 1, INT_MAX))
+ return false;
+ settings_set_miterate (lex_integer (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_MITERATE (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_miterate ());
+}
+
+static bool
+parse_MNEST (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "MNEST", 1, INT_MAX))
+ return false;
+ settings_set_mnest (lex_integer (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_MNEST (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_mnest ());
+}
+
+static bool
+parse_MPRINT (struct lexer *lexer)
+{
+ int mprint = force_parse_bool (lexer);
+ if (mprint != -1)
+ settings_set_mprint (mprint);
+ return mprint != -1;
+}
+
+static char *
+show_MPRINT (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_mprint () ? "ON" : "OFF");
+}
+
+static bool
+parse_MXERRS (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "MXERRS", 1, INT_MAX))
+ return false;
+ settings_set_max_messages (MSG_S_ERROR, lex_integer (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_MXERRS (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_max_messages (MSG_S_ERROR));
+}
+
+static bool
+parse_MXLOOPS (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "MXLOOPS", 1, INT_MAX))
+ return false;
+ settings_set_mxloops (lex_integer (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_MXLOOPS (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_mxloops ());
+}
+
+static bool
+parse_MXWARNS (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "MXWARNS", 0, INT_MAX))
+ return false;
+ settings_set_max_messages (MSG_S_WARNING, lex_integer (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_MXWARNS (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_max_messages (MSG_S_WARNING));
+}
+
+static bool
+parse_PRINTBACK (struct lexer *lexer)
+{
+ return parse_output_routing (lexer, SETTINGS_OUTPUT_SYNTAX);
+}
+
+static char *
+show_PRINTBACK (const struct dataset *ds UNUSED)
+{
+ return show_output_routing (SETTINGS_OUTPUT_SYNTAX);
+}
+
+static bool
+parse_RESULTS (struct lexer *lexer)
+{
+ return parse_output_routing (lexer, SETTINGS_OUTPUT_RESULT);
+}
+
+static char *
+show_RESULTS (const struct dataset *ds UNUSED)
+{
+ return show_output_routing (SETTINGS_OUTPUT_RESULT);
+}
+
+static bool
+parse_RIB (struct lexer *lexer)
+{
+ return parse_integer_format (lexer, settings_set_input_integer_format);
+}
+
+static char *
+show_RIB (const struct dataset *ds UNUSED)
+{
+ return show_integer_format (settings_get_input_integer_format ());
+}
+
+static bool
+parse_RRB (struct lexer *lexer)
+{
+ return parse_real_format (lexer, settings_set_input_float_format);
+}
+
+static char *
+show_RRB (const struct dataset *ds UNUSED)
+{
+ return show_real_format (settings_get_input_float_format ());
+}
+
+static bool
+parse_SAFER (struct lexer *lexer)
+{
+ bool ok = force_parse_enum (lexer, "ON", true, "YES", true) != -1;
+ if (ok)
+ settings_set_safer_mode ();
+ return ok;
+}
+
+static char *
+show_SAFER (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_safer_mode () ? "ON" : "OFF");
+}
+
+static bool
+parse_SCOMPRESSION (struct lexer *lexer)
+{
+ int value = force_parse_bool (lexer);
+ if (value >= 0)
+ settings_set_scompression (value);
+ return value >= 0;
+}
+
+static char *
+show_SCOMPRESSION (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_scompression () ? "ON" : "OFF");
+}
+
+static bool
+parse_SEED (struct lexer *lexer)
+{
+ if (lex_match_id (lexer, "RANDOM"))
+ set_rng (time (0));
+ else
+ {
+ if (!lex_force_num (lexer))
+ return false;
+ set_rng (lex_number (lexer));
+ lex_get (lexer);
+ }
+
+ return true;
+}
+
+static bool
+parse_SMALL (struct lexer *lexer)
+{
+ if (!lex_force_num (lexer))
+ return false;
+ settings_set_small (lex_number (lexer));
+ lex_get (lexer);
+ return true;
+}
+
+static char *
+show_SMALL (const struct dataset *ds UNUSED)
+{
+ char buf[DBL_BUFSIZE_BOUND];
+ if (dtoastr (buf, sizeof buf, 0, 0, settings_get_small ()) < 0)
+ abort ();
+ return xstrdup (buf);
+}
+
+static char *
+show_SUBTITLE (const struct dataset *ds UNUSED)
+{
+ return xstrdup (output_get_subtitle ());
+}
+
+static char *
+show_TEMPDIR (const struct dataset *ds UNUSED)
+{
+ return xstrdup (temp_dir_name ());
+}
+
+static char *
+show_TITLE (const struct dataset *ds UNUSED)
+{
+ return xstrdup (output_get_title ());
+}
+
+static bool
+parse_TNUMBERS (struct lexer *lexer)
+{
+ int value = force_parse_enum (lexer,
+ "LABELS", SETTINGS_VALUE_SHOW_LABEL,
+ "VALUES", SETTINGS_VALUE_SHOW_VALUE,
+ "BOTH", SETTINGS_VALUE_SHOW_BOTH);
+ if (value >= 0)
+ settings_set_show_values (value);
+ return value >= 0;
+}
+
+static char *
+show_TNUMBERS (const struct dataset *ds UNUSED)
+{
+ enum settings_value_show tnumbers = settings_get_show_values ();
+ return xstrdup (tnumbers == SETTINGS_VALUE_SHOW_LABEL ? "LABELS"
+ : tnumbers == SETTINGS_VALUE_SHOW_VALUE ? "VALUES"
+ : "BOTH");
+}
+
+static bool
+parse_TVARS (struct lexer *lexer)
+{
+ int value = force_parse_enum (lexer,
+ "LABELS", SETTINGS_VALUE_SHOW_LABEL,
+ "NAMES", SETTINGS_VALUE_SHOW_VALUE,
+ "BOTH", SETTINGS_VALUE_SHOW_BOTH);
+ if (value >= 0)
+ settings_set_show_variables (value);
+ return value >= 0;
+}
+
+static char *
+show_TVARS (const struct dataset *ds UNUSED)
+{
+ enum settings_value_show tvars = settings_get_show_variables ();
+ return xstrdup (tvars == SETTINGS_VALUE_SHOW_LABEL ? "LABELS"
+ : tvars == SETTINGS_VALUE_SHOW_VALUE ? "NAMES"
+ : "BOTH");
+}
+
+static bool
+parse_TLOOK (struct lexer *lexer)
+{
+ if (lex_match_id (lexer, "NONE"))
+ pivot_table_look_set_default (pivot_table_look_builtin_default ());
+ else if (lex_is_string (lexer))
+ {
+ struct pivot_table_look *look;
+ char *error = pivot_table_look_read (lex_tokcstr (lexer), &look);
+ lex_get (lexer);
+
+ if (error)
+ {
+ msg (SE, "%s", error);
+ free (error);
+ return false;
+ }
+
+ pivot_table_look_set_default (look);
+ pivot_table_look_unref (look);
+ }
+
+ return true;
+}
+
+static bool
+parse_UNDEFINED (struct lexer *lexer)
+{
+ int value = force_parse_enum (lexer,
+ "WARN", true,
+ "NOWARN", false);
+ if (value >= 0)
+ settings_set_undefined (value);
+ return value >= 0;
+}
+
+static char *
+show_UNDEFINED (const struct dataset *ds UNUSED)
+{
+ return xstrdup (settings_get_undefined () ? "WARN" : "NOWARN");
+}
+
+static char *
+show_VERSION (const struct dataset *ds UNUSED)
+{
+ return strdup (announced_version);
+}
+
+static char *
+show_WEIGHT (const struct dataset *ds)
+{
+ const struct variable *var = dict_get_weight (dataset_dict (ds));
+ return xstrdup (var != NULL ? var_get_name (var) : "OFF");
+}
+
+static bool
+parse_WIB (struct lexer *lexer)
+{
+ return parse_integer_format (lexer, settings_set_output_integer_format);
+}
+
+static char *
+show_WIB (const struct dataset *ds UNUSED)
+{
+ return show_integer_format (settings_get_output_integer_format ());
+}
+
+static bool
+parse_WRB (struct lexer *lexer)
+{
+ return parse_real_format (lexer, settings_set_output_float_format);
+}
+
+static char *
+show_WRB (const struct dataset *ds UNUSED)
+{
+ return show_real_format (settings_get_output_float_format ());
+}
+
+static bool
+parse_WIDTH (struct lexer *lexer)
+{
+ if (lex_match_id (lexer, "NARROW"))
+ settings_set_viewwidth (79);
+ else if (lex_match_id (lexer, "WIDE"))
+ settings_set_viewwidth (131);
+ else
+ {
+ if (!lex_force_int_range (lexer, "WIDTH", 40, INT_MAX))
+ return false;
+ settings_set_viewwidth (lex_integer (lexer));
+ lex_get (lexer);
+ }
+
+ return true;
+}
+
+static char *
+show_WIDTH (const struct dataset *ds UNUSED)
+{
+ return xasprintf ("%d", settings_get_viewwidth ());
+}
+
+static bool
+parse_WORKSPACE (struct lexer *lexer)
+{
+ if (!lex_force_int_range (lexer, "WORKSPACE",
+ settings_get_testing_mode () ? 1 : 1024,
+ INT_MAX))
+ return false;
+ int workspace = lex_integer (lexer);
+ lex_get (lexer);
+ settings_set_workspace (MIN (workspace, INT_MAX / 1024) * 1024);
+ return true;
+}
+
+static char *
+show_WORKSPACE (const struct dataset *ds UNUSED)
+{
+ size_t ws = settings_get_workspace () / 1024L;
+ return xasprintf ("%zu", ws);
+}
+\f
+static char *
+show_DIRECTORY (const struct dataset *ds UNUSED)
+{
+ char *buf = NULL;
+ char *wd = NULL;
+ size_t len = 256;
+
+ do
+ {
+ len <<= 1;
+ buf = xrealloc (buf, len);
+ }
+ while (NULL == (wd = getcwd (buf, len)));
+
+ return wd;
+}
+
+static char *
+show_N (const struct dataset *ds)
+{
+ const struct casereader *reader = dataset_source (ds);
+ return (reader
+ ? xasprintf ("%lld", (long long int) casereader_count_cases (reader))
+ : xstrdup (_("Unknown")));
+}
+
+static void
+do_show (const struct dataset *ds, const struct setting *s,
+ struct pivot_table **ptp)
+{
+ struct pivot_table *pt = *ptp;
+ if (!pt)
+ {
+ pt = *ptp = pivot_table_create (N_("Settings"));
+ pivot_dimension_create (pt, PIVOT_AXIS_ROW, N_("Setting"));
+ }
+
+ struct pivot_value *name = pivot_value_new_user_text (s->name, SIZE_MAX);
+ char *text = s->show (ds);
+ if (!text)
+ text = xstrdup("empty");
+ struct pivot_value *value = pivot_value_new_user_text_nocopy (text);
+
+ int row = pivot_category_create_leaf (pt->dimensions[0]->root, name);
+ pivot_table_put1 (pt, row, value);
+}
+
+static void
+show_warranty (const struct dataset *ds UNUSED)
+{
+ fputs (lack_of_warranty, stdout);
+}
+
+static void
+show_copying (const struct dataset *ds UNUSED)
+{
+ fputs (copyleft, stdout);
+}
+
+static void
+add_row (struct pivot_table *table, const char *attribute,
+ const char *value)
+{
+ int row = pivot_category_create_leaf (table->dimensions[0]->root,
+ pivot_value_new_text (attribute));
+ if (value)
+ pivot_table_put1 (table, row, pivot_value_new_user_text (value, -1));
+}
+
+static void
+show_system (const struct dataset *ds UNUSED)
+{
+ struct pivot_table *table = pivot_table_create (N_("System Information"));
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Attribute"));
+
+ add_row (table, N_("Version"), version);
+ add_row (table, N_("Host System"), host_system);
+ add_row (table, N_("Build System"), build_system);
+ add_row (table, N_("Locale Directory"), relocate (locale_dir));
+ add_row (table, N_("Compiler Version"),
+#ifdef __VERSION__
+ __VERSION__
+#else
+ "Unknown"
+#endif
+ );
+ pivot_table_submit (table);
+}
+\f
+static const struct setting settings[] = {
+ { "BASETEXTDIRECTION", parse_BASETEXTDIRECTION, NULL },
+ { "BLANKS", parse_BLANKS, show_BLANKS },
+ { "BLOCK", parse_BLOCK, NULL },
+ { "BOX", parse_BOX, NULL },
+ { "CACHE", parse_CACHE, NULL },
+ { "CCA", parse_CCA, show_CCA },
+ { "CCB", parse_CCB, show_CCB },
+ { "CCC", parse_CCC, show_CCC },
+ { "CCD", parse_CCD, show_CCD },
+ { "CCE", parse_CCE, show_CCE },
+ { "CELLSBREAK", parse_CELLSBREAK, NULL },
+ { "CMPTRANS", parse_CMPTRANS, NULL },
+ { "COMPRESSION", parse_COMPRESSION, NULL },
+ { "CTEMPLATE", parse_CTEMPLATE, NULL },
+ { "DECIMAL", parse_DECIMAL, show_DECIMAL },
+ { "DIRECTORY", NULL, show_DIRECTORY },
+ { "EPOCH", parse_EPOCH, show_EPOCH },
+ { "ERRORS", parse_ERRORS, show_ERRORS },
+ { "FORMAT", parse_FORMAT, show_FORMAT },
+ { "FUZZBITS", parse_FUZZBITS, show_FUZZBITS },
+ { "HEADER", parse_HEADER, NULL },
+ { "INCLUDE", parse_INCLUDE, show_INCLUDE },
+ { "JOURNAL", parse_JOURNAL, show_JOURNAL },
+ { "LEADZERO", parse_LEADZERO, show_LEADZERO },
+ { "LENGTH", parse_LENGTH, show_LENGTH },
+ { "LOCALE", parse_LOCALE, show_LOCALE },
+ { "MDISPLAY", parse_MDISPLAY, show_MDISPLAY },
+ { "MESSAGES", parse_MESSAGES, show_MESSAGES },
+ { "MEXPAND", parse_MEXPAND, show_MEXPAND },
+ { "MITERATE", parse_MITERATE, show_MITERATE },
+ { "MNEST", parse_MNEST, show_MNEST },
+ { "MPRINT", parse_MPRINT, show_MPRINT },
+ { "MXERRS", parse_MXERRS, show_MXERRS },
+ { "MXLOOPS", parse_MXLOOPS, show_MXLOOPS },
+ { "MXWARNS", parse_MXWARNS, show_MXWARNS },
+ { "N", NULL, show_N },
+ { "PRINTBACK", parse_PRINTBACK, show_PRINTBACK },
+ { "RESULTS", parse_RESULTS, show_RESULTS },
+ { "RIB", parse_RIB, show_RIB },
+ { "RRB", parse_RRB, show_RRB },
+ { "SAFER", parse_SAFER, show_SAFER },
+ { "SCOMPRESSION", parse_SCOMPRESSION, show_SCOMPRESSION },
+ { "SEED", parse_SEED, NULL },
+ { "SMALL", parse_SMALL, show_SMALL },
+ { "TEMPDIR", NULL, show_TEMPDIR },
+ { "TNUMBERS", parse_TNUMBERS, show_TNUMBERS },
+ { "TVARS", parse_TVARS, show_TVARS },
+ { "TLOOK", parse_TLOOK, NULL },
+ { "UNDEFINED", parse_UNDEFINED, show_UNDEFINED },
+ { "VERSION", NULL, show_VERSION },
+ { "WEIGHT", NULL, show_WEIGHT },
+ { "WIB", parse_WIB, show_WIB },
+ { "WRB", parse_WRB, show_WRB },
+ { "WIDTH", parse_WIDTH, show_WIDTH },
+ { "WORKSPACE", parse_WORKSPACE, show_WORKSPACE },
+};
+enum { N_SETTINGS = sizeof settings / sizeof *settings };
+
+static bool
+parse_setting (struct lexer *lexer)
+{
+ for (size_t i = 0; i < N_SETTINGS; i++)
+ if (settings[i].set && match_subcommand (lexer, settings[i].name))
+ return settings[i].set (lexer);
+
+ lex_error (lexer, _("Syntax error expecting the name of a setting."));
+ return false;
+}
+
+int
+cmd_set (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ for (;;)
+ {
+ lex_match (lexer, T_SLASH);
+ if (lex_token (lexer) == T_ENDCMD)
+ break;
+
+ if (!parse_setting (lexer))
+ return CMD_FAILURE;
+ }
+
+ return CMD_SUCCESS;
+}
+
+static void
+show_all (const struct dataset *ds, struct pivot_table **ptp)
+{
+ for (size_t i = 0; i < sizeof settings / sizeof *settings; i++)
+ if (settings[i].show)
+ do_show (ds, &settings[i], ptp);
+}
+
+static void
+show_all_cc (const struct dataset *ds, struct pivot_table **ptp)
+{
+ for (size_t i = 0; i < sizeof settings / sizeof *settings; i++)
+ {
+ const struct setting *s = &settings[i];
+ if (s->show && !strncmp (s->name, "CC", 2))
+ do_show (ds, s, ptp);
+ }
+}
+
+int
+cmd_show (struct lexer *lexer, struct dataset *ds)
+{
+ struct pivot_table *pt = NULL;
+ if (lex_token (lexer) == T_ENDCMD)
+ {
+ show_all (ds, &pt);
+ pivot_table_submit (pt);
+ return CMD_SUCCESS;
+ }
+
+ do
+ {
+ if (lex_match (lexer, T_ALL))
+ show_all (ds, &pt);
+ else if (lex_match_id (lexer, "CC"))
+ show_all_cc (ds, &pt);
+ else if (lex_match_id (lexer, "WARRANTY"))
+ show_warranty (ds);
+ else if (lex_match_id (lexer, "COPYING") || lex_match_id (lexer, "LICENSE"))
+ show_copying (ds);
+ else if (lex_match_id (lexer, "SYSTEM"))
+ show_system (ds);
+ else if (lex_match_id (lexer, "TITLE"))
+ {
+ struct setting s = { .name = "TITLE", .show = show_TITLE };
+ do_show (ds, &s, &pt);
+ }
+ else if (lex_match_id (lexer, "SUBTITLE"))
+ {
+ struct setting s = { .name = "SUBTITLE", .show = show_SUBTITLE };
+ do_show (ds, &s, &pt);
+ }
+ else if (lex_token (lexer) == T_ID)
+ {
+ for (size_t i = 0; i < sizeof settings / sizeof *settings; i++)
+ {
+ const struct setting *s = &settings[i];
+ if (s->show && lex_match_id (lexer, s->name))
+ {
+ do_show (ds, s, &pt);
+ goto found;
+ }
+ }
+ lex_error (lexer, _("Syntax error expecting the name of a setting."));
+ return CMD_FAILURE;
+
+ found: ;
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting the name of a setting."));
+ return CMD_FAILURE;
+ }
+
+ lex_match (lexer, T_SLASH);
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ if (pt)
+ pivot_table_submit (pt);
+
+ return CMD_SUCCESS;
+}
+\f
+#define MAX_SAVED_SETTINGS 5
+
+static struct settings *saved_settings[MAX_SAVED_SETTINGS];
+static int n_saved_settings;
+
+int
+cmd_preserve (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ if (n_saved_settings < MAX_SAVED_SETTINGS)
+ {
+ saved_settings[n_saved_settings++] = settings_get ();
+ return CMD_SUCCESS;
+ }
+ else
+ {
+ lex_next_error (lexer, -1, -1,
+ _("Too many %s commands without a %s: at most "
+ "%d levels of saved settings are allowed."),
+ "PRESERVE", "RESTORE",
+ MAX_SAVED_SETTINGS);
+ return CMD_CASCADING_FAILURE;
+ }
+}
+
+int
+cmd_restore (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ if (n_saved_settings > 0)
+ {
+ struct settings *s = saved_settings[--n_saved_settings];
+ settings_set (s);
+ settings_destroy (s);
+ return CMD_SUCCESS;
+ }
+ else
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s without matching %s."), "RESTORE", "PRESERVE");
+ return CMD_FAILURE;
+ }
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/sign.h"
+
+#include <gsl/gsl_cdf.h>
+#include <gsl/gsl_randist.h>
+
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "data/variable.h"
+#include "language/commands/npar.h"
+#include "libpspp/str.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+struct sign_test_params
+{
+ double pos;
+ double ties;
+ double neg;
+
+ double one_tailed_sig;
+ double point_prob;
+};
+
+static int
+add_pair_leaf (struct pivot_dimension *dimension, variable_pair *pair)
+{
+ char *label = xasprintf ("%s - %s", var_to_string ((*pair)[0]),
+ var_to_string ((*pair)[1]));
+ return pivot_category_create_leaf (
+ dimension->root,
+ pivot_value_new_user_text_nocopy (label));
+}
+
+static void
+output_frequency_table (const struct two_sample_test *t2s,
+ const struct sign_test_params *param,
+ const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (N_("Frequencies"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("N"),
+ N_("N"), PIVOT_RC_COUNT);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Differences"),
+ N_("Negative Differences"),
+ N_("Positive Differences"),
+ N_("Ties"), N_("Total"));
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Pairs"));
+
+ for (size_t i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+
+ int pair_idx = add_pair_leaf (pairs, vp);
+
+ const struct sign_test_params *p = ¶m[i];
+ double values[] = { p->neg, p->pos, p->ties, p->ties + p->neg + p->pos };
+ for (size_t j = 0; j < sizeof values / sizeof *values; j++)
+ pivot_table_put3 (table, 0, j, pair_idx,
+ pivot_value_new_number (values[j]));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+output_statistics_table (const struct two_sample_test *t2s,
+ const struct sign_test_params *param)
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Point Probability"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Pairs"));
+
+ for (size_t i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+ int pair_idx = add_pair_leaf (pairs, vp);
+
+ const struct sign_test_params *p = ¶m[i];
+ double values[] = { p->one_tailed_sig * 2,
+ p->one_tailed_sig,
+ p->point_prob };
+ for (size_t j = 0; j < sizeof values / sizeof *values; j++)
+ pivot_table_put2 (table, j, pair_idx,
+ pivot_value_new_number (values[j]));
+ }
+
+ pivot_table_submit (table);
+}
+
+void
+sign_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact UNUSED,
+ double timer UNUSED)
+{
+ int i;
+ bool warn = true;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct two_sample_test *t2s = UP_CAST (test, const struct two_sample_test, parent);
+ struct ccase *c;
+
+ struct sign_test_params *stp = XCALLOC (t2s->n_pairs, struct sign_test_params);
+
+ struct casereader *r = input;
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ const double weight = dict_get_case_weight (dict, c, &warn);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+ const union value *value0 = case_data (c, (*vp)[0]);
+ const union value *value1 = case_data (c, (*vp)[1]);
+ const double diff = value0->f - value1->f;
+
+ if (var_is_value_missing ((*vp)[0], value0) & exclude)
+ continue;
+
+ if (var_is_value_missing ((*vp)[1], value1) & exclude)
+ continue;
+
+ if (diff > 0)
+ stp[i].pos += weight;
+ else if (diff < 0)
+ stp[i].neg += weight;
+ else
+ stp[i].ties += weight;
+ }
+ }
+
+ casereader_destroy (r);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ int r = MIN (stp[i].pos, stp[i].neg);
+ stp[i].one_tailed_sig = gsl_cdf_binomial_P (r,
+ 0.5,
+ stp[i].pos + stp[i].neg);
+
+ stp[i].point_prob = gsl_ran_binomial_pdf (r, 0.5,
+ stp[i].pos + stp[i].neg);
+ }
+
+ output_frequency_table (t2s, stp, dict);
+
+ output_statistics_table (t2s, stp);
+
+ free (stp);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2009, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !sign_h
+#define sign_h 1
+
+
+#include <stdbool.h>
+#include "data/missing-values.h"
+
+struct casereader;
+struct dataset;
+struct npar_test;
+
+void sign_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer);
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <limits.h>
+#include <stdlib.h>
+#include <sys/types.h>
+
+#include "data/dataset.h"
+#include "data/settings.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/commands/sort-criteria.h"
+#include "libpspp/message.h"
+#include "math/sort.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+
+/* Performs the SORT CASES procedures. */
+int
+cmd_sort_cases (struct lexer *lexer, struct dataset *ds)
+{
+ struct subcase ordering = SUBCASE_EMPTY_INITIALIZER;
+ bool ok = false;
+
+ lex_match (lexer, T_BY);
+
+ proc_cancel_temporary_transformations (ds);
+ if (!parse_sort_criteria (lexer, dataset_dict (ds), &ordering, NULL, NULL))
+ return CMD_CASCADING_FAILURE;
+
+ if (settings_get_testing_mode () && lex_match (lexer, T_SLASH))
+ {
+ if (!lex_force_match_id (lexer, "BUFFERS"))
+ goto done;
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_int_range (lexer, "BUFFERS", 2, INT_MAX))
+ goto done;
+ min_buffers = max_buffers = lex_integer (lexer);
+ lex_get (lexer);
+ }
+
+ proc_discard_output (ds);
+ struct casereader *output = sort_execute (proc_open_filtering (ds, false),
+ &ordering);
+ ok = proc_commit (ds);
+ ok = dataset_set_source (ds, output) && ok;
+
+ done:
+ min_buffers = 64;
+ max_buffers = INT_MAX;
+
+ subcase_uninit (&ordering);
+ return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/sort-criteria.h"
+
+#include <stdlib.h>
+
+#include "data/dictionary.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Parses a list of sort fields and appends them to ORDERING,
+ which the caller must already have initialized.
+ Returns true if successful, false on error.
+ If SAW_DIRECTION is nonnull, sets *SAW_DIRECTION to true if at
+ least one parenthesized sort direction was specified, false
+ otherwise. */
+bool
+parse_sort_criteria (struct lexer *lexer, const struct dictionary *dict,
+ struct subcase *ordering,
+ const struct variable ***vars, bool *saw_direction)
+{
+ const struct variable **local_vars = NULL;
+ size_t n_vars = 0;
+
+ if (vars == NULL)
+ vars = &local_vars;
+ *vars = NULL;
+
+ if (saw_direction != NULL)
+ *saw_direction = false;
+
+ int start_ofs = lex_ofs (lexer);
+ do
+ {
+ size_t prev_n_vars = n_vars;
+
+ /* Variables. */
+ if (!parse_variables_const (lexer, dict, vars, &n_vars,
+ PV_APPEND | PV_DUPLICATE | PV_NO_SCRATCH))
+ goto error;
+
+ /* Sort direction. */
+ enum subcase_direction direction;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (lex_match_id (lexer, "D") || lex_match_id (lexer, "DOWN"))
+ direction = SC_DESCEND;
+ else if (lex_match_id (lexer, "A") || lex_match_id (lexer, "UP"))
+ direction = SC_ASCEND;
+ else
+ {
+ lex_error_expecting (lexer, "A", "D");
+ goto error;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+ if (saw_direction != NULL)
+ *saw_direction = true;
+ }
+ else
+ direction = SC_ASCEND;
+
+ for (size_t i = prev_n_vars; i < n_vars; i++)
+ {
+ const struct variable *var = (*vars)[i];
+ if (!subcase_add_var (ordering, var, direction))
+ lex_ofs_msg (lexer, SW, start_ofs, lex_ofs (lexer) - 1,
+ _("Variable %s specified twice in sort criteria."),
+ var_get_name (var));
+ }
+ }
+ while (lex_token (lexer) == T_ID
+ && dict_lookup_var (dict, lex_tokcstr (lexer)) != NULL);
+
+ free (local_vars);
+ return true;
+
+error:
+ subcase_uninit (ordering);
+ subcase_init_empty (ordering);
+ free (local_vars);
+ if (vars)
+ *vars = NULL;
+ return false;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef SORT_CRITERIA_H
+#define SORT_CRITERIA_H
+
+#include <stdbool.h>
+#include <stddef.h>
+
+struct dictionary;
+struct lexer;
+struct variable;
+struct subcase;
+
+bool parse_sort_criteria (struct lexer *, const struct dictionary *,
+ struct subcase *, const struct variable ***vars,
+ bool *saw_direction);
+
+
+#endif /* sort-criteria.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/attributes.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/array.h"
+#include "libpspp/assertion.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+enum key
+ {
+ K_NAME,
+ K_TYPE,
+ K_FORMAT,
+ K_VAR_LABEL,
+ K_VALUE_LABELS,
+ K_MISSING_VALUES,
+ K_MEASURE,
+ K_ROLE,
+ K_COLUMNS,
+ K_ALIGNMENT,
+ K_ATTRIBUTE,
+ };
+
+struct criterion
+ {
+ enum key key;
+ char *attr_name;
+ bool descending;
+ };
+
+static int
+compare_ints (int a, int b)
+{
+ return a < b ? -1 : a > b;
+}
+
+static int
+compare_formats (const struct fmt_spec *a, const struct fmt_spec *b)
+{
+ int retval = compare_ints (fmt_to_io (a->type), fmt_to_io (b->type));
+ if (!retval)
+ retval = compare_ints (a->w, b->w);
+ if (!retval)
+ retval = compare_ints (a->d, b->d);
+ return retval;
+}
+
+static int
+compare_var_labels (const struct variable *a, const struct variable *b)
+{
+ const char *a_label = var_get_label (a);
+ const char *b_label = var_get_label (b);
+ return utf8_strcasecmp (a_label ? a_label : "",
+ b_label ? b_label : "");
+}
+
+static int
+map_measure (enum measure m)
+{
+ return (m == MEASURE_NOMINAL ? 0
+ : m == MEASURE_ORDINAL ? 1
+ : 2);
+}
+
+static int
+map_role (enum var_role r)
+{
+ return (r == ROLE_INPUT ? 0
+ : r == ROLE_TARGET ? 1
+ : r == ROLE_BOTH ? 2
+ : r == ROLE_NONE ? 3
+ : r == ROLE_PARTITION ? 4
+ : 5);
+}
+
+static const char *
+get_attribute (const struct variable *v, const char *name)
+{
+ const struct attrset *set = var_get_attributes (v);
+ const struct attribute *attr = attrset_lookup (set, name);
+ const char *value = attr ? attribute_get_value (attr, 0) : NULL;
+ return value ? value : "";
+}
+
+static int
+map_alignment (enum alignment a)
+{
+ return (a == ALIGN_LEFT ? 0
+ : a == ALIGN_RIGHT ? 1
+ : 2);
+}
+
+static int
+compare_vars (const void *a_, const void *b_, const void *c_)
+{
+ const struct variable *const *ap = a_;
+ const struct variable *const *bp = b_;
+ const struct variable *a = *ap;
+ const struct variable *b = *bp;
+ const struct criterion *c = c_;
+
+ int retval;
+ switch (c->key)
+ {
+ case K_NAME:
+ retval = utf8_strverscasecmp (var_get_name (a), var_get_name (b));
+ break;
+
+ case K_TYPE:
+ retval = compare_ints (var_get_width (a), var_get_width (b));
+ break;
+
+ case K_FORMAT:
+ retval = compare_formats (var_get_print_format (a),
+ var_get_print_format (b));
+ break;
+
+ case K_VAR_LABEL:
+ retval = compare_var_labels (a, b);
+ break;
+
+ case K_VALUE_LABELS:
+ retval = compare_ints (var_has_value_labels (a),
+ var_has_value_labels (b));
+ break;
+
+ case K_MISSING_VALUES:
+ retval = compare_ints (var_has_missing_values (a),
+ var_has_missing_values (b));
+ break;
+
+ case K_MEASURE:
+ retval = compare_ints (map_measure (var_get_measure (a)),
+ map_measure (var_get_measure (b)));
+ break;
+
+ case K_ROLE:
+ retval = compare_ints (map_role (var_get_role (a)),
+ map_role (var_get_role (b)));
+ break;
+
+ case K_COLUMNS:
+ retval = compare_ints (var_get_display_width (a),
+ var_get_display_width (b));
+ break;
+
+ case K_ALIGNMENT:
+ retval = compare_ints (map_alignment (var_get_alignment (a)),
+ map_alignment (var_get_alignment (b)));
+ break;
+
+ case K_ATTRIBUTE:
+ retval = utf8_strcasecmp (get_attribute (a, c->attr_name),
+ get_attribute (b, c->attr_name));
+ break;
+
+ default:
+ NOT_REACHED ();
+ }
+
+ /* Make this a stable sort. */
+ if (!retval)
+ {
+ size_t a_index = var_get_dict_index (a);
+ size_t b_index = var_get_dict_index (b);
+ retval = a_index < b_index ? -1 : a_index > b_index;
+ }
+
+ if (c->descending)
+ retval = -retval;
+
+ return retval;
+}
+
+/* Performs SORT VARIABLES command. */
+int
+cmd_sort_variables (struct lexer *lexer, struct dataset *ds)
+{
+ enum cmd_result result = CMD_FAILURE;
+
+ lex_match (lexer, T_BY);
+
+ /* Parse sort key. */
+ struct criterion c = { .attr_name = NULL };
+ if (lex_match_id (lexer, "NAME"))
+ c.key = K_NAME;
+ else if (lex_match_id (lexer, "TYPE"))
+ c.key = K_TYPE;
+ else if (lex_match_id (lexer, "FORMAT"))
+ c.key = K_FORMAT;
+ else if (lex_match_id (lexer, "LABEL"))
+ c.key = K_VAR_LABEL;
+ else if (lex_match_id (lexer, "VALUES"))
+ c.key = K_VALUE_LABELS;
+ else if (lex_match_id (lexer, "MISSING"))
+ c.key = K_MISSING_VALUES;
+ else if (lex_match_id (lexer, "MEASURE"))
+ c.key = K_MEASURE;
+ else if (lex_match_id (lexer, "ROLE"))
+ c.key = K_ROLE;
+ else if (lex_match_id (lexer, "COLUMNS"))
+ c.key = K_COLUMNS;
+ else if (lex_match_id (lexer, "ALIGNMENT"))
+ c.key = K_ALIGNMENT;
+ else if (lex_match_id (lexer, "ATTRIBUTE"))
+ {
+ if (!lex_force_id (lexer))
+ goto exit;
+ c.key = K_ATTRIBUTE;
+ c.attr_name = xstrdup (lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ else
+ {
+ lex_error_expecting (lexer, "NAME", "TYPE", "FORMAT", "LABEL",
+ "VALUES", "MISSING", "MEASURE", "ROLE",
+ "COLUMNS", "ALIGNMENT", "ATTRIBUTE");
+ return CMD_FAILURE;
+ }
+
+ /* Parse sort direction. */
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (lex_match_id (lexer, "A") || lex_match_id (lexer, "UP"))
+ c.descending = false;
+ else if (lex_match_id (lexer, "D") || lex_match_id (lexer, "DOWN"))
+ c.descending = true;
+ else
+ {
+ lex_error_expecting (lexer, "A", "D");
+ goto exit;
+ }
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto exit;
+ }
+ else
+ c.descending = false;
+
+ /* Sort variables. */
+ struct dictionary *d = dataset_dict (ds);
+ struct variable **vars;
+ size_t n_vars;
+ dict_get_vars_mutable (d, &vars, &n_vars, 0);
+ sort (vars, n_vars, sizeof *vars, compare_vars, &c);
+ dict_reorder_vars (d, CONST_CAST (struct variable *const *, vars), n_vars);
+ free (vars);
+
+ result = CMD_SUCCESS;
+
+exit:
+ free (c.attr_name);
+ return result;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/case.h"
+#include "data/casereader.h"
+#include "data/data-out.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/commands/split-file.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+#include "output/pivot-table.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+int
+cmd_split_file (struct lexer *lexer, struct dataset *ds)
+{
+ if (lex_match_id (lexer, "OFF"))
+ dict_clear_split_vars (dataset_dict (ds));
+ else
+ {
+ struct variable **v;
+ size_t n;
+
+ enum split_type type = (!lex_match_id (lexer, "LAYERED")
+ && lex_match_id (lexer, "SEPARATE")
+ ? SPLIT_SEPARATE
+ : SPLIT_LAYERED);
+
+ lex_match (lexer, T_BY);
+ int vars_start = lex_ofs (lexer);
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &n, PV_NO_DUPLICATE))
+ return CMD_CASCADING_FAILURE;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ if (n > MAX_SPLITS)
+ {
+ verify (MAX_SPLITS == 8);
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("At most 8 split variables may be specified."));
+ free (v);
+ return CMD_CASCADING_FAILURE;
+ }
+
+ dict_set_split_vars (dataset_dict (ds), v, n, type);
+ free (v);
+ }
+
+ return CMD_SUCCESS;
+}
+
+/* Dumps out the values of all the split variables for the case C. */
+void
+output_split_file_values (const struct dataset *ds, const struct ccase *c)
+{
+ const struct dictionary *dict = dataset_dict (ds);
+ size_t n_vars = dict_get_n_splits (dict);
+ if (n_vars == 0)
+ return;
+
+ struct pivot_table *table = pivot_table_create (N_("Split Values"));
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Value"),
+ N_("Value"));
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable"));
+ variables->root->show_label = true;
+
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ const struct variable *v = dict_get_split_vars (dict)[i];
+ int row = pivot_category_create_leaf (variables->root,
+ pivot_value_new_variable (v));
+
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_var_value (v, case_data (c, v)));
+ }
+
+ pivot_table_submit (table);
+}
+
+/* Dumps out the values of all the split variables for the first case in
+ READER. */
+void
+output_split_file_values_peek (const struct dataset *ds,
+ const struct casereader *reader)
+{
+ struct ccase *c = casereader_peek (reader, 0);
+ if (c)
+ {
+ output_split_file_values (ds, c);
+ case_unref (c);
+ }
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2006 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef SPLIT_FILE_H
+#define SPLIT_FILE_H 1
+
+void output_split_file_values (const struct dataset *, const struct ccase *);
+void output_split_file_values_peek (const struct dataset *,
+ const struct casereader *);
+
+#endif /* split-file.h */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+#include <config.h>
+
+#include <ctype.h>
+#include <errno.h>
+#include <float.h>
+#include <stdlib.h>
+
+#include "data/any-reader.h"
+#include "data/attributes.h"
+#include "data/casereader.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/file-handle-def.h"
+#include "data/format.h"
+#include "data/missing-values.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "data/vector.h"
+#include "language/command.h"
+#include "language/commands/file-handle.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/array.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/string-array.h"
+#include "output/pivot-table.h"
+#include "output/output-item.h"
+
+#include "gl/count-one-bits.h"
+#include "gl/localcharset.h"
+#include "gl/intprops.h"
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+#define N_(msgid) (msgid)
+
+/* Information to include in displaying a dictionary. */
+enum
+ {
+ /* Variable table. */
+ DF_NAME = 1 << 0,
+ DF_POSITION = 1 << 1,
+ DF_LABEL = 1 << 2,
+ DF_MEASUREMENT_LEVEL = 1 << 3,
+ DF_ROLE = 1 << 4,
+ DF_WIDTH = 1 << 5,
+ DF_ALIGNMENT = 1 << 6,
+ DF_PRINT_FORMAT = 1 << 7,
+ DF_WRITE_FORMAT = 1 << 8,
+ DF_MISSING_VALUES = 1 << 9,
+#define DF_ALL_VARIABLE ((1 << 10) - 1)
+
+ /* Value labels table. */
+ DF_VALUE_LABELS = 1 << 10,
+
+ /* Attribute table. */
+ DF_AT_ATTRIBUTES = 1 << 11, /* Attributes whose names begin with @. */
+ DF_ATTRIBUTES = 1 << 12, /* All other attributes. */
+ };
+
+static void display_variables (const struct variable **, size_t, int flags);
+static void display_value_labels (const struct variable **, size_t);
+static void display_attributes (const struct attrset *,
+ const struct variable **, size_t, int flags);
+
+static void report_encodings (const struct file_handle *, struct pool *,
+ char **titles, bool *ids,
+ char **strings, size_t n_strings);
+
+static char *get_documents_as_string (const struct dictionary *);
+
+static void
+add_row (struct pivot_table *table, const char *attribute,
+ struct pivot_value *value)
+{
+ int row = pivot_category_create_leaf (table->dimensions[0]->root,
+ pivot_value_new_text (attribute));
+ if (value)
+ pivot_table_put1 (table, row, value);
+}
+
+/* SYSFILE INFO utility. */
+int
+cmd_sysfile_info (struct lexer *lexer, struct dataset *ds)
+{
+ struct any_reader *any_reader;
+ struct file_handle *h;
+ struct dictionary *d;
+ struct casereader *reader;
+ struct any_read_info info;
+ char *encoding;
+
+ h = NULL;
+ encoding = NULL;
+ for (;;)
+ {
+ lex_match (lexer, T_SLASH);
+
+ if (lex_match_id (lexer, "FILE") || lex_is_string (lexer))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ fh_unref (h);
+ h = fh_parse (lexer, FH_REF_FILE, NULL);
+ if (h == NULL)
+ goto error;
+ }
+ else if (lex_match_id (lexer, "ENCODING"))
+ {
+ lex_match (lexer, T_EQUALS);
+
+ if (!lex_force_string (lexer))
+ goto error;
+
+ free (encoding);
+ encoding = ss_xstrdup (lex_tokss (lexer));
+
+ lex_get (lexer);
+ }
+ else
+ break;
+ }
+
+ if (h == NULL)
+ {
+ lex_sbc_missing (lexer, "FILE");
+ goto error;
+ }
+
+ any_reader = any_reader_open (h);
+ if (!any_reader)
+ {
+ free (encoding);
+ return CMD_FAILURE;
+ }
+
+ if (encoding && !strcasecmp (encoding, "detect"))
+ {
+ char **titles, **strings;
+ struct pool *pool;
+ size_t n_strings;
+ bool *ids;
+
+ pool = pool_create ();
+ n_strings = any_reader_get_strings (any_reader, pool,
+ &titles, &ids, &strings);
+ any_reader_close (any_reader);
+
+ report_encodings (h, pool, titles, ids, strings, n_strings);
+ fh_unref (h);
+ pool_destroy (pool);
+ free (encoding);
+
+ return CMD_SUCCESS;
+ }
+
+ reader = any_reader_decode (any_reader, encoding, &d, &info);
+ if (!reader)
+ goto error;
+ casereader_destroy (reader);
+
+ struct pivot_table *table = pivot_table_create (N_("File Information"));
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Attribute"));
+
+ add_row (table, N_("File"),
+ pivot_value_new_user_text (fh_get_file_name (h), -1));
+
+ const char *label = dict_get_label (d);
+ add_row (table, N_("Label"),
+ label ? pivot_value_new_user_text (label, -1) : NULL);
+
+ add_row (table, N_("Created"),
+ pivot_value_new_user_text_nocopy (
+ xasprintf ("%s %s by %s", info.creation_date,
+ info.creation_time, info.product)));
+
+ if (info.product_ext)
+ add_row (table, N_("Product"),
+ pivot_value_new_user_text (info.product_ext, -1));
+
+ add_row (table, N_("Integer Format"),
+ pivot_value_new_text (
+ info.integer_format == INTEGER_MSB_FIRST ? N_("Big Endian")
+ : info.integer_format == INTEGER_LSB_FIRST ? N_("Little Endian")
+ : N_("Unknown")));
+
+ add_row (table, N_("Real Format"),
+ pivot_value_new_text (
+ info.float_format == FLOAT_IEEE_DOUBLE_LE ? N_("IEEE 754 LE.")
+ : info.float_format == FLOAT_IEEE_DOUBLE_BE ? N_("IEEE 754 BE.")
+ : info.float_format == FLOAT_VAX_D ? N_("VAX D.")
+ : info.float_format == FLOAT_VAX_G ? N_("VAX G.")
+ : info.float_format == FLOAT_Z_LONG ? N_("IBM 390 Hex Long.")
+ : N_("Unknown")));
+
+ add_row (table, N_("Variables"),
+ pivot_value_new_integer (dict_get_n_vars (d)));
+
+ add_row (table, N_("Cases"),
+ (info.n_cases == -1
+ ? pivot_value_new_text (N_("Unknown"))
+ : pivot_value_new_integer (info.n_cases)));
+
+ add_row (table, N_("Type"),
+ pivot_value_new_text (info.klass->name));
+
+ struct variable *weight_var = dict_get_weight (d);
+ add_row (table, N_("Weight"),
+ (weight_var
+ ? pivot_value_new_variable (weight_var)
+ : pivot_value_new_text (N_("Not weighted"))));
+
+ add_row (table, N_("Compression"),
+ (info.compression == ANY_COMP_NONE
+ ? pivot_value_new_text (N_("None"))
+ : pivot_value_new_user_text (
+ info.compression == ANY_COMP_SIMPLE ? "SAV" : "ZSAV", -1)));
+
+ add_row (table, N_("Encoding"),
+ pivot_value_new_user_text (dict_get_encoding (d), -1));
+
+ if (dict_get_document_n_lines (d) > 0)
+ add_row (table, N_("Documents"),
+ pivot_value_new_user_text_nocopy (get_documents_as_string (d)));
+
+ pivot_table_submit (table);
+
+ size_t n_vars = dict_get_n_vars (d);
+ const struct variable **vars = xnmalloc (n_vars, sizeof *vars);
+ for (size_t i = 0; i < dict_get_n_vars (d); i++)
+ vars[i] = dict_get_var (d, i);
+ display_variables (vars, n_vars, DF_ALL_VARIABLE);
+ display_value_labels (vars, n_vars);
+ display_attributes (dict_get_attributes (dataset_dict (ds)),
+ vars, n_vars, DF_ATTRIBUTES);
+ free (vars);
+
+ dict_unref (d);
+
+ fh_unref (h);
+ free (encoding);
+ any_read_info_destroy (&info);
+ return CMD_SUCCESS;
+
+error:
+ fh_unref (h);
+ free (encoding);
+ return CMD_FAILURE;
+}
+\f
+/* DISPLAY utility. */
+
+static void display_macros (void);
+static void display_documents (const struct dictionary *dict);
+static void display_vectors (const struct dictionary *dict, int sorted);
+
+int
+cmd_display (struct lexer *lexer, struct dataset *ds)
+{
+ /* Whether to sort the list of variables alphabetically. */
+ int sorted;
+
+ /* Variables to display. */
+ size_t n;
+ const struct variable **vl;
+
+ if (lex_match_id (lexer, "MACROS"))
+ display_macros ();
+ else if (lex_match_id (lexer, "DOCUMENTS"))
+ display_documents (dataset_dict (ds));
+ else if (lex_match_id (lexer, "FILE"))
+ {
+ if (!lex_force_match_id (lexer, "LABEL"))
+ return CMD_FAILURE;
+
+ const char *label = dict_get_label (dataset_dict (ds));
+
+ struct pivot_table *table = pivot_table_create (N_("File Label"));
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Label"),
+ N_("Label"));
+ pivot_table_put1 (table, 0,
+ (label ? pivot_value_new_user_text (label, -1)
+ : pivot_value_new_text (N_("(none)"))));
+ pivot_table_submit (table);
+ }
+ else
+ {
+ int flags;
+
+ sorted = lex_match_id (lexer, "SORTED");
+
+ if (lex_match_id (lexer, "VECTORS"))
+ {
+ display_vectors (dataset_dict(ds), sorted);
+ return CMD_SUCCESS;
+ }
+ else if (lex_match_id (lexer, "SCRATCH"))
+ {
+ dict_get_vars (dataset_dict (ds), &vl, &n, DC_ORDINARY);
+ flags = DF_NAME;
+ }
+ else
+ {
+ struct subcommand
+ {
+ const char *name;
+ int flags;
+ };
+ static const struct subcommand subcommands[] =
+ {
+ {"@ATTRIBUTES", DF_ATTRIBUTES | DF_AT_ATTRIBUTES},
+ {"ATTRIBUTES", DF_ATTRIBUTES},
+ {"DICTIONARY", (DF_NAME | DF_POSITION | DF_LABEL
+ | DF_MEASUREMENT_LEVEL | DF_ROLE | DF_WIDTH
+ | DF_ALIGNMENT | DF_PRINT_FORMAT
+ | DF_WRITE_FORMAT | DF_MISSING_VALUES
+ | DF_VALUE_LABELS)},
+ {"INDEX", DF_NAME | DF_POSITION},
+ {"LABELS", DF_NAME | DF_POSITION | DF_LABEL},
+ {"NAMES", DF_NAME},
+ {"VARIABLES", (DF_NAME | DF_POSITION | DF_PRINT_FORMAT
+ | DF_WRITE_FORMAT | DF_MISSING_VALUES)},
+ {NULL, 0},
+ };
+ const struct subcommand *sbc;
+ struct dictionary *dict = dataset_dict (ds);
+
+ flags = 0;
+ for (sbc = subcommands; sbc->name != NULL; sbc++)
+ if (lex_match_id (lexer, sbc->name))
+ {
+ flags = sbc->flags;
+ break;
+ }
+
+ lex_match (lexer, T_SLASH);
+ lex_match_id (lexer, "VARIABLES");
+ lex_match (lexer, T_EQUALS);
+
+ if (lex_token (lexer) != T_ENDCMD)
+ {
+ if (!parse_variables_const (lexer, dict, &vl, &n, PV_NONE))
+ {
+ free (vl);
+ return CMD_FAILURE;
+ }
+ }
+ else
+ dict_get_vars (dict, &vl, &n, 0);
+ }
+
+ if (n > 0)
+ {
+ sort (vl, n, sizeof *vl, (sorted
+ ? compare_var_ptrs_by_name
+ : compare_var_ptrs_by_dict_index), NULL);
+
+ int variable_flags = flags & DF_ALL_VARIABLE;
+ if (variable_flags)
+ display_variables (vl, n, variable_flags);
+
+ if (flags & DF_VALUE_LABELS)
+ display_value_labels (vl, n);
+
+ int attribute_flags = flags & (DF_ATTRIBUTES | DF_AT_ATTRIBUTES);
+ if (attribute_flags)
+ display_attributes (dict_get_attributes (dataset_dict (ds)),
+ vl, n, attribute_flags);
+ }
+ else
+ msg (SN, _("No variables to display."));
+
+ free (vl);
+ }
+
+ return CMD_SUCCESS;
+}
+
+static void
+display_macros (void)
+{
+ msg (SW, _("Macros not supported."));
+}
+
+static char *
+get_documents_as_string (const struct dictionary *dict)
+{
+ const struct string_array *documents = dict_get_documents (dict);
+ struct string s = DS_EMPTY_INITIALIZER;
+ for (size_t i = 0; i < documents->n; i++)
+ {
+ if (i)
+ ds_put_byte (&s, '\n');
+ ds_put_cstr (&s, documents->strings[i]);
+ }
+ return ds_steal_cstr (&s);
+}
+
+static void
+display_documents (const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (N_("Documents"));
+ struct pivot_dimension *d = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Documents"), N_("Document"));
+ d->hide_all_labels = true;
+
+ if (!dict_get_documents (dict)->n)
+ pivot_table_put1 (table, 0, pivot_value_new_text (N_("(none)")));
+ else
+ {
+ char *docs = get_documents_as_string (dict);
+ pivot_table_put1 (table, 0, pivot_value_new_user_text_nocopy (docs));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+display_variables (const struct variable **vl, size_t n, int flags)
+{
+ struct pivot_table *table = pivot_table_create (N_("Variables"));
+
+ struct pivot_dimension *attributes = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Attributes"));
+
+ struct heading
+ {
+ int flag;
+ const char *title;
+ };
+ static const struct heading headings[] = {
+ { DF_POSITION, N_("Position") },
+ { DF_LABEL, N_("Label") },
+ { DF_MEASUREMENT_LEVEL, N_("Measurement Level") },
+ { DF_ROLE, N_("Role") },
+ { DF_WIDTH, N_("Width") },
+ { DF_ALIGNMENT, N_("Alignment") },
+ { DF_PRINT_FORMAT, N_("Print Format") },
+ { DF_WRITE_FORMAT, N_("Write Format") },
+ { DF_MISSING_VALUES, N_("Missing Values") },
+ };
+ for (size_t i = 0; i < sizeof headings / sizeof *headings; i++)
+ if (flags & headings[i].flag)
+ pivot_category_create_leaf (attributes->root,
+ pivot_value_new_text (headings[i].title));
+
+ struct pivot_dimension *names = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Name"));
+ names->root->show_label = true;
+
+ for (size_t i = 0; i < n; i++)
+ {
+ const struct variable *v = vl[i];
+
+ struct pivot_value *name = pivot_value_new_variable (v);
+ name->variable.show = SETTINGS_VALUE_SHOW_VALUE;
+ int row = pivot_category_create_leaf (names->root, name);
+
+ int x = 0;
+ if (flags & DF_POSITION)
+ pivot_table_put2 (table, x++, row, pivot_value_new_integer (
+ var_get_dict_index (v) + 1));
+
+ if (flags & DF_LABEL)
+ {
+ const char *label = var_get_label (v);
+ if (label)
+ pivot_table_put2 (table, x, row,
+ pivot_value_new_user_text (label, -1));
+ x++;
+ }
+
+ if (flags & DF_MEASUREMENT_LEVEL)
+ pivot_table_put2 (
+ table, x++, row,
+ pivot_value_new_text (measure_to_string (var_get_measure (v))));
+
+ if (flags & DF_ROLE)
+ pivot_table_put2 (
+ table, x++, row,
+ pivot_value_new_text (var_role_to_string (var_get_role (v))));
+
+ if (flags & DF_WIDTH)
+ pivot_table_put2 (
+ table, x++, row,
+ pivot_value_new_integer (var_get_display_width (v)));
+
+ if (flags & DF_ALIGNMENT)
+ pivot_table_put2 (
+ table, x++, row,
+ pivot_value_new_text (alignment_to_string (
+ var_get_alignment (v))));
+
+ if (flags & DF_PRINT_FORMAT)
+ {
+ const struct fmt_spec *print = var_get_print_format (v);
+ char s[FMT_STRING_LEN_MAX + 1];
+
+ pivot_table_put2 (
+ table, x++, row,
+ pivot_value_new_user_text (fmt_to_string (print, s), -1));
+ }
+
+ if (flags & DF_WRITE_FORMAT)
+ {
+ const struct fmt_spec *write = var_get_write_format (v);
+ char s[FMT_STRING_LEN_MAX + 1];
+
+ pivot_table_put2 (
+ table, x++, row,
+ pivot_value_new_user_text (fmt_to_string (write, s), -1));
+ }
+
+ if (flags & DF_MISSING_VALUES)
+ {
+ char *s = mv_to_string (var_get_missing_values (v),
+ var_get_encoding (v));
+ if (s)
+ pivot_table_put2 (
+ table, x, row,
+ pivot_value_new_user_text_nocopy (s));
+
+ x++;
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+static bool
+any_value_labels (const struct variable **vars, size_t n_vars)
+{
+ for (size_t i = 0; i < n_vars; i++)
+ if (val_labs_count (var_get_value_labels (vars[i])))
+ return true;
+ return false;
+}
+
+static void
+display_value_labels (const struct variable **vars, size_t n_vars)
+{
+ if (!any_value_labels (vars, n_vars))
+ return;
+
+ struct pivot_table *table = pivot_table_create (N_("Value Labels"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN,
+ N_("Label"), N_("Label"));
+
+ struct pivot_dimension *values = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable Value"));
+ values->root->show_label = true;
+
+ struct pivot_footnote *missing_footnote = pivot_table_create_footnote (
+ table, pivot_value_new_text (N_("User-missing value")));
+
+ for (size_t i = 0; i < n_vars; i++)
+ {
+ const struct val_labs *val_labs = var_get_value_labels (vars[i]);
+ size_t n_labels = val_labs_count (val_labs);
+ if (!n_labels)
+ continue;
+
+ struct pivot_category *group = pivot_category_create_group__ (
+ values->root, pivot_value_new_variable (vars[i]));
+
+ const struct val_lab **labels = val_labs_sorted (val_labs);
+ for (size_t j = 0; j < n_labels; j++)
+ {
+ const struct val_lab *vl = labels[j];
+
+ struct pivot_value *value = pivot_value_new_var_value (
+ vars[i], &vl->value);
+ if (value->type == PIVOT_VALUE_NUMERIC)
+ value->numeric.show = SETTINGS_VALUE_SHOW_VALUE;
+ else
+ value->string.show = SETTINGS_VALUE_SHOW_VALUE;
+ if (var_is_value_missing (vars[i], &vl->value) == MV_USER)
+ pivot_value_add_footnote (value, missing_footnote);
+ int row = pivot_category_create_leaf (group, value);
+
+ struct pivot_value *label = pivot_value_new_var_value (
+ vars[i], &vl->value);
+ char *escaped_label = xstrdup (val_lab_get_escaped_label (vl));
+ if (label->type == PIVOT_VALUE_NUMERIC)
+ {
+ free (label->numeric.value_label);
+ label->numeric.value_label = escaped_label;
+ label->numeric.show = SETTINGS_VALUE_SHOW_LABEL;
+ }
+ else
+ {
+ free (label->string.value_label);
+ label->string.value_label = escaped_label;
+ label->string.show = SETTINGS_VALUE_SHOW_LABEL;
+ }
+ pivot_table_put2 (table, 0, row, label);
+ }
+ free (labels);
+ }
+ pivot_table_submit (table);
+}
+\f
+static bool
+is_at_name (const char *name)
+{
+ return name[0] == '@' || (name[0] == '$' && name[1] == '@');
+}
+
+static size_t
+count_attributes (const struct attrset *set, int flags)
+{
+ struct attrset_iterator i;
+ struct attribute *attr;
+ size_t n_attrs;
+
+ n_attrs = 0;
+ for (attr = attrset_first (set, &i); attr != NULL;
+ attr = attrset_next (set, &i))
+ if (flags & DF_AT_ATTRIBUTES || !is_at_name (attribute_get_name (attr)))
+ n_attrs += attribute_get_n_values (attr);
+ return n_attrs;
+}
+
+static void
+display_attrset (struct pivot_table *table, struct pivot_value *set_name,
+ const struct attrset *set, int flags)
+{
+ size_t n_total = count_attributes (set, flags);
+ if (!n_total)
+ {
+ pivot_value_destroy (set_name);
+ return;
+ }
+
+ struct pivot_category *group = pivot_category_create_group__ (
+ table->dimensions[1]->root, set_name);
+
+ size_t n_attrs = attrset_count (set);
+ struct attribute **attrs = attrset_sorted (set);
+ for (size_t i = 0; i < n_attrs; i++)
+ {
+ const struct attribute *attr = attrs[i];
+ const char *name = attribute_get_name (attr);
+
+ if (!(flags & DF_AT_ATTRIBUTES) && is_at_name (name))
+ continue;
+
+ size_t n_values = attribute_get_n_values (attr);
+ for (size_t j = 0; j < n_values; j++)
+ {
+ int row = pivot_category_create_leaf (
+ group,
+ (n_values > 1
+ ? pivot_value_new_user_text_nocopy (xasprintf (
+ "%s[%zu]", name, j + 1))
+ : pivot_value_new_user_text (name, -1)));
+ pivot_table_put2 (table, 0, row,
+ pivot_value_new_user_text (
+ attribute_get_value (attr, j), -1));
+ }
+ }
+ free (attrs);
+}
+
+static void
+display_attributes (const struct attrset *dict_attrset,
+ const struct variable **vars, size_t n_vars, int flags)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Variable and Dataset Attributes"));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN,
+ N_("Value"), N_("Value"));
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variable and Name"));
+ variables->root->show_label = true;
+
+ display_attrset (table, pivot_value_new_text (N_("(dataset)")),
+ dict_attrset, flags);
+ for (size_t i = 0; i < n_vars; i++)
+ display_attrset (table, pivot_value_new_variable (vars[i]),
+ var_get_attributes (vars[i]), flags);
+
+ if (pivot_table_is_empty (table))
+ pivot_table_unref (table);
+ else
+ pivot_table_submit (table);
+}
+
+/* Display a list of vectors. If SORTED is nonzero then they are
+ sorted alphabetically. */
+static void
+display_vectors (const struct dictionary *dict, int sorted)
+{
+ size_t n_vectors = dict_get_n_vectors (dict);
+ if (n_vectors == 0)
+ {
+ msg (SN, _("No vectors defined."));
+ return;
+ }
+
+ const struct vector **vectors = xnmalloc (n_vectors, sizeof *vectors);
+ for (size_t i = 0; i < n_vectors; i++)
+ vectors[i] = dict_get_vector (dict, i);
+ if (sorted)
+ qsort (vectors, n_vectors, sizeof *vectors, compare_vector_ptrs_by_name);
+
+ struct pivot_table *table = pivot_table_create (N_("Vectors"));
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Attributes"),
+ N_("Variable"), N_("Print Format"));
+ struct pivot_dimension *vector_dim = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Vector and Position"));
+ vector_dim->root->show_label = true;
+
+ for (size_t i = 0; i < n_vectors; i++)
+ {
+ const struct vector *vec = vectors[i];
+
+ struct pivot_category *group = pivot_category_create_group__ (
+ vector_dim->root, pivot_value_new_user_text (
+ vector_get_name (vectors[i]), -1));
+
+ for (size_t j = 0; j < vector_get_n_vars (vec); j++)
+ {
+ struct variable *var = vector_get_var (vec, j);
+
+ int row = pivot_category_create_leaf (
+ group, pivot_value_new_integer (j + 1));
+
+ pivot_table_put2 (table, 0, row, pivot_value_new_variable (var));
+ char fmt_string[FMT_STRING_LEN_MAX + 1];
+ fmt_to_string (var_get_print_format (var), fmt_string);
+ pivot_table_put2 (table, 1, row,
+ pivot_value_new_user_text (fmt_string, -1));
+ }
+ }
+
+ pivot_table_submit (table);
+
+ free (vectors);
+}
+\f
+/* Encoding analysis. */
+
+static const char *encoding_names[] = {
+ /* These encodings are from http://encoding.spec.whatwg.org/, as retrieved
+ February 2014. Encodings not supported by glibc and encodings relevant
+ only to HTML have been removed. */
+ "utf-8",
+ "windows-1252",
+ "iso-8859-2",
+ "iso-8859-3",
+ "iso-8859-4",
+ "iso-8859-5",
+ "iso-8859-6",
+ "iso-8859-7",
+ "iso-8859-8",
+ "iso-8859-10",
+ "iso-8859-13",
+ "iso-8859-14",
+ "iso-8859-16",
+ "macintosh",
+ "windows-874",
+ "windows-1250",
+ "windows-1251",
+ "windows-1253",
+ "windows-1254",
+ "windows-1255",
+ "windows-1256",
+ "windows-1257",
+ "windows-1258",
+ "koi8-r",
+ "koi8-u",
+ "ibm866",
+ "gb18030",
+ "big5",
+ "euc-jp",
+ "iso-2022-jp",
+ "shift_jis",
+ "euc-kr",
+
+ /* Added by user request. */
+ "ibm850",
+ "din_66003",
+};
+#define N_ENCODING_NAMES (sizeof encoding_names / sizeof *encoding_names)
+
+struct encoding
+ {
+ uint64_t encodings;
+ char **utf8_strings;
+ unsigned int hash;
+ };
+
+static char **
+recode_strings (struct pool *pool,
+ char **strings, bool *ids, size_t n,
+ const char *encoding)
+{
+ char **utf8_strings;
+ size_t i;
+
+ utf8_strings = pool_alloc (pool, n * sizeof *utf8_strings);
+ for (i = 0; i < n; i++)
+ {
+ struct substring utf8;
+ int error;
+
+ error = recode_pedantically ("UTF-8", encoding, ss_cstr (strings[i]),
+ pool, &utf8);
+ if (!error)
+ {
+ ss_rtrim (&utf8, ss_cstr (" "));
+ utf8.string[utf8.length] = '\0';
+
+ if (ids[i] && !id_is_plausible (utf8.string))
+ error = EINVAL;
+ }
+
+ if (error)
+ return NULL;
+
+ utf8_strings[i] = utf8.string;
+ }
+
+ return utf8_strings;
+}
+
+static struct encoding *
+find_duplicate_encoding (struct encoding *encodings, size_t n_encodings,
+ char **utf8_strings, size_t n_strings,
+ unsigned int hash)
+{
+ struct encoding *e;
+
+ for (e = encodings; e < &encodings[n_encodings]; e++)
+ {
+ int i;
+
+ if (e->hash != hash)
+ goto next_encoding;
+
+ for (i = 0; i < n_strings; i++)
+ if (strcmp (utf8_strings[i], e->utf8_strings[i]))
+ goto next_encoding;
+
+ return e;
+ next_encoding:;
+ }
+
+ return NULL;
+}
+
+static bool
+all_equal (const struct encoding *encodings, size_t n_encodings,
+ size_t string_idx)
+{
+ const char *s0;
+ size_t i;
+
+ s0 = encodings[0].utf8_strings[string_idx];
+ for (i = 1; i < n_encodings; i++)
+ if (strcmp (s0, encodings[i].utf8_strings[string_idx]))
+ return false;
+
+ return true;
+}
+
+static int
+equal_prefix (const struct encoding *encodings, size_t n_encodings,
+ size_t string_idx)
+{
+ const char *s0;
+ size_t prefix;
+ size_t i;
+
+ s0 = encodings[0].utf8_strings[string_idx];
+ prefix = strlen (s0);
+ for (i = 1; i < n_encodings; i++)
+ {
+ const char *si = encodings[i].utf8_strings[string_idx];
+ size_t j;
+
+ for (j = 0; j < prefix; j++)
+ if (s0[j] != si[j])
+ {
+ prefix = j;
+ if (!prefix)
+ return 0;
+ break;
+ }
+ }
+
+ while (prefix > 0 && s0[prefix - 1] != ' ')
+ prefix--;
+ return prefix;
+}
+
+static int
+equal_suffix (const struct encoding *encodings, size_t n_encodings,
+ size_t string_idx)
+{
+ const char *s0;
+ size_t s0_len;
+ size_t suffix;
+ size_t i;
+
+ s0 = encodings[0].utf8_strings[string_idx];
+ s0_len = strlen (s0);
+ suffix = s0_len;
+ for (i = 1; i < n_encodings; i++)
+ {
+ const char *si = encodings[i].utf8_strings[string_idx];
+ size_t si_len = strlen (si);
+ size_t j;
+
+ if (si_len < suffix)
+ suffix = si_len;
+ for (j = 0; j < suffix; j++)
+ if (s0[s0_len - j - 1] != si[si_len - j - 1])
+ {
+ suffix = j;
+ if (!suffix)
+ return 0;
+ break;
+ }
+ }
+
+ while (suffix > 0 && s0[s0_len - suffix] != ' ')
+ suffix--;
+ return suffix;
+}
+
+static void
+report_encodings (const struct file_handle *h, struct pool *pool,
+ char **titles, bool *ids, char **strings, size_t n_strings)
+{
+ struct encoding encodings[N_ENCODING_NAMES];
+ size_t n_encodings, n_unique_strings;
+
+ n_encodings = 0;
+ for (size_t i = 0; i < N_ENCODING_NAMES; i++)
+ {
+ char **utf8_strings;
+ struct encoding *e;
+ unsigned int hash;
+
+ utf8_strings = recode_strings (pool, strings, ids, n_strings,
+ encoding_names[i]);
+ if (!utf8_strings)
+ continue;
+
+ /* Hash utf8_strings. */
+ hash = 0;
+ for (size_t j = 0; j < n_strings; j++)
+ hash = hash_string (utf8_strings[j], hash);
+
+ /* If there's a duplicate encoding, just mark it. */
+ e = find_duplicate_encoding (encodings, n_encodings,
+ utf8_strings, n_strings, hash);
+ if (e)
+ {
+ e->encodings |= UINT64_C (1) << i;
+ continue;
+ }
+
+ e = &encodings[n_encodings++];
+ e->encodings = UINT64_C (1) << i;
+ e->utf8_strings = utf8_strings;
+ e->hash = hash;
+ }
+ if (!n_encodings)
+ {
+ msg (SW, _("No valid encodings found."));
+ return;
+ }
+
+ /* Table of valid encodings. */
+ struct pivot_table *table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("Usable encodings for %s."),
+ fh_get_name (h)), "Usable Encodings");
+ pivot_table_set_caption (
+ table, pivot_value_new_text_format (
+ N_("Encodings that can successfully read %s (by specifying the encoding "
+ "name on the GET command's ENCODING subcommand). Encodings that "
+ "yield identical text are listed together."),
+ fh_get_name (h)));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Encodings"),
+ N_("Encodings"));
+ struct pivot_dimension *number = pivot_dimension_create__ (
+ table, PIVOT_AXIS_ROW, pivot_value_new_user_text ("#", -1));
+ number->root->show_label = true;
+
+ for (size_t i = 0; i < n_encodings; i++)
+ {
+ struct string s = DS_EMPTY_INITIALIZER;
+ for (size_t j = 0; j < 64; j++)
+ if (encodings[i].encodings & (UINT64_C (1) << j))
+ ds_put_format (&s, "%s, ", encoding_names[j]);
+ ds_chomp (&s, ss_cstr (", "));
+
+ int row = pivot_category_create_leaf (number->root,
+ pivot_value_new_integer (i + 1));
+ pivot_table_put2 (
+ table, 0, row, pivot_value_new_user_text_nocopy (ds_steal_cstr (&s)));
+ }
+ pivot_table_submit (table);
+
+ n_unique_strings = 0;
+ for (size_t i = 0; i < n_strings; i++)
+ if (!all_equal (encodings, n_encodings, i))
+ n_unique_strings++;
+ if (!n_unique_strings)
+ return;
+
+ /* Table of alternative interpretations. */
+ table = pivot_table_create__ (
+ pivot_value_new_text_format (N_("%s Encoded Text Strings"),
+ fh_get_name (h)),
+ "Alternate Encoded Text Strings");
+ pivot_table_set_caption (
+ table, pivot_value_new_text (
+ N_("Text strings in the file dictionary that the previously listed "
+ "encodings interpret differently, along with the interpretations.")));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Text"), N_("Text"));
+
+ number = pivot_dimension_create__ (table, PIVOT_AXIS_ROW,
+ pivot_value_new_user_text ("#", -1));
+ number->root->show_label = true;
+ for (size_t i = 0; i < n_encodings; i++)
+ pivot_category_create_leaf (number->root,
+ pivot_value_new_integer (i + 1));
+
+ struct pivot_dimension *purpose = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Purpose"));
+ purpose->root->show_label = true;
+
+ for (size_t i = 0; i < n_strings; i++)
+ if (!all_equal (encodings, n_encodings, i))
+ {
+ int prefix = equal_prefix (encodings, n_encodings, i);
+ int suffix = equal_suffix (encodings, n_encodings, i);
+
+ int purpose_idx = pivot_category_create_leaf (
+ purpose->root, pivot_value_new_user_text (titles[i], -1));
+
+ for (size_t j = 0; j < n_encodings; j++)
+ {
+ const char *s = encodings[j].utf8_strings[i] + prefix;
+
+ if (prefix || suffix)
+ {
+ size_t len = strlen (s) - suffix;
+ struct string entry;
+
+ ds_init_empty (&entry);
+ if (prefix)
+ ds_put_cstr (&entry, "...");
+ ds_put_substring (&entry, ss_buffer (s, len));
+ if (suffix)
+ ds_put_cstr (&entry, "...");
+
+ pivot_table_put3 (table, 0, j, purpose_idx,
+ pivot_value_new_user_text_nocopy (
+ ds_steal_cstr (&entry)));
+ }
+ else
+ pivot_table_put3 (table, 0, j, purpose_idx,
+ pivot_value_new_user_text (s, -1));
+ }
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011, 2020 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "t-test.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+#include "libpspp/misc.h"
+
+#include "libpspp/str.h"
+#include "data/casereader.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+#include "math/moments.h"
+#include "math/levene.h"
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct indep_samples
+{
+ const struct variable *gvar;
+ bool cut;
+ const union value *gval0;
+ const union value *gval1;
+};
+
+struct pair_stats
+{
+ struct moments *mom[2];
+ double lev ;
+ struct levene *nl;
+};
+
+
+static void indep_summary (const struct tt *tt, struct indep_samples *is, const struct pair_stats *ps);
+static void indep_test (const struct tt *tt, const struct pair_stats *ps);
+
+static int
+which_group (const union value *v, const struct indep_samples *is)
+{
+ int width = var_get_width (is->gvar);
+ int cmp = value_compare_3way (v, is->gval0, width);
+ if (is->cut)
+ return (cmp < 0);
+
+ if (cmp == 0)
+ return 0;
+
+ if (0 == value_compare_3way (v, is->gval1, width))
+ return 1;
+
+ return -1;
+}
+
+void
+indep_run (struct tt *tt, const struct variable *gvar,
+ bool cut,
+ const union value *gval0, const union value *gval1,
+ struct casereader *reader)
+{
+ struct indep_samples is;
+ struct ccase *c;
+ struct casereader *r;
+
+ struct pair_stats *ps = XCALLOC (tt->n_vars, struct pair_stats);
+
+ int v;
+
+ for (v = 0; v < tt->n_vars; ++v)
+ {
+ ps[v].mom[0] = moments_create (MOMENT_VARIANCE);
+ ps[v].mom[1] = moments_create (MOMENT_VARIANCE);
+ ps[v].nl = levene_create (var_get_width (gvar), cut ? gval0: NULL);
+ }
+
+ is.gvar = gvar;
+ is.gval0 = gval0;
+ is.gval1 = gval1;
+ is.cut = cut;
+
+ r = casereader_clone (reader);
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+
+ const union value *gv = case_data (c, gvar);
+
+ int grp = which_group (gv, &is);
+ if (grp < 0)
+ continue;
+
+ for (v = 0; v < tt->n_vars; ++v)
+ {
+ const union value *val = case_data (c, tt->vars[v]);
+ if (var_is_value_missing (tt->vars[v], val) & tt->exclude)
+ continue;
+
+ moments_pass_one (ps[v].mom[grp], val->f, w);
+ levene_pass_one (ps[v].nl, val->f, w, gv);
+ }
+ }
+ casereader_destroy (r);
+
+ r = casereader_clone (reader);
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+
+ const union value *gv = case_data (c, gvar);
+
+ int grp = which_group (gv, &is);
+ if (grp < 0)
+ continue;
+
+ for (v = 0; v < tt->n_vars; ++v)
+ {
+ const union value *val = case_data (c, tt->vars[v]);
+ if (var_is_value_missing (tt->vars[v], val) & tt->exclude)
+ continue;
+
+ moments_pass_two (ps[v].mom[grp], val->f, w);
+ levene_pass_two (ps[v].nl, val->f, w, gv);
+ }
+ }
+ casereader_destroy (r);
+
+ r = reader;
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+
+ const union value *gv = case_data (c, gvar);
+
+ int grp = which_group (gv, &is);
+ if (grp < 0)
+ continue;
+
+ for (v = 0; v < tt->n_vars; ++v)
+ {
+ const union value *val = case_data (c, tt->vars[v]);
+ if (var_is_value_missing (tt->vars[v], val) & tt->exclude)
+ continue;
+
+ levene_pass_three (ps[v].nl, val->f, w, gv);
+ }
+ }
+ casereader_destroy (r);
+
+
+ for (v = 0; v < tt->n_vars; ++v)
+ ps[v].lev = levene_calculate (ps[v].nl);
+
+ indep_summary (tt, &is, ps);
+ indep_test (tt, ps);
+
+
+ for (v = 0; v < tt->n_vars; ++v)
+ {
+ moments_destroy (ps[v].mom[0]);
+ moments_destroy (ps[v].mom[1]);
+ levene_destroy (ps[v].nl);
+ }
+ free (ps);
+}
+
+
+static void
+indep_summary (const struct tt *tt, struct indep_samples *is, const struct pair_stats *ps)
+{
+ struct pivot_table *table = pivot_table_create (N_("Group Statistics"));
+ pivot_table_set_weight_var (table, tt->wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("S.E. Mean"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *group = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Group"));
+ group->root->show_label = true;
+ if (is->cut)
+ {
+ struct string vallab0 = DS_EMPTY_INITIALIZER;
+ ds_put_cstr (&vallab0, "≥");
+ var_append_value_name (is->gvar, is->gval0, &vallab0);
+ pivot_category_create_leaf (group->root,
+ pivot_value_new_user_text_nocopy (
+ ds_steal_cstr (&vallab0)));
+
+ struct string vallab1 = DS_EMPTY_INITIALIZER;
+ ds_put_cstr (&vallab1, "<");
+ var_append_value_name (is->gvar, is->gval0, &vallab1);
+ pivot_category_create_leaf (group->root,
+ pivot_value_new_user_text_nocopy (
+ ds_steal_cstr (&vallab1)));
+ }
+ else
+ {
+ pivot_category_create_leaf (
+ group->root, pivot_value_new_var_value (is->gvar, is->gval0));
+ pivot_category_create_leaf (
+ group->root, pivot_value_new_var_value (is->gvar, is->gval1));
+ }
+
+ struct pivot_dimension *dep_vars = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ for (size_t v = 0; v < tt->n_vars; ++v)
+ {
+ const struct variable *var = tt->vars[v];
+
+ int dep_var_idx = pivot_category_create_leaf (
+ dep_vars->root, pivot_value_new_variable (var));
+
+ for (int i = 0 ; i < 2; ++i)
+ {
+ double cc, mean, sigma;
+ moments_calculate (ps[v].mom[i], &cc, &mean, &sigma, NULL, NULL);
+
+ double entries[] = { cc, mean, sqrt (sigma), sqrt (sigma / cc) };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put3 (table, j, i, dep_var_idx,
+ pivot_value_new_number (entries[j]));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+indep_test (const struct tt *tt, const struct pair_stats *ps)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Independent Samples Test"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ pivot_category_create_group (
+ statistics->root, N_("Levene's Test for Equality of Variances"),
+ N_("F"), PIVOT_RC_OTHER,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+ struct pivot_category *group = pivot_category_create_group (
+ statistics->root, N_("T-Test for Equality of Means"),
+ N_("t"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_OTHER,
+ N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Mean Difference"), PIVOT_RC_OTHER,
+ N_("Std. Error Difference"), PIVOT_RC_OTHER);
+ pivot_category_create_group (
+ /* xgettext:no-c-format */
+ group, N_("95% Confidence Interval of the Difference"),
+ N_("Lower"), PIVOT_RC_OTHER,
+ N_("Upper"), PIVOT_RC_OTHER);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Assumptions"),
+ N_("Equal variances assumed"),
+ N_("Equal variances not assumed"));
+
+ struct pivot_dimension *dep_vars = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ for (size_t v = 0; v < tt->n_vars; ++v)
+ {
+ int dep_var_idx = pivot_category_create_leaf (
+ dep_vars->root, pivot_value_new_variable (tt->vars[v]));
+
+ double cc0, mean0, sigma0;
+ double cc1, mean1, sigma1;
+ moments_calculate (ps[v].mom[0], &cc0, &mean0, &sigma0, NULL, NULL);
+ moments_calculate (ps[v].mom[1], &cc1, &mean1, &sigma1, NULL, NULL);
+
+ double mean_diff = mean0 - mean1;
+
+
+ /* Equal variances assumed. */
+ double e_df = cc0 + cc1 - 2.0;
+ double e_pooled_variance = ((cc0 - 1)* sigma0 + (cc1 - 1) * sigma1) / e_df;
+ double e_tval = ((mean0 - mean1) / sqrt (e_pooled_variance)
+ / sqrt ((cc0 + cc1) / (cc0 * cc1)));
+ double e_p = gsl_cdf_tdist_P (e_tval, e_df);
+ double e_q = gsl_cdf_tdist_Q (e_tval, e_df);
+ double e_sig = 2.0 * (e_tval > 0 ? e_q : e_p);
+ double e_std_err_diff = sqrt (e_pooled_variance * (1.0/cc0 + 1.0/cc1));
+ double e_tval_qinv = gsl_cdf_tdist_Qinv ((1 - tt->confidence) / 2.0, e_df);
+
+ /* Equal variances not assumed */
+ const double s0 = sigma0 / cc0;
+ const double s1 = sigma1 / cc1;
+ double d_df = (pow2 (s0 + s1) / (pow2 (s0) / (cc0 - 1)
+ + pow2 (s1) / (cc1 - 1)));
+ double d_tval = mean_diff / sqrt (sigma0 / cc0 + sigma1 / cc1);
+ double d_p = gsl_cdf_tdist_P (d_tval, d_df);
+ double d_q = gsl_cdf_tdist_Q (d_tval, d_df);
+ double d_sig = 2.0 * (d_tval > 0 ? d_q : d_p);
+ double d_std_err_diff = sqrt ((sigma0 / cc0) + (sigma1 / cc1));
+ double d_tval_qinv = gsl_cdf_tdist_Qinv ((1 - tt->confidence) / 2.0, d_df);
+
+ struct entry
+ {
+ int assumption_idx;
+ int stat_idx;
+ double x;
+ }
+ entries[] =
+ {
+ { 0, 0, ps[v].lev },
+ { 0, 1, gsl_cdf_fdist_Q (ps[v].lev, 1, cc0 + cc1 - 2) },
+
+ { 0, 2, e_tval },
+ { 0, 3, e_df },
+ { 0, 4, e_sig },
+ { 0, 5, mean_diff },
+ { 0, 6, e_std_err_diff },
+ { 0, 7, mean_diff - e_tval_qinv * e_std_err_diff },
+ { 0, 8, mean_diff + e_tval_qinv * e_std_err_diff },
+
+ { 1, 2, d_tval },
+ { 1, 3, d_df },
+ { 1, 4, d_sig },
+ { 1, 5, mean_diff },
+ { 1, 6, d_std_err_diff },
+ { 1, 7, mean_diff - d_tval_qinv * d_std_err_diff },
+ { 1, 8, mean_diff + d_tval_qinv * d_std_err_diff },
+ };
+
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ {
+ const struct entry *e = &entries[i];
+ pivot_table_put3 (table, e->stat_idx, e->assumption_idx,
+ dep_var_idx, pivot_value_new_number (e->x));
+ }
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <config.h>
+
+
+#include "t-test.h"
+
+#include <math.h>
+#include <gsl/gsl_cdf.h>
+
+#include "data/variable.h"
+#include "data/format.h"
+#include "data/casereader.h"
+#include "data/dictionary.h"
+#include "libpspp/hash-functions.h"
+#include "libpspp/hmapx.h"
+#include "math/moments.h"
+
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct per_var_stats
+{
+ const struct variable *var;
+
+ /* N, Mean, Variance */
+ struct moments *mom;
+
+ /* Sum of the differences */
+ double sum_diff;
+};
+
+
+struct one_samp
+{
+ struct per_var_stats *stats;
+ size_t n_stats;
+ double testval;
+};
+
+
+static void
+one_sample_test (const struct tt *tt, const struct one_samp *os)
+{
+ struct pivot_table *table = pivot_table_create (N_("One-Sample Test"));
+ pivot_table_set_weight_var (table, tt->wv);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ struct pivot_category *group = pivot_category_create_group__ (
+ statistics->root, pivot_value_new_user_text_nocopy (
+ xasprintf (_("Test Value = %.*g"), DBL_DIG + 1, os->testval)));
+ pivot_category_create_leaves (
+ group,
+ N_("t"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_COUNT,
+ N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Mean Difference"), PIVOT_RC_OTHER);
+ struct pivot_category *subgroup = pivot_category_create_group__ (
+ group, pivot_value_new_user_text_nocopy (
+ xasprintf (_("%g%% Confidence Interval of the Difference"),
+ tt->confidence * 100.0)));
+ pivot_category_create_leaves (subgroup,
+ N_("Lower"), PIVOT_RC_OTHER,
+ N_("Upper"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *dep_vars = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
+
+ for (size_t i = 0; i < os->n_stats; i++)
+ {
+ const struct per_var_stats *per_var_stats = &os->stats[i];
+ const struct moments *m = per_var_stats->mom;
+
+ int dep_var_idx = pivot_category_create_leaf (
+ dep_vars->root, pivot_value_new_variable (per_var_stats->var));
+
+ double cc, mean, sigma;
+ moments_calculate (m, &cc, &mean, &sigma, NULL, NULL);
+ double tval = (mean - os->testval) * sqrt (cc / sigma);
+ double mean_diff = per_var_stats->sum_diff / cc;
+ double se_mean = sqrt (sigma / cc);
+ double df = cc - 1.0;
+ double p = gsl_cdf_tdist_P (tval, df);
+ double q = gsl_cdf_tdist_Q (tval, df);
+ double sig = 2.0 * (tval > 0 ? q : p);
+ double tval_qinv = gsl_cdf_tdist_Qinv ((1.0 - tt->confidence) / 2.0, df);
+ double lower = mean_diff - tval_qinv * se_mean;
+ double upper = mean_diff + tval_qinv * se_mean;
+
+ double entries[] = { tval, df, sig, mean_diff, lower, upper };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, dep_var_idx,
+ pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+}
+
+static void
+one_sample_summary (const struct tt *tt, const struct one_samp *os)
+{
+ struct pivot_table *table = pivot_table_create (N_("One-Sample Statistics"));
+ pivot_table_set_weight_var (table, tt->wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("S.E. Mean"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0; i < os->n_stats; i++)
+ {
+ const struct per_var_stats *per_var_stats = &os->stats[i];
+ const struct moments *m = per_var_stats->mom;
+
+ int var_idx = pivot_category_create_leaf (
+ variables->root, pivot_value_new_variable (per_var_stats->var));
+
+ double cc, mean, sigma;
+ moments_calculate (m, &cc, &mean, &sigma, NULL, NULL);
+
+ double entries[] = { cc, mean, sqrt (sigma), sqrt (sigma / cc) };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, var_idx,
+ pivot_value_new_number (entries[j]));
+ }
+
+ pivot_table_submit (table);
+}
+
+void
+one_sample_run (const struct tt *tt, double testval, struct casereader *reader)
+{
+ struct one_samp os;
+ os.testval = testval;
+ os.stats = xcalloc (tt->n_vars, sizeof *os.stats);
+ os.n_stats = tt->n_vars;
+ for (size_t i = 0; i < tt->n_vars; ++i)
+ {
+ struct per_var_stats *per_var_stats = &os.stats[i];
+ per_var_stats->var = tt->vars[i];
+ per_var_stats->mom = moments_create (MOMENT_VARIANCE);
+ }
+
+ struct casereader *r = casereader_clone (reader);
+ struct ccase *c;
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+ for (size_t i = 0; i < os.n_stats; i++)
+ {
+ const struct per_var_stats *per_var_stats = &os.stats[i];
+ const struct variable *var = per_var_stats->var;
+ const union value *val = case_data (c, var);
+ if (var_is_value_missing (var, val) & tt->exclude)
+ continue;
+
+ moments_pass_one (per_var_stats->mom, val->f, w);
+ }
+ }
+ casereader_destroy (r);
+
+ r = reader;
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+ for (size_t i = 0; i < os.n_stats; i++)
+ {
+ struct per_var_stats *per_var_stats = &os.stats[i];
+ const struct variable *var = per_var_stats->var;
+ const union value *val = case_data (c, var);
+ if (var_is_value_missing (var, val) & tt->exclude)
+ continue;
+
+ moments_pass_two (per_var_stats->mom, val->f, w);
+ per_var_stats->sum_diff += w * (val->f - os.testval);
+ }
+ }
+ casereader_destroy (r);
+
+ one_sample_summary (tt, &os);
+ one_sample_test (tt, &os);
+
+ for (size_t i = 0; i < os.n_stats; i++)
+ {
+ const struct per_var_stats *per_var_stats = &os.stats[i];
+ moments_destroy (per_var_stats->mom);
+ }
+ free (os.stats);
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <math.h>
+#include <gsl/gsl_cdf.h>
+
+#include "t-test.h"
+
+#include "math/moments.h"
+#include "math/correlation.h"
+#include "data/casereader.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/variable.h"
+
+#include "output/pivot-table.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+
+struct pair_stats
+{
+ double sum_of_prod;
+ struct moments *mom0;
+ const struct variable *var0;
+
+ struct moments *mom1;
+ const struct variable *var1;
+
+ struct moments *mom_diff;
+};
+
+struct paired_samp
+{
+ struct pair_stats *ps;
+ size_t n_ps;
+};
+
+static void paired_summary (const struct tt *tt, struct paired_samp *os);
+static void paired_correlations (const struct tt *tt, struct paired_samp *os);
+static void paired_test (const struct tt *tt, const struct paired_samp *os);
+
+void
+paired_run (const struct tt *tt, size_t n_pairs, vp *pairs, struct casereader *reader)
+{
+ struct ccase *c;
+ struct paired_samp ps;
+ struct casereader *r;
+
+ ps.ps = xcalloc (n_pairs, sizeof *ps.ps);
+ ps.n_ps = n_pairs;
+ for (size_t i = 0; i < n_pairs; ++i)
+ {
+ vp *pair = &pairs[i];
+ struct pair_stats *pp = &ps.ps[i];
+ pp->var0 = (*pair)[0];
+ pp->var1 = (*pair)[1];
+ pp->mom0 = moments_create (MOMENT_VARIANCE);
+ pp->mom1 = moments_create (MOMENT_VARIANCE);
+ pp->mom_diff = moments_create (MOMENT_VARIANCE);
+ }
+
+ r = casereader_clone (reader);
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+
+ for (int i = 0; i < ps.n_ps; i++)
+ {
+ struct pair_stats *pp = &ps.ps[i];
+ const union value *val0 = case_data (c, pp->var0);
+ const union value *val1 = case_data (c, pp->var1);
+ if (var_is_value_missing (pp->var0, val0) & tt->exclude)
+ continue;
+
+ if (var_is_value_missing (pp->var1, val1) & tt->exclude)
+ continue;
+
+ moments_pass_one (pp->mom0, val0->f, w);
+ moments_pass_one (pp->mom1, val1->f, w);
+ moments_pass_one (pp->mom_diff, val0->f - val1->f, w);
+ }
+ }
+ casereader_destroy (r);
+
+ r = reader;
+ for (; (c = casereader_read (r)); case_unref (c))
+ {
+ double w = dict_get_case_weight (tt->dict, c, NULL);
+
+ for (int i = 0; i < ps.n_ps; i++)
+ {
+ struct pair_stats *pp = &ps.ps[i];
+ const union value *val0 = case_data (c, pp->var0);
+ const union value *val1 = case_data (c, pp->var1);
+ if (var_is_value_missing (pp->var0, val0) & tt->exclude)
+ continue;
+
+ if (var_is_value_missing (pp->var1, val1) & tt->exclude)
+ continue;
+
+ moments_pass_two (pp->mom0, val0->f, w);
+ moments_pass_two (pp->mom1, val1->f, w);
+ moments_pass_two (pp->mom_diff, val0->f - val1->f, w);
+ pp->sum_of_prod += val0->f * val1->f * w;
+ }
+ }
+ casereader_destroy (r);
+
+ paired_summary (tt, &ps);
+ paired_correlations (tt, &ps);
+ paired_test (tt, &ps);
+
+ /* Clean up */
+
+ for (int i = 0; i < ps.n_ps; i++)
+ {
+ struct pair_stats *pp = &ps.ps[i];
+ moments_destroy (pp->mom0);
+ moments_destroy (pp->mom1);
+ moments_destroy (pp->mom_diff);
+ }
+ free (ps.ps);
+}
+
+static void
+paired_summary (const struct tt *tt, struct paired_samp *os)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Paired Sample Statistics"));
+ pivot_table_set_weight_var (table, tt->wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("S.E. Mean"), PIVOT_RC_OTHER);
+
+ struct pivot_dimension *variables = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Variables"));
+
+ for (size_t i = 0; i < os->n_ps; i++)
+ {
+ struct pair_stats *pp = &os->ps[i];
+ struct pivot_category *pair = pivot_category_create_group__ (
+ variables->root, pivot_value_new_text_format (N_("Pair %zu"), i + 1));
+
+ for (int j = 0; j < 2; j++)
+ {
+ const struct variable *var = j ? pp->var1 : pp->var0;
+ const struct moments *mom = j ? pp->mom1 : pp->mom0;
+ double cc, mean, sigma;
+ moments_calculate (mom, &cc, &mean, &sigma, NULL, NULL);
+
+ int var_idx = pivot_category_create_leaf (
+ pair, pivot_value_new_variable (var));
+
+ double entries[] = { cc, mean, sqrt (sigma), sqrt (sigma / cc) };
+ for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
+ pivot_table_put2 (table, j, var_idx,
+ pivot_value_new_number (entries[j]));
+ }
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+paired_correlations (const struct tt *tt, struct paired_samp *os)
+{
+ struct pivot_table *table = pivot_table_create (
+ N_("Paired Samples Correlations"));
+ pivot_table_set_weight_var (table, tt->wv);
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Correlation"), PIVOT_RC_CORRELATION,
+ N_("Sig."), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Pairs"));
+
+ for (size_t i = 0; i < os->n_ps; i++)
+ {
+ struct pair_stats *pp = &os->ps[i];
+ struct pivot_category *group = pivot_category_create_group__ (
+ pairs->root, pivot_value_new_text_format (N_("Pair %zu"), i + 1));
+
+ int row = pivot_category_create_leaf (
+ group, pivot_value_new_text_format (N_("%s & %s"),
+ var_to_string (pp->var0),
+ var_to_string (pp->var1)));
+
+ double cc0, mean0, sigma0;
+ double cc1, mean1, sigma1;
+ moments_calculate (pp->mom0, &cc0, &mean0, &sigma0, NULL, NULL);
+ moments_calculate (pp->mom1, &cc1, &mean1, &sigma1, NULL, NULL);
+ /* If this fails, then we're not dealing with missing values properly */
+ assert (cc0 == cc1);
+
+ double corr = ((pp->sum_of_prod / cc0 - mean0 * mean1)
+ / sqrt (sigma0 * sigma1) * cc0 / (cc0 - 1));
+ double sig = 2.0 * significance_of_correlation (corr, cc0);
+ double entries[] = { cc0, corr, sig };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+paired_test (const struct tt *tt, const struct paired_samp *os)
+{
+ struct pivot_table *table = pivot_table_create (N_("Paired Samples Test"));
+ pivot_table_set_weight_var (table, tt->wv);
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Statistics"));
+ struct pivot_category *group = pivot_category_create_group (
+ statistics->root, N_("Paired Differences"),
+ N_("Mean"), PIVOT_RC_OTHER,
+ N_("Std. Deviation"), PIVOT_RC_OTHER,
+ N_("S.E. Mean"), PIVOT_RC_OTHER);
+ struct pivot_category *interval = pivot_category_create_group__ (
+ group, pivot_value_new_text_format (
+ N_("%g%% Confidence Interval of the Difference"),
+ tt->confidence * 100.0));
+ pivot_category_create_leaves (interval,
+ N_("Lower"), PIVOT_RC_OTHER,
+ N_("Upper"), PIVOT_RC_OTHER);
+ pivot_category_create_leaves (statistics->root,
+ N_("t"), PIVOT_RC_OTHER,
+ N_("df"), PIVOT_RC_COUNT,
+ N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Pairs"));
+
+ for (size_t i = 0; i < os->n_ps; i++)
+ {
+ struct pair_stats *pp = &os->ps[i];
+ struct pivot_category *group = pivot_category_create_group__ (
+ pairs->root, pivot_value_new_text_format (N_("Pair %zu"), i + 1));
+
+ int row = pivot_category_create_leaf (
+ group, pivot_value_new_text_format (N_("%s - %s"),
+ var_to_string (pp->var0),
+ var_to_string (pp->var1)));
+
+ double cc, mean, sigma;
+ moments_calculate (pp->mom_diff, &cc, &mean, &sigma, NULL, NULL);
+
+ double df = cc - 1.0;
+
+ double t = mean * sqrt (cc / sigma);
+ double se_mean = sqrt (sigma / cc);
+
+ double p = gsl_cdf_tdist_P (t, df);
+ double q = gsl_cdf_tdist_Q (t, df);
+ double sig = 2.0 * (t > 0 ? q : p);
+
+ double t_qinv = gsl_cdf_tdist_Qinv ((1.0 - tt->confidence) / 2.0, df);
+
+ double entries[] = {
+ mean,
+ sqrt (sigma),
+ se_mean,
+ mean - t_qinv * se_mean,
+ mean + t_qinv * se_mean,
+ t,
+ df,
+ sig,
+ };
+ for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
+ pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
+
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011, 2015 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "libpspp/message.h"
+
+#include "data/casegrouper.h"
+#include "data/casereader.h"
+#include "data/dictionary.h"
+#include "data/dataset.h"
+#include "data/missing-values.h"
+
+#include "language/lexer/lexer.h"
+#include "language/command.h"
+#include "language/lexer/variable-parser.h"
+#include "language/lexer/value-parser.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+#include "t-test.h"
+
+int
+cmd_t_test (struct lexer *lexer, struct dataset *ds)
+{
+ bool ok = false;
+
+ /* Variables pertaining to the paired mode */
+ const struct variable **v1 = NULL;
+ size_t n_v1 = 0;
+ const struct variable **v2 = NULL;
+ size_t n_v2 = 0;
+
+ size_t n_pairs = 0;
+ vp *pairs = NULL;
+
+ /* One sample mode */
+ double testval = SYSMIS;
+
+ /* Independent samples mode */
+ const struct variable *gvar;
+ union value gval0;
+ union value gval1;
+ int gval_width = -1;
+ bool cut = false;
+
+ const struct dictionary *dict = dataset_dict (ds);
+ struct tt tt = {
+ .wv = dict_get_weight (dict),
+ .dict = dict,
+ .confidence = 0.95,
+ .exclude = MV_ANY,
+ .missing_type = MISS_ANALYSIS,
+ .n_vars = 0,
+ .vars = NULL,
+ };
+
+ lex_match (lexer, T_EQUALS);
+
+ int mode_count = 0;
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ lex_match (lexer, T_SLASH);
+ if (lex_match_id (lexer, "TESTVAL"))
+ {
+ mode_count++;
+ tt.mode = MODE_SINGLE;
+ lex_match (lexer, T_EQUALS);
+ if (!lex_force_num (lexer))
+ goto exit;
+ testval = lex_number (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_match_id (lexer, "GROUPS"))
+ {
+ mode_count++;
+ cut = false;
+ tt.mode = MODE_INDEP;
+ lex_match (lexer, T_EQUALS);
+
+ int groups_start = lex_ofs (lexer);
+ gvar = parse_variable (lexer, dict);
+ if (!gvar)
+ goto exit;
+
+ gval_width = var_get_width (gvar);
+ value_init (&gval0, gval_width);
+ value_init (&gval1, gval_width);
+
+ int n;
+ if (lex_match (lexer, T_LPAREN))
+ {
+ if (!parse_value (lexer, &gval0, gvar))
+ goto exit;
+ if (lex_token (lexer) != T_RPAREN)
+ {
+ lex_match (lexer, T_COMMA);
+ if (!parse_value (lexer, &gval1, gvar))
+ goto exit;
+ cut = false;
+ n = 2;
+ }
+ else
+ {
+ cut = true;
+ n = 1;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto exit;
+ }
+ else
+ {
+ gval0.f = 1.0;
+ gval1.f = 2.0;
+ cut = false;
+ n = 0;
+ }
+ int groups_end = lex_ofs (lexer) - 1;
+
+ if (n != 2 && var_is_alpha (gvar))
+ {
+ lex_ofs_error (lexer, groups_start, groups_end,
+ _("When applying %s to a string variable, two "
+ "values must be specified."), "GROUPS");
+ goto exit;
+ }
+ }
+ else if (lex_match_id (lexer, "PAIRS"))
+ {
+ bool with = false;
+ bool paired = false;
+
+ if (tt.n_vars > 0)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s subcommand may not be used with %s."),
+ "VARIABLES", "PAIRS");
+ goto exit;
+ }
+
+ mode_count++;
+ tt.mode = MODE_PAIRED;
+ lex_match (lexer, T_EQUALS);
+
+ int vars_start = lex_ofs (lexer);
+ if (!parse_variables_const (lexer, dict,
+ &v1, &n_v1,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto exit;
+
+ if (lex_match (lexer, T_WITH))
+ {
+ with = true;
+ if (!parse_variables_const (lexer, dict,
+ &v2, &n_v2,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto exit;
+ int vars_end = lex_ofs (lexer) - 1;
+
+ if (lex_match_phrase (lexer, "(PAIRED)"))
+ {
+ paired = true;
+ if (n_v1 != n_v2)
+ {
+ lex_ofs_error (lexer, vars_start, vars_end,
+ _("PAIRED was specified, but the number "
+ "of variables preceding WITH (%zu) "
+ "does not match the number following "
+ "(%zu)."),
+ n_v1, n_v2);
+ goto exit;
+ }
+ }
+ }
+
+ n_pairs = (paired ? n_v1
+ : with ? n_v1 * n_v2
+ : (n_v1 * (n_v1 - 1)) / 2.0);
+ pairs = xcalloc (n_pairs, sizeof *pairs);
+
+ if (paired)
+ {
+ for (size_t i = 0; i < n_v1; ++i)
+ {
+ vp *pair = &pairs[i];
+ (*pair)[0] = v1[i];
+ (*pair)[1] = v2[i];
+ }
+ }
+ else if (with)
+ {
+ int x = 0;
+ for (size_t i = 0; i < n_v1; ++i)
+ for (size_t j = 0; j < n_v2; ++j)
+ {
+ vp *pair = &pairs[x++];
+ (*pair)[0] = v1[i];
+ (*pair)[1] = v2[j];
+ }
+ }
+ else
+ {
+ int x = 0;
+ for (size_t i = 0; i < n_v1; ++i)
+ for (size_t j = i + 1; j < n_v1; ++j)
+ {
+ vp *pair = &pairs[x++];
+ (*pair)[0] = v1[i];
+ (*pair)[1] = v1[j];
+ }
+ }
+ }
+ else if (lex_match_id (lexer, "VARIABLES"))
+ {
+ if (mode_count && tt.mode == MODE_PAIRED)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("%s subcommand may not be used with %s."),
+ "VARIABLES", "PAIRS");
+ goto exit;
+ }
+
+ lex_match (lexer, T_EQUALS);
+
+ if (!parse_variables_const (lexer, dict,
+ &tt.vars,
+ &tt.n_vars,
+ PV_NO_DUPLICATE | PV_NUMERIC))
+ goto exit;
+ }
+ else if (lex_match_id (lexer, "MISSING"))
+ {
+ lex_match (lexer, T_EQUALS);
+ while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
+ {
+ if (lex_match_id (lexer, "INCLUDE"))
+ tt.exclude = MV_SYSTEM;
+ else if (lex_match_id (lexer, "EXCLUDE"))
+ tt.exclude = MV_ANY;
+ else if (lex_match_id (lexer, "LISTWISE"))
+ tt.missing_type = MISS_LISTWISE;
+ else if (lex_match_id (lexer, "ANALYSIS"))
+ tt.missing_type = MISS_ANALYSIS;
+ else
+ {
+ lex_error_expecting (lexer, "INCLUDE", "EXCLUDE",
+ "LISTWISE", "ANALYSIS");
+ goto exit;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ }
+ else if (lex_match_id (lexer, "CRITERIA"))
+ {
+ lex_match (lexer, T_EQUALS);
+ if (!lex_match_id (lexer, "CIN") && !lex_match_id (lexer, "CI"))
+ {
+ lex_error_expecting (lexer, "CIN", "CI");
+ goto exit;
+ }
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto exit;
+ if (!lex_force_num (lexer))
+ goto exit;
+ tt.confidence = lex_number (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto exit;
+ }
+ else
+ {
+ lex_error_expecting (lexer, "TESTVAL", "GROUPS", "PAIRS",
+ "VARIABLES", "MISSING", "CRITERIA");
+ goto exit;
+ }
+ }
+
+ if (mode_count != 1)
+ {
+ msg (SE, _("Exactly one of TESTVAL, GROUPS and PAIRS subcommands "
+ "must be specified."));
+ goto exit;
+ }
+
+ if (tt.n_vars == 0 && tt.mode != MODE_PAIRED)
+ {
+ lex_sbc_missing (lexer, "VARIABLES");
+ goto exit;
+ }
+
+ struct casereader *group;
+ struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
+ while (casegrouper_get_next_group (grouper, &group))
+ switch (tt.mode)
+ {
+ case MODE_SINGLE:
+ if (tt.missing_type == MISS_LISTWISE)
+ group = casereader_create_filter_missing (group, tt.vars, tt.n_vars,
+ tt.exclude, NULL, NULL);
+ one_sample_run (&tt, testval, group);
+ break;
+
+ case MODE_PAIRED:
+ if (tt.missing_type == MISS_LISTWISE)
+ {
+ group = casereader_create_filter_missing (group, v1, n_v1,
+ tt.exclude, NULL, NULL);
+ group = casereader_create_filter_missing (group, v2, n_v2,
+ tt.exclude, NULL, NULL);
+ }
+ paired_run (&tt, n_pairs, pairs, group);
+ break;
+
+ case MODE_INDEP:
+ if (tt.missing_type == MISS_LISTWISE)
+ {
+ group = casereader_create_filter_missing (group, tt.vars, tt.n_vars,
+ tt.exclude, NULL, NULL);
+ group = casereader_create_filter_missing (group, &gvar, 1,
+ tt.exclude, NULL, NULL);
+ }
+ indep_run (&tt, gvar, cut, &gval0, &gval1, group);
+ break;
+ }
+
+ ok = casegrouper_destroy (grouper);
+ ok = proc_commit (ds) && ok;
+
+exit:
+ if (gval_width != -1)
+ {
+ value_destroy (&gval0, gval_width);
+ value_destroy (&gval1, gval_width);
+ }
+ free (pairs);
+ free (v1);
+ free (v2);
+ free (tt.vars);
+
+ return ok ? CMD_SUCCESS : CMD_FAILURE;
+}
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef T_TEST_H
+#define T_TEST_H 1
+
+#include "data/missing-values.h"
+
+struct variable;
+typedef const struct variable *vp[2];
+
+enum missing_type
+ {
+ MISS_LISTWISE,
+ MISS_ANALYSIS,
+ };
+
+enum mode
+ {
+ MODE_PAIRED,
+ MODE_INDEP,
+ MODE_SINGLE,
+ };
+
+struct tt
+{
+ size_t n_vars;
+ const struct variable **vars;
+ enum mode mode;
+ enum missing_type missing_type;
+ enum mv_class exclude;
+ double confidence;
+ const struct variable *wv;
+ const struct dictionary *dict;
+};
+
+struct casereader;
+union value;
+
+void one_sample_run (const struct tt *tt, double testval, struct casereader *reader);
+void paired_run (const struct tt *tt, size_t n_pairs, vp *pairs, struct casereader *reader);
+void indep_run (struct tt *tt, const struct variable *gvar,
+ bool cut,
+ const union value *gval0, const union value *gval1,
+ struct casereader *reader);
+
+
+#endif
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/transformations.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Parses the TEMPORARY command. */
+int
+cmd_temporary (struct lexer *lexer, struct dataset *ds)
+{
+ if (!proc_in_temporary_transformations (ds))
+ proc_start_temporary_transformations (ds);
+ else
+ lex_ofs_error (lexer, 0, 0,
+ _("This command may only appear once between "
+ "procedures and procedure-like commands."));
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2007, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <ctype.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/token.h"
+#include "libpspp/message.h"
+#include "libpspp/start-date.h"
+#include "libpspp/version.h"
+#include "output/driver.h"
+
+#include "gl/c-ctype.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+static int
+parse_title (struct lexer *lexer, void (*set_title) (const char *))
+{
+ if (lex_token (lexer) == T_STRING)
+ {
+ set_title (lex_tokcstr (lexer));
+ lex_get (lexer);
+ }
+ else
+ {
+ int start_ofs = lex_ofs (lexer);
+ while (lex_token (lexer) != T_ENDCMD)
+ lex_get (lexer);
+
+ /* Get the raw representation of all the tokens, including any space
+ between them, and use it as the title. */
+ char *title = lex_ofs_representation (lexer, start_ofs,
+ lex_ofs (lexer) - 1);
+ set_title (title);
+ free (title);
+ }
+ return CMD_SUCCESS;
+}
+
+int
+cmd_title (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ return parse_title (lexer, output_set_title);
+}
+
+int
+cmd_subtitle (struct lexer *lexer, struct dataset *ds UNUSED)
+{
+ return parse_title (lexer, output_set_subtitle);
+}
+
+/* Performs the FILE LABEL command. */
+int
+cmd_file_label (struct lexer *lexer, struct dataset *ds)
+{
+ if (!lex_force_string (lexer))
+ return CMD_FAILURE;
+
+ dict_set_label (dataset_dict (ds), lex_tokcstr (lexer));
+ lex_get (lexer);
+
+ return CMD_SUCCESS;
+}
+
+/* Performs the DOCUMENT command. */
+int
+cmd_document (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ char *trailer;
+
+ if (!lex_force_string (lexer))
+ return CMD_FAILURE;
+
+ while (lex_is_string (lexer))
+ {
+ dict_add_document_line (dict, lex_tokcstr (lexer), true);
+ lex_get (lexer);
+ }
+
+ trailer = xasprintf (_(" (Entered %s)"), get_start_date ());
+ dict_add_document_line (dict, trailer, true);
+ free (trailer);
+
+ return CMD_SUCCESS;
+}
+
+/* Performs the ADD DOCUMENTS command. */
+int
+cmd_add_documents (struct lexer *lexer, struct dataset *ds)
+{
+ return cmd_document (lexer, ds);
+}
+
+/* Performs the DROP DOCUMENTS command. */
+int
+cmd_drop_documents (struct lexer *lexer UNUSED, struct dataset *ds)
+{
+ dict_clear_documents (dataset_dict (ds));
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2007, 2008, 2010, 2011,
+ 2019, 2020 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include "language/commands/trim.h"
+
+#include <stdlib.h>
+
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+/* Commands that read and write system files share a great deal of common
+ syntactic structure for rearranging and dropping variables. This function
+ parses this syntax and modifies DICT appropriately. Returns true on
+ success, false on failure. */
+bool
+parse_dict_trim (struct lexer *lexer, struct dictionary *dict)
+{
+ if (lex_match_id (lexer, "MAP"))
+ {
+ /* FIXME. */
+ return true;
+ }
+ else if (lex_match_id (lexer, "DROP"))
+ return parse_dict_drop (lexer, dict);
+ else if (lex_match_id (lexer, "KEEP"))
+ return parse_dict_keep (lexer, dict);
+ else if (lex_match_id (lexer, "RENAME"))
+ return parse_dict_rename (lexer, dict);
+ else
+ {
+ lex_error_expecting (lexer, "MAP", "DROP", "KEEP", "RENAME");
+ return false;
+ }
+}
+
+/* Parses and performs the RENAME subcommand of GET, SAVE, and
+ related commands. */
+bool
+parse_dict_rename (struct lexer *lexer, struct dictionary *dict)
+{
+ lex_match (lexer, T_EQUALS);
+ int start_ofs = lex_ofs (lexer);
+
+ struct variable **old_vars = NULL;
+ size_t n_old_vars = 0;
+
+ char **new_vars = NULL;
+ size_t n_new_vars = 0;
+
+ bool ok = false;
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ {
+ size_t prev_n_old = n_old_vars;
+ size_t prev_n_new = n_new_vars;
+
+ bool paren = lex_match (lexer, T_LPAREN);
+ int pv_opts = PV_NO_DUPLICATE | PV_APPEND | (paren ? 0 : PV_SINGLE);
+
+ int old_vars_start = lex_ofs (lexer);
+ if (!parse_variables (lexer, dict, &old_vars, &n_old_vars, pv_opts))
+ goto done;
+ int old_vars_end = lex_ofs (lexer) - 1;
+
+ if (!lex_force_match (lexer, T_EQUALS))
+ goto done;
+
+ int new_vars_start = lex_ofs (lexer);
+ if (!parse_DATA_LIST_vars (lexer, dict, &new_vars, &n_new_vars, pv_opts))
+ goto done;
+ int new_vars_end = lex_ofs (lexer) - 1;
+
+ if (paren && !lex_force_match (lexer, T_RPAREN))
+ goto done;
+
+ if (n_new_vars != n_old_vars)
+ {
+ size_t added_old = n_old_vars - prev_n_old;
+ size_t added_new = n_new_vars - prev_n_new;
+
+ msg (SE, _("Old and new variable counts do not match."));
+ lex_ofs_msg (lexer, SN, old_vars_start, old_vars_end,
+ ngettext ("There is %zu old variable.",
+ "There are %zu old variables.", added_old),
+ added_old);
+ lex_ofs_msg (lexer, SN, new_vars_start, new_vars_end,
+ ngettext ("There is %zu new variable name.",
+ "There are %zu new variable names.",
+ added_new),
+ added_new);
+ goto done;
+ }
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+
+ char *dup_name = NULL;
+ if (!dict_rename_vars (dict, old_vars, new_vars, n_new_vars, &dup_name))
+ {
+ lex_ofs_error (lexer, start_ofs, end_ofs,
+ _("Requested renaming duplicates variable name %s."),
+ dup_name);
+ goto done;
+ }
+ ok = true;
+
+done:
+ free (old_vars);
+ for (size_t i = 0; i < n_new_vars; ++i)
+ free (new_vars[i]);
+ free (new_vars);
+ return ok;
+}
+
+/* Parses and performs the DROP subcommand of GET, SAVE, and
+ related commands.
+ Returns true if successful, false on failure.*/
+bool
+parse_dict_drop (struct lexer *lexer, struct dictionary *dict)
+{
+ int start_ofs = lex_ofs (lexer) - 1;
+ lex_match (lexer, T_EQUALS);
+
+ struct variable **v;
+ size_t nv;
+ if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
+ return false;
+ dict_delete_vars (dict, v, nv);
+ free (v);
+
+ if (dict_get_n_vars (dict) == 0)
+ {
+ lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
+ _("Cannot DROP all variables from dictionary."));
+ return false;
+ }
+ return true;
+}
+
+/* Parses and performs the KEEP subcommand of GET, SAVE, and
+ related commands.
+ Returns true if successful, false on failure.*/
+bool
+parse_dict_keep (struct lexer *lexer, struct dictionary *dict)
+{
+ struct variable **v;
+ size_t nv;
+ size_t i;
+
+ lex_match (lexer, T_EQUALS);
+ if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
+ return false;
+
+ /* Move the specified variables to the beginning. */
+ dict_reorder_vars (dict, v, nv);
+
+ /* Delete the remaining variables. */
+ if (dict_get_n_vars (dict) == nv)
+ {
+ free (v);
+ return true;
+ }
+
+ v = xnrealloc (v, dict_get_n_vars (dict) - nv, sizeof *v);
+ for (i = nv; i < dict_get_n_vars (dict); i++)
+ v[i - nv] = dict_get_var (dict, i);
+ dict_delete_vars (dict, v, dict_get_n_vars (dict) - nv);
+ free (v);
+
+ return true;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2006, 2007, 2008 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#ifndef LANGUAGE_DATA_IO_TRIM_H
+#define LANGUAGE_DATA_IO_TRIM_H
+
+#include <stdbool.h>
+
+struct lexer;
+struct dictionary;
+bool parse_dict_trim (struct lexer *, struct dictionary *);
+bool parse_dict_rename (struct lexer *, struct dictionary *);
+bool parse_dict_drop (struct lexer *, struct dictionary *);
+bool parse_dict_keep (struct lexer *, struct dictionary *);
+
+#endif /* trim.c */
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/value-labels.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/value-parser.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+\f
+/* Declarations. */
+
+static int do_value_labels (struct lexer *,
+ const struct dictionary *dict, bool);
+static void erase_labels (struct variable **vars, size_t n_vars);
+static int get_label (struct lexer *, struct variable **vars, size_t n_vars,
+ const char *dict_encoding);
+\f
+/* Stubs. */
+
+int
+cmd_value_labels (struct lexer *lexer, struct dataset *ds)
+{
+ return do_value_labels (lexer, dataset_dict (ds), true);
+}
+
+int
+cmd_add_value_labels (struct lexer *lexer, struct dataset *ds)
+{
+ return do_value_labels (lexer, dataset_dict (ds), false);
+}
+\f
+/* Do it. */
+
+static int
+do_value_labels (struct lexer *lexer, const struct dictionary *dict, bool erase)
+{
+ struct variable **vars; /* Variable list. */
+ size_t n_vars; /* Number of variables. */
+ int parse_err=0; /* true if error parsing variables */
+
+ lex_match (lexer, T_SLASH);
+
+ while (lex_token (lexer) != T_ENDCMD)
+ {
+ parse_err = !parse_variables (lexer, dict, &vars, &n_vars,
+ PV_SAME_WIDTH);
+ if (n_vars < 1)
+ {
+ free(vars);
+ return CMD_FAILURE;
+ }
+ if (erase)
+ erase_labels (vars, n_vars);
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
+ if (!get_label (lexer, vars, n_vars, dict_get_encoding (dict)))
+ goto lossage;
+
+ if (lex_token (lexer) != T_SLASH)
+ {
+ free (vars);
+ break;
+ }
+
+ lex_get (lexer);
+
+ free (vars);
+ }
+
+ return parse_err ? CMD_FAILURE : CMD_SUCCESS;
+
+ lossage:
+ free (vars);
+ return CMD_FAILURE;
+}
+
+/* Erases all the labels for the N_VARS variables in VARS. */
+static void
+erase_labels (struct variable **vars, size_t n_vars)
+{
+ /* Erase old value labels if desired. */
+ for (size_t i = 0; i < n_vars; i++)
+ var_clear_value_labels (vars[i]);
+}
+
+/* Parse all the labels for the N_VARS variables in VARS and add
+ the specified labels to those variables. */
+static int
+get_label (struct lexer *lexer, struct variable **vars, size_t n_vars,
+ const char *dict_encoding)
+{
+ /* Parse all the labels and add them to the variables. */
+ do
+ {
+ enum { MAX_LABEL_LEN = 255 };
+ int width = var_get_width (vars[0]);
+ union value value;
+ struct string label;
+ size_t trunc_len;
+ size_t i;
+
+ /* Set value. */
+ value_init (&value, width);
+ if (!parse_value (lexer, &value, vars[0]))
+ {
+ value_destroy (&value, width);
+ return 0;
+ }
+ lex_match (lexer, T_COMMA);
+
+ /* Set label. */
+ if (lex_token (lexer) != T_ID && !lex_force_string (lexer))
+ {
+ value_destroy (&value, width);
+ return 0;
+ }
+
+ ds_init_substring (&label, lex_tokss (lexer));
+
+ trunc_len = utf8_encoding_trunc_len (ds_cstr (&label), dict_encoding,
+ MAX_LABEL_LEN);
+ if (ds_length (&label) > trunc_len)
+ {
+ lex_next_msg (lexer, SW, 0, 0,
+ _("Truncating value label to %d bytes."),
+ MAX_LABEL_LEN);
+ ds_truncate (&label, trunc_len);
+ }
+
+ for (i = 0; i < n_vars; i++)
+ var_replace_value_label (vars[i], &value, ds_cstr (&label));
+
+ ds_destroy (&label);
+ value_destroy (&value, width);
+
+ lex_get (lexer);
+ lex_match (lexer, T_COMMA);
+ }
+ while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
+
+ return 1;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011, 2013 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_variable_alignment (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ struct variable **v;
+ size_t nv;
+
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
+ return CMD_FAILURE;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ enum alignment align;
+ if (lex_match_id (lexer, "LEFT"))
+ align = ALIGN_LEFT;
+ else if (lex_match_id (lexer, "RIGHT"))
+ align = ALIGN_RIGHT;
+ else if (lex_match_id (lexer, "CENTER"))
+ align = ALIGN_CENTRE;
+ else
+ {
+ lex_error_expecting (lexer, "LEFT", "RIGHT", "CENTER");
+ goto error;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ for (size_t i = 0; i < nv; ++i)
+ var_set_alignment (v[i], align);
+
+ while (lex_token (lexer) == T_SLASH)
+ lex_get (lexer);
+ free (v);
+ continue;
+
+ error:
+ free (v);
+ return CMD_FAILURE;
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+ return CMD_SUCCESS;
+}
+
+int
+cmd_variable_width (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ struct variable **v;
+ size_t nv;
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
+ return CMD_FAILURE;
+
+ if (!lex_force_match (lexer, T_LPAREN)
+ || !lex_force_int_range (lexer, NULL, 1, INT_MAX))
+ goto error;
+ long width = lex_integer (lexer);
+ lex_get (lexer);
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ width = MIN (width, 2 * MAX_STRING);
+
+ for (size_t i = 0; i < nv; ++i)
+ var_set_display_width (v[i], width);
+
+ while (lex_token (lexer) == T_SLASH)
+ lex_get (lexer);
+ free (v);
+ continue;
+
+ error:
+ free (v);
+ return CMD_FAILURE;
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+ return CMD_SUCCESS;
+}
+
+/* Set variables' measurement level */
+int
+cmd_variable_level (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ struct variable **v;
+ size_t nv;
+
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
+ return CMD_FAILURE;
+
+ if (!lex_force_match (lexer, T_LPAREN))
+ goto error;
+
+ enum measure level;
+ if (lex_match_id (lexer, "SCALE"))
+ level = MEASURE_SCALE;
+ else if (lex_match_id (lexer, "ORDINAL"))
+ level = MEASURE_ORDINAL;
+ else if (lex_match_id (lexer, "NOMINAL"))
+ level = MEASURE_NOMINAL;
+ else
+ {
+ lex_error_expecting (lexer, "SCALE", "ORDINAL", "NOMINAL");
+ goto error;
+ }
+
+ if (!lex_force_match (lexer, T_RPAREN))
+ goto error;
+
+ for (size_t i = 0; i < nv; ++i)
+ var_set_measure (v[i], level);
+
+ while (lex_token (lexer) == T_SLASH)
+ lex_get (lexer);
+ free (v);
+ continue;
+
+ error:
+ free (v);
+ return CMD_FAILURE;
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+ return CMD_SUCCESS;
+}
+
+int
+cmd_variable_role (struct lexer *lexer, struct dataset *ds)
+{
+ do
+ {
+ if (!lex_force_match (lexer, T_SLASH))
+ return CMD_FAILURE;
+
+ enum var_role role;
+ if (lex_match_id (lexer, "INPUT"))
+ role = ROLE_INPUT;
+ else if (lex_match_id (lexer, "TARGET"))
+ role = ROLE_TARGET;
+ else if (lex_match_id (lexer, "BOTH"))
+ role = ROLE_BOTH;
+ else if (lex_match_id (lexer, "NONE"))
+ role = ROLE_NONE;
+ else if (lex_match_id (lexer, "PARTITION"))
+ role = ROLE_PARTITION;
+ else if (lex_match_id (lexer, "SPLIT"))
+ role = ROLE_SPLIT;
+ else
+ {
+ lex_error_expecting (lexer, "INPUT", "TARGET", "BOTH",
+ "NONE", "PARTITION", "SPLIT");
+ return CMD_FAILURE;
+ }
+
+ struct variable **v;
+ size_t nv;
+ if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
+ return CMD_FAILURE;
+
+ for (size_t i = 0; i < nv; i++)
+ var_set_role (v[i], role);
+
+ free (v);
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011, 2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_variable_labels (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+
+ do
+ {
+ struct variable **v;
+ size_t nv;
+
+ size_t i;
+
+ if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
+ return CMD_FAILURE;
+
+ if (!lex_force_string (lexer))
+ {
+ free (v);
+ return CMD_FAILURE;
+ }
+
+ for (i = 0; i < nv; i++)
+ var_set_label (v[i], lex_tokcstr (lexer));
+
+ lex_get (lexer);
+ while (lex_token (lexer) == T_SLASH)
+ lex_get (lexer);
+ free (v);
+ }
+ while (lex_token (lexer) != T_ENDCMD);
+ return CMD_SUCCESS;
+}
+
+
+
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011, 2012, 2016 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdlib.h>
+
+#include "data/dataset.h"
+#include "data/format.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/format-parser.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/assertion.h"
+#include "libpspp/i18n.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "libpspp/pool.h"
+#include "libpspp/str.h"
+
+#include "gl/intprops.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_vector (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ struct pool *pool = pool_create ();
+
+ do
+ {
+ /* Get the name(s) of the new vector(s). */
+ if (!lex_force_id (lexer))
+ goto error;
+
+ int vectors_start = lex_ofs (lexer);
+ char **vectors = NULL;
+ size_t n_vectors = 0;
+ size_t allocated_vectors = 0;
+ while (lex_token (lexer) == T_ID)
+ {
+ char *error = dict_id_is_valid__ (dict, lex_tokcstr (lexer));
+ if (error)
+ {
+ lex_error (lexer, "%s", error);
+ free (error);
+ goto error;
+ }
+
+ if (dict_lookup_vector (dict, lex_tokcstr (lexer)))
+ {
+ lex_next_error (lexer, 0, 0,
+ _("A vector named %s already exists."),
+ lex_tokcstr (lexer));
+ goto error;
+ }
+
+ for (size_t i = 0; i < n_vectors; i++)
+ if (!utf8_strcasecmp (vectors[i], lex_tokcstr (lexer)))
+ {
+ lex_ofs_error (lexer, vectors_start, lex_ofs (lexer),
+ _("Vector name %s is given twice."),
+ lex_tokcstr (lexer));
+ goto error;
+ }
+
+ if (n_vectors == allocated_vectors)
+ vectors = pool_2nrealloc (pool, vectors, &allocated_vectors,
+ sizeof *vectors);
+ vectors[n_vectors++] = pool_strdup (pool, lex_tokcstr (lexer));
+
+ lex_get (lexer);
+ lex_match (lexer, T_COMMA);
+ }
+
+ /* Now that we have the names it's time to check for the short
+ or long forms. */
+ if (lex_match (lexer, T_EQUALS))
+ {
+ if (n_vectors > 1)
+ {
+ lex_ofs_error (lexer, vectors_start, lex_ofs (lexer) - 1,
+ _("Only a single vector name may be specified "
+ "when a list of variables is given."));
+ goto error;
+ }
+
+ struct variable **v;
+ size_t nv;
+ if (!parse_variables_pool (lexer, pool, dict, &v, &nv,
+ PV_SAME_WIDTH | PV_DUPLICATE))
+ goto error;
+
+ dict_create_vector (dict, vectors[0], v, nv);
+ }
+ else if (lex_match (lexer, T_LPAREN))
+ {
+ struct fmt_spec format = fmt_for_output (FMT_F, 8, 2);
+ bool seen_format = false;
+ size_t n_vars = 0;
+ int name_ofs = lex_ofs (lexer) - 2;
+ int lparen_ofs = lex_ofs (lexer) - 1;
+ while (!lex_match (lexer, T_RPAREN))
+ {
+ if (lex_is_integer (lexer))
+ {
+ if (n_vars)
+ {
+ lex_ofs_error (lexer, lparen_ofs, lex_ofs (lexer),
+ _("Vector length may only be specified "
+ "once."));
+ goto error;
+ }
+ if (!lex_force_int_range (lexer, NULL, 1, INT_MAX))
+ goto error;
+ n_vars = lex_integer (lexer);
+ lex_get (lexer);
+ }
+ else if (lex_token (lexer) == T_ID)
+ {
+ if (seen_format)
+ {
+ lex_ofs_error (lexer, lparen_ofs, lex_ofs (lexer),
+ _("Only one format may be specified."));
+ goto error;
+ }
+ seen_format = true;
+ if (!parse_format_specifier (lexer, &format))
+ goto error;
+ char *error = fmt_check_output__ (&format);
+ if (error)
+ {
+ lex_next_error (lexer, -1, -1, "%s", error);
+ free (error);
+ goto error;
+ }
+ }
+ else
+ {
+ lex_error (lexer, _("Syntax error expecting vector length "
+ "or format."));
+ goto error;
+ }
+ lex_match (lexer, T_COMMA);
+ }
+ int end_ofs = lex_ofs (lexer) - 1;
+ if (n_vars == 0)
+ {
+ lex_ofs_error (lexer, lparen_ofs, end_ofs,
+ _("Vector length is required."));
+ goto error;
+ }
+
+ /* Check that none of the variables exist and that their names are
+ not excessively long. */
+ for (size_t i = 0; i < n_vectors; i++)
+ for (size_t j = 0; j < n_vars; j++)
+ {
+ char *name = xasprintf ("%s%zu", vectors[i], j + 1);
+ char *error = dict_id_is_valid__ (dict, name);
+ if (error)
+ {
+ lex_ofs_error (lexer, name_ofs, end_ofs, "%s", error);
+ free (error);
+ free (name);
+ goto error;
+ }
+ if (dict_lookup_var (dict, name))
+ {
+ lex_ofs_error (lexer, name_ofs, end_ofs,
+ _("%s is an existing variable name."),
+ name);
+ free (name);
+ goto error;
+ }
+ free (name);
+ }
+
+ /* Finally create the variables and vectors. */
+ struct variable **vars = pool_nmalloc (pool, n_vars, sizeof *vars);
+ for (size_t i = 0; i < n_vectors; i++)
+ {
+ for (size_t j = 0; j < n_vars; j++)
+ {
+ char *name = xasprintf ("%s%zu", vectors[i], j + 1);
+ vars[j] = dict_create_var_assert (dict, name,
+ fmt_var_width (&format));
+ var_set_both_formats (vars[j], &format);
+ free (name);
+ }
+ dict_create_vector_assert (dict, vectors[i], vars, n_vars);
+ }
+ }
+ else
+ {
+ lex_error_expecting (lexer, "`='", "`('");
+ goto error;
+ }
+ }
+ while (lex_match (lexer, T_SLASH));
+
+ pool_destroy (pool);
+ return CMD_SUCCESS;
+
+error:
+ pool_destroy (pool);
+ return CMD_FAILURE;
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#include <config.h>
+
+#include <stdio.h>
+
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/variable.h"
+#include "language/command.h"
+#include "language/lexer/lexer.h"
+#include "language/lexer/variable-parser.h"
+#include "libpspp/message.h"
+#include "libpspp/str.h"
+
+#include "gettext.h"
+#define _(msgid) gettext (msgid)
+
+int
+cmd_weight (struct lexer *lexer, struct dataset *ds)
+{
+ struct dictionary *dict = dataset_dict (ds);
+ if (lex_match_id (lexer, "OFF"))
+ dict_set_weight (dataset_dict (ds), NULL);
+ else
+ {
+ struct variable *v;
+
+ lex_match (lexer, T_BY);
+ v = parse_variable (lexer, dict);
+ if (!v)
+ return CMD_CASCADING_FAILURE;
+ if (var_is_alpha (v))
+ {
+ lex_next_error (lexer, -1, -1,
+ _("The weighting variable must be numeric."));
+ return CMD_CASCADING_FAILURE;
+ }
+ if (dict_class_from_id (var_get_name (v)) == DC_SCRATCH)
+ {
+ lex_next_error (lexer, -1, -1,
+ _("The weighting variable may not be scratch."));
+ return CMD_CASCADING_FAILURE;
+ }
+
+ dict_set_weight (dict, v);
+ }
+
+ return CMD_SUCCESS;
+}
--- /dev/null
+/* Pspp - a program for statistical analysis.
+ Copyright (C) 2008, 2009, 2010, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+
+
+#include <config.h>
+
+#include "language/commands/wilcoxon.h"
+
+#include <gsl/gsl_cdf.h>
+#include <math.h>
+
+#include "data/casereader.h"
+#include "data/casewriter.h"
+#include "data/dataset.h"
+#include "data/dictionary.h"
+#include "data/format.h"
+#include "data/subcase.h"
+#include "data/variable.h"
+#include "libpspp/assertion.h"
+#include "libpspp/message.h"
+#include "libpspp/misc.h"
+#include "math/sort.h"
+#include "math/wilcoxon-sig.h"
+#include "output/pivot-table.h"
+
+#include "gl/minmax.h"
+#include "gl/xalloc.h"
+
+#include "gettext.h"
+#define N_(msgid) msgid
+#define _(msgid) gettext (msgid)
+
+static double
+append_difference (const struct ccase *c, casenumber n UNUSED, void *aux)
+{
+ const variable_pair *vp = aux;
+
+ return case_num (c, (*vp)[0]) - case_num (c, (*vp)[1]);
+}
+
+static void show_ranks_box (const struct wilcoxon_state *,
+ const struct two_sample_test *,
+ const struct dictionary *);
+
+static void show_tests_box (const struct wilcoxon_state *,
+ const struct two_sample_test *,
+ bool exact, double timer);
+
+
+
+static void
+distinct_callback (double v UNUSED, casenumber n, double w UNUSED, void *aux)
+{
+ struct wilcoxon_state *ws = aux;
+
+ ws->tiebreaker += pow3 (n) - n;
+}
+
+#define WEIGHT_IDX 2
+
+void
+wilcoxon_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer)
+{
+ int i;
+ bool warn = true;
+ const struct dictionary *dict = dataset_dict (ds);
+ const struct two_sample_test *t2s = UP_CAST (test, const struct two_sample_test, parent);
+
+ struct wilcoxon_state *ws = XCALLOC (t2s->n_pairs, struct wilcoxon_state);
+ const struct variable *weight = dict_get_weight (dict);
+ struct variable *weightx = dict_create_internal_var (WEIGHT_IDX, 0);
+ struct caseproto *proto;
+
+ input =
+ casereader_create_filter_weight (input, dict, &warn, NULL);
+
+ proto = caseproto_create ();
+ proto = caseproto_add_width (proto, 0);
+ proto = caseproto_add_width (proto, 0);
+ if (weight != NULL)
+ proto = caseproto_add_width (proto, 0);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ struct casereader *r = casereader_clone (input);
+ struct casewriter *writer;
+ struct ccase *c;
+ struct subcase ordering;
+ variable_pair *vp = &t2s->pairs[i];
+
+ ws[i].sign = dict_create_internal_var (0, 0);
+ ws[i].absdiff = dict_create_internal_var (1, 0);
+
+ r = casereader_create_filter_missing (r, *vp, 2,
+ exclude,
+ NULL, NULL);
+
+ subcase_init_var (&ordering, ws[i].absdiff, SC_ASCEND);
+ writer = sort_create_writer (&ordering, proto);
+ subcase_uninit (&ordering);
+
+ for (; (c = casereader_read (r)) != NULL; case_unref (c))
+ {
+ struct ccase *output = case_create (proto);
+ double d = append_difference (c, 0, vp);
+
+ if (d > 0)
+ *case_num_rw (output, ws[i].sign) = 1.0;
+ else if (d < 0)
+ *case_num_rw (output, ws[i].sign) = -1.0;
+ else
+ {
+ double w = 1.0;
+ if (weight)
+ w = case_num (c, weight);
+
+ /* Central point values should be dropped */
+ ws[i].n_zeros += w;
+ case_unref (output);
+ continue;
+ }
+
+ *case_num_rw (output, ws[i].absdiff) = fabs (d);
+
+ if (weight)
+ *case_num_rw (output, weightx) = case_num (c, weight);
+
+ casewriter_write (writer, output);
+ }
+ casereader_destroy (r);
+ ws[i].reader = casewriter_make_reader (writer);
+ }
+ caseproto_unref (proto);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ struct casereader *rr ;
+ struct ccase *c;
+ enum rank_error err = 0;
+
+ rr = casereader_create_append_rank (ws[i].reader, ws[i].absdiff,
+ weight ? weightx : NULL, &err,
+ distinct_callback, &ws[i]
+ );
+
+ for (; (c = casereader_read (rr)) != NULL; case_unref (c))
+ {
+ double sign = case_num (c, ws[i].sign);
+ double rank = case_num_idx (c, weight ? 3 : 2);
+ double w = weight ? case_num (c, weightx) : 1.0;
+
+ if (sign > 0)
+ {
+ ws[i].positives.sum += rank * w;
+ ws[i].positives.n += w;
+ }
+ else if (sign < 0)
+ {
+ ws[i].negatives.sum += rank * w;
+ ws[i].negatives.n += w;
+ }
+ else
+ NOT_REACHED ();
+ }
+
+ casereader_destroy (rr);
+ }
+
+ casereader_destroy (input);
+
+ dict_destroy_internal_var (weightx);
+
+ show_ranks_box (ws, t2s, dict);
+ show_tests_box (ws, t2s, exact, timer);
+
+ for (i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ dict_destroy_internal_var (ws[i].sign);
+ dict_destroy_internal_var (ws[i].absdiff);
+ }
+
+ free (ws);
+}
+\f
+static void
+put_row (struct pivot_table *table, int var_idx, int sign_idx,
+ double n, double sum)
+{
+ pivot_table_put3 (table, 0, sign_idx, var_idx, pivot_value_new_number (n));
+ if (sum != SYSMIS)
+ {
+ pivot_table_put3 (table, 1, sign_idx, var_idx,
+ pivot_value_new_number (sum / n));
+ pivot_table_put3 (table, 2, sign_idx, var_idx,
+ pivot_value_new_number (sum));
+ }
+}
+
+static int
+add_pair_leaf (struct pivot_dimension *dimension, variable_pair *pair)
+{
+ char *label = xasprintf ("%s - %s", var_to_string ((*pair)[0]),
+ var_to_string ((*pair)[1]));
+ return pivot_category_create_leaf (
+ dimension->root,
+ pivot_value_new_user_text_nocopy (label));
+}
+
+static void
+show_ranks_box (const struct wilcoxon_state *ws,
+ const struct two_sample_test *t2s,
+ const struct dictionary *dict)
+{
+ struct pivot_table *table = pivot_table_create (N_("Ranks"));
+ pivot_table_set_weight_var (table, dict_get_weight (dict));
+
+ pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
+ N_("N"), PIVOT_RC_COUNT,
+ N_("Mean Rank"), PIVOT_RC_OTHER,
+ N_("Sum of Ranks"), PIVOT_RC_OTHER);
+
+ pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Sign"),
+ N_("Negative Ranks"), N_("Positive Ranks"),
+ N_("Ties"), N_("Total"));
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Pairs"));
+
+ for (size_t i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+ int pair_idx = add_pair_leaf (pairs, vp);
+
+ const struct wilcoxon_state *w = &ws[i];
+ put_row (table, pair_idx, 0, w->negatives.n, w->negatives.sum);
+ put_row (table, pair_idx, 1, w->positives.n, w->positives.sum);
+ put_row (table, pair_idx, 2, w->n_zeros, SYSMIS);
+ put_row (table, pair_idx, 3,
+ w->n_zeros + w->positives.n + w->negatives.n, SYSMIS);
+ }
+
+ pivot_table_submit (table);
+}
+
+
+static void
+show_tests_box (const struct wilcoxon_state *ws,
+ const struct two_sample_test *t2s,
+ bool exact,
+ double timer UNUSED
+ )
+{
+ struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
+
+ struct pivot_dimension *statistics = pivot_dimension_create (
+ table, PIVOT_AXIS_ROW, N_("Statistics"),
+ N_("Z"), PIVOT_RC_OTHER,
+ N_("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
+ if (exact)
+ pivot_category_create_leaves (
+ statistics->root,
+ N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
+ N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE);
+
+ struct pivot_dimension *pairs = pivot_dimension_create (
+ table, PIVOT_AXIS_COLUMN, N_("Pairs"));
+
+ struct pivot_footnote *too_many_pairs = pivot_table_create_footnote (
+ table, pivot_value_new_text (
+ N_("Too many pairs to calculate exact significance")));
+
+ for (size_t i = 0 ; i < t2s->n_pairs; ++i)
+ {
+ variable_pair *vp = &t2s->pairs[i];
+ int pair_idx = add_pair_leaf (pairs, vp);
+
+ double n = ws[i].positives.n + ws[i].negatives.n;
+ double z = MIN (ws[i].positives.sum, ws[i].negatives.sum);
+ z -= n * (n + 1)/ 4.0;
+ z /= sqrt (n * (n + 1) * (2*n + 1)/24.0 - ws[i].tiebreaker / 48.0);
+
+ double entries[4];
+ int n_entries = 0;
+ entries[n_entries++] = z;
+ entries[n_entries++] = 2.0 * gsl_cdf_ugaussian_P (z);
+
+ int footnote_idx = -1;
+ if (exact)
+ {
+ double p = LevelOfSignificanceWXMPSR (ws[i].positives.sum, n);
+ if (p < 0)
+ {
+ footnote_idx = n_entries;
+ entries[n_entries++] = SYSMIS;
+ }
+ else
+ {
+ entries[n_entries++] = p;
+ entries[n_entries++] = p / 2.0;
+ }
+ }
+
+ for (int j = 0; j < n_entries; j++)
+ {
+ struct pivot_value *value = pivot_value_new_number (entries[j]);
+ if (j == footnote_idx)
+ pivot_value_add_footnote (value, too_many_pairs);
+ pivot_table_put2 (table, j, pair_idx, value);
+ }
+ }
+
+ pivot_table_submit (table);
+}
--- /dev/null
+/* PSPP - a program for statistical analysis.
+ Copyright (C) 2008, 2011 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+#if !wilcoxon_h
+#define wilcoxon_h 1
+
+#include <stddef.h>
+#include <stdbool.h>
+
+#include "data/case.h"
+#include "language/commands/npar.h"
+
+struct rank_sum
+{
+ double n;
+ double sum;
+};
+
+struct wilcoxon_state
+{
+ struct casereader *reader;
+ struct variable *sign;
+ struct variable *absdiff;
+
+ struct rank_sum positives;
+ struct rank_sum negatives;
+ double n_zeros;
+
+ double tiebreaker;
+};
+
+
+struct wilcoxon_test
+{
+ struct two_sample_test parent;
+};
+
+struct casereader;
+struct dataset;
+
+
+void wilcoxon_execute (const struct dataset *ds,
+ struct casereader *input,
+ enum mv_class exclude,
+ const struct npar_test *test,
+ bool exact,
+ double timer
+ );
+
+
+
+#endif
+++ /dev/null
-# PSPP - a program for statistical analysis.
-# Copyright (C) 2017 Free Software Foundation, Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-## Process this file with automake to produce Makefile.in -*- makefile -*-
-
-
-language_control_sources = \
- src/language/control/define.c \
- src/language/control/do-if.c \
- src/language/control/loop.c \
- src/language/control/repeat.c \
- src/language/control/temporary.c
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2021 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <limits.h>
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/macro.h"
-#include "language/lexer/scan.h"
-#include "language/lexer/token.h"
-#include "libpspp/intern.h"
-#include "libpspp/message.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-static bool
-match_macro_id (struct lexer *lexer, const char *keyword)
-{
- if (keyword[0] != '!')
- return lex_match_id (lexer, keyword);
- else if (lex_token (lexer) == T_MACRO_ID
- && lex_id_match_n (ss_cstr (keyword), lex_tokss (lexer), 4))
- {
- lex_get (lexer);
- return true;
- }
- else
- return false;
-}
-
-/* Obtains a quoted string from LEXER and then tokenizes the quoted string's
- content to obtain a single TOKEN. Returns true if successful, false
- otherwise. The caller takes ownership of TOKEN on success, otherwise TOKEN
- is indeterminate. */
-static bool
-parse_quoted_token (struct lexer *lexer, struct token *token)
-{
- if (!lex_force_string (lexer))
- return false;
-
- struct substring s = lex_tokss (lexer);
- struct string_lexer slex;
- string_lexer_init (&slex, s.string, s.length, SEG_MODE_INTERACTIVE, true);
- struct token another_token = { .type = T_STOP };
- if (string_lexer_next (&slex, token) != SLR_TOKEN
- || string_lexer_next (&slex, &another_token) != SLR_END)
- {
- token_uninit (token);
- token_uninit (&another_token);
- lex_error (lexer, _("String must contain exactly one token."));
- return false;
- }
- lex_get (lexer);
- return true;
-}
-
-static bool
-dup_arg_type (struct lexer *lexer, bool *saw_arg_type)
-{
- if (*saw_arg_type)
- {
- lex_next_error (lexer, -1, -1,
- _("Only one of !TOKENS, !CHAREND, !ENCLOSE, or "
- "!CMDEND is allowed."));
- return false;
- }
- else
- {
- *saw_arg_type = true;
- return true;
- }
-}
-
-static bool
-parse_macro_body (struct lexer *lexer, struct macro_tokens *mts)
-{
- *mts = (struct macro_tokens) { .n = 0 };
- struct string body = DS_EMPTY_INITIALIZER;
- struct msg_point start = lex_ofs_start_point (lexer, lex_ofs (lexer));
- while (!match_macro_id (lexer, "!ENDDEFINE"))
- {
- if (lex_token (lexer) != T_STRING)
- {
- lex_error (lexer,
- _("Syntax error expecting macro body or !ENDDEFINE."));
- ds_destroy (&body);
- return false;
- }
-
- ds_put_substring (&body, lex_tokss (lexer));
- ds_put_byte (&body, '\n');
- lex_get (lexer);
- }
-
- struct segmenter segmenter = segmenter_init (lex_get_syntax_mode (lexer),
- true);
- struct substring p = body.ss;
- bool ok = true;
- while (p.length > 0)
- {
- enum segment_type type;
- int seg_len = segmenter_push (&segmenter, p.string,
- p.length, true, &type);
- assert (seg_len >= 0);
-
- struct macro_token mt = {
- .token = { .type = T_STOP },
- .syntax = ss_head (p, seg_len),
- };
- enum tokenize_result result
- = token_from_segment (type, mt.syntax, &mt.token);
- ss_advance (&p, seg_len);
-
- switch (result)
- {
- case TOKENIZE_EMPTY:
- break;
-
- case TOKENIZE_TOKEN:
- macro_tokens_add (mts, &mt);
- break;
-
- case TOKENIZE_ERROR:
- {
- size_t start_offset = mt.syntax.string - body.ss.string;
- size_t end_offset = start_offset + (mt.syntax.length ? mt.syntax.length - 1 : 0);
-
- const struct msg_location loc = {
- .file_name = intern_new_if_nonnull (lex_get_file_name (lexer)),
- .start = msg_point_advance (start, ss_buffer (body.ss.string, start_offset)),
- .end = msg_point_advance (start, ss_buffer (body.ss.string, end_offset)),
- .src = CONST_CAST (struct lex_source *, lex_source (lexer)),
- };
- msg_at (SE, &loc, "%s", mt.token.string.string);
- intern_unref (loc.file_name);
-
- ok = false;
- }
- break;
- }
-
- token_uninit (&mt.token);
- }
- ds_destroy (&body);
- return ok;
-}
-
-int
-cmd_define (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- /* Parse macro name.
-
- The macro name is a T_STRING token, even though it's an identifier,
- because that's the way that the segmenter prevents it from getting
- macro-expanded. */
- if (lex_token (lexer) != T_STRING)
- {
- lex_error (lexer, _("Syntax error expecting identifier."));
- return CMD_FAILURE;
- }
- const char *name = lex_tokcstr (lexer);
- if (!id_is_plausible (name + (name[0] == '!')))
- {
- lex_error (lexer, _("Syntax error expecting identifier."));
- return CMD_FAILURE;
- }
-
- struct macro *m = xmalloc (sizeof *m);
- *m = (struct macro) { .name = xstrdup (name) };
- struct msg_point macro_start = lex_ofs_start_point (lexer, lex_ofs (lexer));
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- size_t allocated_params = 0;
- int keyword_ofs = 0;
- while (!lex_match (lexer, T_RPAREN))
- {
- if (m->n_params >= allocated_params)
- m->params = x2nrealloc (m->params, &allocated_params,
- sizeof *m->params);
-
- size_t param_index = m->n_params++;
- struct macro_param *p = &m->params[param_index];
- *p = (struct macro_param) { .expand_arg = true };
-
- /* Parse parameter name. */
- if (match_macro_id (lexer, "!POSITIONAL"))
- {
- if (param_index > 0 && !m->params[param_index - 1].positional)
- {
- lex_next_error (lexer, -1, -1,
- _("Positional parameters must precede "
- "keyword parameters."));
- lex_ofs_msg (lexer, SN, keyword_ofs, keyword_ofs,
- _("Here is a previous keyword parameter."));
- goto error;
- }
-
- p->positional = true;
- p->name = xasprintf ("!%zu", param_index + 1);
- }
- else
- {
- if (keyword_ofs == 0)
- keyword_ofs = lex_ofs (lexer);
- if (lex_token (lexer) == T_MACRO_ID)
- {
- lex_error (lexer, _("Keyword macro parameter must be named in "
- "definition without \"!\" prefix."));
- goto error;
- }
- if (!lex_force_id (lexer))
- goto error;
-
- if (is_macro_keyword (lex_tokss (lexer)))
- {
- lex_error (lexer, _("Cannot use macro keyword \"%s\" "
- "as an argument name."),
- lex_tokcstr (lexer));
- goto error;
- }
-
- p->positional = false;
- p->name = xasprintf ("!%s", lex_tokcstr (lexer));
- lex_get (lexer);
- }
- lex_match (lexer, T_EQUALS);
-
- bool saw_default = false;
- bool saw_arg_type = false;
- for (;;)
- {
- if (match_macro_id (lexer, "!DEFAULT"))
- {
- if (saw_default)
- {
- lex_next_error (
- lexer, -1, -1,
- _("!DEFAULT is allowed only once per argument."));
- goto error;
- }
- saw_default = true;
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- /* XXX Should this handle balanced inner parentheses? */
- while (!lex_match (lexer, T_RPAREN))
- {
- if (lex_token (lexer) == T_ENDCMD)
- {
- lex_error_expecting (lexer, ")");
- goto error;
- }
- char *syntax = lex_next_representation (lexer, 0, 0);
- const struct macro_token mt = {
- .token = *lex_next (lexer, 0),
- .syntax = ss_cstr (syntax),
- };
- macro_tokens_add (&p->def, &mt);
- free (syntax);
-
- lex_get (lexer);
- }
- }
- else if (match_macro_id (lexer, "!NOEXPAND"))
- p->expand_arg = false;
- else if (match_macro_id (lexer, "!TOKENS"))
- {
- if (!dup_arg_type (lexer, &saw_arg_type)
- || !lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "!TOKENS", 1, INT_MAX))
- goto error;
- p->arg_type = ARG_N_TOKENS;
- p->n_tokens = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (match_macro_id (lexer, "!CHAREND"))
- {
- if (!dup_arg_type (lexer, &saw_arg_type))
- goto error;
-
- p->arg_type = ARG_CHAREND;
-
- if (!lex_force_match (lexer, T_LPAREN)
- || !parse_quoted_token (lexer, &p->end)
- || !lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (match_macro_id (lexer, "!ENCLOSE"))
- {
- if (!dup_arg_type (lexer, &saw_arg_type))
- goto error;
-
- p->arg_type = ARG_ENCLOSE;
-
- if (!lex_force_match (lexer, T_LPAREN)
- || !parse_quoted_token (lexer, &p->start)
- || !lex_force_match (lexer, T_COMMA)
- || !parse_quoted_token (lexer, &p->end)
- || !lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (match_macro_id (lexer, "!CMDEND"))
- {
- if (!dup_arg_type (lexer, &saw_arg_type))
- goto error;
-
- p->arg_type = ARG_CMDEND;
- }
- else
- break;
- }
- if (!saw_arg_type)
- {
- lex_error_expecting (lexer, "!TOKENS", "!CHAREND", "!ENCLOSE",
- "!CMDEND");
- goto error;
- }
-
- if (lex_token (lexer) != T_RPAREN && !lex_force_match (lexer, T_SLASH))
- goto error;
- }
-
- if (!parse_macro_body (lexer, &m->body))
- goto error;
-
- struct msg_point macro_end = lex_ofs_end_point (lexer, lex_ofs (lexer) - 1);
- m->location = xmalloc (sizeof *m->location);
- *m->location = (struct msg_location) {
- .file_name = intern_new_if_nonnull (lex_get_file_name (lexer)),
- .start = { .line = macro_start.line },
- .end = { .line = macro_end.line },
- };
-
- lex_define_macro (lexer, m);
-
- return CMD_SUCCESS;
-
-error:
- macro_destroy (m);
- return CMD_FAILURE;
-}
-
-int
-cmd_debug_expand (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- settings_set_mprint (true);
-
- while (lex_token (lexer) != T_STOP)
- {
- if (!lex_next_is_from_macro (lexer, 0) && lex_token (lexer) != T_ENDCMD)
- {
- char *rep = lex_next_representation (lexer, 0, 0);
- msg (MN, "unexpanded token \"%s\"", rep);
- free (rep);
- }
- lex_get (lexer);
- }
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009-2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/transformations.h"
-#include "language/command.h"
-#include "language/data-io/inpt-pgm.h"
-#include "language/expressions/public.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/compiler.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* A conditional clause. */
-struct clause
- {
- struct msg_location *location;
- struct expression *condition; /* Test expression; NULL for ELSE clause. */
- struct trns_chain xforms;
- };
-
-/* DO IF transformation. */
-struct do_if_trns
- {
- struct clause *clauses; /* Clauses. */
- size_t n_clauses; /* Number of clauses. */
-
- const struct trns_chain *resume;
- size_t ofs;
- };
-
-static const struct trns_class do_if_trns_class;
-
-static void
-start_clause (struct lexer *lexer, struct dataset *ds,
- bool condition, struct do_if_trns *do_if,
- size_t *allocated_clauses, bool *ok)
-{
- if (*ok && do_if->n_clauses > 0
- && !do_if->clauses[do_if->n_clauses - 1].condition)
- {
- if (condition)
- lex_ofs_error (lexer, 0, 1,
- _("ELSE IF is not allowed following ELSE "
- "within DO IF...END IF."));
- else
- lex_ofs_error (lexer, 0, 0,
- _("Only one ELSE is allowed within DO IF...END IF."));
-
- msg_at (SN, do_if->clauses[do_if->n_clauses - 1].location,
- _("This is the location of the previous ELSE clause."));
-
- msg_at (SN, do_if->clauses[0].location,
- _("This is the location of the DO IF command."));
-
- *ok = false;
- }
-
- if (do_if->n_clauses >= *allocated_clauses)
- do_if->clauses = x2nrealloc (do_if->clauses, allocated_clauses,
- sizeof *do_if->clauses);
- struct clause *clause = &do_if->clauses[do_if->n_clauses++];
-
- *clause = (struct clause) { .location = NULL };
- if (condition)
- {
- clause->condition = expr_parse_bool (lexer, ds);
- if (!clause->condition)
- lex_discard_rest_of_command (lexer);
- }
- clause->location = lex_ofs_location (lexer, 0, lex_ofs (lexer));
-
- lex_end_of_command (lexer);
- lex_get (lexer);
-
- proc_push_transformations (ds);
-}
-
-static void
-finish_clause (struct dataset *ds, struct do_if_trns *do_if)
-{
- struct clause *clause = &do_if->clauses[do_if->n_clauses - 1];
- proc_pop_transformations (ds, &clause->xforms);
-}
-
-/* Parse DO IF. */
-int
-cmd_do_if (struct lexer *lexer, struct dataset *ds)
-{
- struct do_if_trns *do_if = xmalloc (sizeof *do_if);
- *do_if = (struct do_if_trns) { .n_clauses = 0 };
-
- size_t allocated_clauses = 0;
- bool ok = true;
-
- start_clause (lexer, ds, true, do_if, &allocated_clauses, &ok);
- while (!lex_match_phrase (lexer, "END IF"))
- {
- if (lex_token (lexer) == T_STOP)
- {
- lex_error_expecting (lexer, "END IF");
- break;
- }
- else if (lex_match_phrase (lexer, "ELSE IF"))
- {
- finish_clause (ds, do_if);
- start_clause (lexer, ds, true, do_if, &allocated_clauses, &ok);
- }
- else if (lex_match_id (lexer, "ELSE"))
- {
- finish_clause (ds, do_if);
- start_clause (lexer, ds, false, do_if, &allocated_clauses, &ok);
- }
- else
- cmd_parse_in_state (lexer, ds,
- (in_input_program ()
- ? CMD_STATE_NESTED_INPUT_PROGRAM
- : CMD_STATE_NESTED_DATA));
- }
- finish_clause (ds, do_if);
-
- add_transformation (ds, &do_if_trns_class, do_if);
-
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-}
-
-int
-cmd_inside_do_if (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("This command cannot appear outside DO IF...END IF."));
- return CMD_FAILURE;
-}
-
-static const struct trns_chain *
-do_if_find_clause (const struct do_if_trns *do_if,
- struct ccase *c, casenumber case_num)
-{
- for (size_t i = 0; i < do_if->n_clauses; i++)
- {
- const struct clause *clause = &do_if->clauses[i];
- if (!clause->condition)
- return &clause->xforms;
-
- double boolean = expr_evaluate_num (clause->condition, c, case_num);
- if (boolean != 0.0)
- return boolean == SYSMIS ? NULL : &clause->xforms;
- }
- return NULL;
-}
-
-static enum trns_result
-do_if_trns_proc (void *do_if_, struct ccase **c, casenumber case_num)
-{
- struct do_if_trns *do_if = do_if_;
-
- const struct trns_chain *chain;
- size_t start;
- if (do_if->resume)
- {
- chain = do_if->resume;
- start = do_if->ofs;
- do_if->resume = NULL;
- do_if->ofs = 0;
- }
- else
- {
- chain = do_if_find_clause (do_if, *c, case_num);
- if (!chain)
- return TRNS_CONTINUE;
- start = 0;
- }
-
- for (size_t i = start; i < chain->n; i++)
- {
- const struct transformation *trns = &chain->xforms[i];
- enum trns_result r = trns->class->execute (trns->aux, c, case_num);
- switch (r)
- {
- case TRNS_CONTINUE:
- break;
-
- case TRNS_BREAK:
- case TRNS_DROP_CASE:
- case TRNS_ERROR:
- case TRNS_END_FILE:
- return r;
-
- case TRNS_END_CASE:
- do_if->resume = chain;
- do_if->ofs = i;
- return r;
- }
- }
- return TRNS_CONTINUE;
-}
-
-static bool
-do_if_trns_free (void *do_if_)
-{
- struct do_if_trns *do_if = do_if_;
-
- for (size_t i = 0; i < do_if->n_clauses; i++)
- {
- struct clause *clause = &do_if->clauses[i];
-
- msg_location_destroy (clause->location);
- expr_free (clause->condition);
-
- trns_chain_uninit (&clause->xforms);
- }
- free (do_if->clauses);
- free (do_if);
- return true;
-}
-
-static const struct trns_class do_if_trns_class = {
- .name = "DO IF",
- .execute = do_if_trns_proc,
- .destroy = do_if_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009-2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <limits.h>
-
-#include "data/case.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/settings.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/inpt-pgm.h"
-#include "language/expressions/public.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-struct loop_trns
- {
- /* a=a TO b [BY c]. */
- struct variable *index_var; /* Index variable. */
- struct expression *first_expr; /* Starting index. */
- struct expression *by_expr; /* Index increment (or NULL). */
- struct expression *last_expr; /* Terminal index. */
-
- /* IF condition for LOOP or END LOOP. */
- struct expression *loop_condition;
- struct expression *end_loop_condition;
-
- /* Inner transformations. */
- struct trns_chain xforms;
-
- /* State. */
- double cur, by, last; /* Index data. */
- int iteration; /* For MXLOOPS. */
- size_t resume_idx; /* For resuming after END CASE. */
- };
-
-static struct trns_class loop_trns_class;
-
-static int in_loop;
-
-static bool parse_if_clause (struct lexer *, struct dataset *,
- struct expression **);
-static bool parse_index_clause (struct dataset *, struct lexer *,
- struct loop_trns *);
-\f
-/* LOOP. */
-
-/* Parses LOOP. */
-int
-cmd_loop (struct lexer *lexer, struct dataset *ds)
-{
- struct loop_trns *loop = xmalloc (sizeof *loop);
- *loop = (struct loop_trns) { .resume_idx = SIZE_MAX };
-
- bool ok = true;
- while (lex_token (lexer) != T_ENDCMD && ok)
- {
- if (lex_match_id (lexer, "IF"))
- ok = parse_if_clause (lexer, ds, &loop->loop_condition);
- else
- ok = parse_index_clause (ds, lexer, loop);
- }
- if (ok)
- lex_end_of_command (lexer);
- lex_discard_rest_of_command (lexer);
-
- proc_push_transformations (ds);
- in_loop++;
- for (;;)
- {
- if (lex_token (lexer) == T_STOP)
- {
- lex_error_expecting (lexer, "END LOOP");
- ok = false;
- break;
- }
- else if (lex_match_phrase (lexer, "END LOOP"))
- {
- if (lex_match_id (lexer, "IF"))
- ok = parse_if_clause (lexer, ds, &loop->end_loop_condition) && ok;
- break;
- }
- else
- cmd_parse_in_state (lexer, ds,
- (in_input_program ()
- ? CMD_STATE_NESTED_INPUT_PROGRAM
- : CMD_STATE_NESTED_DATA));
- }
- in_loop--;
- proc_pop_transformations (ds, &loop->xforms);
-
- add_transformation (ds, &loop_trns_class, loop);
-
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-}
-
-int
-cmd_inside_loop (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("This command cannot appear outside LOOP...END LOOP."));
- return CMD_FAILURE;
-}
-
-static enum trns_result
-break_trns_proc (void *aux UNUSED, struct ccase **c UNUSED,
- casenumber case_num UNUSED)
-{
- return TRNS_BREAK;
-}
-
-/* Parses BREAK. */
-int
-cmd_break (struct lexer *lexer, struct dataset *ds)
-{
- if (!in_loop)
- {
- cmd_inside_loop (lexer, ds);
- return CMD_FAILURE;
- }
-
- static const struct trns_class trns_class = {
- .name = "BREAK",
- .execute = break_trns_proc
- };
- add_transformation (ds, &trns_class, NULL);
-
- return CMD_SUCCESS;
-}
-
-/* Parses an IF clause for LOOP or END LOOP and stores the
- resulting expression to *CONDITION.
- Returns true if successful, false on failure. */
-static bool
-parse_if_clause (struct lexer *lexer, struct dataset *ds,
- struct expression **condition)
-{
- if (*condition != NULL)
- {
- lex_sbc_only_once (lexer, "IF");
- return false;
- }
-
- *condition = expr_parse_bool (lexer, ds);
- return *condition != NULL;
-}
-
-/* Parses an indexing clause into LOOP. Returns true if successful, false on
- failure. */
-static bool
-parse_index_clause (struct dataset *ds, struct lexer *lexer,
- struct loop_trns *loop)
-{
- if (loop->index_var != NULL)
- {
- lex_error (lexer, _("Only one index clause may be specified."));
- return false;
- }
-
- if (!lex_force_id (lexer))
- return false;
-
- loop->index_var = dict_lookup_var (dataset_dict (ds), lex_tokcstr (lexer));
- if (!loop->index_var)
- loop->index_var = dict_create_var_assert (dataset_dict (ds),
- lex_tokcstr (lexer), 0);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_EQUALS))
- return false;
-
- loop->first_expr = expr_parse (lexer, ds, VAL_NUMERIC);
- if (loop->first_expr == NULL)
- return false;
-
- for (;;)
- {
- struct expression **e;
- if (lex_match (lexer, T_TO))
- e = &loop->last_expr;
- else if (lex_match (lexer, T_BY))
- e = &loop->by_expr;
- else
- break;
-
- if (*e != NULL)
- {
- lex_sbc_only_once (lexer, e == &loop->last_expr ? "TO" : "BY");
- return false;
- }
- *e = expr_parse (lexer, ds, VAL_NUMERIC);
- if (*e == NULL)
- return false;
- }
- if (loop->last_expr == NULL)
- {
- lex_sbc_missing (lexer, "TO");
- return false;
- }
-
- return true;
-}
-
-/* Sets up LOOP for the first pass. */
-static enum trns_result
-loop_trns_proc (void *loop_, struct ccase **c, casenumber case_num)
-{
- struct loop_trns *loop = loop_;
-
- size_t start_idx = loop->resume_idx;
- loop->resume_idx = SIZE_MAX;
- if (start_idx != SIZE_MAX)
- goto resume;
-
- if (loop->index_var)
- {
- /* Evaluate loop index expressions. */
- loop->cur = expr_evaluate_num (loop->first_expr, *c, case_num);
- loop->by = (loop->by_expr
- ? expr_evaluate_num (loop->by_expr, *c, case_num)
- : 1.0);
- loop->last = expr_evaluate_num (loop->last_expr, *c, case_num);
-
- /* Even if the loop is never entered, set the index
- variable to the initial value. */
- *c = case_unshare (*c);
- *case_num_rw (*c, loop->index_var) = loop->cur;
-
- /* Throw out pathological cases. */
- if (!isfinite (loop->cur)
- || !isfinite (loop->by)
- || !isfinite (loop->last)
- || loop->by == 0.0
- || (loop->by > 0.0 && loop->cur > loop->last)
- || (loop->by < 0.0 && loop->cur < loop->last))
- return TRNS_CONTINUE;
- }
-
- for (loop->iteration = 0;
- loop->index_var || loop->iteration < settings_get_mxloops ();
- loop->iteration++)
- {
- if (loop->loop_condition
- && expr_evaluate_num (loop->loop_condition, *c, case_num) != 1.0)
- break;
-
- start_idx = 0;
- resume:
- for (size_t i = start_idx; i < loop->xforms.n; i++)
- {
- const struct transformation *trns = &loop->xforms.xforms[i];
- enum trns_result r = trns->class->execute (trns->aux, c, case_num);
- switch (r)
- {
- case TRNS_CONTINUE:
- break;
-
- case TRNS_BREAK:
- return TRNS_CONTINUE;
-
- case TRNS_END_CASE:
- loop->resume_idx = i;
- return TRNS_END_CASE;
-
- case TRNS_ERROR:
- case TRNS_END_FILE:
- return r;
-
- case TRNS_DROP_CASE:
- NOT_REACHED ();
- }
- }
-
- if (loop->end_loop_condition != NULL
- && expr_evaluate_num (loop->end_loop_condition, *c, case_num) != 0.0)
- break;
-
- if (loop->index_var)
- {
- loop->cur += loop->by;
- if (loop->by > 0.0 ? loop->cur > loop->last : loop->cur < loop->last)
- break;
-
- *c = case_unshare (*c);
- *case_num_rw (*c, loop->index_var) = loop->cur;
- }
- }
- return TRNS_CONTINUE;
-}
-
-/* Frees LOOP. */
-static bool
-loop_trns_free (void *loop_)
-{
- struct loop_trns *loop = loop_;
-
- expr_free (loop->first_expr);
- expr_free (loop->by_expr);
- expr_free (loop->last_expr);
-
- expr_free (loop->loop_condition);
- expr_free (loop->end_loop_condition);
-
- trns_chain_uninit (&loop->xforms);
-
- free (loop);
- return true;
-}
-
-static struct trns_class loop_trns_class = {
- .name = "LOOP",
- .execute = loop_trns_proc,
- .destroy = loop_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2007, 2009-2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/settings.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/segment.h"
-#include "language/lexer/token.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmap.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "libpspp/misc.h"
-#include "output/output-item.h"
-
-#include "gl/ftoastr.h"
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-#include "gl/xmemdup0.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-struct dummy_var
- {
- struct hmap_node hmap_node;
- struct substring name;
- char **values;
- size_t n_values;
- int start_ofs, end_ofs;
- };
-
-static bool parse_specification (struct lexer *, struct dictionary *,
- struct hmap *dummies);
-static bool parse_commands (struct lexer *, struct hmap *dummies);
-static void destroy_dummies (struct hmap *dummies);
-
-static bool parse_ids (struct lexer *, const struct dictionary *,
- struct dummy_var *);
-static bool parse_numbers (struct lexer *, struct dummy_var *);
-static bool parse_strings (struct lexer *, struct dummy_var *);
-
-int
-cmd_do_repeat (struct lexer *lexer, struct dataset *ds)
-{
- struct hmap dummies = HMAP_INITIALIZER (dummies);
- bool ok = parse_specification (lexer, dataset_dict (ds), &dummies);
- ok = parse_commands (lexer, &dummies) && ok;
- destroy_dummies (&dummies);
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-}
-
-static const struct dummy_var *
-find_dummy_var (struct hmap *hmap, struct substring name)
-{
- const struct dummy_var *dv;
-
- HMAP_FOR_EACH_WITH_HASH (dv, struct dummy_var, hmap_node,
- utf8_hash_case_substring (name, 0), hmap)
- if (!utf8_sscasecmp (dv->name, name))
- return dv;
-
- return NULL;
-}
-
-/* Parses the whole DO REPEAT command specification.
- Returns success. */
-static bool
-parse_specification (struct lexer *lexer, struct dictionary *dict,
- struct hmap *dummies)
-{
- struct dummy_var *first_dv = NULL;
-
- do
- {
- int start_ofs = lex_ofs (lexer);
-
- /* Get a stand-in variable name and make sure it's unique. */
- if (!lex_force_id (lexer))
- goto error;
- struct substring name = lex_tokss (lexer);
- if (dict_lookup_var (dict, name.string))
- lex_msg (lexer, SW,
- _("Dummy variable name `%s' hides dictionary variable `%s'."),
- name.string, name.string);
- if (find_dummy_var (dummies, name))
- {
- lex_error (lexer, _("Dummy variable name `%s' is given twice."),
- name.string);
- goto error;
- }
-
- /* Make a new macro. */
- struct dummy_var *dv = xmalloc (sizeof *dv);
- *dv = (struct dummy_var) {
- .name = ss_clone (name),
- .start_ofs = start_ofs,
- };
- hmap_insert (dummies, &dv->hmap_node, utf8_hash_case_substring (name, 0));
-
- /* Skip equals sign. */
- lex_get (lexer);
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- /* Get the details of the variable's possible values. */
- bool ok;
- if (lex_token (lexer) == T_ID || lex_token (lexer) == T_ALL)
- ok = parse_ids (lexer, dict, dv);
- else if (lex_is_number (lexer))
- ok = parse_numbers (lexer, dv);
- else if (lex_is_string (lexer))
- ok = parse_strings (lexer, dv);
- else
- {
- lex_error (lexer, _("Syntax error expecting substitution values."));
- goto error;
- }
- if (!ok)
- goto error;
- assert (dv->n_values > 0);
- if (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- lex_error (lexer, _("Syntax error expecting `/' or end of command."));
- goto error;
- }
- dv->end_ofs = lex_ofs (lexer) - 1;
-
- /* If this is the first variable then it defines how many replacements
- there must be; otherwise enforce this number of replacements. */
- if (first_dv == NULL)
- first_dv = dv;
- else if (first_dv->n_values != dv->n_values)
- {
- msg (SE, _("Each dummy variable must have the same number of "
- "substitutions."));
-
- lex_ofs_msg (lexer, SN, first_dv->start_ofs, first_dv->end_ofs,
- ngettext ("Dummy variable %s had %zu substitution.",
- "Dummy variable %s had %zu substitutions.",
- first_dv->n_values),
- first_dv->name.string, first_dv->n_values);
- lex_ofs_msg (lexer, SN, dv->start_ofs, dv->end_ofs,
- ngettext ("Dummy variable %s had %zu substitution.",
- "Dummy variable %s had %zu substitutions.",
- dv->n_values),
- dv->name.string, dv->n_values);
- goto error;
- }
-
- lex_match (lexer, T_SLASH);
- }
- while (!lex_match (lexer, T_ENDCMD));
-
- while (lex_match (lexer, T_ENDCMD))
- continue;
-
- return true;
-
-error:
- lex_discard_rest_of_command (lexer);
- while (lex_match (lexer, T_ENDCMD))
- continue;
- destroy_dummies (dummies);
- hmap_init (dummies);
- return false;
-}
-
-static size_t
-count_values (struct hmap *dummies)
-{
- if (hmap_is_empty (dummies))
- return 0;
- const struct dummy_var *dv = HMAP_FIRST (struct dummy_var, hmap_node, dummies);
- return dv->n_values;
-}
-
-static void
-do_parse_commands (struct substring s, enum segmenter_mode mode,
- struct hmap *dummies,
- struct string *outputs, size_t n_outputs)
-{
- struct segmenter segmenter = segmenter_init (mode, false);
- while (!ss_is_empty (s))
- {
- enum segment_type type;
- int n = segmenter_push (&segmenter, s.string, s.length, true, &type);
- assert (n >= 0);
-
- if (type == SEG_DO_REPEAT_COMMAND)
- {
- for (;;)
- {
- int k = segmenter_push (&segmenter, s.string + n, s.length - n,
- true, &type);
- if (type != SEG_NEWLINE && type != SEG_DO_REPEAT_COMMAND)
- break;
-
- n += k;
- }
-
- do_parse_commands (ss_head (s, n), mode, dummies,
- outputs, n_outputs);
- }
- else if (type != SEG_END)
- {
- const struct dummy_var *dv
- = (type == SEG_IDENTIFIER ? find_dummy_var (dummies, ss_head (s, n))
- : NULL);
- for (size_t i = 0; i < n_outputs; i++)
- if (dv != NULL)
- ds_put_cstr (&outputs[i], dv->values[i]);
- else
- ds_put_substring (&outputs[i], ss_head (s, n));
- }
-
- ss_advance (&s, n);
- }
-}
-
-static bool
-parse_commands (struct lexer *lexer, struct hmap *dummies)
-{
- char *file_name = xstrdup_if_nonnull (lex_get_file_name (lexer));
- int line_number = lex_ofs_start_point (lexer, lex_ofs (lexer)).line;
-
- struct string input = DS_EMPTY_INITIALIZER;
- while (lex_is_string (lexer))
- {
- ds_put_substring (&input, lex_tokss (lexer));
- ds_put_byte (&input, '\n');
- lex_get (lexer);
- }
-
- size_t n_values = count_values (dummies);
- struct string *outputs = xmalloc (n_values * sizeof *outputs);
- for (size_t i = 0; i < n_values; i++)
- ds_init_empty (&outputs[i]);
-
- do_parse_commands (ds_ss (&input), lex_get_syntax_mode (lexer),
- dummies, outputs, n_values);
-
- ds_destroy (&input);
-
- while (lex_match (lexer, T_ENDCMD))
- continue;
-
- bool ok = lex_match_phrase (lexer, "END REPEAT");
- if (!ok)
- lex_error (lexer, _("Syntax error expecting END REPEAT."));
- bool print = ok && lex_match_id (lexer, "PRINT");
- lex_discard_rest_of_command (lexer);
-
- for (size_t i = 0; i < n_values; i++)
- {
- struct string *output = &outputs[n_values - i - 1];
- if (print)
- {
- struct substring s = output->ss;
- ss_chomp_byte (&s, '\n');
- char *label = xasprintf (_("Expansion %zu of %zu"), i + 1, n_values);
- output_item_submit (
- text_item_create_nocopy (TEXT_ITEM_LOG, ss_xstrdup (s), label));
- }
- const char *encoding = lex_get_encoding (lexer);
- struct lex_reader *reader = lex_reader_for_substring_nocopy (ds_ss (output), encoding);
- lex_reader_set_file_name (reader, file_name);
- reader->line_number = line_number;
- lex_include (lexer, reader);
- }
- free (file_name);
- free (outputs);
-
- return ok;
-}
-
-static void
-destroy_dummies (struct hmap *dummies)
-{
- struct dummy_var *dv, *next;
-
- HMAP_FOR_EACH_SAFE (dv, next, struct dummy_var, hmap_node, dummies)
- {
- hmap_delete (dummies, &dv->hmap_node);
-
- ss_dealloc (&dv->name);
- for (size_t i = 0; i < dv->n_values; i++)
- free (dv->values[i]);
- free (dv->values);
- free (dv);
- }
- hmap_destroy (dummies);
-}
-
-/* Parses a set of ids for DO REPEAT. */
-static bool
-parse_ids (struct lexer *lexer, const struct dictionary *dict,
- struct dummy_var *dv)
-{
- return parse_mixed_vars (lexer, dict, &dv->values, &dv->n_values, PV_NONE);
-}
-
-/* Adds REPLACEMENT to MACRO's list of replacements, which has
- *USED elements and has room for *ALLOCATED. Allocates memory
- from POOL. */
-static void
-add_replacement (struct dummy_var *dv, char *value, size_t *allocated)
-{
- if (dv->n_values == *allocated)
- dv->values = x2nrealloc (dv->values, allocated, sizeof *dv->values);
- dv->values[dv->n_values++] = value;
-}
-
-/* Parses a list or range of numbers for DO REPEAT. */
-static bool
-parse_numbers (struct lexer *lexer, struct dummy_var *dv)
-{
- size_t allocated = 0;
-
- do
- {
- if (!lex_force_num (lexer))
- return false;
-
- if (lex_next_token (lexer, 1) == T_TO)
- {
- if (!lex_is_integer (lexer))
- {
- lex_error (lexer, _("Ranges may only have integer bounds."));
- return false;
- }
-
- long a = lex_integer (lexer);
- lex_get (lexer);
- lex_get (lexer);
-
- if (!lex_force_int_range (lexer, NULL, a, LONG_MAX))
- return false;
-
- long b = lex_integer (lexer);
- if (b < a)
- {
- lex_next_error (lexer, -2, 0,
- _("%ld TO %ld is an invalid range."), a, b);
- return false;
- }
- lex_get (lexer);
-
- for (long i = a; i <= b; i++)
- add_replacement (dv, xasprintf ("%ld", i), &allocated);
- }
- else
- {
- char s[DBL_BUFSIZE_BOUND];
-
- c_dtoastr (s, sizeof s, 0, 0, lex_number (lexer));
- add_replacement (dv, xstrdup (s), &allocated);
- lex_get (lexer);
- }
-
- lex_match (lexer, T_COMMA);
- }
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
-
- return true;
-}
-
-/* Parses a list of strings for DO REPEAT. */
-static bool
-parse_strings (struct lexer *lexer, struct dummy_var *dv)
-{
- size_t allocated = 0;
-
- do
- {
- if (!lex_force_string (lexer))
- return false;
-
- add_replacement (dv, token_to_string (lex_next (lexer, 0)), &allocated);
-
- lex_get (lexer);
- lex_match (lexer, T_COMMA);
- }
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
-
- return true;
-}
-\f
-int
-cmd_end_repeat (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- lex_ofs_error (lexer, 0, 1, _("No matching %s."), "DO REPEAT");
- return CMD_CASCADING_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/transformations.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Parses the TEMPORARY command. */
-int
-cmd_temporary (struct lexer *lexer, struct dataset *ds)
-{
- if (!proc_in_temporary_transformations (ds))
- proc_start_temporary_transformations (ds);
- else
- lex_ofs_error (lexer, 0, 0,
- _("This command may only appear once between "
- "procedures and procedure-like commands."));
- return CMD_SUCCESS;
-}
+++ /dev/null
-# PSPP - a program for statistical analysis.
-# Copyright (C) 2017 Free Software Foundation, Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-## Process this file with automake to produce Makefile.in -*- makefile -*-
-
-language_data_io_sources = \
- src/language/data-io/combine-files.c \
- src/language/data-io/data-list.c \
- src/language/data-io/data-parser.c \
- src/language/data-io/data-parser.h \
- src/language/data-io/data-reader.c \
- src/language/data-io/data-reader.h \
- src/language/data-io/data-writer.c \
- src/language/data-io/data-writer.h \
- src/language/data-io/dataset.c \
- src/language/data-io/file-handle.c \
- src/language/data-io/file-handle.h \
- src/language/data-io/get-data.c \
- src/language/data-io/get.c \
- src/language/data-io/inpt-pgm.c \
- src/language/data-io/inpt-pgm.h \
- src/language/data-io/list.c \
- src/language/data-io/placement-parser.c \
- src/language/data-io/placement-parser.h \
- src/language/data-io/print-space.c \
- src/language/data-io/print.c \
- src/language/data-io/matrix-data.c \
- src/language/data-io/matrix-reader.c \
- src/language/data-io/matrix-reader.h \
- src/language/data-io/mconvert.c \
- src/language/data-io/save-translate.c \
- src/language/data-io/save.c \
- src/language/data-io/trim.c \
- src/language/data-io/trim.h
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/any-reader.h"
-#include "data/case-matcher.h"
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/trim.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "language/stats/sort-criteria.h"
-#include "libpspp/assertion.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/string-array.h"
-#include "libpspp/taint.h"
-#include "math/sort.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-enum comb_command_type
- {
- COMB_ADD,
- COMB_MATCH,
- COMB_UPDATE
- };
-
-/* File types. */
-enum comb_file_type
- {
- COMB_FILE, /* Specified on FILE= subcommand. */
- COMB_TABLE /* Specified on TABLE= subcommand. */
- };
-
-/* One FILE or TABLE subcommand. */
-struct comb_file
- {
- /* Basics. */
- enum comb_file_type type; /* COMB_FILE or COMB_TABLE. */
- int start_ofs, end_ofs; /* Lexer offsets. */
-
- /* Variables. */
- struct subcase by_vars; /* BY variables in this input file. */
- struct subcase src, dst; /* Data to copy to output; where to put it. */
- const struct missing_values **mv; /* Each variable's missing values. */
-
- /* Input files. */
- struct file_handle *handle; /* Input file handle. */
- struct dictionary *dict; /* Input file dictionary. */
- struct casereader *reader; /* Input data source. */
- struct ccase *data; /* The current input case. */
- bool is_minimal; /* Does 'data' have minimum BY values across
- all input files? */
- bool is_sorted; /* Is file presorted on the BY variables? */
-
- /* IN subcommand. */
- char *in_name;
- int in_ofs;
- struct variable *in_var;
- };
-
-struct comb_proc
- {
- struct comb_file *files; /* All the files being merged. */
- size_t n_files; /* Number of files. */
-
- struct dictionary *dict; /* Dictionary of output file. */
- struct subcase by_vars; /* BY variables in the output. */
- struct casewriter *output; /* Destination for output. */
-
- size_t *var_sources;
- size_t n_var_sources, allocated_var_sources;
-
- struct case_matcher *matcher;
-
- /* FIRST, LAST.
- Only if "first" or "last" is nonnull are the remaining
- members used. */
- struct variable *first; /* Variable specified on FIRST (if any). */
- struct variable *last; /* Variable specified on LAST (if any). */
- struct ccase *buffered_case; /* Case ready for output except that we don't
- know the value for the LAST var yet. */
- union value *prev_BY; /* Values of BY vars in buffered_case. */
- };
-
-static int combine_files (enum comb_command_type, struct lexer *,
- struct dataset *);
-static void free_comb_proc (struct comb_proc *);
-
-static void close_all_comb_files (struct comb_proc *);
-static bool merge_dictionary (struct comb_proc *, struct lexer *,
- struct comb_file *);
-
-static void execute_update (struct comb_proc *);
-static void execute_match_files (struct comb_proc *);
-static void execute_add_files (struct comb_proc *);
-
-static bool create_flag_var (struct lexer *lexer, const char *subcommand_name,
- const char *var_name, int var_ofs,
- struct dictionary *, struct variable **);
-static void output_case (struct comb_proc *, struct ccase *, union value *by);
-static void output_buffered_case (struct comb_proc *);
-
-int
-cmd_add_files (struct lexer *lexer, struct dataset *ds)
-{
- return combine_files (COMB_ADD, lexer, ds);
-}
-
-int
-cmd_match_files (struct lexer *lexer, struct dataset *ds)
-{
- return combine_files (COMB_MATCH, lexer, ds);
-}
-
-int
-cmd_update (struct lexer *lexer, struct dataset *ds)
-{
- return combine_files (COMB_UPDATE, lexer, ds);
-}
-
-static int
-combine_files (enum comb_command_type command,
- struct lexer *lexer, struct dataset *ds)
-{
- struct comb_proc proc = {
- .dict = dict_create (get_default_encoding ()),
- };
-
- bool saw_by = false;
- bool saw_sort = false;
- struct casereader *active_file = NULL;
-
- char *first_name = NULL;
- int first_ofs = 0;
- char *last_name = NULL;
- int last_ofs = 0;
-
- struct taint *taint = NULL;
-
- size_t table_idx = SIZE_MAX;
- int sort_ofs = INT_MAX;
- size_t allocated_files = 0;
-
- dict_set_case_limit (proc.dict, dict_get_case_limit (dataset_dict (ds)));
-
- lex_match (lexer, T_SLASH);
- for (;;)
- {
- int start_ofs = lex_ofs (lexer);
- enum comb_file_type type;
- if (lex_match_id (lexer, "FILE"))
- type = COMB_FILE;
- else if (command == COMB_MATCH && lex_match_id (lexer, "TABLE"))
- {
- type = COMB_TABLE;
- table_idx = MIN (table_idx, proc.n_files);
- }
- else
- break;
- lex_match (lexer, T_EQUALS);
-
- if (proc.n_files >= allocated_files)
- proc.files = x2nrealloc (proc.files, &allocated_files,
- sizeof *proc.files);
- struct comb_file *file = &proc.files[proc.n_files++];
- *file = (struct comb_file) {
- .type = type,
- .start_ofs = start_ofs,
- .is_sorted = true,
- };
-
- if (lex_match (lexer, T_ASTERISK))
- {
- if (!dataset_has_source (ds))
- {
- lex_next_error (lexer, -1, -1,
- _("Cannot specify the active dataset since none "
- "has been defined."));
- goto error;
- }
-
- if (proc_make_temporary_transformations_permanent (ds))
- lex_next_error (lexer, -1, -1,
- _("This command may not be used after TEMPORARY "
- "when the active dataset is an input source. "
- "Temporary transformations will be made "
- "permanent."));
-
- file->dict = dict_clone (dataset_dict (ds));
- }
- else
- {
- file->handle = fh_parse (lexer, FH_REF_FILE, dataset_session (ds));
- if (file->handle == NULL)
- goto error;
-
- file->reader = any_reader_open_and_decode (file->handle, NULL,
- &file->dict, NULL);
- if (file->reader == NULL)
- goto error;
- }
- file->end_ofs = lex_ofs (lexer) - 1;
-
- while (lex_match (lexer, T_SLASH))
- if (lex_match_id (lexer, "RENAME"))
- {
- if (!parse_dict_rename (lexer, file->dict))
- goto error;
- }
- else if (lex_match_id (lexer, "IN"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_id (lexer))
- goto error;
-
- if (file->in_name)
- {
- lex_error (lexer, _("Multiple IN subcommands for a single FILE "
- "or TABLE."));
- goto error;
- }
- file->in_name = xstrdup (lex_tokcstr (lexer));
- file->in_ofs = lex_ofs (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "SORT"))
- {
- file->is_sorted = false;
- saw_sort = true;
- sort_ofs = MIN (sort_ofs, lex_ofs (lexer) - 1);
- }
-
- if (!merge_dictionary (&proc, lexer, file))
- goto error;
- }
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match (lexer, T_BY))
- {
- if (saw_by)
- {
- lex_sbc_only_once (lexer, "BY");
- goto error;
- }
- saw_by = true;
-
- lex_match (lexer, T_EQUALS);
-
- const struct variable **by_vars;
- if (!parse_sort_criteria (lexer, proc.dict, &proc.by_vars,
- &by_vars, NULL))
- goto error;
-
- bool ok = true;
- for (size_t i = 0; i < proc.n_files; i++)
- {
- struct comb_file *file = &proc.files[i];
- for (size_t j = 0; j < subcase_get_n_fields (&proc.by_vars); j++)
- {
- const char *name = var_get_name (by_vars[j]);
- struct variable *var = dict_lookup_var (file->dict, name);
- if (var != NULL)
- subcase_add_var (&file->by_vars, var,
- subcase_get_direction (&proc.by_vars, j));
- else
- {
- const char *fn
- = file->handle ? fh_get_name (file->handle) : "*";
- lex_ofs_error (lexer, file->start_ofs, file->end_ofs,
- _("File %s lacks BY variable %s."),
- fn, name);
- ok = false;
- }
- }
- assert (!ok || subcase_conformable (&file->by_vars,
- &proc.files[0].by_vars));
- }
- free (by_vars);
-
- if (!ok)
- goto error;
- }
- else if (command != COMB_UPDATE && lex_match_id (lexer, "FIRST"))
- {
- if (first_name != NULL)
- {
- lex_sbc_only_once (lexer, "FIRST");
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
- if (!lex_force_id (lexer))
- goto error;
- first_name = xstrdup (lex_tokcstr (lexer));
- first_ofs = lex_ofs (lexer);
- lex_get (lexer);
- }
- else if (command != COMB_UPDATE && lex_match_id (lexer, "LAST"))
- {
- if (last_name != NULL)
- {
- lex_sbc_only_once (lexer, "LAST");
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
- if (!lex_force_id (lexer))
- goto error;
- last_name = xstrdup (lex_tokcstr (lexer));
- last_ofs = lex_ofs (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "MAP"))
- {
- /* FIXME. */
- }
- else if (lex_match_id (lexer, "DROP"))
- {
- if (!parse_dict_drop (lexer, proc.dict))
- goto error;
- }
- else if (lex_match_id (lexer, "KEEP"))
- {
- if (!parse_dict_keep (lexer, proc.dict))
- goto error;
- }
- else
- {
- if (command == COMB_UPDATE)
- lex_error_expecting (lexer, "BY", "MAP", "DROP", "KEEP");
- else
- lex_error_expecting (lexer, "BY", "FIRST", "LAST",
- "MAP", "DROP", "KEEP");
- goto error;
- }
-
- if (!lex_match (lexer, T_SLASH) && lex_token (lexer) != T_ENDCMD)
- {
- lex_end_of_command (lexer);
- goto error;
- }
- }
-
- if (!saw_by)
- {
- if (command == COMB_UPDATE)
- {
- lex_sbc_missing (lexer, "BY");
- goto error;
- }
- if (table_idx != SIZE_MAX)
- {
- const struct comb_file *table = &proc.files[table_idx];
- lex_ofs_error (lexer, table->start_ofs, table->end_ofs,
- _("BY is required when %s is specified."), "TABLE");
- goto error;
- }
- if (saw_sort)
- {
- lex_ofs_error (lexer, sort_ofs, sort_ofs,
- _("BY is required when %s is specified."), "SORT");
- goto error;
- }
- }
-
- /* Add IN, FIRST, and LAST variables to master dictionary. */
- for (size_t i = 0; i < proc.n_files; i++)
- {
- struct comb_file *file = &proc.files[i];
- if (!create_flag_var (lexer, "IN", file->in_name, file->in_ofs,
- proc.dict, &file->in_var))
- goto error;
- }
- if (!create_flag_var (lexer, "FIRST", first_name, first_ofs, proc.dict, &proc.first)
- || !create_flag_var (lexer, "LAST", last_name, last_ofs, proc.dict, &proc.last))
- goto error;
-
- dict_delete_scratch_vars (proc.dict);
- dict_compact_values (proc.dict);
-
- /* Set up mapping from each file's variables to master
- variables. */
- for (size_t i = 0; i < proc.n_files; i++)
- {
- struct comb_file *file = &proc.files[i];
- size_t src_n_vars = dict_get_n_vars (file->dict);
-
- file->mv = xnmalloc (src_n_vars, sizeof *file->mv);
- for (size_t j = 0; j < src_n_vars; j++)
- {
- struct variable *src_var = dict_get_var (file->dict, j);
- struct variable *dst_var = dict_lookup_var (proc.dict,
- var_get_name (src_var));
- if (dst_var != NULL)
- {
- size_t n = subcase_get_n_fields (&file->src);
- file->mv[n] = var_get_missing_values (src_var);
- subcase_add_var (&file->src, src_var, SC_ASCEND);
- subcase_add_var (&file->dst, dst_var, SC_ASCEND);
- }
- }
- }
-
- proc.output = autopaging_writer_create (dict_get_proto (proc.dict));
- taint = taint_clone (casewriter_get_taint (proc.output));
-
- /* Set up case matcher. */
- proc.matcher = case_matcher_create ();
- for (size_t i = 0; i < proc.n_files; i++)
- {
- struct comb_file *file = &proc.files[i];
- if (file->reader == NULL)
- {
- if (active_file == NULL)
- {
- proc_discard_output (ds);
- file->reader = active_file = proc_open_filtering (ds, false);
- }
- else
- file->reader = casereader_clone (active_file);
- }
- if (!file->is_sorted)
- file->reader = sort_execute (file->reader, &file->by_vars);
- taint_propagate (casereader_get_taint (file->reader), taint);
- file->data = casereader_read (file->reader);
- if (file->type == COMB_FILE)
- case_matcher_add_input (proc.matcher, &file->by_vars,
- &file->data, &file->is_minimal);
- }
-
- if (command == COMB_ADD)
- execute_add_files (&proc);
- else if (command == COMB_MATCH)
- execute_match_files (&proc);
- else if (command == COMB_UPDATE)
- execute_update (&proc);
- else
- NOT_REACHED ();
-
- case_matcher_destroy (proc.matcher);
- proc.matcher = NULL;
- close_all_comb_files (&proc);
- if (active_file != NULL)
- proc_commit (ds);
-
- dataset_set_dict (ds, proc.dict);
- dataset_set_source (ds, casewriter_make_reader (proc.output));
- proc.dict = NULL;
- proc.output = NULL;
-
- free_comb_proc (&proc);
-
- free (first_name);
- free (last_name);
-
- return taint_destroy (taint) ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-
- error:
- if (active_file != NULL)
- proc_commit (ds);
- free_comb_proc (&proc);
- taint_destroy (taint);
- free (first_name);
- free (last_name);
- return CMD_CASCADING_FAILURE;
-}
-
-/* Merge the dictionary for file F into master dictionary for PROC. */
-static bool
-merge_dictionary (struct comb_proc *proc, struct lexer *lexer,
- struct comb_file *f)
-{
- struct dictionary *m = proc->dict;
- struct dictionary *d = f->dict;
-
- if (dict_get_label (m) == NULL)
- dict_set_label (m, dict_get_label (d));
-
- /* FIXME: If the input files have different encodings, then
- the result is undefined.
- The correct thing to do would be to convert to an encoding
- which can cope with all the input files (eg UTF-8).
- */
- if (strcmp (dict_get_encoding (f->dict), dict_get_encoding (m)))
- msg (MW, _("Combining files with incompatible encodings. String data may "
- "not be represented correctly."));
-
- const struct string_array *d_docs = dict_get_documents (d);
- const struct string_array *m_docs = dict_get_documents (m);
- if (d_docs)
- {
- if (!m_docs)
- dict_set_documents (m, d_docs);
- else
- {
- size_t n = m_docs->n + d_docs->n;
- struct string_array new_docs = {
- .strings = xmalloc (n * sizeof *new_docs.strings),
- };
- for (size_t i = 0; i < m_docs->n; i++)
- new_docs.strings[new_docs.n++] = m_docs->strings[i];
- for (size_t i = 0; i < d_docs->n; i++)
- new_docs.strings[new_docs.n++] = d_docs->strings[i];
-
- dict_set_documents (m, &new_docs);
-
- free (new_docs.strings);
- }
- }
-
- for (size_t i = 0; i < dict_get_n_vars (d); i++)
- {
- struct variable *dv = dict_get_var (d, i);
- struct variable *mv = dict_lookup_var (m, var_get_name (dv));
-
- if (dict_class_from_id (var_get_name (dv)) == DC_SCRATCH)
- continue;
-
- if (!mv)
- {
- mv = dict_clone_var_assert (m, dv);
- if (proc->n_var_sources >= proc->allocated_var_sources)
- proc->var_sources = x2nrealloc (proc->var_sources,
- &proc->allocated_var_sources,
- sizeof *proc->var_sources);
- proc->var_sources[proc->n_var_sources++] = f - proc->files;
- }
- else
- {
- if (var_get_width (mv) != var_get_width (dv))
- {
- const char *var_name = var_get_name (dv);
- msg (SE, _("Variable %s has different type or width in different "
- "files."), var_name);
-
- for (size_t j = 0; j < 2; j++)
- {
- const struct variable *ev = !j ? mv : dv;
- const struct comb_file *ef
- = !j ? &proc->files[proc->var_sources[var_get_dict_index (mv)]] : f;
- const char *fn = ef->handle ? fh_get_name (ef->handle) : "*";
-
- if (var_is_numeric (ev))
- lex_ofs_msg (lexer, SN, ef->start_ofs, ef->end_ofs,
- _("In file %s, %s is numeric."),
- fn, var_name);
- else
- lex_ofs_msg (lexer, SN, ef->start_ofs, ef->end_ofs,
- _("In file %s, %s is a string with width %d."),
- fn, var_name, var_get_width (ev));
- }
-
- return false;
- }
-
- if (var_has_value_labels (dv) && !var_has_value_labels (mv))
- var_set_value_labels (mv, var_get_value_labels (dv));
- if (var_has_missing_values (dv) && !var_has_missing_values (mv))
- var_set_missing_values (mv, var_get_missing_values (dv));
- if (var_get_label (dv) && !var_get_label (mv))
- var_set_label (mv, var_get_label (dv));
- }
- }
-
- return true;
-}
-
-/* If VAR_NAME is non-NULL, attempts to create a
- variable named VAR_NAME, with format F1.0, in DICT, and stores
- a pointer to the variable in *VAR. Returns true if
- successful, false if the variable name is a duplicate (in
- which case a message saying that the variable specified on the
- given SUBCOMMAND is a duplicate is emitted).
-
- Does nothing and returns true if VAR_NAME is null. */
-static bool
-create_flag_var (struct lexer *lexer, const char *subcommand,
- const char *var_name, int var_ofs,
- struct dictionary *dict, struct variable **var)
-{
- if (var_name != NULL)
- {
- struct fmt_spec format = fmt_for_output (FMT_F, 1, 0);
- *var = dict_create_var (dict, var_name, 0);
- if (*var == NULL)
- {
- lex_ofs_error (lexer, var_ofs, var_ofs,
- _("Variable name %s specified on %s subcommand "
- "duplicates an existing variable name."),
- var_name, subcommand);
- return false;
- }
- var_set_both_formats (*var, &format);
- }
- else
- *var = NULL;
- return true;
-}
-
-/* Closes all the files in PROC and frees their associated data. */
-static void
-close_all_comb_files (struct comb_proc *proc)
-{
- for (size_t i = 0; i < proc->n_files; i++)
- {
- struct comb_file *file = &proc->files[i];
- subcase_uninit (&file->by_vars);
- subcase_uninit (&file->src);
- subcase_uninit (&file->dst);
- free (file->mv);
- fh_unref (file->handle);
- dict_unref (file->dict);
- casereader_destroy (file->reader);
- case_unref (file->data);
- free (file->in_name);
- }
- free (proc->files);
- proc->files = NULL;
- proc->n_files = 0;
-}
-
-/* Frees all the data for the procedure. */
-static void
-free_comb_proc (struct comb_proc *proc)
-{
- close_all_comb_files (proc);
- dict_unref (proc->dict);
- casewriter_destroy (proc->output);
- case_matcher_destroy (proc->matcher);
- if (proc->prev_BY)
- {
- caseproto_destroy_values (subcase_get_proto (&proc->by_vars),
- proc->prev_BY);
- free (proc->prev_BY);
- }
- subcase_uninit (&proc->by_vars);
- case_unref (proc->buffered_case);
- free (proc->var_sources);
-}
-\f
-static bool scan_table (struct comb_file *, union value by[]);
-static struct ccase *create_output_case (const struct comb_proc *);
-static void apply_case (const struct comb_file *, struct ccase *);
-static void apply_nonmissing_case (const struct comb_file *, struct ccase *);
-static void advance_file (struct comb_file *, union value by[]);
-static void output_case (struct comb_proc *, struct ccase *, union value by[]);
-static void output_buffered_case (struct comb_proc *);
-
-/* Executes the ADD FILES command. */
-static void
-execute_add_files (struct comb_proc *proc)
-{
- union value *by;
-
- while (case_matcher_match (proc->matcher, &by))
- for (size_t i = 0; i < proc->n_files; i++)
- {
- struct comb_file *file = &proc->files[i];
- while (file->is_minimal)
- {
- struct ccase *output = create_output_case (proc);
- apply_case (file, output);
- advance_file (file, by);
- output_case (proc, output, by);
- }
- }
- output_buffered_case (proc);
-}
-
-/* Executes the MATCH FILES command. */
-static void
-execute_match_files (struct comb_proc *proc)
-{
- union value *by;
-
- while (case_matcher_match (proc->matcher, &by))
- {
- struct ccase *output = create_output_case (proc);
- for (size_t i = proc->n_files; i-- > 0;)
- {
- struct comb_file *file = &proc->files[i];
- if (file->type == COMB_FILE)
- {
- if (file->is_minimal)
- {
- apply_case (file, output);
- advance_file (file, NULL);
- }
- }
- else
- {
- if (scan_table (file, by))
- apply_case (file, output);
- }
- }
- output_case (proc, output, by);
- }
- output_buffered_case (proc);
-}
-
-/* Executes the UPDATE command. */
-static void
-execute_update (struct comb_proc *proc)
-{
- union value *by;
- size_t n_duplicates = 0;
-
- while (case_matcher_match (proc->matcher, &by))
- {
- struct comb_file *first, *file;
- struct ccase *output;
-
- /* Find first nonnull case in array and make an output case
- from it. */
- output = create_output_case (proc);
- for (first = &proc->files[0]; ; first++)
- if (first->is_minimal)
- break;
- apply_case (first, output);
- advance_file (first, by);
-
- /* Read additional cases and update the output case from
- them. (Don't update the output case from any duplicate
- cases in the master file.) */
- for (file = first + (first == proc->files);
- file < &proc->files[proc->n_files]; file++)
- {
- while (file->is_minimal)
- {
- apply_nonmissing_case (file, output);
- advance_file (file, by);
- }
- }
- casewriter_write (proc->output, output);
-
- /* Write duplicate cases in the master file directly to the
- output. */
- if (first == proc->files && first->is_minimal)
- {
- n_duplicates++;
- while (first->is_minimal)
- {
- output = create_output_case (proc);
- apply_case (first, output);
- advance_file (first, by);
- casewriter_write (proc->output, output);
- }
- }
- }
-
- if (n_duplicates)
- msg (SW, _("Encountered %zu sets of duplicate cases in the master file."),
- n_duplicates);
-}
-
-/* Reads FILE, which must be of type COMB_TABLE, until it
- encounters a case with BY or greater for its BY variables.
- Returns true if a case with exactly BY for its BY variables
- was found, otherwise false. */
-static bool
-scan_table (struct comb_file *file, union value by[])
-{
- while (file->data != NULL)
- {
- int cmp = subcase_compare_3way_xc (&file->by_vars, by, file->data);
- if (cmp > 0)
- {
- case_unref (file->data);
- file->data = casereader_read (file->reader);
- }
- else
- return cmp == 0;
- }
- return false;
-}
-
-/* Creates and returns an output case for PROC, initializing each
- of its values to system-missing or blanks, except that the
- values of IN variables are set to 0. */
-static struct ccase *
-create_output_case (const struct comb_proc *proc)
-{
- size_t n_vars = dict_get_n_vars (proc->dict);
- struct ccase *output = case_create (dict_get_proto (proc->dict));
- for (size_t i = 0; i < n_vars; i++)
- {
- struct variable *v = dict_get_var (proc->dict, i);
- value_set_missing (case_data_rw (output, v), var_get_width (v));
- }
- for (size_t i = 0; i < proc->n_files; i++)
- {
- struct comb_file *file = &proc->files[i];
- if (file->in_var != NULL)
- *case_num_rw (output, file->in_var) = false;
- }
- return output;
-}
-
-static void
-mark_file_used (const struct comb_file *file, struct ccase *output)
-{
- if (file->in_var != NULL)
- *case_num_rw (output, file->in_var) = true;
-}
-
-/* Copies the data from FILE's case into output case OUTPUT.
- If FILE has an IN variable, then it is set to 1 in OUTPUT. */
-static void
-apply_case (const struct comb_file *file, struct ccase *output)
-{
- subcase_copy (&file->src, file->data, &file->dst, output);
- mark_file_used (file, output);
-}
-
-/* Copies the data from FILE's case into output case OUTPUT,
- skipping values that are missing or all spaces.
-
- If FILE has an IN variable, then it is set to 1 in OUTPUT. */
-static void
-apply_nonmissing_case (const struct comb_file *file, struct ccase *output)
-{
- for (size_t i = 0; i < subcase_get_n_fields (&file->src); i++)
- {
- const struct subcase_field *src_field = &file->src.fields[i];
- const struct subcase_field *dst_field = &file->dst.fields[i];
- const union value *src_value
- = case_data_idx (file->data, src_field->case_index);
- int width = src_field->width;
-
- if (!mv_is_value_missing (file->mv[i], src_value)
- && !(width > 0 && value_is_spaces (src_value, width)))
- value_copy (case_data_rw_idx (output, dst_field->case_index),
- src_value, width);
- }
- mark_file_used (file, output);
-}
-
-/* Advances FILE to its next case. If BY is nonnull, then FILE's is_minimal
- member is updated based on whether the new case's BY values still match
- those in BY. */
-static void
-advance_file (struct comb_file *file, union value by[])
-{
- case_unref (file->data);
- file->data = casereader_read (file->reader);
- if (by)
- file->is_minimal = (file->data != NULL
- && subcase_equal_cx (&file->by_vars, file->data, by));
-}
-
-/* Writes OUTPUT, whose BY values has been extracted into BY, to
- PROC's output file, first initializing any FIRST or LAST
- variables in OUTPUT to the correct values. */
-static void
-output_case (struct comb_proc *proc, struct ccase *output, union value by[])
-{
- if (proc->first == NULL && proc->last == NULL)
- casewriter_write (proc->output, output);
- else
- {
- /* It's harder with LAST, because we can't know whether
- this case is the last in a group until we've prepared
- the *next* case also. Thus, we buffer the previous
- output case until the next one is ready. */
- bool new_BY;
- if (proc->prev_BY != NULL)
- {
- new_BY = !subcase_equal_xx (&proc->by_vars, proc->prev_BY, by);
- if (proc->last != NULL)
- *case_num_rw (proc->buffered_case, proc->last) = new_BY;
- casewriter_write (proc->output, proc->buffered_case);
- }
- else
- new_BY = true;
-
- proc->buffered_case = output;
- if (proc->first != NULL)
- *case_num_rw (proc->buffered_case, proc->first) = new_BY;
-
- if (new_BY)
- {
- size_t n_values = subcase_get_n_fields (&proc->by_vars);
- const struct caseproto *proto = subcase_get_proto (&proc->by_vars);
- if (proc->prev_BY == NULL)
- {
- proc->prev_BY = xmalloc (n_values * sizeof *proc->prev_BY);
- caseproto_init_values (proto, proc->prev_BY);
- }
- caseproto_copy (subcase_get_proto (&proc->by_vars), 0, n_values,
- proc->prev_BY, by);
- }
- }
-}
-
-/* Writes a trailing buffered case to the output, if FIRST or
- LAST is in use. */
-static void
-output_buffered_case (struct comb_proc *proc)
-{
- if (proc->prev_BY != NULL)
- {
- if (proc->last != NULL)
- *case_num_rw (proc->buffered_case, proc->last) = 1.0;
- casewriter_write (proc->output, proc->buffered_case);
- proc->buffered_case = NULL;
- }
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2007, 2009, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <float.h>
-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/data-in.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/settings.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/data-parser.h"
-#include "language/data-io/data-reader.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/inpt-pgm.h"
-#include "language/data-io/placement-parser.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-
-#include "gl/xsize.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-\f
-/* DATA LIST transformation data. */
-struct data_list_trns
- {
- struct data_parser *parser; /* Parser. */
- struct dictionary *dict; /* Dictionary. */
- struct dfm_reader *reader; /* Data file reader. */
- struct variable *end; /* Variable specified on END subcommand. */
- };
-
-static bool parse_fixed (struct lexer *, struct dictionary *,
- struct pool *, struct data_parser *);
-static bool parse_free (struct lexer *, struct dictionary *,
- struct pool *, struct data_parser *);
-
-static const struct trns_class data_list_trns_class;
-
-int
-cmd_data_list (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = (in_input_program ()
- ? dataset_dict (ds)
- : dict_create (get_default_encoding ()));
- struct data_parser *parser = data_parser_create ();
- struct dfm_reader *reader = NULL;
-
- struct variable *end = NULL;
- struct file_handle *fh = NULL;
-
- char *encoding = NULL;
- int encoding_start = 0, encoding_end = 0;
-
- int table = -1; /* Print table if nonzero, -1=undecided. */
-
- bool has_type = false;
-
- int end_start = 0, end_end = 0;
- while (lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "FILE"))
- {
- lex_match (lexer, T_EQUALS);
- fh_unref (fh);
- fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
- if (fh == NULL)
- goto error;
- }
- else if (lex_match_id (lexer, "ENCODING"))
- {
- encoding_start = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- encoding_end = lex_ofs (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "RECORDS"))
- {
- if (data_parser_get_records (parser) > 0)
- {
- lex_sbc_only_once (lexer, "RECORDS");
- goto error;
- }
- lex_match (lexer, T_EQUALS);
- lex_match (lexer, T_LPAREN);
- if (!lex_force_int_range (lexer, "RECORDS", 0, INT_MAX))
- goto error;
- data_parser_set_records (parser, lex_integer (lexer));
- lex_get (lexer);
- lex_match (lexer, T_RPAREN);
- }
- else if (lex_match_id (lexer, "SKIP"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "SKIP", 0, INT_MAX))
- goto error;
- data_parser_set_skip (parser, lex_integer (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "END"))
- {
- if (!in_input_program ())
- {
- lex_next_error (lexer, -1, -1,
- _("The %s subcommand may only be used within %s."),
- "END", "INPUT PROGRAM");
- goto error;
- }
- if (end)
- {
- lex_sbc_only_once (lexer, "END");
- goto error;
- }
-
- end_start = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
- if (!lex_force_id (lexer))
- goto error;
- end_end = lex_ofs (lexer);
-
- end = dict_lookup_var (dict, lex_tokcstr (lexer));
- if (!end)
- end = dict_create_var_assert (dict, lex_tokcstr (lexer), 0);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "NOTABLE"))
- table = 0;
- else if (lex_match_id (lexer, "TABLE"))
- table = 1;
- else if (lex_token (lexer) == T_ID)
- {
- if (lex_match_id (lexer, "FIXED"))
- data_parser_set_type (parser, DP_FIXED);
- else if (lex_match_id (lexer, "FREE"))
- {
- data_parser_set_type (parser, DP_DELIMITED);
- data_parser_set_span (parser, true);
- }
- else if (lex_match_id (lexer, "LIST"))
- {
- data_parser_set_type (parser, DP_DELIMITED);
- data_parser_set_span (parser, false);
- }
- else
- {
- lex_error_expecting (lexer, "FILE", "ENCODING", "RECORDS",
- "SKIP", "END", "NOTABLE", "TABLE",
- "FIXED", "FREE", "LIST");
- goto error;
- }
-
- if (has_type)
- {
- lex_next_error (lexer, -1, -1,
- _("Only one of FIXED, FREE, or LIST may "
- "be specified."));
- goto error;
- }
- has_type = true;
-
- if (data_parser_get_type (parser) == DP_DELIMITED)
- {
- if (lex_match (lexer, T_LPAREN))
- {
- struct string delims = DS_EMPTY_INITIALIZER;
-
- do
- {
- int delim;
-
- if (lex_match_id (lexer, "TAB"))
- delim = '\t';
- else if (lex_is_string (lexer)
- && ss_length (lex_tokss (lexer)) == 1)
- {
- delim = ss_first (lex_tokss (lexer));
- lex_get (lexer);
- }
- else
- {
- /* XXX should support multibyte UTF-8 characters */
- lex_error (lexer, _("Syntax error expecting TAB "
- "or delimiter string."));
- ds_destroy (&delims);
- goto error;
- }
- ds_put_byte (&delims, delim);
-
- lex_match (lexer, T_COMMA);
- }
- while (!lex_match (lexer, T_RPAREN));
-
- data_parser_set_empty_line_has_field (parser, true);
- data_parser_set_quotes (parser, ss_empty ());
- data_parser_set_soft_delimiters (parser, ss_empty ());
- data_parser_set_hard_delimiters (parser, ds_ss (&delims));
- ds_destroy (&delims);
- }
- else
- {
- data_parser_set_empty_line_has_field (parser, false);
- data_parser_set_quotes (parser, ss_cstr ("'\""));
- data_parser_set_soft_delimiters (parser,
- ss_cstr (CC_SPACES));
- const char decimal = settings_get_fmt_settings ()->decimal;
- data_parser_set_hard_delimiters (parser,
- ss_buffer (",", (decimal == '.') ? 1 : 0));
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "FILE", "ENCODING", "RECORDS",
- "SKIP", "END", "NOTABLE", "TABLE",
- "FIXED", "FREE", "LIST");
- goto error;
- }
- }
-
- if (!fh)
- {
- fh = fh_inline_file ();
-
- if (encoding)
- lex_ofs_msg (lexer, SW, encoding_start, encoding_end,
- _("Encoding should not be specified for inline data. "
- "It will be ignored."));
- }
- fh_set_default_handle (fh);
-
- enum data_parser_type type = data_parser_get_type (parser);
- if (type != DP_FIXED && end != NULL)
- {
- lex_ofs_error (lexer, end_start, end_end,
- _("The %s subcommand may be used only with %s."),
- "END", "DATA LIST FIXED");
- goto error;
- }
-
- struct pool *tmp_pool = pool_create ();
- bool ok = (type == DP_FIXED
- ? parse_fixed (lexer, dict, tmp_pool, parser)
- : parse_free (lexer, dict, tmp_pool, parser));
- pool_destroy (tmp_pool);
- if (!ok)
- goto error;
- assert (data_parser_any_fields (parser));
-
- if (lex_end_of_command (lexer) != CMD_SUCCESS)
- goto error;
-
- if (table == -1)
- table = type == DP_FIXED || !data_parser_get_span (parser);
- if (table)
- data_parser_output_description (parser, fh);
-
- reader = dfm_open_reader (fh, lexer, encoding);
- if (reader == NULL)
- goto error;
-
- if (in_input_program ())
- {
- struct data_list_trns *trns = xmalloc (sizeof *trns);
- *trns = (struct data_list_trns) {
- .parser = parser,
- .dict = dict_ref (dict),
- .reader = reader,
- .end = end,
- };
- add_transformation (ds, &data_list_trns_class, trns);
- }
- else
- data_parser_make_active_file (parser, ds, reader, dict, NULL, NULL);
-
- fh_unref (fh);
- free (encoding);
-
- data_list_seen ();
-
- return CMD_SUCCESS;
-
- error:
- data_parser_destroy (parser);
- if (!in_input_program ())
- dict_unref (dict);
- fh_unref (fh);
- free (encoding);
- return CMD_CASCADING_FAILURE;
-}
-\f
-/* Fixed-format parsing. */
-
-/* Parses all the variable specifications for DATA LIST FIXED,
- storing them into DLS. Uses TMP_POOL for temporary storage;
- the caller may destroy it. Returns true only if
- successful. */
-static bool
-parse_fixed (struct lexer *lexer, struct dictionary *dict,
- struct pool *tmp_pool, struct data_parser *parser)
-{
- int max_records = data_parser_get_records (parser);
- int record = 0;
- int column = 1;
-
- do
- {
- /* Parse everything. */
- int records_start = lex_ofs (lexer);
- if (!parse_record_placement (lexer, &record, &column))
- return false;
-
- int vars_start = lex_ofs (lexer);
- char **names;
- size_t n_names;
- if (!parse_DATA_LIST_vars_pool (lexer, dict, tmp_pool,
- &names, &n_names, PV_NONE))
- return false;
- int vars_end = lex_ofs (lexer) - 1;
- struct fmt_spec *formats;
- size_t n_formats;
- if (!parse_var_placements (lexer, tmp_pool, n_names, FMT_FOR_INPUT,
- &formats, &n_formats))
- return false;
- int placements_end = lex_ofs (lexer) - 1;
-
- /* Create variables and var specs. */
- size_t name_idx = 0;
- for (struct fmt_spec *f = formats; f < &formats[n_formats]; f++)
- if (!execute_placement_format (f, &record, &column))
- {
- /* Create variable. */
- const char *name = names[name_idx++];
- int width = fmt_var_width (f);
- struct variable *v = dict_create_var (dict, name, width);
- if (v != NULL)
- {
- /* Success. */
- struct fmt_spec output = fmt_for_output_from_input (
- f, settings_get_fmt_settings ());
- var_set_both_formats (v, &output);
- }
- else
- {
- /* Failure.
- This can be acceptable if we're in INPUT
- PROGRAM, but only if the existing variable has
- the same width as the one we would have
- created. */
- if (!in_input_program ())
- {
- lex_ofs_error (lexer, vars_start, vars_end,
- _("%s is a duplicate variable name."), name);
- return false;
- }
-
- v = dict_lookup_var_assert (dict, name);
- if ((width != 0) != (var_get_width (v) != 0))
- {
- lex_ofs_error (lexer, vars_start, placements_end,
- _("There is already a variable %s of a "
- "different type."), name);
- return false;
- }
- if (width != 0 && width != var_get_width (v))
- {
- lex_ofs_error (lexer, vars_start, placements_end,
- _("There is already a string variable %s of "
- "a different width."), name);
- return false;
- }
- }
-
- if (max_records && record > max_records)
- {
- lex_ofs_error (lexer, records_start, vars_end,
- _("Cannot place variable %s on record %d when "
- "RECORDS=%d is specified."),
- var_get_name (v), record,
- data_parser_get_records (parser));
- return false;
- }
-
- data_parser_add_fixed_field (parser, f,
- var_get_case_index (v),
- var_get_name (v), record, column);
-
- column += f->w;
- }
- assert (name_idx == n_names);
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- return true;
-}
-\f
-/* Free-format parsing. */
-
-/* Parses variable specifications for DATA LIST FREE and adds
- them to DLS. Uses TMP_POOL for temporary storage; the caller
- may destroy it. Returns true only if successful. */
-static bool
-parse_free (struct lexer *lexer, struct dictionary *dict,
- struct pool *tmp_pool, struct data_parser *parser)
-{
- lex_get (lexer);
- do
- {
- char **names;
- size_t n_names;
-
- int vars_start = lex_ofs (lexer);
- if (!parse_DATA_LIST_vars_pool (lexer, dict, tmp_pool,
- &names, &n_names, PV_NONE))
- return false;
- int vars_end = lex_ofs (lexer) - 1;
-
- struct fmt_spec input, output;
- if (lex_match (lexer, T_LPAREN))
- {
- char type[FMT_TYPE_LEN_MAX + 1];
-
- if (!parse_abstract_format_specifier (lexer, type, &input.w,
- &input.d))
- return NULL;
- if (!fmt_from_name (type, &input.type))
- {
- lex_next_error (lexer, -1, -1,
- _("Unknown format type `%s'."), type);
- return NULL;
- }
-
- /* If no width was included, use the minimum width for the type.
- This isn't quite right, because DATETIME by itself seems to become
- DATETIME20 (see bug #30690), whereas this will become
- DATETIME17. The correct behavior is not documented. */
- if (input.w == 0)
- {
- input.w = fmt_min_input_width (input.type);
- input.d = 0;
- }
-
- char *error = fmt_check_input__ (&input);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- return NULL;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- return NULL;
-
- /* As a special case, N format is treated as F format
- for free-field input. */
- if (input.type == FMT_N)
- input.type = FMT_F;
-
- output = fmt_for_output_from_input (&input,
- settings_get_fmt_settings ());
- }
- else
- {
- lex_match (lexer, T_ASTERISK);
- input = fmt_for_input (FMT_F, 8, 0);
- output = *settings_get_format ();
- }
-
- for (size_t i = 0; i < n_names; i++)
- {
- struct variable *v = dict_create_var (dict, names[i],
- fmt_var_width (&input));
- if (!v)
- {
- lex_ofs_error (lexer, vars_start, vars_end,
- _("%s is a duplicate variable name."), names[i]);
- return false;
- }
- var_set_both_formats (v, &output);
-
- data_parser_add_delimited_field (parser,
- &input, var_get_case_index (v),
- var_get_name (v));
- }
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- return true;
-}
-\f
-/* Input procedure. */
-
-/* Destroys DATA LIST transformation TRNS.
- Returns true if successful, false if an I/O error occurred. */
-static bool
-data_list_trns_free (void *trns_)
-{
- struct data_list_trns *trns = trns_;
- data_parser_destroy (trns->parser);
- dfm_close_reader (trns->reader);
- dict_unref (trns->dict);
- free (trns);
- return true;
-}
-
-/* Handle DATA LIST transformation TRNS, parsing data into *C. */
-static enum trns_result
-data_list_trns_proc (void *trns_, struct ccase **c, casenumber case_num UNUSED)
-{
- struct data_list_trns *trns = trns_;
- enum trns_result retval;
-
- *c = case_unshare (*c);
- if (data_parser_parse (trns->parser, trns->reader, trns->dict, *c))
- retval = TRNS_CONTINUE;
- else if (dfm_reader_error (trns->reader) || dfm_eof (trns->reader) > 1)
- {
- /* An I/O error, or encountering end of file for a second
- time, should be escalated into a more serious error. */
- retval = TRNS_ERROR;
- }
- else
- retval = TRNS_END_FILE;
-
- /* If there was an END subcommand handle it. */
- if (trns->end != NULL)
- {
- double *end = case_num_rw (*c, trns->end);
- if (retval == TRNS_END_FILE)
- {
- *end = 1.0;
- retval = TRNS_CONTINUE;
- }
- else
- *end = 0.0;
- }
-
- return retval;
-}
-
-static const struct trns_class data_list_trns_class = {
- .name = "DATA LIST",
- .execute = data_list_trns_proc,
- .destroy = data_list_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2007, 2009, 2010, 2011, 2012, 2013, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/data-io/data-parser.h"
-
-#include <stdint.h>
-#include <stdlib.h>
-
-#include "data/casereader-provider.h"
-#include "data/data-in.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/file-handle-def.h"
-#include "data/settings.h"
-#include "language/data-io/data-reader.h"
-#include "libpspp/intern.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "libpspp/string-array.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-/* Data parser for textual data like that read by DATA LIST. */
-struct data_parser
- {
- enum data_parser_type type; /* Type of data to parse. */
- int skip_records; /* Records to skip before first real data. */
-
- struct field *fields; /* Fields to parse. */
- size_t n_fields; /* Number of fields. */
- size_t field_allocated; /* Number of fields spaced allocated for. */
-
- /* DP_DELIMITED parsers only. */
- bool span; /* May cases span multiple records? */
- bool empty_line_has_field; /* Does an empty line have an (empty) field? */
- bool warn_missing_fields; /* Should missing fields be considered errors? */
- struct substring quotes; /* Characters that can quote separators. */
- bool quote_escape; /* Doubled quote acts as escape? */
- struct substring soft_seps; /* Two soft separators act like just one. */
- struct substring hard_seps; /* Two hard separators yield empty fields. */
- struct string any_sep; /* Concatenation of soft_seps and hard_seps. */
-
- /* DP_FIXED parsers only. */
- int records_per_case; /* Number of records in each case. */
- };
-
-/* How to parse one variable. */
-struct field
- {
- struct fmt_spec format; /* Input format of this field. */
- int case_idx; /* First value in case. */
- char *name; /* Var name for error messages and tables. */
-
- /* DP_FIXED only. */
- int record; /* Record number (1-based). */
- int first_column; /* First column in record (1-based). */
- };
-
-static void set_any_sep (struct data_parser *parser);
-
-/* Creates and returns a new data parser. */
-struct data_parser *
-data_parser_create (void)
-{
- struct data_parser *parser = xmalloc (sizeof *parser);
- *parser = (struct data_parser) {
- .type = DP_FIXED,
- .span = true,
- .warn_missing_fields = true,
- .quotes = ss_clone (ss_cstr ("\"'")),
- .soft_seps = ss_clone (ss_cstr (CC_SPACES)),
- .hard_seps = ss_clone (ss_cstr (",")),
- };
- set_any_sep (parser);
-
- return parser;
-}
-
-/* Destroys PARSER. */
-void
-data_parser_destroy (struct data_parser *parser)
-{
- if (parser != NULL)
- {
- size_t i;
-
- for (i = 0; i < parser->n_fields; i++)
- free (parser->fields[i].name);
- free (parser->fields);
- ss_dealloc (&parser->quotes);
- ss_dealloc (&parser->soft_seps);
- ss_dealloc (&parser->hard_seps);
- ds_destroy (&parser->any_sep);
- free (parser);
- }
-}
-
-/* Returns the type of PARSER (either DP_DELIMITED or DP_FIXED). */
-enum data_parser_type
-data_parser_get_type (const struct data_parser *parser)
-{
- return parser->type;
-}
-
-/* Sets the type of PARSER to TYPE (either DP_DELIMITED or
- DP_FIXED). */
-void
-data_parser_set_type (struct data_parser *parser, enum data_parser_type type)
-{
- assert (parser->n_fields == 0);
- assert (type == DP_FIXED || type == DP_DELIMITED);
- parser->type = type;
-}
-
-/* Configures PARSER to skip the specified number of
- INITIAL_RECORDS_TO_SKIP before parsing any data. By default,
- no records are skipped. */
-void
-data_parser_set_skip (struct data_parser *parser, int initial_records_to_skip)
-{
- assert (initial_records_to_skip >= 0);
- parser->skip_records = initial_records_to_skip;
-}
-
-/* Returns true if PARSER is configured to allow cases to span
- multiple records. */
-bool
-data_parser_get_span (const struct data_parser *parser)
-{
- return parser->span;
-}
-
-/* If MAY_CASES_SPAN_RECORDS is true, configures PARSER to allow
- a single case to span multiple records and multiple cases to
- occupy a single record. If MAY_CASES_SPAN_RECORDS is false,
- configures PARSER to require each record to contain exactly
- one case.
-
- This setting affects parsing of DP_DELIMITED files only. */
-void
-data_parser_set_span (struct data_parser *parser, bool may_cases_span_records)
-{
- parser->span = may_cases_span_records;
-}
-
-/* If EMPTY_LINE_HAS_FIELD is true, configures PARSER to parse an
- empty line as an empty field and to treat a hard delimiter
- followed by end-of-line as an empty field. If
- EMPTY_LINE_HAS_FIELD is false, PARSER will skip empty lines
- and hard delimiters at the end of lines without emitting empty
- fields.
-
- This setting affects parsing of DP_DELIMITED files only. */
-void
-data_parser_set_empty_line_has_field (struct data_parser *parser,
- bool empty_line_has_field)
-{
- parser->empty_line_has_field = empty_line_has_field;
-}
-
-
-/* If WARN_MISSING_FIELDS is true, configures PARSER to emit a warning
- and cause an error condition when a missing field is encountered.
- If WARN_MISSING_FIELDS is false, PARSER will silently fill such
- fields with the system missing value.
-
- This setting affects parsing of DP_DELIMITED files only. */
-void
-data_parser_set_warn_missing_fields (struct data_parser *parser,
- bool warn_missing_fields)
-{
- parser->warn_missing_fields = warn_missing_fields;
-}
-
-
-/* Sets the characters that may be used for quoting field
- contents to QUOTES. If QUOTES is empty, quoting will be
- disabled.
-
- The caller retains ownership of QUOTES.
-
- This setting affects parsing of DP_DELIMITED files only. */
-void
-data_parser_set_quotes (struct data_parser *parser, struct substring quotes)
-{
- ss_dealloc (&parser->quotes);
- parser->quotes = ss_clone (quotes);
-}
-
-/* If ESCAPE is false (the default setting), a character used for
- quoting cannot itself be embedded within a quoted field. If
- ESCAPE is true, then a quote character can be embedded within
- a quoted field by doubling it.
-
- This setting affects parsing of DP_DELIMITED files only, and
- only when at least one quote character has been set (with
- data_parser_set_quotes). */
-void
-data_parser_set_quote_escape (struct data_parser *parser, bool escape)
-{
- parser->quote_escape = escape;
-}
-
-/* Sets PARSER's soft delimiters to DELIMITERS. Soft delimiters
- separate fields, but consecutive soft delimiters do not yield
- empty fields. (Ordinarily, only white space characters are
- appropriate soft delimiters.)
-
- The caller retains ownership of DELIMITERS.
-
- This setting affects parsing of DP_DELIMITED files only. */
-void
-data_parser_set_soft_delimiters (struct data_parser *parser,
- struct substring delimiters)
-{
- ss_dealloc (&parser->soft_seps);
- parser->soft_seps = ss_clone (delimiters);
- set_any_sep (parser);
-}
-
-/* Sets PARSER's hard delimiters to DELIMITERS. Hard delimiters
- separate fields. A consecutive pair of hard delimiters yield
- an empty field.
-
- The caller retains ownership of DELIMITERS.
-
- This setting affects parsing of DP_DELIMITED files only. */
-void
-data_parser_set_hard_delimiters (struct data_parser *parser,
- struct substring delimiters)
-{
- ss_dealloc (&parser->hard_seps);
- parser->hard_seps = ss_clone (delimiters);
- set_any_sep (parser);
-}
-
-/* Returns the number of records per case. */
-int
-data_parser_get_records (const struct data_parser *parser)
-{
- return parser->records_per_case;
-}
-
-/* Sets the number of records per case to RECORDS_PER_CASE.
-
- This setting affects parsing of DP_FIXED files only. */
-void
-data_parser_set_records (struct data_parser *parser, int records_per_case)
-{
- assert (records_per_case >= 0);
- assert (records_per_case >= parser->records_per_case);
- parser->records_per_case = records_per_case;
-}
-
-static void
-add_field (struct data_parser *p, const struct fmt_spec *format, int case_idx,
- const char *name, int record, int first_column)
-{
- struct field *field;
-
- if (p->n_fields == p->field_allocated)
- p->fields = x2nrealloc (p->fields, &p->field_allocated, sizeof *p->fields);
- field = &p->fields[p->n_fields++];
- field->format = *format;
- field->case_idx = case_idx;
- field->name = xstrdup (name);
- field->record = record;
- field->first_column = first_column;
-}
-
-/* Adds a delimited field to the field parsed by PARSER, which
- must be configured as a DP_DELIMITED parser. The field is
- parsed as input format FORMAT. Its data will be stored into case
- index CASE_INDEX. Errors in input data will be reported
- against variable NAME. */
-void
-data_parser_add_delimited_field (struct data_parser *parser,
- const struct fmt_spec *format, int case_idx,
- const char *name)
-{
- assert (parser->type == DP_DELIMITED);
- add_field (parser, format, case_idx, name, 0, 0);
-}
-
-/* Adds a fixed field to the field parsed by PARSER, which
- must be configured as a DP_FIXED parser. The field is
- parsed as input format FORMAT. Its data will be stored into case
- index CASE_INDEX. Errors in input data will be reported
- against variable NAME. The field will be drawn from the
- FORMAT->w columns in 1-based RECORD starting at 1-based
- column FIRST_COLUMN.
-
- RECORD must be at least as great as that of any field already
- added; that is, fields must be added in increasing order of
- record number. If RECORD is greater than the current number
- of records per case, the number of records per case are
- increased as needed. */
-void
-data_parser_add_fixed_field (struct data_parser *parser,
- const struct fmt_spec *format, int case_idx,
- const char *name,
- int record, int first_column)
-{
- assert (parser->type == DP_FIXED);
- assert (parser->n_fields == 0
- || record >= parser->fields[parser->n_fields - 1].record);
- if (record > parser->records_per_case)
- parser->records_per_case = record;
- add_field (parser, format, case_idx, name, record, first_column);
-}
-
-/* Returns true if any fields have been added to PARSER, false
- otherwise. */
-bool
-data_parser_any_fields (const struct data_parser *parser)
-{
- return parser->n_fields > 0;
-}
-
-static void
-set_any_sep (struct data_parser *parser)
-{
- ds_assign_substring (&parser->any_sep, parser->soft_seps);
- ds_put_substring (&parser->any_sep, parser->hard_seps);
-}
-\f
-static bool parse_delimited_span (const struct data_parser *,
- struct dfm_reader *,
- struct dictionary *, struct ccase *);
-static bool parse_delimited_no_span (const struct data_parser *,
- struct dfm_reader *,
- struct dictionary *, struct ccase *);
-static bool parse_fixed (const struct data_parser *, struct dfm_reader *,
- struct dictionary *, struct ccase *);
-
-/* Reads a case from DFM into C, which matches dictionary DICT, parsing it with
- PARSER. Returns true if successful, false at end of file or on I/O error.
-
- Case C must not be shared. */
-bool
-data_parser_parse (struct data_parser *parser, struct dfm_reader *reader,
- struct dictionary *dict, struct ccase *c)
-{
- bool retval;
-
- assert (!case_is_shared (c));
- assert (data_parser_any_fields (parser));
-
- /* Skip the requested number of records before reading the
- first case. */
- for (; parser->skip_records > 0; parser->skip_records--)
- {
- if (dfm_eof (reader))
- return false;
- dfm_forward_record (reader);
- }
-
- /* Limit cases. */
- if (parser->type == DP_DELIMITED)
- {
- if (parser->span)
- retval = parse_delimited_span (parser, reader, dict, c);
- else
- retval = parse_delimited_no_span (parser, reader, dict, c);
- }
- else
- retval = parse_fixed (parser, reader, dict, c);
-
- return retval;
-}
-
-static void
-cut_field__ (const struct data_parser *parser, const struct substring *line,
- struct substring *p, size_t *n_columns,
- struct string *tmp, struct substring *field)
-{
- bool quoted = ss_find_byte (parser->quotes, ss_first (*p)) != SIZE_MAX;
- if (quoted)
- {
- /* Quoted field. */
- int quote = ss_get_byte (p);
- if (!ss_get_until (p, quote, field))
- msg (DW, _("Quoted string extends beyond end of line."));
- if (parser->quote_escape && ss_first (*p) == quote)
- {
- ds_assign_substring (tmp, *field);
- while (ss_match_byte (p, quote))
- {
- struct substring ss;
- ds_put_byte (tmp, quote);
- if (!ss_get_until (p, quote, &ss))
- msg (DW, _("Quoted string extends beyond end of line."));
- ds_put_substring (tmp, ss);
- }
- *field = ds_ss (tmp);
- }
- *n_columns = ss_length (*line) - ss_length (*p);
- }
- else
- {
- /* Regular field. */
- ss_get_bytes (p, ss_cspan (*p, ds_ss (&parser->any_sep)), field);
- *n_columns = ss_length (*field);
- }
-
- /* Skip trailing soft separator and a single hard separator if present. */
- size_t length_before_separators = ss_length (*p);
- ss_ltrim (p, parser->soft_seps);
- if (!ss_is_empty (*p)
- && ss_find_byte (parser->hard_seps, ss_first (*p)) != SIZE_MAX)
- {
- ss_advance (p, 1);
- ss_ltrim (p, parser->soft_seps);
- }
-
- if (!ss_is_empty (*p) && quoted && length_before_separators == ss_length (*p))
- msg (DW, _("Missing delimiter following quoted string."));
-}
-
-/* Extracts a delimited field from the current position in the
- current record according to PARSER, reading data from READER.
-
- *FIELD is set to the field content. The caller must not or
- destroy this constant string.
-
- Sets *FIRST_COLUMN to the 1-based column number of the start of
- the extracted field, and *LAST_COLUMN to the end of the extracted
- field.
-
- Returns true on success, false on failure. */
-static bool
-cut_field (const struct data_parser *parser, struct dfm_reader *reader,
- int *first_column, int *last_column, struct string *tmp,
- struct substring *field)
-{
- struct substring line, p;
-
- if (dfm_eof (reader))
- return false;
- if (ss_is_empty (parser->hard_seps))
- dfm_expand_tabs (reader);
- line = p = dfm_get_record (reader);
-
- /* Skip leading soft separators. */
- ss_ltrim (&p, parser->soft_seps);
-
- /* Handle empty or completely consumed lines. */
- if (ss_is_empty (p))
- {
- if (!parser->empty_line_has_field || dfm_columns_past_end (reader) > 0)
- return false;
- else
- {
- *field = p;
- *first_column = dfm_column_start (reader);
- *last_column = *first_column + 1;
- dfm_forward_columns (reader, 1);
- return true;
- }
- }
-
- size_t n_columns;
- cut_field__ (parser, &line, &p, &n_columns, tmp, field);
- *first_column = dfm_column_start (reader);
- *last_column = *first_column + n_columns;
-
- if (ss_is_empty (p))
- dfm_forward_columns (reader, 1);
- dfm_forward_columns (reader, ss_length (line) - ss_length (p));
-
- return true;
-}
-
-static void
-parse_error (const struct dfm_reader *reader, const struct field *field,
- int first_column, int last_column, char *error)
-{
- int line_number = dfm_get_line_number (reader);
- struct msg_location *location = xmalloc (sizeof *location);
- *location = (struct msg_location) {
- .file_name = intern_new (dfm_get_file_name (reader)),
- .start = { .line = line_number, .column = first_column },
- .end = { .line = line_number, .column = last_column - 1 },
- };
- struct msg *m = xmalloc (sizeof *m);
- *m = (struct msg) {
- .category = MSG_C_DATA,
- .severity = MSG_S_WARNING,
- .location = location,
- .text = xasprintf (_("Data for variable %s is not valid as format %s: %s"),
- field->name, fmt_name (field->format.type), error),
- };
- msg_emit (m);
-
- free (error);
-}
-
-/* Reads a case from READER into C, which matches DICT, parsing it according to
- fixed-format syntax rules in PARSER. Returns true if successful, false at
- end of file or on I/O error. */
-static bool
-parse_fixed (const struct data_parser *parser, struct dfm_reader *reader,
- struct dictionary *dict, struct ccase *c)
-{
- const char *input_encoding = dfm_reader_get_encoding (reader);
- const char *output_encoding = dict_get_encoding (dict);
- struct field *f;
- int row;
-
- if (dfm_eof (reader))
- return false;
-
- f = parser->fields;
- for (row = 1; row <= parser->records_per_case; row++)
- {
- struct substring line;
-
- if (dfm_eof (reader))
- {
- msg (DW, _("Partial case of %d of %d records discarded."),
- row - 1, parser->records_per_case);
- return false;
- }
- dfm_expand_tabs (reader);
- line = dfm_get_record (reader);
-
- for (; f < &parser->fields[parser->n_fields] && f->record == row; f++)
- {
- struct substring s = ss_substr (line, f->first_column - 1,
- f->format.w);
- union value *value = case_data_rw_idx (c, f->case_idx);
- char *error = data_in (s, input_encoding, f->format.type,
- settings_get_fmt_settings (),
- value, fmt_var_width (&f->format),
- output_encoding);
-
- if (error == NULL)
- data_in_imply_decimals (s, input_encoding, f->format.type,
- f->format.d, settings_get_fmt_settings (),
- value);
- else
- parse_error (reader, f, f->first_column,
- f->first_column + f->format.w, error);
- }
-
- dfm_forward_record (reader);
- }
-
- return true;
-}
-
-/* Splits the data line in LINE into individual text fields and returns the
- number of fields. If SA is nonnull, appends each field to SA; the caller
- retains ownership of SA and its contents. */
-size_t
-data_parser_split (const struct data_parser *parser,
- struct substring line, struct string_array *sa)
-{
- size_t n = 0;
-
- struct string tmp = DS_EMPTY_INITIALIZER;
- for (;;)
- {
- struct substring p = line;
- ss_ltrim (&p, parser->soft_seps);
- if (ss_is_empty (p))
- {
- ds_destroy (&tmp);
- return n;
- }
-
- size_t n_columns;
- struct substring field;
-
- msg_disable ();
- cut_field__ (parser, &line, &p, &n_columns, &tmp, &field);
- msg_enable ();
-
- if (sa)
- string_array_append_nocopy (sa, ss_xstrdup (field));
- n++;
- line = p;
- }
-}
-
-/* Reads a case from READER into C, which matches dictionary DICT, parsing it
- according to free-format syntax rules in PARSER. Returns true if
- successful, false at end of file or on I/O error. */
-static bool
-parse_delimited_span (const struct data_parser *parser,
- struct dfm_reader *reader,
- struct dictionary *dict, struct ccase *c)
-{
- const char *output_encoding = dict_get_encoding (dict);
- struct string tmp = DS_EMPTY_INITIALIZER;
- struct field *f;
-
- for (f = parser->fields; f < &parser->fields[parser->n_fields]; f++)
- {
- struct substring s;
- int first_column, last_column;
- char *error;
-
- /* Cut out a field and read in a new record if necessary. */
- while (!cut_field (parser, reader,
- &first_column, &last_column, &tmp, &s))
- {
- if (!dfm_eof (reader))
- dfm_forward_record (reader);
- if (dfm_eof (reader))
- {
- if (f > parser->fields)
- msg (DW, _("Partial case discarded. The first variable "
- "missing was %s."), f->name);
- ds_destroy (&tmp);
- return false;
- }
- }
-
- const char *input_encoding = dfm_reader_get_encoding (reader);
- error = data_in (s, input_encoding, f->format.type,
- settings_get_fmt_settings (),
- case_data_rw_idx (c, f->case_idx),
- fmt_var_width (&f->format), output_encoding);
- if (error != NULL)
- parse_error (reader, f, first_column, last_column, error);
- }
- ds_destroy (&tmp);
- return true;
-}
-
-/* Reads a case from READER into C, which matches dictionary DICT, parsing it
- according to delimited syntax rules with one case per record in PARSER.
- Returns true if successful, false at end of file or on I/O error. */
-static bool
-parse_delimited_no_span (const struct data_parser *parser,
- struct dfm_reader *reader,
- struct dictionary *dict, struct ccase *c)
-{
- const char *output_encoding = dict_get_encoding (dict);
- struct string tmp = DS_EMPTY_INITIALIZER;
- struct substring s;
- struct field *f, *end;
-
- if (dfm_eof (reader))
- return false;
-
- end = &parser->fields[parser->n_fields];
- for (f = parser->fields; f < end; f++)
- {
- int first_column, last_column;
- char *error;
-
- if (!cut_field (parser, reader, &first_column, &last_column, &tmp, &s))
- {
- if (f < end - 1 && settings_get_undefined () && parser->warn_missing_fields)
- msg (DW, _("Missing value(s) for all variables from %s onward. "
- "These will be filled with the system-missing value "
- "or blanks, as appropriate."),
- f->name);
- for (; f < end; f++)
- value_set_missing (case_data_rw_idx (c, f->case_idx),
- fmt_var_width (&f->format));
- goto exit;
- }
-
- const char *input_encoding = dfm_reader_get_encoding (reader);
- error = data_in (s, input_encoding, f->format.type,
- settings_get_fmt_settings (),
- case_data_rw_idx (c, f->case_idx),
- fmt_var_width (&f->format), output_encoding);
- if (error != NULL)
- parse_error (reader, f, first_column, last_column, error);
- }
-
- s = dfm_get_record (reader);
- ss_ltrim (&s, parser->soft_seps);
- if (!ss_is_empty (s))
- msg (DW, _("Record ends in data not part of any field."));
-
-exit:
- dfm_forward_record (reader);
- ds_destroy (&tmp);
- return true;
-}
-\f
-/* Displays a table giving information on fixed-format variable
- parsing on DATA LIST. */
-static void
-dump_fixed_table (const struct data_parser *parser,
- const struct file_handle *fh)
-{
- /* XXX This should not be preformatted. */
- char *title = xasprintf (ngettext ("Reading %d record from %s.",
- "Reading %d records from %s.",
- parser->records_per_case),
- parser->records_per_case, fh_get_name (fh));
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_user_text (title, -1), "Fixed Data Records");
- free (title);
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Attributes"),
- N_("Record"), N_("Columns"), N_("Format"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
- variables->root->show_label = true;
- for (size_t i = 0; i < parser->n_fields; i++)
- {
- struct field *f = &parser->fields[i];
-
- /* XXX It would be better to have the actual variable here. */
- int variable_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_user_text (f->name, -1));
-
- pivot_table_put2 (table, 0, variable_idx,
- pivot_value_new_integer (f->record));
-
- int first_column = f->first_column;
- int last_column = f->first_column + f->format.w - 1;
- char *columns = xasprintf ("%d-%d", first_column, last_column);
- pivot_table_put2 (table, 1, variable_idx,
- pivot_value_new_user_text (columns, -1));
- free (columns);
-
- char str[FMT_STRING_LEN_MAX + 1];
- pivot_table_put2 (table, 2, variable_idx,
- pivot_value_new_user_text (
- fmt_to_string (&f->format, str), -1));
-
- }
-
- pivot_table_submit (table);
-}
-
-/* Displays a table giving information on free-format variable parsing
- on DATA LIST. */
-static void
-dump_delimited_table (const struct data_parser *parser,
- const struct file_handle *fh)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_text_format (N_("Reading free-form data from %s."),
- fh_get_name (fh)),
- "Free-Form Data Records");
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Attributes"), N_("Format"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
- variables->root->show_label = true;
- for (size_t i = 0; i < parser->n_fields; i++)
- {
- struct field *f = &parser->fields[i];
-
- /* XXX It would be better to have the actual variable here. */
- int variable_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_user_text (f->name, -1));
-
- char str[FMT_STRING_LEN_MAX + 1];
- pivot_table_put2 (table, 0, variable_idx,
- pivot_value_new_user_text (
- fmt_to_string (&f->format, str), -1));
- }
-
- pivot_table_submit (table);
-}
-
-/* Displays a table giving information on how PARSER will read
- data from FH. */
-void
-data_parser_output_description (struct data_parser *parser,
- const struct file_handle *fh)
-{
- if (parser->type == DP_FIXED)
- dump_fixed_table (parser, fh);
- else
- dump_delimited_table (parser, fh);
-}
-\f
-/* Data parser input program. */
-struct data_parser_casereader
- {
- struct data_parser *parser; /* Parser. */
- struct dictionary *dict; /* Dictionary. */
- struct dfm_reader *reader; /* Data file reader. */
- struct caseproto *proto; /* Format of cases. */
- };
-
-static const struct casereader_class data_parser_casereader_class;
-
-/* Replaces DS's active dataset by an input program that reads data
- from READER according to the rules in PARSER, using DICT as
- the underlying dictionary. Ownership of PARSER and READER is
- transferred to the input program, and ownership of DICT is
- transferred to the dataset. */
-void
-data_parser_make_active_file (struct data_parser *parser, struct dataset *ds,
- struct dfm_reader *reader,
- struct dictionary *dict,
- struct casereader* (*func)(struct casereader *,
- const struct dictionary *,
- void *),
- void *ud)
-{
- struct data_parser_casereader *r;
- struct casereader *casereader0;
- struct casereader *casereader1;
-
- r = xmalloc (sizeof *r);
- r->parser = parser;
- r->dict = dict_ref (dict);
- r->reader = reader;
- r->proto = caseproto_ref (dict_get_proto (dict));
- casereader0 = casereader_create_sequential (NULL, r->proto,
- CASENUMBER_MAX,
- &data_parser_casereader_class, r);
-
- if (func)
- casereader1 = func (casereader0, dict, ud);
- else
- casereader1 = casereader0;
-
- dataset_set_dict (ds, dict);
- dataset_set_source (ds, casereader1);
-}
-
-
-static struct ccase *
-data_parser_casereader_read (struct casereader *reader UNUSED, void *r_)
-{
- struct data_parser_casereader *r = r_;
- struct ccase *c = case_create (r->proto);
- if (data_parser_parse (r->parser, r->reader, r->dict, c))
- return c;
- else
- {
- case_unref (c);
- return NULL;
- }
-}
-
-static void
-data_parser_casereader_destroy (struct casereader *reader, void *r_)
-{
- struct data_parser_casereader *r = r_;
- if (dfm_reader_error (r->reader))
- casereader_force_error (reader);
- dfm_close_reader (r->reader);
- caseproto_unref (r->proto);
- dict_unref (r->dict);
- data_parser_destroy (r->parser);
- free (r);
-}
-
-static const struct casereader_class data_parser_casereader_class =
- {
- data_parser_casereader_read,
- data_parser_casereader_destroy,
- NULL,
- NULL,
- };
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2007, 2011, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef LANGUAGE_DATA_IO_DATA_PARSER_H
-#define LANGUAGE_DATA_IO_DATA_PARSER_H
-
-/* Abstraction of a DATA LIST or GET DATA TYPE=TXT data parser. */
-
-#include <stdbool.h>
-#include "data/case.h"
-#include "libpspp/str.h"
-
-struct dataset;
-struct dfm_reader;
-struct dictionary;
-struct file_handle;
-struct fmt_spec;
-struct string_array;
-struct substring;
-
-/* Type of data read by a data parser. */
-enum data_parser_type
- {
- DP_FIXED, /* Fields in fixed column positions. */
- DP_DELIMITED /* Fields delimited by e.g. commas. */
- };
-
-/* Creating and configuring any parser. */
-struct data_parser *data_parser_create (void);
-void data_parser_destroy (struct data_parser *);
-
-enum data_parser_type data_parser_get_type (const struct data_parser *);
-void data_parser_set_type (struct data_parser *, enum data_parser_type);
-
-void data_parser_set_skip (struct data_parser *, int initial_records_to_skip);
-
-/* For configuring delimited parsers only. */
-bool data_parser_get_span (const struct data_parser *);
-void data_parser_set_span (struct data_parser *, bool may_cases_span_records);
-
-void data_parser_set_empty_line_has_field (struct data_parser *,
- bool empty_line_has_field);
-void data_parser_set_warn_missing_fields (struct data_parser *parser,
- bool warn_missing_fields);
-
-void data_parser_set_quotes (struct data_parser *, struct substring);
-void data_parser_set_quote_escape (struct data_parser *, bool escape);
-void data_parser_set_soft_delimiters (struct data_parser *, struct substring);
-void data_parser_set_hard_delimiters (struct data_parser *, struct substring);
-
-/* For configuring fixed parsers only. */
-int data_parser_get_records (const struct data_parser *);
-void data_parser_set_records (struct data_parser *, int records_per_case);
-
-/* Field setup and parsing. */
-void data_parser_add_delimited_field (struct data_parser *,
- const struct fmt_spec *, int fv,
- const char *name);
-void data_parser_add_fixed_field (struct data_parser *,
- const struct fmt_spec *, int fv,
- const char *name,
- int record, int first_column);
-bool data_parser_any_fields (const struct data_parser *);
-bool data_parser_parse (struct data_parser *, struct dfm_reader *,
- struct dictionary *, struct ccase *);
-size_t data_parser_split (const struct data_parser *, struct substring line,
- struct string_array *);
-
-/* Uses for a configured parser. */
-void data_parser_output_description (struct data_parser *,
- const struct file_handle *);
-struct casereader;
-void data_parser_make_active_file (struct data_parser *, struct dataset *,
- struct dfm_reader *, struct dictionary *,
- struct casereader* (*func)(struct casereader *,
- const struct dictionary *,
- void *),
- void *ud);
-
-
-#endif /* language/data-io/data-parser.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-2004, 2006, 2010, 2011, 2012, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/data-io/data-reader.h"
-
-#include <ctype.h>
-#include <errno.h>
-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/file-handle-def.h"
-#include "data/file-name.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/encoding-guesser.h"
-#include "libpspp/integer-format.h"
-#include "libpspp/line-reader.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-/* Flags for DFM readers. */
-enum dfm_reader_flags
- {
- DFM_ADVANCE = 002, /* Read next line on dfm_get_record() call? */
- DFM_SAW_BEGIN_DATA = 004, /* For inline_file only, whether we've
- already read a BEGIN DATA line. */
- DFM_TABS_EXPANDED = 010, /* Tabs have been expanded. */
- DFM_CONSUME = 020 /* read_inline_record() should get a token? */
- };
-
-/* Data file reader. */
-struct dfm_reader
- {
- struct file_handle *fh; /* File handle. */
- struct fh_lock *lock; /* Mutual exclusion lock for file. */
- int line_number; /* Current line or record number. */
- struct string line; /* Current line. */
- struct string scratch; /* Extra line buffer. */
- enum dfm_reader_flags flags; /* Zero or more of DFM_*. */
- FILE *file; /* Associated file. */
- size_t pos; /* Offset in line of current character. */
- unsigned n_eofs; /* # of attempts to advance past EOF. */
- struct lexer *lexer; /* The lexer reading the file */
- char *encoding; /* Current encoding. */
-
- /* For FH_MODE_TEXT only. */
- struct line_reader *line_reader;
-
- /* For FH_MODE_360_VARIABLE and FH_MODE_360_SPANNED files only. */
- size_t block_left; /* Bytes left in current block. */
- };
-
-/* Closes reader R opened by dfm_open_reader(). */
-void
-dfm_close_reader (struct dfm_reader *r)
-{
- if (r == NULL)
- return;
-
- if (fh_unlock (r->lock))
- {
- /* File is still locked by another client. */
- return;
- }
-
- /* This was the last client, so close the underlying file. */
- if (fh_get_referent (r->fh) != FH_REF_INLINE)
- fn_close (r->fh, r->file);
- else
- {
- /* Skip any remaining data on the inline file. */
- if (r->flags & DFM_SAW_BEGIN_DATA)
- {
- dfm_reread_record (r, 0);
- while (!dfm_eof (r))
- dfm_forward_record (r);
- }
- }
-
- line_reader_free (r->line_reader);
- free (r->encoding);
- fh_unref (r->fh);
- ds_destroy (&r->line);
- ds_destroy (&r->scratch);
- free (r);
-}
-
-/* Opens the file designated by file handle FH for reading as a data file.
- Returns a reader if successful, or a null pointer otherwise.
-
- If FH is fh_inline_file() then the new reader reads data included inline in
- the command file between BEGIN FILE and END FILE, obtaining data from LEXER.
- LEXER must remain valid as long as the new reader is in use. ENCODING is
- ignored.
-
- If FH is not fh_inline_file(), then the encoding of the file read is by
- default that of FH itself. If ENCODING is nonnull, then it overrides the
- default encoding. LEXER is ignored. */
-struct dfm_reader *
-dfm_open_reader (struct file_handle *fh, struct lexer *lexer,
- const char *encoding)
-{
- struct dfm_reader *r;
- struct fh_lock *lock;
-
- /* TRANSLATORS: this fragment will be interpolated into
- messages in fh_lock() that identify types of files. */
- lock = fh_lock (fh, FH_REF_FILE | FH_REF_INLINE, N_("data file"),
- FH_ACC_READ, false);
- if (lock == NULL)
- return NULL;
-
- r = fh_lock_get_aux (lock);
- if (r != NULL)
- return r;
-
- r = xmalloc (sizeof *r);
- r->fh = fh_ref (fh);
- r->lock = lock;
- r->lexer = lexer;
- ds_init_empty (&r->line);
- ds_init_empty (&r->scratch);
- r->flags = DFM_ADVANCE;
- r->n_eofs = 0;
- r->block_left = 0;
- if (fh_get_referent (fh) != FH_REF_INLINE)
- {
- r->line_number = 0;
- r->file = fn_open (fh, "rb");
- if (r->file == NULL)
- {
- msg (ME, _("Could not open `%s' for reading as a data file: %s."),
- fh_get_file_name (r->fh), strerror (errno));
- goto error;
- }
- }
- fh_lock_set_aux (lock, r);
-
- if (encoding == NULL)
- encoding = fh_get_encoding (fh);
- if (fh_get_referent (fh) == FH_REF_FILE && fh_get_mode (fh) == FH_MODE_TEXT)
- {
- r->line_reader = line_reader_for_fd (encoding, fileno (r->file));
- if (r->line_reader == NULL)
- {
- msg (ME, _("Could not read `%s' as a text file with encoding `%s': "
- "%s."),
- fh_get_file_name (r->fh), encoding, strerror (errno));
- goto error;
- }
- r->encoding = xstrdup (line_reader_get_encoding (r->line_reader));
- }
- else
- {
- r->line_reader = NULL;
- r->encoding = xstrdup (encoding_guess_parse_encoding (encoding));
- }
-
- return r;
-
-error:
- fh_unlock (r->lock);
- fh_unref (fh);
- free (r);
- return NULL;
-}
-
-/* Returns true if an I/O error occurred on READER, false otherwise. */
-bool
-dfm_reader_error (const struct dfm_reader *r)
-{
- return (fh_get_referent (r->fh) == FH_REF_FILE
- && (r->line_reader != NULL
- ? line_reader_error (r->line_reader) != 0
- : ferror (r->file)));
-}
-
-/* Reads a record from the inline file into R.
- Returns true if successful, false on failure. */
-static bool
-read_inline_record (struct dfm_reader *r)
-{
- if ((r->flags & DFM_SAW_BEGIN_DATA) == 0)
- {
- r->flags |= DFM_SAW_BEGIN_DATA;
- r->flags &= ~DFM_CONSUME;
-
- while (lex_token (r->lexer) == T_ENDCMD)
- lex_get (r->lexer);
-
- if (!lex_force_match_phrase (r->lexer, "BEGIN DATA"))
- return false;
-
- lex_match (r->lexer, T_ENDCMD);
- }
-
- if (r->flags & DFM_CONSUME)
- lex_get (r->lexer);
-
- if (!lex_is_string (r->lexer))
- {
- if (!lex_match_id (r->lexer, "END") || !lex_match_id (r->lexer, "DATA"))
- {
- msg (SE, _("Missing %s while reading inline data. "
- "This probably indicates a missing or incorrectly "
- "formatted %s command. %s must appear "
- "by itself on a single line with exactly one space "
- "between words."), "END DATA", "END DATA", "END DATA");
- lex_discard_rest_of_command (r->lexer);
- }
- return false;
- }
-
- ds_assign_substring (&r->line, lex_tokss (r->lexer));
- r->flags |= DFM_CONSUME;
-
- return true;
-}
-
-/* Report a read error on R. */
-static void
-read_error (struct dfm_reader *r)
-{
- msg (ME, _("Error reading file %s: %s."),
- fh_get_name (r->fh), strerror (errno));
-}
-
-/* Report a partial read at end of file reading R. */
-static void
-partial_record (struct dfm_reader *r)
-{
- msg (ME, _("Unexpected end of file in partial record reading %s."),
- fh_get_name (r->fh));
-}
-
-/* Tries to read SIZE bytes from R into BUFFER. Returns 1 if
- successful, 0 if end of file was reached before any bytes
- could be read, and -1 if some bytes were read but fewer than
- SIZE due to end of file or an error mid-read. In the latter
- case, reports an error. */
-static int
-try_to_read_fully (struct dfm_reader *r, void *buffer, size_t size)
-{
- size_t bytes_read = fread (buffer, 1, size, r->file);
- if (bytes_read == size)
- return 1;
- else if (bytes_read == 0)
- return 0;
- else
- {
- partial_record (r);
- return -1;
- }
-}
-
-/* Type of a descriptor word. */
-enum descriptor_type
- {
- BLOCK,
- RECORD
- };
-
-/* Reads a block descriptor word or record descriptor word
- (according to TYPE) from R. Returns 1 if successful, 0 if
- end of file was reached before any bytes could be read, -1 if
- an error occurred. Reports an error in the latter case.
-
- If successful, stores the number of remaining bytes in the
- block or record (that is, the block or record length, minus
- the 4 bytes in the BDW or RDW itself) into *REMAINING_SIZE.
- If SEGMENT is nonnull, also stores the segment control
- character (SCC) into *SEGMENT. */
-static int
-read_descriptor_word (struct dfm_reader *r, enum descriptor_type type,
- size_t *remaining_size, int *segment)
-{
- uint8_t raw_descriptor[4];
- int status;
-
- status = try_to_read_fully (r, raw_descriptor, sizeof raw_descriptor);
- if (status <= 0)
- return status;
-
- *remaining_size = (raw_descriptor[0] << 8) | raw_descriptor[1];
- if (segment != NULL)
- *segment = raw_descriptor[2];
-
- if (*remaining_size < 4)
- {
- msg (ME,
- (type == BLOCK
- ? _("Corrupt block descriptor word at offset 0x%lx in %s.")
- : _("Corrupt record descriptor word at offset 0x%lx in %s.")),
- (long) ftello (r->file) - 4, fh_get_name (r->fh));
- return -1;
- }
-
- *remaining_size -= 4;
- return 1;
-}
-
-/* Reports that reader R has read a corrupt record size. */
-static void
-corrupt_size (struct dfm_reader *r)
-{
- msg (ME, _("Corrupt record size at offset 0x%lx in %s."),
- (long) ftello (r->file) - 4, fh_get_name (r->fh));
-}
-
-/* Reads a 32-byte little-endian signed number from R and stores
- its value into *SIZE_OUT. Returns 1 if successful, 0 if end
- of file was reached before any bytes could be read, -1 if an
- error occurred. Reports an error in the latter case. Numbers
- less than 0 are considered errors. */
-static int
-read_size (struct dfm_reader *r, size_t *size_out)
-{
- int32_t size;
- int status;
-
- status = try_to_read_fully (r, &size, sizeof size);
- if (status <= 0)
- return status;
-
- integer_convert (INTEGER_LSB_FIRST, &size, INTEGER_NATIVE, &size,
- sizeof size);
- if (size < 0)
- {
- corrupt_size (r);
- return -1;
- }
-
- *size_out = size;
- return 1;
-}
-
-static bool
-read_text_record (struct dfm_reader *r)
-{
- bool is_auto;
- bool ok;
-
- /* Read a line. If the line reader's encoding changes, update r->encoding to
- match. */
- is_auto = line_reader_is_auto (r->line_reader);
- ok = line_reader_read (r->line_reader, &r->line, SIZE_MAX);
- if (is_auto && !line_reader_is_auto (r->line_reader))
- {
- free (r->encoding);
- r->encoding = xstrdup (line_reader_get_encoding (r->line_reader));
- }
-
- /* Detect and report read error. */
- if (!ok)
- {
- int error = line_reader_error (r->line_reader);
- if (error != 0)
- msg (ME, _("Error reading file %s: %s."),
- fh_get_name (r->fh), strerror (error));
- }
-
- return ok;
-}
-
-/* Reads a record from a disk file into R.
- Returns true if successful, false on error or at end of file. */
-static bool
-read_file_record (struct dfm_reader *r)
-{
- assert (r->fh != fh_inline_file ());
-
- ds_clear (&r->line);
- switch (fh_get_mode (r->fh))
- {
- case FH_MODE_TEXT:
- return read_text_record (r);
-
- case FH_MODE_FIXED:
- if (ds_read_stream (&r->line, 1, fh_get_record_width (r->fh), r->file))
- return true;
- else
- {
- if (ferror (r->file))
- read_error (r);
- else if (!ds_is_empty (&r->line))
- partial_record (r);
- return false;
- }
-
- case FH_MODE_VARIABLE:
- {
- size_t leading_size;
- size_t trailing_size;
- int status;
-
- /* Read leading record size. */
- status = read_size (r, &leading_size);
- if (status <= 0)
- return false;
-
- /* Read record data. */
- if (!ds_read_stream (&r->line, leading_size, 1, r->file))
- {
- if (ferror (r->file))
- read_error (r);
- else
- partial_record (r);
- return false;
- }
-
- /* Read trailing record size and check that it's the same
- as the leading record size. */
- status = read_size (r, &trailing_size);
- if (status <= 0)
- {
- if (status == 0)
- partial_record (r);
- return false;
- }
- if (leading_size != trailing_size)
- {
- corrupt_size (r);
- return false;
- }
-
- return true;
- }
-
- case FH_MODE_360_VARIABLE:
- case FH_MODE_360_SPANNED:
- for (;;)
- {
- size_t record_size;
- int segment;
- int status;
-
- /* If we've exhausted our current block, start another
- one by reading the new block descriptor word. */
- if (r->block_left == 0)
- {
- status = read_descriptor_word (r, BLOCK, &r->block_left, NULL);
- if (status < 0)
- return false;
- else if (status == 0)
- return !ds_is_empty (&r->line);
- }
-
- /* Read record descriptor. */
- if (r->block_left < 4)
- {
- partial_record (r);
- return false;
- }
- r->block_left -= 4;
- status = read_descriptor_word (r, RECORD, &record_size, &segment);
- if (status <= 0)
- {
- if (status == 0)
- partial_record (r);
- return false;
- }
- if (record_size > r->block_left)
- {
- msg (ME, _("Record exceeds remaining block length."));
- return false;
- }
-
- /* Read record data. */
- if (!ds_read_stream (&r->line, record_size, 1, r->file))
- {
- if (ferror (r->file))
- read_error (r);
- else
- partial_record (r);
- return false;
- }
- r->block_left -= record_size;
-
- /* In variable mode, read only a single record.
- In spanned mode, a segment value of 0 should
- designate a whole record without spanning, 1 the
- first segment in a record, 2 the last segment in a
- record, and 3 an intermediate segment in a record.
- For compatibility, though, we actually pay attention
- only to whether the segment value is even or odd. */
- if (fh_get_mode (r->fh) == FH_MODE_360_VARIABLE
- || (segment & 1) == 0)
- return true;
- }
- }
-
- NOT_REACHED ();
-}
-
-/* Reads a record from R, setting the current position to the
- start of the line. If an error occurs or end-of-file is
- encountered, the current line is set to null. */
-static bool
-read_record (struct dfm_reader *r)
-{
- if (fh_get_referent (r->fh) == FH_REF_FILE)
- {
- bool ok = read_file_record (r);
- if (ok)
- r->line_number++;
- return ok;
- }
- else
- return read_inline_record (r);
-}
-
-/* Returns the number of attempts, thus far, to advance past
- end-of-file in reader R. Reads forward in HANDLE's file, if
- necessary, to find out.
-
- Normally, the user stops attempting to read from the file the
- first time EOF is reached (a return value of 1). If the user
- tries to read past EOF again (a return value of 2 or more),
- an error message is issued, and the caller should more
- forcibly abort to avoid an infinite loop. */
-unsigned
-dfm_eof (struct dfm_reader *r)
-{
- if (r->flags & DFM_ADVANCE)
- {
- r->flags &= ~DFM_ADVANCE;
-
- if (r->n_eofs == 0 && read_record (r))
- {
- r->pos = 0;
- return 0;
- }
-
- r->n_eofs++;
- if (r->n_eofs == 2)
- {
- if (r->fh != fh_inline_file ())
- msg (ME, _("Attempt to read beyond end-of-file on file %s."),
- fh_get_name (r->fh));
- else
- msg (ME, _("Attempt to read beyond %s."), "END DATA");
- }
- }
-
- return r->n_eofs;
-}
-
-/* Returns the current record in the file corresponding to
- HANDLE. Aborts if reading from the file is necessary or at
- end of file, so call dfm_eof() first. */
-struct substring
-dfm_get_record (struct dfm_reader *r)
-{
- assert ((r->flags & DFM_ADVANCE) == 0);
- assert (r->n_eofs == 0);
-
- return ds_substr (&r->line, r->pos, SIZE_MAX);
-}
-
-/* Expands tabs in the current line into the equivalent number of
- spaces, if appropriate for this kind of file. Aborts if
- reading from the file is necessary or at end of file, so call
- dfm_eof() first.*/
-void
-dfm_expand_tabs (struct dfm_reader *r)
-{
- size_t ofs, new_pos, tab_width;
-
- assert ((r->flags & DFM_ADVANCE) == 0);
- assert (r->n_eofs == 0);
-
- if (r->flags & DFM_TABS_EXPANDED)
- return;
- r->flags |= DFM_TABS_EXPANDED;
-
- if (r->fh != fh_inline_file ()
- && (fh_get_mode (r->fh) != FH_MODE_TEXT
- || fh_get_tab_width (r->fh) == 0
- || ds_find_byte (&r->line, '\t') == SIZE_MAX))
- return;
-
- /* Expand tabs from r->line into r->scratch, and figure out
- new value for r->pos. */
- tab_width = fh_get_tab_width (r->fh);
- ds_clear (&r->scratch);
- new_pos = SIZE_MAX;
- for (ofs = 0; ofs < ds_length (&r->line); ofs++)
- {
- unsigned char c;
-
- if (ofs == r->pos)
- new_pos = ds_length (&r->scratch);
-
- c = ds_data (&r->line)[ofs];
- if (c != '\t')
- ds_put_byte (&r->scratch, c);
- else
- {
- do
- ds_put_byte (&r->scratch, ' ');
- while (ds_length (&r->scratch) % tab_width != 0);
- }
- }
- if (new_pos == SIZE_MAX)
- {
- /* Maintain the same relationship between position and line
- length that we had before. DATA LIST uses a
- beyond-the-end position to deal with an empty field at
- the end of the line. */
- assert (r->pos >= ds_length (&r->line));
- new_pos = (r->pos - ds_length (&r->line)) + ds_length (&r->scratch);
- }
-
- /* Swap r->line and r->scratch and set new r->pos. */
- ds_swap (&r->line, &r->scratch);
- r->pos = new_pos;
-}
-
-/* Returns the character encoding of data read from READER. */
-const char *
-dfm_reader_get_encoding (const struct dfm_reader *reader)
-{
- return reader->encoding;
-}
-
-/* Causes dfm_get_record() or dfm_get_whole_record() to read in
- the next record the next time it is executed on file
- HANDLE. */
-void
-dfm_forward_record (struct dfm_reader *r)
-{
- r->flags |= DFM_ADVANCE;
-}
-
-/* Cancels the effect of any previous dfm_fwd_record() executed
- on file HANDLE. Sets the current line to begin in the 1-based
- column COLUMN. */
-void
-dfm_reread_record (struct dfm_reader *r, size_t column)
-{
- r->flags &= ~DFM_ADVANCE;
- r->pos = MAX (column, 1) - 1;
-}
-
-/* Sets the current line to begin COLUMNS characters following
- the current start. */
-void
-dfm_forward_columns (struct dfm_reader *r, size_t columns)
-{
- dfm_reread_record (r, (r->pos + 1) + columns);
-}
-
-/* Returns the 1-based column to which the line pointer in HANDLE
- is set. Unless dfm_reread_record() or dfm_forward_columns()
- have been called, this is 1. */
-size_t
-dfm_column_start (const struct dfm_reader *r)
-{
- return r->pos + 1;
-}
-
-/* Returns the number of columns we are currently beyond the end
- of the line. At or before end-of-line, this is 0; one column
- after end-of-line, this is 1; and so on. */
-size_t
-dfm_columns_past_end (const struct dfm_reader *r)
-{
- return r->pos < ds_length (&r->line) ? 0 : ds_length (&r->line) - r->pos;
-}
-
-/* Returns the 1-based column within the current line that P
- designates. */
-size_t
-dfm_get_column (const struct dfm_reader *r, const char *p)
-{
- return ds_pointer_to_position (&r->line, p) + 1;
-}
-
-const char *
-dfm_get_file_name (const struct dfm_reader *r)
-{
- enum fh_referent referent = fh_get_referent (r->fh);
- return (referent == FH_REF_FILE ? fh_get_file_name (r->fh)
- : referent == FH_REF_INLINE ? lex_get_file_name (r->lexer)
- : NULL);
-}
-
-int
-dfm_get_line_number (const struct dfm_reader *r)
-{
- switch (fh_get_referent (r->fh))
- {
- case FH_REF_FILE:
- return r->line_number;
-
- case FH_REF_INLINE:
- return lex_ofs_start_point (r->lexer, lex_ofs (r->lexer)).line;
-
- case FH_REF_DATASET:
- default:
- return -1;
- }
-}
-\f
-/* BEGIN DATA...END DATA procedure. */
-
-/* Perform BEGIN DATA...END DATA as a procedure in itself. */
-int
-cmd_begin_data (struct lexer *lexer, struct dataset *ds)
-{
- struct dfm_reader *r;
- bool ok;
-
- if (!fh_is_locked (fh_inline_file (), FH_ACC_READ))
- {
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("This command is not valid here since the current "
- "input program does not access the inline file."));
- return CMD_CASCADING_FAILURE;
- }
- lex_match (lexer, T_ENDCMD);
-
- /* Open inline file. */
- r = dfm_open_reader (fh_inline_file (), lexer, NULL);
- r->flags |= DFM_SAW_BEGIN_DATA;
- r->flags &= ~DFM_CONSUME;
-
- /* Input procedure reads from inline file. */
- casereader_destroy (proc_open (ds));
- ok = proc_commit (ds);
- dfm_close_reader (r);
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011, 2012, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef DFM_READ_H
-#define DFM_READ_H
-
-/* Data file manager (dfm).
-
- This module is in charge of reading and writing data files (other
- than system files). dfm is an fhuser, so see file-handle.h for the
- fhuser interface. */
-
-#include <stdbool.h>
-#include <stddef.h>
-
-struct file_handle;
-struct string;
-struct lexer;
-
-/* Input. */
-struct dfm_reader *dfm_open_reader (struct file_handle *, struct lexer *,
- const char *encoding);
-void dfm_close_reader (struct dfm_reader *);
-bool dfm_reader_error (const struct dfm_reader *);
-unsigned dfm_eof (struct dfm_reader *);
-struct substring dfm_get_record (struct dfm_reader *);
-void dfm_expand_tabs (struct dfm_reader *);
-const char *dfm_reader_get_encoding (const struct dfm_reader *);
-
-/* Line control. */
-void dfm_forward_record (struct dfm_reader *);
-void dfm_reread_record (struct dfm_reader *, size_t column);
-void dfm_forward_columns (struct dfm_reader *, size_t columns);
-size_t dfm_column_start (const struct dfm_reader *);
-size_t dfm_columns_past_end (const struct dfm_reader *);
-size_t dfm_get_column (const struct dfm_reader *, const char *);
-
-/* Information. */
-const char *dfm_get_file_name (const struct dfm_reader *);
-int dfm_get_line_number (const struct dfm_reader *);
-
-#endif /* data-reader.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-2004, 2006, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/data-io/data-writer.h"
-
-#include <assert.h>
-#include <errno.h>
-#include <stdint.h>
-#include <stdlib.h>
-#include <sys/stat.h>
-
-#include "data/file-name.h"
-#include "data/make-file.h"
-#include "language/data-io/file-handle.h"
-#include "libpspp/assertion.h"
-#include "libpspp/encoding-guesser.h"
-#include "libpspp/integer-format.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-/* Data file writer. */
-struct dfm_writer
- {
- struct file_handle *fh; /* File handle. */
- struct fh_lock *lock; /* Exclusive access to file. */
- FILE *file; /* Associated file. */
- struct replace_file *rf; /* Atomic file replacement support. */
- char *encoding; /* Encoding. */
- enum fh_line_ends line_ends; /* Line ends for text files. */
-
- int unit; /* Unit width, in bytes. */
- char cr[MAX_UNIT]; /* \r in encoding, 'unit' bytes long. */
- char lf[MAX_UNIT]; /* \n in encoding, 'unit' bytes long. */
- char spaces[32]; /* 32 bytes worth of ' ' in encoding. */
- };
-
-/* Opens a file handle for writing as a data file.
-
- The encoding of the file written is by default that of FH itself. If
- ENCODING is nonnull, then it overrides the default encoding.
-
- *However*: ENCODING directly affects only text strings written by the data
- writer code itself, that is, new-lines in FH_MODE_TEXT and space padding in
- FH_MODE_FIXED mode. The client must do its own encoding translation for the
- data that it writes. (This is unavoidable because sometimes the data
- written includes binary data that reencoding would mangle.) The client can
- obtain the encoding to re-encode into with dfm_writer_get_encoding(). */
-struct dfm_writer *
-dfm_open_writer (struct file_handle *fh, const char *encoding)
-{
- struct encoding_info ei;
- struct dfm_writer *w;
- struct fh_lock *lock;
- int ofs;
-
- lock = fh_lock (fh, FH_REF_FILE, N_("data file"), FH_ACC_WRITE, false);
- if (lock == NULL)
- return NULL;
-
- w = fh_lock_get_aux (lock);
- if (w != NULL)
- return w;
-
- encoding = encoding_guess_parse_encoding (encoding != NULL
- ? encoding
- : fh_get_encoding (fh));
- get_encoding_info (&ei, encoding);
-
- w = xmalloc (sizeof *w);
- w->fh = fh_ref (fh);
- w->lock = lock;
- w->rf = replace_file_start (w->fh, "wb", 0666, &w->file);
- w->encoding = xstrdup (encoding);
- w->line_ends = fh_get_line_ends (fh);
- w->unit = ei.unit;
- memcpy (w->cr, ei.cr, sizeof w->cr);
- memcpy (w->lf, ei.lf, sizeof w->lf);
- for (ofs = 0; ofs + ei.unit <= sizeof w->spaces; ofs += ei.unit)
- memcpy (&w->spaces[ofs], ei.space, ei.unit);
-
- if (w->rf == NULL)
- {
- msg (ME, _("An error occurred while opening `%s' for writing "
- "as a data file: %s."),
- fh_get_file_name (w->fh), strerror (errno));
- dfm_close_writer (w);
- return NULL;
- }
- fh_lock_set_aux (lock, w);
-
- return w;
-}
-
-/* Returns true if an I/O error occurred on WRITER, false otherwise. */
-bool
-dfm_write_error (const struct dfm_writer *writer)
-{
- return ferror (writer->file);
-}
-
-/* Writes record REC (which need not be null-terminated) having
- length LEN to the file corresponding to HANDLE. Adds any
- needed formatting, such as a trailing new-line. Returns true
- on success, false on failure. */
-bool
-dfm_put_record (struct dfm_writer *w, const char *rec, size_t len)
-{
- assert (w != NULL);
-
- if (dfm_write_error (w))
- return false;
-
- switch (fh_get_mode (w->fh))
- {
- case FH_MODE_TEXT:
- fwrite (rec, len, 1, w->file);
- if (w->line_ends == FH_END_CRLF)
- fwrite (w->cr, w->unit, 1, w->file);
- fwrite (w->lf, w->unit, 1, w->file);
- break;
-
- case FH_MODE_FIXED:
- {
- size_t record_width = fh_get_record_width (w->fh);
- size_t write_bytes = MIN (len, record_width);
- size_t pad_bytes = record_width - write_bytes;
- fwrite (rec, write_bytes, 1, w->file);
- while (pad_bytes > 0)
- {
- size_t chunk = MIN (pad_bytes, sizeof w->spaces);
- fwrite (w->spaces, chunk, 1, w->file);
- pad_bytes -= chunk;
- }
- }
- break;
-
- case FH_MODE_VARIABLE:
- {
- uint32_t size = len;
- integer_convert (INTEGER_NATIVE, &size, INTEGER_LSB_FIRST, &size,
- sizeof size);
- fwrite (&size, sizeof size, 1, w->file);
- fwrite (rec, len, 1, w->file);
- fwrite (&size, sizeof size, 1, w->file);
- }
- break;
-
- case FH_MODE_360_VARIABLE:
- case FH_MODE_360_SPANNED:
- {
- size_t ofs = 0;
- if (fh_get_mode (w->fh) == FH_MODE_360_VARIABLE)
- len = MIN (65527, len);
- while (ofs < len)
- {
- size_t chunk = MIN (65527, len - ofs);
- uint32_t bdw = (chunk + 8) << 16;
- int scc = (ofs == 0 && chunk == len ? 0
- : ofs == 0 ? 1
- : ofs + chunk == len ? 2
- : 3);
- uint32_t rdw = ((chunk + 4) << 16) | (scc << 8);
-
- integer_convert (INTEGER_NATIVE, &bdw, INTEGER_MSB_FIRST, &bdw,
- sizeof bdw);
- integer_convert (INTEGER_NATIVE, &rdw, INTEGER_MSB_FIRST, &rdw,
- sizeof rdw);
- fwrite (&bdw, 1, sizeof bdw, w->file);
- fwrite (&rdw, 1, sizeof rdw, w->file);
- fwrite (rec + ofs, 1, chunk, w->file);
- ofs += chunk;
- }
- }
- break;
-
- default:
- NOT_REACHED ();
- }
-
- return !dfm_write_error (w);
-}
-
-/* Writes record REC (which need not be null-terminated) having length LEN to
- the file corresponding to HANDLE. REC is encoded in UTF-8, which this
- function recodes to the correct encoding for W before writing. Adds any
- needed formatting, such as a trailing new-line. Returns true on success,
- false on failure. */
-bool
-dfm_put_record_utf8 (struct dfm_writer *w, const char *rec, size_t len)
-{
- if (is_encoding_utf8 (w->encoding))
- return dfm_put_record (w, rec, len);
- else
- {
- char *recoded = recode_string (w->encoding, UTF8, rec, len);
- bool ok = dfm_put_record (w, recoded, strlen (recoded));
- free (recoded);
- return ok;
- }
-}
-
-/* Closes data file writer W. */
-bool
-dfm_close_writer (struct dfm_writer *w)
-{
- bool ok;
-
- if (w == NULL)
- return true;
- if (fh_unlock (w->lock))
- return true;
-
- ok = true;
- if (w->file != NULL)
- {
- const char *file_name = fh_get_file_name (w->fh);
- ok = !dfm_write_error (w) && !fn_close (w->fh, w->file);
-
- if (!ok)
- msg (ME, _("I/O error occurred writing data file `%s'."), file_name);
-
- if (ok ? !replace_file_commit (w->rf) : !replace_file_abort (w->rf))
- ok = false;
- }
- fh_unref (w->fh);
- free (w->encoding);
- free (w);
-
- return ok;
-}
-
-/* Returns the encoding of data written to WRITER. */
-const char *
-dfm_writer_get_encoding (const struct dfm_writer *writer)
-{
- return writer->encoding;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2011, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef DFM_WRITE_H
-#define DFM_WRITE_H
-
-/* Writing data files. */
-
-#include <stdbool.h>
-#include <stddef.h>
-
-struct file_handle;
-struct dfm_writer *dfm_open_writer (struct file_handle *,
- const char *encoding);
-bool dfm_close_writer (struct dfm_writer *);
-bool dfm_write_error (const struct dfm_writer *);
-bool dfm_put_record (struct dfm_writer *, const char *rec, size_t len);
-bool dfm_put_record_utf8 (struct dfm_writer *, const char *rec, size_t len);
-const char *dfm_writer_get_encoding (const struct dfm_writer *);
-
-#endif /* data-writer.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/command.h"
-
-#include "data/dataset.h"
-#include "data/session.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-\f
-static int
-parse_window (struct lexer *lexer, unsigned int allowed,
- enum dataset_display def)
-{
- if (!lex_match_id (lexer, "WINDOW"))
- return def;
- lex_match (lexer, T_EQUALS);
-
- if (allowed & (1 << DATASET_MINIMIZED) && lex_match_id (lexer, "MINIMIZED"))
- return DATASET_MINIMIZED;
- else if (allowed & (1 << DATASET_ASIS) && lex_match_id (lexer, "ASIS"))
- return DATASET_ASIS;
- else if (allowed & (1 << DATASET_FRONT) && lex_match_id (lexer, "FRONT"))
- return DATASET_FRONT;
- else if (allowed & (1 << DATASET_HIDDEN) && lex_match_id (lexer, "HIDDEN"))
- return DATASET_HIDDEN;
-
- const char *allowed_s[4];
- size_t n_allowed = 0;
- if (allowed & (1 << DATASET_MINIMIZED))
- allowed_s[n_allowed++] = "MINIMIZED";
- if (allowed & (1 << DATASET_ASIS))
- allowed_s[n_allowed++] = "ASIS";
- if (allowed & (1 << DATASET_FRONT))
- allowed_s[n_allowed++] = "FRONT";
- if (allowed & (1 << DATASET_HIDDEN))
- allowed_s[n_allowed++] = "HIDDEN";
- lex_error_expecting_array (lexer, allowed_s, n_allowed);
- return -1;
-}
-
-static struct dataset *
-parse_dataset_name (struct lexer *lexer, struct session *session)
-{
- if (!lex_force_id (lexer))
- return NULL;
-
- struct dataset *ds = session_lookup_dataset (session, lex_tokcstr (lexer));
- if (ds != NULL)
- lex_get (lexer);
- else
- lex_error (lexer, _("There is no dataset named %s."), lex_tokcstr (lexer));
- return ds;
-}
-
-int
-cmd_dataset_name (struct lexer *lexer, struct dataset *active)
-{
- if (!lex_force_id (lexer))
- return CMD_FAILURE;
- dataset_set_name (active, lex_tokcstr (lexer));
- lex_get (lexer);
-
- int display = parse_window (lexer, (1 << DATASET_ASIS) | (1 << DATASET_FRONT),
- DATASET_ASIS);
- if (display < 0)
- return CMD_FAILURE;
- else if (display != DATASET_ASIS)
- dataset_set_display (active, display);
-
- return CMD_SUCCESS;
-}
-
-int
-cmd_dataset_activate (struct lexer *lexer, struct dataset *active)
-{
- struct session *session = dataset_session (active);
- struct dataset *ds;
- int display;
-
- ds = parse_dataset_name (lexer, session);
- if (ds == NULL)
- return CMD_FAILURE;
-
- if (ds != active)
- {
- proc_execute (active);
- session_set_active_dataset (session, ds);
- if (dataset_name (active)[0] == '\0')
- dataset_destroy (active);
- return CMD_SUCCESS;
- }
-
- display = parse_window (lexer, (1 << DATASET_ASIS) | (1 << DATASET_FRONT),
- DATASET_ASIS);
- if (display < 0)
- return CMD_FAILURE;
- else if (display != DATASET_ASIS)
- dataset_set_display (ds, display);
-
- return CMD_SUCCESS;
-}
-
-int
-cmd_dataset_copy (struct lexer *lexer, struct dataset *old)
-{
- struct session *session = dataset_session (old);
-
- /* Parse the entire command first. proc_execute() can attempt to parse
- BEGIN DATA...END DATA and it will fail confusingly if we are in the
- middle of the command at the point. */
- if (!lex_force_id (lexer))
- return CMD_FAILURE;
- char *name = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
-
- int display = parse_window (lexer, ((1 << DATASET_MINIMIZED)
- | (1 << DATASET_HIDDEN)
- | (1 << DATASET_FRONT)),
- DATASET_MINIMIZED);
- if (display < 0)
- {
- free (name);
- return CMD_FAILURE;
- }
-
- struct dataset *new;
- if (session_lookup_dataset (session, name) == old)
- {
- new = old;
- dataset_set_name (old, "");
- }
- else
- {
- proc_execute (old);
- new = dataset_clone (old, name);
- }
- dataset_set_display (new, display);
-
- free (name);
- return CMD_SUCCESS;
-}
-
-int
-cmd_dataset_declare (struct lexer *lexer, struct dataset *ds)
-{
- struct session *session = dataset_session (ds);
-
- if (!lex_force_id (lexer))
- return CMD_FAILURE;
-
- struct dataset *new = session_lookup_dataset (session, lex_tokcstr (lexer));
- if (new == NULL)
- new = dataset_create (session, lex_tokcstr (lexer));
- lex_get (lexer);
-
- int display = parse_window (lexer, ((1 << DATASET_MINIMIZED)
- | (1 << DATASET_HIDDEN)
- | (1 << DATASET_FRONT)),
- DATASET_MINIMIZED);
- if (display < 0)
- return CMD_FAILURE;
- dataset_set_display (new, display);
-
- return CMD_SUCCESS;
-}
-
-static void
-dataset_close_cb (struct dataset *ds, void *session_)
-{
- struct session *session = session_;
-
- if (ds != session_active_dataset (session))
- dataset_destroy (ds);
-}
-
-int
-cmd_dataset_close (struct lexer *lexer, struct dataset *ds)
-{
- struct session *session = dataset_session (ds);
-
- if (lex_match (lexer, T_ALL))
- {
- session_for_each_dataset (session, dataset_close_cb, session);
- dataset_set_name (session_active_dataset (session), "");
- }
- else
- {
- if (!lex_match (lexer, T_ASTERISK))
- {
- ds = parse_dataset_name (lexer, session);
- if (ds == NULL)
- return CMD_FAILURE;
- }
-
- if (ds == session_active_dataset (session))
- dataset_set_name (ds, "");
- else
- dataset_destroy (ds);
- }
-
- return CMD_SUCCESS;
-}
-
-static void
-dataset_display_cb (struct dataset *ds, void *p_)
-{
- struct dataset ***p = p_;
- **p = ds;
- (*p)++;
-}
-
-static int
-sort_datasets (const void *a_, const void *b_)
-{
- struct dataset *const *a = a_;
- struct dataset *const *b = b_;
-
- return strcmp (dataset_name (*a), dataset_name (*b));
-}
-
-int
-cmd_dataset_display (struct lexer *lexer UNUSED, struct dataset *ds)
-{
- struct session *session = dataset_session (ds);
- size_t n = session_n_datasets (session);
- struct dataset **datasets = xmalloc (n * sizeof *datasets);
- struct dataset **p = datasets;
- session_for_each_dataset (session, dataset_display_cb, &p);
- qsort (datasets, n, sizeof *datasets, sort_datasets);
-
- struct pivot_table *table = pivot_table_create (N_("Datasets"));
-
- struct pivot_dimension *datasets_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dataset"));
- datasets_dim->hide_all_labels = true;
-
- for (size_t i = 0; i < n; i++)
- {
- struct dataset *ds = datasets[i];
- const char *name;
-
- name = dataset_name (ds);
- if (name[0] == '\0')
- name = _("unnamed dataset");
-
- char *text = (ds == session_active_dataset (session)
- ? xasprintf ("%s (%s)", name, _("active dataset"))
- : xstrdup (name));
-
- int dataset_idx = pivot_category_create_leaf (
- datasets_dim->root, pivot_value_new_integer (i));
-
- pivot_table_put1 (table, dataset_idx,
- pivot_value_new_user_text_nocopy (text));
- }
-
- free (datasets);
-
- pivot_table_submit (table);
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-2000, 2006, 2010-2013, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "data/file-handle-def.h"
-
-#include <limits.h>
-#include <errno.h>
-#include <stdlib.h>
-
-#include "data/file-name.h"
-#include "data/session.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_file_handle (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- enum cmd_result result = CMD_CASCADING_FAILURE;
- char *handle_name = NULL;
- char *file_name = NULL;
- int lrecl = 0;
- int tabwidth = -1;
- enum { MODE_DEFAULT, MODE_CHARACTER, MODE_BINARY, MODE_IMAGE, MODE_360 }
- mode = MODE_DEFAULT;
- int ends = -1;
- enum { RECFORM_FIXED = 1, RECFORM_VARIABLE, RECFORM_SPANNED } recform = 0;
- char *encoding = NULL;
-
- if (!lex_force_id (lexer))
- goto exit;
-
- handle_name = xstrdup (lex_tokcstr (lexer));
- struct file_handle *fh = fh_from_id (handle_name);
- if (fh)
- {
- fh_unref (fh);
- lex_error (lexer, _("File handle %s is already defined. "
- "Use %s before redefining a file handle."),
- handle_name, "CLOSE FILE HANDLE");
- goto exit;
- }
-
- lex_get (lexer);
- if (!lex_force_match (lexer, T_SLASH))
- goto exit;
-
- int mode_start = 0;
- int mode_end = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "NAME"))
- {
- if (file_name)
- {
- lex_sbc_only_once (lexer, "NAME");
- goto exit;
- }
-
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto exit;
- free (file_name);
- file_name = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "LRECL"))
- {
- if (lrecl)
- {
- lex_sbc_only_once (lexer, "LRECL");
- goto exit;
- }
-
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "LRECL", 1, (1UL << 31) - 1))
- goto exit;
- lrecl = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "TABWIDTH"))
- {
- if (tabwidth >= 0)
- {
- lex_sbc_only_once (lexer, "TABWIDTH");
- goto exit;
- }
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_int_range (lexer, "TABWIDTH", 1, INT_MAX))
- goto exit;
- tabwidth = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "MODE"))
- {
- if (mode != MODE_DEFAULT)
- {
- lex_sbc_only_once (lexer, "MODE");
- goto exit;
- }
- mode_start = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "CHARACTER"))
- mode = MODE_CHARACTER;
- else if (lex_match_id (lexer, "BINARY"))
- mode = MODE_BINARY;
- else if (lex_match_id (lexer, "IMAGE"))
- mode = MODE_IMAGE;
- else if (lex_match_int (lexer, 360))
- mode = MODE_360;
- else
- {
- lex_error_expecting (lexer, "CHARACTER", "BINARY",
- "IMAGE", "360");
- goto exit;
- }
- mode_end = lex_ofs (lexer) - 1;
- }
- else if (lex_match_id (lexer, "ENDS"))
- {
- if (ends >= 0)
- {
- lex_sbc_only_once (lexer, "ENDS");
- goto exit;
- }
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "LF"))
- ends = FH_END_LF;
- else if (lex_match_id (lexer, "CRLF"))
- ends = FH_END_CRLF;
- else
- {
- lex_error_expecting (lexer, "LF", "CRLF");
- goto exit;
- }
- }
- else if (lex_match_id (lexer, "RECFORM"))
- {
- if (recform)
- {
- lex_sbc_only_once (lexer, "RECFORM");
- goto exit;
- }
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "FIXED") || lex_match_id (lexer, "F"))
- recform = RECFORM_FIXED;
- else if (lex_match_id (lexer, "VARIABLE")
- || lex_match_id (lexer, "V"))
- recform = RECFORM_VARIABLE;
- else if (lex_match_id (lexer, "SPANNED")
- || lex_match_id (lexer, "VS"))
- recform = RECFORM_SPANNED;
- else
- {
- lex_error_expecting (lexer, "FIXED", "VARIABLE", "SPANNED");
- goto exit;
- }
- }
- else if (lex_match_id (lexer, "ENCODING"))
- {
- if (encoding)
- {
- lex_sbc_only_once (lexer, "ENCODING");
- goto exit;
- }
-
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto exit;
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- if (!lex_match (lexer, T_SLASH))
- break;
- }
-
- if (lex_end_of_command (lexer) != CMD_SUCCESS)
- goto exit;
-
- struct fh_properties properties = *fh_default_properties ();
- if (file_name == NULL)
- {
- lex_sbc_missing (lexer, "NAME");
- goto exit;
- }
-
- switch (mode)
- {
- case MODE_DEFAULT:
- case MODE_CHARACTER:
- properties.mode = FH_MODE_TEXT;
- if (tabwidth >= 0)
- properties.tab_width = tabwidth;
- if (ends)
- properties.line_ends = ends;
- break;
- case MODE_IMAGE:
- properties.mode = FH_MODE_FIXED;
- break;
- case MODE_BINARY:
- properties.mode = FH_MODE_VARIABLE;
- break;
- case MODE_360:
- properties.encoding = CONST_CAST (char *, "EBCDIC-US");
- if (recform == RECFORM_FIXED)
- properties.mode = FH_MODE_FIXED;
- else if (recform == RECFORM_VARIABLE)
- {
- properties.mode = FH_MODE_360_VARIABLE;
- properties.record_width = 8192;
- }
- else if (recform == RECFORM_SPANNED)
- {
- properties.mode = FH_MODE_360_SPANNED;
- properties.record_width = 8192;
- }
- else
- {
- lex_ofs_error (lexer, mode_start, mode_end,
- _("%s must be specified with %s."),
- "RECFORM", "MODE=360");
- goto exit;
- }
- break;
- default:
- NOT_REACHED ();
- }
-
- if (properties.mode == FH_MODE_FIXED || lrecl)
- {
- if (!lrecl)
- msg (SE, _("The specified file mode requires LRECL. "
- "Assuming %zu-character records."),
- properties.record_width);
- else
- properties.record_width = lrecl;
- }
-
- if (encoding)
- properties.encoding = encoding;
-
- fh_create_file (handle_name, file_name, lex_get_encoding (lexer),
- &properties);
-
- result = CMD_SUCCESS;
-
-exit:
- free (handle_name);
- free (file_name);
- free (encoding);
- return result;
-}
-
-int
-cmd_close_file_handle (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- struct file_handle *handle;
-
- if (!lex_force_id (lexer))
- return CMD_CASCADING_FAILURE;
- handle = fh_from_id (lex_tokcstr (lexer));
- if (handle == NULL)
- return CMD_CASCADING_FAILURE;
-
- fh_unname (handle);
- return CMD_SUCCESS;
-}
-
-/* Returns the name for REFERENT. */
-static const char *
-referent_name (enum fh_referent referent)
-{
- switch (referent)
- {
- case FH_REF_FILE:
- return _("file");
- case FH_REF_INLINE:
- return _("inline file");
- case FH_REF_DATASET:
- return _("dataset");
- default:
- NOT_REACHED ();
- }
-}
-
-/* Parses a file handle name:
-
- - If SESSION is nonnull, then the parsed syntax may be the name of a
- dataset within SESSION. Dataset names take precedence over file handle
- names.
-
- - If REFERENT_MASK includes FH_REF_FILE, the parsed syntax may be a file
- name as a string or a file handle name as an identifier.
-
- - If REFERENT_MASK includes FH_REF_INLINE, the parsed syntax may be the
- identifier INLINE to represent inline data.
-
- Returns the file handle when successful, a null pointer on failure.
-
- The caller is responsible for fh_unref()'ing the returned file handle when
- it is no longer needed. */
-struct file_handle *
-fh_parse (struct lexer *lexer, enum fh_referent referent_mask,
- struct session *session)
-{
- if (session != NULL && lex_token (lexer) == T_ID)
- {
- struct dataset *ds;
-
- ds = session_lookup_dataset (session, lex_tokcstr (lexer));
- if (ds != NULL)
- {
- lex_get (lexer);
- return fh_create_dataset (ds);
- }
- }
-
- int start_ofs = lex_ofs (lexer);
- struct file_handle *handle;
- if (lex_match_id (lexer, "INLINE"))
- handle = fh_inline_file ();
- else
- {
- if (lex_token (lexer) != T_ID && !lex_is_string (lexer))
- {
- lex_error (lexer,
- _("Syntax error expecting a file name or handle name."));
- return NULL;
- }
-
- handle = NULL;
- if (lex_token (lexer) == T_ID)
- handle = fh_from_id (lex_tokcstr (lexer));
- if (handle == NULL)
- handle = fh_create_file (NULL, lex_tokcstr (lexer), lex_get_encoding (lexer),
- fh_default_properties ());
- lex_get (lexer);
- }
-
- if (!(fh_get_referent (handle) & referent_mask))
- {
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("Handle for %s not allowed here."),
- referent_name (fh_get_referent (handle)));
- fh_unref (handle);
- return NULL;
- }
-
- return handle;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef LANGUAGE_DATA_IO_FILE_HANDLE_H
-#define LANGUAGE_DATA_IO_FILE_HANDLE_H 1
-
-/* Parsing file handles. */
-
-#include <stdbool.h>
-#include <stddef.h>
-#include "data/file-handle-def.h"
-
-struct lexer;
-struct session;
-
-struct file_handle *fh_parse (struct lexer *, enum fh_referent,
- struct session *);
-
-#endif /* language/data-io/file-handle.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012,
- 2013, 2015, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include <string.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/gnumeric-reader.h"
-#include "data/ods-reader.h"
-#include "data/spreadsheet-reader.h"
-#include "data/psql-reader.h"
-#include "data/settings.h"
-#include "language/command.h"
-#include "language/data-io/data-parser.h"
-#include "language/data-io/data-reader.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/placement-parser.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/cast.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-static bool parse_spreadsheet (struct lexer *lexer, char **filename,
- struct spreadsheet_read_options *opts);
-
-static void destroy_spreadsheet_read_info (struct spreadsheet_read_options *);
-
-static int parse_get_txt (struct lexer *, struct dataset *);
-static int parse_get_psql (struct lexer *, struct dataset *);
-static int parse_get_spreadsheet (struct lexer *, struct dataset *,
- struct spreadsheet *(*probe)(
- const char *filename, bool report_errors));
-
-int
-cmd_get_data (struct lexer *lexer, struct dataset *ds)
-{
- if (!lex_force_match_phrase (lexer, "/TYPE="))
- return CMD_FAILURE;
-
- if (lex_match_id (lexer, "TXT"))
- return parse_get_txt (lexer, ds);
- else if (lex_match_id (lexer, "PSQL"))
- return parse_get_psql (lexer, ds);
- else if (lex_match_id (lexer, "GNM"))
- return parse_get_spreadsheet (lexer, ds, gnumeric_probe);
- else if (lex_match_id (lexer, "ODS"))
- return parse_get_spreadsheet (lexer, ds, ods_probe);
- else
- {
- lex_error_expecting (lexer, "TXT", "PSQL", "GNM", "ODS");
- return CMD_FAILURE;
- }
-}
-
-static int
-parse_get_spreadsheet (struct lexer *lexer, struct dataset *ds,
- struct spreadsheet *(*probe)(
- const char *filename, bool report_errors))
-{
- struct spreadsheet_read_options opts;
- char *filename;
- if (!parse_spreadsheet (lexer, &filename, &opts))
- return CMD_FAILURE;
-
- bool ok = false;
- struct spreadsheet *spreadsheet = probe (filename, true);
- if (!spreadsheet)
- {
- msg (SE, _("error reading file `%s'"), filename);
- goto done;
- }
-
- struct casereader *reader = spreadsheet_make_reader (spreadsheet, &opts);
- if (reader)
- {
- dataset_set_dict (ds, dict_clone (spreadsheet->dict));
- dataset_set_source (ds, reader);
- ok = true;
- }
- spreadsheet_unref (spreadsheet);
-
-done:
- free (filename);
- destroy_spreadsheet_read_info (&opts);
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-}
-
-static int
-parse_get_psql (struct lexer *lexer, struct dataset *ds)
-{
- if (!lex_force_match_phrase (lexer, "/CONNECT=") || !lex_force_string (lexer))
- return CMD_FAILURE;
-
- struct psql_read_info psql = {
- .str_width = -1,
- .bsize = -1,
- .conninfo = ss_xstrdup (lex_tokss (lexer)),
- };
- bool ok = false;
-
- lex_get (lexer);
-
- while (lex_match (lexer, T_SLASH))
- {
- if (lex_match_id (lexer, "ASSUMEDSTRWIDTH"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "ASSUMEDSTRWIDTH", 1, 32767))
- goto done;
- psql.str_width = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "BSIZE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "BSIZE", 1, INT_MAX))
- goto done;
- psql.bsize = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "UNENCRYPTED"))
- psql.allow_clear = true;
- else if (lex_match_id (lexer, "SQL"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto done;
-
- free (psql.sql);
- psql.sql = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- }
-
- struct dictionary *dict = NULL;
- struct casereader *reader = psql_open_reader (&psql, &dict);
- if (reader)
- {
- dataset_set_dict (ds, dict);
- dataset_set_source (ds, reader);
- }
-
- done:
- free (psql.conninfo);
- free (psql.sql);
-
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-}
-
-static bool
-parse_spreadsheet (struct lexer *lexer, char **filename,
- struct spreadsheet_read_options *opts)
-{
- *opts = (struct spreadsheet_read_options) {
- .sheet_index = 1,
- .read_names = true,
- .asw = -1,
- };
- *filename = NULL;
-
- if (!lex_force_match_phrase (lexer, "/FILE=") || !lex_force_string (lexer))
- goto error;
-
- *filename = utf8_to_filename (lex_tokcstr (lexer));
- lex_get (lexer);
-
- while (lex_match (lexer, T_SLASH))
- {
- if (lex_match_id (lexer, "ASSUMEDSTRWIDTH"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "ASSUMEDSTRWIDTH", 1, 32767))
- goto error;
- opts->asw = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "SHEET"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "NAME"))
- {
- if (!lex_force_string (lexer))
- goto error;
-
- opts->sheet_name = ss_xstrdup (lex_tokss (lexer));
- opts->sheet_index = -1;
-
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "INDEX"))
- {
- if (!lex_force_int_range (lexer, "INDEX", 1, INT_MAX))
- goto error;
- opts->sheet_index = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "NAME", "INDEX");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "CELLRANGE"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "FULL"))
- opts->cell_range = NULL;
- else if (lex_match_id (lexer, "RANGE"))
- {
- if (!lex_force_string (lexer))
- goto error;
-
- opts->cell_range = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "FULL", "RANGE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "READNAMES"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "ON"))
- opts->read_names = true;
- else if (lex_match_id (lexer, "OFF"))
- opts->read_names = false;
- else
- {
- lex_error_expecting (lexer, "ON", "OFF");
- goto error;
- }
- }
- else
- {
- lex_error_expecting (lexer, "ASSUMEDSTRWIDTH", "SHEET", "CELLRANGE",
- "READNAMES");
- goto error;
- }
- }
-
- return true;
-
- error:
- destroy_spreadsheet_read_info (opts);
- free (*filename);
- return false;
-}
-
-
-static bool
-set_type (struct lexer *lexer, struct data_parser *parser,
- enum data_parser_type type,
- int type_start, int type_end, int *type_startp, int *type_endp)
-{
- if (!*type_startp)
- {
- data_parser_set_type (parser, type);
- *type_startp = type_start;
- *type_endp = type_end;
- }
- else if (type != data_parser_get_type (parser))
- {
- msg (SE, _("FIXED and DELIMITED arrangements are mutually exclusive."));
- lex_ofs_msg (lexer, SN, type_start, type_end,
- _("This syntax requires %s arrangement."),
- type == DP_FIXED ? "FIXED" : "DELIMITED");
- lex_ofs_msg (lexer, SN, *type_startp, *type_endp,
- _("This syntax requires %s arrangement."),
- type == DP_FIXED ? "DELIMITED" : "FIXED");
- return false;
- }
- return true;
-}
-
-static int
-parse_get_txt (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dict_create (get_default_encoding ());
- struct data_parser *parser = data_parser_create ();
- struct file_handle *fh = NULL;
- char *encoding = NULL;
- char *name = NULL;
-
- if (!lex_force_match_phrase (lexer, "/FILE="))
- goto error;
- fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
- if (fh == NULL)
- goto error;
-
- int type_start = 0, type_end = 0;
- data_parser_set_type (parser, DP_DELIMITED);
- data_parser_set_span (parser, false);
- data_parser_set_quotes (parser, ss_empty ());
- data_parser_set_quote_escape (parser, true);
- data_parser_set_empty_line_has_field (parser, true);
-
- for (;;)
- {
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
-
- if (lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "ARRANGEMENT"))
- {
- bool ok;
-
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "FIXED"))
- ok = set_type (lexer, parser, DP_FIXED,
- lex_ofs (lexer) - 3, lex_ofs (lexer) - 1,
- &type_start, &type_end);
- else if (lex_match_id (lexer, "DELIMITED"))
- ok = set_type (lexer, parser, DP_DELIMITED,
- lex_ofs (lexer) - 3, lex_ofs (lexer) - 1,
- &type_start, &type_end);
- else
- {
- lex_error_expecting (lexer, "FIXED", "DELIMITED");
- goto error;
- }
- if (!ok)
- goto error;
- }
- else if (lex_match_id (lexer, "FIRSTCASE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "FIRSTCASE", 1, INT_MAX))
- goto error;
- data_parser_set_skip (parser, lex_integer (lexer) - 1);
- lex_get (lexer);
- }
- else if (lex_match_id_n (lexer, "DELCASE", 4))
- {
- if (!set_type (lexer, parser, DP_DELIMITED,
- lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
- &type_start, &type_end))
- goto error;
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "LINE"))
- data_parser_set_span (parser, false);
- else if (lex_match_id (lexer, "VARIABLES"))
- {
- data_parser_set_span (parser, true);
-
- /* VARIABLES takes an integer argument, but for no
- good reason. We just ignore it. */
- if (!lex_force_int (lexer))
- goto error;
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "LINE", "VARIABLES");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "FIXCASE"))
- {
- if (!set_type (lexer, parser, DP_FIXED,
- lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
- &type_start, &type_end))
- goto error;
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "FIXCASE", 1, INT_MAX))
- goto error;
- data_parser_set_records (parser, lex_integer (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "IMPORTCASES"))
- {
- int start_ofs = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
- if (lex_match (lexer, T_ALL))
- {
- /* Nothing to do. */
- }
- else if (lex_match_id (lexer, "FIRST"))
- {
- if (!lex_force_int (lexer))
- goto error;
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "PERCENT"))
- {
- if (!lex_force_int (lexer))
- goto error;
- lex_get (lexer);
- }
- lex_ofs_msg (lexer, SW, start_ofs, lex_ofs (lexer) - 1,
- _("Ignoring obsolete IMPORTCASES subcommand. (N OF "
- "CASES or SAMPLE may be used to substitute.)"));
- }
- else if (lex_match_id_n (lexer, "DELIMITERS", 4))
- {
- if (!set_type (lexer, parser, DP_DELIMITED,
- lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
- &type_start, &type_end))
- goto error;
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_string (lexer))
- goto error;
-
- /* XXX should support multibyte UTF-8 characters */
- struct substring s = lex_tokss (lexer);
- struct string hard_seps = DS_EMPTY_INITIALIZER;
- const char *soft_seps = "";
- if (ss_match_string (&s, ss_cstr ("\\t")))
- ds_put_cstr (&hard_seps, "\t");
- if (ss_match_string (&s, ss_cstr ("\\\\")))
- ds_put_cstr (&hard_seps, "\\");
- int c;
- while ((c = ss_get_byte (&s)) != EOF)
- if (c == ' ')
- soft_seps = " ";
- else
- ds_put_byte (&hard_seps, c);
- data_parser_set_soft_delimiters (parser, ss_cstr (soft_seps));
- data_parser_set_hard_delimiters (parser, ds_ss (&hard_seps));
- ds_destroy (&hard_seps);
-
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "QUALIFIERS"))
- {
- if (!set_type (lexer, parser, DP_DELIMITED,
- lex_ofs (lexer) - 1, lex_ofs (lexer) - 1,
- &type_start, &type_end))
- goto error;
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_string (lexer))
- goto error;
-
- /* XXX should support multibyte UTF-8 characters */
- if (settings_get_syntax () == COMPATIBLE
- && ss_length (lex_tokss (lexer)) != 1)
- {
- lex_error (lexer, _("In compatible syntax mode, the QUALIFIER "
- "string must contain exactly one character."));
- goto error;
- }
-
- data_parser_set_quotes (parser, lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "VARIABLES"))
- break;
- else
- {
- lex_error_expecting (lexer, "VARIABLES");
- goto error;
- }
- }
- lex_match (lexer, T_EQUALS);
-
- int record = 1;
- enum data_parser_type type = data_parser_get_type (parser);
- do
- {
- while (type == DP_FIXED && lex_match (lexer, T_SLASH))
- {
- if (!lex_force_int_range (lexer, NULL, record,
- data_parser_get_records (parser)))
- goto error;
- record = lex_integer (lexer);
- lex_get (lexer);
- }
-
- int name_ofs = lex_ofs (lexer);
- if (!lex_force_id (lexer))
- goto error;
- name = xstrdup (lex_tokcstr (lexer));
- char *error = dict_id_is_valid__ (dict, name);
- if (error)
- {
- lex_error (lexer, "%s", error);
- free (error);
- goto error;
- }
- lex_get (lexer);
-
- struct fmt_spec input, output;
- int fc, lc;
- if (type == DP_DELIMITED)
- {
- if (!parse_format_specifier (lexer, &input))
- goto error;
- error = fmt_check_input__ (&input);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- goto error;
- }
- output = fmt_for_output_from_input (&input,
- settings_get_fmt_settings ());
- }
- else
- {
- int start_ofs = lex_ofs (lexer);
- if (!parse_column_range (lexer, 0, &fc, &lc, NULL))
- goto error;
-
- /* Accept a format (e.g. F8.2) or just a type name (e.g. DOLLAR). */
- char fmt_type_name[FMT_TYPE_LEN_MAX + 1];
- uint16_t w;
- uint8_t d;
- if (!parse_abstract_format_specifier (lexer, fmt_type_name, &w, &d))
- goto error;
-
- enum fmt_type fmt_type;
- if (!fmt_from_name (fmt_type_name, &fmt_type))
- {
- lex_next_error (lexer, -1, -1,
- _("Unknown format type `%s'."), fmt_type_name);
- goto error;
- }
- int end_ofs = lex_ofs (lexer) - 1;
-
- /* Compose input format. */
- input = (struct fmt_spec) { .type = fmt_type, .w = lc - fc + 1 };
- error = fmt_check_input__ (&input);
- if (error)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs, "%s", error);
- free (error);
- goto error;
- }
-
- /* Compose output format. */
- if (w != 0)
- {
- output = (struct fmt_spec) { .type = fmt_type, .w = w, .d = d };
- error = fmt_check_output__ (&output);
- if (error)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs, "%s", error);
- free (error);
- goto error;
- }
- }
- else
- output = fmt_for_output_from_input (&input,
- settings_get_fmt_settings ());
- }
- struct variable *v = dict_create_var (dict, name, fmt_var_width (&input));
- if (!v)
- {
- lex_ofs_error (lexer, name_ofs, name_ofs,
- _("%s is a duplicate variable name."), name);
- goto error;
- }
- var_set_both_formats (v, &output);
- if (type == DP_DELIMITED)
- data_parser_add_delimited_field (parser, &input,
- var_get_case_index (v),
- name);
- else
- data_parser_add_fixed_field (parser, &input, var_get_case_index (v),
- name, record, fc);
- free (name);
- name = NULL;
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- struct dfm_reader *reader = dfm_open_reader (fh, lexer, encoding);
- if (!reader)
- goto error;
-
- data_parser_make_active_file (parser, ds, reader, dict, NULL, NULL);
- fh_unref (fh);
- free (encoding);
- return CMD_SUCCESS;
-
- error:
- data_parser_destroy (parser);
- dict_unref (dict);
- fh_unref (fh);
- free (name);
- free (encoding);
- return CMD_CASCADING_FAILURE;
-}
-
-static void
-destroy_spreadsheet_read_info (struct spreadsheet_read_options *opts)
-{
- free (opts->cell_range);
- free (opts->sheet_name);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006-2007, 2010-15 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/any-reader.h"
-#include "data/case-map.h"
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/por-file-writer.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/trim.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/compiler.h"
-#include "libpspp/misc.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-\f
-/* Reading system and portable files. */
-
-/* Type of command. */
-enum reader_command
- {
- GET_CMD,
- IMPORT_CMD
- };
-
-static int parse_read_command (struct lexer *, struct dataset *,
- enum reader_command);
-
-/* GET. */
-int
-cmd_get (struct lexer *lexer, struct dataset *ds)
-{
- return parse_read_command (lexer, ds, GET_CMD);
-}
-
-/* IMPORT. */
-int
-cmd_import (struct lexer *lexer, struct dataset *ds)
-{
- return parse_read_command (lexer, ds, IMPORT_CMD);
-}
-
-/* Parses a GET or IMPORT command. */
-static int
-parse_read_command (struct lexer *lexer, struct dataset *ds,
- enum reader_command command)
-{
- struct casereader *reader = NULL;
- struct file_handle *fh = NULL;
- struct dictionary *dict = NULL;
- struct case_map *map = NULL;
- struct case_map_stage *stage = NULL;
- char *encoding = NULL;
-
- for (;;)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "FILE") || lex_is_string (lexer))
- {
- lex_match (lexer, T_EQUALS);
-
- fh_unref (fh);
- fh = fh_parse (lexer, FH_REF_FILE, NULL);
- if (fh == NULL)
- goto error;
- }
- else if (command == GET_CMD && lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_string (lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- }
- else if (command == IMPORT_CMD && lex_match_id (lexer, "TYPE"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (!lex_match_id (lexer, "COMM")
- && !lex_match_id (lexer, "TAPE"))
- {
- lex_error_expecting (lexer, "COMM", "TAPE");
- goto error;
- }
- }
- else
- break;
- }
-
- if (fh == NULL)
- {
- lex_sbc_missing (lexer, "FILE");
- goto error;
- }
-
- reader = any_reader_open_and_decode (fh, encoding, &dict, NULL);
- if (reader == NULL)
- goto error;
-
- if (dict_get_n_vars (dict) == 0)
- {
- msg (SE, _("%s: Data file dictionary has no variables."),
- fh_get_name (fh));
- goto error;
- }
-
- stage = case_map_stage_create (dict);
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
- if (!parse_dict_trim (lexer, dict))
- goto error;
- }
- dict_compact_values (dict);
-
- map = case_map_stage_get_case_map (stage);
- case_map_stage_destroy (stage);
- if (map != NULL)
- reader = case_map_create_input_translator (map, reader);
-
- dataset_set_dict (ds, dict);
- dataset_set_source (ds, reader);
-
- fh_unref (fh);
- free (encoding);
- return CMD_SUCCESS;
-
- error:
- case_map_stage_destroy (stage);
- fh_unref (fh);
- casereader_destroy (reader);
- if (dict != NULL)
- dict_unref (dict);
- free (encoding);
- return CMD_CASCADING_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2012, 2013 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/caseinit.h"
-#include "data/casereader-provider.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/session.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/data-reader.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/inpt-pgm.h"
-#include "language/expressions/public.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Indicates how a `union value' should be initialized. */
-struct input_program_pgm
- {
- struct session *session;
- struct dataset *ds;
-
- struct trns_chain xforms;
- size_t idx;
- bool eof;
-
- casenumber case_nr; /* Incremented by END CASE transformation. */
-
- struct caseinit *init;
- struct caseproto *proto;
- };
-
-static void destroy_input_program (struct input_program_pgm *);
-static const struct trns_class end_case_trns_class;
-static const struct trns_class reread_trns_class;
-static const struct trns_class end_file_trns_class;
-
-static const struct casereader_class input_program_casereader_class;
-
-static bool inside_input_program;
-static bool saw_END_CASE;
-static bool saw_END_FILE;
-static bool saw_DATA_LIST;
-
-/* Returns true if we're parsing the inside of a INPUT
- PROGRAM...END INPUT PROGRAM construct, false otherwise. */
-bool
-in_input_program (void)
-{
- return inside_input_program;
-}
-
-void
-data_list_seen (void)
-{
- saw_DATA_LIST = true;
-}
-
-/* Emits an END CASE transformation for INP. */
-static void
-emit_END_CASE (struct dataset *ds)
-{
- add_transformation (ds, &end_case_trns_class, xzalloc (sizeof (bool)));
-}
-
-int
-cmd_input_program (struct lexer *lexer, struct dataset *ds)
-{
- struct msg_location *location = lex_ofs_location (lexer, 0, 1);
- if (!lex_match (lexer, T_ENDCMD))
- {
- msg_location_destroy (location);
- return lex_end_of_command (lexer);
- }
-
- struct session *session = session_create (dataset_session (ds));
- struct dataset *inp_ds = dataset_create (session, "INPUT PROGRAM");
-
- struct input_program_pgm *inp = xmalloc (sizeof *inp);
- *inp = (struct input_program_pgm) { .session = session, .ds = inp_ds };
-
- proc_push_transformations (inp->ds);
- inside_input_program = true;
- saw_END_CASE = saw_END_FILE = saw_DATA_LIST = false;
- while (!lex_match_phrase (lexer, "END INPUT PROGRAM"))
- {
- enum cmd_result result;
-
- result = cmd_parse_in_state (lexer, inp->ds, CMD_STATE_INPUT_PROGRAM);
- if (result == CMD_EOF
- || result == CMD_FINISH
- || result == CMD_CASCADING_FAILURE)
- {
- proc_pop_transformations (inp->ds, &inp->xforms);
-
- if (result == CMD_EOF)
- msg (SE, _("Unexpected end-of-file within %s."), "INPUT PROGRAM");
- inside_input_program = false;
- destroy_input_program (inp);
- msg_location_destroy (location);
- return result;
- }
- }
- if (!saw_END_CASE)
- emit_END_CASE (inp->ds);
- inside_input_program = false;
- proc_pop_transformations (inp->ds, &inp->xforms);
-
- struct msg_location *end = lex_ofs_location (lexer, 0, 2);
- msg_location_merge (&location, end);
- location->omit_underlines = true;
- msg_location_destroy (end);
-
- if (!saw_DATA_LIST && !saw_END_FILE)
- {
- msg_at (SE, location, _("Input program does not contain %s or %s."),
- "DATA LIST", "END FILE");
- destroy_input_program (inp);
- msg_location_destroy (location);
- return CMD_FAILURE;
- }
- if (dict_get_next_value_idx (dataset_dict (inp->ds)) == 0)
- {
- msg_at (SE, location, _("Input program did not create any variables."));
- destroy_input_program (inp);
- msg_location_destroy (location);
- return CMD_FAILURE;
- }
- msg_location_destroy (location);
-
- /* Figure out how to initialize each input case. */
- inp->init = caseinit_create ();
- caseinit_mark_for_init (inp->init, dataset_dict (inp->ds));
- inp->proto = caseproto_ref (dict_get_proto (dataset_dict (inp->ds)));
-
- dataset_set_dict (ds, dict_clone (dataset_dict (inp->ds)));
- dataset_set_source (
- ds, casereader_create_sequential (NULL, inp->proto, CASENUMBER_MAX,
- &input_program_casereader_class, inp));
-
- return CMD_SUCCESS;
-}
-
-/* Reads and returns one case.
- Returns the case if successful, null at end of file or if an
- I/O error occurred. */
-static struct ccase *
-input_program_casereader_read (struct casereader *reader UNUSED, void *inp_)
-{
- struct input_program_pgm *inp = inp_;
-
- if (inp->eof || !inp->xforms.n)
- return NULL;
-
- struct ccase *c = case_create (inp->proto);
- caseinit_init_vars (inp->init, c);
-
- for (size_t i = inp->idx < inp->xforms.n ? inp->idx : 0; ; i++)
- {
- if (i >= inp->xforms.n)
- {
- i = 0;
- c = case_unshare (c);
- caseinit_update_left_vars (inp->init, c);
- caseinit_init_vars (inp->init, c);
- }
-
- const struct transformation *trns = &inp->xforms.xforms[i];
- switch (trns->class->execute (trns->aux, &c, inp->case_nr))
- {
- case TRNS_END_CASE:
- inp->case_nr++;
- inp->idx = i;
- return c;
-
- case TRNS_ERROR:
- casereader_force_error (reader);
- /* Fall through. */
- case TRNS_END_FILE:
- inp->eof = true;
- case_unref (c);
- return NULL;
-
- case TRNS_CONTINUE:
- break;
-
- default:
- NOT_REACHED ();
- }
- }
-}
-
-static void
-destroy_input_program (struct input_program_pgm *pgm)
-{
- if (pgm != NULL)
- {
- session_destroy (pgm->session);
- trns_chain_uninit (&pgm->xforms);
- caseinit_destroy (pgm->init);
- caseproto_unref (pgm->proto);
- free (pgm);
- }
-}
-
-/* Destroys the casereader. */
-static void
-input_program_casereader_destroy (struct casereader *reader UNUSED, void *inp_)
-{
- struct input_program_pgm *inp = inp_;
- destroy_input_program (inp);
-}
-
-static const struct casereader_class input_program_casereader_class =
- {
- input_program_casereader_read,
- input_program_casereader_destroy,
- NULL,
- NULL,
- };
-\f
-int
-cmd_end_case (struct lexer *lexer UNUSED, struct dataset *ds)
-{
- assert (in_input_program ());
- emit_END_CASE (ds);
- saw_END_CASE = true;
- return CMD_SUCCESS;
-}
-
-/* Outputs the current case */
-static enum trns_result
-end_case_trns_proc (void *resume_, struct ccase **c UNUSED,
- casenumber case_nr UNUSED)
-{
- bool *resume = resume_;
- enum trns_result retval = *resume ? TRNS_CONTINUE : TRNS_END_CASE;
- *resume = !*resume;
- return retval;
-}
-
-static bool
-end_case_trns_free (void *resume)
-{
- free (resume);
- return true;
-}
-
-static const struct trns_class end_case_trns_class = {
- .name = "END CASE",
- .execute = end_case_trns_proc,
- .destroy = end_case_trns_free,
-};
-
-/* REREAD transformation. */
-struct reread_trns
- {
- struct dfm_reader *reader; /* File to move file pointer back on. */
- struct expression *column; /* Column to reset file pointer to. */
- };
-
-/* Parses REREAD command. */
-int
-cmd_reread (struct lexer *lexer, struct dataset *ds)
-{
- char *encoding = NULL;
- struct file_handle *fh = fh_get_default_handle ();
- struct expression *e = NULL;
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "COLUMN"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (e)
- {
- lex_sbc_only_once (lexer, "COLUMN");
- goto error;
- }
-
- e = expr_parse (lexer, ds, VAL_NUMERIC);
- if (!e)
- goto error;
- }
- else if (lex_match_id (lexer, "FILE"))
- {
- lex_match (lexer, T_EQUALS);
- fh_unref (fh);
- fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
- if (fh == NULL)
- goto error;
- }
- else if (lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "COLUMN", "FILE", "ENCODING");
- goto error;
- }
- }
-
- struct reread_trns *t = xmalloc (sizeof *t);
- *t = (struct reread_trns) {
- .reader = dfm_open_reader (fh, lexer, encoding),
- .column = e,
- };
- add_transformation (ds, &reread_trns_class, t);
-
- fh_unref (fh);
- free (encoding);
- return CMD_SUCCESS;
-
-error:
- expr_free (e);
- free (encoding);
- return CMD_CASCADING_FAILURE;
-}
-
-/* Executes a REREAD transformation. */
-static enum trns_result
-reread_trns_proc (void *t_, struct ccase **c, casenumber case_num)
-{
- struct reread_trns *t = t_;
-
- if (t->column == NULL)
- dfm_reread_record (t->reader, 1);
- else
- {
- double column = expr_evaluate_num (t->column, *c, case_num);
- if (!isfinite (column) || column < 1)
- {
- msg (SE, _("REREAD: Column numbers must be positive finite "
- "numbers. Column set to 1."));
- dfm_reread_record (t->reader, 1);
- }
- else
- dfm_reread_record (t->reader, column);
- }
- return TRNS_CONTINUE;
-}
-
-/* Frees a REREAD transformation.
- Returns true if successful, false if an I/O error occurred. */
-static bool
-reread_trns_free (void *t_)
-{
- struct reread_trns *t = t_;
- expr_free (t->column);
- dfm_close_reader (t->reader);
- return true;
-}
-
-static const struct trns_class reread_trns_class = {
- .name = "REREAD",
- .execute = reread_trns_proc,
- .destroy = reread_trns_free,
-};
-
-/* Parses END FILE command. */
-int
-cmd_end_file (struct lexer *lexer UNUSED, struct dataset *ds)
-{
- assert (in_input_program ());
-
- add_transformation (ds, &end_file_trns_class, NULL);
- saw_END_FILE = true;
-
- return CMD_SUCCESS;
-}
-
-/* Executes an END FILE transformation. */
-static enum trns_result
-end_file_trns_proc (void *trns_ UNUSED, struct ccase **c UNUSED,
- casenumber case_num UNUSED)
-{
- return TRNS_END_FILE;
-}
-
-static const struct trns_class end_file_trns_class = {
- .name = "END FILE",
- .execute = end_file_trns_proc,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef INPT_PGM_H
-#define INPT_PGM_H 1
-
-#include <stdbool.h>
-
-bool in_input_program (void);
-
-void data_list_seen (void);
-
-#endif /* inpt-pgm.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2009-2011, 2013, 2014, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/data-out.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/compiler.h"
-#include "libpspp/ll.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "output/pivot-table.h"
-
-#include "gl/intprops.h"
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-#include "gl/xmalloca.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-struct lst_cmd
- {
- long first;
- long last;
- long step;
- const struct variable **vars;
- size_t n_vars;
- bool number_cases;
- };
-
-static int
-list_execute (const struct lst_cmd *lcmd, struct dataset *ds)
-{
- const struct dictionary *dict = dataset_dict (ds);
-
- struct subcase sc;
- subcase_init_empty (&sc);
- for (size_t i = 0; i < lcmd->n_vars; i++)
- subcase_add_var (&sc, lcmd->vars[i], SC_ASCEND);
-
- struct casegrouper *grouper;
- struct casereader *group;
- grouper = casegrouper_create_splits (proc_open (ds), dict);
- while (casegrouper_get_next_group (grouper, &group))
- {
- output_split_file_values_peek (ds, group);
- group = casereader_project (group, &sc);
- group = casereader_select (group, lcmd->first - 1,
- (lcmd->last != LONG_MAX ? lcmd->last
- : CASENUMBER_MAX), lcmd->step);
-
- struct pivot_table *table = pivot_table_create (N_("Data List"));
- table->show_values = table->show_variables = SETTINGS_VALUE_SHOW_VALUE;
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Variables"));
- for (size_t i = 0; i < lcmd->n_vars; i++)
- pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (lcmd->vars[i]));
-
- struct pivot_dimension *cases = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Case Number"));
- if (lcmd->number_cases)
- cases->root->show_label = true;
- else
- cases->hide_all_labels = true;
-
- casenumber case_num = lcmd->first;
- struct ccase *c;
- for (; (c = casereader_read (group)) != NULL; case_unref (c))
- {
- int case_idx = pivot_category_create_leaf (
- cases->root, pivot_value_new_integer (case_num));
- case_num += lcmd->step;
-
- for (size_t i = 0; i < lcmd->n_vars; i++)
- pivot_table_put2 (table, i, case_idx,
- pivot_value_new_var_value (
- lcmd->vars[i], case_data_idx (c, i)));
- }
- casereader_destroy (group);
-
- pivot_table_submit (table);
- }
-
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- subcase_uninit (&sc);
- free (lcmd->vars);
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-}
-
-
-/* Parses and executes the LIST procedure. */
-int
-cmd_list (struct lexer *lexer, struct dataset *ds)
-{
- const struct dictionary *dict = dataset_dict (ds);
-
- struct lst_cmd cmd = {
- .step = 1,
- .first = 1,
- .last = LONG_MAX,
- };
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "VARIABLES"))
- {
- lex_match (lexer, T_EQUALS);
- free (cmd.vars);
- cmd.vars = NULL;
- if (!parse_variables_const (lexer, dict, &cmd.vars, &cmd.n_vars,
- PV_DUPLICATE))
- goto error;
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "NUMBERED"))
- cmd.number_cases = true;
- else if (lex_match_id (lexer, "UNNUMBERED"))
- cmd.number_cases = false;
- else
- {
- lex_error_expecting (lexer, "NUMBERED", "UNNUMBERED");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "CASES"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "FROM"))
- {
- if (!lex_force_int_range (lexer, "FROM", 1, LONG_MAX))
- goto error;
- cmd.first = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- cmd.first = 1;
-
- if (lex_match (lexer, T_TO) || lex_is_integer (lexer))
- {
- if (!lex_force_int_range (lexer, "TO", cmd.first, LONG_MAX))
- goto error;
- cmd.last = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- cmd.last = LONG_MAX;
-
- if (lex_match (lexer, T_BY))
- {
- if (!lex_force_int_range (lexer, "TO", 1, LONG_MAX))
- goto error;
- cmd.step = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- cmd.step = 1;
- }
- else
- {
- free (cmd.vars);
- cmd.vars = NULL;
- if (!parse_variables_const (lexer, dict, &cmd.vars, &cmd.n_vars,
- PV_DUPLICATE))
- goto error;
- }
- }
-
- if (!cmd.n_vars)
- dict_get_vars (dict, &cmd.vars, &cmd.n_vars, DC_SYSTEM | DC_SCRATCH);
- return list_execute (&cmd, ds);
-
- error:
- free (cmd.vars);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2017 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_matrix.h>
-#include <gsl/gsl_vector.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/data-in.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/short-names.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/data-parser.h"
-#include "language/data-io/data-reader.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/inpt-pgm.h"
-#include "language/data-io/placement-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/i18n.h"
-#include "libpspp/intern.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/c-ctype.h"
-#include "gl/minmax.h"
-#include "gl/xsize.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-\f
-#define ROWTYPES \
- /* Matrix row types. */ \
- RT(CORR, 2) \
- RT(COV, 2) \
- RT(MAT, 2) \
- RT(N_MATRIX, 2) \
- RT(PROX, 2) \
- \
- /* Vector row types. */ \
- RT(COUNT, 1) \
- RT(DFE, 1) \
- RT(MEAN, 1) \
- RT(MSE, 1) \
- RT(STDDEV, 1) \
- RT(N, 1) \
- \
- /* Scalar row types. */ \
- RT(N_SCALAR, 0)
-
-enum rowtype
- {
-#define RT(NAME, DIMS) C_##NAME,
- ROWTYPES
-#undef RT
- };
-
-enum
- {
-#define RT(NAME, DIMS) +1
- N_ROWTYPES = ROWTYPES
-#undef RT
- };
-verify (N_ROWTYPES < 32);
-
-/* Returns the number of dimensions in the indexes for row type RT. A matrix
- has 2 dimensions, a vector has 1, a scalar has 0. */
-static int
-rowtype_dimensions (enum rowtype rt)
-{
- static const int rowtype_dims[N_ROWTYPES] = {
-#define RT(NAME, DIMS) [C_##NAME] = DIMS,
- ROWTYPES
-#undef RT
- };
- return rowtype_dims[rt];
-}
-
-static struct substring
-rowtype_name (enum rowtype rt)
-{
- static const struct substring rowtype_names[N_ROWTYPES] = {
-#define RT(NAME, DIMS) [C_##NAME] = SS_LITERAL_INITIALIZER (#NAME),
- ROWTYPES
-#undef RT
- };
-
- return rowtype_names[rt];
-}
-
-static bool
-rowtype_from_string (struct substring token, enum rowtype *rt)
-{
- ss_trim (&token, ss_cstr (CC_SPACES));
- for (size_t i = 0; i < N_ROWTYPES; i++)
- if (lex_id_match (rowtype_name (i), token))
- {
- *rt = i;
- return true;
- }
-
- if (lex_id_match (ss_cstr ("N_VECTOR"), token))
- {
- *rt = C_N;
- return true;
- }
- else if (lex_id_match (ss_cstr ("SD"), token))
- {
- *rt = C_STDDEV;
- return true;
- }
-
- return false;
-}
-
-static bool
-rowtype_parse (struct lexer *lexer, enum rowtype *rt)
-{
- bool parsed = (lex_token (lexer) == T_ID
- && rowtype_from_string (lex_tokss (lexer), rt));
- if (parsed)
- lex_get (lexer);
- return parsed;
-}
-\f
-struct matrix_format
- {
- bool span;
- enum triangle
- {
- LOWER,
- UPPER,
- FULL
- }
- triangle;
- enum diagonal
- {
- DIAGONAL,
- NO_DIAGONAL
- }
- diagonal;
-
- bool input_rowtype;
- struct variable **input_vars;
- size_t n_input_vars;
-
- /* How to read matrices with each possible number of dimensions (0=scalar,
- 1=vector, 2=matrix). */
- struct matrix_sched
- {
- /* Number of rows and columns in the matrix: (1,1) for a scalar, (1,n) for
- a vector, (n,n) for a matrix. */
- int nr, nc;
-
- /* Rows of data to read and the number of columns in each. Because we
- often read just a triangle and sometimes omit the diagonal, 'n_rp' can
- be less than 'nr' and 'rp[i]->y' isn't always 'y'. */
- struct row_sched
- {
- /* The y-value of the row inside the matrix. */
- int y;
-
- /* first and last (exclusive) columns to read in this row. */
- int x0, x1;
- }
- *rp;
- size_t n_rp;
- }
- ms[3];
-
- struct variable *rowtype;
- struct variable *varname;
- struct variable **cvars;
- int n_cvars;
- struct variable **svars;
- size_t *svar_indexes;
- size_t n_svars;
- struct variable **fvars;
- size_t *fvar_indexes;
- size_t n_fvars;
- int cells;
- int n;
-
- unsigned int pooled_rowtype_mask;
- unsigned int factor_rowtype_mask;
-
- struct content
- {
- bool open;
- enum rowtype rowtype;
- bool close;
- }
- *contents;
- size_t n_contents;
- };
-
-static void
-matrix_format_uninit (struct matrix_format *mf)
-{
- free (mf->input_vars);
- for (int i = 0; i < 3; i++)
- free (mf->ms[i].rp);
- free (mf->cvars);
- free (mf->svars);
- free (mf->svar_indexes);
- free (mf->fvars);
- free (mf->fvar_indexes);
- free (mf->contents);
-}
-
-static void
-set_string (struct ccase *outcase, const struct variable *var,
- struct substring src)
-{
- struct substring dst = case_ss (outcase, var);
- for (size_t i = 0; i < dst.length; i++)
- dst.string[i] = i < src.length ? src.string[i] : ' ';
-}
-
-static void
-parse_msg (struct dfm_reader *reader, const struct substring *token,
- char *text, enum msg_severity severity)
-{
- int first_column = 0;
- if (token)
- {
- struct substring line = dfm_get_record (reader);
- if (token->string >= line.string && token->string < ss_end (line))
- first_column = ss_pointer_to_position (line, token->string) + 1;
- }
-
- int line_number = dfm_get_line_number (reader);
- struct msg_location *location = xmalloc (sizeof *location);
- int last_column = (first_column && token->length
- ? first_column + token->length - 1
- : 0);
- *location = (struct msg_location) {
- .file_name = intern_new (dfm_get_file_name (reader)),
- .start = { .line = line_number, .column = first_column },
- .end = { .line = line_number, .column = last_column },
- };
- struct msg *m = xmalloc (sizeof *m);
- *m = (struct msg) {
- .category = MSG_C_DATA,
- .severity = severity,
- .location = location,
- .text = text,
- };
- msg_emit (m);
-}
-
-static void PRINTF_FORMAT (3, 4)
-parse_warning (struct dfm_reader *reader, const struct substring *token,
- const char *format, ...)
-{
- va_list args;
- va_start (args, format);
- parse_msg (reader, token, xvasprintf (format, args), MSG_S_WARNING);
- va_end (args);
-}
-
-static void PRINTF_FORMAT (3, 4)
-parse_error (struct dfm_reader *reader, const struct substring *token,
- const char *format, ...)
-{
- va_list args;
- va_start (args, format);
- parse_msg (reader, token, xvasprintf (format, args), MSG_S_ERROR);
- va_end (args);
-}
-
-/* Advance to beginning of next token. */
-static bool
-more_tokens (struct substring *p, struct dfm_reader *r)
-{
- for (;;)
- {
- ss_ltrim (p, ss_cstr (CC_SPACES ","));
- if (p->length)
- return true;
-
- dfm_forward_record (r);
- if (dfm_eof (r))
- return false;
- *p = dfm_get_record (r);
- }
-}
-
-static bool
-next_token (struct substring *p, struct dfm_reader *r, struct substring *token)
-{
- if (!more_tokens (p, r))
- return false;
-
- /* Collect token. */
- int c = ss_first (*p);
- if (c == '\'' || c == '"')
- {
- ss_advance (p, 1);
- ss_get_until (p, c, token);
- }
- else
- {
- size_t n = 1;
- for (;;)
- {
- c = ss_at (*p, n);
- if (c == EOF
- || ss_find_byte (ss_cstr (CC_SPACES ","), c) != SIZE_MAX
- || ((c == '+' || c == '-')
- && ss_find_byte (ss_cstr ("dDeE"),
- ss_at (*p, n - 1)) == SIZE_MAX))
- break;
- n++;
- }
- ss_get_bytes (p, n, token);
- }
- return true;
-}
-
-static bool
-next_number (struct substring *p, struct dfm_reader *r, double *d)
-{
- struct substring token;
- if (!next_token (p, r, &token))
- return false;
-
- union value v;
- char *error = data_in (token, dfm_reader_get_encoding (r), FMT_F,
- settings_get_fmt_settings (), &v, 0, NULL);
- if (error)
- {
- parse_error (r, &token, "%s", error);
- free (error);
- }
- *d = v.f;
- return true;
-}
-
-static bool
-next_rowtype (struct substring *p, struct dfm_reader *r, enum rowtype *rt)
-{
- struct substring token;
- if (!next_token (p, r, &token))
- return false;
-
- if (rowtype_from_string (token, rt))
- return true;
-
- parse_error (r, &token, _("Unknown row type \"%.*s\"."),
- (int) token.length, token.string);
- return false;
-}
-
-struct read_matrix_params
- {
- /* Adjustments to first and last row to read. */
- int dy0, dy1;
-
- /* Left and right columns to read in first row, inclusive.
- For x1, INT_MAX is the rightmost column. */
- int x0, x1;
-
- /* Adjustment to x0 and x1 for each subsequent row we read. Each of these
- is 0 to keep it the same or -1 or +1 to adjust it by that much. */
- int dx0, dx1;
- };
-
-static const struct read_matrix_params *
-get_read_matrix_params (const struct matrix_format *mf)
-{
- if (mf->triangle == FULL)
- {
- /* 1 2 3 4
- 2 1 5 6
- 3 5 1 7
- 4 6 7 1 */
- static const struct read_matrix_params rmp = { 0, 0, 0, INT_MAX, 0, 0 };
- return &rmp;
- }
- else if (mf->triangle == LOWER)
- {
- if (mf->diagonal == DIAGONAL)
- {
- /* 1 . . .
- 2 1 . .
- 3 5 1 .
- 4 6 7 1 */
- static const struct read_matrix_params rmp = { 0, 0, 0, 0, 0, 1 };
- return &rmp;
- }
- else
- {
- /* . . . .
- 2 . . .
- 3 5 . .
- 4 6 7 . */
- static const struct read_matrix_params rmp = { 1, 0, 0, 0, 0, 1 };
- return &rmp;
- }
- }
- else if (mf->triangle == UPPER)
- {
- if (mf->diagonal == DIAGONAL)
- {
- /* 1 2 3 4
- . 1 5 6
- . . 1 7
- . . . 1 */
- static const struct read_matrix_params rmp = { 0, 0, 0, INT_MAX, 1, 0 };
- return &rmp;
- }
- else
- {
- /* . 2 3 4
- . . 5 6
- . . . 7
- . . . . */
- static const struct read_matrix_params rmp = { 0, -1, 1, INT_MAX, 1, 0 };
- return &rmp;
- }
- }
- else
- NOT_REACHED ();
-}
-
-static void
-schedule_matrices (struct matrix_format *mf)
-{
- struct matrix_sched *ms0 = &mf->ms[0];
- ms0->nr = 1;
- ms0->nc = 1;
- ms0->rp = xmalloc (sizeof *ms0->rp);
- ms0->rp[0] = (struct row_sched) { .y = 0, .x0 = 0, .x1 = 1 };
- ms0->n_rp = 1;
-
- struct matrix_sched *ms1 = &mf->ms[1];
- ms1->nr = 1;
- ms1->nc = mf->n_cvars;
- ms1->rp = xmalloc (sizeof *ms1->rp);
- ms1->rp[0] = (struct row_sched) { .y = 0, .x0 = 0, .x1 = mf->n_cvars };
- ms1->n_rp = 1;
-
- struct matrix_sched *ms2 = &mf->ms[2];
- ms2->nr = mf->n_cvars;
- ms2->nc = mf->n_cvars;
- ms2->rp = xmalloc (mf->n_cvars * sizeof *ms2->rp);
- ms2->n_rp = 0;
-
- const struct read_matrix_params *rmp = get_read_matrix_params (mf);
- int x0 = rmp->x0;
- int x1 = rmp->x1 < mf->n_cvars ? rmp->x1 : mf->n_cvars - 1;
- int y0 = rmp->dy0;
- int y1 = (int) mf->n_cvars + rmp->dy1;
- for (int y = y0; y < y1; y++)
- {
- assert (x0 >= 0 && x0 < mf->n_cvars);
- assert (x1 >= 0 && x1 < mf->n_cvars);
- assert (x1 >= x0);
-
- ms2->rp[ms2->n_rp++] = (struct row_sched) {
- .y = y, .x0 = x0, .x1 = x1 + 1
- };
-
- x0 += rmp->dx0;
- x1 += rmp->dx1;
- }
-}
-
-static bool
-read_id_columns (const struct matrix_format *mf,
- struct substring *p, struct dfm_reader *r,
- double *d, enum rowtype *rt)
-{
- for (size_t i = 0; mf->input_vars[i] != mf->cvars[0]; i++)
- if (!(mf->input_vars[i] == mf->rowtype
- ? next_rowtype (p, r, rt)
- : next_number (p, r, &d[i])))
- return false;
- return true;
-}
-
-static bool
-equal_id_columns (const struct matrix_format *mf,
- const double *a, const double *b)
-{
- for (size_t i = 0; mf->input_vars[i] != mf->cvars[0]; i++)
- if (mf->input_vars[i] != mf->rowtype && a[i] != b[i])
- return false;
- return true;
-}
-
-static bool
-equal_split_columns (const struct matrix_format *mf,
- const double *a, const double *b)
-{
- for (size_t i = 0; i < mf->n_svars; i++)
- {
- size_t idx = mf->svar_indexes[i];
- if (a[idx] != b[idx])
- return false;
- }
- return true;
-}
-
-static bool
-is_pooled (const struct matrix_format *mf, const double *d)
-{
- for (size_t i = 0; i < mf->n_fvars; i++)
- if (d[mf->fvar_indexes[i]] != SYSMIS)
- return false;
- return true;
-}
-
-static void
-matrix_sched_init (const struct matrix_format *mf, enum rowtype rt,
- gsl_matrix *m)
-{
- int n_dims = rowtype_dimensions (rt);
- const struct matrix_sched *ms = &mf->ms[n_dims];
- double diagonal = n_dims < 2 || rt != C_CORR ? SYSMIS : 1.0;
- for (size_t y = 0; y < ms->nr; y++)
- for (size_t x = 0; x < ms->nc; x++)
- gsl_matrix_set (m, y, x, y == x ? diagonal : SYSMIS);
-}
-
-static void
-matrix_sched_output (const struct matrix_format *mf, enum rowtype rt,
- gsl_matrix *m, const double *d, int split_num,
- struct casewriter *w)
-{
- int n_dims = rowtype_dimensions (rt);
- const struct matrix_sched *ms = &mf->ms[n_dims];
-
- if (rt == C_N_SCALAR)
- {
- for (size_t x = 1; x < mf->n_cvars; x++)
- gsl_matrix_set (m, 0, x, gsl_matrix_get (m, 0, 0));
- rt = C_N;
- }
-
- for (int y = 0; y < ms->nr; y++)
- {
- struct ccase *c = case_create (casewriter_get_proto (w));
- for (size_t i = 0; mf->input_vars[i] != mf->cvars[0]; i++)
- if (mf->input_vars[i] != mf->rowtype)
- *case_num_rw (c, mf->input_vars[i]) = d[i];
- if (mf->n_svars && !mf->svar_indexes)
- *case_num_rw (c, mf->svars[0]) = split_num;
- set_string (c, mf->rowtype, rowtype_name (rt));
- const char *varname = n_dims == 2 ? var_get_name (mf->cvars[y]) : "";
- set_string (c, mf->varname, ss_cstr (varname));
- for (int x = 0; x < mf->n_cvars; x++)
- *case_num_rw (c, mf->cvars[x]) = gsl_matrix_get (m, y, x);
- casewriter_write (w, c);
- }
-}
-
-static void
-matrix_sched_output_n (const struct matrix_format *mf, double n,
- gsl_matrix *m, const double *d, int split_num,
- struct casewriter *w)
-{
- gsl_matrix_set (m, 0, 0, n);
- matrix_sched_output (mf, C_N_SCALAR, m, d, split_num, w);
-}
-
-static void
-check_eol (const struct matrix_format *mf, struct substring *p,
- struct dfm_reader *r)
-{
- if (!mf->span)
- {
- ss_ltrim (p, ss_cstr (CC_SPACES ","));
- if (p->length)
- {
- parse_error (r, p, _("Extraneous data expecting end of line."));
- p->length = 0;
- }
- }
-}
-
-static void
-parse_data_with_rowtype (const struct matrix_format *mf,
- struct dfm_reader *r, struct casewriter *w)
-{
- if (dfm_eof (r))
- return;
- struct substring p = dfm_get_record (r);
-
- double *prev = NULL;
- gsl_matrix *m = gsl_matrix_alloc (mf->n_cvars, mf->n_cvars);
-
- double *d = xnmalloc (mf->n_input_vars, sizeof *d);
- enum rowtype rt;
-
- double *d_next = xnmalloc (mf->n_input_vars, sizeof *d_next);
-
- if (!read_id_columns (mf, &p, r, d, &rt))
- goto exit;
- for (;;)
- {
- /* If this has rowtype N but there was an N subcommand, then the
- subcommand takes precedence, so we will suppress outputting this
- record. We still need to parse it, though, so we can't skip other
- work. */
- bool suppress_output = mf->n >= 0 && (rt == C_N || rt == C_N_SCALAR);
- if (suppress_output)
- parse_error (r, NULL, _("N record is not allowed with N subcommand. "
- "Ignoring N record."));
-
- /* If there's an N subcommand, and this is a new split, then output an N
- record. */
- if (mf->n >= 0 && (!prev || !equal_split_columns (mf, prev, d)))
- {
- matrix_sched_output_n (mf, mf->n, m, d, 0, w);
-
- if (!prev)
- prev = xnmalloc (mf->n_input_vars, sizeof *prev);
- memcpy (prev, d, mf->n_input_vars * sizeof *prev);
- }
-
- /* Usually users don't provide the CONTENTS subcommand with ROWTYPE_, but
- if they did then warn if ROWTYPE_ is an unexpected type. */
- if (mf->factor_rowtype_mask || mf->pooled_rowtype_mask)
- {
- const char *name = rowtype_name (rt).string;
- if (is_pooled (mf, d))
- {
- if (!((1u << rt) & mf->pooled_rowtype_mask))
- parse_warning (r, NULL, _("Data contains pooled row type %s not "
- "included in CONTENTS."), name);
- }
- else
- {
- if (!((1u << rt) & mf->factor_rowtype_mask))
- parse_warning (r, NULL, _("Data contains with-factors row type "
- "%s not included in CONTENTS."), name);
- }
- }
-
- /* Initialize the matrix to be filled-in. */
- int n_dims = rowtype_dimensions (rt);
- const struct matrix_sched *ms = &mf->ms[n_dims];
- matrix_sched_init (mf, rt, m);
-
- enum rowtype rt_next;
- bool eof;
-
- size_t n_rows;
- for (n_rows = 1; ; n_rows++)
- {
- if (n_rows <= ms->n_rp)
- {
- const struct row_sched *rs = &ms->rp[n_rows - 1];
- size_t y = rs->y;
- for (size_t x = rs->x0; x < rs->x1; x++)
- {
- double e;
- if (!next_number (&p, r, &e))
- goto exit;
- gsl_matrix_set (m, y, x, e);
- if (n_dims == 2 && mf->triangle != FULL)
- gsl_matrix_set (m, x, y, e);
- }
- check_eol (mf, &p, r);
- }
- else
- {
- /* Suppress bad input data. We'll issue an error later. */
- p.length = 0;
- }
-
- eof = (!more_tokens (&p, r)
- || !read_id_columns (mf, &p, r, d_next, &rt_next));
- if (eof)
- break;
-
- if (!equal_id_columns (mf, d, d_next) || rt_next != rt)
- break;
- }
- if (!suppress_output)
- matrix_sched_output (mf, rt, m, d, 0, w);
-
- if (n_rows != ms->n_rp)
- parse_error (r, NULL,
- _("Matrix %s had %zu rows but %zu rows were expected."),
- rowtype_name (rt).string, n_rows, ms->n_rp);
- if (eof)
- break;
-
- double *d_tmp = d;
- d = d_next;
- d_next = d_tmp;
-
- rt = rt_next;
- }
-
-exit:
- free (prev);
- gsl_matrix_free (m);
- free (d);
- free (d_next);
-}
-
-static void
-parse_matrix_without_rowtype (const struct matrix_format *mf,
- struct substring *p, struct dfm_reader *r,
- gsl_matrix *m, enum rowtype rowtype, bool pooled,
- int split_num, struct casewriter *w)
-{
- int n_dims = rowtype_dimensions (rowtype);
- const struct matrix_sched *ms = &mf->ms[n_dims];
-
- double *d = xnmalloc (mf->n_input_vars, sizeof *d);
- matrix_sched_init (mf, rowtype, m);
- for (size_t i = 0; i < ms->n_rp; i++)
- {
- int y = ms->rp[i].y;
- int k = 0;
- int h = 0;
- for (size_t j = 0; j < mf->n_input_vars; j++)
- {
- const struct variable *iv = mf->input_vars[j];
- if (k < mf->n_cvars && iv == mf->cvars[k])
- {
- if (k < ms->rp[i].x1 - ms->rp[i].x0)
- {
- double e;
- if (!next_number (p, r, &e))
- goto exit;
-
- int x = k + ms->rp[i].x0;
- gsl_matrix_set (m, y, x, e);
- if (n_dims == 2 && mf->triangle != FULL)
- gsl_matrix_set (m, x, y, e);
- }
- k++;
- continue;
- }
- if (h < mf->n_fvars && iv == mf->fvars[h])
- {
- h++;
- if (pooled)
- {
- d[j] = SYSMIS;
- continue;
- }
- }
-
- double e;
- if (!next_number (p, r, &e))
- goto exit;
- d[j] = e;
- }
- check_eol (mf, p, r);
- }
-
- matrix_sched_output (mf, rowtype, m, d, split_num, w);
-exit:
- free (d);
-}
-
-static void
-parse_data_without_rowtype (const struct matrix_format *mf,
- struct dfm_reader *r, struct casewriter *w)
-{
- if (dfm_eof (r))
- return;
- struct substring p = dfm_get_record (r);
-
- gsl_matrix *m = gsl_matrix_alloc (mf->n_cvars, mf->n_cvars);
-
- int split_num = 1;
- do
- {
- for (size_t i = 0; i < mf->n_contents; )
- {
- size_t j = i;
- if (mf->contents[i].open)
- while (!mf->contents[j].close)
- j++;
-
- if (mf->contents[i].open)
- {
- for (size_t k = 0; k < mf->cells; k++)
- for (size_t h = i; h <= j; h++)
- parse_matrix_without_rowtype (mf, &p, r, m,
- mf->contents[h].rowtype, false,
- split_num, w);
- }
- else
- parse_matrix_without_rowtype (mf, &p, r, m, mf->contents[i].rowtype,
- true, split_num, w);
- i = j + 1;
- }
-
- split_num++;
- }
- while (more_tokens (&p, r));
-
- gsl_matrix_free (m);
-}
-
-/* Parses VARIABLES=varnames for MATRIX DATA and returns a dictionary with the
- named variables in it. */
-static struct dictionary *
-parse_matrix_data_variables (struct lexer *lexer)
-{
- if (!lex_force_match_id (lexer, "VARIABLES"))
- return NULL;
- lex_match (lexer, T_EQUALS);
-
- struct dictionary *dict = dict_create (get_default_encoding ());
-
- size_t n_names = 0;
- char **names = NULL;
- int vars_start = lex_ofs (lexer);
- if (!parse_DATA_LIST_vars (lexer, dict, &names, &n_names, PV_NO_DUPLICATE))
- {
- dict_unref (dict);
- return NULL;
- }
- int vars_end = lex_ofs (lexer) - 1;
-
- for (size_t i = 0; i < n_names; i++)
- if (!strcasecmp (names[i], "ROWTYPE_"))
- dict_create_var_assert (dict, "ROWTYPE_", 8);
- else
- {
- struct variable *var = dict_create_var_assert (dict, names[i], 0);
- var_set_measure (var, MEASURE_SCALE);
- }
-
- for (size_t i = 0; i < n_names; ++i)
- free (names[i]);
- free (names);
-
- if (dict_lookup_var (dict, "VARNAME_"))
- {
- lex_ofs_error (lexer, vars_start, vars_end,
- _("VARIABLES may not include VARNAME_."));
- dict_unref (dict);
- return NULL;
- }
- return dict;
-}
-
-static bool
-parse_matrix_data_subvars (struct lexer *lexer, struct dictionary *dict,
- bool *taken_vars,
- struct variable ***vars, size_t **indexes,
- size_t *n_vars)
-{
- int start_ofs = lex_ofs (lexer);
- if (!parse_variables (lexer, dict, vars, n_vars, 0))
- return false;
- int end_ofs = lex_ofs (lexer) - 1;
-
- *indexes = xnmalloc (*n_vars, sizeof **indexes);
- for (size_t i = 0; i < *n_vars; i++)
- {
- struct variable *v = (*vars)[i];
- if (!strcasecmp (var_get_name (v), "ROWTYPE_"))
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("ROWTYPE_ is not allowed on SPLIT or FACTORS."));
- goto error;
- }
- (*indexes)[i] = var_get_dict_index (v);
-
- bool *tv = &taken_vars[var_get_dict_index (v)];
- if (*tv)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("%s may not appear on both SPLIT and FACTORS."),
- var_get_name (v));
- goto error;
- }
- *tv = true;
-
- var_set_measure (v, MEASURE_NOMINAL);
- var_set_both_formats (v, &(struct fmt_spec) { .type = FMT_F, .w = 4 });
- }
- return true;
-
-error:
- free (*vars);
- *vars = NULL;
- *n_vars = 0;
- free (*indexes);
- *indexes = NULL;
- return false;
-}
-
-int
-cmd_matrix_data (struct lexer *lexer, struct dataset *ds)
-{
- int input_vars_start = lex_ofs (lexer);
- struct dictionary *dict = parse_matrix_data_variables (lexer);
- if (!dict)
- return CMD_FAILURE;
- int input_vars_end = lex_ofs (lexer) - 1;
-
- size_t n_input_vars = dict_get_n_vars (dict);
- struct variable **input_vars = xnmalloc (n_input_vars, sizeof *input_vars);
- for (size_t i = 0; i < n_input_vars; i++)
- input_vars[i] = dict_get_var (dict, i);
-
- int varname_width = 8;
- for (size_t i = 0; i < n_input_vars; i++)
- {
- int w = strlen (var_get_name (input_vars[i]));
- varname_width = MAX (w, varname_width);
- }
-
- struct variable *rowtype = dict_lookup_var (dict, "ROWTYPE_");
- bool input_rowtype = rowtype != NULL;
- if (!rowtype)
- rowtype = dict_create_var_assert (dict, "ROWTYPE_", 8);
-
- struct matrix_format mf = {
- .input_rowtype = input_rowtype,
- .input_vars = input_vars,
- .n_input_vars = n_input_vars,
-
- .rowtype = rowtype,
- .varname = dict_create_var_assert (dict, "VARNAME_", varname_width),
-
- .triangle = LOWER,
- .diagonal = DIAGONAL,
- .n = -1,
- .cells = -1,
- };
-
- bool *taken_vars = XCALLOC (n_input_vars, bool);
- if (input_rowtype)
- taken_vars[var_get_dict_index (rowtype)] = true;
-
- struct file_handle *fh = NULL;
- int n_start = 0;
- int n_end = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
-
- if (lex_match_id (lexer, "N"))
- {
- n_start = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_int_range (lexer, "N", 0, INT_MAX))
- goto error;
-
- mf.n = lex_integer (lexer);
- n_end = lex_ofs (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- int start_ofs = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "LIST"))
- mf.span = false;
- else if (lex_match_id (lexer, "FREE"))
- mf.span = true;
- else if (lex_match_id (lexer, "UPPER"))
- mf.triangle = UPPER;
- else if (lex_match_id (lexer, "LOWER"))
- mf.triangle = LOWER;
- else if (lex_match_id (lexer, "FULL"))
- mf.triangle = FULL;
- else if (lex_match_id (lexer, "DIAGONAL"))
- mf.diagonal = DIAGONAL;
- else if (lex_match_id (lexer, "NODIAGONAL"))
- mf.diagonal = NO_DIAGONAL;
- else
- {
- lex_error_expecting (lexer, "LIST", "FREE",
- "UPPER", "LOWER", "FULL",
- "DIAGONAL", "NODIAGONAL");
- goto error;
- }
- }
- int end_ofs = lex_ofs (lexer) - 1;
-
- if (mf.diagonal == NO_DIAGONAL && mf.triangle == FULL)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("FORMAT=FULL and FORMAT=NODIAGONAL are "
- "mutually exclusive."));
- goto error;
- }
- }
- else if (lex_match_id (lexer, "FILE"))
- {
- lex_match (lexer, T_EQUALS);
- fh_unref (fh);
- fh = fh_parse (lexer, FH_REF_FILE | FH_REF_INLINE, NULL);
- if (!fh)
- goto error;
- }
- else if (!mf.n_svars && lex_match_id (lexer, "SPLIT"))
- {
- lex_match (lexer, T_EQUALS);
- if (!mf.input_rowtype
- && lex_token (lexer) == T_ID
- && !dict_lookup_var (dict, lex_tokcstr (lexer)))
- {
- mf.svars = xmalloc (sizeof *mf.svars);
- mf.svars[0] = dict_create_var_assert (dict, lex_tokcstr (lexer),
- 0);
- var_set_measure (mf.svars[0], MEASURE_NOMINAL);
- var_set_both_formats (
- mf.svars[0], &(struct fmt_spec) { .type = FMT_F, .w = 4 });
- mf.n_svars = 1;
- lex_get (lexer);
- }
- else if (!parse_matrix_data_subvars (lexer, dict, taken_vars,
- &mf.svars, &mf.svar_indexes,
- &mf.n_svars))
- goto error;
- }
- else if (!mf.n_fvars && lex_match_id (lexer, "FACTORS"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_matrix_data_subvars (lexer, dict, taken_vars,
- &mf.fvars, &mf.fvar_indexes,
- &mf.n_fvars))
- goto error;
- }
- else if (lex_match_id (lexer, "CELLS"))
- {
- if (mf.input_rowtype)
- lex_next_msg (lexer, SW,
- -1, -1, _("CELLS is ignored when VARIABLES "
- "includes ROWTYPE_"));
-
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_int_range (lexer, "CELLS", 0, INT_MAX))
- goto error;
-
- mf.cells = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "CONTENTS"))
- {
- lex_match (lexer, T_EQUALS);
-
- size_t allocated_contents = mf.n_contents;
- bool in_parens = false;
- for (;;)
- {
- bool open = !in_parens && lex_match (lexer, T_LPAREN);
- enum rowtype rt;
- if (!rowtype_parse (lexer, &rt))
- {
- if (open || in_parens || (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH))
- {
- const char *rowtypes[] = {
-#define RT(NAME, DIMS) #NAME,
- ROWTYPES
-#undef RT
- "N_VECTOR", "SD",
- };
- lex_error_expecting_array (
- lexer, rowtypes, sizeof rowtypes / sizeof *rowtypes);
- goto error;
- }
- break;
- }
-
- if (open)
- in_parens = true;
-
- if (in_parens)
- mf.factor_rowtype_mask |= 1u << rt;
- else
- mf.pooled_rowtype_mask |= 1u << rt;
-
- bool close = in_parens && lex_match (lexer, T_RPAREN);
- if (close)
- in_parens = false;
-
- if (mf.n_contents >= allocated_contents)
- mf.contents = x2nrealloc (mf.contents, &allocated_contents,
- sizeof *mf.contents);
- mf.contents[mf.n_contents++] = (struct content) {
- .open = open, .rowtype = rt, .close = close
- };
- }
- }
- else
- {
- lex_error_expecting (lexer, "N", "FORMAT", "FILE", "SPLIT", "FACTORS",
- "CELLS", "CONTENTS");
- goto error;
- }
- }
- if (!mf.input_rowtype)
- {
- if (mf.cells < 0)
- {
- if (mf.n_fvars)
- {
- msg (SE, _("CELLS is required when factor variables are specified "
- "and VARIABLES does not include ROWTYPE_."));
- goto error;
- }
- mf.cells = 1;
- }
-
- if (!mf.n_contents)
- {
- msg (SW, _("CONTENTS was not specified and VARIABLES does not "
- "include ROWTYPE_. Assuming CONTENTS=CORR."));
-
- mf.n_contents = 1;
- mf.contents = xmalloc (sizeof *mf.contents);
- *mf.contents = (struct content) { .rowtype = C_CORR };
- }
- }
- mf.cvars = xmalloc (mf.n_input_vars * sizeof *mf.cvars);
- for (size_t i = 0; i < mf.n_input_vars; i++)
- if (!taken_vars[i])
- {
- struct variable *v = input_vars[i];
- mf.cvars[mf.n_cvars++] = v;
- var_set_both_formats (v, &(struct fmt_spec) { .type = FMT_F, .w = 10,
- .d = 4 });
- }
- if (!mf.n_cvars)
- {
- lex_ofs_error (lexer, input_vars_start, input_vars_end,
- _("At least one continuous variable is required."));
- goto error;
- }
- if (mf.input_rowtype)
- {
- for (size_t i = 0; i < mf.n_cvars; i++)
- if (mf.cvars[i] != input_vars[n_input_vars - mf.n_cvars + i])
- {
- lex_ofs_error (lexer, input_vars_start, input_vars_end,
- _("VARIABLES includes ROWTYPE_ but the continuous "
- "variables are not the last ones on VARIABLES."));
- goto error;
- }
- }
- unsigned int rowtype_mask = mf.pooled_rowtype_mask | mf.factor_rowtype_mask;
- if (rowtype_mask & (1u << C_N) && mf.n >= 0)
- {
- lex_ofs_error (lexer, n_start, n_end,
- _("Cannot specify N on CONTENTS along with the "
- "N subcommand."));
- goto error;
- }
-
- struct variable **order = xnmalloc (dict_get_n_vars (dict), sizeof *order);
- size_t n_order = 0;
- for (size_t i = 0; i < mf.n_svars; i++)
- order[n_order++] = mf.svars[i];
- order[n_order++] = mf.rowtype;
- for (size_t i = 0; i < mf.n_fvars; i++)
- order[n_order++] = mf.fvars[i];
- order[n_order++] = mf.varname;
- for (size_t i = 0; i < mf.n_cvars; i++)
- order[n_order++] = mf.cvars[i];
- assert (n_order == dict_get_n_vars (dict));
- dict_reorder_vars (dict, order, n_order);
- free (order);
-
- dict_set_split_vars (dict, mf.svars, mf.n_svars, SPLIT_LAYERED);
-
- schedule_matrices (&mf);
-
- if (fh == NULL)
- fh = fh_inline_file ();
-
- if (lex_end_of_command (lexer) != CMD_SUCCESS)
- goto error;
-
- struct dfm_reader *reader = dfm_open_reader (fh, lexer, NULL);
- if (reader == NULL)
- goto error;
-
- struct casewriter *writer = autopaging_writer_create (dict_get_proto (dict));
- if (mf.input_rowtype)
- parse_data_with_rowtype (&mf, reader, writer);
- else
- parse_data_without_rowtype (&mf, reader, writer);
- dfm_close_reader (reader);
-
- dataset_set_dict (ds, dict);
- dataset_set_source (ds, casewriter_make_reader (writer));
-
- matrix_format_uninit (&mf);
- free (taken_vars);
- fh_unref (fh);
-
- return CMD_SUCCESS;
-
- error:
- matrix_format_uninit (&mf);
- free (taken_vars);
- dict_unref (dict);
- fh_unref (fh);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2017, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "matrix-reader.h"
-
-#include <stdbool.h>
-#include <math.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-struct lexer;
-
-/*
-This module interprets a "data matrix", typically generated by the command
-MATRIX DATA. The dictionary of such a matrix takes the form:
-
- s_0, s_1, ... s_m, ROWTYPE_, VARNAME_, v_0, v_1, .... v_n
-
-where s_0, s_1 ... s_m are the variables defining the splits, and
-v_0, v_1 ... v_n are the continuous variables.
-
-m >= 0; n >= 0
-
-The ROWTYPE_ variable is of type A8.
-The VARNAME_ variable is a string type whose width is not predetermined.
-The variables s_x are of type F4.0 (although this reader accepts any type),
-and v_x are of any numeric type.
-
-The values of the ROWTYPE_ variable are in the set {MEAN, STDDEV, N, CORR, COV}
-and determine the purpose of that case.
-The values of the VARNAME_ variable must correspond to the names of the varibles
-in {v_0, v_1 ... v_n} and indicate the rows of the correlation or covariance
-matrices.
-
-
-
-A typical example is as follows:
-
-s_0 ROWTYPE_ VARNAME_ v_0 v_1 v_2
-
-0 MEAN 5.0000 4.0000 3.0000
-0 STDDEV 1.0000 2.0000 3.0000
-0 N 9.0000 9.0000 9.0000
-0 CORR V1 1.0000 .6000 .7000
-0 CORR V2 .6000 1.0000 .8000
-0 CORR V3 .7000 .8000 1.0000
-1 MEAN 9.0000 8.0000 7.0000
-1 STDDEV 5.0000 6.0000 7.0000
-1 N 9.0000 9.0000 9.0000
-1 CORR V1 1.0000 .4000 .3000
-1 CORR V2 .4000 1.0000 .2000
-1 CORR V3 .3000 .2000 1.0000
-
-*/
-
-void
-matrix_material_uninit (struct matrix_material *mm)
-{
- gsl_matrix_free (mm->corr);
- gsl_matrix_free (mm->cov);
- gsl_matrix_free (mm->n);
- gsl_matrix_free (mm->mean_matrix);
- gsl_matrix_free (mm->var_matrix);
-}
-\f
-static const struct variable *
-find_matrix_string_var (const struct dictionary *dict, const char *name)
-{
- const struct variable *var = dict_lookup_var (dict, name);
- if (var == NULL)
- {
- msg (SE, _("Matrix dataset lacks a variable called %s."), name);
- return NULL;
- }
- if (!var_is_alpha (var))
- {
- msg (SE, _("Matrix dataset variable %s should be of string type."), name);
- return NULL;
- }
- return var;
-}
-
-struct matrix_reader *
-matrix_reader_create (const struct dictionary *dict,
- struct casereader *in_reader)
-{
- const struct variable *varname = find_matrix_string_var (dict, "VARNAME_");
- const struct variable *rowtype = find_matrix_string_var (dict, "ROWTYPE_");
- if (!varname || !rowtype)
- return NULL;
-
- size_t varname_idx = var_get_dict_index (varname);
- size_t rowtype_idx = var_get_dict_index (rowtype);
- if (varname_idx < rowtype_idx)
- {
- msg (SE, _("Variable %s must precede %s in matrix file dictionary."),
- "ROWTYPE_", "VARNAME_");
- return NULL;
- }
-
- for (size_t i = 0; i < dict_get_n_vars (dict); i++)
- {
- const struct variable *v = dict_get_var (dict, i);
- if (!var_is_numeric (v) && v != rowtype && v != varname)
- {
- msg (SE, _("Matrix dataset variable %s should be numeric."),
- var_get_name (v));
- return NULL;
- }
- }
-
- size_t n_vars;
- const struct variable **vars;
- dict_get_vars (dict, &vars, &n_vars, DC_SCRATCH);
-
- /* Different kinds of variables. */
- size_t first_svar = 0;
- size_t n_svars = rowtype_idx;
- size_t first_fvar = rowtype_idx + 1;
- size_t n_fvars = varname_idx - rowtype_idx - 1;
- size_t first_cvar = varname_idx + 1;
- size_t n_cvars = n_vars - varname_idx - 1;
- if (!n_cvars)
- {
- msg (SE, _("Matrix dataset does not have any continuous variables."));
- free (vars);
- return NULL;
- }
-
- struct matrix_reader *mr = xmalloc (sizeof *mr);
- *mr = (struct matrix_reader) {
- .dict = dict,
- .grouper = casegrouper_create_vars (in_reader, &vars[first_svar], n_svars),
- .svars = xmemdup (vars + first_svar, n_svars * sizeof *mr->svars),
- .n_svars = n_svars,
- .rowtype = rowtype,
- .fvars = xmemdup (vars + first_fvar, n_fvars * sizeof *mr->fvars),
- .n_fvars = n_fvars,
- .varname = varname,
- .cvars = xmemdup (vars + first_cvar, n_cvars * sizeof *mr->cvars),
- .n_cvars = n_cvars,
- };
- free (vars);
-
- return mr;
-}
-
-bool
-matrix_reader_destroy (struct matrix_reader *mr)
-{
- if (mr == NULL)
- return false;
- bool ret = casegrouper_destroy (mr->grouper);
- free (mr->svars);
- free (mr->cvars);
- free (mr->fvars);
- free (mr);
- return ret;
-}
-
-
-/*
- Allocates MATRIX if necessary,
- and populates row MROW, from the data in C corresponding to
- variables in VARS. N_VARS is the length of VARS.
-*/
-static void
-matrix_fill_row (gsl_matrix **matrix,
- const struct ccase *c, int mrow,
- const struct variable **vars, size_t n_vars)
-{
- int col;
- if (*matrix == NULL)
- {
- *matrix = gsl_matrix_alloc (n_vars, n_vars);
- gsl_matrix_set_all (*matrix, SYSMIS);
- }
-
- for (col = 0; col < n_vars; ++col)
- {
- const struct variable *cv = vars [col];
- double x = case_num (c, cv);
- assert (col < (*matrix)->size2);
- assert (mrow < (*matrix)->size1);
- gsl_matrix_set (*matrix, mrow, col, x);
- }
-}
-
-static int
-find_varname (const struct variable **vars, int n_vars,
- const char *varname)
-{
- for (int i = 0; i < n_vars; i++)
- if (!strcasecmp (var_get_name (vars[i]), varname))
- return i;
- return -1;
-}
-
-struct substring
-matrix_reader_get_string (const struct ccase *c, const struct variable *var)
-{
- struct substring s = case_ss (c, var);
- ss_rtrim (&s, ss_cstr (CC_SPACES));
- return s;
-}
-
-void
-matrix_reader_set_string (struct ccase *c, const struct variable *var,
- struct substring src)
-{
- struct substring dst = case_ss (c, var);
- for (size_t i = 0; i < dst.length; i++)
- dst.string[i] = i < src.length ? src.string[i] : ' ';
-}
-
-bool
-matrix_reader_next (struct matrix_material *mm, struct matrix_reader *mr,
- struct casereader **groupp)
-{
- struct casereader *group;
- if (!casegrouper_get_next_group (mr->grouper, &group))
- {
- *mm = (struct matrix_material) MATRIX_MATERIAL_INIT;
- if (groupp)
- *groupp = NULL;
- return false;
- }
-
- if (groupp)
- *groupp = casereader_clone (group);
-
- const struct variable **vars = mr->cvars;
- size_t n_vars = mr->n_cvars;
-
- *mm = (struct matrix_material) { .n = NULL };
-
- struct matrix
- {
- const char *name;
- gsl_matrix **m;
- size_t good_rows;
- size_t bad_rows;
- };
- struct matrix matrices[] = {
- { .name = "CORR", .m = &mm->corr },
- { .name = "COV", .m = &mm->cov },
- };
- enum { N_MATRICES = 2 };
-
- struct ccase *c;
- for (; (c = casereader_read (group)); case_unref (c))
- {
- struct substring rowtype = matrix_reader_get_string (c, mr->rowtype);
-
- gsl_matrix **v
- = (ss_equals_case (rowtype, ss_cstr ("N")) ? &mm->n
- : ss_equals_case (rowtype, ss_cstr ("MEAN")) ? &mm->mean_matrix
- : ss_equals_case (rowtype, ss_cstr ("STDDEV")) ? &mm->var_matrix
- : NULL);
- if (v)
- {
- if (!*v)
- *v = gsl_matrix_calloc (n_vars, n_vars);
-
- for (int x = 0; x < n_vars; ++x)
- {
- double n = case_num (c, vars[x]);
- if (v == &mm->var_matrix)
- n *= n;
- for (int y = 0; y < n_vars; ++y)
- gsl_matrix_set (*v, y, x, n);
- }
- continue;
- }
-
- struct matrix *m = NULL;
- for (size_t i = 0; i < N_MATRICES; i++)
- if (ss_equals_case (rowtype, ss_cstr (matrices[i].name)))
- {
- m = &matrices[i];
- break;
- }
- if (m)
- {
- struct substring varname_raw = case_ss (c, mr->varname);
- struct substring varname = ss_cstr (
- recode_string (UTF8, dict_get_encoding (mr->dict),
- varname_raw.string, varname_raw.length));
- ss_rtrim (&varname, ss_cstr (CC_SPACES));
- varname.string[varname.length] = '\0';
-
- int y = find_varname (vars, n_vars, varname.string);
- if (y >= 0)
- {
- m->good_rows++;
- matrix_fill_row (m->m, c, y, vars, n_vars);
- }
- else
- m->bad_rows++;
- ss_dealloc (&varname);
- }
- }
- casereader_destroy (group);
-
- for (size_t i = 0; i < N_MATRICES; i++)
- if (matrices[i].good_rows && matrices[i].good_rows != n_vars)
- msg (SW, _("%s matrix has %zu columns but %zu rows named variables "
- "to be analyzed (and %zu rows named unknown variables)."),
- matrices[i].name, n_vars, matrices[i].good_rows,
- matrices[i].bad_rows);
-
- return true;
-}
-
-int
-cmd_debug_matrix_read (struct lexer *lexer, struct dataset *ds)
-{
- if (lex_match_id (lexer, "NODATA"))
- {
- struct casereader *cr = casewriter_make_reader (
- mem_writer_create (dict_get_proto (dataset_dict (ds))));
- struct matrix_reader *mr = matrix_reader_create (dataset_dict (ds), cr);
- if (!mr)
- {
- casereader_destroy (cr);
- return CMD_FAILURE;
- }
- matrix_reader_destroy (mr);
- return CMD_SUCCESS;
- }
-
- struct matrix_reader *mr = matrix_reader_create (dataset_dict (ds),
- proc_open (ds));
- if (!mr)
- return CMD_FAILURE;
-
- struct pivot_table *pt = pivot_table_create ("Debug Matrix Reader");
-
- enum mm_stat
- {
- MM_CORR,
- MM_COV,
- MM_N,
- MM_MEAN,
- MM_STDDEV,
- };
- const char *mm_stat_names[] = {
- [MM_CORR] = "Correlation",
- [MM_COV] = "Covariance",
- [MM_N] = "N",
- [MM_MEAN] = "Mean",
- [MM_STDDEV] = "Standard Deviation",
- };
- enum { N_STATS = sizeof mm_stat_names / sizeof *mm_stat_names };
- for (size_t i = 0; i < 2; i++)
- {
- struct pivot_dimension *d = pivot_dimension_create (
- pt,
- i ? PIVOT_AXIS_COLUMN : PIVOT_AXIS_ROW,
- i ? "Column" : "Row");
- if (!i)
- pivot_category_create_leaf_rc (d->root, pivot_value_new_text ("Value"),
- PIVOT_RC_CORRELATION);
- for (size_t j = 0; j < mr->n_cvars; j++)
- pivot_category_create_leaf_rc (
- d->root, pivot_value_new_variable (mr->cvars[j]),
- PIVOT_RC_CORRELATION);
- }
-
- struct pivot_dimension *stat = pivot_dimension_create (pt, PIVOT_AXIS_ROW,
- "Statistic");
- for (size_t i = 0; i < N_STATS; i++)
- pivot_category_create_leaf (stat->root,
- pivot_value_new_text (mm_stat_names[i]));
-
- struct pivot_dimension *split = pivot_dimension_create (
- pt, PIVOT_AXIS_ROW, "Split");
-
- int split_num = 0;
-
- struct matrix_material mm = MATRIX_MATERIAL_INIT;
- while (matrix_reader_next (&mm, mr, NULL))
- {
- pivot_category_create_leaf (split->root,
- pivot_value_new_integer (split_num + 1));
-
- const gsl_matrix *m[N_STATS] = {
- [MM_CORR] = mm.corr,
- [MM_COV] = mm.cov,
- [MM_N] = mm.n,
- [MM_MEAN] = mm.mean_matrix,
- [MM_STDDEV] = mm.var_matrix,
- };
-
- for (size_t i = 0; i < N_STATS; i++)
- if (m[i])
- {
- if (i == MM_COV || i == MM_CORR)
- {
- for (size_t y = 0; y < mr->n_cvars; y++)
- for (size_t x = 0; x < mr->n_cvars; x++)
- pivot_table_put4 (
- pt, y + 1, x, i, split_num,
- pivot_value_new_number (gsl_matrix_get (m[i], y, x)));
- }
- else
- for (size_t x = 0; x < mr->n_cvars; x++)
- {
- double n = gsl_matrix_get (m[i], 0, x);
- if (i == MM_STDDEV)
- n = sqrt (n);
- pivot_table_put4 (pt, 0, x, i, split_num,
- pivot_value_new_number (n));
- }
- }
-
- split_num++;
- matrix_material_uninit (&mm);
- }
- pivot_table_submit (pt);
-
- proc_commit (ds);
-
- matrix_reader_destroy (mr);
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2017 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef MATRIX_READER_H
-#define MATRIX_READER_H
-
-#include <gsl/gsl_matrix.h>
-#include <stdbool.h>
-
-struct casereader;
-struct ccase;
-struct dictionary;
-struct matrix_reader;
-struct variable;
-
-struct matrix_reader
- {
- const struct dictionary *dict;
- struct casegrouper *grouper;
-
- /* Variables in 'dict'. */
- const struct variable **svars; /* Split variables. */
- size_t n_svars;
- const struct variable *rowtype; /* ROWTYPE_. */
- const struct variable **fvars; /* Factor variables. */
- size_t n_fvars;
- const struct variable *varname; /* VARNAME_. */
- const struct variable **cvars; /* Continuous variables. */
- size_t n_cvars;
- };
-
-struct matrix_material
-{
- gsl_matrix *corr; /* The correlation matrix */
- gsl_matrix *cov; /* The covariance matrix */
-
- /* Moment matrices */
- gsl_matrix *n; /* MOMENT 0 */
- gsl_matrix *mean_matrix; /* MOMENT 1 */
- gsl_matrix *var_matrix; /* MOMENT 2 */
-};
-
-#define MATRIX_MATERIAL_INIT { .corr = NULL }
-void matrix_material_uninit (struct matrix_material *);
-
-struct matrix_reader *matrix_reader_create (const struct dictionary *,
- struct casereader *);
-
-bool matrix_reader_destroy (struct matrix_reader *mr);
-
-bool matrix_reader_next (struct matrix_material *mm, struct matrix_reader *mr,
- struct casereader **groupp);
-
-struct substring matrix_reader_get_string (const struct ccase *,
- const struct variable *);
-void matrix_reader_set_string (struct ccase *, const struct variable *,
- struct substring);
-
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2021 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <math.h>
-
-#include "data/any-reader.h"
-#include "data/any-writer.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/matrix-reader.h"
-#include "language/lexer/lexer.h"
-#include "language/command.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_mconvert (struct lexer *lexer, struct dataset *ds)
-{
- bool append = false;
- struct file_handle *in = NULL;
- struct file_handle *out = NULL;
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "APPEND"))
- append = true;
- else if (lex_match_id (lexer, "REPLACE"))
- append = false;
- else
- {
- if (lex_match_id (lexer, "MATRIX"))
- lex_match (lexer, T_EQUALS);
-
- struct file_handle **fhp = (lex_match_id (lexer, "IN") ? &in
- : lex_match_id (lexer, "OUT") ? &out
- : NULL);
- if (!fhp)
- {
- lex_error_expecting (lexer, "IN", "OUT", "APPEND", "REPLACE");
- goto error;
- }
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- fh_unref (*fhp);
- if (lex_match (lexer, T_ASTERISK))
- *fhp = NULL;
- else
- {
- *fhp = fh_parse (lexer, FH_REF_FILE, dataset_session (ds));
- if (!*fhp)
- goto error;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
-
- if (!in && !dataset_has_source (ds))
- {
- msg (SE, _("No active file is defined and no external file is "
- "specified on MATRIX=IN."));
- goto error;
- }
-
- struct dictionary *d;
- struct casereader *cr;
- if (in)
- {
- cr = any_reader_open_and_decode (in, NULL, &d, NULL);
- if (!cr)
- goto error;
- }
- else
- {
- d = dict_clone (dataset_dict (ds));
- cr = proc_open (ds);
- }
-
- struct matrix_reader *mr = matrix_reader_create (d, cr);
- if (!mr)
- {
- casereader_destroy (cr);
- dict_unref (d);
- if (!in)
- proc_commit (ds);
- goto error;
- }
-
- struct casewriter *cw;
- if (out)
- {
- cw = any_writer_open (out, d);
- if (!cw)
- {
- matrix_reader_destroy (mr);
- casereader_destroy (cr);
- dict_unref (d);
- if (!in)
- proc_commit (ds);
- goto error;
- }
- }
- else
- cw = autopaging_writer_create (dict_get_proto (d));
-
- for (;;)
- {
- struct matrix_material mm;
- struct casereader *group;
- if (!matrix_reader_next (&mm, mr, &group))
- break;
-
- bool add_corr = mm.cov && !mm.corr;
- bool add_cov = mm.corr && !mm.cov && mm.var_matrix;
- bool add_stddev = add_corr && !mm.var_matrix;
- bool remove_corr = add_cov && !append;
- bool remove_cov = add_corr && !append;
-
- struct ccase *model = casereader_peek (group, 0);
- for (size_t i = 0; i < mr->n_fvars; i++)
- *case_num_rw (model, mr->fvars[i]) = SYSMIS;
-
- for (;;)
- {
- struct ccase *c = casereader_read (group);
- if (!c)
- break;
-
- struct substring rowtype = matrix_reader_get_string (c, mr->rowtype);
- if ((remove_cov && ss_equals_case (rowtype, ss_cstr ("COV")))
- || (remove_corr && ss_equals_case (rowtype, ss_cstr ("CORR"))))
- case_unref (c);
- else
- casewriter_write (cw, c);
- }
- casereader_destroy (group);
-
- if (add_corr)
- {
- for (size_t y = 0; y < mr->n_cvars; y++)
- {
- struct ccase *c = case_clone (model);
- for (size_t x = 0; x < mr->n_cvars; x++)
- {
- double d1 = gsl_matrix_get (mm.cov, x, x);
- double d2 = gsl_matrix_get (mm.cov, y, y);
- double cov = gsl_matrix_get (mm.cov, y, x);
- *case_num_rw (c, mr->cvars[x]) = cov / sqrt (d1 * d2);
- }
- matrix_reader_set_string (c, mr->rowtype, ss_cstr ("CORR"));
- matrix_reader_set_string (c, mr->varname,
- ss_cstr (var_get_name (mr->cvars[y])));
- casewriter_write (cw, c);
- }
- }
-
- if (add_stddev)
- {
- struct ccase *c = case_clone (model);
- for (size_t x = 0; x < mr->n_cvars; x++)
- {
- double variance = gsl_matrix_get (mm.cov, x, x);
- *case_num_rw (c, mr->cvars[x]) = sqrt (variance);
- }
- matrix_reader_set_string (c, mr->rowtype, ss_cstr ("STDDEV"));
- matrix_reader_set_string (c, mr->varname, ss_empty ());
- casewriter_write (cw, c);
- }
-
- if (add_cov)
- {
- for (size_t y = 0; y < mr->n_cvars; y++)
- {
- struct ccase *c = case_clone (model);
- for (size_t x = 0; x < mr->n_cvars; x++)
- {
- double d1 = gsl_matrix_get (mm.var_matrix, x, x);
- double d2 = gsl_matrix_get (mm.var_matrix, y, y);
- double corr = gsl_matrix_get (mm.corr, y, x);
- *case_num_rw (c, mr->cvars[x]) = corr * sqrt (d1 * d2);
- }
- matrix_reader_set_string (c, mr->rowtype, ss_cstr ("COV"));
- matrix_reader_set_string (c, mr->varname,
- ss_cstr (var_get_name (mr->cvars[y])));
- casewriter_write (cw, c);
- }
- }
-
- case_unref (model);
- matrix_material_uninit (&mm);
- }
-
- matrix_reader_destroy (mr);
- if (!in)
- proc_commit (ds);
- if (out)
- casewriter_destroy (cw);
- else
- {
- dataset_set_dict (ds, dict_ref (d));
- dataset_set_source (ds, casewriter_make_reader (cw));
- }
-
- fh_unref (in);
- fh_unref (out);
- dict_unref (d);
- return CMD_SUCCESS;
-
-error:
- fh_unref (in);
- fh_unref (out);
- return CMD_FAILURE;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2010, 2011, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/data-io/placement-parser.h"
-
-#include <assert.h>
-
-#include "data/format.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-
-#include "gl/c-strcase.h"
-#include "gl/xalloc.h"
-#include "gl/xsize.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Extensions to the format specifiers used only for
- placement. */
-enum
- {
- PRS_TYPE_T = SCHAR_MAX - 3, /* Tab to absolute column. */
- PRS_TYPE_X, /* Skip columns. */
- PRS_TYPE_NEW_REC /* Next record. */
- };
-
-static bool fixed_parse_columns (struct lexer *, struct pool *, size_t n_vars,
- enum fmt_use, struct fmt_spec **, size_t *);
-static bool fixed_parse_fortran (struct lexer *l, struct pool *, enum fmt_use,
- struct fmt_spec **, size_t *);
-
-/* Parses Fortran-like or column-based specifications for placing
- variable data in fixed positions in columns and rows, that is,
- formats like those parsed by DATA LIST or PRINT. Returns true
- only if successful.
-
- The formats parsed are either input or output formats, according
- to USE.
-
- If USE is FMT_FOR_INPUT, then T, X, and / "formats" are parsed,
- in addition to regular formats. If USE is FMT_FOR_OUTPUT, then
- T and X "formats" are parsed but not /.
-
- If successful, formats for N_VARS variables are stored in
- *FORMATS, and the number of formats required is stored in
- *FORMAT_CNT. *FORMAT_CNT may be greater than N_VARS because
- of T, X, and / "formats", but success guarantees that exactly
- N_VARS variables will be placed by the output formats. The
- caller should call execute_placement_format to process those
- "formats" in interpreting the output.
-
- Uses POOL for allocation. When the caller is finished
- interpreting *FORMATS, POOL may be destroyed. */
-bool
-parse_var_placements (struct lexer *lexer, struct pool *pool, size_t n_vars,
- enum fmt_use use,
- struct fmt_spec **formats, size_t *n_formats)
-{
- assert (n_vars > 0);
- if (lex_is_number (lexer))
- return fixed_parse_columns (lexer, pool, n_vars, use,
- formats, n_formats);
- else if (lex_match (lexer, T_LPAREN))
- {
- int start_ofs = lex_ofs (lexer);
- if (!fixed_parse_fortran (lexer, pool, use, formats, n_formats))
- return false;
- int end_ofs = lex_ofs (lexer) - 1;
-
- size_t n_assignments = 0;
- for (size_t i = 0; i < *n_formats; i++)
- n_assignments += (*formats)[i].type < FMT_NUMBER_OF_FORMATS;
-
- if (n_assignments != n_vars)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("Number of variables specified (%zu) "
- "differs from number of variable formats (%zu)."),
- n_vars, n_assignments);
- return false;
- }
-
- return true;
- }
- else
- {
- lex_error (lexer, _("SPSS-like or Fortran-like format "
- "specification expected after variable names."));
- return false;
- }
-}
-
-/* Implements parse_var_placements for column-based formats. */
-static bool
-fixed_parse_columns (struct lexer *lexer, struct pool *pool, size_t n_vars,
- enum fmt_use use,
- struct fmt_spec **formats, size_t *n_formats)
-{
- int start_ofs = lex_ofs (lexer);
-
- int fc, lc;
- if (!parse_column_range (lexer, 1, &fc, &lc, NULL))
- return false;
-
- /* Divide columns evenly. */
- int w = (lc - fc + 1) / n_vars;
- if ((lc - fc + 1) % n_vars)
- {
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("The %d columns %d-%d "
- "can't be evenly divided into %zu fields."),
- lc - fc + 1, fc, lc, n_vars);
- return false;
- }
-
- /* Format specifier. */
- enum fmt_type type;
- int d;
- if (lex_match (lexer, T_LPAREN))
- {
- /* Get format type. */
- if (lex_token (lexer) == T_ID)
- {
- if (!parse_format_specifier_name (lexer, &type))
- return false;
- lex_match (lexer, T_COMMA);
- }
- else
- type = FMT_F;
-
- /* Get decimal places. */
- if (lex_is_integer (lexer))
- {
- d = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- d = 0;
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- else
- {
- type = FMT_F;
- d = 0;
- }
- int end_ofs = lex_ofs (lexer) - 1;
-
- struct fmt_spec format = { .type = type, .w = w, .d = d };
- char *error = fmt_check__ (&format, use);
- if (error)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs, "%s", error);
- free (error);
- return false;
- }
-
- *formats = pool_nalloc (pool, n_vars + 1, sizeof **formats);
- *n_formats = n_vars + 1;
- (*formats)[0].type = (enum fmt_type) PRS_TYPE_T;
- (*formats)[0].w = fc;
- for (size_t i = 1; i <= n_vars; i++)
- (*formats)[i] = format;
- return true;
-}
-
-/* Implements parse_var_placements for Fortran-like formats. */
-static bool
-fixed_parse_fortran (struct lexer *lexer, struct pool *pool, enum fmt_use use,
- struct fmt_spec **formats, size_t *n_formats)
-{
- size_t formats_allocated = 0;
- size_t formats_used = 0;
-
- *formats = NULL;
- while (!lex_match (lexer, T_RPAREN))
- {
- struct fmt_spec f;
- struct fmt_spec *new_formats;
- size_t n_new_formats;
- size_t count;
- size_t formats_needed;
-
- /* Parse count. */
- if (lex_is_integer (lexer))
- {
- count = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- count = 1;
-
- /* Parse format specifier. */
- if (lex_match (lexer, T_LPAREN))
- {
- /* Call ourselves recursively to handle parentheses. */
- if (!fixed_parse_fortran (lexer, pool, use,
- &new_formats, &n_new_formats))
- return false;
- }
- else
- {
- new_formats = &f;
- n_new_formats = 1;
- if (use == FMT_FOR_INPUT && lex_match (lexer, T_SLASH))
- f.type = (enum fmt_type) PRS_TYPE_NEW_REC;
- else
- {
- int ofs = lex_ofs (lexer);
- char type[FMT_TYPE_LEN_MAX + 1];
- if (!parse_abstract_format_specifier (lexer, type, &f.w, &f.d))
- return false;
-
- if (!c_strcasecmp (type, "T"))
- f.type = (enum fmt_type) PRS_TYPE_T;
- else if (!c_strcasecmp (type, "X"))
- {
- f.type = (enum fmt_type) PRS_TYPE_X;
- f.w = count;
- count = 1;
- }
- else
- {
- if (!fmt_from_name (type, &f.type))
- {
- lex_ofs_error (lexer, ofs, ofs,
- _("Unknown format type `%s'."), type);
- return false;
- }
- char *error = fmt_check__ (&f, use);
- if (error)
- {
- lex_ofs_error (lexer, ofs, ofs, "%s", error);
- free (error);
- return false;
- }
- }
- }
- }
-
- /* Add COUNT copies of the NEW_FORMAT_CNT formats in
- NEW_FORMATS to FORMATS. */
- if (n_new_formats != 0
- && size_overflow_p (xtimes (xsum (formats_used,
- xtimes (count, n_new_formats)),
- sizeof *formats)))
- xalloc_die ();
- formats_needed = count * n_new_formats;
- if (formats_used + formats_needed > formats_allocated)
- {
- formats_allocated = formats_used + formats_needed;
- *formats = pool_2nrealloc (pool, *formats, &formats_allocated,
- sizeof **formats);
- }
- for (; count > 0; count--)
- {
- memcpy (&(*formats)[formats_used], new_formats,
- sizeof **formats * n_new_formats);
- formats_used += n_new_formats;
- }
-
- lex_match (lexer, T_COMMA);
- }
-
- *n_formats = formats_used;
- return true;
-}
-
-/* Checks whether FORMAT represents one of the special "formats"
- for T, X, or /. If so, updates *RECORD or *COLUMN (or both)
- as appropriate, and returns true. Otherwise, returns false
- without any side effects. */
-bool
-execute_placement_format (const struct fmt_spec *format,
- int *record, int *column)
-{
- switch ((int) format->type)
- {
- case PRS_TYPE_X:
- *column += format->w;
- return true;
-
- case PRS_TYPE_T:
- *column = format->w;
- return true;
-
- case PRS_TYPE_NEW_REC:
- (*record)++;
- *column = 1;
- return true;
-
- default:
- assert (format->type < FMT_NUMBER_OF_FORMATS);
- return false;
- }
-}
-
-static bool
-parse_column__ (struct lexer *lexer, bool negative, int base, int *column)
-{
- assert (base == 0 || base == 1);
-
- if (!lex_force_int (lexer))
- return false;
- long int value = lex_integer (lexer);
- if (negative)
- value = -value;
- lex_get (lexer);
-
- *column = value - base + 1;
- if (*column < 1)
- {
- if (base == 1)
- lex_next_error (lexer, -1, -1,
- _("Column positions for fields must be positive."));
- else
- lex_next_error (lexer, -1, -1,
- _("Column positions for fields must not be negative."));
- return false;
- }
- return true;
-}
-
-/* Parses a BASE-based column using LEXER. Returns true and
- stores a 1-based column number into *COLUMN if successful,
- otherwise emits an error message and returns false.
-
- If BASE is 0, zero-based column numbers are parsed; if BASE is
- 1, 1-based column numbers are parsed. Regardless of BASE, the
- values stored in *FIRST_COLUMN and *LAST_COLUMN are
- 1-based. */
-bool
-parse_column (struct lexer *lexer, int base, int *column)
-{
- return parse_column__ (lexer, false, base, column);
-}
-
-/* Parse a column or a range of columns, specified as a single
- integer or two integers delimited by a dash. Stores the range
- in *FIRST_COLUMN and *LAST_COLUMN. (If only a single integer
- is given, it is stored in both.) If RANGE_SPECIFIED is
- non-null, then *RANGE_SPECIFIED is set to true if the syntax
- contained a dash, false otherwise. Returns true if
- successful, false if the syntax was invalid or the values
- specified did not make sense.
-
- If BASE is 0, zero-based column numbers are parsed; if BASE is
- 1, 1-based column numbers are parsed. Regardless of BASE, the
- values stored in *FIRST_COLUMN and *LAST_COLUMN are
- 1-based. */
-bool
-parse_column_range (struct lexer *lexer, int base,
- int *first_column, int *last_column,
- bool *range_specified)
-{
- int start_ofs = lex_ofs (lexer);
-
- /* First column. */
- if (!parse_column__ (lexer, false, base, first_column))
- return false;
-
- /* Last column. */
- if (lex_is_integer (lexer) && lex_integer (lexer) < 0)
- {
- if (!parse_column__ (lexer, true, base, last_column))
- return false;
-
- if (*last_column < *first_column)
- {
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("The ending column for a field must be "
- "greater than the starting column."));
- return false;
- }
-
- if (range_specified)
- *range_specified = true;
- }
- else
- {
- *last_column = *first_column;
- if (range_specified)
- *range_specified = false;
- }
-
- return true;
-}
-
-/* Parses a (possibly empty) sequence of slashes, each of which
- may be followed by an integer. A slash on its own increases
- *RECORD by 1 and sets *COLUMN to 1. A slash followed by an
- integer sets *RECORD to the integer, as long as that increases
- *RECORD, and sets *COLUMN to 1.
-
- Returns true if successful, false on syntax error. */
-bool
-parse_record_placement (struct lexer *lexer, int *record, int *column)
-{
- while (lex_match (lexer, T_SLASH))
- {
- if (lex_is_number (lexer))
- {
- if (!lex_force_int_range (lexer, NULL, *record + 1, INT_MAX))
- return false;
- *record = lex_integer (lexer);
- lex_get (lexer);
- }
- else
- (*record)++;
- *column = 1;
- }
- assert (*record >= 1);
-
- return true;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef LANGUAGE_DATA_IO_PLACEMENT_PARSER_H
-#define LANGUAGE_DATA_IO_PLACEMENT_PARSER_H 1
-
-#include <stdbool.h>
-#include <stddef.h>
-#include "data/format.h"
-
-struct pool;
-struct lexer;
-
-bool parse_record_placement (struct lexer *, int *record, int *column);
-bool parse_var_placements (struct lexer *, struct pool *, size_t n_vars,
- enum fmt_use,
- struct fmt_spec **, size_t *n_formats);
-bool execute_placement_format (const struct fmt_spec *,
- int *record, int *column);
-bool parse_column (struct lexer *lexer, int base, int *column);
-bool parse_column_range (struct lexer *, int base,
- int *first_column, int *last_column,
- bool *range_specified);
-
-#endif /* language/data-io/placement-parser.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <limits.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/value.h"
-#include "language/command.h"
-#include "language/data-io/data-writer.h"
-#include "language/data-io/file-handle.h"
-#include "language/expressions/public.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "output/driver.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* PRINT SPACE transformation. */
-struct print_space_trns
- {
- struct dfm_writer *writer; /* Output data file. */
- struct expression *expr; /* Number of lines; NULL means 1. */
- struct msg_location *expr_location;
- };
-
-static const struct trns_class print_space_class;
-
-int
-cmd_print_space (struct lexer *lexer, struct dataset *ds)
-{
- struct file_handle *handle = NULL;
- struct expression *expr = NULL;
- struct msg_location *expr_location = NULL;
- char *encoding = NULL;
-
- if (lex_match_id (lexer, "OUTFILE"))
- {
- lex_match (lexer, T_EQUALS);
-
- handle = fh_parse (lexer, FH_REF_FILE, NULL);
- if (handle == NULL)
- return CMD_FAILURE;
-
- if (lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- }
- }
- else
- handle = NULL;
-
- if (lex_token (lexer) != T_ENDCMD)
- {
- int start_ofs = lex_ofs (lexer);
- expr = expr_parse (lexer, ds, VAL_NUMERIC);
- int end_ofs = lex_ofs (lexer) - 1;
- expr_location = lex_ofs_location (lexer, start_ofs, end_ofs);
- if (!expr)
- goto error;
-
- if (lex_token (lexer) != T_ENDCMD)
- {
- lex_error (lexer, _("Syntax error expecting end of command."));
- goto error;
- }
- }
- else
- expr = NULL;
-
- struct dfm_writer *writer = NULL;
- if (handle != NULL)
- {
- writer = dfm_open_writer (handle, encoding);
- if (writer == NULL)
- goto error;
- }
-
- struct print_space_trns *trns = xmalloc (sizeof *trns);
- *trns = (struct print_space_trns) {
- .writer = writer,
- .expr = expr,
- .expr_location = expr_location,
- };
-
- add_transformation (ds, &print_space_class, trns);
- fh_unref (handle);
- free (encoding);
- return CMD_SUCCESS;
-
-error:
- msg_location_destroy (expr_location);
- fh_unref (handle);
- expr_free (expr);
- free (encoding);
- return CMD_FAILURE;
-}
-
-/* Executes a PRINT SPACE transformation. */
-static enum trns_result
-print_space_trns_proc (void *t_, struct ccase **c,
- casenumber case_num UNUSED)
-{
- struct print_space_trns *trns = t_;
- int n;
-
- n = 1;
- if (trns->expr)
- {
- double f = expr_evaluate_num (trns->expr, *c, case_num);
- if (f == SYSMIS)
- msg_at (SW, trns->expr_location,
- _("The expression on %s evaluated to the "
- "system-missing value."), "PRINT SPACE");
- else if (f < 0 || f > INT_MAX)
- msg_at (SW, trns->expr_location,
- _("The expression on %s evaluated to %g."), "PRINT SPACE", f);
- else
- n = f;
- }
-
- while (n--)
- if (trns->writer == NULL)
- output_log ("%s", "");
- else
- dfm_put_record (trns->writer, " ", 1); /* XXX */
-
- if (trns->writer != NULL && dfm_write_error (trns->writer))
- return TRNS_ERROR;
- return TRNS_CONTINUE;
-}
-
-/* Frees a PRINT SPACE transformation.
- Returns true if successful, false if an I/O error occurred. */
-static bool
-print_space_trns_free (void *trns_)
-{
- struct print_space_trns *trns = trns_;
- bool ok = dfm_close_writer (trns->writer);
- expr_free (trns->expr);
- msg_location_destroy (trns->expr_location);
- free (trns);
- return ok;
-}
-
-static const struct trns_class print_space_class = {
- .name = "PRINT SPACE",
- .execute = print_space_trns_proc,
- .destroy = print_space_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-#include <uniwidth.h>
-
-#include "data/case.h"
-#include "data/dataset.h"
-#include "data/data-out.h"
-#include "data/format.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/data-writer.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/placement-parser.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/u8-line.h"
-#include "output/driver.h"
-#include "output/pivot-table.h"
-#include "output/output-item.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-/* Describes what to do when an output field is encountered. */
-enum field_type
- {
- PRT_LITERAL, /* Literal string. */
- PRT_VAR /* Variable. */
- };
-
-/* Describes how to output one field. */
-struct prt_out_spec
- {
- /* All fields. */
- enum field_type type; /* What type of field this is. */
- int record; /* 1-based record number. */
- int first_column; /* 0-based first column. */
- int start_ofs, end_ofs;
-
- /* PRT_VAR only. */
- const struct variable *var; /* Associated variable. */
- struct fmt_spec format; /* Output spec. */
- bool add_space; /* Add trailing space? */
- bool sysmis_as_spaces; /* Output SYSMIS as spaces? */
-
- /* PRT_LITERAL only. */
- struct substring string; /* String to output. */
- int width; /* Width of 'string', in display columns. */
- };
-
-/* PRINT, PRINT EJECT, WRITE private data structure. */
-struct print_trns
- {
- struct pool *pool; /* Stores related data. */
- bool eject; /* Eject page before printing? */
- bool include_prefix; /* Prefix lines with space? */
- const char *encoding; /* Encoding to use for output. */
- struct dfm_writer *writer; /* Output file, NULL=listing file. */
- struct prt_out_spec *specs;
- size_t n_specs;
- size_t n_records; /* Number of records to write. */
- };
-
-enum which_formats
- {
- PRINT,
- WRITE
- };
-
-static const struct trns_class print_binary_trns_class;
-static const struct trns_class print_text_trns_class;
-
-static int cmd_print__ (struct lexer *, struct dataset *,
- enum which_formats, bool eject);
-static bool parse_specs (struct lexer *, struct pool *tmp_pool,
- struct print_trns *, int records_ofs,
- struct dictionary *, enum which_formats);
-static void dump_table (struct print_trns *);
-
-static bool print_trns_free (void *trns_);
-
-static const struct prt_out_spec *find_binary_spec (const struct print_trns *);
-\f
-/* Basic parsing. */
-
-/* Parses PRINT command. */
-int
-cmd_print (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_print__ (lexer, ds, PRINT, false);
-}
-
-/* Parses PRINT EJECT command. */
-int
-cmd_print_eject (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_print__ (lexer, ds, PRINT, true);
-}
-
-/* Parses WRITE command. */
-int
-cmd_write (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_print__ (lexer, ds, WRITE, false);
-}
-
-/* Parses the output commands. */
-static int
-cmd_print__ (struct lexer *lexer, struct dataset *ds,
- enum which_formats which_formats, bool eject)
-{
- bool print_table = false;
- struct file_handle *fh = NULL;
- char *encoding = NULL;
-
- /* Fill in prt to facilitate error-handling. */
- struct pool *pool = pool_create ();
- struct print_trns *trns = pool_alloc (pool, sizeof *trns);
- *trns = (struct print_trns) { .pool = pool, .eject = eject };
- struct pool *tmp_pool = pool_create_subpool (trns->pool);
-
- /* Parse the command options. */
- int records_ofs = 0;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "OUTFILE"))
- {
- lex_match (lexer, T_EQUALS);
-
- fh = fh_parse (lexer, FH_REF_FILE, NULL);
- if (fh == NULL)
- goto error;
- }
- else if (lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "RECORDS"))
- {
- lex_match (lexer, T_EQUALS);
- lex_match (lexer, T_LPAREN);
- if (!lex_force_int_range (lexer, "RECORDS", 0, INT_MAX))
- goto error;
- trns->n_records = lex_integer (lexer);
- records_ofs = lex_ofs (lexer);
- lex_get (lexer);
- lex_match (lexer, T_RPAREN);
- }
- else if (lex_match_id (lexer, "TABLE"))
- print_table = true;
- else if (lex_match_id (lexer, "NOTABLE"))
- print_table = false;
- else
- {
- lex_error_expecting (lexer, "OUTFILE", "ENCODING", "RECORDS",
- "TABLE", "NOTABLE");
- goto error;
- }
- }
-
- /* When PRINT or PRINT EJECT writes to an external file, we
- prefix each line with a space for compatibility. */
- trns->include_prefix = which_formats == PRINT && fh != NULL;
-
- /* Parse variables and strings. */
- if (!parse_specs (lexer, tmp_pool, trns, records_ofs,
- dataset_dict (ds), which_formats))
- goto error;
-
- /* Are there any binary formats?
-
- There are real difficulties figuring out what to do when both binary
- formats and nontrivial encodings enter the picture. So when binary
- formats are present we fall back to much simpler handling. */
- const struct prt_out_spec *binary_spec = find_binary_spec (trns);
- if (binary_spec && !fh)
- {
- lex_ofs_error (lexer, binary_spec->start_ofs, binary_spec->end_ofs,
- _("%s is required when binary formats are specified."),
- "OUTFILE");
- goto error;
- }
-
- if (lex_end_of_command (lexer) != CMD_SUCCESS)
- goto error;
-
- if (fh != NULL)
- {
- trns->writer = dfm_open_writer (fh, encoding);
- if (trns->writer == NULL)
- goto error;
- trns->encoding = dfm_writer_get_encoding (trns->writer);
- }
- else
- trns->encoding = UTF8;
-
- /* Output the variable table if requested. */
- if (print_table)
- dump_table (trns);
-
- /* Put the transformation in the queue. */
- add_transformation (ds, (binary_spec
- ? &print_binary_trns_class
- : &print_text_trns_class), trns);
-
- pool_destroy (tmp_pool);
- fh_unref (fh);
-
- return CMD_SUCCESS;
-
- error:
- print_trns_free (trns);
- fh_unref (fh);
- return CMD_FAILURE;
-}
-\f
-static bool parse_string_argument (struct lexer *, struct print_trns *,
- size_t *allocated_specs,
- int record, int *column);
-static bool parse_variable_argument (struct lexer *, const struct dictionary *,
- struct print_trns *,
- size_t *allocated_specs,
- struct pool *tmp_pool,
- int *record, int *column,
- enum which_formats);
-
-/* Parses all the variable and string specifications on a single
- PRINT, PRINT EJECT, or WRITE command into the prt structure.
- Returns success. */
-static bool
-parse_specs (struct lexer *lexer, struct pool *tmp_pool,
- struct print_trns *trns, int records_ofs, struct dictionary *dict,
- enum which_formats which_formats)
-{
- int record = 0;
- int column = 1;
-
- if (lex_token (lexer) == T_ENDCMD)
- {
- trns->n_records = 1;
- return true;
- }
-
- size_t allocated_specs = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (!parse_record_placement (lexer, &record, &column))
- return false;
-
- bool ok = (lex_is_string (lexer)
- ? parse_string_argument (lexer, trns, &allocated_specs,
- record, &column)
- : parse_variable_argument (lexer, dict, trns, &allocated_specs,
- tmp_pool, &record, &column,
- which_formats));
- if (!ok)
- return 0;
-
- lex_match (lexer, T_COMMA);
- }
-
- if (trns->n_records != 0 && trns->n_records != record)
- lex_ofs_error (lexer, records_ofs, records_ofs,
- _("Output calls for %d records but %zu specified on RECORDS "
- "subcommand."),
- record, trns->n_records);
- trns->n_records = record;
-
- return true;
-}
-
-static struct prt_out_spec *
-add_spec (struct print_trns *trns, size_t *allocated_specs)
-{
- if (trns->n_specs >= *allocated_specs)
- trns->specs = pool_2nrealloc (trns->pool, trns->specs, allocated_specs,
- sizeof *trns->specs);
- return &trns->specs[trns->n_specs++];
-}
-
-/* Parses a string argument to the PRINT commands. Returns success. */
-static bool
-parse_string_argument (struct lexer *lexer, struct print_trns *trns,
- size_t *allocated_specs, int record, int *column)
-{
- struct prt_out_spec *spec = add_spec (trns, allocated_specs);
- *spec = (struct prt_out_spec) {
- .type = PRT_LITERAL,
- .record = record,
- .first_column = *column,
- .string = ss_clone_pool (lex_tokss (lexer), trns->pool),
- .start_ofs = lex_ofs (lexer),
- };
- lex_get (lexer);
-
- /* Parse the included column range. */
- if (lex_is_number (lexer))
- {
- int first_column, last_column;
- bool range_specified;
-
- if (!parse_column_range (lexer, 1,
- &first_column, &last_column, &range_specified))
- return false;
-
- spec->first_column = first_column;
- if (range_specified)
- {
- struct string s;
- ds_init_substring (&s, spec->string);
- ds_set_length (&s, last_column - first_column + 1, ' ');
- spec->string = ss_clone_pool (s.ss, trns->pool);
- ds_destroy (&s);
- }
- }
- spec->end_ofs = lex_ofs (lexer) - 1;
-
- spec->width = u8_width (CHAR_CAST (const uint8_t *, spec->string.string),
- spec->string.length, UTF8);
- *column = spec->first_column + spec->width;
-
- return true;
-}
-
-/* Parses a variable argument to the PRINT commands by passing it off
- to fixed_parse_compatible() or fixed_parse_fortran() as appropriate.
- Returns success. */
-static bool
-parse_variable_argument (struct lexer *lexer, const struct dictionary *dict,
- struct print_trns *trns, size_t *allocated_specs,
- struct pool *tmp_pool, int *record, int *column,
- enum which_formats which_formats)
-{
- const struct variable **vars;
- size_t n_vars;
- if (!parse_variables_const_pool (lexer, tmp_pool, dict,
- &vars, &n_vars, PV_DUPLICATE))
- return false;
-
- struct fmt_spec *formats, *f;
- size_t n_formats;
- bool add_space;
- int formats_start = lex_ofs (lexer);
- if (lex_is_number (lexer) || lex_token (lexer) == T_LPAREN)
- {
- if (!parse_var_placements (lexer, tmp_pool, n_vars, FMT_FOR_OUTPUT,
- &formats, &n_formats))
- return false;
- add_space = false;
- }
- else
- {
- lex_match (lexer, T_ASTERISK);
-
- formats = pool_nmalloc (tmp_pool, n_vars, sizeof *formats);
- n_formats = n_vars;
- for (size_t i = 0; i < n_vars; i++)
- {
- const struct variable *v = vars[i];
- formats[i] = (which_formats == PRINT
- ? *var_get_print_format (v)
- : *var_get_write_format (v));
- }
- add_space = which_formats == PRINT;
- }
- int formats_end = lex_ofs (lexer) - 1;
-
- size_t var_idx = 0;
- for (f = formats; f < &formats[n_formats]; f++)
- if (!execute_placement_format (f, record, column))
- {
- const struct variable *var = vars[var_idx++];
- char *error = fmt_check_width_compat__ (f, var_get_name (var),
- var_get_width (var));
- if (error)
- {
- lex_ofs_error (lexer, formats_start, formats_end, "%s", error);
- free (error);
- return false;
- }
-
- struct prt_out_spec *spec = add_spec (trns, allocated_specs);
- *spec = (struct prt_out_spec) {
- .type = PRT_VAR,
- .record = *record,
- .first_column = *column,
- .var = var,
- .format = *f,
- .add_space = add_space,
-
- /* This is a completely bizarre twist for compatibility: WRITE
- outputs the system-missing value as a field filled with spaces,
- instead of using the normal format that usually contains a
- period. */
- .sysmis_as_spaces = (which_formats == WRITE
- && var_is_numeric (var)
- && (fmt_get_category (f->type)
- != FMT_CAT_BINARY)),
- };
-
- *column += f->w + add_space;
- }
- assert (var_idx == n_vars);
-
- return true;
-}
-
-/* Prints the table produced by the TABLE subcommand to the listing
- file. */
-static void
-dump_table (struct print_trns *trns)
-{
- struct pivot_table *table = pivot_table_create (N_("Print Summary"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Attributes"),
- N_("Record"), N_("Columns"), N_("Format"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t i = 0; i < trns->n_specs; i++)
- {
- const struct prt_out_spec *spec = &trns->specs[i];
- if (spec->type != PRT_VAR)
- continue;
-
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (spec->var));
-
- pivot_table_put2 (table, 0, row,
- pivot_value_new_integer (spec->record));
- int last_column = spec->first_column + spec->format.w - 1;
- pivot_table_put2 (table, 1, row, pivot_value_new_user_text_nocopy (
- xasprintf ("%d-%d",
- spec->first_column, last_column)));
-
- char fmt_string[FMT_STRING_LEN_MAX + 1];
- pivot_table_put2 (table, 2, row, pivot_value_new_user_text (
- fmt_to_string (&spec->format, fmt_string), -1));
- }
-
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_text (N_("N of Records")));
- pivot_table_put2 (table, 0, row,
- pivot_value_new_integer (trns->n_records));
-
- pivot_table_submit (table);
-}
-
-static const struct prt_out_spec *
-find_binary_spec (const struct print_trns *trns)
-{
- for (size_t i = 0; i < trns->n_specs; i++)
- {
- const struct prt_out_spec *spec = &trns->specs[i];
- if (spec->type == PRT_VAR
- && fmt_get_category (spec->format.type) == FMT_CAT_BINARY)
- return spec;
- }
- return NULL;
-}
-\f
-/* Transformation, for all-text output. */
-
-static void print_text_flush_records (struct print_trns *, struct u8_line *,
- int target_record,
- bool *eject, int *record);
-
-/* Performs the transformation inside print_trns T on case C. */
-static enum trns_result
-print_text_trns_proc (void *trns_, struct ccase **c,
- casenumber case_num UNUSED)
-{
- struct print_trns *trns = trns_;
- struct u8_line line;
-
- bool eject = trns->eject;
- int record = 1;
-
- u8_line_init (&line);
- for (size_t i = 0; i < trns->n_specs; i++)
- {
- const struct prt_out_spec *spec = &trns->specs[i];
- int x0 = spec->first_column;
-
- print_text_flush_records (trns, &line, spec->record, &eject, &record);
-
- u8_line_set_length (&line, spec->first_column);
- if (spec->type == PRT_VAR)
- {
- const union value *input = case_data (*c, spec->var);
- int x1;
-
- if (!spec->sysmis_as_spaces || input->f != SYSMIS)
- {
- size_t len;
- int width;
- char *s;
-
- s = data_out (input, var_get_encoding (spec->var),
- &spec->format, settings_get_fmt_settings ());
- len = strlen (s);
- width = u8_width (CHAR_CAST (const uint8_t *, s), len, UTF8);
- x1 = x0 + width;
- u8_line_put (&line, x0, x1, s, len);
- free (s);
- }
- else
- {
- int n = spec->format.w;
-
- x1 = x0 + n;
- memset (u8_line_reserve (&line, x0, x1, n), ' ', n);
- }
-
- if (spec->add_space)
- *u8_line_reserve (&line, x1, x1 + 1, 1) = ' ';
- }
- else
- {
- const struct substring *s = &spec->string;
-
- u8_line_put (&line, x0, x0 + spec->width, s->string, s->length);
- }
- }
- print_text_flush_records (trns, &line, trns->n_records + 1,
- &eject, &record);
- u8_line_destroy (&line);
-
- if (trns->writer != NULL && dfm_write_error (trns->writer))
- return TRNS_ERROR;
- return TRNS_CONTINUE;
-}
-
-/* Advance from *RECORD to TARGET_RECORD, outputting records
- along the way. If *EJECT is true, then the first record
- output is preceded by ejecting the page (and *EJECT is set
- false). */
-static void
-print_text_flush_records (struct print_trns *trns, struct u8_line *line,
- int target_record, bool *eject, int *record)
-{
- for (; target_record > *record; (*record)++)
- {
- char leader = ' ';
-
- if (*eject)
- {
- *eject = false;
- if (trns->writer == NULL)
- output_item_submit (page_break_item_create ());
- else
- leader = '1';
- }
- *u8_line_reserve (line, 0, 1, 1) = leader;
-
- if (trns->writer == NULL)
- output_log ("%s", ds_cstr (&line->s) + 1);
- else
- {
- size_t len = ds_length (&line->s);
- char *s = ds_cstr (&line->s);
-
- if (!trns->include_prefix)
- {
- s++;
- len--;
- }
-
- dfm_put_record_utf8 (trns->writer, s, len);
- }
- }
-}
-\f
-/* Transformation, for output involving binary. */
-
-static void print_binary_flush_records (struct print_trns *,
- struct string *line, int target_record,
- bool *eject, int *record);
-
-/* Performs the transformation inside print_trns T on case C. */
-static enum trns_result
-print_binary_trns_proc (void *trns_, struct ccase **c,
- casenumber case_num UNUSED)
-{
- struct print_trns *trns = trns_;
- bool eject = trns->eject;
- char encoded_space = recode_byte (trns->encoding, C_ENCODING, ' ');
- int record = 1;
- struct string line = DS_EMPTY_INITIALIZER;
-
- ds_put_byte (&line, ' ');
- for (size_t i = 0; i < trns->n_specs; i++)
- {
- const struct prt_out_spec *spec = &trns->specs[i];
- print_binary_flush_records (trns, &line, spec->record, &eject, &record);
-
- ds_set_length (&line, spec->first_column, encoded_space);
- if (spec->type == PRT_VAR)
- {
- const union value *input = case_data (*c, spec->var);
- if (!spec->sysmis_as_spaces || input->f != SYSMIS)
- data_out_recode (input, var_get_encoding (spec->var),
- &spec->format, settings_get_fmt_settings (),
- &line, trns->encoding);
- else
- ds_put_byte_multiple (&line, encoded_space, spec->format.w);
- if (spec->add_space)
- ds_put_byte (&line, encoded_space);
- }
- else
- {
- ds_put_substring (&line, spec->string);
- if (0 != strcmp (trns->encoding, UTF8))
- {
- size_t length = spec->string.length;
- char *data = ss_data (ds_tail (&line, length));
- char *s = recode_string (trns->encoding, UTF8, data, length);
- memcpy (data, s, length);
- free (s);
- }
- }
- }
- print_binary_flush_records (trns, &line, trns->n_records + 1,
- &eject, &record);
- ds_destroy (&line);
-
- if (trns->writer != NULL && dfm_write_error (trns->writer))
- return TRNS_ERROR;
- return TRNS_CONTINUE;
-}
-
-/* Advance from *RECORD to TARGET_RECORD, outputting records
- along the way. If *EJECT is true, then the first record
- output is preceded by ejecting the page (and *EJECT is set
- false). */
-static void
-print_binary_flush_records (struct print_trns *trns, struct string *line,
- int target_record, bool *eject, int *record)
-{
- for (; target_record > *record; (*record)++)
- {
- char *s = ds_cstr (line);
- size_t length = ds_length (line);
- char leader = ' ';
-
- if (*eject)
- {
- *eject = false;
- leader = '1';
- }
- s[0] = recode_byte (trns->encoding, C_ENCODING, leader);
-
- if (!trns->include_prefix)
- {
- s++;
- length--;
- }
- dfm_put_record (trns->writer, s, length);
-
- ds_truncate (line, 1);
- }
-}
-\f
-/* Frees TRNS. */
-static bool
-print_trns_free (void *trns_)
-{
- struct print_trns *trns = trns_;
- bool ok = true;
-
- if (trns->writer != NULL)
- ok = dfm_close_writer (trns->writer);
- pool_destroy (trns->pool);
-
- return ok;
-}
-
-static const struct trns_class print_binary_trns_class = {
- .name = "PRINT",
- .execute = print_binary_trns_proc,
- .destroy = print_trns_free,
-};
-
-static const struct trns_class print_text_trns_class = {
- .name = "PRINT",
- .execute = print_text_trns_proc,
- .destroy = print_trns_free,
-};
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011, 2013, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/case-map.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/csv-file-writer.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/file-name.h"
-#include "data/format.h"
-#include "data/settings.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/trim.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-int
-cmd_save_translate (struct lexer *lexer, struct dataset *ds)
-{
- enum { CSV_FILE = 1, TAB_FILE } type = 0;
-
- struct dictionary *dict = dict_clone (dataset_dict (ds));
- dict_set_names_must_be_ids (dict, false);
-
- struct case_map_stage *stage = case_map_stage_create (dict);
- dict_delete_scratch_vars (dict);
-
- struct file_handle *handle = NULL;
-
- bool replace = false;
-
- bool retain_unselected = true;
- bool recode_user_missing = false;
- bool include_var_names = false;
- bool use_value_labels = false;
- bool use_print_formats = false;
- char decimal = settings_get_fmt_settings ()->decimal;
- char delimiter = 0;
- char qualifier = '"';
-
- int outfile_start = 0;
- int outfile_end = 0;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
-
- if (lex_match_id (lexer, "OUTFILE"))
- {
- outfile_start = lex_ofs (lexer) - 1;
- if (handle != NULL)
- {
- lex_sbc_only_once (lexer, "OUTFILE");
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
-
- handle = fh_parse (lexer, FH_REF_FILE, NULL);
- if (handle == NULL)
- goto error;
- outfile_end = lex_ofs (lexer) - 1;
- }
- else if (lex_match_id (lexer, "TYPE"))
- {
- if (type != 0)
- {
- lex_sbc_only_once (lexer, "TYPE");
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "CSV"))
- type = CSV_FILE;
- else if (lex_match_id (lexer, "TAB"))
- type = TAB_FILE;
- else
- {
- lex_error_expecting (lexer, "CSV", "TAB");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "REPLACE"))
- replace = true;
- else if (lex_match_id (lexer, "FIELDNAMES"))
- include_var_names = true;
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "IGNORE"))
- recode_user_missing = false;
- else if (lex_match_id (lexer, "RECODE"))
- recode_user_missing = true;
- else
- {
- lex_error_expecting (lexer, "IGNORE", "RECODE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "CELLS"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "VALUES"))
- use_value_labels = false;
- else if (lex_match_id (lexer, "LABELS"))
- use_value_labels = true;
- else
- {
- lex_error_expecting (lexer, "VALUES", "LABELS");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "TEXTOPTIONS"))
- {
- lex_match (lexer, T_EQUALS);
- for (;;)
- {
- if (lex_match_id (lexer, "DELIMITER"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
- /* XXX should support multibyte UTF-8 delimiters */
- if (ss_length (lex_tokss (lexer)) != 1)
- {
- lex_error (lexer, _("The %s string must contain exactly "
- "one character."), "DELIMITER");
- goto error;
- }
- delimiter = ss_first (lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "QUALIFIER"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
- /* XXX should support multibyte UTF-8 qualifiers */
- if (ss_length (lex_tokss (lexer)) != 1)
- {
- lex_error (lexer, _("The %s string must contain exactly "
- "one character."), "QUALIFIER");
- goto error;
- }
- qualifier = ss_first (lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "DECIMAL"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "DOT"))
- decimal = '.';
- else if (lex_match_id (lexer, "COMMA"))
- decimal = ',';
- else
- {
- lex_error_expecting (lexer, "DOT", "COMMA");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "PLAIN"))
- use_print_formats = false;
- else if (lex_match_id (lexer, "VARIABLE"))
- use_print_formats = true;
- else
- {
- lex_error_expecting (lexer, "PLAIN", "VARIABLE");
- goto error;
- }
- }
- else
- break;
- }
- }
- else if (lex_match_id (lexer, "UNSELECTED"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "RETAIN"))
- retain_unselected = true;
- else if (lex_match_id (lexer, "DELETE"))
- retain_unselected = false;
- else
- {
- lex_error_expecting (lexer, "RETAIN", "DELETE");
- goto error;
- }
- }
- else if (!parse_dict_trim (lexer, dict))
- goto error;
- }
-
- if (type == 0)
- {
- lex_sbc_missing (lexer, "TYPE");
- goto error;
- }
- else if (handle == NULL)
- {
- lex_sbc_missing (lexer, "OUTFILE");
- goto error;
- }
- else if (!replace && fn_exists (handle))
- {
- lex_ofs_error (lexer, outfile_start, outfile_end,
- _("Output file `%s' exists but %s was not specified."),
- fh_get_file_name (handle), "REPLACE");
- goto error;
- }
-
- dict_delete_scratch_vars (dict);
- dict_compact_values (dict);
-
- struct csv_writer_options csv_opts = {
- .recode_user_missing = recode_user_missing,
- .include_var_names = include_var_names,
- .use_value_labels = use_value_labels,
- .use_print_formats = use_print_formats,
- .decimal = decimal,
- .delimiter = (delimiter ? delimiter
- : type == TAB_FILE ? '\t'
- : decimal == '.' ? ','
- : ';'),
- .qualifier = qualifier,
- };
- struct casewriter *writer = csv_writer_open (handle, dict, &csv_opts);
- if (writer == NULL)
- goto error;
- fh_unref (handle);
-
- struct case_map *map = case_map_stage_get_case_map (stage);
- case_map_stage_destroy (stage);
- if (map != NULL)
- writer = case_map_create_output_translator (map, writer);
- dict_unref (dict);
-
- casereader_transfer (proc_open_filtering (ds, !retain_unselected), writer);
- bool ok = casewriter_destroy (writer);
- ok = proc_commit (ds) && ok;
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-
-error:
- case_map_stage_destroy (stage);
- fh_unref (handle);
- dict_unref (dict);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/any-writer.h"
-#include "data/case-map.h"
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/mdd-writer.h"
-#include "data/por-file-writer.h"
-#include "data/sys-file-writer.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/trim.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/message.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Writing system and portable files. */
-
-/* Type of output file. */
-enum writer_type
- {
- SYSFILE_WRITER, /* System file. */
- PORFILE_WRITER /* Portable file. */
- };
-
-/* Type of a command. */
-enum command_type
- {
- XFORM_CMD, /* Transformation. */
- PROC_CMD /* Procedure. */
- };
-
-static int parse_output_proc (struct lexer *, struct dataset *,
- enum writer_type);
-static int parse_output_trns (struct lexer *, struct dataset *,
- enum writer_type);
-
-int
-cmd_save (struct lexer *lexer, struct dataset *ds)
-{
- return parse_output_proc (lexer, ds, SYSFILE_WRITER);
-}
-
-int
-cmd_save_data_collection (struct lexer *lexer, struct dataset *ds)
-{
- return parse_output_proc (lexer, ds, SYSFILE_WRITER);
-}
-
-int
-cmd_export (struct lexer *lexer, struct dataset *ds)
-{
- return parse_output_proc (lexer, ds, PORFILE_WRITER);
-}
-
-int
-cmd_xsave (struct lexer *lexer, struct dataset *ds)
-{
- return parse_output_trns (lexer, ds, SYSFILE_WRITER);
-}
-
-int
-cmd_xexport (struct lexer *lexer, struct dataset *ds)
-{
- return parse_output_trns (lexer, ds, PORFILE_WRITER);
-}
-\f
-struct output_trns
- {
- struct casewriter *writer; /* Writer. */
- };
-
-static const struct trns_class output_trns_class;
-static struct casewriter *parse_write_command (struct lexer *,
- struct dataset *,
- enum writer_type,
- enum command_type,
- bool *retain_unselected);
-
-/* Parses and performs the SAVE or EXPORT procedure. */
-static int
-parse_output_proc (struct lexer *lexer, struct dataset *ds,
- enum writer_type writer_type)
-{
- bool retain_unselected;
- struct casewriter *output;
- bool ok;
-
- output = parse_write_command (lexer, ds, writer_type, PROC_CMD,
- &retain_unselected);
- if (output == NULL)
- return CMD_CASCADING_FAILURE;
-
- casereader_transfer (proc_open_filtering (ds, !retain_unselected), output);
- ok = casewriter_destroy (output);
- ok = proc_commit (ds) && ok;
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-}
-
-/* Parses the XSAVE or XEXPORT transformation command. */
-static int
-parse_output_trns (struct lexer *lexer, struct dataset *ds, enum writer_type writer_type)
-{
- struct output_trns *t = xmalloc (sizeof *t);
- t->writer = parse_write_command (lexer, ds, writer_type, XFORM_CMD, NULL);
- if (t->writer == NULL)
- {
- free (t);
- return CMD_CASCADING_FAILURE;
- }
-
- add_transformation (ds, &output_trns_class, t);
- return CMD_SUCCESS;
-}
-
-/* Parses SAVE or XSAVE or EXPORT or XEXPORT command.
- WRITER_TYPE identifies the type of file to write,
- and COMMAND_TYPE identifies the type of command.
-
- On success, returns a writer.
- For procedures only, sets *RETAIN_UNSELECTED to true if cases
- that would otherwise be excluded by FILTER or USE should be
- included.
-
- On failure, returns a null pointer. */
-static struct casewriter *
-parse_write_command (struct lexer *lexer, struct dataset *ds,
- enum writer_type writer_type,
- enum command_type command_type,
- bool *retain_unselected)
-{
- /* Common data. */
- struct file_handle *handle; /* Output file. */
- struct file_handle *metadata; /* MDD output file. */
- struct dictionary *dict; /* Dictionary for output file. */
- struct casewriter *writer; /* Writer. */
- struct case_map_stage *stage; /* Preparation for 'map'. */
- struct case_map *map; /* Map from input data to data for writer. */
- const char *sav_name = "";
-
- /* Common options. */
- struct sfm_write_options sysfile_opts;
- struct pfm_write_options porfile_opts;
-
- assert (writer_type == SYSFILE_WRITER || writer_type == PORFILE_WRITER);
- assert (command_type == XFORM_CMD || command_type == PROC_CMD);
- assert ((retain_unselected != NULL) == (command_type == PROC_CMD));
-
- if (command_type == PROC_CMD)
- *retain_unselected = true;
-
- handle = NULL;
- metadata = NULL;
- dict = dict_clone (dataset_dict (ds));
- writer = NULL;
- stage = NULL;
- map = NULL;
- sysfile_opts = sfm_writer_default_options ();
- porfile_opts = pfm_writer_default_options ();
-
- stage = case_map_stage_create (dict);
- dict_delete_scratch_vars (dict);
-
- lex_match (lexer, T_SLASH);
- for (;;)
- {
- if (lex_match_id (lexer, "OUTFILE"))
- {
- if (handle != NULL)
- {
- lex_sbc_only_once (lexer, "OUTFILE");
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
-
- handle = fh_parse (lexer, FH_REF_FILE, NULL);
- if (handle == NULL)
- goto error;
- }
- else if (lex_match_id (lexer, "METADATA"))
- {
- if (metadata != NULL)
- {
- lex_sbc_only_once (lexer, "METADATA");
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
-
- metadata = fh_parse (lexer, FH_REF_FILE, NULL);
- if (metadata == NULL)
- goto error;
- }
- else if (lex_match_id (lexer, "NAMES"))
- {
- /* Not yet implemented. */
- }
- else if (lex_match_id (lexer, "PERMISSIONS"))
- {
- bool cw;
-
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "READONLY"))
- cw = false;
- else if (lex_match_id (lexer, "WRITEABLE"))
- cw = true;
- else
- {
- lex_error_expecting (lexer, "READONLY", "WRITEABLE");
- goto error;
- }
- sysfile_opts.create_writeable = porfile_opts.create_writeable = cw;
- }
- else if (command_type == PROC_CMD && lex_match_id (lexer, "UNSELECTED"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "RETAIN"))
- *retain_unselected = true;
- else if (lex_match_id (lexer, "DELETE"))
- *retain_unselected = false;
- else
- {
- lex_error_expecting (lexer, "RETAIN", "DELETE");
- goto error;
- }
- }
- else if (writer_type == SYSFILE_WRITER
- && lex_match_id (lexer, "COMPRESSED"))
- sysfile_opts.compression = ANY_COMP_SIMPLE;
- else if (writer_type == SYSFILE_WRITER
- && lex_match_id (lexer, "UNCOMPRESSED"))
- sysfile_opts.compression = ANY_COMP_NONE;
- else if (writer_type == SYSFILE_WRITER
- && lex_match_id (lexer, "ZCOMPRESSED"))
- sysfile_opts.compression = ANY_COMP_ZLIB;
- else if (writer_type == SYSFILE_WRITER
- && lex_match_id (lexer, "VERSION"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "VERSION", 2, 3))
- goto error;
- sysfile_opts.version = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (writer_type == PORFILE_WRITER && lex_match_id (lexer, "TYPE"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "COMMUNICATIONS"))
- porfile_opts.type = PFM_COMM;
- else if (lex_match_id (lexer, "TAPE"))
- porfile_opts.type = PFM_TAPE;
- else
- {
- lex_error_expecting (lexer, "COMM", "TAPE");
- goto error;
- }
- }
- else if (writer_type == PORFILE_WRITER && lex_match_id (lexer, "DIGITS"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "DIGITS", 1, INT_MAX))
- goto error;
- porfile_opts.digits = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (!parse_dict_trim (lexer, dict))
- goto error;
-
- if (!lex_match (lexer, T_SLASH))
- break;
- }
- if (lex_end_of_command (lexer) != CMD_SUCCESS)
- goto error;
-
- if (!handle && !metadata)
- {
- msg (SE, _("The OUTFILE or METADATA subcommand is required."));
- goto error;
- }
-
- dict_delete_scratch_vars (dict);
- dict_compact_values (dict);
-
- if (handle)
- {
- if (metadata)
- sav_name = (fh_get_referent (handle) == FH_REF_FILE
- ? fh_get_file_name (handle)
- : fh_get_name (handle));
- if (fh_get_referent (handle) == FH_REF_FILE)
- {
- switch (writer_type)
- {
- case SYSFILE_WRITER:
- writer = sfm_open_writer (handle, dict, sysfile_opts);
- break;
- case PORFILE_WRITER:
- writer = pfm_open_writer (handle, dict, porfile_opts);
- break;
- }
- }
- else
- writer = any_writer_open (handle, dict);
- if (writer == NULL)
- goto error;
- }
-
- if (metadata)
- {
- if (!mdd_write (metadata, dict, sav_name))
- goto error;
- }
-
- map = case_map_stage_get_case_map (stage);
- case_map_stage_destroy (stage);
- if (map != NULL)
- writer = case_map_create_output_translator (map, writer);
- dict_unref (dict);
-
- fh_unref (handle);
- fh_unref (metadata);
- return writer;
-
- error:
- case_map_stage_destroy (stage);
- fh_unref (handle);
- fh_unref (metadata);
- casewriter_destroy (writer);
- dict_unref (dict);
- case_map_destroy (map);
- return NULL;
-}
-
-/* Writes case *C to the system file specified on XSAVE or XEXPORT. */
-static enum trns_result
-output_trns_proc (void *trns_, struct ccase **c, casenumber case_num UNUSED)
-{
- struct output_trns *t = trns_;
- casewriter_write (t->writer, case_ref (*c));
- return TRNS_CONTINUE;
-}
-
-/* Frees an XSAVE or XEXPORT transformation.
- Returns true if successful, false if an I/O error occurred. */
-static bool
-output_trns_free (void *trns_)
-{
- struct output_trns *t = trns_;
- bool ok = casewriter_destroy (t->writer);
- free (t);
- return ok;
-}
-
-static const struct trns_class output_trns_class = {
- .name = "XSAVE/XEXPORT",
- .execute = output_trns_proc,
- .destroy = output_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2007, 2008, 2010, 2011,
- 2019, 2020 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/data-io/trim.h"
-
-#include <stdlib.h>
-
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Commands that read and write system files share a great deal of common
- syntactic structure for rearranging and dropping variables. This function
- parses this syntax and modifies DICT appropriately. Returns true on
- success, false on failure. */
-bool
-parse_dict_trim (struct lexer *lexer, struct dictionary *dict)
-{
- if (lex_match_id (lexer, "MAP"))
- {
- /* FIXME. */
- return true;
- }
- else if (lex_match_id (lexer, "DROP"))
- return parse_dict_drop (lexer, dict);
- else if (lex_match_id (lexer, "KEEP"))
- return parse_dict_keep (lexer, dict);
- else if (lex_match_id (lexer, "RENAME"))
- return parse_dict_rename (lexer, dict);
- else
- {
- lex_error_expecting (lexer, "MAP", "DROP", "KEEP", "RENAME");
- return false;
- }
-}
-
-/* Parses and performs the RENAME subcommand of GET, SAVE, and
- related commands. */
-bool
-parse_dict_rename (struct lexer *lexer, struct dictionary *dict)
-{
- lex_match (lexer, T_EQUALS);
- int start_ofs = lex_ofs (lexer);
-
- struct variable **old_vars = NULL;
- size_t n_old_vars = 0;
-
- char **new_vars = NULL;
- size_t n_new_vars = 0;
-
- bool ok = false;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- size_t prev_n_old = n_old_vars;
- size_t prev_n_new = n_new_vars;
-
- bool paren = lex_match (lexer, T_LPAREN);
- int pv_opts = PV_NO_DUPLICATE | PV_APPEND | (paren ? 0 : PV_SINGLE);
-
- int old_vars_start = lex_ofs (lexer);
- if (!parse_variables (lexer, dict, &old_vars, &n_old_vars, pv_opts))
- goto done;
- int old_vars_end = lex_ofs (lexer) - 1;
-
- if (!lex_force_match (lexer, T_EQUALS))
- goto done;
-
- int new_vars_start = lex_ofs (lexer);
- if (!parse_DATA_LIST_vars (lexer, dict, &new_vars, &n_new_vars, pv_opts))
- goto done;
- int new_vars_end = lex_ofs (lexer) - 1;
-
- if (paren && !lex_force_match (lexer, T_RPAREN))
- goto done;
-
- if (n_new_vars != n_old_vars)
- {
- size_t added_old = n_old_vars - prev_n_old;
- size_t added_new = n_new_vars - prev_n_new;
-
- msg (SE, _("Old and new variable counts do not match."));
- lex_ofs_msg (lexer, SN, old_vars_start, old_vars_end,
- ngettext ("There is %zu old variable.",
- "There are %zu old variables.", added_old),
- added_old);
- lex_ofs_msg (lexer, SN, new_vars_start, new_vars_end,
- ngettext ("There is %zu new variable name.",
- "There are %zu new variable names.",
- added_new),
- added_new);
- goto done;
- }
- }
- int end_ofs = lex_ofs (lexer) - 1;
-
- char *dup_name = NULL;
- if (!dict_rename_vars (dict, old_vars, new_vars, n_new_vars, &dup_name))
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("Requested renaming duplicates variable name %s."),
- dup_name);
- goto done;
- }
- ok = true;
-
-done:
- free (old_vars);
- for (size_t i = 0; i < n_new_vars; ++i)
- free (new_vars[i]);
- free (new_vars);
- return ok;
-}
-
-/* Parses and performs the DROP subcommand of GET, SAVE, and
- related commands.
- Returns true if successful, false on failure.*/
-bool
-parse_dict_drop (struct lexer *lexer, struct dictionary *dict)
-{
- int start_ofs = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
-
- struct variable **v;
- size_t nv;
- if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
- return false;
- dict_delete_vars (dict, v, nv);
- free (v);
-
- if (dict_get_n_vars (dict) == 0)
- {
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("Cannot DROP all variables from dictionary."));
- return false;
- }
- return true;
-}
-
-/* Parses and performs the KEEP subcommand of GET, SAVE, and
- related commands.
- Returns true if successful, false on failure.*/
-bool
-parse_dict_keep (struct lexer *lexer, struct dictionary *dict)
-{
- struct variable **v;
- size_t nv;
- size_t i;
-
- lex_match (lexer, T_EQUALS);
- if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
- return false;
-
- /* Move the specified variables to the beginning. */
- dict_reorder_vars (dict, v, nv);
-
- /* Delete the remaining variables. */
- if (dict_get_n_vars (dict) == nv)
- {
- free (v);
- return true;
- }
-
- v = xnrealloc (v, dict_get_n_vars (dict) - nv, sizeof *v);
- for (i = nv; i < dict_get_n_vars (dict); i++)
- v[i - nv] = dict_get_var (dict, i);
- dict_delete_vars (dict, v, dict_get_n_vars (dict) - nv);
- free (v);
-
- return true;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2007, 2008 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef LANGUAGE_DATA_IO_TRIM_H
-#define LANGUAGE_DATA_IO_TRIM_H
-
-#include <stdbool.h>
-
-struct lexer;
-struct dictionary;
-bool parse_dict_trim (struct lexer *, struct dictionary *);
-bool parse_dict_rename (struct lexer *, struct dictionary *);
-bool parse_dict_drop (struct lexer *, struct dictionary *);
-bool parse_dict_keep (struct lexer *, struct dictionary *);
-
-#endif /* trim.c */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2012, 2014, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/any-reader.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/file-handle-def.h"
-#include "data/missing-values.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Parses and executes APPLY DICTIONARY. */
-int
-cmd_apply_dictionary (struct lexer *lexer, struct dataset *ds)
-{
- lex_match_id (lexer, "FROM");
- lex_match (lexer, T_EQUALS);
-
- struct file_handle *handle = fh_parse (lexer, FH_REF_FILE,
- dataset_session (ds));
- if (!handle)
- return CMD_FAILURE;
-
- struct dictionary *dict;
- struct casereader *reader = any_reader_open_and_decode (handle, NULL, &dict,
- NULL);
- fh_unref (handle);
- if (!reader)
- return CMD_FAILURE;
-
- casereader_destroy (reader);
-
- size_t n_matched = 0;
- for (size_t i = 0; i < dict_get_n_vars (dict); i++)
- {
- const struct variable *s = dict_get_var (dict, i);
- struct variable *t = dict_lookup_var (dataset_dict (ds),
- var_get_name (s));
- if (t == NULL)
- continue;
-
- n_matched++;
- if (var_get_type (s) != var_get_type (t))
- {
- msg (SW, _("Variable %s is %s in target file, but %s in "
- "source file."),
- var_get_name (s),
- var_is_alpha (t) ? _("string") : _("numeric"),
- var_is_alpha (s) ? _("string") : _("numeric"));
- continue;
- }
-
- if (var_has_label (s))
- var_set_label (t, var_get_label (s));
-
- if (var_has_value_labels (s))
- {
- const struct val_labs *value_labels = var_get_value_labels (s);
- if (val_labs_can_set_width (value_labels, var_get_width (t)))
- var_set_value_labels (t, value_labels);
- }
-
- if (var_has_missing_values (s))
- {
- const struct missing_values *miss = var_get_missing_values (s);
- if (mv_is_resizable (miss, var_get_width (t)))
- var_set_missing_values (t, miss);
- }
-
- if (var_is_numeric (s))
- {
- var_set_print_format (t, var_get_print_format (s));
- var_set_write_format (t, var_get_write_format (s));
- }
-
- if (var_has_attributes (s))
- var_set_attributes (t, var_get_attributes (s));
- }
-
- if (!n_matched)
- msg (SW, _("No matching variables found between the source "
- "and target files."));
-
- /* Data file attributes. */
- if (dict_has_attributes (dict))
- dict_set_attributes (dataset_dict (ds), dict_get_attributes (dict));
-
- /* Weighting. */
- if (dict_get_weight (dict) != NULL)
- {
- struct variable *new_weight
- = dict_lookup_var (dataset_dict (ds),
- var_get_name (dict_get_weight (dict)));
-
- if (new_weight != NULL)
- dict_set_weight (dataset_dict (ds), new_weight);
- }
-
- dict_unref (dict);
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2008, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/attributes.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-static enum cmd_result parse_attributes (struct lexer *,
- const char *dict_encoding,
- struct attrset **, size_t n);
-
-/* Parses the DATAFILE ATTRIBUTE command. */
-int
-cmd_datafile_attribute (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct attrset *set = dict_get_attributes (dict);
- return parse_attributes (lexer, dict_get_encoding (dict), &set, 1);
-}
-
-/* Parses the VARIABLE ATTRIBUTE command. */
-int
-cmd_variable_attribute (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- const char *dict_encoding = dict_get_encoding (dict);
-
- do
- {
- struct variable **vars;
- struct attrset **sets;
- size_t n_vars, i;
- bool ok;
-
- if (!lex_force_match_phrase (lexer, "VARIABLES=")
- || !parse_variables (lexer, dict, &vars, &n_vars, PV_NONE))
- return CMD_FAILURE;
-
- sets = xmalloc (n_vars * sizeof *sets);
- for (i = 0; i < n_vars; i++)
- sets[i] = var_get_attributes (vars[i]);
-
- ok = parse_attributes (lexer, dict_encoding, sets, n_vars);
- free (vars);
- free (sets);
- if (!ok)
- return CMD_FAILURE;
- }
- while (lex_match (lexer, T_SLASH));
-
- return CMD_SUCCESS;
-}
-
-/* Parses an attribute name and verifies that it is valid in DICT_ENCODING,
- optionally followed by an index inside square brackets. Returns the
- attribute name or NULL if there was a parse error. Stores the index into
- *INDEX. */
-static char *
-parse_attribute_name (struct lexer *lexer, const char *dict_encoding,
- size_t *index)
-{
- if (!lex_force_id (lexer))
- return NULL;
- char *error = id_is_valid__ (lex_tokcstr (lexer), dict_encoding);
- if (error)
- {
- lex_error (lexer, "%s", error);
- free (error);
- return NULL;
- }
- char *name = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
-
- if (lex_match (lexer, T_LBRACK))
- {
- if (!lex_force_int_range (lexer, NULL, 1, 65535))
- goto error;
- *index = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RBRACK))
- goto error;
- }
- else
- *index = 0;
- return name;
-
-error:
- free (name);
- return NULL;
-}
-
-static bool
-add_attribute (struct lexer *lexer, const char *dict_encoding,
- struct attrset **sets, size_t n)
-{
- const char *value;
- size_t index, i;
- char *name;
-
- name = parse_attribute_name (lexer, dict_encoding, &index);
- if (name == NULL)
- return false;
- if (!lex_force_match (lexer, T_LPAREN) || !lex_force_string (lexer))
- {
- free (name);
- return false;
- }
- value = lex_tokcstr (lexer);
-
- for (i = 0; i < n; i++)
- {
- struct attribute *attr = attrset_lookup (sets[i], name);
- if (attr == NULL)
- {
- attr = attribute_create (name);
- attrset_add (sets[i], attr);
- }
- attribute_set_value (attr, index ? index - 1 : 0, value);
- }
-
- lex_get (lexer);
- free (name);
- return lex_force_match (lexer, T_RPAREN);
-}
-
-static bool
-delete_attribute (struct lexer *lexer, const char *dict_encoding,
- struct attrset **sets, size_t n)
-{
- size_t index, i;
- char *name;
-
- name = parse_attribute_name (lexer, dict_encoding, &index);
- if (name == NULL)
- return false;
-
- for (i = 0; i < n; i++)
- {
- struct attrset *set = sets[i];
- if (index == 0)
- attrset_delete (set, name);
- else
- {
- struct attribute *attr = attrset_lookup (set, name);
- if (attr != NULL)
- {
- attribute_del_value (attr, index - 1);
- if (attribute_get_n_values (attr) == 0)
- attrset_delete (set, name);
- }
- }
- }
-
- free (name);
- return true;
-}
-
-static enum cmd_result
-parse_attributes (struct lexer *lexer, const char *dict_encoding,
- struct attrset **sets, size_t n)
-{
- enum { UNKNOWN, ADD, DELETE } command = UNKNOWN;
- do
- {
- if (lex_match_phrase (lexer, "ATTRIBUTE="))
- command = ADD;
- else if (lex_match_phrase (lexer, "DELETE="))
- command = DELETE;
- else if (command == UNKNOWN)
- {
- lex_error_expecting (lexer, "ATTRIBUTE=", "DELETE=");
- return CMD_FAILURE;
- }
-
- if (!(command == ADD
- ? add_attribute (lexer, dict_encoding, sets, n)
- : delete_attribute (lexer, dict_encoding, sets, n)))
- return CMD_FAILURE;
- }
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
- return CMD_SUCCESS;
-}
+++ /dev/null
-# PSPP - a program for statistical analysis.
-# Copyright (C) 2017 Free Software Foundation, Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-## Process this file with automake to produce Makefile.in -*- makefile -*-
-
-language_dictionary_sources = \
- src/language/dictionary/attributes.c \
- src/language/dictionary/apply-dictionary.c \
- src/language/dictionary/delete-variables.c \
- src/language/dictionary/formats.c \
- src/language/dictionary/missing-values.c \
- src/language/dictionary/mrsets.c \
- src/language/dictionary/numeric.c \
- src/language/dictionary/rename-variables.c \
- src/language/dictionary/sort-variables.c \
- src/language/dictionary/split-file.c \
- src/language/dictionary/split-file.h \
- src/language/dictionary/sys-file-info.c \
- src/language/dictionary/value-labels.c \
- src/language/dictionary/variable-label.c \
- src/language/dictionary/vector.c \
- src/language/dictionary/variable-display.c \
- src/language/dictionary/weight.c
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2007, 2010, 2011, 2013 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Performs DELETE VARIABLES command. */
-int
-cmd_delete_variables (struct lexer *lexer, struct dataset *ds)
-{
- struct variable **vars;
- size_t n_vars;
- bool ok;
-
- if (proc_make_temporary_transformations_permanent (ds))
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("%s may not be used after %s. "
- "Temporary transformations will be made permanent."),
- "DELETE VARIABLES", "TEMPORARY");
-
- if (!parse_variables (lexer, dataset_dict (ds), &vars, &n_vars, PV_NONE))
- goto error;
- if (n_vars == dict_get_n_vars (dataset_dict (ds)))
- {
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("%s may not be used to delete all variables "
- "from the active dataset dictionary. "
- "Use %s instead."), "DELETE VARIABLES", "NEW FILE");
- goto error;
- }
-
- ok = casereader_destroy (proc_open_filtering (ds, false));
- ok = proc_commit (ds) && ok;
- if (!ok)
- goto error;
-
- dict_delete_vars (dataset_dict (ds), vars, n_vars);
-
- /* XXX A bunch of bugs conspire to make executing transformations again here
- necessary, even though it shouldn't be.
-
- Consider the following (which is included in delete-variables.at):
-
- DATA LIST NOTABLE /s1 TO s2 1-2(A).
- BEGIN DATA
- 12
- END DATA.
- DELETE VARIABLES s1.
- NUMERIC n1.
- LIST.
-
- The DATA LIST gives us a caseproto with widths 1,1. DELETE VARIABLES
- deletes the first variable so we now have -1,1. This already is
- technically a problem because proc_casereader_read() calls
- case_unshare_and_resize() from the former to the latter caseproto, and
- these caseprotos are not conformable (which is a requirement for
- case_resize()). It doesn't cause an assert by default because
- case_resize() uses expensive_assert() to check for it though. However, in
- practice we don't see a problem yet because case_resize() only does work
- if the number of widths in the source and dest caseproto are different.
-
- Executing NUMERIC adds a third variable, though, so we have -1,1,0. This
- makes caseproto_resize() notice that there are fewer strings in the new
- caseproto. Therefore it destroys the second one (s2). It should destroy
- the first one (s1), but if the caseprotos were really conformable then it
- would have destroyed the right one. This mistake eventually causes a bad
- memory reference.
-
- Executing transformations a second time after DELETE VARIABLES, like we do
- below, works around the problem because we can never run into a situation
- where we've got both new variables (triggering a resize) and deleted
- variables (triggering the bad free).
-
- We should fix this in a better way. Doing it cleanly seems hard. This
- seems to work for now. */
- ok = casereader_destroy (proc_open_filtering (ds, false));
- ok = proc_commit (ds) && ok;
- if (!ok)
- goto error;
-
- free (vars);
-
- return CMD_SUCCESS;
-
- error:
- free (vars);
- return CMD_CASCADING_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <limits.h>
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/str.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-static int cmd_formats__ (struct lexer *, struct dataset *ds,
- bool print_format, bool write_format);
-
-int
-cmd_print_formats (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_formats__ (lexer, ds, true, false);
-}
-
-int
-cmd_write_formats (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_formats__ (lexer, ds, false, true);
-}
-
-int
-cmd_formats (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_formats__ (lexer, ds, true, true);
-}
-
-static int
-cmd_formats__ (struct lexer *lexer, struct dataset *ds,
- bool print_format, bool write_format)
-{
- /* Variables. */
- struct variable **v;
- size_t cv;
-
- for (;;)
- {
- struct fmt_spec f;
- int width;
- size_t i;
-
- lex_match (lexer, T_SLASH);
-
- if (lex_token (lexer) == T_ENDCMD)
- break;
-
- if (!parse_variables (lexer, dataset_dict (ds), &v, &cv, PV_SAME_WIDTH))
- return CMD_FAILURE;
- width = var_get_width (v[0]);
-
- if (!lex_match (lexer, T_LPAREN))
- {
- lex_error_expecting (lexer, "`('");
- goto fail;
- }
- if (!parse_format_specifier (lexer, &f))
- goto fail;
- char *error = fmt_check_output__ (&f);
- if (!error)
- error = fmt_check_width_compat__ (&f, var_get_name (v[0]), width);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- goto fail;
- }
-
- if (!lex_match (lexer, T_RPAREN))
- {
- lex_error_expecting (lexer, "`)'");
- goto fail;
- }
-
- for (i = 0; i < cv; i++)
- {
- if (print_format)
- var_set_print_format (v[i], &f);
- if (write_format)
- var_set_write_format (v[i], &f);
- }
- free (v);
- v = NULL;
- }
- return CMD_SUCCESS;
-
-fail:
- free (v);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2013, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/data-in.h"
-#include "data/dictionary.h"
-#include "data/dataset.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "data/value.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/token.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_missing_values (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- struct missing_values mv = MV_INIT_EMPTY_NUMERIC;
- struct variable **v = NULL;
- size_t nv;
-
- if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
- goto error;
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- int values_start = lex_ofs (lexer);
- int values_end;
- for (values_end = values_start; ; values_end++)
- {
- enum token_type next = lex_ofs_token (lexer, values_end + 1)->type;
- if (next == T_RPAREN || next == T_ENDCMD || next == T_STOP)
- break;
- }
-
- if (!lex_match (lexer, T_RPAREN))
- {
- if (var_is_numeric (v[0]))
- {
- while (!lex_match (lexer, T_RPAREN))
- {
- enum fmt_type type = var_get_print_format (v[0])->type;
- double x, y;
-
- if (!parse_num_range (lexer, &x, &y, &type))
- goto error;
-
- if (!(x == y
- ? mv_add_num (&mv, x)
- : mv_add_range (&mv, x, y)))
- {
- lex_ofs_error (lexer, values_start, values_end,
- _("Too many numeric missing values. At "
- "most three individual values or one "
- "value and one range are allowed."));
- goto error;
- }
-
- lex_match (lexer, T_COMMA);
- }
- }
- else
- {
- const char *encoding = dict_get_encoding (dict);
-
- mv_init (&mv, MV_MAX_STRING);
- while (!lex_match (lexer, T_RPAREN))
- {
- if (!lex_force_string (lexer))
- goto error;
-
- /* Truncate the string to fit in 8 bytes in the dictionary
- encoding. */
- const char *utf8_s = lex_tokcstr (lexer);
- size_t utf8_len = ss_length (lex_tokss (lexer));
- size_t utf8_trunc_len = utf8_encoding_trunc_len (
- utf8_s, encoding, MV_MAX_STRING);
- if (utf8_trunc_len < utf8_len)
- lex_error (lexer, _("Truncating missing value to maximum "
- "acceptable length (%d bytes)."),
- MV_MAX_STRING);
-
- /* Recode to dictionary encoding and add. */
- char *raw_s = recode_string (encoding, "UTF-8",
- utf8_s, utf8_trunc_len);
- bool ok = mv_add_str (&mv, CHAR_CAST (const uint8_t *, raw_s),
- strlen (raw_s));
- free (raw_s);
- if (!ok)
- {
- lex_ofs_error (lexer, values_start, values_end,
- _("Too many string missing values. "
- "At most three individual values "
- "are allowed."));
- goto error;
- }
-
- lex_get (lexer);
- lex_match (lexer, T_COMMA);
- }
- }
- }
- lex_match (lexer, T_SLASH);
-
- bool ok = true;
- for (size_t i = 0; i < nv; i++)
- {
- int var_width = var_get_width (v[i]);
-
- if (mv_is_resizable (&mv, var_width))
- var_set_missing_values (v[i], &mv);
- else
- {
- ok = false;
- if (!var_width)
- lex_ofs_error (lexer, values_start, values_end,
- _("Cannot assign string missing values to "
- "numeric variable %s."), var_get_name (v[i]));
- else
- lex_ofs_error (lexer, values_start, values_end,
- _("Missing values are too long to assign "
- "to variable %s with width %d."),
- var_get_name (v[i]), var_get_width (v[i]));
- }
- }
- mv_destroy (&mv);
- free (v);
- if (!ok)
- return CMD_FAILURE;
- continue;
-
- error:
- mv_destroy (&mv);
- free (v);
- return CMD_FAILURE;
- }
-
- return CMD_SUCCESS;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/mrset.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/hmap.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "libpspp/stringi-map.h"
-#include "libpspp/stringi-set.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-static bool parse_group (struct lexer *, struct dictionary *, enum mrset_type);
-static bool parse_delete (struct lexer *, struct dictionary *);
-static bool parse_display (struct lexer *, struct dictionary *);
-
-int
-cmd_mrsets (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
-
- while (lex_match (lexer, T_SLASH))
- {
- bool ok;
-
- if (lex_match_id (lexer, "MDGROUP"))
- ok = parse_group (lexer, dict, MRSET_MD);
- else if (lex_match_id (lexer, "MCGROUP"))
- ok = parse_group (lexer, dict, MRSET_MC);
- else if (lex_match_id (lexer, "DELETE"))
- ok = parse_delete (lexer, dict);
- else if (lex_match_id (lexer, "DISPLAY"))
- ok = parse_display (lexer, dict);
- else
- {
- ok = false;
- lex_error_expecting (lexer, "MDGROUP", "MCGROUP",
- "DELETE", "DISPLAY");
- }
-
- if (!ok)
- return CMD_FAILURE;
- }
-
- return CMD_SUCCESS;
-}
-
-static bool
-parse_group (struct lexer *lexer, struct dictionary *dict,
- enum mrset_type type)
-{
- const char *subcommand_name = type == MRSET_MD ? "MDGROUP" : "MCGROUP";
-
- struct mrset *mrset = XZALLOC (struct mrset);
- mrset->type = type;
- mrset->cat_source = MRSET_VARLABELS;
-
- bool labelsource_varlabel = false;
- bool has_value = false;
-
- int vars_start = 0;
- int vars_end = 0;
- int value_ofs = 0;
- int labelsource_start = 0;
- int labelsource_end = 0;
- int label_start = 0;
- int label_end = 0;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "NAME"))
- {
- if (!lex_force_match (lexer, T_EQUALS) || !lex_force_id (lexer))
- goto error;
- char *error = mrset_is_valid_name__ (lex_tokcstr (lexer),
- dict_get_encoding (dict));
- if (error)
- {
- lex_error (lexer, "%s", error);
- free (error);
- goto error;
- }
-
- free (mrset->name);
- mrset->name = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "VARIABLES"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- free (mrset->vars);
- vars_start = lex_ofs (lexer);
- if (!parse_variables (lexer, dict, &mrset->vars, &mrset->n_vars,
- PV_SAME_TYPE | PV_NO_SCRATCH))
- goto error;
- vars_end = lex_ofs (lexer) - 1;
-
- if (mrset->n_vars < 2)
- {
- lex_ofs_error (lexer, vars_start, vars_end,
- _("At least two variables are required."));
- goto error;
- }
- }
- else if (lex_match_id (lexer, "LABEL"))
- {
- label_start = lex_ofs (lexer) - 1;
- if (!lex_force_match (lexer, T_EQUALS) || !lex_force_string (lexer))
- goto error;
- label_end = lex_ofs (lexer);
-
- free (mrset->label);
- mrset->label = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (type == MRSET_MD && lex_match_id (lexer, "LABELSOURCE"))
- {
- if (!lex_force_match_phrase (lexer, "=VARLABEL"))
- goto error;
-
- labelsource_varlabel = true;
- labelsource_start = lex_ofs (lexer) - 3;
- labelsource_end = lex_ofs (lexer) - 1;
- }
- else if (type == MRSET_MD && lex_match_id (lexer, "VALUE"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- has_value = true;
- value_ofs = lex_ofs (lexer);
- if (lex_is_number (lexer))
- {
- if (!lex_is_integer (lexer))
- {
- lex_error (lexer, _("Numeric VALUE must be an integer."));
- goto error;
- }
- value_destroy (&mrset->counted, mrset->width);
- mrset->counted.f = lex_integer (lexer);
- mrset->width = 0;
- }
- else if (lex_is_string (lexer))
- {
- size_t width;
- char *s;
-
- s = recode_string (dict_get_encoding (dict), "UTF-8",
- lex_tokcstr (lexer), -1);
- width = strlen (s);
-
- /* Trim off trailing spaces, but don't trim the string until
- it's empty because a width of 0 is a numeric type. */
- while (width > 1 && s[width - 1] == ' ')
- width--;
-
- value_destroy (&mrset->counted, mrset->width);
- value_init (&mrset->counted, width);
- memcpy (mrset->counted.s, s, width);
- mrset->width = width;
-
- free (s);
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting integer or string."));
- goto error;
- }
- lex_get (lexer);
- }
- else if (type == MRSET_MD && lex_match_id (lexer, "CATEGORYLABELS"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- if (lex_match_id (lexer, "VARLABELS"))
- mrset->cat_source = MRSET_VARLABELS;
- else if (lex_match_id (lexer, "COUNTEDVALUES"))
- mrset->cat_source = MRSET_COUNTEDVALUES;
- else
- {
- lex_error_expecting (lexer, "VARLABELS", "COUNTEDVALUES");
- goto error;
- }
- }
- else
- {
- if (type == MRSET_MD)
- lex_error_expecting (lexer, "NAME", "VARIABLES", "LABEL",
- "LABELSOURCE", "VALUE", "CATEGORYLABELS");
- else
- lex_error_expecting (lexer, "NAME", "VARIABLES", "LABEL");
- goto error;
- }
- }
-
- if (mrset->name == NULL)
- {
- lex_spec_missing (lexer, subcommand_name, "NAME");
- goto error;
- }
- else if (mrset->n_vars == 0)
- {
- lex_spec_missing (lexer, subcommand_name, "VARIABLES");
- goto error;
- }
-
- if (type == MRSET_MD)
- {
- /* Check that VALUE is specified and is valid for the VARIABLES. */
- if (!has_value)
- {
- lex_spec_missing (lexer, subcommand_name, "VALUE");
- goto error;
- }
-
- if (var_is_alpha (mrset->vars[0]) != (mrset->width > 0))
- {
- msg (SE, _("VARIABLES and VALUE must have the same type."));
- if (var_is_alpha (mrset->vars[0]))
- lex_ofs_msg (lexer, SN, vars_start, vars_end,
- _("These are string variables."));
- else
- lex_ofs_msg (lexer, SN, vars_start, vars_end,
- _("These are numeric variables."));
- if (mrset->width > 0)
- lex_ofs_msg (lexer, SN, value_ofs, value_ofs,
- _("This is a string value."));
- else
- lex_ofs_msg (lexer, SN, value_ofs, value_ofs,
- _("This is a numeric value."));
- goto error;
- }
- if (var_is_alpha (mrset->vars[0]))
- {
- const struct variable *shortest_var = NULL;
- int min_width = INT_MAX;
-
- for (size_t i = 0; i < mrset->n_vars; i++)
- {
- int width = var_get_width (mrset->vars[i]);
- if (width < min_width)
- {
- shortest_var = mrset->vars[i];
- min_width = width;
- }
- }
- if (mrset->width > min_width)
- {
- msg (SE, _("The VALUE string must be no longer than the "
- "narrowest variable in the group."));
- lex_ofs_msg (lexer, SN, value_ofs, value_ofs,
- _("The VALUE string is %d bytes long."),
- mrset->width);
- lex_ofs_msg (lexer, SN, vars_start, vars_end,
- _("Variable %s has a width of %d bytes."),
- var_get_name (shortest_var), min_width);
- goto error;
- }
- }
-
- /* Implement LABELSOURCE=VARLABEL. */
- if (labelsource_varlabel)
- {
- if (mrset->cat_source != MRSET_COUNTEDVALUES)
- lex_ofs_msg (lexer, SW, labelsource_start, labelsource_end,
- _("MDGROUP subcommand for group %s specifies "
- "LABELSOURCE=VARLABEL but not "
- "CATEGORYLABELS=COUNTEDVALUES. "
- "Ignoring LABELSOURCE."),
- mrset->name);
- else if (mrset->label)
- {
- msg (SW, _("MDGROUP subcommand for group %s specifies both "
- "LABEL and LABELSOURCE, but only one of these "
- "subcommands may be used at a time. "
- "Ignoring LABELSOURCE."),
- mrset->name);
- lex_ofs_msg (lexer, SN, label_start, label_end,
- _("Here is the %s setting."), "LABEL");
- lex_ofs_msg (lexer, SN, labelsource_start, labelsource_end,
- _("Here is the %s setting."), "LABELSOURCE");
- }
- else
- {
- mrset->label_from_var_label = true;
- for (size_t i = 0; mrset->label == NULL && i < mrset->n_vars; i++)
- {
- const char *label = var_get_label (mrset->vars[i]);
- if (label != NULL)
- {
- mrset->label = xstrdup (label);
- break;
- }
- }
- }
- }
-
- /* Warn if categories cannot be distinguished in output. */
- if (mrset->cat_source == MRSET_VARLABELS)
- {
- struct stringi_map seen;
- size_t i;
-
- stringi_map_init (&seen);
- for (i = 0; i < mrset->n_vars; i++)
- {
- const struct variable *var = mrset->vars[i];
- const char *name = var_get_name (var);
- const char *label = var_get_label (var);
- if (label != NULL)
- {
- const char *other_name = stringi_map_find (&seen, label);
-
- if (other_name == NULL)
- stringi_map_insert (&seen, label, name);
- else
- lex_ofs_msg (lexer, SW, vars_start, vars_end,
- _("Variables %s and %s specified as part of "
- "multiple dichotomy group %s have the same "
- "variable label. Categories represented by "
- "these variables will not be distinguishable "
- "in output."),
- other_name, name, mrset->name);
- }
- }
- stringi_map_destroy (&seen);
- }
- else
- {
- struct stringi_map seen = STRINGI_MAP_INITIALIZER (seen);
- for (size_t i = 0; i < mrset->n_vars; i++)
- {
- const struct variable *var = mrset->vars[i];
- const char *name = var_get_name (var);
-
- union value value;
- value_clone (&value, &mrset->counted, mrset->width);
- value_resize (&value, mrset->width, var_get_width (var));
-
- const struct val_labs *val_labs = var_get_value_labels (var);
- const char *label = val_labs_find (val_labs, &value);
- if (label == NULL)
- lex_ofs_msg (lexer, SW, vars_start, vars_end,
- _("Variable %s specified as part of multiple "
- "dichotomy group %s (which has "
- "CATEGORYLABELS=COUNTEDVALUES) has no value "
- "label for its counted value. This category "
- "will not be distinguishable in output."),
- name, mrset->name);
- else
- {
- const char *other_name = stringi_map_find (&seen, label);
-
- if (other_name == NULL)
- stringi_map_insert (&seen, label, name);
- else
- lex_ofs_msg (lexer, SW, vars_start, vars_end,
- _("Variables %s and %s specified as part of "
- "multiple dichotomy group %s (which has "
- "CATEGORYLABELS=COUNTEDVALUES) have the same "
- "value label for the group's counted "
- "value. These categories will not be "
- "distinguishable in output."),
- other_name, name, mrset->name);
- }
-
- value_destroy (&value, var_get_width (var));
- }
- stringi_map_destroy (&seen);
- }
- }
- else /* MCGROUP. */
- {
- /* Warn if categories cannot be distinguished in output. */
- struct category
- {
- struct hmap_node hmap_node;
- union value value;
- int width;
- const char *label;
- const char *var_name;
- bool warned;
- };
-
- struct hmap categories = HMAP_INITIALIZER (categories);
- for (size_t i = 0; i < mrset->n_vars; i++)
- {
- const struct variable *var = mrset->vars[i];
- const char *name = var_get_name (var);
- int width = var_get_width (var);
- const struct val_labs *val_labs = var_get_value_labels (var);
-
- const struct val_lab *vl;
- for (vl = val_labs_first (val_labs); vl != NULL;
- vl = val_labs_next (val_labs, vl))
- {
- const union value *value = val_lab_get_value (vl);
- const char *label = val_lab_get_label (vl);
- unsigned int hash = value_hash (value, width, 0);
-
- struct category *c;
- HMAP_FOR_EACH_WITH_HASH (c, struct category, hmap_node,
- hash, &categories)
- {
- if (width == c->width
- && value_equal (value, &c->value, width))
- {
- if (!c->warned && utf8_strcasecmp (c->label, label))
- {
- char *s = data_out (value, var_get_encoding (var),
- var_get_print_format (var),
- settings_get_fmt_settings ());
- c->warned = true;
- lex_ofs_msg (lexer, SW, vars_start, vars_end,
- _("Variables specified on MCGROUP should "
- "have the same categories, but %s and "
- "%s (and possibly others) in multiple "
- "category group %s have different "
- "value labels for value %s."),
- c->var_name, name, mrset->name, s);
- free (s);
- }
- goto found;
- }
- }
-
- c = xmalloc (sizeof *c);
- *c = (struct category) {
- .width = width,
- .label = label,
- .var_name = name,
- .warned = false,
- };
- value_clone (&c->value, value, width);
- hmap_insert (&categories, &c->hmap_node, hash);
-
- found: ;
- }
- }
-
- struct category *c, *next;
- HMAP_FOR_EACH_SAFE (c, next, struct category, hmap_node, &categories)
- {
- value_destroy (&c->value, c->width);
- hmap_delete (&categories, &c->hmap_node);
- free (c);
- }
- hmap_destroy (&categories);
- }
-
- dict_add_mrset (dict, mrset);
- return true;
-
-error:
- mrset_destroy (mrset);
- return false;
-}
-
-static bool
-parse_mrset_names (struct lexer *lexer, struct dictionary *dict,
- struct stringi_set *mrset_names)
-{
- if (!lex_force_match_phrase (lexer, "NAME="))
- return false;
-
- stringi_set_init (mrset_names);
- if (lex_match (lexer, T_LBRACK))
- {
- while (!lex_match (lexer, T_RBRACK))
- {
- if (!lex_force_id (lexer))
- return false;
- if (dict_lookup_mrset (dict, lex_tokcstr (lexer)) == NULL)
- {
- lex_error (lexer, _("No multiple response set named %s."),
- lex_tokcstr (lexer));
- stringi_set_destroy (mrset_names);
- return false;
- }
- stringi_set_insert (mrset_names, lex_tokcstr (lexer));
- lex_get (lexer);
- }
- }
- else if (lex_match (lexer, T_ALL))
- {
- size_t n_sets = dict_get_n_mrsets (dict);
- size_t i;
-
- for (i = 0; i < n_sets; i++)
- stringi_set_insert (mrset_names, dict_get_mrset (dict, i)->name);
- }
- else
- {
- lex_error_expecting (lexer, "`['", "ALL");
- return false;
- }
-
- return true;
-}
-
-static bool
-parse_delete (struct lexer *lexer, struct dictionary *dict)
-{
- struct stringi_set mrset_names;
- const char *name;
- if (!parse_mrset_names (lexer, dict, &mrset_names))
- return false;
-
- const struct stringi_set_node *node;
- STRINGI_SET_FOR_EACH (name, node, &mrset_names)
- dict_delete_mrset (dict, name);
- stringi_set_destroy (&mrset_names);
-
- return true;
-}
-
-static bool
-parse_display (struct lexer *lexer, struct dictionary *dict)
-{
- struct stringi_set mrset_names_set;
- if (!parse_mrset_names (lexer, dict, &mrset_names_set))
- return false;
-
- size_t n = stringi_set_count (&mrset_names_set);
- if (n == 0)
- {
- if (dict_get_n_mrsets (dict) == 0)
- lex_next_msg (lexer, SN, -1, -1,
- _("The active dataset dictionary does not contain any "
- "multiple response sets."));
- stringi_set_destroy (&mrset_names_set);
- return true;
- }
-
- struct pivot_table *table = pivot_table_create (
- N_("Multiple Response Sets"));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Attributes"),
- N_("Label"), N_("Encoding"), N_("Counted Value"), N_("Member Variables"));
-
- struct pivot_dimension *mrsets = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Name"));
- mrsets->root->show_label = true;
-
- char **mrset_names = stringi_set_get_sorted_array (&mrset_names_set);
- for (size_t i = 0; i < n; i++)
- {
- const struct mrset *mrset = dict_lookup_mrset (dict, mrset_names[i]);
-
- int row = pivot_category_create_leaf (
- mrsets->root, pivot_value_new_user_text (mrset->name, -1));
-
- if (mrset->label != NULL)
- pivot_table_put2 (table, 0, row,
- pivot_value_new_user_text (mrset->label, -1));
-
- pivot_table_put2 (table, 1, row,
- pivot_value_new_text (mrset->type == MRSET_MD
- ? _("Dichotomies")
- : _("Categories")));
-
- if (mrset->type == MRSET_MD)
- pivot_table_put2 (table, 2, row,
- pivot_value_new_value (
- &mrset->counted, mrset->width,
- &F_8_0, dict_get_encoding (dict)));
-
- /* Variable names. */
- struct string var_names = DS_EMPTY_INITIALIZER;
- for (size_t j = 0; j < mrset->n_vars; j++)
- ds_put_format (&var_names, "%s\n", var_get_name (mrset->vars[j]));
- ds_chomp_byte (&var_names, '\n');
- pivot_table_put2 (table, 3, row,
- pivot_value_new_user_text_nocopy (
- ds_steal_cstr (&var_names)));
- }
- free (mrset_names);
- stringi_set_destroy (&mrset_names_set);
-
- pivot_table_submit (table);
-
- return true;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2010, 2011, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "data/format.h"
-#include "language/command.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Parses the NUMERIC command. */
-int
-cmd_numeric (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- char **v;
- size_t nv;
- int vars_start = lex_ofs (lexer);
- if (!parse_DATA_LIST_vars (lexer, dataset_dict (ds),
- &v, &nv, PV_NO_DUPLICATE))
- return CMD_FAILURE;
- int vars_end = lex_ofs (lexer) - 1;
-
- bool ok = false;
-
- /* Get the optional format specification. */
- struct fmt_spec f = var_default_formats (0);
- if (lex_match (lexer, T_LPAREN))
- {
- if (!parse_format_specifier (lexer, &f))
- goto done;
-
- char *error = fmt_check_output__ (&f);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- goto done;
- }
-
- if (fmt_is_string (f.type))
- {
- char str[FMT_STRING_LEN_MAX + 1];
- lex_next_error (lexer, -1, -1,
- _("Format type %s may not be used with a numeric "
- "variable."), fmt_to_string (&f, str));
- goto done;
- }
-
- if (!lex_match (lexer, T_RPAREN))
- {
- lex_error_expecting (lexer, "`)'");
- goto done;
- }
- }
-
- /* Create each variable. */
- for (size_t i = 0; i < nv; i++)
- {
- struct variable *new_var = dict_create_var (dataset_dict (ds),
- v[i], 0);
- if (!new_var)
- lex_ofs_error (lexer, vars_start, vars_end,
- _("There is already a variable named %s."), v[i]);
- else
- var_set_both_formats (new_var, &f);
- }
- ok = true;
-
- done:
- for (size_t i = 0; i < nv; i++)
- free (v[i]);
- free (v);
- if (!ok)
- return CMD_FAILURE;
- }
- while (lex_match (lexer, T_SLASH));
-
- return CMD_SUCCESS;
-}
-
-/* Parses the STRING command. */
-int
-cmd_string (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- char **v;
- size_t nv;
- int vars_start = lex_ofs (lexer);
- if (!parse_DATA_LIST_vars (lexer, dataset_dict (ds),
- &v, &nv, PV_NO_DUPLICATE))
- return CMD_FAILURE;
- int vars_end = lex_ofs (lexer) - 1;
-
- bool ok = false;
-
- struct fmt_spec f;
- if (!lex_force_match (lexer, T_LPAREN)
- || !parse_format_specifier (lexer, &f))
- goto done;
-
- char *error = fmt_check_type_compat__ (&f, NULL, VAL_STRING);
- if (!error)
- error = fmt_check_output__ (&f);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- goto done;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
-
- /* Create each variable. */
- int width = fmt_var_width (&f);
- for (size_t i = 0; i < nv; i++)
- {
- struct variable *new_var = dict_create_var (dataset_dict (ds), v[i],
- width);
- if (!new_var)
- lex_ofs_error (lexer, vars_start, vars_end,
- _("There is already a variable named %s."), v[i]);
- else
- var_set_both_formats (new_var, &f);
- }
- ok = true;
-
- done:
- for (size_t i = 0; i < nv; i++)
- free (v[i]);
- free (v);
- if (!ok)
- return CMD_FAILURE;
- }
- while (lex_match (lexer, T_SLASH));
-
- return CMD_SUCCESS;
-}
-
-/* Parses the LEAVE command. */
-int
-cmd_leave (struct lexer *lexer, struct dataset *ds)
-{
- struct variable **v;
- size_t nv;
-
- if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
- return CMD_CASCADING_FAILURE;
- for (size_t i = 0; i < nv; i++)
- var_set_leave (v[i], true);
- free (v);
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011, 2015, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_rename_variables (struct lexer *lexer, struct dataset *ds)
-{
- struct variable **vars_to_be_renamed = NULL;
- size_t n_vars_to_be_renamed = 0;
-
- char **new_names = NULL;
- size_t n_new_names = 0;
-
- char *err_name;
-
- int status = CMD_CASCADING_FAILURE;
-
- if (proc_make_temporary_transformations_permanent (ds))
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("%s may not be used after %s. "
- "Temporary transformations will be made permanent."),
- "RENAME VARS", "TEMPORARY");
-
- do
- {
- int opts = PV_APPEND | PV_NO_DUPLICATE;
-
- if (!lex_match (lexer, T_LPAREN))
- opts |= PV_SINGLE;
- int start_ofs = lex_ofs (lexer);
- if (!parse_variables (lexer, dataset_dict (ds),
- &vars_to_be_renamed, &n_vars_to_be_renamed, opts))
- {
- goto lossage;
- }
- if (!lex_force_match (lexer, T_EQUALS))
- {
- goto lossage;
- }
- if (!parse_DATA_LIST_vars (lexer, dataset_dict (ds),
- &new_names, &n_new_names, opts))
- {
- goto lossage;
- }
- int end_ofs = lex_ofs (lexer) - 1;
- if (n_new_names != n_vars_to_be_renamed)
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("Differing number of variables in old name list "
- "(%zu) and in new name list (%zu)."),
- n_vars_to_be_renamed, n_new_names);
- goto lossage;
- }
- if (!(opts & PV_SINGLE) && !lex_force_match (lexer, T_RPAREN))
- {
- goto lossage;
- }
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- if (!dict_rename_vars (dataset_dict (ds),
- vars_to_be_renamed, new_names, n_new_names,
- &err_name))
- {
- lex_ofs_error (lexer, 2, lex_ofs (lexer) - 1,
- _("Renaming would duplicate variable name %s."), err_name);
- goto lossage;
- }
-
- status = CMD_SUCCESS;
-
- lossage:
- free (vars_to_be_renamed);
- if (new_names != NULL)
- {
- size_t i;
- for (i = 0; i < n_new_names; ++i)
- free (new_names[i]);
- free (new_names);
- }
- return status;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/attributes.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-enum key
- {
- K_NAME,
- K_TYPE,
- K_FORMAT,
- K_VAR_LABEL,
- K_VALUE_LABELS,
- K_MISSING_VALUES,
- K_MEASURE,
- K_ROLE,
- K_COLUMNS,
- K_ALIGNMENT,
- K_ATTRIBUTE,
- };
-
-struct criterion
- {
- enum key key;
- char *attr_name;
- bool descending;
- };
-
-static int
-compare_ints (int a, int b)
-{
- return a < b ? -1 : a > b;
-}
-
-static int
-compare_formats (const struct fmt_spec *a, const struct fmt_spec *b)
-{
- int retval = compare_ints (fmt_to_io (a->type), fmt_to_io (b->type));
- if (!retval)
- retval = compare_ints (a->w, b->w);
- if (!retval)
- retval = compare_ints (a->d, b->d);
- return retval;
-}
-
-static int
-compare_var_labels (const struct variable *a, const struct variable *b)
-{
- const char *a_label = var_get_label (a);
- const char *b_label = var_get_label (b);
- return utf8_strcasecmp (a_label ? a_label : "",
- b_label ? b_label : "");
-}
-
-static int
-map_measure (enum measure m)
-{
- return (m == MEASURE_NOMINAL ? 0
- : m == MEASURE_ORDINAL ? 1
- : 2);
-}
-
-static int
-map_role (enum var_role r)
-{
- return (r == ROLE_INPUT ? 0
- : r == ROLE_TARGET ? 1
- : r == ROLE_BOTH ? 2
- : r == ROLE_NONE ? 3
- : r == ROLE_PARTITION ? 4
- : 5);
-}
-
-static const char *
-get_attribute (const struct variable *v, const char *name)
-{
- const struct attrset *set = var_get_attributes (v);
- const struct attribute *attr = attrset_lookup (set, name);
- const char *value = attr ? attribute_get_value (attr, 0) : NULL;
- return value ? value : "";
-}
-
-static int
-map_alignment (enum alignment a)
-{
- return (a == ALIGN_LEFT ? 0
- : a == ALIGN_RIGHT ? 1
- : 2);
-}
-
-static int
-compare_vars (const void *a_, const void *b_, const void *c_)
-{
- const struct variable *const *ap = a_;
- const struct variable *const *bp = b_;
- const struct variable *a = *ap;
- const struct variable *b = *bp;
- const struct criterion *c = c_;
-
- int retval;
- switch (c->key)
- {
- case K_NAME:
- retval = utf8_strverscasecmp (var_get_name (a), var_get_name (b));
- break;
-
- case K_TYPE:
- retval = compare_ints (var_get_width (a), var_get_width (b));
- break;
-
- case K_FORMAT:
- retval = compare_formats (var_get_print_format (a),
- var_get_print_format (b));
- break;
-
- case K_VAR_LABEL:
- retval = compare_var_labels (a, b);
- break;
-
- case K_VALUE_LABELS:
- retval = compare_ints (var_has_value_labels (a),
- var_has_value_labels (b));
- break;
-
- case K_MISSING_VALUES:
- retval = compare_ints (var_has_missing_values (a),
- var_has_missing_values (b));
- break;
-
- case K_MEASURE:
- retval = compare_ints (map_measure (var_get_measure (a)),
- map_measure (var_get_measure (b)));
- break;
-
- case K_ROLE:
- retval = compare_ints (map_role (var_get_role (a)),
- map_role (var_get_role (b)));
- break;
-
- case K_COLUMNS:
- retval = compare_ints (var_get_display_width (a),
- var_get_display_width (b));
- break;
-
- case K_ALIGNMENT:
- retval = compare_ints (map_alignment (var_get_alignment (a)),
- map_alignment (var_get_alignment (b)));
- break;
-
- case K_ATTRIBUTE:
- retval = utf8_strcasecmp (get_attribute (a, c->attr_name),
- get_attribute (b, c->attr_name));
- break;
-
- default:
- NOT_REACHED ();
- }
-
- /* Make this a stable sort. */
- if (!retval)
- {
- size_t a_index = var_get_dict_index (a);
- size_t b_index = var_get_dict_index (b);
- retval = a_index < b_index ? -1 : a_index > b_index;
- }
-
- if (c->descending)
- retval = -retval;
-
- return retval;
-}
-
-/* Performs SORT VARIABLES command. */
-int
-cmd_sort_variables (struct lexer *lexer, struct dataset *ds)
-{
- enum cmd_result result = CMD_FAILURE;
-
- lex_match (lexer, T_BY);
-
- /* Parse sort key. */
- struct criterion c = { .attr_name = NULL };
- if (lex_match_id (lexer, "NAME"))
- c.key = K_NAME;
- else if (lex_match_id (lexer, "TYPE"))
- c.key = K_TYPE;
- else if (lex_match_id (lexer, "FORMAT"))
- c.key = K_FORMAT;
- else if (lex_match_id (lexer, "LABEL"))
- c.key = K_VAR_LABEL;
- else if (lex_match_id (lexer, "VALUES"))
- c.key = K_VALUE_LABELS;
- else if (lex_match_id (lexer, "MISSING"))
- c.key = K_MISSING_VALUES;
- else if (lex_match_id (lexer, "MEASURE"))
- c.key = K_MEASURE;
- else if (lex_match_id (lexer, "ROLE"))
- c.key = K_ROLE;
- else if (lex_match_id (lexer, "COLUMNS"))
- c.key = K_COLUMNS;
- else if (lex_match_id (lexer, "ALIGNMENT"))
- c.key = K_ALIGNMENT;
- else if (lex_match_id (lexer, "ATTRIBUTE"))
- {
- if (!lex_force_id (lexer))
- goto exit;
- c.key = K_ATTRIBUTE;
- c.attr_name = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "NAME", "TYPE", "FORMAT", "LABEL",
- "VALUES", "MISSING", "MEASURE", "ROLE",
- "COLUMNS", "ALIGNMENT", "ATTRIBUTE");
- return CMD_FAILURE;
- }
-
- /* Parse sort direction. */
- if (lex_match (lexer, T_LPAREN))
- {
- if (lex_match_id (lexer, "A") || lex_match_id (lexer, "UP"))
- c.descending = false;
- else if (lex_match_id (lexer, "D") || lex_match_id (lexer, "DOWN"))
- c.descending = true;
- else
- {
- lex_error_expecting (lexer, "A", "D");
- goto exit;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto exit;
- }
- else
- c.descending = false;
-
- /* Sort variables. */
- struct dictionary *d = dataset_dict (ds);
- struct variable **vars;
- size_t n_vars;
- dict_get_vars_mutable (d, &vars, &n_vars, 0);
- sort (vars, n_vars, sizeof *vars, compare_vars, &c);
- dict_reorder_vars (d, CONST_CAST (struct variable *const *, vars), n_vars);
- free (vars);
-
- result = CMD_SUCCESS;
-
-exit:
- free (c.attr_name);
- return result;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-int
-cmd_split_file (struct lexer *lexer, struct dataset *ds)
-{
- if (lex_match_id (lexer, "OFF"))
- dict_clear_split_vars (dataset_dict (ds));
- else
- {
- struct variable **v;
- size_t n;
-
- enum split_type type = (!lex_match_id (lexer, "LAYERED")
- && lex_match_id (lexer, "SEPARATE")
- ? SPLIT_SEPARATE
- : SPLIT_LAYERED);
-
- lex_match (lexer, T_BY);
- int vars_start = lex_ofs (lexer);
- if (!parse_variables (lexer, dataset_dict (ds), &v, &n, PV_NO_DUPLICATE))
- return CMD_CASCADING_FAILURE;
- int vars_end = lex_ofs (lexer) - 1;
-
- if (n > MAX_SPLITS)
- {
- verify (MAX_SPLITS == 8);
- lex_ofs_error (lexer, vars_start, vars_end,
- _("At most 8 split variables may be specified."));
- free (v);
- return CMD_CASCADING_FAILURE;
- }
-
- dict_set_split_vars (dataset_dict (ds), v, n, type);
- free (v);
- }
-
- return CMD_SUCCESS;
-}
-
-/* Dumps out the values of all the split variables for the case C. */
-void
-output_split_file_values (const struct dataset *ds, const struct ccase *c)
-{
- const struct dictionary *dict = dataset_dict (ds);
- size_t n_vars = dict_get_n_splits (dict);
- if (n_vars == 0)
- return;
-
- struct pivot_table *table = pivot_table_create (N_("Split Values"));
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Value"),
- N_("Value"));
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
- variables->root->show_label = true;
-
- for (size_t i = 0; i < n_vars; i++)
- {
- const struct variable *v = dict_get_split_vars (dict)[i];
- int row = pivot_category_create_leaf (variables->root,
- pivot_value_new_variable (v));
-
- pivot_table_put2 (table, 0, row,
- pivot_value_new_var_value (v, case_data (c, v)));
- }
-
- pivot_table_submit (table);
-}
-
-/* Dumps out the values of all the split variables for the first case in
- READER. */
-void
-output_split_file_values_peek (const struct dataset *ds,
- const struct casereader *reader)
-{
- struct ccase *c = casereader_peek (reader, 0);
- if (c)
- {
- output_split_file_values (ds, c);
- case_unref (c);
- }
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef SPLIT_FILE_H
-#define SPLIT_FILE_H 1
-
-void output_split_file_values (const struct dataset *, const struct ccase *);
-void output_split_file_values_peek (const struct dataset *,
- const struct casereader *);
-
-#endif /* split-file.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-#include <config.h>
-
-#include <ctype.h>
-#include <errno.h>
-#include <float.h>
-#include <stdlib.h>
-
-#include "data/any-reader.h"
-#include "data/attributes.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/file-handle-def.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "data/vector.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/string-array.h"
-#include "output/pivot-table.h"
-#include "output/output-item.h"
-
-#include "gl/count-one-bits.h"
-#include "gl/localcharset.h"
-#include "gl/intprops.h"
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-/* Information to include in displaying a dictionary. */
-enum
- {
- /* Variable table. */
- DF_NAME = 1 << 0,
- DF_POSITION = 1 << 1,
- DF_LABEL = 1 << 2,
- DF_MEASUREMENT_LEVEL = 1 << 3,
- DF_ROLE = 1 << 4,
- DF_WIDTH = 1 << 5,
- DF_ALIGNMENT = 1 << 6,
- DF_PRINT_FORMAT = 1 << 7,
- DF_WRITE_FORMAT = 1 << 8,
- DF_MISSING_VALUES = 1 << 9,
-#define DF_ALL_VARIABLE ((1 << 10) - 1)
-
- /* Value labels table. */
- DF_VALUE_LABELS = 1 << 10,
-
- /* Attribute table. */
- DF_AT_ATTRIBUTES = 1 << 11, /* Attributes whose names begin with @. */
- DF_ATTRIBUTES = 1 << 12, /* All other attributes. */
- };
-
-static void display_variables (const struct variable **, size_t, int flags);
-static void display_value_labels (const struct variable **, size_t);
-static void display_attributes (const struct attrset *,
- const struct variable **, size_t, int flags);
-
-static void report_encodings (const struct file_handle *, struct pool *,
- char **titles, bool *ids,
- char **strings, size_t n_strings);
-
-static char *get_documents_as_string (const struct dictionary *);
-
-static void
-add_row (struct pivot_table *table, const char *attribute,
- struct pivot_value *value)
-{
- int row = pivot_category_create_leaf (table->dimensions[0]->root,
- pivot_value_new_text (attribute));
- if (value)
- pivot_table_put1 (table, row, value);
-}
-
-/* SYSFILE INFO utility. */
-int
-cmd_sysfile_info (struct lexer *lexer, struct dataset *ds)
-{
- struct any_reader *any_reader;
- struct file_handle *h;
- struct dictionary *d;
- struct casereader *reader;
- struct any_read_info info;
- char *encoding;
-
- h = NULL;
- encoding = NULL;
- for (;;)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "FILE") || lex_is_string (lexer))
- {
- lex_match (lexer, T_EQUALS);
-
- fh_unref (h);
- h = fh_parse (lexer, FH_REF_FILE, NULL);
- if (h == NULL)
- goto error;
- }
- else if (lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_string (lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- }
- else
- break;
- }
-
- if (h == NULL)
- {
- lex_sbc_missing (lexer, "FILE");
- goto error;
- }
-
- any_reader = any_reader_open (h);
- if (!any_reader)
- {
- free (encoding);
- return CMD_FAILURE;
- }
-
- if (encoding && !strcasecmp (encoding, "detect"))
- {
- char **titles, **strings;
- struct pool *pool;
- size_t n_strings;
- bool *ids;
-
- pool = pool_create ();
- n_strings = any_reader_get_strings (any_reader, pool,
- &titles, &ids, &strings);
- any_reader_close (any_reader);
-
- report_encodings (h, pool, titles, ids, strings, n_strings);
- fh_unref (h);
- pool_destroy (pool);
- free (encoding);
-
- return CMD_SUCCESS;
- }
-
- reader = any_reader_decode (any_reader, encoding, &d, &info);
- if (!reader)
- goto error;
- casereader_destroy (reader);
-
- struct pivot_table *table = pivot_table_create (N_("File Information"));
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Attribute"));
-
- add_row (table, N_("File"),
- pivot_value_new_user_text (fh_get_file_name (h), -1));
-
- const char *label = dict_get_label (d);
- add_row (table, N_("Label"),
- label ? pivot_value_new_user_text (label, -1) : NULL);
-
- add_row (table, N_("Created"),
- pivot_value_new_user_text_nocopy (
- xasprintf ("%s %s by %s", info.creation_date,
- info.creation_time, info.product)));
-
- if (info.product_ext)
- add_row (table, N_("Product"),
- pivot_value_new_user_text (info.product_ext, -1));
-
- add_row (table, N_("Integer Format"),
- pivot_value_new_text (
- info.integer_format == INTEGER_MSB_FIRST ? N_("Big Endian")
- : info.integer_format == INTEGER_LSB_FIRST ? N_("Little Endian")
- : N_("Unknown")));
-
- add_row (table, N_("Real Format"),
- pivot_value_new_text (
- info.float_format == FLOAT_IEEE_DOUBLE_LE ? N_("IEEE 754 LE.")
- : info.float_format == FLOAT_IEEE_DOUBLE_BE ? N_("IEEE 754 BE.")
- : info.float_format == FLOAT_VAX_D ? N_("VAX D.")
- : info.float_format == FLOAT_VAX_G ? N_("VAX G.")
- : info.float_format == FLOAT_Z_LONG ? N_("IBM 390 Hex Long.")
- : N_("Unknown")));
-
- add_row (table, N_("Variables"),
- pivot_value_new_integer (dict_get_n_vars (d)));
-
- add_row (table, N_("Cases"),
- (info.n_cases == -1
- ? pivot_value_new_text (N_("Unknown"))
- : pivot_value_new_integer (info.n_cases)));
-
- add_row (table, N_("Type"),
- pivot_value_new_text (info.klass->name));
-
- struct variable *weight_var = dict_get_weight (d);
- add_row (table, N_("Weight"),
- (weight_var
- ? pivot_value_new_variable (weight_var)
- : pivot_value_new_text (N_("Not weighted"))));
-
- add_row (table, N_("Compression"),
- (info.compression == ANY_COMP_NONE
- ? pivot_value_new_text (N_("None"))
- : pivot_value_new_user_text (
- info.compression == ANY_COMP_SIMPLE ? "SAV" : "ZSAV", -1)));
-
- add_row (table, N_("Encoding"),
- pivot_value_new_user_text (dict_get_encoding (d), -1));
-
- if (dict_get_document_n_lines (d) > 0)
- add_row (table, N_("Documents"),
- pivot_value_new_user_text_nocopy (get_documents_as_string (d)));
-
- pivot_table_submit (table);
-
- size_t n_vars = dict_get_n_vars (d);
- const struct variable **vars = xnmalloc (n_vars, sizeof *vars);
- for (size_t i = 0; i < dict_get_n_vars (d); i++)
- vars[i] = dict_get_var (d, i);
- display_variables (vars, n_vars, DF_ALL_VARIABLE);
- display_value_labels (vars, n_vars);
- display_attributes (dict_get_attributes (dataset_dict (ds)),
- vars, n_vars, DF_ATTRIBUTES);
- free (vars);
-
- dict_unref (d);
-
- fh_unref (h);
- free (encoding);
- any_read_info_destroy (&info);
- return CMD_SUCCESS;
-
-error:
- fh_unref (h);
- free (encoding);
- return CMD_FAILURE;
-}
-\f
-/* DISPLAY utility. */
-
-static void display_macros (void);
-static void display_documents (const struct dictionary *dict);
-static void display_vectors (const struct dictionary *dict, int sorted);
-
-int
-cmd_display (struct lexer *lexer, struct dataset *ds)
-{
- /* Whether to sort the list of variables alphabetically. */
- int sorted;
-
- /* Variables to display. */
- size_t n;
- const struct variable **vl;
-
- if (lex_match_id (lexer, "MACROS"))
- display_macros ();
- else if (lex_match_id (lexer, "DOCUMENTS"))
- display_documents (dataset_dict (ds));
- else if (lex_match_id (lexer, "FILE"))
- {
- if (!lex_force_match_id (lexer, "LABEL"))
- return CMD_FAILURE;
-
- const char *label = dict_get_label (dataset_dict (ds));
-
- struct pivot_table *table = pivot_table_create (N_("File Label"));
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Label"),
- N_("Label"));
- pivot_table_put1 (table, 0,
- (label ? pivot_value_new_user_text (label, -1)
- : pivot_value_new_text (N_("(none)"))));
- pivot_table_submit (table);
- }
- else
- {
- int flags;
-
- sorted = lex_match_id (lexer, "SORTED");
-
- if (lex_match_id (lexer, "VECTORS"))
- {
- display_vectors (dataset_dict(ds), sorted);
- return CMD_SUCCESS;
- }
- else if (lex_match_id (lexer, "SCRATCH"))
- {
- dict_get_vars (dataset_dict (ds), &vl, &n, DC_ORDINARY);
- flags = DF_NAME;
- }
- else
- {
- struct subcommand
- {
- const char *name;
- int flags;
- };
- static const struct subcommand subcommands[] =
- {
- {"@ATTRIBUTES", DF_ATTRIBUTES | DF_AT_ATTRIBUTES},
- {"ATTRIBUTES", DF_ATTRIBUTES},
- {"DICTIONARY", (DF_NAME | DF_POSITION | DF_LABEL
- | DF_MEASUREMENT_LEVEL | DF_ROLE | DF_WIDTH
- | DF_ALIGNMENT | DF_PRINT_FORMAT
- | DF_WRITE_FORMAT | DF_MISSING_VALUES
- | DF_VALUE_LABELS)},
- {"INDEX", DF_NAME | DF_POSITION},
- {"LABELS", DF_NAME | DF_POSITION | DF_LABEL},
- {"NAMES", DF_NAME},
- {"VARIABLES", (DF_NAME | DF_POSITION | DF_PRINT_FORMAT
- | DF_WRITE_FORMAT | DF_MISSING_VALUES)},
- {NULL, 0},
- };
- const struct subcommand *sbc;
- struct dictionary *dict = dataset_dict (ds);
-
- flags = 0;
- for (sbc = subcommands; sbc->name != NULL; sbc++)
- if (lex_match_id (lexer, sbc->name))
- {
- flags = sbc->flags;
- break;
- }
-
- lex_match (lexer, T_SLASH);
- lex_match_id (lexer, "VARIABLES");
- lex_match (lexer, T_EQUALS);
-
- if (lex_token (lexer) != T_ENDCMD)
- {
- if (!parse_variables_const (lexer, dict, &vl, &n, PV_NONE))
- {
- free (vl);
- return CMD_FAILURE;
- }
- }
- else
- dict_get_vars (dict, &vl, &n, 0);
- }
-
- if (n > 0)
- {
- sort (vl, n, sizeof *vl, (sorted
- ? compare_var_ptrs_by_name
- : compare_var_ptrs_by_dict_index), NULL);
-
- int variable_flags = flags & DF_ALL_VARIABLE;
- if (variable_flags)
- display_variables (vl, n, variable_flags);
-
- if (flags & DF_VALUE_LABELS)
- display_value_labels (vl, n);
-
- int attribute_flags = flags & (DF_ATTRIBUTES | DF_AT_ATTRIBUTES);
- if (attribute_flags)
- display_attributes (dict_get_attributes (dataset_dict (ds)),
- vl, n, attribute_flags);
- }
- else
- msg (SN, _("No variables to display."));
-
- free (vl);
- }
-
- return CMD_SUCCESS;
-}
-
-static void
-display_macros (void)
-{
- msg (SW, _("Macros not supported."));
-}
-
-static char *
-get_documents_as_string (const struct dictionary *dict)
-{
- const struct string_array *documents = dict_get_documents (dict);
- struct string s = DS_EMPTY_INITIALIZER;
- for (size_t i = 0; i < documents->n; i++)
- {
- if (i)
- ds_put_byte (&s, '\n');
- ds_put_cstr (&s, documents->strings[i]);
- }
- return ds_steal_cstr (&s);
-}
-
-static void
-display_documents (const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (N_("Documents"));
- struct pivot_dimension *d = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Documents"), N_("Document"));
- d->hide_all_labels = true;
-
- if (!dict_get_documents (dict)->n)
- pivot_table_put1 (table, 0, pivot_value_new_text (N_("(none)")));
- else
- {
- char *docs = get_documents_as_string (dict);
- pivot_table_put1 (table, 0, pivot_value_new_user_text_nocopy (docs));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-display_variables (const struct variable **vl, size_t n, int flags)
-{
- struct pivot_table *table = pivot_table_create (N_("Variables"));
-
- struct pivot_dimension *attributes = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Attributes"));
-
- struct heading
- {
- int flag;
- const char *title;
- };
- static const struct heading headings[] = {
- { DF_POSITION, N_("Position") },
- { DF_LABEL, N_("Label") },
- { DF_MEASUREMENT_LEVEL, N_("Measurement Level") },
- { DF_ROLE, N_("Role") },
- { DF_WIDTH, N_("Width") },
- { DF_ALIGNMENT, N_("Alignment") },
- { DF_PRINT_FORMAT, N_("Print Format") },
- { DF_WRITE_FORMAT, N_("Write Format") },
- { DF_MISSING_VALUES, N_("Missing Values") },
- };
- for (size_t i = 0; i < sizeof headings / sizeof *headings; i++)
- if (flags & headings[i].flag)
- pivot_category_create_leaf (attributes->root,
- pivot_value_new_text (headings[i].title));
-
- struct pivot_dimension *names = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Name"));
- names->root->show_label = true;
-
- for (size_t i = 0; i < n; i++)
- {
- const struct variable *v = vl[i];
-
- struct pivot_value *name = pivot_value_new_variable (v);
- name->variable.show = SETTINGS_VALUE_SHOW_VALUE;
- int row = pivot_category_create_leaf (names->root, name);
-
- int x = 0;
- if (flags & DF_POSITION)
- pivot_table_put2 (table, x++, row, pivot_value_new_integer (
- var_get_dict_index (v) + 1));
-
- if (flags & DF_LABEL)
- {
- const char *label = var_get_label (v);
- if (label)
- pivot_table_put2 (table, x, row,
- pivot_value_new_user_text (label, -1));
- x++;
- }
-
- if (flags & DF_MEASUREMENT_LEVEL)
- pivot_table_put2 (
- table, x++, row,
- pivot_value_new_text (measure_to_string (var_get_measure (v))));
-
- if (flags & DF_ROLE)
- pivot_table_put2 (
- table, x++, row,
- pivot_value_new_text (var_role_to_string (var_get_role (v))));
-
- if (flags & DF_WIDTH)
- pivot_table_put2 (
- table, x++, row,
- pivot_value_new_integer (var_get_display_width (v)));
-
- if (flags & DF_ALIGNMENT)
- pivot_table_put2 (
- table, x++, row,
- pivot_value_new_text (alignment_to_string (
- var_get_alignment (v))));
-
- if (flags & DF_PRINT_FORMAT)
- {
- const struct fmt_spec *print = var_get_print_format (v);
- char s[FMT_STRING_LEN_MAX + 1];
-
- pivot_table_put2 (
- table, x++, row,
- pivot_value_new_user_text (fmt_to_string (print, s), -1));
- }
-
- if (flags & DF_WRITE_FORMAT)
- {
- const struct fmt_spec *write = var_get_write_format (v);
- char s[FMT_STRING_LEN_MAX + 1];
-
- pivot_table_put2 (
- table, x++, row,
- pivot_value_new_user_text (fmt_to_string (write, s), -1));
- }
-
- if (flags & DF_MISSING_VALUES)
- {
- char *s = mv_to_string (var_get_missing_values (v),
- var_get_encoding (v));
- if (s)
- pivot_table_put2 (
- table, x, row,
- pivot_value_new_user_text_nocopy (s));
-
- x++;
- }
- }
-
- pivot_table_submit (table);
-}
-
-static bool
-any_value_labels (const struct variable **vars, size_t n_vars)
-{
- for (size_t i = 0; i < n_vars; i++)
- if (val_labs_count (var_get_value_labels (vars[i])))
- return true;
- return false;
-}
-
-static void
-display_value_labels (const struct variable **vars, size_t n_vars)
-{
- if (!any_value_labels (vars, n_vars))
- return;
-
- struct pivot_table *table = pivot_table_create (N_("Value Labels"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN,
- N_("Label"), N_("Label"));
-
- struct pivot_dimension *values = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable Value"));
- values->root->show_label = true;
-
- struct pivot_footnote *missing_footnote = pivot_table_create_footnote (
- table, pivot_value_new_text (N_("User-missing value")));
-
- for (size_t i = 0; i < n_vars; i++)
- {
- const struct val_labs *val_labs = var_get_value_labels (vars[i]);
- size_t n_labels = val_labs_count (val_labs);
- if (!n_labels)
- continue;
-
- struct pivot_category *group = pivot_category_create_group__ (
- values->root, pivot_value_new_variable (vars[i]));
-
- const struct val_lab **labels = val_labs_sorted (val_labs);
- for (size_t j = 0; j < n_labels; j++)
- {
- const struct val_lab *vl = labels[j];
-
- struct pivot_value *value = pivot_value_new_var_value (
- vars[i], &vl->value);
- if (value->type == PIVOT_VALUE_NUMERIC)
- value->numeric.show = SETTINGS_VALUE_SHOW_VALUE;
- else
- value->string.show = SETTINGS_VALUE_SHOW_VALUE;
- if (var_is_value_missing (vars[i], &vl->value) == MV_USER)
- pivot_value_add_footnote (value, missing_footnote);
- int row = pivot_category_create_leaf (group, value);
-
- struct pivot_value *label = pivot_value_new_var_value (
- vars[i], &vl->value);
- char *escaped_label = xstrdup (val_lab_get_escaped_label (vl));
- if (label->type == PIVOT_VALUE_NUMERIC)
- {
- free (label->numeric.value_label);
- label->numeric.value_label = escaped_label;
- label->numeric.show = SETTINGS_VALUE_SHOW_LABEL;
- }
- else
- {
- free (label->string.value_label);
- label->string.value_label = escaped_label;
- label->string.show = SETTINGS_VALUE_SHOW_LABEL;
- }
- pivot_table_put2 (table, 0, row, label);
- }
- free (labels);
- }
- pivot_table_submit (table);
-}
-\f
-static bool
-is_at_name (const char *name)
-{
- return name[0] == '@' || (name[0] == '$' && name[1] == '@');
-}
-
-static size_t
-count_attributes (const struct attrset *set, int flags)
-{
- struct attrset_iterator i;
- struct attribute *attr;
- size_t n_attrs;
-
- n_attrs = 0;
- for (attr = attrset_first (set, &i); attr != NULL;
- attr = attrset_next (set, &i))
- if (flags & DF_AT_ATTRIBUTES || !is_at_name (attribute_get_name (attr)))
- n_attrs += attribute_get_n_values (attr);
- return n_attrs;
-}
-
-static void
-display_attrset (struct pivot_table *table, struct pivot_value *set_name,
- const struct attrset *set, int flags)
-{
- size_t n_total = count_attributes (set, flags);
- if (!n_total)
- {
- pivot_value_destroy (set_name);
- return;
- }
-
- struct pivot_category *group = pivot_category_create_group__ (
- table->dimensions[1]->root, set_name);
-
- size_t n_attrs = attrset_count (set);
- struct attribute **attrs = attrset_sorted (set);
- for (size_t i = 0; i < n_attrs; i++)
- {
- const struct attribute *attr = attrs[i];
- const char *name = attribute_get_name (attr);
-
- if (!(flags & DF_AT_ATTRIBUTES) && is_at_name (name))
- continue;
-
- size_t n_values = attribute_get_n_values (attr);
- for (size_t j = 0; j < n_values; j++)
- {
- int row = pivot_category_create_leaf (
- group,
- (n_values > 1
- ? pivot_value_new_user_text_nocopy (xasprintf (
- "%s[%zu]", name, j + 1))
- : pivot_value_new_user_text (name, -1)));
- pivot_table_put2 (table, 0, row,
- pivot_value_new_user_text (
- attribute_get_value (attr, j), -1));
- }
- }
- free (attrs);
-}
-
-static void
-display_attributes (const struct attrset *dict_attrset,
- const struct variable **vars, size_t n_vars, int flags)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Variable and Dataset Attributes"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN,
- N_("Value"), N_("Value"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable and Name"));
- variables->root->show_label = true;
-
- display_attrset (table, pivot_value_new_text (N_("(dataset)")),
- dict_attrset, flags);
- for (size_t i = 0; i < n_vars; i++)
- display_attrset (table, pivot_value_new_variable (vars[i]),
- var_get_attributes (vars[i]), flags);
-
- if (pivot_table_is_empty (table))
- pivot_table_unref (table);
- else
- pivot_table_submit (table);
-}
-
-/* Display a list of vectors. If SORTED is nonzero then they are
- sorted alphabetically. */
-static void
-display_vectors (const struct dictionary *dict, int sorted)
-{
- size_t n_vectors = dict_get_n_vectors (dict);
- if (n_vectors == 0)
- {
- msg (SN, _("No vectors defined."));
- return;
- }
-
- const struct vector **vectors = xnmalloc (n_vectors, sizeof *vectors);
- for (size_t i = 0; i < n_vectors; i++)
- vectors[i] = dict_get_vector (dict, i);
- if (sorted)
- qsort (vectors, n_vectors, sizeof *vectors, compare_vector_ptrs_by_name);
-
- struct pivot_table *table = pivot_table_create (N_("Vectors"));
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Attributes"),
- N_("Variable"), N_("Print Format"));
- struct pivot_dimension *vector_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Vector and Position"));
- vector_dim->root->show_label = true;
-
- for (size_t i = 0; i < n_vectors; i++)
- {
- const struct vector *vec = vectors[i];
-
- struct pivot_category *group = pivot_category_create_group__ (
- vector_dim->root, pivot_value_new_user_text (
- vector_get_name (vectors[i]), -1));
-
- for (size_t j = 0; j < vector_get_n_vars (vec); j++)
- {
- struct variable *var = vector_get_var (vec, j);
-
- int row = pivot_category_create_leaf (
- group, pivot_value_new_integer (j + 1));
-
- pivot_table_put2 (table, 0, row, pivot_value_new_variable (var));
- char fmt_string[FMT_STRING_LEN_MAX + 1];
- fmt_to_string (var_get_print_format (var), fmt_string);
- pivot_table_put2 (table, 1, row,
- pivot_value_new_user_text (fmt_string, -1));
- }
- }
-
- pivot_table_submit (table);
-
- free (vectors);
-}
-\f
-/* Encoding analysis. */
-
-static const char *encoding_names[] = {
- /* These encodings are from http://encoding.spec.whatwg.org/, as retrieved
- February 2014. Encodings not supported by glibc and encodings relevant
- only to HTML have been removed. */
- "utf-8",
- "windows-1252",
- "iso-8859-2",
- "iso-8859-3",
- "iso-8859-4",
- "iso-8859-5",
- "iso-8859-6",
- "iso-8859-7",
- "iso-8859-8",
- "iso-8859-10",
- "iso-8859-13",
- "iso-8859-14",
- "iso-8859-16",
- "macintosh",
- "windows-874",
- "windows-1250",
- "windows-1251",
- "windows-1253",
- "windows-1254",
- "windows-1255",
- "windows-1256",
- "windows-1257",
- "windows-1258",
- "koi8-r",
- "koi8-u",
- "ibm866",
- "gb18030",
- "big5",
- "euc-jp",
- "iso-2022-jp",
- "shift_jis",
- "euc-kr",
-
- /* Added by user request. */
- "ibm850",
- "din_66003",
-};
-#define N_ENCODING_NAMES (sizeof encoding_names / sizeof *encoding_names)
-
-struct encoding
- {
- uint64_t encodings;
- char **utf8_strings;
- unsigned int hash;
- };
-
-static char **
-recode_strings (struct pool *pool,
- char **strings, bool *ids, size_t n,
- const char *encoding)
-{
- char **utf8_strings;
- size_t i;
-
- utf8_strings = pool_alloc (pool, n * sizeof *utf8_strings);
- for (i = 0; i < n; i++)
- {
- struct substring utf8;
- int error;
-
- error = recode_pedantically ("UTF-8", encoding, ss_cstr (strings[i]),
- pool, &utf8);
- if (!error)
- {
- ss_rtrim (&utf8, ss_cstr (" "));
- utf8.string[utf8.length] = '\0';
-
- if (ids[i] && !id_is_plausible (utf8.string))
- error = EINVAL;
- }
-
- if (error)
- return NULL;
-
- utf8_strings[i] = utf8.string;
- }
-
- return utf8_strings;
-}
-
-static struct encoding *
-find_duplicate_encoding (struct encoding *encodings, size_t n_encodings,
- char **utf8_strings, size_t n_strings,
- unsigned int hash)
-{
- struct encoding *e;
-
- for (e = encodings; e < &encodings[n_encodings]; e++)
- {
- int i;
-
- if (e->hash != hash)
- goto next_encoding;
-
- for (i = 0; i < n_strings; i++)
- if (strcmp (utf8_strings[i], e->utf8_strings[i]))
- goto next_encoding;
-
- return e;
- next_encoding:;
- }
-
- return NULL;
-}
-
-static bool
-all_equal (const struct encoding *encodings, size_t n_encodings,
- size_t string_idx)
-{
- const char *s0;
- size_t i;
-
- s0 = encodings[0].utf8_strings[string_idx];
- for (i = 1; i < n_encodings; i++)
- if (strcmp (s0, encodings[i].utf8_strings[string_idx]))
- return false;
-
- return true;
-}
-
-static int
-equal_prefix (const struct encoding *encodings, size_t n_encodings,
- size_t string_idx)
-{
- const char *s0;
- size_t prefix;
- size_t i;
-
- s0 = encodings[0].utf8_strings[string_idx];
- prefix = strlen (s0);
- for (i = 1; i < n_encodings; i++)
- {
- const char *si = encodings[i].utf8_strings[string_idx];
- size_t j;
-
- for (j = 0; j < prefix; j++)
- if (s0[j] != si[j])
- {
- prefix = j;
- if (!prefix)
- return 0;
- break;
- }
- }
-
- while (prefix > 0 && s0[prefix - 1] != ' ')
- prefix--;
- return prefix;
-}
-
-static int
-equal_suffix (const struct encoding *encodings, size_t n_encodings,
- size_t string_idx)
-{
- const char *s0;
- size_t s0_len;
- size_t suffix;
- size_t i;
-
- s0 = encodings[0].utf8_strings[string_idx];
- s0_len = strlen (s0);
- suffix = s0_len;
- for (i = 1; i < n_encodings; i++)
- {
- const char *si = encodings[i].utf8_strings[string_idx];
- size_t si_len = strlen (si);
- size_t j;
-
- if (si_len < suffix)
- suffix = si_len;
- for (j = 0; j < suffix; j++)
- if (s0[s0_len - j - 1] != si[si_len - j - 1])
- {
- suffix = j;
- if (!suffix)
- return 0;
- break;
- }
- }
-
- while (suffix > 0 && s0[s0_len - suffix] != ' ')
- suffix--;
- return suffix;
-}
-
-static void
-report_encodings (const struct file_handle *h, struct pool *pool,
- char **titles, bool *ids, char **strings, size_t n_strings)
-{
- struct encoding encodings[N_ENCODING_NAMES];
- size_t n_encodings, n_unique_strings;
-
- n_encodings = 0;
- for (size_t i = 0; i < N_ENCODING_NAMES; i++)
- {
- char **utf8_strings;
- struct encoding *e;
- unsigned int hash;
-
- utf8_strings = recode_strings (pool, strings, ids, n_strings,
- encoding_names[i]);
- if (!utf8_strings)
- continue;
-
- /* Hash utf8_strings. */
- hash = 0;
- for (size_t j = 0; j < n_strings; j++)
- hash = hash_string (utf8_strings[j], hash);
-
- /* If there's a duplicate encoding, just mark it. */
- e = find_duplicate_encoding (encodings, n_encodings,
- utf8_strings, n_strings, hash);
- if (e)
- {
- e->encodings |= UINT64_C (1) << i;
- continue;
- }
-
- e = &encodings[n_encodings++];
- e->encodings = UINT64_C (1) << i;
- e->utf8_strings = utf8_strings;
- e->hash = hash;
- }
- if (!n_encodings)
- {
- msg (SW, _("No valid encodings found."));
- return;
- }
-
- /* Table of valid encodings. */
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_text_format (N_("Usable encodings for %s."),
- fh_get_name (h)), "Usable Encodings");
- pivot_table_set_caption (
- table, pivot_value_new_text_format (
- N_("Encodings that can successfully read %s (by specifying the encoding "
- "name on the GET command's ENCODING subcommand). Encodings that "
- "yield identical text are listed together."),
- fh_get_name (h)));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Encodings"),
- N_("Encodings"));
- struct pivot_dimension *number = pivot_dimension_create__ (
- table, PIVOT_AXIS_ROW, pivot_value_new_user_text ("#", -1));
- number->root->show_label = true;
-
- for (size_t i = 0; i < n_encodings; i++)
- {
- struct string s = DS_EMPTY_INITIALIZER;
- for (size_t j = 0; j < 64; j++)
- if (encodings[i].encodings & (UINT64_C (1) << j))
- ds_put_format (&s, "%s, ", encoding_names[j]);
- ds_chomp (&s, ss_cstr (", "));
-
- int row = pivot_category_create_leaf (number->root,
- pivot_value_new_integer (i + 1));
- pivot_table_put2 (
- table, 0, row, pivot_value_new_user_text_nocopy (ds_steal_cstr (&s)));
- }
- pivot_table_submit (table);
-
- n_unique_strings = 0;
- for (size_t i = 0; i < n_strings; i++)
- if (!all_equal (encodings, n_encodings, i))
- n_unique_strings++;
- if (!n_unique_strings)
- return;
-
- /* Table of alternative interpretations. */
- table = pivot_table_create__ (
- pivot_value_new_text_format (N_("%s Encoded Text Strings"),
- fh_get_name (h)),
- "Alternate Encoded Text Strings");
- pivot_table_set_caption (
- table, pivot_value_new_text (
- N_("Text strings in the file dictionary that the previously listed "
- "encodings interpret differently, along with the interpretations.")));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Text"), N_("Text"));
-
- number = pivot_dimension_create__ (table, PIVOT_AXIS_ROW,
- pivot_value_new_user_text ("#", -1));
- number->root->show_label = true;
- for (size_t i = 0; i < n_encodings; i++)
- pivot_category_create_leaf (number->root,
- pivot_value_new_integer (i + 1));
-
- struct pivot_dimension *purpose = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Purpose"));
- purpose->root->show_label = true;
-
- for (size_t i = 0; i < n_strings; i++)
- if (!all_equal (encodings, n_encodings, i))
- {
- int prefix = equal_prefix (encodings, n_encodings, i);
- int suffix = equal_suffix (encodings, n_encodings, i);
-
- int purpose_idx = pivot_category_create_leaf (
- purpose->root, pivot_value_new_user_text (titles[i], -1));
-
- for (size_t j = 0; j < n_encodings; j++)
- {
- const char *s = encodings[j].utf8_strings[i] + prefix;
-
- if (prefix || suffix)
- {
- size_t len = strlen (s) - suffix;
- struct string entry;
-
- ds_init_empty (&entry);
- if (prefix)
- ds_put_cstr (&entry, "...");
- ds_put_substring (&entry, ss_buffer (s, len));
- if (suffix)
- ds_put_cstr (&entry, "...");
-
- pivot_table_put3 (table, 0, j, purpose_idx,
- pivot_value_new_user_text_nocopy (
- ds_steal_cstr (&entry)));
- }
- else
- pivot_table_put3 (table, 0, j, purpose_idx,
- pivot_value_new_user_text (s, -1));
- }
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-\f
-/* Declarations. */
-
-static int do_value_labels (struct lexer *,
- const struct dictionary *dict, bool);
-static void erase_labels (struct variable **vars, size_t n_vars);
-static int get_label (struct lexer *, struct variable **vars, size_t n_vars,
- const char *dict_encoding);
-\f
-/* Stubs. */
-
-int
-cmd_value_labels (struct lexer *lexer, struct dataset *ds)
-{
- return do_value_labels (lexer, dataset_dict (ds), true);
-}
-
-int
-cmd_add_value_labels (struct lexer *lexer, struct dataset *ds)
-{
- return do_value_labels (lexer, dataset_dict (ds), false);
-}
-\f
-/* Do it. */
-
-static int
-do_value_labels (struct lexer *lexer, const struct dictionary *dict, bool erase)
-{
- struct variable **vars; /* Variable list. */
- size_t n_vars; /* Number of variables. */
- int parse_err=0; /* true if error parsing variables */
-
- lex_match (lexer, T_SLASH);
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- parse_err = !parse_variables (lexer, dict, &vars, &n_vars,
- PV_SAME_WIDTH);
- if (n_vars < 1)
- {
- free(vars);
- return CMD_FAILURE;
- }
- if (erase)
- erase_labels (vars, n_vars);
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- if (!get_label (lexer, vars, n_vars, dict_get_encoding (dict)))
- goto lossage;
-
- if (lex_token (lexer) != T_SLASH)
- {
- free (vars);
- break;
- }
-
- lex_get (lexer);
-
- free (vars);
- }
-
- return parse_err ? CMD_FAILURE : CMD_SUCCESS;
-
- lossage:
- free (vars);
- return CMD_FAILURE;
-}
-
-/* Erases all the labels for the N_VARS variables in VARS. */
-static void
-erase_labels (struct variable **vars, size_t n_vars)
-{
- /* Erase old value labels if desired. */
- for (size_t i = 0; i < n_vars; i++)
- var_clear_value_labels (vars[i]);
-}
-
-/* Parse all the labels for the N_VARS variables in VARS and add
- the specified labels to those variables. */
-static int
-get_label (struct lexer *lexer, struct variable **vars, size_t n_vars,
- const char *dict_encoding)
-{
- /* Parse all the labels and add them to the variables. */
- do
- {
- enum { MAX_LABEL_LEN = 255 };
- int width = var_get_width (vars[0]);
- union value value;
- struct string label;
- size_t trunc_len;
- size_t i;
-
- /* Set value. */
- value_init (&value, width);
- if (!parse_value (lexer, &value, vars[0]))
- {
- value_destroy (&value, width);
- return 0;
- }
- lex_match (lexer, T_COMMA);
-
- /* Set label. */
- if (lex_token (lexer) != T_ID && !lex_force_string (lexer))
- {
- value_destroy (&value, width);
- return 0;
- }
-
- ds_init_substring (&label, lex_tokss (lexer));
-
- trunc_len = utf8_encoding_trunc_len (ds_cstr (&label), dict_encoding,
- MAX_LABEL_LEN);
- if (ds_length (&label) > trunc_len)
- {
- lex_next_msg (lexer, SW, 0, 0,
- _("Truncating value label to %d bytes."),
- MAX_LABEL_LEN);
- ds_truncate (&label, trunc_len);
- }
-
- for (i = 0; i < n_vars; i++)
- var_replace_value_label (vars[i], &value, ds_cstr (&label));
-
- ds_destroy (&label);
- value_destroy (&value, width);
-
- lex_get (lexer);
- lex_match (lexer, T_COMMA);
- }
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD);
-
- return 1;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011, 2013 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_variable_alignment (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- struct variable **v;
- size_t nv;
-
- if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
- return CMD_FAILURE;
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- enum alignment align;
- if (lex_match_id (lexer, "LEFT"))
- align = ALIGN_LEFT;
- else if (lex_match_id (lexer, "RIGHT"))
- align = ALIGN_RIGHT;
- else if (lex_match_id (lexer, "CENTER"))
- align = ALIGN_CENTRE;
- else
- {
- lex_error_expecting (lexer, "LEFT", "RIGHT", "CENTER");
- goto error;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
-
- for (size_t i = 0; i < nv; ++i)
- var_set_alignment (v[i], align);
-
- while (lex_token (lexer) == T_SLASH)
- lex_get (lexer);
- free (v);
- continue;
-
- error:
- free (v);
- return CMD_FAILURE;
- }
- while (lex_token (lexer) != T_ENDCMD);
- return CMD_SUCCESS;
-}
-
-int
-cmd_variable_width (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- struct variable **v;
- size_t nv;
- if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
- return CMD_FAILURE;
-
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, NULL, 1, INT_MAX))
- goto error;
- long width = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
-
- width = MIN (width, 2 * MAX_STRING);
-
- for (size_t i = 0; i < nv; ++i)
- var_set_display_width (v[i], width);
-
- while (lex_token (lexer) == T_SLASH)
- lex_get (lexer);
- free (v);
- continue;
-
- error:
- free (v);
- return CMD_FAILURE;
- }
- while (lex_token (lexer) != T_ENDCMD);
- return CMD_SUCCESS;
-}
-
-/* Set variables' measurement level */
-int
-cmd_variable_level (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- struct variable **v;
- size_t nv;
-
- if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
- return CMD_FAILURE;
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- enum measure level;
- if (lex_match_id (lexer, "SCALE"))
- level = MEASURE_SCALE;
- else if (lex_match_id (lexer, "ORDINAL"))
- level = MEASURE_ORDINAL;
- else if (lex_match_id (lexer, "NOMINAL"))
- level = MEASURE_NOMINAL;
- else
- {
- lex_error_expecting (lexer, "SCALE", "ORDINAL", "NOMINAL");
- goto error;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
-
- for (size_t i = 0; i < nv; ++i)
- var_set_measure (v[i], level);
-
- while (lex_token (lexer) == T_SLASH)
- lex_get (lexer);
- free (v);
- continue;
-
- error:
- free (v);
- return CMD_FAILURE;
- }
- while (lex_token (lexer) != T_ENDCMD);
- return CMD_SUCCESS;
-}
-
-int
-cmd_variable_role (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- if (!lex_force_match (lexer, T_SLASH))
- return CMD_FAILURE;
-
- enum var_role role;
- if (lex_match_id (lexer, "INPUT"))
- role = ROLE_INPUT;
- else if (lex_match_id (lexer, "TARGET"))
- role = ROLE_TARGET;
- else if (lex_match_id (lexer, "BOTH"))
- role = ROLE_BOTH;
- else if (lex_match_id (lexer, "NONE"))
- role = ROLE_NONE;
- else if (lex_match_id (lexer, "PARTITION"))
- role = ROLE_PARTITION;
- else if (lex_match_id (lexer, "SPLIT"))
- role = ROLE_SPLIT;
- else
- {
- lex_error_expecting (lexer, "INPUT", "TARGET", "BOTH",
- "NONE", "PARTITION", "SPLIT");
- return CMD_FAILURE;
- }
-
- struct variable **v;
- size_t nv;
- if (!parse_variables (lexer, dataset_dict (ds), &v, &nv, PV_NONE))
- return CMD_FAILURE;
-
- for (size_t i = 0; i < nv; i++)
- var_set_role (v[i], role);
-
- free (v);
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_variable_labels (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
-
- do
- {
- struct variable **v;
- size_t nv;
-
- size_t i;
-
- if (!parse_variables (lexer, dict, &v, &nv, PV_NONE))
- return CMD_FAILURE;
-
- if (!lex_force_string (lexer))
- {
- free (v);
- return CMD_FAILURE;
- }
-
- for (i = 0; i < nv; i++)
- var_set_label (v[i], lex_tokcstr (lexer));
-
- lex_get (lexer);
- while (lex_token (lexer) == T_SLASH)
- lex_get (lexer);
- free (v);
- }
- while (lex_token (lexer) != T_ENDCMD);
- return CMD_SUCCESS;
-}
-
-
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011, 2012, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/format.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-
-#include "gl/intprops.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_vector (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct pool *pool = pool_create ();
-
- do
- {
- /* Get the name(s) of the new vector(s). */
- if (!lex_force_id (lexer))
- goto error;
-
- int vectors_start = lex_ofs (lexer);
- char **vectors = NULL;
- size_t n_vectors = 0;
- size_t allocated_vectors = 0;
- while (lex_token (lexer) == T_ID)
- {
- char *error = dict_id_is_valid__ (dict, lex_tokcstr (lexer));
- if (error)
- {
- lex_error (lexer, "%s", error);
- free (error);
- goto error;
- }
-
- if (dict_lookup_vector (dict, lex_tokcstr (lexer)))
- {
- lex_next_error (lexer, 0, 0,
- _("A vector named %s already exists."),
- lex_tokcstr (lexer));
- goto error;
- }
-
- for (size_t i = 0; i < n_vectors; i++)
- if (!utf8_strcasecmp (vectors[i], lex_tokcstr (lexer)))
- {
- lex_ofs_error (lexer, vectors_start, lex_ofs (lexer),
- _("Vector name %s is given twice."),
- lex_tokcstr (lexer));
- goto error;
- }
-
- if (n_vectors == allocated_vectors)
- vectors = pool_2nrealloc (pool, vectors, &allocated_vectors,
- sizeof *vectors);
- vectors[n_vectors++] = pool_strdup (pool, lex_tokcstr (lexer));
-
- lex_get (lexer);
- lex_match (lexer, T_COMMA);
- }
-
- /* Now that we have the names it's time to check for the short
- or long forms. */
- if (lex_match (lexer, T_EQUALS))
- {
- if (n_vectors > 1)
- {
- lex_ofs_error (lexer, vectors_start, lex_ofs (lexer) - 1,
- _("Only a single vector name may be specified "
- "when a list of variables is given."));
- goto error;
- }
-
- struct variable **v;
- size_t nv;
- if (!parse_variables_pool (lexer, pool, dict, &v, &nv,
- PV_SAME_WIDTH | PV_DUPLICATE))
- goto error;
-
- dict_create_vector (dict, vectors[0], v, nv);
- }
- else if (lex_match (lexer, T_LPAREN))
- {
- struct fmt_spec format = fmt_for_output (FMT_F, 8, 2);
- bool seen_format = false;
- size_t n_vars = 0;
- int name_ofs = lex_ofs (lexer) - 2;
- int lparen_ofs = lex_ofs (lexer) - 1;
- while (!lex_match (lexer, T_RPAREN))
- {
- if (lex_is_integer (lexer))
- {
- if (n_vars)
- {
- lex_ofs_error (lexer, lparen_ofs, lex_ofs (lexer),
- _("Vector length may only be specified "
- "once."));
- goto error;
- }
- if (!lex_force_int_range (lexer, NULL, 1, INT_MAX))
- goto error;
- n_vars = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (lex_token (lexer) == T_ID)
- {
- if (seen_format)
- {
- lex_ofs_error (lexer, lparen_ofs, lex_ofs (lexer),
- _("Only one format may be specified."));
- goto error;
- }
- seen_format = true;
- if (!parse_format_specifier (lexer, &format))
- goto error;
- char *error = fmt_check_output__ (&format);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- goto error;
- }
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting vector length "
- "or format."));
- goto error;
- }
- lex_match (lexer, T_COMMA);
- }
- int end_ofs = lex_ofs (lexer) - 1;
- if (n_vars == 0)
- {
- lex_ofs_error (lexer, lparen_ofs, end_ofs,
- _("Vector length is required."));
- goto error;
- }
-
- /* Check that none of the variables exist and that their names are
- not excessively long. */
- for (size_t i = 0; i < n_vectors; i++)
- for (size_t j = 0; j < n_vars; j++)
- {
- char *name = xasprintf ("%s%zu", vectors[i], j + 1);
- char *error = dict_id_is_valid__ (dict, name);
- if (error)
- {
- lex_ofs_error (lexer, name_ofs, end_ofs, "%s", error);
- free (error);
- free (name);
- goto error;
- }
- if (dict_lookup_var (dict, name))
- {
- lex_ofs_error (lexer, name_ofs, end_ofs,
- _("%s is an existing variable name."),
- name);
- free (name);
- goto error;
- }
- free (name);
- }
-
- /* Finally create the variables and vectors. */
- struct variable **vars = pool_nmalloc (pool, n_vars, sizeof *vars);
- for (size_t i = 0; i < n_vectors; i++)
- {
- for (size_t j = 0; j < n_vars; j++)
- {
- char *name = xasprintf ("%s%zu", vectors[i], j + 1);
- vars[j] = dict_create_var_assert (dict, name,
- fmt_var_width (&format));
- var_set_both_formats (vars[j], &format);
- free (name);
- }
- dict_create_vector_assert (dict, vectors[i], vars, n_vars);
- }
- }
- else
- {
- lex_error_expecting (lexer, "`='", "`('");
- goto error;
- }
- }
- while (lex_match (lexer, T_SLASH));
-
- pool_destroy (pool);
- return CMD_SUCCESS;
-
-error:
- pool_destroy (pool);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdio.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_weight (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- if (lex_match_id (lexer, "OFF"))
- dict_set_weight (dataset_dict (ds), NULL);
- else
- {
- struct variable *v;
-
- lex_match (lexer, T_BY);
- v = parse_variable (lexer, dict);
- if (!v)
- return CMD_CASCADING_FAILURE;
- if (var_is_alpha (v))
- {
- lex_next_error (lexer, -1, -1,
- _("The weighting variable must be numeric."));
- return CMD_CASCADING_FAILURE;
- }
- if (dict_class_from_id (var_get_name (v)) == DC_SCRATCH)
- {
- lex_next_error (lexer, -1, -1,
- _("The weighting variable may not be scratch."));
- return CMD_CASCADING_FAILURE;
- }
-
- dict_set_weight (dict, v);
- }
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2008, 2009, 2010, 2011, 2012, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/aggregate.h"
-
-#include <stdlib.h>
-
-#include "data/any-writer.h"
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/file-handle-def.h"
-#include "data/format.h"
-#include "data/settings.h"
-#include "data/subcase.h"
-#include "data/sys-file-writer.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/data-io/file-handle.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "language/stats/sort-criteria.h"
-#include "libpspp/assertion.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-#include "math/moments.h"
-#include "math/percentiles.h"
-#include "math/sort.h"
-#include "math/statistic.h"
-
-#include "gl/c-strcase.h"
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-/* Argument for AGGREGATE function.
-
- Only one of the members is used, so this could be a union, but it's simpler
- to just have both. */
-struct agr_argument
- {
- double f; /* Numeric. */
- struct substring s; /* String. */
- };
-
-/* Specifies how to make an aggregate variable. */
-struct agr_var
- {
- /* Collected during parsing. */
- const struct variable *src; /* Source variable. */
- struct variable *dest; /* Target variable. */
- enum agr_function function; /* Function. */
- enum mv_class exclude; /* Classes of missing values to exclude. */
- struct agr_argument arg[2]; /* Arguments. */
-
- /* Accumulated during AGGREGATE execution. */
- double dbl;
- double W; /* Total non-missing weight. */
- int int1;
- char *string;
- bool saw_missing;
- struct moments1 *moments;
-
- struct variable *subject;
- struct variable *weight;
- struct casewriter *writer;
- };
-
-/* Attributes of aggregation functions. */
-const struct agr_func agr_func_tab[] =
- {
-#define AGRF(ENUM, NAME, DESCRIPTION, SRC_VARS, N_ARGS, ALPHA_TYPE, W, D) \
- [ENUM] = { NAME, DESCRIPTION, SRC_VARS, N_ARGS, ALPHA_TYPE, \
- { .type = (W) > 0 ? FMT_F : -1, .w = W, .d = D } },
-AGGREGATE_FUNCTIONS
-#undef AGRF
- {NULL, NULL, AGR_SV_NO, 0, -1, {-1, -1, -1}},
- };
-
-/* Missing value types. */
-enum missing_treatment
- {
- ITEMWISE, /* Missing values item by item. */
- COLUMNWISE /* Missing values column by column. */
- };
-
-/* An entire AGGREGATE procedure. */
-struct agr_proc
- {
- /* Break variables. */
- struct subcase sort; /* Sort criteria (break variables). */
- const struct variable **break_vars; /* Break variables. */
- size_t break_n_vars; /* Number of break variables. */
-
- enum missing_treatment missing; /* How to treat missing values. */
- struct agr_var *agr_vars; /* Aggregate variables. */
- size_t n_agr_vars;
- struct dictionary *dict; /* Aggregate dictionary. */
- const struct dictionary *src_dict; /* Dict of the source */
- int n_cases; /* Counts aggregated cases. */
-
- bool add_variables; /* True iff the aggregated variables should
- be appended to the existing dictionary */
- };
-
-static void initialize_aggregate_info (struct agr_proc *);
-
-static void accumulate_aggregate_info (struct agr_proc *,
- const struct ccase *);
-/* Prototypes. */
-static bool parse_aggregate_functions (struct lexer *, const struct dictionary *,
- struct agr_proc *);
-static void agr_destroy (struct agr_proc *);
-static void dump_aggregate_info (const struct agr_proc *agr,
- struct casewriter *output,
- const struct ccase *break_case);
-\f
-/* Parsing. */
-
-/* Parses and executes the AGGREGATE procedure. */
-int
-cmd_aggregate (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct agr_proc agr = {
- .missing = ITEMWISE,
- .src_dict = dict,
- };
- struct file_handle *out_file = NULL;
- struct casereader *input = NULL;
- struct casewriter *output = NULL;
-
- bool copy_documents = false;
- bool presorted = false;
- int addvariables_ofs = 0;
-
- /* OUTFILE subcommand must be first. */
- if (lex_match_phrase (lexer, "/OUTFILE") || lex_match_id (lexer, "OUTFILE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_match (lexer, T_ASTERISK))
- {
- out_file = fh_parse (lexer, FH_REF_FILE, dataset_session (ds));
- if (out_file == NULL)
- goto error;
- }
-
- if (!out_file && lex_match_id (lexer, "MODE"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "ADDVARIABLES"))
- {
- addvariables_ofs = lex_ofs (lexer) - 1;
- agr.add_variables = true;
- presorted = true;
- }
- else if (lex_match_id (lexer, "REPLACE"))
- agr.add_variables = false;
- else
- {
- lex_error_expecting (lexer, "ADDVARIABLES", "REPLACE");
- goto error;
- }
- }
- }
- else
- {
- agr.add_variables = true;
- presorted = true;
- }
-
- if (lex_match_phrase (lexer, "/MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_match_id (lexer, "COLUMNWISE"))
- {
- lex_error_expecting (lexer, "COLUMNWISE");
- goto error;
- }
- agr.missing = COLUMNWISE;
- }
-
- int presorted_ofs = 0;
- for (;;)
- if (lex_match_phrase (lexer, "/DOCUMENT"))
- copy_documents = true;
- else if (lex_match_phrase (lexer, "/PRESORTED"))
- {
- presorted = true;
- presorted_ofs = lex_ofs (lexer) - 1;
- }
- else
- break;
-
- if (agr.add_variables)
- agr.dict = dict_clone (dict);
- else
- agr.dict = dict_create (dict_get_encoding (dict));
-
- dict_set_label (agr.dict, dict_get_label (dict));
- dict_set_documents (agr.dict, dict_get_documents (dict));
-
- if (lex_match_phrase (lexer, "/BREAK"))
- {
- lex_match (lexer, T_EQUALS);
- bool saw_direction;
- int break_start = lex_ofs (lexer);
- if (!parse_sort_criteria (lexer, dict, &agr.sort, &agr.break_vars,
- &saw_direction))
- goto error;
- int break_end = lex_ofs (lexer) - 1;
- agr.break_n_vars = subcase_get_n_fields (&agr.sort);
-
- if (! agr.add_variables)
- for (size_t i = 0; i < agr.break_n_vars; i++)
- dict_clone_var_assert (agr.dict, agr.break_vars[i]);
-
- if (presorted && saw_direction)
- {
- lex_ofs_msg (lexer, SW, break_start, break_end,
- _("When the input data is presorted, specifying "
- "sorting directions with (A) or (D) has no effect. "
- "Output data will be sorted the same way as the "
- "input data."));
- if (presorted_ofs)
- lex_ofs_msg (lexer, SN, presorted_ofs, presorted_ofs,
- _("The PRESORTED subcommand state that the "
- "input data is presorted."));
- else if (addvariables_ofs)
- lex_ofs_msg (lexer, SN, addvariables_ofs, addvariables_ofs,
- _("ADDVARIABLES implies that the input data "
- "is presorted."));
- else
- msg (SN, _("The input data must be presorted because the "
- "OUTFILE subcommand is not specified."));
- }
- }
-
- /* Read in the aggregate functions. */
- if (!parse_aggregate_functions (lexer, dict, &agr))
- goto error;
-
- /* Delete documents. */
- if (!copy_documents)
- dict_clear_documents (agr.dict);
-
- /* Cancel SPLIT FILE. */
- dict_clear_split_vars (agr.dict);
-
- /* Initialize. */
- agr.n_cases = 0;
-
- if (out_file == NULL)
- {
- /* The active dataset will be replaced by the aggregated data,
- so TEMPORARY is moot. */
- proc_cancel_temporary_transformations (ds);
- proc_discard_output (ds);
- output = autopaging_writer_create (dict_get_proto (agr.dict));
- }
- else
- {
- output = any_writer_open (out_file, agr.dict);
- if (output == NULL)
- goto error;
- }
-
- input = proc_open (ds);
- if (!subcase_is_empty (&agr.sort) && !presorted)
- {
- input = sort_execute (input, &agr.sort);
- subcase_clear (&agr.sort);
- }
-
- struct casegrouper *grouper;
- struct casereader *group;
- for (grouper = casegrouper_create_vars (input, agr.break_vars,
- agr.break_n_vars);
- casegrouper_get_next_group (grouper, &group);
- casereader_destroy (group))
- {
- struct casereader *placeholder = NULL;
- struct ccase *c = casereader_peek (group, 0);
-
- if (c == NULL)
- {
- casereader_destroy (group);
- continue;
- }
-
- initialize_aggregate_info (&agr);
-
- if (agr.add_variables)
- placeholder = casereader_clone (group);
-
- {
- struct ccase *cg;
- for (; (cg = casereader_read (group)) != NULL; case_unref (cg))
- accumulate_aggregate_info (&agr, cg);
- }
-
-
- if (agr.add_variables)
- {
- struct ccase *cg;
- for (; (cg = casereader_read (placeholder)) != NULL; case_unref (cg))
- dump_aggregate_info (&agr, output, cg);
-
- casereader_destroy (placeholder);
- }
- else
- {
- dump_aggregate_info (&agr, output, c);
- }
- case_unref (c);
- }
- if (!casegrouper_destroy (grouper))
- goto error;
-
- bool ok = proc_commit (ds);
- input = NULL;
- if (!ok)
- goto error;
-
- if (out_file == NULL)
- {
- struct casereader *next_input = casewriter_make_reader (output);
- if (next_input == NULL)
- goto error;
-
- dataset_set_dict (ds, agr.dict);
- dataset_set_source (ds, next_input);
- agr.dict = NULL;
- }
- else
- {
- ok = casewriter_destroy (output);
- output = NULL;
- if (!ok)
- goto error;
- }
-
- agr_destroy (&agr);
- fh_unref (out_file);
- return CMD_SUCCESS;
-
-error:
- if (input != NULL)
- proc_commit (ds);
- casewriter_destroy (output);
- agr_destroy (&agr);
- fh_unref (out_file);
- return CMD_CASCADING_FAILURE;
-}
-
-static bool
-parse_agr_func_name (struct lexer *lexer, int *func_index,
- enum mv_class *exclude)
-{
- if (lex_token (lexer) != T_ID)
- {
- lex_error (lexer, _("Syntax error expecting aggregation function."));
- return false;
- }
-
- struct substring name = lex_tokss (lexer);
- *exclude = ss_chomp_byte (&name, '.') ? MV_SYSTEM : MV_ANY;
-
- for (const struct agr_func *f = agr_func_tab; f->name; f++)
- if (ss_equals_case (ss_cstr (f->name), name))
- {
- *func_index = f - agr_func_tab;
- lex_get (lexer);
- return true;
- }
- lex_error (lexer, _("Unknown aggregation function %s."), lex_tokcstr (lexer));
- return false;
-}
-
-/* Parse all the aggregate functions. */
-static bool
-parse_aggregate_functions (struct lexer *lexer, const struct dictionary *dict,
- struct agr_proc *agr)
-{
- if (!lex_force_match (lexer, T_SLASH))
- return false;
-
- size_t starting_n_vars = dict_get_n_vars (dict);
- size_t allocated_agr_vars = 0;
-
- /* Parse everything. */
- for (;;)
- {
- char **dest = NULL;
- char **dest_label = NULL;
- size_t n_vars = 0;
-
- struct agr_argument arg[2] = { { .f = 0 }, { .f = 0 } };
-
- const struct variable **src = NULL;
-
- /* Parse the list of target variables. */
- int dst_start_ofs = lex_ofs (lexer);
- while (!lex_match (lexer, T_EQUALS))
- {
- size_t n_vars_prev = n_vars;
-
- if (!parse_DATA_LIST_vars (lexer, dict, &dest, &n_vars,
- (PV_APPEND | PV_SINGLE | PV_NO_SCRATCH
- | PV_NO_DUPLICATE)))
- goto error;
-
- /* Assign empty labels. */
- dest_label = xnrealloc (dest_label, n_vars, sizeof *dest_label);
- for (size_t j = n_vars_prev; j < n_vars; j++)
- dest_label[j] = NULL;
-
- if (lex_is_string (lexer))
- {
- dest_label[n_vars - 1] = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
- }
- }
- int dst_end_ofs = lex_ofs (lexer) - 2;
-
- /* Get the name of the aggregation function. */
- int func_index;
- enum mv_class exclude;
- if (!parse_agr_func_name (lexer, &func_index, &exclude))
- goto error;
- const struct agr_func *function = &agr_func_tab[func_index];
-
- /* Check for leading lparen. */
- if (!lex_match (lexer, T_LPAREN))
- {
- if (function->src_vars == AGR_SV_YES)
- {
- bool ok UNUSED = lex_force_match (lexer, T_LPAREN);
- goto error;
- }
- }
- else
- {
- /* Parse list of source variables. */
- int pv_opts = PV_NO_SCRATCH;
- if (func_index == AGRF_SUM || func_index == AGRF_MEAN
- || func_index == AGRF_MEDIAN || func_index == AGRF_SD)
- pv_opts |= PV_NUMERIC;
- else if (function->n_args)
- pv_opts |= PV_SAME_TYPE;
-
- int src_start_ofs = lex_ofs (lexer);
- size_t n_src;
- if (!parse_variables_const (lexer, dict, &src, &n_src, pv_opts))
- goto error;
- int src_end_ofs = lex_ofs (lexer) - 1;
-
- /* Parse function arguments, for those functions that
- require arguments. */
- int args_start_ofs = 0;
- if (function->n_args != 0)
- for (size_t i = 0; i < function->n_args; i++)
- {
- lex_match (lexer, T_COMMA);
-
- enum val_type type;
- if (lex_is_string (lexer))
- type = VAL_STRING;
- else if (lex_is_number (lexer))
- type = VAL_NUMERIC;
- else
- {
- lex_error (lexer, _("Missing argument %zu to %s."),
- i + 1, function->name);
- goto error;
- }
-
- if (type != var_get_type (src[0]))
- {
- msg (SE, _("Arguments to %s must be of same type as "
- "source variables."),
- function->name);
- if (type == VAL_NUMERIC)
- {
- lex_next_msg (lexer, SN, 0, 0,
- _("The argument is numeric."));
- lex_ofs_msg (lexer, SN, src_start_ofs, src_end_ofs,
- _("The variables have string type."));
- }
- else
- {
- lex_next_msg (lexer, SN, 0, 0,
- _("The argument is a string."));
- lex_ofs_msg (lexer, SN, src_start_ofs, src_end_ofs,
- _("The variables are numeric."));
- }
- goto error;
- }
-
- if (i == 0)
- args_start_ofs = lex_ofs (lexer);
- if (type == VAL_NUMERIC)
- arg[i].f = lex_tokval (lexer);
- else
- arg[i].s = recode_substring_pool (dict_get_encoding (agr->dict),
- "UTF-8", lex_tokss (lexer),
- NULL);
- lex_get (lexer);
- }
- int args_end_ofs = lex_ofs (lexer) - 1;
-
- /* Trailing rparen. */
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
-
- /* Now check that the number of source variables match
- the number of target variables. If we check earlier
- than this, the user can get very misleading error
- message, i.e. `AGGREGATE x=SUM(y t).' will get this
- error message when a proper message would be more
- like `unknown variable t'. */
- if (n_src != n_vars)
- {
- msg (SE, _("Number of source variables (%zu) does not match "
- "number of target variables (%zu)."),
- n_src, n_vars);
- lex_ofs_msg (lexer, SN, src_start_ofs, src_end_ofs,
- _("These are the source variables."));
- lex_ofs_msg (lexer, SN, dst_start_ofs, dst_end_ofs,
- _("These are the target variables."));
- goto error;
- }
-
- if ((func_index == AGRF_PIN || func_index == AGRF_POUT
- || func_index == AGRF_FIN || func_index == AGRF_FOUT)
- && (var_is_numeric (src[0])
- ? arg[0].f > arg[1].f
- : buf_compare_rpad (arg[0].s.string, arg[0].s.length,
- arg[1].s.string, arg[1].s.length) > 0))
- {
- struct agr_argument tmp = arg[0];
- arg[0] = arg[1];
- arg[1] = tmp;
-
- lex_ofs_msg (lexer, SW, args_start_ofs, args_end_ofs,
- _("The value arguments passed to the %s function "
- "are out of order. They will be treated as if "
- "they had been specified in the correct order."),
- function->name);
- }
- }
-
- /* Finally add these to the aggregation variables. */
- for (size_t i = 0; i < n_vars; i++)
- {
- const struct variable *existing_var = dict_lookup_var (agr->dict,
- dest[i]);
- if (existing_var)
- {
- if (var_get_dict_index (existing_var) >= starting_n_vars)
- lex_ofs_error (lexer, dst_start_ofs, dst_end_ofs,
- _("Duplicate target variable name %s."),
- dest[i]);
- else if (agr->add_variables)
- lex_ofs_error (lexer, dst_start_ofs, dst_end_ofs,
- _("Variable name %s duplicates the name of a "
- "variable in the active file dictionary."),
- dest[i]);
- else
- lex_ofs_error (lexer, dst_start_ofs, dst_end_ofs,
- _("Variable name %s duplicates the name of a "
- "break variable."), dest[i]);
- goto error;
- }
-
- /* Add variable. */
- if (agr->n_agr_vars >= allocated_agr_vars)
- agr->agr_vars = x2nrealloc (agr->agr_vars, &allocated_agr_vars,
- sizeof *agr->agr_vars);
- struct agr_var *v = &agr->agr_vars[agr->n_agr_vars++];
- *v = (struct agr_var) {
- .exclude = exclude,
- .moments = NULL,
- .function = func_index,
- .src = src ? src[i] : NULL,
- };
-
- /* Create the target variable in the aggregate dictionary. */
- if (v->src && var_is_alpha (v->src))
- v->string = xmalloc (var_get_width (v->src));
-
- if (v->src && function->alpha_type == VAL_STRING)
- v->dest = dict_clone_var_as_assert (agr->dict, v->src, dest[i]);
- else
- {
- v->dest = dict_create_var_assert (agr->dict, dest[i], 0);
-
- struct fmt_spec f;
- if ((func_index == AGRF_N || func_index == AGRF_NMISS)
- && dict_get_weight (dict) != NULL)
- f = fmt_for_output (FMT_F, 8, 2);
- else
- f = function->format;
- var_set_both_formats (v->dest, &f);
- }
- if (dest_label[i])
- var_set_label (v->dest, dest_label[i]);
-
- if (v->src != NULL)
- for (size_t j = 0; j < function->n_args; j++)
- v->arg[j] = (struct agr_argument) {
- .f = arg[j].f,
- .s = arg[j].s.string ? ss_clone (arg[j].s) : ss_empty (),
- };
- }
-
- ss_dealloc (&arg[0].s);
- ss_dealloc (&arg[1].s);
-
- free (src);
- for (size_t i = 0; i < n_vars; i++)
- {
- free (dest[i]);
- free (dest_label[i]);
- }
- free (dest);
- free (dest_label);
-
- if (!lex_match (lexer, T_SLASH))
- {
- if (lex_token (lexer) == T_ENDCMD)
- return true;
-
- lex_error (lexer, "Syntax error expecting end of command.");
- return false;
- }
- continue;
-
- error:
- for (size_t i = 0; i < n_vars; i++)
- {
- free (dest[i]);
- free (dest_label[i]);
- }
- free (dest);
- free (dest_label);
- ss_dealloc (&arg[0].s);
- ss_dealloc (&arg[1].s);
- free (src);
-
- return false;
- }
-}
-
-/* Destroys AGR. */
-static void
-agr_destroy (struct agr_proc *agr)
-{
- subcase_uninit (&agr->sort);
- free (agr->break_vars);
- for (size_t i = 0; i < agr->n_agr_vars; i++)
- {
- struct agr_var *av = &agr->agr_vars[i];
-
- ss_dealloc (&av->arg[0].s);
- ss_dealloc (&av->arg[1].s);
- free (av->string);
-
- if (av->function == AGRF_SD)
- moments1_destroy (av->moments);
-
- dict_destroy_internal_var (av->subject);
- dict_destroy_internal_var (av->weight);
- }
- free (agr->agr_vars);
- if (agr->dict != NULL)
- dict_unref (agr->dict);
-}
-\f
-/* Execution. */
-
-/* Accumulates aggregation data from the case INPUT. */
-static void
-accumulate_aggregate_info (struct agr_proc *agr, const struct ccase *input)
-{
- bool bad_warn = true;
- double weight = dict_get_case_weight (agr->src_dict, input, &bad_warn);
- for (size_t i = 0; i < agr->n_agr_vars; i++)
- {
- struct agr_var *av = &agr->agr_vars[i];
- if (av->src)
- {
- bool is_string = var_is_alpha (av->src);
- const union value *v = case_data (input, av->src);
- int src_width = var_get_width (av->src);
- const struct substring vs = (src_width > 0
- ? value_ss (v, src_width)
- : ss_empty ());
-
- if (var_is_value_missing (av->src, v) & av->exclude)
- {
- switch (av->function)
- {
- case AGRF_NMISS:
- av->dbl += weight;
- break;
-
- case AGRF_NUMISS:
- av->int1++;
- break;
-
- case AGRF_SUM:
- case AGRF_MEAN:
- case AGRF_MEDIAN:
- case AGRF_SD:
- case AGRF_MAX:
- case AGRF_MIN:
- case AGRF_PGT:
- case AGRF_PLT:
- case AGRF_PIN:
- case AGRF_POUT:
- case AGRF_FGT:
- case AGRF_FLT:
- case AGRF_FIN:
- case AGRF_FOUT:
- case AGRF_CGT:
- case AGRF_CLT:
- case AGRF_CIN:
- case AGRF_COUT:
- case AGRF_N:
- case AGRF_NU:
- case AGRF_FIRST:
- case AGRF_LAST:
- break;
- }
- av->saw_missing = true;
- continue;
- }
-
- /* This is horrible. There are too many possibilities. */
- av->W += weight;
- switch (av->function)
- {
- case AGRF_SUM:
- av->dbl += v->f * weight;
- av->int1 = 1;
- break;
-
- case AGRF_MEAN:
- av->dbl += v->f * weight;
- break;
-
- case AGRF_MEDIAN:
- {
- struct ccase *cout = case_create (casewriter_get_proto (av->writer));
- *case_num_rw (cout, av->subject) = case_num (input, av->src);
- *case_num_rw (cout, av->weight) = weight;
- casewriter_write (av->writer, cout);
- }
- break;
-
- case AGRF_SD:
- moments1_add (av->moments, v->f, weight);
- break;
-
- case AGRF_MAX:
- if (!is_string)
- av->dbl = MAX (av->dbl, v->f);
- else if (memcmp (av->string, v->s, src_width) < 0)
- memcpy (av->string, v->s, src_width);
- av->int1 = 1;
- break;
-
- case AGRF_MIN:
- if (!is_string)
- av->dbl = MIN (av->dbl, v->f);
- else if (memcmp (av->string, v->s, src_width) > 0)
- memcpy (av->string, v->s, src_width);
- av->dbl = MIN (av->dbl, v->f);
- av->int1 = 1;
- break;
-
- case AGRF_FGT:
- case AGRF_PGT:
- case AGRF_CGT:
- if (is_string
- ? ss_compare_rpad (av->arg[0].s, vs) < 0
- : v->f > av->arg[0].f)
- av->dbl += weight;
- break;
-
- case AGRF_FLT:
- case AGRF_PLT:
- case AGRF_CLT:
- if (is_string
- ? ss_compare_rpad (av->arg[0].s, vs) > 0
- : v->f < av->arg[0].f)
- av->dbl += weight;
- break;
-
- case AGRF_FIN:
- case AGRF_PIN:
- case AGRF_CIN:
- if (is_string
- ? (ss_compare_rpad (av->arg[0].s, vs) <= 0
- && ss_compare_rpad (av->arg[1].s, vs) >= 0)
- : av->arg[0].f <= v->f && v->f <= av->arg[1].f)
- av->dbl += weight;
- break;
-
- case AGRF_FOUT:
- case AGRF_POUT:
- case AGRF_COUT:
- if (is_string
- ? (ss_compare_rpad (av->arg[0].s, vs) > 0
- || ss_compare_rpad (av->arg[1].s, vs) < 0)
- : av->arg[0].f > v->f || v->f > av->arg[1].f)
- av->dbl += weight;
- break;
-
- case AGRF_N:
- av->dbl += weight;
- break;
-
- case AGRF_NU:
- av->int1++;
- break;
-
- case AGRF_FIRST:
- if (av->int1 == 0)
- {
- if (is_string)
- memcpy (av->string, v->s, src_width);
- else
- av->dbl = v->f;
- av->int1 = 1;
- }
- break;
-
- case AGRF_LAST:
- if (is_string)
- memcpy (av->string, v->s, src_width);
- else
- av->dbl = v->f;
- av->int1 = 1;
- break;
-
- case AGRF_NMISS:
- case AGRF_NUMISS:
- /* Our value is not missing or it would have been
- caught earlier. Nothing to do. */
- break;
- }
- }
- else
- {
- av->W += weight;
- switch (av->function)
- {
- case AGRF_N:
- break;
-
- case AGRF_NU:
- av->int1++;
- break;
-
- case AGRF_SUM:
- case AGRF_MEAN:
- case AGRF_MEDIAN:
- case AGRF_SD:
- case AGRF_MAX:
- case AGRF_MIN:
- case AGRF_PGT:
- case AGRF_PLT:
- case AGRF_PIN:
- case AGRF_POUT:
- case AGRF_FGT:
- case AGRF_FLT:
- case AGRF_FIN:
- case AGRF_FOUT:
- case AGRF_CGT:
- case AGRF_CLT:
- case AGRF_CIN:
- case AGRF_COUT:
- case AGRF_NMISS:
- case AGRF_NUMISS:
- case AGRF_FIRST:
- case AGRF_LAST:
- NOT_REACHED ();
- }
- }
- }
-}
-
-/* Writes an aggregated record to OUTPUT. */
-static void
-dump_aggregate_info (const struct agr_proc *agr, struct casewriter *output, const struct ccase *break_case)
-{
- struct ccase *c = case_create (dict_get_proto (agr->dict));
-
- if (agr->add_variables)
- {
- case_copy (c, 0, break_case, 0, dict_get_n_vars (agr->src_dict));
- }
- else
- {
- int value_idx = 0;
-
- for (size_t i = 0; i < agr->break_n_vars; i++)
- {
- const struct variable *v = agr->break_vars[i];
- value_copy (case_data_rw_idx (c, value_idx),
- case_data (break_case, v),
- var_get_width (v));
- value_idx++;
- }
- }
-
- for (size_t i = 0; i < agr->n_agr_vars; i++)
- {
- struct agr_var *av = &agr->agr_vars[i];
- union value *v = case_data_rw (c, av->dest);
- int width = var_get_width (av->dest);
-
- if (agr->missing == COLUMNWISE && av->saw_missing
- && av->function != AGRF_N
- && av->function != AGRF_NU
- && av->function != AGRF_NMISS
- && av->function != AGRF_NUMISS)
- {
- value_set_missing (v, width);
- casewriter_destroy (av->writer);
- continue;
- }
-
- switch (av->function)
- {
- case AGRF_SUM:
- v->f = av->int1 ? av->dbl : SYSMIS;
- break;
-
- case AGRF_MEAN:
- v->f = av->W != 0.0 ? av->dbl / av->W : SYSMIS;
- break;
-
- case AGRF_MEDIAN:
- {
- if (av->writer)
- {
- struct percentile *median = percentile_create (0.5, av->W);
- struct order_stats *os = &median->parent;
- struct casereader *sorted_reader = casewriter_make_reader (av->writer);
- av->writer = NULL;
-
- order_stats_accumulate (&os, 1,
- sorted_reader,
- av->weight,
- av->subject,
- av->exclude);
- av->dbl = percentile_calculate (median, PC_HAVERAGE);
- statistic_destroy (&median->parent.parent);
- }
- v->f = av->dbl;
- }
- break;
-
- case AGRF_SD:
- {
- double variance;
-
- moments1_calculate (av->moments, NULL, NULL, &variance,
- NULL, NULL);
- v->f = variance != SYSMIS ? sqrt (variance) : SYSMIS;
- }
- break;
-
- case AGRF_MAX:
- case AGRF_MIN:
- case AGRF_FIRST:
- case AGRF_LAST:
- if (!width)
- v->f = av->int1 ? av->dbl : SYSMIS;
- else
- {
- if (av->int1)
- memcpy (v->s, av->string, width);
- else
- value_set_missing (v, width);
- }
- break;
-
- case AGRF_FGT:
- case AGRF_FLT:
- case AGRF_FIN:
- case AGRF_FOUT:
- v->f = av->W ? av->dbl / av->W : SYSMIS;
- break;
-
- case AGRF_PGT:
- case AGRF_PLT:
- case AGRF_PIN:
- case AGRF_POUT:
- v->f = av->W ? av->dbl / av->W * 100.0 : SYSMIS;
- break;
-
- case AGRF_CGT:
- case AGRF_CLT:
- case AGRF_CIN:
- case AGRF_COUT:
- v->f = av->dbl;
- break;
-
- case AGRF_N:
- v->f = av->W;
- break;
-
- case AGRF_NU:
- case AGRF_NUMISS:
- v->f = av->int1;
- break;
-
- case AGRF_NMISS:
- v->f = av->dbl;
- break;
- }
- }
-
- casewriter_write (output, c);
-}
-
-/* Resets the state for all the aggregate functions. */
-static void
-initialize_aggregate_info (struct agr_proc *agr)
-{
- for (size_t i = 0; i < agr->n_agr_vars; i++)
- {
- struct agr_var *av = &agr->agr_vars[i];
- av->saw_missing = false;
- av->dbl = av->W = 0.0;
- av->int1 = 0;
-
- int width = av->src ? var_get_width (av->src) : 0;
- switch (av->function)
- {
- case AGRF_MIN:
- if (!width)
- av->dbl = DBL_MAX;
- else
- memset (av->string, 255, width);
- break;
-
- case AGRF_MAX:
- if (!width)
- av->dbl = -DBL_MAX;
- else
- memset (av->string, 0, width);
- break;
-
- case AGRF_MEDIAN:
- {
- struct caseproto *proto = caseproto_create ();
- proto = caseproto_add_width (proto, 0);
- proto = caseproto_add_width (proto, 0);
-
- if (! av->subject)
- av->subject = dict_create_internal_var (0, 0);
-
- if (! av->weight)
- av->weight = dict_create_internal_var (1, 0);
-
- struct subcase ordering;
- subcase_init_var (&ordering, av->subject, SC_ASCEND);
- av->writer = sort_create_writer (&ordering, proto);
- subcase_uninit (&ordering);
- caseproto_unref (proto);
- }
- break;
-
- case AGRF_SD:
- if (av->moments == NULL)
- av->moments = moments1_create (MOMENT_VARIANCE);
- else
- moments1_clear (av->moments);
- break;
-
- case AGRF_SUM:
- case AGRF_MEAN:
- case AGRF_PGT:
- case AGRF_PLT:
- case AGRF_PIN:
- case AGRF_POUT:
- case AGRF_FGT:
- case AGRF_FLT:
- case AGRF_FIN:
- case AGRF_FOUT:
- case AGRF_CGT:
- case AGRF_CLT:
- case AGRF_CIN:
- case AGRF_COUT:
- case AGRF_N:
- case AGRF_NU:
- case AGRF_NMISS:
- case AGRF_NUMISS:
- case AGRF_FIRST:
- case AGRF_LAST:
- break;
- }
- }
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-
-#ifndef AGGREGATE_H
-#define AGGREGATE_H
-
-#include <stddef.h>
-
-#include "data/format.h"
-#include "data/val-type.h"
-
-enum agr_src_vars
- {
- AGR_SV_NO,
- AGR_SV_YES,
- AGR_SV_OPT
- };
-
-#define AGGREGATE_FUNCTIONS \
- AGRF(AGRF_SUM, "SUM", N_("Sum of values"), AGR_SV_YES, 0, -1, 8, 2) \
- AGRF(AGRF_MEAN, "MEAN", N_("Mean average"), AGR_SV_YES, 0, -1, 8, 2) \
- AGRF(AGRF_MEDIAN, "MEDIAN", N_("Median"), AGR_SV_YES, 0, -1, 8, 2) \
- AGRF(AGRF_SD, "SD", N_("Standard deviation"), AGR_SV_YES, 0, -1, 8, 2) \
- AGRF(AGRF_MAX, "MAX", N_("Maximum value"), AGR_SV_YES, 0, VAL_STRING, -1, -1) \
- AGRF(AGRF_MIN, "MIN", N_("Minimum value"), AGR_SV_YES, 0, VAL_STRING, -1, -1) \
- AGRF(AGRF_PGT, "PGT", N_("Percentage greater than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_PLT, "PLT", N_("Percentage less than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_PIN, "PIN", N_("Percentage included in range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_POUT, "POUT", N_("Percentage excluded from range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_FGT, "FGT", N_("Fraction greater than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 3) \
- AGRF(AGRF_FLT, "FLT", N_("Fraction less than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 3) \
- AGRF(AGRF_FIN, "FIN", N_("Fraction included in range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 3) \
- AGRF(AGRF_FOUT, "FOUT", N_("Fraction excluded from range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 3) \
- AGRF(AGRF_CGT, "CGT", N_("Count greater than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_CLT, "CLT", N_("Count less than"), AGR_SV_YES, 1, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_CIN, "CIN", N_("Count included in range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_COUT, "COUT", N_("Count excluded from range"), AGR_SV_YES, 2, VAL_NUMERIC, 5, 1) \
- AGRF(AGRF_N, "N", N_("Number of cases"), AGR_SV_NO, 0, VAL_NUMERIC, 7, 0) \
- AGRF(AGRF_NU, "NU", N_("Number of cases (unweighted)"), AGR_SV_OPT, 0, VAL_NUMERIC, 7, 0) \
- AGRF(AGRF_NMISS, "NMISS", N_("Number of missing values"), AGR_SV_YES, 0, VAL_NUMERIC, 7, 0) \
- AGRF(AGRF_NUMISS, "NUMISS", N_("Number of missing values (unweighted)"), AGR_SV_YES, 0, VAL_NUMERIC, 7, 0) \
- AGRF(AGRF_FIRST, "FIRST", N_("First non-missing value"), AGR_SV_YES, 0, VAL_STRING, -1, -1) \
- AGRF(AGRF_LAST, "LAST", N_("Last non-missing value"), AGR_SV_YES, 0, VAL_STRING, -1, -1)
-
-/* Aggregation functions. */
-enum agr_function
- {
-#define AGRF(ENUM, NAME, DESCRIPTION, SRC_VARS, N_ARGS, ALPHA_TYPE, W, D) \
- ENUM,
-AGGREGATE_FUNCTIONS
-#undef AGRF
- };
-
-/* Attributes of an aggregation function. */
-struct agr_func
- {
- const char *name; /* Aggregation function name. */
- const char *description; /* Translatable string describing the function. */
- enum agr_src_vars src_vars; /* Whether source variables are a parameter of the function */
- size_t n_args; /* Number of arguments (not including src vars). */
- enum val_type alpha_type; /* When given ALPHA arguments, output type. */
- struct fmt_spec format; /* Format spec if alpha_type != ALPHA. */
- };
-
-extern const struct agr_func agr_func_tab[];
-
-
-#endif
+++ /dev/null
-# PSPP - a program for statistical analysis.
-# Copyright (C) 2017 Free Software Foundation, Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-## Process this file with automake to produce Makefile.in -*- makefile -*-
-
-AM_CPPFLAGS += -I"$(top_srcdir)/src/language/stats"
-
-language_stats_sources = \
- src/language/stats/aggregate.c \
- src/language/stats/aggregate.h \
- src/language/stats/autorecode.c \
- src/language/stats/binomial.c \
- src/language/stats/binomial.h \
- src/language/stats/chart-category.h \
- src/language/stats/chisquare.c \
- src/language/stats/chisquare.h \
- src/language/stats/cochran.c \
- src/language/stats/cochran.h \
- src/language/stats/correlations.c \
- src/language/stats/crosstabs.c \
- src/language/stats/ctables.c \
- src/language/stats/ctables.inc \
- src/language/stats/descriptives.c \
- src/language/stats/examine.c \
- src/language/stats/factor.c \
- src/language/stats/flip.c \
- src/language/stats/freq.c \
- src/language/stats/freq.h \
- src/language/stats/frequencies.c \
- src/language/stats/friedman.c \
- src/language/stats/friedman.h \
- src/language/stats/glm.c \
- src/language/stats/graph.c \
- src/language/stats/kruskal-wallis.c \
- src/language/stats/kruskal-wallis.h \
- src/language/stats/ks-one-sample.c \
- src/language/stats/ks-one-sample.h \
- src/language/stats/logistic.c \
- src/language/stats/jonckheere-terpstra.c \
- src/language/stats/jonckheere-terpstra.h \
- src/language/stats/mann-whitney.c \
- src/language/stats/mann-whitney.h \
- src/language/stats/matrix.c \
- src/language/stats/means.c \
- src/language/stats/means.h \
- src/language/stats/means-calc.c \
- src/language/stats/means-parser.c \
- src/language/stats/mcnemar.c \
- src/language/stats/mcnemar.h \
- src/language/stats/median.c \
- src/language/stats/median.h \
- src/language/stats/npar.c \
- src/language/stats/npar.h \
- src/language/stats/npar-summary.c \
- src/language/stats/npar-summary.h \
- src/language/stats/oneway.c \
- src/language/stats/quick-cluster.c \
- src/language/stats/rank.c \
- src/language/stats/reliability.c \
- src/language/stats/roc.c \
- src/language/stats/roc.h \
- src/language/stats/regression.c \
- src/language/stats/runs.h \
- src/language/stats/runs.c \
- src/language/stats/sign.c \
- src/language/stats/sign.h \
- src/language/stats/sort-cases.c \
- src/language/stats/sort-criteria.c \
- src/language/stats/sort-criteria.h \
- src/language/stats/t-test.h \
- src/language/stats/t-test-indep.c \
- src/language/stats/t-test-one-sample.c \
- src/language/stats/t-test-paired.c \
- src/language/stats/t-test-parser.c \
- src/language/stats/wilcoxon.c \
- src/language/stats/wilcoxon.h
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2012, 2013, 2014
- 2021, Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/compiler.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmap.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-#include "gl/c-xvasprintf.h"
-#include "gl/mbiter.h"
-#include "gl/size_max.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-/* Explains how to recode one value. */
-struct arc_item
- {
- struct hmap_node hmap_node; /* Element in "struct arc_spec" hash table. */
- union value from; /* Original value. */
- int width; /* Width of the original value */
- bool missing; /* Is 'from' missing in its source varible? */
- char *value_label; /* Value label in source variable, if any. */
-
- double to; /* Recoded value. */
- };
-
-/* Explains how to recode an AUTORECODE variable. */
-struct arc_spec
- {
- int width; /* Variable width. */
- int src_idx; /* Case index of source variable. */
- char *src_name; /* Name of source variable. */
- struct fmt_spec format; /* Print format in source variable. */
- struct variable *dst; /* Target variable. */
- struct missing_values mv; /* Missing values of source variable. */
- char *label; /* Variable label of source variable. */
- struct rec_items *items;
- };
-
-/* Descending or ascending sort order. */
-enum arc_direction
- {
- ASCENDING,
- DESCENDING
- };
-
-struct rec_items
-{
- struct hmap ht; /* Hash table of "struct arc_item"s. */
-};
-
-
-
-/* AUTORECODE data. */
-struct autorecode_pgm
-{
- struct arc_spec *specs;
- size_t n_specs;
-
- bool blank_valid;
-};
-
-static const struct trns_class autorecode_trns_class;
-
-static int compare_arc_items (const void *, const void *, const void *aux);
-static void arc_free (struct autorecode_pgm *);
-static struct arc_item *find_arc_item (
- const struct rec_items *, const union value *, int width,
- size_t hash);
-
-/* Returns WIDTH with any trailing spaces in VALUE trimmed off (except that a
- minimum width of 1 is always returned because otherwise the width would
- indicate a numeric type). */
-static int
-value_trim_spaces (const union value *value, int width)
-{
- while (width > 1 && value->s[width - 1] == ' ')
- width--;
- return width;
-}
-
-/* Performs the AUTORECODE procedure. */
-int
-cmd_autorecode (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
-
- const struct variable **src_vars = NULL;
- size_t n_srcs = 0;
-
- char **dst_names = NULL;
- size_t n_dsts = 0;
-
- enum arc_direction direction = ASCENDING;
- bool print = false;
-
- /* Create procedure. */
- struct autorecode_pgm *arc = XZALLOC (struct autorecode_pgm);
- arc->blank_valid = true;
-
- /* Parse variable lists. */
- lex_match_id (lexer, "VARIABLES");
- lex_match (lexer, T_EQUALS);
- if (!parse_variables_const (lexer, dict, &src_vars, &n_srcs,
- PV_NO_DUPLICATE | PV_NO_SCRATCH))
- goto error;
- lex_match (lexer, T_SLASH);
- if (!lex_force_match_id (lexer, "INTO"))
- goto error;
- lex_match (lexer, T_EQUALS);
- if (!parse_DATA_LIST_vars (lexer, dict, &dst_names, &n_dsts,
- PV_NO_DUPLICATE))
- goto error;
- if (n_dsts != n_srcs)
- {
- msg (SE, _("Source variable count (%zu) does not match "
- "target variable count (%zu)."),
- n_srcs, n_dsts);
-
- goto error;
- }
- for (size_t i = 0; i < n_dsts; i++)
- {
- const char *name = dst_names[i];
-
- if (dict_lookup_var (dict, name) != NULL)
- {
- msg (SE, _("Target variable %s duplicates existing variable %s."),
- name, name);
- goto error;
- }
- }
-
- /* Parse options. */
- bool group = false;
- while (lex_match (lexer, T_SLASH))
- {
- if (lex_match_id (lexer, "DESCENDING"))
- direction = DESCENDING;
- else if (lex_match_id (lexer, "PRINT"))
- print = true;
- else if (lex_match_id (lexer, "GROUP"))
- group = true;
- else if (lex_match_id (lexer, "BLANK"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "VALID"))
- {
- arc->blank_valid = true;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- arc->blank_valid = false;
- }
- else
- {
- lex_error_expecting (lexer, "VALID", "MISSING");
- goto error;
- }
- }
- else
- {
- lex_error_expecting (lexer, "DESCENDING", "PRINT", "GROUP", "BLANK");
- goto error;
- }
- }
-
- if (lex_token (lexer) != T_ENDCMD)
- {
- lex_error (lexer, _("Syntax error expecting end of command."));
- goto error;
- }
-
- /* If GROUP is specified, verify that the variables are all string or all
- numeric. */
- if (group)
- {
- enum val_type type = var_get_type (src_vars[0]);
- for (size_t i = 1; i < n_dsts; i++)
- {
- if (var_get_type (src_vars[i]) != type)
- {
- size_t string_idx = type == VAL_STRING ? 0 : i;
- size_t numeric_idx = type == VAL_STRING ? i : 0;
- lex_error (lexer, _("With GROUP, variables may not mix string "
- "variables (such as %s) and numeric "
- "variables (such as %s)."),
- var_get_name (src_vars[string_idx]),
- var_get_name (src_vars[numeric_idx]));
- goto error;
- }
- }
- }
-
- /* Allocate all the specs and the rec_items that they point to.
-
- If GROUP is specified, there is only a single global rec_items, with the
- maximum width 'width', and all of the specs point to it; otherwise each
- spec has its own rec_items. */
- arc->specs = xmalloc (n_dsts * sizeof *arc->specs);
- arc->n_specs = n_dsts;
- for (size_t i = 0; i < n_dsts; i++)
- {
- struct arc_spec *spec = &arc->specs[i];
-
- spec->width = var_get_width (src_vars[i]);
- spec->src_idx = var_get_case_index (src_vars[i]);
- spec->src_name = xstrdup (var_get_name (src_vars[i]));
- spec->format = *var_get_print_format (src_vars[i]);
-
- const char *label = var_get_label (src_vars[i]);
- spec->label = xstrdup_if_nonnull (label);
-
- if (group && i > 0)
- spec->items = arc->specs[0].items;
- else
- {
- spec->items = xzalloc (sizeof (*spec->items));
- hmap_init (&spec->items->ht);
- }
- }
-
- /* Initialize specs[*]->mv to the user-missing values for each
- source variable. */
- if (group)
- {
- /* Use the first source variable that has any user-missing values. */
- size_t mv_idx = 0;
- for (size_t i = 0; i < n_dsts; i++)
- if (var_has_missing_values (src_vars[i]))
- {
- mv_idx = i;
- break;
- }
-
- for (size_t i = 0; i < n_dsts; i++)
- mv_copy (&arc->specs[i].mv, var_get_missing_values (src_vars[mv_idx]));
- }
- else
- {
- /* Each variable uses its own user-missing values. */
- for (size_t i = 0; i < n_dsts; i++)
- mv_copy (&arc->specs[i].mv, var_get_missing_values (src_vars[i]));
- }
-
- /* Execute procedure. */
- struct casereader *input = proc_open (ds);
- struct ccase *c;
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- for (size_t i = 0; i < arc->n_specs; i++)
- {
- struct arc_spec *spec = &arc->specs[i];
- const union value *value = case_data_idx (c, spec->src_idx);
- if (spec->width == 0 && value->f == SYSMIS)
- {
- /* AUTORECODE never changes the system-missing value.
- (Leaving it out of the translation table has this
- effect automatically because values not found in the
- translation table get translated to system-missing.) */
- continue;
- }
-
- int width = value_trim_spaces (value, spec->width);
- if (width == 1 && value->s[0] == ' ' && !arc->blank_valid)
- continue;
-
- size_t hash = value_hash (value, width, 0);
- if (find_arc_item (spec->items, value, width, hash))
- continue;
-
- struct string value_label = DS_EMPTY_INITIALIZER;
- var_append_value_name__ (src_vars[i], value,
- SETTINGS_VALUE_SHOW_LABEL, &value_label);
-
- struct arc_item *item = xmalloc (sizeof *item);
- item->width = width;
- value_clone (&item->from, value, width);
- item->missing = mv_is_value_missing_varwidth (&spec->mv, value,
- spec->width);
- item->value_label = ds_steal_cstr (&value_label);
- hmap_insert (&spec->items->ht, &item->hmap_node, hash);
-
- ds_destroy (&value_label);
- }
- bool ok = casereader_destroy (input);
- ok = proc_commit (ds) && ok;
-
- /* Re-fetch dictionary because it might have changed (if TEMPORARY was in
- use). */
- dict = dataset_dict (ds);
-
- /* Create transformation. */
- for (size_t i = 0; i < arc->n_specs; i++)
- {
- struct arc_spec *spec = &arc->specs[i];
-
- /* Create destination variable. */
- spec->dst = dict_create_var_assert (dict, dst_names[i], 0);
- var_set_label (spec->dst, spec->label);
-
- /* Set print format. */
- size_t n_items = hmap_count (&spec->items->ht);
- char *longest_value = xasprintf ("%zu", n_items);
- struct fmt_spec format = { .type = FMT_F, .w = strlen (longest_value) };
- var_set_both_formats (spec->dst, &format);
- free (longest_value);
-
- /* Create array of pointers to items. */
- struct arc_item **items = xmalloc (n_items * sizeof *items);
- struct arc_item *item;
- size_t j = 0;
- HMAP_FOR_EACH (item, struct arc_item, hmap_node, &spec->items->ht)
- items[j++] = item;
- assert (j == n_items);
-
- /* Sort array by value. */
- sort (items, n_items, sizeof *items, compare_arc_items, &direction);
-
- /* Assign recoded values in sorted order. */
- for (j = 0; j < n_items; j++)
- items[j]->to = j + 1;
-
- if (print && (!group || i == 0))
- {
- struct pivot_value *title
- = (group
- ? pivot_value_new_text (N_("Recoding grouped variables."))
- : spec->label && spec->label[0]
- ? pivot_value_new_text_format (N_("Recoding %s into %s (%s)."),
- spec->src_name,
- var_get_name (spec->dst),
- spec->label)
- : pivot_value_new_text_format (N_("Recoding %s into %s."),
- spec->src_name,
- var_get_name (spec->dst)));
- struct pivot_table *table = pivot_table_create__ (title, "Recoding");
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Attributes"),
- N_("New Value"), N_("Value Label"));
-
- struct pivot_dimension *old_values = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Old Value"));
- old_values->root->show_label = true;
-
- for (size_t k = 0; k < n_items; k++)
- {
- const struct arc_item *item = items[k];
- int old_value_idx = pivot_category_create_leaf (
- old_values->root, pivot_value_new_value (
- &item->from, item->width,
- (item->width
- ? &(struct fmt_spec) { .type = FMT_F, .w = item->width }
- : &spec->format),
- dict_get_encoding (dict)));
- pivot_table_put2 (table, 0, old_value_idx,
- pivot_value_new_integer (item->to));
-
- const char *value_label = item->value_label;
- if (value_label && value_label[0])
- pivot_table_put2 (table, 1, old_value_idx,
- pivot_value_new_user_text (value_label, -1));
- }
-
- pivot_table_submit (table);
- }
-
- /* Assign user-missing values.
-
- User-missing values in the source variable(s) must be marked
- as user-missing values in the destination variable. There
- might be an arbitrary number of missing values, since the
- source variable might have a range. Our sort function always
- puts missing values together at the top of the range, so that
- means that we can use a missing value range to cover all of
- the user-missing values in any case (but we avoid it unless
- necessary because user-missing value ranges are an obscure
- feature). */
- size_t n_missing = n_items;
- for (size_t k = 0; k < n_items; k++)
- if (!items[n_items - k - 1]->missing)
- {
- n_missing = k;
- break;
- }
- if (n_missing > 0)
- {
- size_t lo = n_items - (n_missing - 1);
- size_t hi = n_items;
-
- struct missing_values mv;
- mv_init (&mv, 0);
- if (n_missing > 3)
- mv_add_range (&mv, lo, hi);
- else
- for (size_t k = 0; k < n_missing; k++)
- mv_add_num (&mv, lo + k);
- var_set_missing_values (spec->dst, &mv);
- mv_destroy (&mv);
- }
-
- /* Add value labels to the destination variable. */
- for (j = 0; j < n_items; j++)
- {
- const char *value_label = items[j]->value_label;
- if (value_label && value_label[0])
- {
- union value to_val = { .f = items[j]->to };
- var_add_value_label (spec->dst, &to_val, value_label);
- }
- }
-
- /* Free array. */
- free (items);
- }
- add_transformation (ds, &autorecode_trns_class, arc);
-
- for (size_t i = 0; i < n_dsts; i++)
- free (dst_names[i]);
- free (dst_names);
- free (src_vars);
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-
-error:
- for (size_t i = 0; i < n_dsts; i++)
- free (dst_names[i]);
- free (dst_names);
- free (src_vars);
- arc_free (arc);
- return CMD_CASCADING_FAILURE;
-}
-
-static void
-arc_free (struct autorecode_pgm *arc)
-{
- if (arc != NULL)
- {
- for (size_t i = 0; i < arc->n_specs; i++)
- {
- struct arc_spec *spec = &arc->specs[i];
- struct arc_item *item, *next;
-
- HMAP_FOR_EACH_SAFE (item, next, struct arc_item, hmap_node,
- &spec->items->ht)
- {
- value_destroy (&item->from, item->width);
- free (item->value_label);
- hmap_delete (&spec->items->ht, &item->hmap_node);
- free (item);
- }
- free (spec->label);
- free (spec->src_name);
- mv_destroy (&spec->mv);
- }
-
- size_t n_rec_items =
- (arc->n_specs >= 2 && arc->specs[0].items == arc->specs[1].items
- ? 1
- : arc->n_specs);
-
- for (size_t i = 0; i < n_rec_items; i++)
- {
- struct arc_spec *spec = &arc->specs[i];
- hmap_destroy (&spec->items->ht);
- free (spec->items);
- }
-
- free (arc->specs);
- free (arc);
- }
-}
-
-static struct arc_item *
-find_arc_item (const struct rec_items *items,
- const union value *value, int width,
- size_t hash)
-{
- struct arc_item *item;
-
- HMAP_FOR_EACH_WITH_HASH (item, struct arc_item, hmap_node, hash, &items->ht)
- if (item->width == width && value_equal (value, &item->from, width))
- return item;
- return NULL;
-}
-
-static int
-compare_arc_items (const void *a_, const void *b_, const void *direction_)
-{
- const struct arc_item *const *ap = a_;
- const struct arc_item *const *bp = b_;
- const struct arc_item *a = *ap;
- const struct arc_item *b = *bp;
-
- /* User-missing values always sort to the highest target values
- (regardless of sort direction). */
- if (a->missing != b->missing)
- return a->missing < b->missing ? -1 : 1;
-
- /* Otherwise, compare the data. */
- int aw = a->width;
- int bw = b->width;
- int cmp;
- if (aw == bw)
- cmp = value_compare_3way (&a->from, &b->from, aw);
- else
- {
- assert (aw && bw);
- cmp = buf_compare_rpad (CHAR_CAST_BUG (const char *, a->from.s), aw,
- CHAR_CAST_BUG (const char *, b->from.s), bw);
- }
-
- /* Then apply sort direction. */
- const enum arc_direction *directionp = direction_;
- enum arc_direction direction = *directionp;
- return direction == ASCENDING ? cmp : -cmp;
-}
-
-static enum trns_result
-autorecode_trns_proc (void *arc_, struct ccase **c,
- casenumber case_idx UNUSED)
-{
- struct autorecode_pgm *arc = arc_;
-
- *c = case_unshare (*c);
- for (size_t i = 0; i < arc->n_specs; i++)
- {
- const struct arc_spec *spec = &arc->specs[i];
- const union value *value = case_data_idx (*c, spec->src_idx);
- int width = value_trim_spaces (value, spec->width);
- size_t hash = value_hash (value, width, 0);
- const struct arc_item *item = find_arc_item (spec->items, value, width,
- hash);
- *case_num_rw (*c, spec->dst) = item ? item->to : SYSMIS;
- }
-
- return TRNS_CONTINUE;
-}
-
-static bool
-autorecode_trns_free (void *arc_)
-{
- struct autorecode_pgm *arc = arc_;
-
- arc_free (arc);
- return true;
-}
-
-static const struct trns_class autorecode_trns_class = {
- .name = "AUTORECODE",
- .execute = autorecode_trns_proc,
- .destroy = autorecode_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2009, 2010, 2011, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/binomial.h"
-
-#include <float.h>
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_randist.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value-labels.h"
-#include "data/value.h"
-#include "data/variable.h"
-#include "language/stats/freq.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-#include "gl/minmax.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-static double calculate_binomial_internal (double n1, double n2,
- double p);
-
-
-static void
-swap (double *i1, double *i2)
-{
- double temp = *i1;
- *i1 = *i2;
- *i2 = temp;
-}
-
-static double
-calculate_binomial (double n1, double n2, double p)
-{
- const double n = n1 + n2;
- const bool test_reversed = (n1 / n > p) ;
- if (test_reversed)
- {
- p = 1 - p ;
- swap (&n1, &n2);
- }
-
- return calculate_binomial_internal (n1, n2, p);
-}
-
-static double
-calculate_binomial_internal (double n1, double n2, double p)
-{
- /* SPSS Statistical Algorithms has completely different and WRONG
- advice here. */
-
- double sig1tailed = gsl_cdf_binomial_P (n1, p, n1 + n2);
-
- if (p == 0.5)
- return sig1tailed > 0.5 ? 1.0 :sig1tailed * 2.0;
-
- return sig1tailed ;
-}
-
-static bool
-do_binomial (const struct dictionary *dict,
- struct casereader *input,
- const struct one_sample_test *ost,
- struct freq *cat1,
- struct freq *cat2,
- enum mv_class exclude
- )
-{
- const struct binomial_test *bst = UP_CAST (ost, const struct binomial_test, parent);
- bool warn = true;
-
- struct ccase *c;
-
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- {
- int v;
- double w = dict_get_case_weight (dict, c, &warn);
-
- for (v = 0 ; v < ost->n_vars ; ++v)
- {
- const struct variable *var = ost->vars[v];
- double value = case_num (c, var);
-
- if (var_is_num_missing (var, value) & exclude)
- continue;
-
- if (bst->cutpoint != SYSMIS)
- {
- if (cat1[v].values[0].f >= value)
- cat1[v].count += w;
- else
- cat2[v].count += w;
- }
- else
- {
- if (SYSMIS == cat1[v].values[0].f)
- {
- cat1[v].values[0].f = value;
- cat1[v].count = w;
- }
- else if (cat1[v].values[0].f == value)
- cat1[v].count += w;
- else if (SYSMIS == cat2[v].values[0].f)
- {
- cat2[v].values[0].f = value;
- cat2[v].count = w;
- }
- else if (cat2[v].values[0].f == value)
- cat2[v].count += w;
- else if (bst->category1 == SYSMIS)
- msg (ME, _("Variable %s is not dichotomous"), var_get_name (var));
- }
- }
- }
- return casereader_destroy (input);
-}
-
-
-
-void
-binomial_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- const struct dictionary *dict = dataset_dict (ds);
- const struct one_sample_test *ost = UP_CAST (test, const struct one_sample_test, parent);
- const struct binomial_test *bst = UP_CAST (ost, const struct binomial_test, parent);
-
- struct freq *cat[2];
- int i;
-
- assert ((bst->category1 == SYSMIS) == (bst->category2 == SYSMIS) || bst->cutpoint != SYSMIS);
-
- for (i = 0; i < 2; i++)
- {
- double value;
- if (i == 0)
- value = bst->cutpoint != SYSMIS ? bst->cutpoint : bst->category1;
- else
- value = bst->category2;
-
- cat[i] = xnmalloc (ost->n_vars, sizeof *cat[i]);
- for (size_t v = 0; v < ost->n_vars; v++)
- {
- cat[i][v].values[0].f = value;
- cat[i][v].count = 0;
- }
- }
-
- if (do_binomial (dataset_dict (ds), input, ost, cat[0], cat[1], exclude))
- {
- struct pivot_table *table = pivot_table_create (N_("Binomial Test"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Category"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Observed Prop."), PIVOT_RC_OTHER,
- N_("Test Prop."), PIVOT_RC_OTHER,
- (bst->p == 0.5
- ? N_("Exact Sig. (2-tailed)")
- : N_("Exact Sig. (1-tailed)")), PIVOT_RC_SIGNIFICANCE);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Groups"),
- N_("Group 1"), N_("Group 2"), N_("Total"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t v = 0; v < ost->n_vars; ++v)
- {
- const struct variable *var = ost->vars[v];
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (var));
-
- /* Category. */
- if (bst->cutpoint != SYSMIS)
- pivot_table_put3 (
- table, 0, 0, var_idx,
- pivot_value_new_user_text_nocopy (
- xasprintf ("<= %.*g", DBL_DIG + 1, bst->cutpoint)));
- else
- for (int i = 0; i < 2; i++)
- pivot_table_put3 (
- table, 0, i, var_idx,
- pivot_value_new_var_value (var, cat[i][v].values));
-
- double n_total = cat[0][v].count + cat[1][v].count;
- double sig = calculate_binomial (cat[0][v].count, cat[1][v].count,
- bst->p);
- struct entry
- {
- int stat_idx;
- int group_idx;
- double x;
- }
- entries[] = {
- /* N. */
- { 1, 0, cat[0][v].count },
- { 1, 1, cat[1][v].count },
- { 1, 2, n_total },
- /* Observed Prop. */
- { 2, 0, cat[0][v].count / n_total },
- { 2, 1, cat[1][v].count / n_total },
- { 2, 2, 1.0 },
- /* Test Prop. */
- { 3, 0, bst->p },
- /* Significance. */
- { 4, 0, sig }
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put3 (table, e->stat_idx, e->group_idx,
- var_idx, pivot_value_new_number (e->x));
- }
- }
-
- pivot_table_submit (table);
- }
-
- for (i = 0; i < 2; i++)
- free (cat[i]);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !binomial_h
-#define binomial_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-
-#include "npar.h"
-
-
-struct binomial_test
-{
- struct one_sample_test parent;
- double p;
- double category1;
- double category2;
- double cutpoint;
-};
-
-
-struct casereader;
-struct dataset;
-
-
-void binomial_execute (const struct dataset *,
- struct casereader *,
- enum mv_class,
- const struct npar_test *,
- bool, double);
-
-#endif
+++ /dev/null
-/*
-PSPP - a program for statistical analysis.
-Copyright (C) 2017 Free Software Foundation, Inc.
-
-This program is free software: you can redistribute it and/or modify
-it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or
-(at your option) any later version.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#ifndef BARCHART_DEF_H
-#define BARCHART_DEF_H 1
-
-#include <stdbool.h>
-
-struct ag_func
-{
- const char *name;
- const char *description;
-
- int arity;
- bool cumulative;
- double (*pre) (void);
- double (*calc) (double acc, double x, double w);
- double (*post) (double acc, double cc);
- double (*ppost) (double acc, double ccc);
-};
-
-extern const struct ag_func ag_func[];
-
-extern const int N_AG_FUNCS;
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2007, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/chisquare.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/stats/freq.h"
-#include "language/stats/npar.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/compiler.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/message.h"
-#include "libpspp/taint.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-/* Adds frequency counts of each value of VAR in INPUT between LO and HI to
- FREQ_HASH. LO and HI and each input value is truncated to an integer.
- Returns true if successful, false on input error. It is the caller's
- responsibility to initialize FREQ_HASH and to free it when no longer
- required, even on failure. */
-static bool
-create_freq_hash_with_range (const struct dictionary *dict,
- struct casereader *input,
- const struct variable *var,
- double lo_, double hi_,
- struct hmap *freq_hash)
-{
- struct freq **entries;
- bool warn = true;
- struct ccase *c;
- double lo, hi;
- double i_d;
-
- assert (var_is_numeric (var));
- lo = trunc (lo_);
- hi = trunc (hi_);
-
- /* Populate the hash with zero entries */
- entries = xnmalloc (hi - lo + 1, sizeof *entries);
- for (i_d = lo; i_d <= hi; i_d += 1.0)
- {
- size_t ofs = i_d - lo;
- union value value = { i_d };
- entries[ofs] = freq_hmap_insert (freq_hash, &value, 0,
- value_hash (&value, 0, 0));
- }
-
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- {
- double x = trunc (case_num (c, var));
- if (x >= lo && x <= hi)
- {
- size_t ofs = x - lo;
- struct freq *fr = entries[ofs];
- fr->count += dict_get_case_weight (dict, c, &warn);
- }
- }
-
- free (entries);
-
- return casereader_destroy (input);
-}
-
-/* Adds frequency counts of each value of VAR in INPUT to FREQ_HASH. LO and HI
- and each input value is truncated to an integer. Returns true if
- successful, false on input error. It is the caller's responsibility to
- initialize FREQ_HASH and to free it when no longer required, even on
- failure. */
-static bool
-create_freq_hash (const struct dictionary *dict,
- struct casereader *input,
- const struct variable *var,
- struct hmap *freq_hash)
-{
- int width = var_get_width (var);
- bool warn = true;
- struct ccase *c;
-
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- {
- const union value *value = case_data (c, var);
- size_t hash = value_hash (value, width, 0);
- double weight = dict_get_case_weight (dict, c, &warn);
- struct freq *f;
-
- f = freq_hmap_search (freq_hash, value, width, hash);
- if (f == NULL)
- f = freq_hmap_insert (freq_hash, value, width, hash);
-
- f->count += weight;
- }
-
- return casereader_destroy (input);
-}
-
-void
-chisquare_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- const struct dictionary *dict = dataset_dict (ds);
- int v, i;
- struct chisquare_test *cst = UP_CAST (test, struct chisquare_test,
- parent.parent);
- struct one_sample_test *ost = &cst->parent;
- double total_expected = 0.0;
-
- double *df = XCALLOC (ost->n_vars, double);
- double *xsq = XCALLOC (ost->n_vars, double);
- bool ok;
-
- for (i = 0 ; i < cst->n_expected ; ++i)
- total_expected += cst->expected[i];
-
- if (cst->ranged == false)
- {
- for (v = 0 ; v < ost->n_vars ; ++v)
- {
- const struct variable *var = ost->vars[v];
-
- struct hmap freq_hash = HMAP_INITIALIZER (freq_hash);
- struct casereader *reader =
- casereader_create_filter_missing (casereader_clone (input),
- &var, 1, exclude,
- NULL, NULL);
- if (!create_freq_hash (dict, reader, var, &freq_hash))
- {
- freq_hmap_destroy (&freq_hash, var_get_width (var));
- return;
- }
-
- size_t n_cells = hmap_count (&freq_hash);
- if (cst->n_expected > 0 && n_cells != cst->n_expected)
- {
- msg (ME, _("CHISQUARE test specified %d expected values, but "
- "variable %s has %zu distinct values."),
- cst->n_expected, var_get_name (var), n_cells);
- freq_hmap_destroy (&freq_hash, var_get_width (var));
- continue;
- }
-
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_variable (var), "Chisquare");
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Observed N"), PIVOT_RC_COUNT,
- N_("Expected N"), PIVOT_RC_OTHER,
- N_("Residual"), PIVOT_RC_RESIDUAL);
-
- struct freq **ff = freq_hmap_sort (&freq_hash, var_get_width (var));
-
- double total_obs = 0.0;
- for (size_t i = 0; i < n_cells; i++)
- total_obs += ff[i]->count;
-
- struct pivot_dimension *values = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Value"));
- values->root->show_label = true;
-
- xsq[v] = 0.0;
- for (size_t i = 0; i < n_cells; i++)
- {
- int row = pivot_category_create_leaf (
- values->root, pivot_value_new_var_value (
- var, &ff[i]->values[0]));
-
- double exp = (cst->n_expected > 0
- ? cst->expected[i] * total_obs / total_expected
- : total_obs / (double) n_cells);
- double entries[] = {
- ff[i]->count,
- exp,
- ff[i]->count - exp,
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (
- table, j, row, pivot_value_new_number (entries[j]));
-
- xsq[v] += (ff[i]->count - exp) * (ff[i]->count - exp) / exp;
- }
-
- df[v] = n_cells - 1.0;
-
- int row = pivot_category_create_leaf (
- values->root, pivot_value_new_text (N_("Total")));
- pivot_table_put2 (table, 0, row,
- pivot_value_new_number (total_obs));
-
- pivot_table_submit (table);
-
- freq_hmap_destroy (&freq_hash, var_get_width (var));
- free (ff);
- }
- }
- else /* ranged == true */
- {
- struct pivot_table *table = pivot_table_create (N_("Frequencies"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Category"),
- N_("Observed N"), PIVOT_RC_COUNT,
- N_("Expected N"), PIVOT_RC_OTHER,
- N_("Residual"), PIVOT_RC_RESIDUAL);
-
- struct pivot_dimension *var_dim
- = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Variable"));
- for (size_t i = 0 ; i < ost->n_vars ; ++i)
- pivot_category_create_leaf (var_dim->root,
- pivot_value_new_variable (ost->vars[i]));
-
- struct pivot_dimension *category_dim
- = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Category"));
- size_t n_cells = cst->hi - cst->lo + 1;
- for (size_t i = 0 ; i < n_cells; ++i)
- pivot_category_create_leaf (category_dim->root,
- pivot_value_new_integer (i + 1));
- pivot_category_create_leaves (category_dim->root, N_("Total"));
-
- for (size_t v = 0 ; v < ost->n_vars ; ++v)
- {
- const struct variable *var = ost->vars[v];
- struct casereader *reader =
- casereader_create_filter_missing (casereader_clone (input),
- &var, 1, exclude,
- NULL, NULL);
- struct hmap freq_hash = HMAP_INITIALIZER (freq_hash);
- if (!create_freq_hash_with_range (dict, reader, var,
- cst->lo, cst->hi, &freq_hash))
- {
- freq_hmap_destroy (&freq_hash, var_get_width (var));
- continue;
- }
-
- struct freq **ff = freq_hmap_sort (&freq_hash, var_get_width (var));
-
- double total_obs = 0.0;
- for (size_t i = 0 ; i < hmap_count (&freq_hash) ; ++i)
- total_obs += ff[i]->count;
-
- xsq[v] = 0.0;
- for (size_t i = 0 ; i < hmap_count (&freq_hash) ; ++i)
- {
- /* Category. */
- pivot_table_put3 (table, 0, v, i,
- pivot_value_new_var_value (
- var, &ff[i]->values[0]));
-
- double exp = (cst->n_expected > 0
- ? cst->expected[i] * total_obs / total_expected
- : total_obs / (double) hmap_count (&freq_hash));
- double entries[] = {
- ff[i]->count,
- exp,
- ff[i]->count - exp,
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put3 (table, j + 1, v, i,
- pivot_value_new_number (entries[j]));
-
-
- xsq[v] += (ff[i]->count - exp) * (ff[i]->count - exp) / exp;
- }
-
- df[v] = n_cells - 1.0;
-
- freq_hmap_destroy (&freq_hash, var_get_width (var));
- free (ff);
-
- pivot_table_put3 (table, 1, v, n_cells,
- pivot_value_new_number (total_obs));
- }
-
- pivot_table_submit (table);
- }
- ok = !taint_has_tainted_successor (casereader_get_taint (input));
- casereader_destroy (input);
-
- if (ok)
- {
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Chi-square"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t v = 0 ; v < ost->n_vars ; ++v)
- {
- const struct variable *var = ost->vars[v];
-
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (var));
-
- double sig = gsl_cdf_chisq_Q (xsq[v], df[v]);
- double entries[] = { xsq[v], df[v], sig };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, row,
- pivot_value_new_number (entries[i]));
- }
- pivot_table_submit (table);
- }
-
- free (xsq);
- free (df);
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !chisquare_h
-#define chisquare_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-struct chisquare_test
-{
- struct one_sample_test parent;
-
- bool ranged ; /* True if this test has a range specified */
-
- int lo; /* Lower bound of range (undefined if RANGED is false) */
- int hi; /* Upper bound of range (undefined if RANGED is false) */
-
- double *expected;
- int n_expected;
-};
-
-struct casereader;
-struct dataset;
-
-
-void chisquare_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool,
- double);
-
-
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/cochran.h"
-
-#include <float.h>
-#include <gsl/gsl_cdf.h>
-#include <stdbool.h>
-
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/val-type.h"
-#include "data/variable.h"
-#include "language/stats/npar.h"
-#include "libpspp/cast.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-struct cochran
-{
- double success;
- double failure;
-
- double *hits;
- double *misses;
-
- const struct dictionary *dict;
- double cc;
- double df;
- double q;
-};
-
-static void show_freqs_box (const struct one_sample_test *ost, const struct cochran *ch);
-static void show_sig_box (const struct cochran *ch);
-
-void
-cochran_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED, double timer UNUSED)
-{
- struct one_sample_test *ct = UP_CAST (test, struct one_sample_test, parent);
- int v;
- struct cochran ch;
- const struct dictionary *dict = dataset_dict (ds);
- const struct variable *weight = dict_get_weight (dict);
-
- struct ccase *c;
- double rowsq = 0;
- ch.cc = 0.0;
- ch.dict = dict;
- ch.success = SYSMIS;
- ch.failure = SYSMIS;
- ch.hits = xcalloc (ct->n_vars, sizeof *ch.hits);
- ch.misses = xcalloc (ct->n_vars, sizeof *ch.misses);
-
- for (; (c = casereader_read (input)); case_unref (c))
- {
- double case_hits = 0.0;
- const double w = weight ? case_num (c, weight) : 1.0;
- for (v = 0; v < ct->n_vars; ++v)
- {
- const struct variable *var = ct->vars[v];
- const union value *val = case_data (c, var);
-
- if (var_is_value_missing (var, val) & exclude)
- continue;
-
- if (ch.success == SYSMIS)
- {
- ch.success = val->f;
- }
- else if (ch.failure == SYSMIS && val->f != ch.success)
- {
- ch.failure = val->f;
- }
- if (ch.success == val->f)
- {
- ch.hits[v] += w;
- case_hits += w;
- }
- else if (ch.failure == val->f)
- {
- ch.misses[v] += w;
- }
- else
- {
- msg (MW, _("More than two values encountered. Cochran Q test will not be run."));
- goto finish;
- }
- }
- ch.cc += w;
- rowsq += pow2 (case_hits);
- }
- casereader_destroy (input);
-
- {
- double c_l = 0;
- double c_l2 = 0;
- for (v = 0; v < ct->n_vars; ++v)
- {
- c_l += ch.hits[v];
- c_l2 += pow2 (ch.hits[v]);
- }
-
- ch.q = ct->n_vars * c_l2;
- ch.q -= pow2 (c_l);
- ch.q *= ct->n_vars - 1;
-
- ch.q /= ct->n_vars * c_l - rowsq;
-
- ch.df = ct->n_vars - 1;
- }
-
- show_freqs_box (ct, &ch);
- show_sig_box (&ch);
-
- finish:
-
- free (ch.hits);
- free (ch.misses);
-}
-
-static void
-show_freqs_box (const struct one_sample_test *ost, const struct cochran *ct)
-{
- struct pivot_table *table = pivot_table_create (N_("Frequencies"));
- pivot_table_set_weight_var (table, dict_get_weight (ct->dict));
-
- char *success = xasprintf (_("Success (%.*g)"), DBL_DIG + 1, ct->success);
- char *failure = xasprintf (_("Failure (%.*g)"), DBL_DIG + 1, ct->failure);
- struct pivot_dimension *values = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Value"),
- success, PIVOT_RC_COUNT,
- failure, PIVOT_RC_COUNT);
- values->root->show_label = true;
- free (failure);
- free (success);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t i = 0 ; i < ost->n_vars ; ++i)
- {
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (ost->vars[i]));
-
- pivot_table_put2 (table, 0, row, pivot_value_new_number (ct->hits[i]));
- pivot_table_put2 (table, 1, row, pivot_value_new_number (ct->misses[i]));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_sig_box (const struct cochran *ch)
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
-
- pivot_table_set_weight_format (table, dict_get_weight_format (ch->dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Value"), N_("Value"));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Cochran's Q"), PIVOT_RC_SIGNIFICANCE,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- double sig = gsl_cdf_chisq_Q (ch->q, ch->df);
- double entries[] = { ch->cc, ch->q, ch->df, sig };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, 0, i, pivot_value_new_number (entries[i]));
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !cochran_h
-#define cochran_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-
-
-void cochran_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool,
- double);
-
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_matrix.h>
-#include <math.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/correlation.h"
-#include "math/covariance.h"
-#include "math/moments.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-#include "gl/minmax.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-
-struct corr
- {
- size_t n_vars_total;
- size_t n_vars1;
-
- const struct variable **vars;
- };
-
-
-/* Handling of missing values. */
-enum corr_missing_type
- {
- CORR_PAIRWISE, /* Handle missing values on a per-variable-pair basis. */
- CORR_LISTWISE /* Discard entire case if any variable is missing. */
- };
-
-struct corr_opts
-{
- enum corr_missing_type missing_type;
- enum mv_class exclude; /* Classes of missing values to exclude. */
-
- bool sig; /* Flag significant values or not */
- int tails; /* Report significance with how many tails ? */
- bool descriptive_stats;
- bool xprod_stats;
-
- const struct variable *wv; /* The weight variable (if any) */
-};
-
-
-static void
-output_descriptives (const struct corr *corr, const struct corr_opts *opts,
- const gsl_matrix *means,
- const gsl_matrix *vars, const gsl_matrix *ns)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Descriptive Statistics"));
- pivot_table_set_weight_var (table, opts->wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("N"), PIVOT_RC_COUNT);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t r = 0; r < corr->n_vars_total; ++r)
- {
- const struct variable *v = corr->vars[r];
-
- int row = pivot_category_create_leaf (variables->root,
- pivot_value_new_variable (v));
-
- double mean = gsl_matrix_get (means, r, 0);
- /* Here we want to display the non-biased estimator */
- double n = gsl_matrix_get (ns, r, 0);
- double stddev = sqrt (gsl_matrix_get (vars, r, 0) * n / (n - 1));
- double entries[] = { mean, stddev, n };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-output_correlation (const struct corr *corr, const struct corr_opts *opts,
- const gsl_matrix *cm, const gsl_matrix *samples,
- const gsl_matrix *cv)
-{
- struct pivot_table *table = pivot_table_create (N_("Correlations"));
- pivot_table_set_weight_var (table, opts->wv);
-
- /* Column variable dimension. */
- struct pivot_dimension *columns = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Variables"));
-
- size_t matrix_cols = (corr->n_vars_total > corr->n_vars1
- ? corr->n_vars_total - corr->n_vars1
- : corr->n_vars1);
- for (size_t c = 0; c < matrix_cols; c++)
- {
- const struct variable *v = corr->n_vars_total > corr->n_vars1 ?
- corr->vars[corr->n_vars1 + c] : corr->vars[c];
- pivot_category_create_leaf (columns->root, pivot_value_new_variable (v));
- }
-
- /* Statistics dimension. */
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Pearson Correlation"), PIVOT_RC_CORRELATION,
- opts->tails == 2 ? N_("Sig. (2-tailed)") : N_("Sig. (1-tailed)"),
- PIVOT_RC_SIGNIFICANCE);
-
- if (opts->xprod_stats)
- pivot_category_create_leaves (statistics->root, N_("Cross-products"),
- N_("Covariance"));
-
- if (opts->missing_type != CORR_LISTWISE)
- pivot_category_create_leaves (statistics->root, N_("N"), PIVOT_RC_COUNT);
-
- /* Row variable dimension. */
- struct pivot_dimension *rows = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
- for (size_t r = 0; r < corr->n_vars1; r++)
- pivot_category_create_leaf (rows->root,
- pivot_value_new_variable (corr->vars[r]));
-
- struct pivot_footnote *sig_footnote = pivot_table_create_footnote (
- table, pivot_value_new_text (N_("Significant at .05 level")));
-
- for (size_t r = 0; r < corr->n_vars1; r++)
- for (size_t c = 0; c < matrix_cols; c++)
- {
- const int col_index = (corr->n_vars_total > corr->n_vars1
- ? corr->n_vars1 + c
- : c);
- double pearson = gsl_matrix_get (cm, r, col_index);
- double w = gsl_matrix_get (samples, r, col_index);
- double sig = opts->tails * significance_of_correlation (pearson, w);
-
- double entries[5];
- int n = 0;
- entries[n++] = pearson;
- entries[n++] = col_index != r ? sig : SYSMIS;
- if (opts->xprod_stats)
- {
- double cov = gsl_matrix_get (cv, r, col_index);
- const double xprod_dev = cov * w;
- cov *= w / (w - 1.0);
-
- entries[n++] = xprod_dev;
- entries[n++] = cov;
- }
- if (opts->missing_type != CORR_LISTWISE)
- entries[n++] = w;
-
- for (int i = 0; i < n; i++)
- if (entries[i] != SYSMIS)
- {
- struct pivot_value *v = pivot_value_new_number (entries[i]);
- if (!i && opts->sig && col_index != r && sig < 0.05)
- pivot_value_add_footnote (v, sig_footnote);
- pivot_table_put3 (table, c, i, r, v);
- }
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-run_corr (struct casereader *r, const struct corr_opts *opts, const struct corr *corr)
-{
- struct covariance *cov = covariance_2pass_create (
- corr->n_vars_total, corr->vars, NULL,opts->wv, opts->exclude, true);
-
- struct casereader *rc = casereader_clone (r);
- struct ccase *c;
- for (; (c = casereader_read (r)); case_unref (c))
- covariance_accumulate_pass1 (cov, c);
- for (; (c = casereader_read (rc)); case_unref (c))
- covariance_accumulate_pass2 (cov, c);
- casereader_destroy (rc);
-
- gsl_matrix *cov_matrix = covariance_calculate (cov);
- if (!cov_matrix)
- {
- msg (SE, _("The data for the chosen variables are all missing or empty."));
- covariance_destroy (cov);
- return;
- }
-
- const gsl_matrix *samples_matrix = covariance_moments (cov, MOMENT_NONE);
- const gsl_matrix *var_matrix = covariance_moments (cov, MOMENT_VARIANCE);
- const gsl_matrix *mean_matrix = covariance_moments (cov, MOMENT_MEAN);
-
- gsl_matrix *corr_matrix = correlation_from_covariance (cov_matrix, var_matrix);
-
- if (opts->descriptive_stats)
- output_descriptives (corr, opts, mean_matrix, var_matrix, samples_matrix);
-
- output_correlation (corr, opts, corr_matrix, samples_matrix, cov_matrix);
-
- covariance_destroy (cov);
- gsl_matrix_free (corr_matrix);
- gsl_matrix_free (cov_matrix);
-}
-
-int
-cmd_correlations (struct lexer *lexer, struct dataset *ds)
-{
- size_t n_all_vars = 0; /* Total number of variables involved in this command */
- const struct dictionary *dict = dataset_dict (ds);
-
- struct corr *corrs = NULL;
- size_t n_corrs = 0;
- size_t allocated_corrs = 0;
-
- struct corr_opts opts = {
- .missing_type = CORR_PAIRWISE,
- .wv = dict_get_weight (dict),
- .tails = 2,
- .exclude = MV_ANY,
- };
-
- /* Parse CORRELATIONS. */
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "PAIRWISE"))
- opts.missing_type = CORR_PAIRWISE;
- else if (lex_match_id (lexer, "LISTWISE"))
- opts.missing_type = CORR_LISTWISE;
- else if (lex_match_id (lexer, "INCLUDE"))
- opts.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- opts.exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "PAIRWISE", "LISTWISE",
- "INCLUDE", "EXCLUDE");
- goto error;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "PRINT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "TWOTAIL"))
- opts.tails = 2;
- else if (lex_match_id (lexer, "ONETAIL"))
- opts.tails = 1;
- else if (lex_match_id (lexer, "SIG"))
- opts.sig = false;
- else if (lex_match_id (lexer, "NOSIG"))
- opts.sig = true;
- else
- {
- lex_error_expecting (lexer, "TWOTAIL", "ONETAIL",
- "SIG", "NOSIG");
- goto error;
- }
-
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "DESCRIPTIVES"))
- opts.descriptive_stats = true;
- else if (lex_match_id (lexer, "XPROD"))
- opts.xprod_stats = true;
- else if (lex_token (lexer) == T_ALL)
- {
- opts.descriptive_stats = opts.xprod_stats = true;
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "DESCRIPTIVES", "XPROD", "ALL");
- goto error;
- }
-
- lex_match (lexer, T_COMMA);
- }
- }
- else
- {
- if (lex_match_id (lexer, "VARIABLES"))
- lex_match (lexer, T_EQUALS);
-
- const struct variable **vars;
- size_t n_vars1;
- if (!parse_variables_const (lexer, dict, &vars, &n_vars1, PV_NUMERIC))
- goto error;
-
- size_t n_vars_total = n_vars1;
- if (lex_match (lexer, T_WITH)
- && !parse_variables_const (lexer, dict, &vars, &n_vars_total,
- PV_NUMERIC | PV_APPEND))
- goto error;
-
- if (n_corrs >= allocated_corrs)
- corrs = x2nrealloc (corrs, &allocated_corrs, sizeof *corrs);
- corrs[n_corrs++] = (struct corr) {
- .n_vars1 = n_vars1,
- .n_vars_total = n_vars_total,
- .vars = vars,
- };
-
- n_all_vars += n_vars_total;
- }
- }
- if (n_corrs == 0)
- {
- lex_ofs_error (lexer, 0, lex_ofs (lexer) - 1,
- _("No variables specified."));
- goto error;
- }
-
- const struct variable **all_vars = xmalloc (n_all_vars * sizeof *all_vars);
- const struct variable **vv = all_vars;
- for (size_t i = 0; i < n_corrs; ++i)
- {
- const struct corr *c = &corrs[i];
- for (size_t v = 0; v < c->n_vars_total; ++v)
- *vv++ = c->vars[v];
- }
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- for (size_t i = 0; i < n_corrs; ++i)
- {
- /* FIXME: No need to iterate the data multiple times */
- struct casereader *r = casereader_clone (group);
-
- if (opts.missing_type == CORR_LISTWISE)
- r = casereader_create_filter_missing (r, all_vars, n_all_vars,
- opts.exclude, NULL, NULL);
-
-
- run_corr (r, &opts, &corrs[i]);
- casereader_destroy (r);
- }
- casereader_destroy (group);
- }
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- free (all_vars);
-
- /* Done. */
- for (size_t i = 0; i < n_corrs; i++)
- free (corrs[i].vars);
- free (corrs);
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-
-error:
- for (size_t i = 0; i < n_corrs; i++)
- free (corrs[i].vars);
- free (corrs);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012, 2013, 2014, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-/* FIXME:
-
- - How to calculate significance of some symmetric and directional measures?
- - How to calculate ASE for symmetric Somers ' d?
- - How to calculate ASE for Goodman and Kruskal's tau?
- - How to calculate approx. T of symmetric uncertainty coefficient?
-
-*/
-
-#include <config.h>
-
-#include <ctype.h>
-#include <float.h>
-#include <gsl/gsl_cdf.h>
-#include <stdlib.h>
-#include <stdio.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/stats/freq.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmap.h"
-#include "libpspp/hmapx.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-#include "output/pivot-table.h"
-#include "output/charts/barchart.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc-oversized.h"
-#include "gl/xalloc.h"
-#include "gl/xsize.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-/* Kinds of cells in the crosstabulation. */
-#define CRS_CELLS \
- C(COUNT, N_("Count"), PIVOT_RC_COUNT) \
- C(EXPECTED, N_("Expected"), PIVOT_RC_OTHER) \
- C(ROW, N_("Row %"), PIVOT_RC_PERCENT) \
- C(COLUMN, N_("Column %"), PIVOT_RC_PERCENT) \
- C(TOTAL, N_("Total %"), PIVOT_RC_PERCENT) \
- C(RESIDUAL, N_("Residual"), PIVOT_RC_RESIDUAL) \
- C(SRESIDUAL, N_("Std. Residual"), PIVOT_RC_RESIDUAL) \
- C(ASRESIDUAL, N_("Adjusted Residual"), PIVOT_RC_RESIDUAL)
-enum crs_cell
- {
-#define C(KEYWORD, STRING, RC) CRS_CL_##KEYWORD,
- CRS_CELLS
-#undef C
- };
-enum {
-#define C(KEYWORD, STRING, RC) + 1
- CRS_N_CELLS = CRS_CELLS
-#undef C
-};
-#define CRS_ALL_CELLS ((1u << CRS_N_CELLS) - 1)
-
-/* Kinds of statistics. */
-#define CRS_STATISTICS \
- S(CHISQ) \
- S(PHI) \
- S(CC) \
- S(LAMBDA) \
- S(UC) \
- S(BTAU) \
- S(CTAU) \
- S(RISK) \
- S(GAMMA) \
- S(D) \
- S(KAPPA) \
- S(ETA) \
- S(CORR)
-enum crs_statistic_index {
-#define S(KEYWORD) CRS_ST_##KEYWORD##_INDEX,
- CRS_STATISTICS
-#undef S
-};
-enum crs_statistic_bit {
-#define S(KEYWORD) CRS_ST_##KEYWORD = 1u << CRS_ST_##KEYWORD##_INDEX,
- CRS_STATISTICS
-#undef S
-};
-enum {
-#define S(KEYWORD) + 1
- CRS_N_STATISTICS = CRS_STATISTICS
-#undef S
-};
-#define CRS_ALL_STATISTICS ((1u << CRS_N_STATISTICS) - 1)
-
-/* Number of chi-square statistics. */
-#define N_CHISQ 5
-
-/* Number of symmetric statistics. */
-#define N_SYMMETRIC 9
-
-/* Number of directional statistics. */
-#define N_DIRECTIONAL 13
-
-/* Indexes into the 'vars' member of struct crosstabulation and
- struct crosstab member. */
-enum
- {
- ROW_VAR = 0, /* Row variable. */
- COL_VAR = 1 /* Column variable. */
- /* Higher indexes cause multiple tables to be output. */
- };
-
-struct xtab_var
- {
- const struct variable *var;
- union value *values;
- size_t n_values;
- };
-
-/* A crosstabulation of 2 or more variables. */
-struct crosstabulation
- {
- struct crosstabs_proc *proc;
- struct fmt_spec weight_format; /* Format for weight variable. */
- double missing; /* Weight of missing cases. */
-
- /* Variables (2 or more). */
- size_t n_vars;
- struct xtab_var *vars;
-
- /* Constants (0 or more). */
- size_t n_consts;
- struct xtab_var *const_vars;
- size_t *const_indexes;
-
- /* Data. */
- struct hmap data;
- struct freq **entries;
- size_t n_entries;
-
- /* Number of statistically interesting columns/rows
- (columns/rows with data in them). */
- size_t ns_cols, ns_rows;
-
- /* Matrix contents. */
- double *mat; /* Matrix proper. */
- double *row_tot; /* Row totals. */
- double *col_tot; /* Column totals. */
- double total; /* Grand total. */
-
- /* Syntax. */
- int start_ofs;
- int end_ofs;
- };
-
-/* Integer mode variable info. */
-struct var_range
- {
- struct hmap_node hmap_node; /* In struct crosstabs_proc var_ranges map. */
- const struct variable *var; /* The variable. */
- int min; /* Minimum value. */
- int max; /* Maximum value + 1. */
- int count; /* max - min. */
- };
-
-struct crosstabs_proc
- {
- const struct dictionary *dict;
- enum { INTEGER, GENERAL } mode;
- enum mv_class exclude;
- bool barchart;
- bool bad_warn;
- struct fmt_spec weight_format;
-
- /* Variables specifies on VARIABLES. */
- const struct variable **variables;
- size_t n_variables;
- struct hmap var_ranges;
-
- /* TABLES. */
- struct crosstabulation *pivots;
- size_t n_pivots;
-
- /* CELLS. */
- size_t n_cells; /* Number of cells requested. */
- unsigned int cells; /* Bit k is 1 if cell k is requested. */
- int a_cells[CRS_N_CELLS]; /* 0...n_cells-1 are the requested cells. */
-
- /* Rounding of cells. */
- bool round_case_weights; /* Round case weights? */
- bool round_cells; /* If !round_case_weights, round cells? */
- bool round_down; /* Round down? (otherwise to nearest) */
-
- /* STATISTICS. */
- unsigned int statistics; /* Bit k is 1 if statistic k is requested. */
-
- bool descending; /* True if descending sort order is requested. */
- };
-
-static bool parse_crosstabs_tables (struct lexer *, struct dataset *,
- struct crosstabs_proc *);
-static bool parse_crosstabs_variables (struct lexer *, struct dataset *,
- struct crosstabs_proc *);
-
-static const struct var_range *get_var_range (const struct crosstabs_proc *,
- const struct variable *);
-
-static bool should_tabulate_case (const struct crosstabulation *,
- const struct ccase *, enum mv_class exclude);
-static void tabulate_general_case (struct crosstabulation *, const struct ccase *,
- double weight);
-static void tabulate_integer_case (struct crosstabulation *, const struct ccase *,
- double weight);
-static void postcalc (struct crosstabs_proc *, struct lexer *);
-
-static double
-round_weight (const struct crosstabs_proc *proc, double weight)
-{
- return proc->round_down ? floor (weight) : floor (weight + 0.5);
-}
-
-#define FOR_EACH_POPULATED_COLUMN(C, XT) \
- for (size_t C = next_populated_column (0, XT); \
- C < (XT)->vars[COL_VAR].n_values; \
- C = next_populated_column (C + 1, XT))
-static size_t
-next_populated_column (size_t c, const struct crosstabulation *xt)
-{
- size_t n_columns = xt->vars[COL_VAR].n_values;
- for (; c < n_columns; c++)
- if (xt->col_tot[c])
- break;
- return c;
-}
-
-#define FOR_EACH_POPULATED_ROW(R, XT) \
- for (size_t R = next_populated_row (0, XT); R < (XT)->vars[ROW_VAR].n_values; \
- R = next_populated_row (R + 1, XT))
-static size_t
-next_populated_row (size_t r, const struct crosstabulation *xt)
-{
- size_t n_rows = xt->vars[ROW_VAR].n_values;
- for (; r < n_rows; r++)
- if (xt->row_tot[r])
- break;
- return r;
-}
-
-/* Parses and executes the CROSSTABS procedure. */
-int
-cmd_crosstabs (struct lexer *lexer, struct dataset *ds)
-{
- int result = CMD_FAILURE;
-
- struct crosstabs_proc proc = {
- .dict = dataset_dict (ds),
- .mode = GENERAL,
- .exclude = MV_ANY,
- .barchart = false,
- .bad_warn = true,
- .weight_format = *dict_get_weight_format (dataset_dict (ds)),
-
- .variables = NULL,
- .n_variables = 0,
- .var_ranges = HMAP_INITIALIZER (proc.var_ranges),
-
- .pivots = NULL,
- .n_pivots = 0,
-
- .cells = 1u << CRS_CL_COUNT,
- /* n_cells and a_cells will be filled in later. */
-
- .round_case_weights = false,
- .round_cells = false,
- .round_down = false,
-
- .statistics = 0,
-
- .descending = false,
- };
- bool show_tables = true;
- int exclude_ofs = 0;
- lex_match (lexer, T_SLASH);
- for (;;)
- {
- if (lex_match_id (lexer, "VARIABLES"))
- {
- if (!parse_crosstabs_variables (lexer, ds, &proc))
- goto exit;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- exclude_ofs = lex_ofs (lexer);
- if (lex_match_id (lexer, "TABLE"))
- proc.exclude = MV_ANY;
- else if (lex_match_id (lexer, "INCLUDE"))
- proc.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "REPORT"))
- proc.exclude = 0;
- else
- {
- lex_error_expecting (lexer, "TABLE", "INCLUDE", "REPORT");
- goto exit;
- }
- }
- else if (lex_match_id (lexer, "COUNT"))
- {
- lex_match (lexer, T_EQUALS);
-
- /* Default is CELL. */
- proc.round_case_weights = false;
- proc.round_cells = true;
-
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "ASIS"))
- {
- proc.round_case_weights = false;
- proc.round_cells = false;
- }
- else if (lex_match_id (lexer, "CASE"))
- {
- proc.round_case_weights = true;
- proc.round_cells = false;
- }
- else if (lex_match_id (lexer, "CELL"))
- {
- proc.round_case_weights = false;
- proc.round_cells = true;
- }
- else if (lex_match_id (lexer, "ROUND"))
- proc.round_down = false;
- else if (lex_match_id (lexer, "TRUNCATE"))
- proc.round_down = true;
- else
- {
- lex_error_expecting (lexer, "ASIS", "CASE", "CELL",
- "ROUND", "TRUNCATE");
- goto exit;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "AVALUE"))
- proc.descending = false;
- else if (lex_match_id (lexer, "DVALUE"))
- proc.descending = true;
- else if (lex_match_id (lexer, "TABLES"))
- show_tables = true;
- else if (lex_match_id (lexer, "NOTABLES"))
- show_tables = false;
- else
- {
- lex_error_expecting (lexer, "AVALUE", "DVALUE",
- "TABLES", "NOTABLES");
- goto exit;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "BARCHART"))
- proc.barchart = true;
- else if (lex_match_id (lexer, "CELLS"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "NONE"))
- proc.cells = 0;
- else if (lex_match (lexer, T_ALL))
- proc.cells = CRS_ALL_CELLS;
- else
- {
- proc.cells = 0;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
-#define C(KEYWORD, STRING, RC) \
- if (lex_match_id (lexer, #KEYWORD)) \
- { \
- proc.cells |= 1u << CRS_CL_##KEYWORD; \
- continue; \
- }
- CRS_CELLS
-#undef C
-
- static const char *cells[] =
- {
-#define C(KEYWORD, STRING, RC) #KEYWORD,
- CRS_CELLS
-#undef C
- };
- lex_error_expecting_array (lexer, cells,
- sizeof cells / sizeof *cells);
- goto exit;
- }
- if (!proc.cells)
- proc.cells = ((1u << CRS_CL_COUNT) | (1u << CRS_CL_ROW)
- | (1u << CRS_CL_COLUMN) | (1u << CRS_CL_TOTAL));
- }
- }
- else if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "NONE"))
- proc.statistics = 0;
- else if (lex_match (lexer, T_ALL))
- proc.statistics = CRS_ALL_STATISTICS;
- else
- {
- proc.statistics = 0;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
-#define S(KEYWORD) \
- if (lex_match_id (lexer, #KEYWORD)) \
- { \
- proc.statistics |= CRS_ST_##KEYWORD; \
- continue; \
- }
- CRS_STATISTICS
-#undef S
- static const char *stats[] =
- {
-#define S(KEYWORD) #KEYWORD,
- CRS_STATISTICS
-#undef S
- };
- lex_error_expecting_array (lexer, stats,
- sizeof stats / sizeof *stats);
- goto exit;
- }
- if (!proc.statistics)
- proc.statistics = CRS_ST_CHISQ;
- }
- }
- else if (!parse_crosstabs_tables (lexer, ds, &proc))
- goto exit;
-
- if (!lex_match (lexer, T_SLASH))
- break;
- }
- if (!lex_end_of_command (lexer))
- goto exit;
-
- if (!proc.n_pivots)
- {
- msg (SE, _("At least one crosstabulation must be requested (using "
- "the TABLES subcommand)."));
- goto exit;
- }
-
- /* Cells. */
- if (!show_tables)
- proc.cells = 0;
- for (size_t i = 0; i < CRS_N_CELLS; i++)
- if (proc.cells & (1u << i))
- proc.a_cells[proc.n_cells++] = i;
- assert (proc.n_cells < CRS_N_CELLS);
-
- /* Missing values. */
- if (proc.mode == GENERAL && !proc.exclude)
- {
- lex_ofs_msg (lexer, SW, exclude_ofs, exclude_ofs,
- _("Missing mode %s not allowed in general mode. "
- "Assuming %s."), "REPORT", "MISSING=TABLE");
- proc.exclude = MV_ANY;
- }
-
- struct casereader *input = casereader_create_filter_weight (proc_open (ds),
- dataset_dict (ds),
- NULL, NULL);
- struct casegrouper *grouper = casegrouper_create_splits (input, dataset_dict (ds));
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- output_split_file_values_peek (ds, group);
-
- /* Initialize hash tables. */
- for (struct crosstabulation *xt = &proc.pivots[0];
- xt < &proc.pivots[proc.n_pivots]; xt++)
- hmap_init (&xt->data);
-
- /* Tabulate. */
- struct ccase *c;
- for (; (c = casereader_read (group)) != NULL; case_unref (c))
- for (struct crosstabulation *xt = &proc.pivots[0];
- xt < &proc.pivots[proc.n_pivots]; xt++)
- {
- double weight = dict_get_case_weight (dataset_dict (ds), c,
- &proc.bad_warn);
- if (proc.round_case_weights)
- {
- weight = round_weight (&proc, weight);
- if (weight == 0.)
- continue;
- }
- if (should_tabulate_case (xt, c, proc.exclude))
- {
- if (proc.mode == GENERAL)
- tabulate_general_case (xt, c, weight);
- else
- tabulate_integer_case (xt, c, weight);
- }
- else
- xt->missing += weight;
- }
- casereader_destroy (group);
-
- /* Output. */
- postcalc (&proc, lexer);
- }
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- result = ok ? CMD_SUCCESS : CMD_FAILURE;
-
-exit:
- free (proc.variables);
-
- struct var_range *range, *next_range;
- HMAP_FOR_EACH_SAFE (range, next_range, struct var_range, hmap_node,
- &proc.var_ranges)
- {
- hmap_delete (&proc.var_ranges, &range->hmap_node);
- free (range);
- }
- for (struct crosstabulation *xt = &proc.pivots[0];
- xt < &proc.pivots[proc.n_pivots]; xt++)
- {
- free (xt->vars);
- free (xt->const_vars);
- free (xt->const_indexes);
- }
- free (proc.pivots);
-
- return result;
-}
-
-/* Parses the TABLES subcommand. */
-static bool
-parse_crosstabs_tables (struct lexer *lexer, struct dataset *ds,
- struct crosstabs_proc *proc)
-{
- const struct variable ***by = NULL;
- size_t *by_nvar = NULL;
- bool ok = false;
-
- /* Ensure that this is a TABLES subcommand. */
- if (!lex_match_id (lexer, "TABLES")
- && (lex_token (lexer) != T_ID ||
- dict_lookup_var (dataset_dict (ds), lex_tokcstr (lexer)) == NULL)
- && lex_token (lexer) != T_ALL)
- {
- lex_error (lexer, _("Syntax error expecting subcommand name or "
- "variable name."));
- return false;
- }
- lex_match (lexer, T_EQUALS);
-
- struct const_var_set *var_set
- = (proc->variables
- ? const_var_set_create_from_array (proc->variables,
- proc->n_variables)
- : const_var_set_create_from_dict (dataset_dict (ds)));
-
- size_t nx = 1;
- size_t n_by = 0;
- int vars_start = lex_ofs (lexer);
- do
- {
- by = xnrealloc (by, n_by + 1, sizeof *by);
- by_nvar = xnrealloc (by_nvar, n_by + 1, sizeof *by_nvar);
- if (!parse_const_var_set_vars (lexer, var_set, &by[n_by], &by_nvar[n_by],
- PV_NO_DUPLICATE | PV_NO_SCRATCH))
- goto done;
- size_t n = by_nvar[n_by++];
- if (xalloc_oversized (nx, n))
- {
- lex_ofs_error (
- lexer, vars_start, lex_ofs (lexer) - 1,
- _("Too many cross-tabulation variables or dimensions."));
- goto done;
- }
- nx *= n;
- }
- while (lex_match (lexer, T_BY));
- if (n_by < 2)
- {
- bool unused UNUSED = lex_force_match (lexer, T_BY);
- goto done;
- }
- int vars_end = lex_ofs (lexer) - 1;
-
- size_t *by_iter = XCALLOC (n_by, size_t);
- proc->pivots = xnrealloc (proc->pivots,
- proc->n_pivots + nx, sizeof *proc->pivots);
- for (size_t i = 0; i < nx; i++)
- {
- struct crosstabulation *xt = &proc->pivots[proc->n_pivots++];
-
- *xt = (struct crosstabulation) {
- .proc = proc,
- .weight_format = proc->weight_format,
- .missing = 0.,
- .n_vars = n_by,
- .vars = xcalloc (n_by, sizeof *xt->vars),
- .n_consts = 0,
- .const_vars = NULL,
- .const_indexes = NULL,
- .start_ofs = vars_start,
- .end_ofs = vars_end,
- };
-
- for (size_t j = 0; j < n_by; j++)
- xt->vars[j].var = by[j][by_iter[j]];
-
- for (int j = n_by - 1; j >= 0; j--)
- {
- if (++by_iter[j] < by_nvar[j])
- break;
- by_iter[j] = 0;
- }
- }
- free (by_iter);
- ok = true;
-
-done:
- /* All return paths lead here. */
- for (size_t i = 0; i < n_by; i++)
- free (by[i]);
- free (by);
- free (by_nvar);
-
- const_var_set_destroy (var_set);
-
- return ok;
-}
-
-/* Parses the VARIABLES subcommand. */
-static bool
-parse_crosstabs_variables (struct lexer *lexer, struct dataset *ds,
- struct crosstabs_proc *proc)
-{
- if (proc->n_pivots)
- {
- lex_next_error (lexer, -1, -1, _("%s must be specified before %s."),
- "VARIABLES", "TABLES");
- return false;
- }
-
- lex_match (lexer, T_EQUALS);
-
- for (;;)
- {
- size_t orig_nv = proc->n_variables;
-
- if (!parse_variables_const (lexer, dataset_dict (ds),
- &proc->variables, &proc->n_variables,
- (PV_APPEND | PV_NUMERIC
- | PV_NO_DUPLICATE | PV_NO_SCRATCH)))
- return false;
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- if (!lex_force_int (lexer))
- goto error;
- long min = lex_integer (lexer);
- lex_get (lexer);
-
- lex_match (lexer, T_COMMA);
-
- if (!lex_force_int_range (lexer, NULL, min, LONG_MAX))
- goto error;
- long max = lex_integer (lexer);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
-
- for (size_t i = orig_nv; i < proc->n_variables; i++)
- {
- const struct variable *var = proc->variables[i];
- struct var_range *vr = xmalloc (sizeof *vr);
- *vr = (struct var_range) {
- .var = var,
- .min = min,
- .max = max,
- .count = max - min + 1,
- };
- hmap_insert (&proc->var_ranges, &vr->hmap_node,
- hash_pointer (var, 0));
- }
-
- if (lex_token (lexer) == T_SLASH)
- break;
- }
-
- proc->mode = INTEGER;
- return true;
-
- error:
- free (proc->variables);
- proc->variables = NULL;
- proc->n_variables = 0;
- return false;
-}
-\f
-/* Data file processing. */
-
-static const struct var_range *
-get_var_range (const struct crosstabs_proc *proc, const struct variable *var)
-{
- if (!hmap_is_empty (&proc->var_ranges))
- {
- const struct var_range *range;
-
- HMAP_FOR_EACH_IN_BUCKET (range, struct var_range, hmap_node,
- hash_pointer (var, 0), &proc->var_ranges)
- if (range->var == var)
- return range;
- }
-
- return NULL;
-}
-
-static bool
-should_tabulate_case (const struct crosstabulation *xt, const struct ccase *c,
- enum mv_class exclude)
-{
- for (size_t j = 0; j < xt->n_vars; j++)
- {
- const struct variable *var = xt->vars[j].var;
- const struct var_range *range = get_var_range (xt->proc, var);
-
- if (var_is_value_missing (var, case_data (c, var)) & exclude)
- return false;
-
- if (range != NULL)
- {
- double num = case_num (c, var);
- if (num < range->min || num >= range->max + 1.)
- return false;
- }
- }
- return true;
-}
-
-static void
-tabulate_integer_case (struct crosstabulation *xt, const struct ccase *c,
- double weight)
-{
- size_t hash = 0;
- for (size_t j = 0; j < xt->n_vars; j++)
- {
- /* Throw away fractional parts of values. */
- hash = hash_int (case_num (c, xt->vars[j].var), hash);
- }
-
- struct freq *te;
- HMAP_FOR_EACH_WITH_HASH (te, struct freq, node, hash, &xt->data)
- {
- for (size_t j = 0; j < xt->n_vars; j++)
- if ((int) case_num (c, xt->vars[j].var) != (int) te->values[j].f)
- goto no_match;
-
- /* Found an existing entry. */
- te->count += weight;
- return;
-
- no_match: ;
- }
-
- /* No existing entry. Create a new one. */
- te = xmalloc (table_entry_size (xt->n_vars));
- te->count = weight;
- for (size_t j = 0; j < xt->n_vars; j++)
- te->values[j].f = (int) case_num (c, xt->vars[j].var);
- hmap_insert (&xt->data, &te->node, hash);
-}
-
-static void
-tabulate_general_case (struct crosstabulation *xt, const struct ccase *c,
- double weight)
-{
- size_t hash = 0;
- for (size_t j = 0; j < xt->n_vars; j++)
- {
- const struct variable *var = xt->vars[j].var;
- hash = value_hash (case_data (c, var), var_get_width (var), hash);
- }
-
- struct freq *te;
- HMAP_FOR_EACH_WITH_HASH (te, struct freq, node, hash, &xt->data)
- {
- for (size_t j = 0; j < xt->n_vars; j++)
- {
- const struct variable *var = xt->vars[j].var;
- if (!value_equal (case_data (c, var), &te->values[j],
- var_get_width (var)))
- goto no_match;
- }
-
- /* Found an existing entry. */
- te->count += weight;
- return;
-
- no_match: ;
- }
-
- /* No existing entry. Create a new one. */
- te = xmalloc (table_entry_size (xt->n_vars));
- te->count = weight;
- for (size_t j = 0; j < xt->n_vars; j++)
- {
- const struct variable *var = xt->vars[j].var;
- value_clone (&te->values[j], case_data (c, var), var_get_width (var));
- }
- hmap_insert (&xt->data, &te->node, hash);
-}
-\f
-/* Post-data reading calculations. */
-
-static int compare_table_entry_vars_3way (const struct freq *a,
- const struct freq *b,
- const struct crosstabulation *xt,
- int idx0, int idx1);
-static int compare_table_entry_3way (const void *ap_, const void *bp_,
- const void *xt_);
-static int compare_table_entry_3way_inv (const void *ap_, const void *bp_,
- const void *xt_);
-
-static void enum_var_values (const struct crosstabulation *, int var_idx,
- bool descending);
-static void free_var_values (const struct crosstabulation *, int var_idx);
-static void output_crosstabulation (struct crosstabs_proc *,
- struct crosstabulation *,
- struct lexer *);
-static void make_crosstabulation_subset (struct crosstabulation *xt,
- size_t row0, size_t row1,
- struct crosstabulation *subset);
-static void make_summary_table (struct crosstabs_proc *);
-static bool find_crosstab (struct crosstabulation *, size_t *row0p,
- size_t *row1p);
-
-static void
-postcalc (struct crosstabs_proc *proc, struct lexer *lexer)
-{
- /* Round hash table entries, if requested
-
- If this causes any of the cell counts to fall to zero, delete those
- cells. */
- if (proc->round_cells)
- for (struct crosstabulation *xt = proc->pivots;
- xt < &proc->pivots[proc->n_pivots]; xt++)
- {
- struct freq *e, *next;
- HMAP_FOR_EACH_SAFE (e, next, struct freq, node, &xt->data)
- {
- e->count = round_weight (proc, e->count);
- if (e->count == 0.0)
- {
- hmap_delete (&xt->data, &e->node);
- free (e);
- }
- }
- }
-
- /* Convert hash tables into sorted arrays of entries. */
- for (struct crosstabulation *xt = proc->pivots;
- xt < &proc->pivots[proc->n_pivots]; xt++)
- {
- xt->n_entries = hmap_count (&xt->data);
- xt->entries = xnmalloc (xt->n_entries, sizeof *xt->entries);
-
- size_t i = 0;
- struct freq *e;
- HMAP_FOR_EACH (e, struct freq, node, &xt->data)
- xt->entries[i++] = e;
-
- hmap_destroy (&xt->data);
-
- sort (xt->entries, xt->n_entries, sizeof *xt->entries,
- proc->descending ? compare_table_entry_3way_inv : compare_table_entry_3way,
- xt);
- }
-
- make_summary_table (proc);
-
- /* Output each pivot table. */
- for (struct crosstabulation *xt = proc->pivots;
- xt < &proc->pivots[proc->n_pivots]; xt++)
- {
- output_crosstabulation (proc, xt, lexer);
- if (proc->barchart)
- {
- int n_vars = (xt->n_vars > 2 ? 2 : xt->n_vars);
- const struct variable **vars = XCALLOC (n_vars, const struct variable*);
- for (size_t i = 0; i < n_vars; i++)
- vars[i] = xt->vars[i].var;
- chart_submit (barchart_create (vars, n_vars, _("Count"),
- false,
- xt->entries, xt->n_entries));
- free (vars);
- }
- }
-
- /* Free output and prepare for next split file. */
- for (struct crosstabulation *xt = proc->pivots;
- xt < &proc->pivots[proc->n_pivots]; xt++)
- {
- xt->missing = 0.0;
-
- /* Free the members that were allocated in this function(and the values
- owned by the entries.
-
- The other pointer members are either both allocated and destroyed at a
- lower level (in output_crosstabulation), or both allocated and
- destroyed at a higher level (in crs_custom_tables and free_proc,
- respectively). */
- for (size_t i = 0; i < xt->n_vars; i++)
- {
- int width = var_get_width (xt->vars[i].var);
- if (value_needs_init (width))
- for (size_t j = 0; j < xt->n_entries; j++)
- value_destroy (&xt->entries[j]->values[i], width);
- }
-
- for (size_t i = 0; i < xt->n_entries; i++)
- free (xt->entries[i]);
- free (xt->entries);
- }
-}
-
-static void
-make_crosstabulation_subset (struct crosstabulation *xt, size_t row0,
- size_t row1, struct crosstabulation *subset)
-{
- *subset = *xt;
- if (xt->n_vars > 2)
- {
- assert (xt->n_consts == 0);
- subset->n_vars = 2;
- subset->vars = xt->vars;
-
- subset->n_consts = xt->n_vars - 2;
- subset->const_vars = xt->vars + 2;
- subset->const_indexes = xcalloc (subset->n_consts,
- sizeof *subset->const_indexes);
- for (size_t i = 0; i < subset->n_consts; i++)
- {
- const union value *value = &xt->entries[row0]->values[2 + i];
-
- for (size_t j = 0; j < xt->vars[2 + i].n_values; j++)
- if (value_equal (&xt->vars[2 + i].values[j], value,
- var_get_width (xt->vars[2 + i].var)))
- {
- subset->const_indexes[i] = j;
- goto found;
- }
- NOT_REACHED ();
- found: ;
- }
- }
- subset->entries = &xt->entries[row0];
- subset->n_entries = row1 - row0;
-}
-
-static int
-compare_table_entry_var_3way (const struct freq *a,
- const struct freq *b,
- const struct crosstabulation *xt,
- int idx)
-{
- return value_compare_3way (&a->values[idx], &b->values[idx],
- var_get_width (xt->vars[idx].var));
-}
-
-static int
-compare_table_entry_vars_3way (const struct freq *a,
- const struct freq *b,
- const struct crosstabulation *xt,
- int idx0, int idx1)
-{
- for (int i = idx1 - 1; i >= idx0; i--)
- {
- int cmp = compare_table_entry_var_3way (a, b, xt, i);
- if (cmp != 0)
- return cmp;
- }
- return 0;
-}
-
-/* Compare the struct freq at *AP to the one at *BP and
- return a strcmp()-type result. */
-static int
-compare_table_entry_3way (const void *ap_, const void *bp_, const void *xt_)
-{
- const struct freq *const *ap = ap_;
- const struct freq *const *bp = bp_;
- const struct freq *a = *ap;
- const struct freq *b = *bp;
- const struct crosstabulation *xt = xt_;
-
- int cmp = compare_table_entry_vars_3way (a, b, xt, 2, xt->n_vars);
- if (cmp != 0)
- return cmp;
-
- cmp = compare_table_entry_var_3way (a, b, xt, ROW_VAR);
- if (cmp != 0)
- return cmp;
-
- return compare_table_entry_var_3way (a, b, xt, COL_VAR);
-}
-
-/* Inverted version of compare_table_entry_3way */
-static int
-compare_table_entry_3way_inv (const void *ap_, const void *bp_, const void *xt_)
-{
- return -compare_table_entry_3way (ap_, bp_, xt_);
-}
-
-/* Output a table summarizing the cases processed. */
-static void
-make_summary_table (struct crosstabs_proc *proc)
-{
- struct pivot_table *table = pivot_table_create (N_("Summary"));
- pivot_table_set_weight_var (table, dict_get_weight (proc->dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Percent"), PIVOT_RC_PERCENT);
-
- struct pivot_dimension *cases = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Cases"),
- N_("Valid"), N_("Missing"), N_("Total"));
- cases->root->show_label = true;
-
- struct pivot_dimension *tables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Crosstabulation"));
- for (struct crosstabulation *xt = &proc->pivots[0];
- xt < &proc->pivots[proc->n_pivots]; xt++)
- {
- struct string name = DS_EMPTY_INITIALIZER;
- for (size_t i = 0; i < xt->n_vars; i++)
- {
- if (i > 0)
- ds_put_cstr (&name, " × ");
- ds_put_cstr (&name, var_to_string (xt->vars[i].var));
- }
-
- int row = pivot_category_create_leaf (
- tables->root,
- pivot_value_new_user_text_nocopy (ds_steal_cstr (&name)));
-
- double valid = 0.;
- for (size_t i = 0; i < xt->n_entries; i++)
- valid += xt->entries[i]->count;
-
- double n[3];
- n[0] = valid;
- n[1] = xt->missing;
- n[2] = n[0] + n[1];
- for (int i = 0; i < 3; i++)
- {
- pivot_table_put3 (table, 0, i, row, pivot_value_new_number (n[i]));
- pivot_table_put3 (table, 1, i, row,
- pivot_value_new_number (n[i] / n[2] * 100.0));
- }
- }
-
- pivot_table_submit (table);
-}
-\f
-/* Output. */
-
-static struct pivot_table *create_crosstab_table (
- struct crosstabs_proc *, struct crosstabulation *,
- size_t crs_leaves[CRS_N_CELLS]);
-static struct pivot_table *create_chisq_table (struct crosstabulation *);
-static struct pivot_table *create_sym_table (struct crosstabulation *);
-static struct pivot_table *create_risk_table (
- struct crosstabulation *, struct pivot_dimension **risk_statistics);
-static struct pivot_table *create_direct_table (struct crosstabulation *);
-static void display_crosstabulation (struct crosstabs_proc *,
- struct crosstabulation *,
- struct pivot_table *,
- size_t crs_leaves[CRS_N_CELLS]);
-static void display_chisq (struct crosstabulation *, struct pivot_table *);
-static void display_symmetric (struct crosstabs_proc *,
- struct crosstabulation *, struct pivot_table *);
-static void display_risk (struct crosstabulation *, struct pivot_table *,
- struct pivot_dimension *risk_statistics);
-static void display_directional (struct crosstabs_proc *,
- struct crosstabulation *,
- struct pivot_table *);
-static void delete_missing (struct crosstabulation *);
-static void build_matrix (struct crosstabulation *);
-
-/* Output pivot table XT in the context of PROC. */
-static void
-output_crosstabulation (struct crosstabs_proc *proc, struct crosstabulation *xt,
- struct lexer *lexer)
-{
- for (size_t i = 0; i < xt->n_vars; i++)
- enum_var_values (xt, i, proc->descending);
-
- if (xt->vars[COL_VAR].n_values == 0)
- {
- struct string vars;
-
- ds_init_cstr (&vars, var_to_string (xt->vars[0].var));
- for (size_t i = 1; i < xt->n_vars; i++)
- ds_put_format (&vars, " × %s", var_to_string (xt->vars[i].var));
-
- /* TRANSLATORS: The %s here describes a crosstabulation. It takes the
- form "var1 * var2 * var3 * ...". */
- lex_ofs_msg (lexer, SW, xt->start_ofs, xt->end_ofs,
- _("Crosstabulation %s contained no non-missing cases."),
- ds_cstr (&vars));
-
- ds_destroy (&vars);
- for (size_t i = 0; i < xt->n_vars; i++)
- free_var_values (xt, i);
- return;
- }
-
- size_t crs_leaves[CRS_N_CELLS];
- struct pivot_table *table = (proc->cells
- ? create_crosstab_table (proc, xt, crs_leaves)
- : NULL);
- struct pivot_table *chisq = (proc->statistics & CRS_ST_CHISQ
- ? create_chisq_table (xt)
- : NULL);
- struct pivot_table *sym
- = (proc->statistics & (CRS_ST_PHI | CRS_ST_CC | CRS_ST_BTAU | CRS_ST_CTAU
- | CRS_ST_GAMMA | CRS_ST_CORR | CRS_ST_KAPPA)
- ? create_sym_table (xt)
- : NULL);
- struct pivot_dimension *risk_statistics = NULL;
- struct pivot_table *risk = (proc->statistics & CRS_ST_RISK
- ? create_risk_table (xt, &risk_statistics)
- : NULL);
- struct pivot_table *direct
- = (proc->statistics & (CRS_ST_LAMBDA | CRS_ST_UC | CRS_ST_D | CRS_ST_ETA)
- ? create_direct_table (xt)
- : NULL);
-
- size_t row0 = 0;
- size_t row1 = 0;
- while (find_crosstab (xt, &row0, &row1))
- {
- struct crosstabulation x;
-
- make_crosstabulation_subset (xt, row0, row1, &x);
-
- size_t n_rows = x.vars[ROW_VAR].n_values;
- size_t n_cols = x.vars[COL_VAR].n_values;
- if (size_overflow_p (xtimes (xtimes (n_rows, n_cols), sizeof (double))))
- xalloc_die ();
- x.row_tot = xmalloc (n_rows * sizeof *x.row_tot);
- x.col_tot = xmalloc (n_cols * sizeof *x.col_tot);
- x.mat = xmalloc (n_rows * n_cols * sizeof *x.mat);
-
- build_matrix (&x);
-
- /* Find the first variable that differs from the last subtable. */
- if (table)
- display_crosstabulation (proc, &x, table, crs_leaves);
-
- if (proc->exclude == 0)
- delete_missing (&x);
-
- if (chisq)
- display_chisq (&x, chisq);
-
- if (sym)
- display_symmetric (proc, &x, sym);
- if (risk)
- display_risk (&x, risk, risk_statistics);
- if (direct)
- display_directional (proc, &x, direct);
-
- free (x.mat);
- free (x.row_tot);
- free (x.col_tot);
- free (x.const_indexes);
- }
-
- if (table)
- pivot_table_submit (table);
-
- if (chisq)
- pivot_table_submit (chisq);
-
- if (sym)
- pivot_table_submit (sym);
-
- if (risk)
- {
- if (!pivot_table_is_empty (risk))
- pivot_table_submit (risk);
- else
- pivot_table_unref (risk);
- }
-
- if (direct)
- pivot_table_submit (direct);
-
- for (size_t i = 0; i < xt->n_vars; i++)
- free_var_values (xt, i);
-}
-
-static void
-build_matrix (struct crosstabulation *x)
-{
- const int col_var_width = var_get_width (x->vars[COL_VAR].var);
- const int row_var_width = var_get_width (x->vars[ROW_VAR].var);
- size_t n_rows = x->vars[ROW_VAR].n_values;
- size_t n_cols = x->vars[COL_VAR].n_values;
-
- double *mp = x->mat;
- size_t col = 0;
- size_t row = 0;
- for (struct freq **p = x->entries; p < &x->entries[x->n_entries]; p++)
- {
- const struct freq *te = *p;
-
- while (!value_equal (&x->vars[ROW_VAR].values[row],
- &te->values[ROW_VAR], row_var_width))
- {
- for (; col < n_cols; col++)
- *mp++ = 0.0;
- col = 0;
- row++;
- }
-
- while (!value_equal (&x->vars[COL_VAR].values[col],
- &te->values[COL_VAR], col_var_width))
- {
- *mp++ = 0.0;
- col++;
- }
-
- *mp++ = te->count;
- if (++col >= n_cols)
- {
- col = 0;
- row++;
- }
- }
- while (mp < &x->mat[n_cols * n_rows])
- *mp++ = 0.0;
- assert (mp == &x->mat[n_cols * n_rows]);
-
- /* Column totals, row totals, ns_rows. */
- mp = x->mat;
- for (col = 0; col < n_cols; col++)
- x->col_tot[col] = 0.0;
- for (row = 0; row < n_rows; row++)
- x->row_tot[row] = 0.0;
- x->ns_rows = 0;
- for (row = 0; row < n_rows; row++)
- {
- bool row_is_empty = true;
- for (col = 0; col < n_cols; col++)
- {
- if (*mp != 0.0)
- {
- row_is_empty = false;
- x->col_tot[col] += *mp;
- x->row_tot[row] += *mp;
- }
- mp++;
- }
- if (!row_is_empty)
- x->ns_rows++;
- }
- assert (mp == &x->mat[n_cols * n_rows]);
-
- /* ns_cols. */
- x->ns_cols = 0;
- for (col = 0; col < n_cols; col++)
- for (row = 0; row < n_rows; row++)
- if (x->mat[col + row * n_cols] != 0.0)
- {
- x->ns_cols++;
- break;
- }
-
- /* Grand total. */
- x->total = 0.0;
- for (col = 0; col < n_cols; col++)
- x->total += x->col_tot[col];
-}
-
-static void
-add_var_dimension (struct pivot_table *table, const struct xtab_var *var,
- enum pivot_axis_type axis_type, bool total)
-{
- struct pivot_dimension *d = pivot_dimension_create__ (
- table, axis_type, pivot_value_new_variable (var->var));
-
- struct pivot_footnote *missing_footnote = pivot_table_create_footnote (
- table, pivot_value_new_text (N_("Missing value")));
-
- struct pivot_category *group = pivot_category_create_group__ (
- d->root, pivot_value_new_variable (var->var));
- for (size_t j = 0; j < var->n_values; j++)
- {
- struct pivot_value *value = pivot_value_new_var_value (
- var->var, &var->values[j]);
- if (var_is_value_missing (var->var, &var->values[j]))
- pivot_value_add_footnote (value, missing_footnote);
- pivot_category_create_leaf (group, value);
- }
-
- if (total)
- pivot_category_create_leaf (d->root, pivot_value_new_text (N_("Total")));
-}
-
-static struct pivot_table *
-create_crosstab_table (struct crosstabs_proc *proc, struct crosstabulation *xt,
- size_t crs_leaves[CRS_N_CELLS])
-{
- /* Title. */
- struct string title = DS_EMPTY_INITIALIZER;
- for (size_t i = 0; i < xt->n_vars; i++)
- {
- if (i)
- ds_put_cstr (&title, " × ");
- ds_put_cstr (&title, var_to_string (xt->vars[i].var));
- }
- for (size_t i = 0; i < xt->n_consts; i++)
- {
- const struct variable *var = xt->const_vars[i].var;
- const union value *value = &xt->entries[0]->values[2 + i];
- char *s;
-
- ds_put_format (&title, ", %s=", var_to_string (var));
-
- /* Insert the formatted value of VAR without any leading spaces. */
- s = data_out (value, var_get_encoding (var), var_get_print_format (var),
- settings_get_fmt_settings ());
- ds_put_cstr (&title, s + strspn (s, " "));
- free (s);
- }
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_user_text_nocopy (ds_steal_cstr (&title)),
- "Crosstabulation");
- pivot_table_set_weight_format (table, &proc->weight_format);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"));
-
- struct statistic
- {
- const char *label;
- const char *rc;
- };
- static const struct statistic stats[CRS_N_CELLS] =
- {
-#define C(KEYWORD, STRING, RC) { STRING, RC },
- CRS_CELLS
-#undef C
- };
- for (size_t i = 0; i < CRS_N_CELLS; i++)
- if (proc->cells & (1u << i) && stats[i].label)
- crs_leaves[i] = pivot_category_create_leaf_rc (
- statistics->root, pivot_value_new_text (stats[i].label),
- stats[i].rc);
-
- for (size_t i = 0; i < xt->n_vars; i++)
- add_var_dimension (table, &xt->vars[i],
- i == COL_VAR ? PIVOT_AXIS_COLUMN : PIVOT_AXIS_ROW,
- true);
-
- return table;
-}
-
-static struct pivot_table *
-create_chisq_table (struct crosstabulation *xt)
-{
- struct pivot_table *chisq = pivot_table_create (N_("Chi-Square Tests"));
- pivot_table_set_weight_format (chisq, &xt->weight_format);
-
- pivot_dimension_create (
- chisq, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Pearson Chi-Square"),
- N_("Likelihood Ratio"),
- N_("Fisher's Exact Test"),
- N_("Continuity Correction"),
- N_("Linear-by-Linear Association"),
- N_("N of Valid Cases"), PIVOT_RC_COUNT);
-
- pivot_dimension_create (
- chisq, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Value"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_COUNT,
- N_("Asymptotic Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- for (size_t i = 2; i < xt->n_vars; i++)
- add_var_dimension (chisq, &xt->vars[i], PIVOT_AXIS_ROW, false);
-
- return chisq;
-}
-
-/* Symmetric measures. */
-static struct pivot_table *
-create_sym_table (struct crosstabulation *xt)
-{
- struct pivot_table *sym = pivot_table_create (N_("Symmetric Measures"));
- pivot_table_set_weight_format (sym, &xt->weight_format);
-
- pivot_dimension_create (
- sym, PIVOT_AXIS_COLUMN, N_("Values"),
- N_("Value"), PIVOT_RC_OTHER,
- N_("Asymp. Std. Error"), PIVOT_RC_OTHER,
- N_("Approx. T"), PIVOT_RC_OTHER,
- N_("Approx. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- sym, PIVOT_AXIS_ROW, N_("Statistics"));
- pivot_category_create_group (
- statistics->root, N_("Nominal by Nominal"),
- N_("Phi"), N_("Cramer's V"), N_("Contingency Coefficient"));
- pivot_category_create_group (
- statistics->root, N_("Ordinal by Ordinal"),
- N_("Kendall's tau-b"), N_("Kendall's tau-c"),
- N_("Gamma"), N_("Spearman Correlation"));
- pivot_category_create_group (
- statistics->root, N_("Interval by Interval"),
- N_("Pearson's R"));
- pivot_category_create_group (
- statistics->root, N_("Measure of Agreement"),
- N_("Kappa"));
- pivot_category_create_leaves (statistics->root, N_("N of Valid Cases"),
- PIVOT_RC_COUNT);
-
- for (size_t i = 2; i < xt->n_vars; i++)
- add_var_dimension (sym, &xt->vars[i], PIVOT_AXIS_ROW, false);
-
- return sym;
-}
-
-/* Risk estimate. */
-static struct pivot_table *
-create_risk_table (struct crosstabulation *xt,
- struct pivot_dimension **risk_statistics)
-{
- struct pivot_table *risk = pivot_table_create (N_("Risk Estimate"));
- pivot_table_set_weight_format (risk, &xt->weight_format);
-
- struct pivot_dimension *values = pivot_dimension_create (
- risk, PIVOT_AXIS_COLUMN, N_("Values"),
- N_("Value"), PIVOT_RC_OTHER);
- pivot_category_create_group (
- /* xgettext:no-c-format */
- values->root, N_("95% Confidence Interval"),
- N_("Lower"), PIVOT_RC_OTHER,
- N_("Upper"), PIVOT_RC_OTHER);
-
- *risk_statistics = pivot_dimension_create (
- risk, PIVOT_AXIS_ROW, N_("Statistics"));
-
- for (size_t i = 2; i < xt->n_vars; i++)
- add_var_dimension (risk, &xt->vars[i], PIVOT_AXIS_ROW, false);
-
- return risk;
-}
-
-static void
-create_direct_stat (struct pivot_category *parent,
- const struct crosstabulation *xt,
- const char *name, bool symmetric)
-{
- struct pivot_category *group = pivot_category_create_group (
- parent, name);
- if (symmetric)
- pivot_category_create_leaf (group, pivot_value_new_text (N_("Symmetric")));
-
- char *row_label = xasprintf (_("%s Dependent"),
- var_to_string (xt->vars[ROW_VAR].var));
- pivot_category_create_leaf (group, pivot_value_new_user_text_nocopy (
- row_label));
-
- char *col_label = xasprintf (_("%s Dependent"),
- var_to_string (xt->vars[COL_VAR].var));
- pivot_category_create_leaf (group, pivot_value_new_user_text_nocopy (
- col_label));
-}
-
-/* Directional measures. */
-static struct pivot_table *
-create_direct_table (struct crosstabulation *xt)
-{
- struct pivot_table *direct = pivot_table_create (N_("Directional Measures"));
- pivot_table_set_weight_format (direct, &xt->weight_format);
-
- pivot_dimension_create (
- direct, PIVOT_AXIS_COLUMN, N_("Values"),
- N_("Value"), PIVOT_RC_OTHER,
- N_("Asymp. Std. Error"), PIVOT_RC_OTHER,
- N_("Approx. T"), PIVOT_RC_OTHER,
- N_("Approx. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- direct, PIVOT_AXIS_ROW, N_("Statistics"));
- struct pivot_category *nn = pivot_category_create_group (
- statistics->root, N_("Nominal by Nominal"));
- create_direct_stat (nn, xt, N_("Lambda"), true);
- create_direct_stat (nn, xt, N_("Goodman and Kruskal tau"), false);
- create_direct_stat (nn, xt, N_("Uncertainty Coefficient"), true);
- struct pivot_category *oo = pivot_category_create_group (
- statistics->root, N_("Ordinal by Ordinal"));
- create_direct_stat (oo, xt, N_("Somers' d"), true);
- struct pivot_category *ni = pivot_category_create_group (
- statistics->root, N_("Nominal by Interval"));
- create_direct_stat (ni, xt, N_("Eta"), false);
-
- for (size_t i = 2; i < xt->n_vars; i++)
- add_var_dimension (direct, &xt->vars[i], PIVOT_AXIS_ROW, false);
-
- return direct;
-}
-
-/* Delete missing rows and columns for statistical analysis when
- /MISSING=REPORT. */
-static void
-delete_missing (struct crosstabulation *xt)
-{
- size_t n_rows = xt->vars[ROW_VAR].n_values;
- size_t n_cols = xt->vars[COL_VAR].n_values;
-
- for (size_t r = 0; r < n_rows; r++)
- if (var_is_num_missing (xt->vars[ROW_VAR].var,
- xt->vars[ROW_VAR].values[r].f) == MV_USER)
- {
- for (size_t c = 0; c < n_cols; c++)
- xt->mat[c + r * n_cols] = 0.;
- xt->ns_rows--;
- }
-
-
- for (size_t c = 0; c < n_cols; c++)
- if (var_is_num_missing (xt->vars[COL_VAR].var,
- xt->vars[COL_VAR].values[c].f) == MV_USER)
- {
- for (size_t r = 0; r < n_rows; r++)
- xt->mat[c + r * n_cols] = 0.;
- xt->ns_cols--;
- }
-}
-
-static bool
-find_crosstab (struct crosstabulation *xt, size_t *row0p, size_t *row1p)
-{
- size_t row0 = *row1p;
- if (row0 >= xt->n_entries)
- return false;
-
- size_t row1;
- for (row1 = row0 + 1; row1 < xt->n_entries; row1++)
- {
- struct freq *a = xt->entries[row0];
- struct freq *b = xt->entries[row1];
- if (compare_table_entry_vars_3way (a, b, xt, 2, xt->n_vars) != 0)
- break;
- }
- *row0p = row0;
- *row1p = row1;
- return true;
-}
-
-/* Compares `union value's A_ and B_ and returns a strcmp()-like
- result. WIDTH_ points to an int which is either 0 for a
- numeric value or a string width for a string value. */
-static int
-compare_value_3way (const void *a_, const void *b_, const void *width_)
-{
- const union value *a = a_;
- const union value *b = b_;
- const int *width = width_;
-
- return value_compare_3way (a, b, *width);
-}
-
-/* Inverted version of the above */
-static int
-compare_value_3way_inv (const void *a_, const void *b_, const void *width_)
-{
- return -compare_value_3way (a_, b_, width_);
-}
-
-
-/* Given an array of ENTRY_CNT table_entry structures starting at
- ENTRIES, creates a sorted list of the values that the variable
- with index VAR_IDX takes on. Stores the array of the values in
- XT->values and the number of values in XT->n_values. */
-static void
-enum_var_values (const struct crosstabulation *xt, int var_idx,
- bool descending)
-{
- struct xtab_var *xv = &xt->vars[var_idx];
- const struct var_range *range = get_var_range (xt->proc, xv->var);
-
- if (range)
- {
- xv->values = xnmalloc (range->count, sizeof *xv->values);
- xv->n_values = range->count;
- for (size_t i = 0; i < range->count; i++)
- xv->values[i].f = range->min + i;
- }
- else
- {
- int width = var_get_width (xv->var);
- struct hmapx set = HMAPX_INITIALIZER (set);
-
- for (size_t i = 0; i < xt->n_entries; i++)
- {
- const struct freq *te = xt->entries[i];
- const union value *value = &te->values[var_idx];
- size_t hash = value_hash (value, width, 0);
-
- const union value *iter;
- struct hmapx_node *node;
- HMAPX_FOR_EACH_WITH_HASH (iter, node, hash, &set)
- if (value_equal (iter, value, width))
- goto next_entry;
-
- hmapx_insert (&set, (union value *) value, hash);
-
- next_entry: ;
- }
-
- xv->n_values = hmapx_count (&set);
- xv->values = xnmalloc (xv->n_values, sizeof *xv->values);
- size_t i = 0;
- const union value *iter;
- struct hmapx_node *node;
- HMAPX_FOR_EACH (iter, node, &set)
- xv->values[i++] = *iter;
- hmapx_destroy (&set);
-
- sort (xv->values, xv->n_values, sizeof *xv->values,
- descending ? compare_value_3way_inv : compare_value_3way,
- &width);
- }
-}
-
-static void
-free_var_values (const struct crosstabulation *xt, int var_idx)
-{
- struct xtab_var *xv = &xt->vars[var_idx];
- free (xv->values);
- xv->values = NULL;
- xv->n_values = 0;
-}
-
-/* Displays the crosstabulation table. */
-static void
-display_crosstabulation (struct crosstabs_proc *proc,
- struct crosstabulation *xt, struct pivot_table *table,
- size_t crs_leaves[CRS_N_CELLS])
-{
- size_t n_rows = xt->vars[ROW_VAR].n_values;
- size_t n_cols = xt->vars[COL_VAR].n_values;
-
- size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
- assert (xt->n_vars == 2);
- for (size_t i = 0; i < xt->n_consts; i++)
- indexes[i + 3] = xt->const_indexes[i];
-
- /* Put in the actual cells. */
- double *mp = xt->mat;
- for (size_t r = 0; r < n_rows; r++)
- {
- if (!xt->row_tot[r] && proc->mode != INTEGER)
- continue;
-
- indexes[ROW_VAR + 1] = r;
- for (size_t c = 0; c < n_cols; c++)
- {
- if (!xt->col_tot[c] && proc->mode != INTEGER)
- continue;
-
- indexes[COL_VAR + 1] = c;
-
- double expected_value = xt->row_tot[r] * xt->col_tot[c] / xt->total;
- double residual = *mp - expected_value;
- double sresidual = residual / sqrt (expected_value);
- double asresidual
- = residual / sqrt (expected_value
- * (1. - xt->row_tot[r] / xt->total)
- * (1. - xt->col_tot[c] / xt->total));
- double entries[CRS_N_CELLS] = {
- [CRS_CL_COUNT] = *mp,
- [CRS_CL_ROW] = *mp / xt->row_tot[r] * 100.,
- [CRS_CL_COLUMN] = *mp / xt->col_tot[c] * 100.,
- [CRS_CL_TOTAL] = *mp / xt->total * 100.,
- [CRS_CL_EXPECTED] = expected_value,
- [CRS_CL_RESIDUAL] = residual,
- [CRS_CL_SRESIDUAL] = sresidual,
- [CRS_CL_ASRESIDUAL] = asresidual,
- };
- for (size_t i = 0; i < proc->n_cells; i++)
- {
- int cell = proc->a_cells[i];
- indexes[0] = crs_leaves[cell];
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (entries[cell]));
- }
-
- mp++;
- }
- }
-
- /* Row totals. */
- for (size_t r = 0; r < n_rows; r++)
- {
- if (!xt->row_tot[r] && proc->mode != INTEGER)
- continue;
-
- double expected_value = xt->row_tot[r] / xt->total;
- double entries[CRS_N_CELLS] = {
- [CRS_CL_COUNT] = xt->row_tot[r],
- [CRS_CL_ROW] = 100.0,
- [CRS_CL_COLUMN] = expected_value * 100.,
- [CRS_CL_TOTAL] = expected_value * 100.,
- [CRS_CL_EXPECTED] = expected_value,
- [CRS_CL_RESIDUAL] = SYSMIS,
- [CRS_CL_SRESIDUAL] = SYSMIS,
- [CRS_CL_ASRESIDUAL] = SYSMIS,
- };
- for (size_t i = 0; i < proc->n_cells; i++)
- {
- int cell = proc->a_cells[i];
- double entry = entries[cell];
- if (entry != SYSMIS)
- {
- indexes[ROW_VAR + 1] = r;
- indexes[COL_VAR + 1] = n_cols;
- indexes[0] = crs_leaves[cell];
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (entry));
- }
- }
- }
-
- for (size_t c = 0; c <= n_cols; c++)
- {
- if (c < n_cols && !xt->col_tot[c] && proc->mode != INTEGER)
- continue;
-
- double ct = c < n_cols ? xt->col_tot[c] : xt->total;
- double expected_value = ct / xt->total;
- double entries[CRS_N_CELLS] = {
- [CRS_CL_COUNT] = ct,
- [CRS_CL_ROW] = expected_value * 100.0,
- [CRS_CL_COLUMN] = 100.0,
- [CRS_CL_TOTAL] = expected_value * 100.,
- [CRS_CL_EXPECTED] = expected_value,
- [CRS_CL_RESIDUAL] = SYSMIS,
- [CRS_CL_SRESIDUAL] = SYSMIS,
- [CRS_CL_ASRESIDUAL] = SYSMIS,
- };
- for (size_t i = 0; i < proc->n_cells; i++)
- {
- size_t cell = proc->a_cells[i];
- double entry = entries[cell];
- if (entry != SYSMIS)
- {
- indexes[ROW_VAR + 1] = n_rows;
- indexes[COL_VAR + 1] = c;
- indexes[0] = crs_leaves[cell];
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (entry));
- }
- }
- }
-
- free (indexes);
-}
-
-static void calc_r (struct crosstabulation *,
- double *XT, double *Y, double *, double *, double *);
-static void calc_chisq (struct crosstabulation *,
- double[N_CHISQ], int[N_CHISQ], double *, double *);
-
-/* Display chi-square statistics. */
-static void
-display_chisq (struct crosstabulation *xt, struct pivot_table *chisq)
-{
- double chisq_v[N_CHISQ];
- double fisher1, fisher2;
- int df[N_CHISQ];
- calc_chisq (xt, chisq_v, df, &fisher1, &fisher2);
-
- size_t *indexes = xnmalloc (chisq->n_dimensions, sizeof *indexes);
- assert (xt->n_vars == 2);
- for (size_t i = 0; i < xt->n_consts; i++)
- indexes[i + 2] = xt->const_indexes[i];
- for (size_t i = 0; i < N_CHISQ; i++)
- {
- indexes[0] = i;
-
- double entries[5] = { SYSMIS, SYSMIS, SYSMIS, SYSMIS, SYSMIS };
- if (i == 2)
- {
- entries[3] = fisher2;
- entries[4] = fisher1;
- }
- else if (chisq_v[i] != SYSMIS)
- {
- entries[0] = chisq_v[i];
- entries[1] = df[i];
- entries[2] = gsl_cdf_chisq_Q (chisq_v[i], df[i]);
- }
-
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- if (entries[j] != SYSMIS)
- {
- indexes[1] = j;
- pivot_table_put (chisq, indexes, chisq->n_dimensions,
- pivot_value_new_number (entries[j]));
- }
- }
-
- indexes[0] = 5;
- indexes[1] = 0;
- pivot_table_put (chisq, indexes, chisq->n_dimensions,
- pivot_value_new_number (xt->total));
-
- free (indexes);
-}
-
-static bool calc_symmetric (struct crosstabs_proc *, struct crosstabulation *,
- double[N_SYMMETRIC], double[N_SYMMETRIC],
- double[N_SYMMETRIC],
- double[3], double[3], double[3]);
-
-/* Display symmetric measures. */
-static void
-display_symmetric (struct crosstabs_proc *proc, struct crosstabulation *xt,
- struct pivot_table *sym)
-{
- double sym_v[N_SYMMETRIC], sym_ase[N_SYMMETRIC], sym_t[N_SYMMETRIC];
- double somers_d_v[3], somers_d_ase[3], somers_d_t[3];
-
- if (!calc_symmetric (proc, xt, sym_v, sym_ase, sym_t,
- somers_d_v, somers_d_ase, somers_d_t))
- return;
-
- size_t *indexes = xnmalloc (sym->n_dimensions, sizeof *indexes);
- assert (xt->n_vars == 2);
- for (size_t i = 0; i < xt->n_consts; i++)
- indexes[i + 2] = xt->const_indexes[i];
-
- for (size_t i = 0; i < N_SYMMETRIC; i++)
- {
- if (sym_v[i] == SYSMIS)
- continue;
-
- indexes[1] = i;
-
- double entries[] = { sym_v[i], sym_ase[i], sym_t[i] };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- if (entries[j] != SYSMIS)
- {
- indexes[0] = j;
- pivot_table_put (sym, indexes, sym->n_dimensions,
- pivot_value_new_number (entries[j]));
- }
- }
-
- indexes[1] = N_SYMMETRIC;
- indexes[0] = 0;
- struct pivot_value *total = pivot_value_new_number (xt->total);
- pivot_value_set_rc (sym, total, PIVOT_RC_COUNT);
- pivot_table_put (sym, indexes, sym->n_dimensions, total);
-
- free (indexes);
-}
-
-static bool calc_risk (struct crosstabulation *,
- double[], double[], double[], union value *,
- double *);
-
-/* Display risk estimate. */
-static void
-display_risk (struct crosstabulation *xt, struct pivot_table *risk,
- struct pivot_dimension *risk_statistics)
-{
- double risk_v[3], lower[3], upper[3], n_valid;
- union value c[2];
- if (!calc_risk (xt, risk_v, upper, lower, c, &n_valid))
- return;
- assert (risk_statistics);
-
- size_t *indexes = xnmalloc (risk->n_dimensions, sizeof *indexes);
- assert (xt->n_vars == 2);
- for (size_t i = 0; i < xt->n_consts; i++)
- indexes[i + 2] = xt->const_indexes[i];
-
- for (size_t i = 0; i < 3; i++)
- {
- const struct variable *cv = xt->vars[COL_VAR].var;
- const struct variable *rv = xt->vars[ROW_VAR].var;
-
- if (risk_v[i] == SYSMIS)
- continue;
-
- struct string label = DS_EMPTY_INITIALIZER;
- switch (i)
- {
- case 0:
- ds_put_format (&label, _("Odds Ratio for %s"), var_to_string (rv));
- ds_put_cstr (&label, " (");
- var_append_value_name (rv, &c[0], &label);
- ds_put_cstr (&label, " / ");
- var_append_value_name (rv, &c[1], &label);
- ds_put_cstr (&label, ")");
- break;
- case 1:
- case 2:
- ds_put_format (&label, _("For cohort %s = "), var_to_string (cv));
- var_append_value_name (cv, &xt->vars[ROW_VAR].values[i - 1], &label);
- break;
- }
-
- indexes[1] = pivot_category_create_leaf (
- risk_statistics->root,
- pivot_value_new_user_text_nocopy (ds_steal_cstr (&label)));
-
- double entries[] = { risk_v[i], lower[i], upper[i] };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- {
- indexes[0] = j;
- pivot_table_put (risk, indexes, risk->n_dimensions,
- pivot_value_new_number (entries[i]));
- }
- }
- indexes[1] = pivot_category_create_leaf (
- risk_statistics->root,
- pivot_value_new_text (N_("N of Valid Cases")));
- indexes[0] = 0;
- pivot_table_put (risk, indexes, risk->n_dimensions,
- pivot_value_new_number (n_valid));
- free (indexes);
-}
-
-static void calc_directional (struct crosstabs_proc *, struct crosstabulation *,
- double[N_DIRECTIONAL], double[N_DIRECTIONAL],
- double[N_DIRECTIONAL], double[N_DIRECTIONAL]);
-
-/* Display directional measures. */
-static void
-display_directional (struct crosstabs_proc *proc,
- struct crosstabulation *xt, struct pivot_table *direct)
-{
- double direct_v[N_DIRECTIONAL];
- double direct_ase[N_DIRECTIONAL];
- double direct_t[N_DIRECTIONAL];
- double sig[N_DIRECTIONAL];
- calc_directional (proc, xt, direct_v, direct_ase, direct_t, sig);
-
- size_t *indexes = xnmalloc (direct->n_dimensions, sizeof *indexes);
- assert (xt->n_vars == 2);
- for (size_t i = 0; i < xt->n_consts; i++)
- indexes[i + 2] = xt->const_indexes[i];
-
- for (size_t i = 0; i < N_DIRECTIONAL; i++)
- {
- if (direct_v[i] == SYSMIS)
- continue;
-
- indexes[1] = i;
-
- double entries[] = {
- direct_v[i], direct_ase[i], direct_t[i], sig[i],
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- if (entries[j] != SYSMIS)
- {
- indexes[0] = j;
- pivot_table_put (direct, indexes, direct->n_dimensions,
- pivot_value_new_number (entries[j]));
- }
- }
-
- free (indexes);
-}
-\f
-/* Statistical calculations. */
-
-/* Returns the value of the logarithm of gamma (factorial) function for an integer
- argument XT. */
-static double
-log_gamma_int (double xt)
-{
- double r = 0;
- for (int i = 2; i < xt; i++)
- r += log(i);
- return r;
-}
-
-/* Calculate P_r as specified in _SPSS Statistical Algorithms_,
- Appendix 5. */
-static inline double
-Pr (int a, int b, int c, int d)
-{
- return exp (log_gamma_int (a + b + 1.) - log_gamma_int (a + 1.)
- + log_gamma_int (c + d + 1.) - log_gamma_int (b + 1.)
- + log_gamma_int (a + c + 1.) - log_gamma_int (c + 1.)
- + log_gamma_int (b + d + 1.) - log_gamma_int (d + 1.)
- - log_gamma_int (a + b + c + d + 1.));
-}
-
-/* Swap the contents of A and B. */
-static inline void
-swap (int *a, int *b)
-{
- int t = *a;
- *a = *b;
- *b = t;
-}
-
-/* Calculate significance for Fisher's exact test as specified in
- _SPSS Statistical Algorithms_, Appendix 5. */
-static void
-calc_fisher (int a, int b, int c, int d, double *fisher1, double *fisher2)
-{
- if (MIN (c, d) < MIN (a, b))
- swap (&a, &c), swap (&b, &d);
- if (MIN (b, d) < MIN (a, c))
- swap (&a, &b), swap (&c, &d);
- if (b * c < a * d)
- {
- if (b < c)
- swap (&a, &b), swap (&c, &d);
- else
- swap (&a, &c), swap (&b, &d);
- }
-
- double pn1 = Pr (a, b, c, d);
- *fisher1 = pn1;
- for (int xt = 1; xt <= a; xt++)
- *fisher1 += Pr (a - xt, b + xt, c + xt, d - xt);
-
- *fisher2 = *fisher1;
- for (int xt = 1; xt <= b; xt++)
- {
- double p = Pr (a + xt, b - xt, c - xt, d + xt);
- if (p < pn1)
- *fisher2 += p;
- }
-}
-
-/* Calculates chi-squares into CHISQ. MAT is a matrix with N_COLS
- columns with values COLS and N_ROWS rows with values ROWS. Values
- in the matrix sum to xt->total. */
-static void
-calc_chisq (struct crosstabulation *xt,
- double chisq[N_CHISQ], int df[N_CHISQ],
- double *fisher1, double *fisher2)
-{
- chisq[0] = chisq[1] = 0.;
- chisq[2] = chisq[3] = chisq[4] = SYSMIS;
- *fisher1 = *fisher2 = SYSMIS;
-
- df[0] = df[1] = (xt->ns_cols - 1) * (xt->ns_rows - 1);
-
- if (xt->ns_rows <= 1 || xt->ns_cols <= 1)
- {
- chisq[0] = chisq[1] = SYSMIS;
- return;
- }
-
- size_t n_cols = xt->vars[COL_VAR].n_values;
- FOR_EACH_POPULATED_ROW (r, xt)
- FOR_EACH_POPULATED_COLUMN (c, xt)
- {
- const double expected = xt->row_tot[r] * xt->col_tot[c] / xt->total;
- const double freq = xt->mat[n_cols * r + c];
- const double residual = freq - expected;
-
- chisq[0] += residual * residual / expected;
- if (freq)
- chisq[1] += freq * log (expected / freq);
- }
-
- if (chisq[0] == 0.)
- chisq[0] = SYSMIS;
-
- if (chisq[1] != 0.)
- chisq[1] *= -2.;
- else
- chisq[1] = SYSMIS;
-
- /* Calculate Yates and Fisher exact test. */
- if (xt->ns_cols == 2 && xt->ns_rows == 2)
- {
- int nz_cols[2];
-
- size_t j = 0;
- FOR_EACH_POPULATED_COLUMN (c, xt)
- {
- nz_cols[j++] = c;
- if (j == 2)
- break;
- }
- assert (j == 2);
-
- double f11 = xt->mat[nz_cols[0]];
- double f12 = xt->mat[nz_cols[1]];
- double f21 = xt->mat[nz_cols[0] + n_cols];
- double f22 = xt->mat[nz_cols[1] + n_cols];
-
- /* Yates. */
- const double xt_ = fabs (f11 * f22 - f12 * f21) - 0.5 * xt->total;
-
- if (xt_ > 0.)
- chisq[3] = (xt->total * pow2 (xt_)
- / (f11 + f12) / (f21 + f22)
- / (f11 + f21) / (f12 + f22));
- else
- chisq[3] = 0.;
-
- df[3] = 1.;
-
- /* Fisher. */
- calc_fisher (f11 + .5, f12 + .5, f21 + .5, f22 + .5, fisher1, fisher2);
- }
-
- /* Calculate Mantel-Haenszel. */
- if (var_is_numeric (xt->vars[ROW_VAR].var)
- && var_is_numeric (xt->vars[COL_VAR].var))
- {
- double r, ase_0, ase_1;
- calc_r (xt, (double *) xt->vars[ROW_VAR].values,
- (double *) xt->vars[COL_VAR].values,
- &r, &ase_0, &ase_1);
-
- chisq[4] = (xt->total - 1.) * r * r;
- df[4] = 1;
- }
-}
-
-/* Calculate the value of Pearson's r. r is stored into R, its T value into
- T, and standard error into ERROR. The row and column values must be
- passed in XT and Y. */
-static void
-calc_r (struct crosstabulation *xt,
- double *XT, double *Y, double *r, double *t, double *error)
-{
- size_t n_rows = xt->vars[ROW_VAR].n_values;
- size_t n_cols = xt->vars[COL_VAR].n_values;
-
- double sum_XYf = 0;
- for (size_t i = 0; i < n_rows; i++)
- for (size_t j = 0; j < n_cols; j++)
- {
- double fij = xt->mat[j + i * n_cols];
- double product = XT[i] * Y[j];
- double temp = fij * product;
- sum_XYf += temp;
- }
-
- double sum_Xr = 0;
- double sum_X2r = 0;
- for (size_t i = 0; i < n_rows; i++)
- {
- sum_Xr += XT[i] * xt->row_tot[i];
- sum_X2r += pow2 (XT[i]) * xt->row_tot[i];
- }
- double Xbar = sum_Xr / xt->total;
-
- double sum_Yc = 0;
- double sum_Y2c = 0;
- for (size_t i = 0; i < n_cols; i++)
- {
- sum_Yc += Y[i] * xt->col_tot[i];
- sum_Y2c += Y[i] * Y[i] * xt->col_tot[i];
- }
- double Ybar = sum_Yc / xt->total;
-
- double S = sum_XYf - sum_Xr * sum_Yc / xt->total;
- double SX = sum_X2r - pow2 (sum_Xr) / xt->total;
- double SY = sum_Y2c - pow2 (sum_Yc) / xt->total;
- double T = sqrt (SX * SY);
- *r = S / T;
- *t = *r / sqrt (1 - pow2 (*r)) * sqrt (xt->total - 2);
-
- double s = 0;
- double c = 0;
- for (size_t i = 0; i < n_rows; i++)
- for (size_t j = 0; j < n_cols; j++)
- {
- double Xresid = XT[i] - Xbar;
- double Yresid = Y[j] - Ybar;
- double temp = (T * Xresid * Yresid
- - ((S / (2. * T))
- * (Xresid * Xresid * SY + Yresid * Yresid * SX)));
- double y = xt->mat[j + i * n_cols] * temp * temp - c;
- double t = s + y;
- c = (t - s) - y;
- s = t;
- }
- *error = sqrt (s) / (T * T);
-}
-
-/* Calculate symmetric statistics and their asymptotic standard
- errors. Returns false if none could be calculated. */
-static bool
-calc_symmetric (struct crosstabs_proc *proc, struct crosstabulation *xt,
- double v[N_SYMMETRIC], double ase[N_SYMMETRIC],
- double t[N_SYMMETRIC],
- double somers_d_v[3], double somers_d_ase[3],
- double somers_d_t[3])
-{
- size_t n_rows = xt->vars[ROW_VAR].n_values;
- size_t n_cols = xt->vars[COL_VAR].n_values;
-
- size_t q = MIN (xt->ns_rows, xt->ns_cols);
- if (q <= 1)
- return false;
-
- for (size_t i = 0; i < N_SYMMETRIC; i++)
- v[i] = ase[i] = t[i] = SYSMIS;
-
- /* Phi, Cramer's V, contingency coefficient. */
- if (proc->statistics & (CRS_ST_PHI | CRS_ST_CC))
- {
- double Xp = 0.; /* Pearson chi-square. */
-
- FOR_EACH_POPULATED_ROW (r, xt)
- FOR_EACH_POPULATED_COLUMN (c, xt)
- {
- double expected = xt->row_tot[r] * xt->col_tot[c] / xt->total;
- double freq = xt->mat[n_cols * r + c];
- double residual = freq - expected;
-
- Xp += residual * residual / expected;
- }
-
- if (proc->statistics & CRS_ST_PHI)
- {
- v[0] = sqrt (Xp / xt->total);
- v[1] = sqrt (Xp / (xt->total * (q - 1)));
- }
- if (proc->statistics & CRS_ST_CC)
- v[2] = sqrt (Xp / (Xp + xt->total));
- }
-
- if (proc->statistics & (CRS_ST_BTAU | CRS_ST_CTAU
- | CRS_ST_GAMMA | CRS_ST_D))
- {
- double Dr = pow2 (xt->total);
- for (size_t r = 0; r < n_rows; r++)
- Dr -= pow2 (xt->row_tot[r]);
-
- double Dc = pow2 (xt->total);
- for (size_t c = 0; c < n_cols; c++)
- Dc -= pow2 (xt->col_tot[c]);
-
- double *cum = xnmalloc (n_cols * n_rows, sizeof *cum);
- for (size_t c = 0; c < n_cols; c++)
- {
- double ct = 0.;
-
- for (size_t r = 0; r < n_rows; r++)
- cum[c + r * n_cols] = ct += xt->mat[c + r * n_cols];
- }
-
- /* P and Q. */
- double P = 0;
- double Q = 0;
- for (size_t i = 0; i < n_rows; i++)
- {
- double Cij = 0;
- for (size_t j = 1; j < n_cols; j++)
- Cij += xt->col_tot[j] - cum[j + i * n_cols];
-
- double Dij = 0;
- if (i > 0)
- for (size_t j = 1; j < n_cols; j++)
- Dij += cum[j + (i - 1) * n_cols];
-
- for (size_t j = 0;;)
- {
- double fij = xt->mat[j + i * n_cols];
- P += fij * Cij;
- Q += fij * Dij;
-
- if (++j >= n_cols)
- break;
-
- Cij -= xt->col_tot[j] - cum[j + i * n_cols];
- Dij += xt->col_tot[j - 1] - cum[j - 1 + i * n_cols];
-
- if (i > 0)
- {
- Cij += cum[j - 1 + (i - 1) * n_cols];
- Dij -= cum[j + (i - 1) * n_cols];
- }
- }
- }
-
- if (proc->statistics & CRS_ST_BTAU)
- v[3] = (P - Q) / sqrt (Dr * Dc);
- if (proc->statistics & CRS_ST_CTAU)
- v[4] = (q * (P - Q)) / (pow2 (xt->total) * (q - 1));
- if (proc->statistics & CRS_ST_GAMMA)
- v[5] = (P - Q) / (P + Q);
-
- /* ASE for tau-b, tau-c, gamma. Calculations could be
- eliminated here, at expense of memory. */
- double btau_cum = 0;
- double ctau_cum = 0;
- double gamma_cum = 0;
- double d_yx_cum = 0;
- double d_xy_cum = 0;
- for (size_t i = 0; i < n_rows; i++)
- {
- double Cij = 0;
- for (size_t j = 1; j < n_cols; j++)
- Cij += xt->col_tot[j] - cum[j + i * n_cols];
-
- double Dij = 0;
- if (i > 0)
- for (size_t j = 1; j < n_cols; j++)
- Dij += cum[j + (i - 1) * n_cols];
-
- for (size_t j = 0;;)
- {
- double fij = xt->mat[j + i * n_cols];
-
- if (proc->statistics & CRS_ST_BTAU)
- btau_cum += fij * pow2 (2. * sqrt (Dr * Dc) * (Cij - Dij)
- + v[3] * (xt->row_tot[i] * Dc
- + xt->col_tot[j] * Dr));
- ctau_cum += fij * pow2 (Cij - Dij);
-
- if (proc->statistics & CRS_ST_GAMMA)
- gamma_cum += fij * pow2 (Q * Cij - P * Dij);
-
- if (proc->statistics & CRS_ST_D)
- {
- d_yx_cum += fij * pow2 (Dr * (Cij - Dij)
- - (P - Q) * (xt->total - xt->row_tot[i]));
- d_xy_cum += fij * pow2 (Dc * (Dij - Cij)
- - (Q - P) * (xt->total - xt->col_tot[j]));
- }
-
- if (++j >= n_cols)
- break;
-
- Cij -= xt->col_tot[j] - cum[j + i * n_cols];
- Dij += xt->col_tot[j - 1] - cum[j - 1 + i * n_cols];
-
- if (i > 0)
- {
- Cij += cum[j - 1 + (i - 1) * n_cols];
- Dij -= cum[j + (i - 1) * n_cols];
- }
- }
- }
-
- if (proc->statistics & CRS_ST_BTAU)
- {
- double btau_var = ((btau_cum
- - (xt->total * pow2 (xt->total * (P - Q) / sqrt (Dr * Dc) * (Dr + Dc))))
- / pow2 (Dr * Dc));
- ase[3] = sqrt (btau_var);
- t[3] = v[3] / (2 * sqrt ((ctau_cum - (P - Q) * (P - Q) / xt->total)
- / (Dr * Dc)));
- }
- if (proc->statistics & CRS_ST_CTAU)
- {
- ase[4] = ((2 * q / ((q - 1) * pow2 (xt->total)))
- * sqrt (ctau_cum - (P - Q) * (P - Q) / xt->total));
- t[4] = v[4] / ase[4];
- }
- if (proc->statistics & CRS_ST_GAMMA)
- {
- ase[5] = ((4. / ((P + Q) * (P + Q))) * sqrt (gamma_cum));
- t[5] = v[5] / (2. / (P + Q)
- * sqrt (ctau_cum - (P - Q) * (P - Q) / xt->total));
- }
- if (proc->statistics & CRS_ST_D)
- {
- somers_d_v[0] = (P - Q) / (.5 * (Dc + Dr));
- somers_d_ase[0] = SYSMIS;
- somers_d_t[0] = (somers_d_v[0]
- / (4 / (Dc + Dr)
- * sqrt (ctau_cum - pow2 (P - Q) / xt->total)));
- somers_d_v[1] = (P - Q) / Dc;
- somers_d_ase[1] = 2. / pow2 (Dc) * sqrt (d_xy_cum);
- somers_d_t[1] = (somers_d_v[1]
- / (2. / Dc
- * sqrt (ctau_cum - pow2 (P - Q) / xt->total)));
- somers_d_v[2] = (P - Q) / Dr;
- somers_d_ase[2] = 2. / pow2 (Dr) * sqrt (d_yx_cum);
- somers_d_t[2] = (somers_d_v[2]
- / (2. / Dr
- * sqrt (ctau_cum - pow2 (P - Q) / xt->total)));
- }
-
- free (cum);
- }
-
- /* Spearman correlation, Pearson's r. */
- if (proc->statistics & CRS_ST_CORR)
- {
- double *R = xmalloc (sizeof *R * n_rows);
- double c = 0;
- double s = 0;
- for (size_t i = 0; i < n_rows; i++)
- {
- R[i] = s + (xt->row_tot[i] + 1.) / 2.;
- double y = xt->row_tot[i] - c;
- double t = s + y;
- c = (t - s) - y;
- s = t;
- }
-
- double *C = xmalloc (sizeof *C * n_cols);
- c = s = 0;
- for (size_t j = 0; j < n_cols; j++)
- {
- C[j] = s + (xt->col_tot[j] + 1.) / 2;
- double y = xt->col_tot[j] - c;
- double t = s + y;
- c = (t - s) - y;
- s = t;
- }
-
- calc_r (xt, R, C, &v[6], &t[6], &ase[6]);
-
- free (R);
- free (C);
-
- calc_r (xt, (double *) xt->vars[ROW_VAR].values,
- (double *) xt->vars[COL_VAR].values,
- &v[7], &t[7], &ase[7]);
- }
-
- /* Cohen's kappa. */
- if (proc->statistics & CRS_ST_KAPPA && xt->ns_rows == xt->ns_cols)
- {
- double sum_fii = 0;
- double sum_rici = 0;
- double sum_fiiri_ci = 0;
- double sum_riciri_ci = 0;
- for (size_t i = 0, j = 0; i < xt->ns_rows; i++, j++)
- {
- while (xt->col_tot[j] == 0.)
- j++;
-
- double prod = xt->row_tot[i] * xt->col_tot[j];
- double sum = xt->row_tot[i] + xt->col_tot[j];
-
- sum_fii += xt->mat[j + i * n_cols];
- sum_rici += prod;
- sum_fiiri_ci += xt->mat[j + i * n_cols] * sum;
- sum_riciri_ci += prod * sum;
- }
-
- double sum_fijri_ci2 = 0;
- for (size_t i = 0; i < xt->ns_rows; i++)
- for (size_t j = 0; j < xt->ns_cols; j++)
- {
- double sum = xt->row_tot[i] + xt->col_tot[j];
- sum_fijri_ci2 += xt->mat[j + i * n_cols] * sum * sum;
- }
-
- v[8] = (xt->total * sum_fii - sum_rici) / (pow2 (xt->total) - sum_rici);
-
- double ase_under_h0 = sqrt ((pow2 (xt->total) * sum_rici
- + sum_rici * sum_rici
- - xt->total * sum_riciri_ci)
- / (xt->total * (pow2 (xt->total) - sum_rici) * (pow2 (xt->total) - sum_rici)));
-
- ase[8] = sqrt (xt->total * (((sum_fii * (xt->total - sum_fii))
- / pow2 (pow2 (xt->total) - sum_rici))
- + ((2. * (xt->total - sum_fii)
- * (2. * sum_fii * sum_rici
- - xt->total * sum_fiiri_ci))
- / pow3 (pow2 (xt->total) - sum_rici))
- + (pow2 (xt->total - sum_fii)
- * (xt->total * sum_fijri_ci2 - 4.
- * sum_rici * sum_rici)
- / pow4 (pow2 (xt->total) - sum_rici))));
-
- t[8] = v[8] / ase_under_h0;
- }
-
- return true;
-}
-
-/* Calculate risk estimate. */
-static bool
-calc_risk (struct crosstabulation *xt,
- double *value, double *upper, double *lower, union value *c,
- double *n_valid)
-{
- size_t n_cols = xt->vars[COL_VAR].n_values;
-
- for (size_t i = 0; i < 3; i++)
- value[i] = upper[i] = lower[i] = SYSMIS;
-
- if (xt->ns_rows != 2 || xt->ns_cols != 2)
- return false;
-
- /* Find populated columns. */
- size_t nz_cols[2];
- size_t n = 0;
- FOR_EACH_POPULATED_COLUMN (c, xt)
- nz_cols[n++] = c;
- assert (n == 2);
-
- /* Find populated rows. */
- size_t nz_rows[2];
- n = 0;
- FOR_EACH_POPULATED_ROW (r, xt)
- nz_rows[n++] = r;
- assert (n == 2);
-
- double f11 = xt->mat[nz_cols[0] + n_cols * nz_rows[0]];
- double f12 = xt->mat[nz_cols[1] + n_cols * nz_rows[0]];
- double f21 = xt->mat[nz_cols[0] + n_cols * nz_rows[1]];
- double f22 = xt->mat[nz_cols[1] + n_cols * nz_rows[1]];
- *n_valid = f11 + f12 + f21 + f22;
-
- c[0] = xt->vars[COL_VAR].values[nz_cols[0]];
- c[1] = xt->vars[COL_VAR].values[nz_cols[1]];
-
- value[0] = (f11 * f22) / (f12 * f21);
- double v = sqrt (1. / f11 + 1. / f12 + 1. / f21 + 1. / f22);
- lower[0] = value[0] * exp (-1.960 * v);
- upper[0] = value[0] * exp (1.960 * v);
-
- value[1] = (f11 * (f21 + f22)) / (f21 * (f11 + f12));
- v = sqrt ((f12 / (f11 * (f11 + f12)))
- + (f22 / (f21 * (f21 + f22))));
- lower[1] = value[1] * exp (-1.960 * v);
- upper[1] = value[1] * exp (1.960 * v);
-
- value[2] = (f12 * (f21 + f22)) / (f22 * (f11 + f12));
- v = sqrt ((f11 / (f12 * (f11 + f12)))
- + (f21 / (f22 * (f21 + f22))));
- lower[2] = value[2] * exp (-1.960 * v);
- upper[2] = value[2] * exp (1.960 * v);
-
- return true;
-}
-
-/* Calculate directional measures. */
-static void
-calc_directional (struct crosstabs_proc *proc, struct crosstabulation *xt,
- double v[N_DIRECTIONAL], double ase[N_DIRECTIONAL],
- double t[N_DIRECTIONAL], double sig[N_DIRECTIONAL])
-{
- size_t n_rows = xt->vars[ROW_VAR].n_values;
- size_t n_cols = xt->vars[COL_VAR].n_values;
- for (size_t i = 0; i < N_DIRECTIONAL; i++)
- v[i] = ase[i] = t[i] = sig[i] = SYSMIS;
-
- /* Lambda. */
- if (proc->statistics & CRS_ST_LAMBDA)
- {
- /* Find maximum for each row and their sum. */
- double *fim = xnmalloc (n_rows, sizeof *fim);
- size_t *fim_index = xnmalloc (n_rows, sizeof *fim_index);
- double sum_fim = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- {
- double max = xt->mat[i * n_cols];
- size_t index = 0;
-
- for (size_t j = 1; j < n_cols; j++)
- if (xt->mat[j + i * n_cols] > max)
- {
- max = xt->mat[j + i * n_cols];
- index = j;
- }
-
- fim[i] = max;
- sum_fim += max;
- fim_index[i] = index;
- }
-
- /* Find maximum for each column. */
- double *fmj = xnmalloc (n_cols, sizeof *fmj);
- size_t *fmj_index = xnmalloc (n_cols, sizeof *fmj_index);
- double sum_fmj = 0.0;
- for (size_t j = 0; j < n_cols; j++)
- {
- double max = xt->mat[j];
- size_t index = 0;
-
- for (size_t i = 1; i < n_rows; i++)
- if (xt->mat[j + i * n_cols] > max)
- {
- max = xt->mat[j + i * n_cols];
- index = i;
- }
-
- fmj[j] = max;
- sum_fmj += max;
- fmj_index[j] = index;
- }
-
- /* Find maximum row total. */
- double rm = xt->row_tot[0];
- size_t rm_index = 0;
- for (size_t i = 1; i < n_rows; i++)
- if (xt->row_tot[i] > rm)
- {
- rm = xt->row_tot[i];
- rm_index = i;
- }
-
- /* Find maximum column total. */
- double cm = xt->col_tot[0];
- size_t cm_index = 0;
- for (size_t j = 1; j < n_cols; j++)
- if (xt->col_tot[j] > cm)
- {
- cm = xt->col_tot[j];
- cm_index = j;
- }
-
- v[0] = (sum_fim + sum_fmj - cm - rm) / (2. * xt->total - rm - cm);
- v[1] = (sum_fmj - rm) / (xt->total - rm);
- v[2] = (sum_fim - cm) / (xt->total - cm);
-
- /* ASE1 for Y given XT. */
- {
- double accum = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- if (cm_index == fim_index[i])
- accum += fim[i];
- ase[2] = sqrt ((xt->total - sum_fim) * (sum_fim + cm - 2. * accum)
- / pow3 (xt->total - cm));
- }
-
- /* ASE0 for Y given XT. */
- {
- double accum = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- if (cm_index != fim_index[i])
- accum += (xt->mat[i * n_cols + fim_index[i]]
- + xt->mat[i * n_cols + cm_index]);
- t[2] = v[2] / (sqrt (accum - pow2 (sum_fim - cm) / xt->total) / (xt->total - cm));
- }
-
- /* ASE1 for XT given Y. */
- {
- double accum = 0.0;
- for (size_t j = 0; j < n_cols; j++)
- if (rm_index == fmj_index[j])
- accum += fmj[j];
- ase[1] = sqrt ((xt->total - sum_fmj) * (sum_fmj + rm - 2. * accum)
- / pow3 (xt->total - rm));
- }
-
- /* ASE0 for XT given Y. */
- {
- double accum = 0.0;
- for (size_t j = 0; j < n_cols; j++)
- if (rm_index != fmj_index[j])
- accum += (xt->mat[j + n_cols * fmj_index[j]]
- + xt->mat[j + n_cols * rm_index]);
- t[1] = v[1] / (sqrt (accum - pow2 (sum_fmj - rm) / xt->total) / (xt->total - rm));
- }
-
- /* Symmetric ASE0 and ASE1. */
- {
- double accum0 = 0.0;
- double accum1 = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- for (size_t j = 0; j < n_cols; j++)
- {
- int temp0 = (fmj_index[j] == i) + (fim_index[i] == j);
- int temp1 = (i == rm_index) + (j == cm_index);
- accum0 += xt->mat[j + i * n_cols] * pow2 (temp0 - temp1);
- accum1 += (xt->mat[j + i * n_cols]
- * pow2 (temp0 + (v[0] - 1.) * temp1));
- }
- ase[0] = sqrt (accum1 - 4. * xt->total * v[0] * v[0]) / (2. * xt->total - rm - cm);
- t[0] = v[0] / (sqrt (accum0 - pow2 (sum_fim + sum_fmj - cm - rm) / xt->total)
- / (2. * xt->total - rm - cm));
- }
-
- for (size_t i = 0; i < 3; i++)
- sig[i] = 2 * gsl_cdf_ugaussian_Q (t[i]);
-
- free (fim);
- free (fim_index);
- free (fmj);
- free (fmj_index);
-
- /* Tau. */
- double sum_fij2_ri = 0.0;
- double sum_fij2_ci = 0.0;
- FOR_EACH_POPULATED_ROW (i, xt)
- FOR_EACH_POPULATED_COLUMN (j, xt)
- {
- double temp = pow2 (xt->mat[j + i * n_cols]);
- sum_fij2_ri += temp / xt->row_tot[i];
- sum_fij2_ci += temp / xt->col_tot[j];
- }
-
- double sum_ri2 = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- sum_ri2 += pow2 (xt->row_tot[i]);
-
- double sum_cj2 = 0.0;
- for (size_t j = 0; j < n_cols; j++)
- sum_cj2 += pow2 (xt->col_tot[j]);
-
- v[3] = (xt->total * sum_fij2_ci - sum_ri2) / (pow2 (xt->total) - sum_ri2);
- v[4] = (xt->total * sum_fij2_ri - sum_cj2) / (pow2 (xt->total) - sum_cj2);
- }
-
- if (proc->statistics & CRS_ST_UC)
- {
- double UX = 0.0;
- FOR_EACH_POPULATED_ROW (i, xt)
- UX -= xt->row_tot[i] / xt->total * log (xt->row_tot[i] / xt->total);
-
- double UY = 0.0;
- FOR_EACH_POPULATED_COLUMN (j, xt)
- UY -= xt->col_tot[j] / xt->total * log (xt->col_tot[j] / xt->total);
-
- double UXY = 0.0;
- double P = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- for (size_t j = 0; j < n_cols; j++)
- {
- double entry = xt->mat[j + i * n_cols];
-
- if (entry <= 0.)
- continue;
-
- P += entry * pow2 (log (xt->col_tot[j] * xt->row_tot[i] / (xt->total * entry)));
- UXY -= entry / xt->total * log (entry / xt->total);
- }
-
- double ase1_yx = 0.0;
- double ase1_xy = 0.0;
- double ase1_sym = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- for (size_t j = 0; j < n_cols; j++)
- {
- double entry = xt->mat[j + i * n_cols];
-
- if (entry <= 0.)
- continue;
-
- ase1_yx += entry * pow2 (UY * log (entry / xt->row_tot[i])
- + (UX - UXY) * log (xt->col_tot[j] / xt->total));
- ase1_xy += entry * pow2 (UX * log (entry / xt->col_tot[j])
- + (UY - UXY) * log (xt->row_tot[i] / xt->total));
- ase1_sym += entry * pow2 ((UXY
- * log (xt->row_tot[i] * xt->col_tot[j] / pow2 (xt->total)))
- - (UX + UY) * log (entry / xt->total));
- }
-
- v[5] = 2. * ((UX + UY - UXY) / (UX + UY));
- ase[5] = (2. / (xt->total * pow2 (UX + UY))) * sqrt (ase1_sym);
- t[5] = SYSMIS;
-
- v[6] = (UX + UY - UXY) / UX;
- ase[6] = sqrt (ase1_xy) / (xt->total * UX * UX);
- t[6] = v[6] / (sqrt (P - xt->total * pow2 (UX + UY - UXY)) / (xt->total * UX));
-
- v[7] = (UX + UY - UXY) / UY;
- ase[7] = sqrt (ase1_yx) / (xt->total * UY * UY);
- t[7] = v[7] / (sqrt (P - xt->total * pow2 (UX + UY - UXY)) / (xt->total * UY));
- }
-
- /* Somers' D. */
- if (proc->statistics & CRS_ST_D)
- {
- double v_dummy[N_SYMMETRIC];
- double ase_dummy[N_SYMMETRIC];
- double t_dummy[N_SYMMETRIC];
- double somers_d_v[3];
- double somers_d_ase[3];
- double somers_d_t[3];
-
- if (calc_symmetric (proc, xt, v_dummy, ase_dummy, t_dummy,
- somers_d_v, somers_d_ase, somers_d_t))
- {
- for (size_t i = 0; i < 3; i++)
- {
- v[8 + i] = somers_d_v[i];
- ase[8 + i] = somers_d_ase[i];
- t[8 + i] = somers_d_t[i];
- sig[8 + i] = 2 * gsl_cdf_ugaussian_Q (fabs (somers_d_t[i]));
- }
- }
- }
-
- /* Eta. */
- if (proc->statistics & CRS_ST_ETA)
- {
- /* X dependent. */
- double sum_Xr = 0.0;
- double sum_X2r = 0.0;
- for (size_t i = 0; i < n_rows; i++)
- {
- sum_Xr += xt->vars[ROW_VAR].values[i].f * xt->row_tot[i];
- sum_X2r += pow2 (xt->vars[ROW_VAR].values[i].f) * xt->row_tot[i];
- }
- double SX = sum_X2r - pow2 (sum_Xr) / xt->total;
-
- double SXW = 0.0;
- FOR_EACH_POPULATED_COLUMN (j, xt)
- {
- double cum = 0.0;
-
- for (size_t i = 0; i < n_rows; i++)
- {
- SXW += (pow2 (xt->vars[ROW_VAR].values[i].f)
- * xt->mat[j + i * n_cols]);
- cum += (xt->vars[ROW_VAR].values[i].f
- * xt->mat[j + i * n_cols]);
- }
-
- SXW -= cum * cum / xt->col_tot[j];
- }
- v[11] = sqrt (1. - SXW / SX);
-
- /* Y dependent. */
- double sum_Yc = 0.0;
- double sum_Y2c = 0.0;
- for (size_t i = 0; i < n_cols; i++)
- {
- sum_Yc += xt->vars[COL_VAR].values[i].f * xt->col_tot[i];
- sum_Y2c += pow2 (xt->vars[COL_VAR].values[i].f) * xt->col_tot[i];
- }
- double SY = sum_Y2c - pow2 (sum_Yc) / xt->total;
-
- double SYW = 0.0;
- FOR_EACH_POPULATED_ROW (i, xt)
- {
- double cum = 0.0;
- for (size_t j = 0; j < n_cols; j++)
- {
- SYW += (pow2 (xt->vars[COL_VAR].values[j].f)
- * xt->mat[j + i * n_cols]);
- cum += (xt->vars[COL_VAR].values[j].f
- * xt->mat[j + i * n_cols]);
- }
-
- SYW -= cum * cum / xt->row_tot[i];
- }
- v[12] = sqrt (1. - SYW / SY);
- }
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2021 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <math.h>
-#include <errno.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/data-in.h"
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/mrset.h"
-#include "data/subcase.h"
-#include "data/value-labels.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/token.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmap.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/string-array.h"
-#include "math/mode.h"
-#include "math/moments.h"
-#include "math/percentiles.h"
-#include "math/sort.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-struct ctables;
-
-/* The three forms of weighting supported by CTABLES. */
-enum ctables_weighting
- {
- CTW_EFFECTIVE, /* Effective base weight (WEIGHT subcommand). */
- CTW_DICTIONARY, /* Dictionary weight. */
- CTW_UNWEIGHTED /* No weight. */
-#define N_CTWS 3
- };
-\f
-/* CTABLES table areas. */
-
-enum ctables_area_type
- {
- /* Within a section, where stacked variables divide one section from
- another.
-
- Keep CTAT_LAYER after CTAT_LAYERROW and CTAT_LAYERCOL so that
- parse_ctables_summary_function() parses correctly. */
- CTAT_TABLE, /* All layers of a whole section. */
- CTAT_LAYERROW, /* Row in one layer within a section. */
- CTAT_LAYERCOL, /* Column in one layer within a section. */
- CTAT_LAYER, /* One layer within a section. */
-
- /* Within a subtable, where a subtable pairs an innermost row variable with
- an innermost column variable within a single layer. */
- CTAT_SUBTABLE, /* Whole subtable. */
- CTAT_ROW, /* Row within a subtable. */
- CTAT_COL, /* Column within a subtable. */
-#define N_CTATS 7
- };
-
-static const char *ctables_area_type_name[N_CTATS] = {
- [CTAT_TABLE] = "TABLE",
- [CTAT_LAYER] = "LAYER",
- [CTAT_LAYERROW] = "LAYERROW",
- [CTAT_LAYERCOL] = "LAYERCOL",
- [CTAT_SUBTABLE] = "SUBTABLE",
- [CTAT_ROW] = "ROW",
- [CTAT_COL] = "COL",
-};
-
-/* Summary statistics for an area. */
-struct ctables_area
- {
- struct hmap_node node;
- const struct ctables_cell *example;
-
- /* Sequence number used for CTSF_ID. */
- size_t sequence;
-
- /* Weights for CTSF_areaPCT_COUNT, CTSF_areaPCT_VALIDN, and
- CTSF_areaPCT_TOTALN. */
- double count[N_CTWS];
- double valid[N_CTWS];
- double total[N_CTWS];
-
- /* Sums for CTSF_areaPCT_SUM. */
- struct ctables_sum *sums;
- };
-
-struct ctables_sum
- {
- double sum[N_CTWS];
- };
-\f
-/* CTABLES summary functions. */
-
-enum ctables_function_type
- {
- /* A function that operates on data in a single cell. It operates on
- effective weights. It does not have an unweighted version. */
- CTFT_CELL,
-
- /* A function that operates on data in a single cell. The function
- operates on effective weights and has a U-prefixed unweighted
- version. */
- CTFT_UCELL,
-
- /* A function that operates on data in a single cell. It operates on
- dictionary weights, and has U-prefixed unweighted version and an
- E-prefixed effective weight version. */
- CTFT_UECELL,
-
- /* A function that operates on an area of cells. It operates on effective
- weights and has a U-prefixed unweighted version. */
- CTFT_AREA,
- };
-
-enum ctables_format
- {
- CTF_COUNT, /* F40.0. */
- CTF_PERCENT, /* PCT40.1. */
- CTF_GENERAL /* Variable's print format. */
- };
-
-enum ctables_function_availability
- {
- CTFA_ALL, /* Any variables. */
- CTFA_SCALE, /* Only scale variables, totals, and subtotals. */
- //CTFA_MRSETS, /* Only multiple-response sets */
- };
-
-enum ctables_summary_function
- {
-#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) ENUM,
-#include "ctables.inc"
-#undef S
- };
-
-enum {
-#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) +1
- N_CTSF_FUNCTIONS =
-#include "ctables.inc"
-#undef S
-};
-
-struct ctables_function_info
- {
- struct substring basename;
- enum ctables_function_type type;
- enum ctables_format format;
- enum ctables_function_availability availability;
-
- bool u_prefix; /* Accepts a 'U' prefix (for unweighted)? */
- bool e_prefix; /* Accepts an 'E' prefix (for effective)? */
- bool is_area; /* Needs an area prefix. */
- };
-static const struct ctables_function_info ctables_function_info[N_CTSF_FUNCTIONS] = {
-#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) \
- [ENUM] = { \
- .basename = SS_LITERAL_INITIALIZER (NAME), \
- .type = TYPE, \
- .format = FORMAT, \
- .availability = AVAILABILITY, \
- .u_prefix = (TYPE) == CTFT_UCELL || (TYPE) == CTFT_UECELL || (TYPE) == CTFT_AREA, \
- .e_prefix = (TYPE) == CTFT_UECELL, \
- .is_area = (TYPE) == CTFT_AREA \
- },
-#include "ctables.inc"
-#undef S
-};
-
-static struct fmt_spec
-ctables_summary_default_format (enum ctables_summary_function function,
- const struct variable *var)
-{
- static const enum ctables_format default_formats[] = {
-#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) [ENUM] = FORMAT,
-#include "ctables.inc"
-#undef S
- };
- switch (default_formats[function])
- {
- case CTF_COUNT:
- return (struct fmt_spec) { .type = FMT_F, .w = 40 };
-
- case CTF_PERCENT:
- return (struct fmt_spec) { .type = FMT_PCT, .w = 40, .d = 1 };
-
- case CTF_GENERAL:
- return *var_get_print_format (var);
-
- default:
- NOT_REACHED ();
- }
-}
-
-static enum ctables_function_availability
-ctables_function_availability (enum ctables_summary_function f)
-{
- static enum ctables_function_availability availability[] = {
-#define S(ENUM, NAME, TYPE, FORMAT, AVAILABILITY) [ENUM] = AVAILABILITY,
-#include "ctables.inc"
-#undef S
- };
-
- return availability[f];
-}
-
-static bool
-parse_ctables_summary_function (struct lexer *lexer,
- enum ctables_summary_function *function,
- enum ctables_weighting *weighting,
- enum ctables_area_type *area)
-{
- if (!lex_force_id (lexer))
- return false;
-
- struct substring name = lex_tokss (lexer);
- if (ss_ends_with_case (name, ss_cstr (".LCL"))
- || ss_ends_with_case (name, ss_cstr (".UCL"))
- || ss_ends_with_case (name, ss_cstr (".SE")))
- {
- lex_error (lexer, _("Support for LCL, UCL, and SE summary functions "
- "is not yet implemented."));
- return false;
- }
-
- bool u = ss_match_byte (&name, 'U') || ss_match_byte (&name, 'u');
- bool e = !u && (ss_match_byte (&name, 'E') || ss_match_byte (&name, 'e'));
-
- bool has_area = false;
- *area = 0;
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- if (ss_match_string_case (&name, ss_cstr (ctables_area_type_name[at])))
- {
- has_area = true;
- *area = at;
-
- if (ss_equals_case (name, ss_cstr ("PCT")))
- {
- /* Special case where .COUNT suffix is omitted. */
- *function = CTSF_areaPCT_COUNT;
- *weighting = CTW_EFFECTIVE;
- lex_get (lexer);
- return true;
- }
- break;
- }
-
- for (int f = 0; f < N_CTSF_FUNCTIONS; f++)
- {
- const struct ctables_function_info *cfi = &ctables_function_info[f];
- if (ss_equals_case (cfi->basename, name))
- {
- *function = f;
- if ((u && !cfi->u_prefix) || (e && !cfi->e_prefix) || (has_area != cfi->is_area))
- break;
-
- *weighting = (e ? CTW_EFFECTIVE
- : u ? CTW_UNWEIGHTED
- : cfi->e_prefix ? CTW_DICTIONARY
- : CTW_EFFECTIVE);
- lex_get (lexer);
- return true;
- }
- }
-
- lex_error (lexer, _("Syntax error expecting summary function name."));
- return false;
-}
-
-static const char *
-ctables_summary_function_name (enum ctables_summary_function function,
- enum ctables_weighting weighting,
- enum ctables_area_type area,
- char *buffer, size_t bufsize)
-{
- const struct ctables_function_info *cfi = &ctables_function_info[function];
- snprintf (buffer, bufsize, "%s%s%s",
- (weighting == CTW_UNWEIGHTED ? "U"
- : weighting == CTW_DICTIONARY ? ""
- : cfi->e_prefix ? "E"
- : ""),
- cfi->is_area ? ctables_area_type_name[area] : "",
- cfi->basename.string);
- return buffer;
-}
-
-static const char *
-ctables_summary_function_label__ (enum ctables_summary_function function,
- enum ctables_weighting weighting,
- enum ctables_area_type area)
-{
- bool w = weighting != CTW_UNWEIGHTED;
- bool d = weighting == CTW_DICTIONARY;
- enum ctables_area_type a = area;
- switch (function)
- {
- case CTSF_COUNT:
- return d ? N_("Count") : w ? N_("Adjusted Count") : N_("Unweighted Count");
-
- case CTSF_areaPCT_COUNT:
- switch (a)
- {
- case CTAT_TABLE: return w ? N_("Table %") : N_("Unweighted Table %");
- case CTAT_LAYER: return w ? N_("Layer %") : N_("Unweighted Layer %");
- case CTAT_LAYERROW: return w ? N_("Layer Row %") : N_("Unweighted Layer Row %");
- case CTAT_LAYERCOL: return w ? N_("Layer Column %") : N_("Unweighted Layer Column %");
- case CTAT_SUBTABLE: return w ? N_("Subtable %") : N_("Unweighted Subtable %");
- case CTAT_ROW: return w ? N_("Row %") : N_("Unweighted Row %");
- case CTAT_COL: return w ? N_("Column %") : N_("Unweighted Column %");
- }
- NOT_REACHED ();
-
- case CTSF_areaPCT_VALIDN:
- switch (a)
- {
- case CTAT_TABLE: return w ? N_("Table Valid N %") : N_("Unweighted Table Valid N %");
- case CTAT_LAYER: return w ? N_("Layer Valid N %") : N_("Unweighted Layer Valid N %");
- case CTAT_LAYERROW: return w ? N_("Layer Row Valid N %") : N_("Unweighted Layer Row Valid N %");
- case CTAT_LAYERCOL: return w ? N_("Layer Column Valid N %") : N_("Unweighted Layer Column Valid N %");
- case CTAT_SUBTABLE: return w ? N_("Subtable Valid N %") : N_("Unweighted Subtable Valid N %");
- case CTAT_ROW: return w ? N_("Row Valid N %") : N_("Unweighted Row Valid N %");
- case CTAT_COL: return w ? N_("Column Valid N %") : N_("Unweighted Column Valid N %");
- }
- NOT_REACHED ();
-
- case CTSF_areaPCT_TOTALN:
- switch (a)
- {
- case CTAT_TABLE: return w ? N_("Table Total N %") : N_("Unweighted Table Total N %");
- case CTAT_LAYER: return w ? N_("Layer Total N %") : N_("Unweighted Layer Total N %");
- case CTAT_LAYERROW: return w ? N_("Layer Row Total N %") : N_("Unweighted Layer Row Total N %");
- case CTAT_LAYERCOL: return w ? N_("Layer Column Total N %") : N_("Unweighted Layer Column Total N %");
- case CTAT_SUBTABLE: return w ? N_("Subtable Total N %") : N_("Unweighted Subtable Total N %");
- case CTAT_ROW: return w ? N_("Row Total N %") : N_("Unweighted Row Total N %");
- case CTAT_COL: return w ? N_("Column Total N %") : N_("Unweighted Column Total N %");
- }
- NOT_REACHED ();
-
- case CTSF_MAXIMUM: return N_("Maximum");
- case CTSF_MEAN: return w ? N_("Mean") : N_("Unweighted Mean");
- case CTSF_MEDIAN: return w ? N_("Median") : N_("Unweighted Median");
- case CTSF_MINIMUM: return N_("Minimum");
- case CTSF_MISSING: return w ? N_("Missing") : N_("Unweighted Missing");
- case CTSF_MODE: return w ? N_("Mode") : N_("Unweighted Mode");
- case CTSF_PTILE: NOT_REACHED ();
- case CTSF_RANGE: return N_("Range");
- case CTSF_SEMEAN: return w ? N_("Std Error of Mean") : N_("Unweighted Std Error of Mean");
- case CTSF_STDDEV: return w ? N_("Std Deviation") : N_("Unweighted Std Deviation");
- case CTSF_SUM: return w ? N_("Sum") : N_("Unweighted Sum");
- case CTSF_TOTALN: return (d ? N_("Total N")
- : w ? N_("Adjusted Total N")
- : N_("Unweighted Total N"));
- case CTSF_VALIDN: return (d ? N_("Valid N")
- : w ? N_("Adjusted Valid N")
- : N_("Unweighted Valid N"));
- case CTSF_VARIANCE: return w ? N_("Variance") : N_("Unweighted Variance");
- case CTSF_areaPCT_SUM:
- switch (a)
- {
- case CTAT_TABLE: return w ? N_("Table Sum %") : N_("Unweighted Table Sum %");
- case CTAT_LAYER: return w ? N_("Layer Sum %") : N_("Unweighted Layer Sum %");
- case CTAT_LAYERROW: return w ? N_("Layer Row Sum %") : N_("Unweighted Layer Row Sum %");
- case CTAT_LAYERCOL: return w ? N_("Layer Column Sum %") : N_("Unweighted Layer Column Sum %");
- case CTAT_SUBTABLE: return w ? N_("Subtable Sum %") : N_("Unweighted Subtable Sum %");
- case CTAT_ROW: return w ? N_("Row Sum %") : N_("Unweighted Row Sum %");
- case CTAT_COL: return w ? N_("Column Sum %") : N_("Unweighted Column Sum %");
- }
- NOT_REACHED ();
-
- case CTSF_areaID:
- switch (a)
- {
- /* Don't bother translating these: they are for developers only. */
- case CTAT_TABLE: return "Table ID";
- case CTAT_LAYER: return "Layer ID";
- case CTAT_LAYERROW: return "Layer Row ID";
- case CTAT_LAYERCOL: return "Layer Column ID";
- case CTAT_SUBTABLE: return "Subtable ID";
- case CTAT_ROW: return "Row ID";
- case CTAT_COL: return "Column ID";
- }
- NOT_REACHED ();
- }
-
- NOT_REACHED ();
-}
-
-static struct pivot_value *
-ctables_summary_function_label (enum ctables_summary_function function,
- enum ctables_weighting weighting,
- enum ctables_area_type area,
- double percentile)
-{
- if (function == CTSF_PTILE)
- {
- char *s = (weighting != CTW_UNWEIGHTED
- ? xasprintf (_("Percentile %.2f"), percentile)
- : xasprintf (_("Unweighted Percentile %.2f"), percentile));
- return pivot_value_new_user_text_nocopy (s);
- }
- else
- return pivot_value_new_text (ctables_summary_function_label__ (
- function, weighting, area));
-}
-\f
-/* CTABLES summaries. */
-
-struct ctables_summary_spec
- {
- /* The calculation to be performed.
-
- 'function' is the function to calculate. 'weighted' specifies whether
- to use weighted or unweighted data (for functions that do not support a
- choice, it must be true). 'calc_area' is the area over which the
- calculation takes place (for functions that target only an individual
- cell, it must be 0). For CTSF_PTILE only, 'percentile' is the
- percentile between 0 and 100 (for other functions it must be 0). */
- enum ctables_summary_function function;
- enum ctables_weighting weighting;
- enum ctables_area_type calc_area;
- double percentile; /* CTSF_PTILE only. */
-
- /* How to display the result of the calculation.
-
- 'label' is a user-specified label, NULL if the user didn't specify
- one.
-
- 'user_area' is usually the same as 'calc_area', but when category labels
- are rotated from one axis to another it swaps rows and columns.
-
- 'format' is the format for displaying the output. If
- 'is_ctables_format' is true, then 'format.type' is one of the special
- CTEF_* formats instead of the standard ones. */
- char *label;
- enum ctables_area_type user_area;
- struct fmt_spec format;
- bool is_ctables_format; /* Is 'format' one of CTEF_*? */
-
- size_t axis_idx; /* Leaf index if summary dimension in use. */
- size_t sum_var_idx; /* Offset into 'sums' in ctables_area. */
- };
-
-static void
-ctables_summary_spec_clone (struct ctables_summary_spec *dst,
- const struct ctables_summary_spec *src)
-{
- *dst = *src;
- dst->label = xstrdup_if_nonnull (src->label);
-}
-
-static void
-ctables_summary_spec_uninit (struct ctables_summary_spec *s)
-{
- if (s)
- free (s->label);
-}
-\f
-/* Collections of summary functions. */
-
-struct ctables_summary_spec_set
- {
- struct ctables_summary_spec *specs;
- size_t n;
- size_t allocated;
-
- /* The variable to which the summary specs are applied. */
- struct variable *var;
-
- /* Whether the variable to which the summary specs are applied is a scale
- variable for the purpose of summarization.
-
- (VALIDN and TOTALN act differently for summarizing scale and categorical
- variables.) */
- bool is_scale;
-
- /* If any of these optional additional scale variables are missing, then
- treat 'var' as if it's missing too. This is for implementing
- SMISSING=LISTWISE. */
- struct variable **listwise_vars;
- size_t n_listwise_vars;
- };
-
-static void
-ctables_summary_spec_set_clone (struct ctables_summary_spec_set *dst,
- const struct ctables_summary_spec_set *src)
-{
- struct ctables_summary_spec *specs
- = (src->n ? xnmalloc (src->n, sizeof *specs) : NULL);
- for (size_t i = 0; i < src->n; i++)
- ctables_summary_spec_clone (&specs[i], &src->specs[i]);
-
- *dst = (struct ctables_summary_spec_set) {
- .specs = specs,
- .n = src->n,
- .allocated = src->n,
- .var = src->var,
- .is_scale = src->is_scale,
- };
-}
-
-static void
-ctables_summary_spec_set_uninit (struct ctables_summary_spec_set *set)
-{
- for (size_t i = 0; i < set->n; i++)
- ctables_summary_spec_uninit (&set->specs[i]);
- free (set->listwise_vars);
- free (set->specs);
-}
-
-static bool
-is_listwise_missing (const struct ctables_summary_spec_set *specs,
- const struct ccase *c)
-{
- for (size_t i = 0; i < specs->n_listwise_vars; i++)
- {
- const struct variable *var = specs->listwise_vars[i];
- if (var_is_num_missing (var, case_num (c, var)))
- return true;
- }
-
- return false;
-}
-\f
-/* CTABLES postcompute expressions. */
-
-struct ctables_postcompute
- {
- struct hmap_node hmap_node; /* In struct ctables's 'pcompute' hmap. */
- char *name; /* Name, without leading &. */
-
- struct msg_location *location; /* Location of definition. */
- struct ctables_pcexpr *expr;
- char *label;
- struct ctables_summary_spec_set *specs;
- bool hide_source_cats;
- };
-
-struct ctables_pcexpr
- {
- /* Precedence table:
-
- ()
- **
- -
- * /
- - +
- */
- enum ctables_pcexpr_op
- {
- /* Terminals. */
- CTPO_CONSTANT, /* 5 */
- CTPO_CAT_NUMBER, /* [5] */
- CTPO_CAT_STRING, /* ["STRING"] */
- CTPO_CAT_NRANGE, /* [LO THRU 5] */
- CTPO_CAT_SRANGE, /* ["A" THRU "B"] */
- CTPO_CAT_MISSING, /* MISSING */
- CTPO_CAT_OTHERNM, /* OTHERNM */
- CTPO_CAT_SUBTOTAL, /* SUBTOTAL */
- CTPO_CAT_TOTAL, /* TOTAL */
-
- /* Nonterminals. */
- CTPO_ADD,
- CTPO_SUB,
- CTPO_MUL,
- CTPO_DIV,
- CTPO_POW,
- CTPO_NEG,
- }
- op;
-
- union
- {
- /* CTPO_CAT_NUMBER. */
- double number;
-
- /* CTPO_CAT_STRING, in dictionary encoding. */
- struct substring string;
-
- /* CTPO_CAT_NRANGE. */
- double nrange[2];
-
- /* CTPO_CAT_SRANGE. */
- struct substring srange[2];
-
- /* CTPO_CAT_SUBTOTAL. */
- size_t subtotal_index;
-
- /* Two elements: CTPO_ADD, CTPO_SUB, CTPO_MUL, CTPO_DIV, CTPO_POW.
- One element: CTPO_NEG. */
- struct ctables_pcexpr *subs[2];
- };
-
- /* Source location. */
- struct msg_location *location;
- };
-
-static struct ctables_postcompute *ctables_find_postcompute (struct ctables *,
- const char *name);
-
-static struct ctables_pcexpr *ctables_pcexpr_allocate_binary (
- enum ctables_pcexpr_op, struct ctables_pcexpr *sub0,
- struct ctables_pcexpr *sub1);
-
-typedef struct ctables_pcexpr *parse_recursively_func (struct lexer *,
- struct dictionary *);
-
-static void
-ctables_pcexpr_destroy (struct ctables_pcexpr *e)
-{
- if (e)
- {
- switch (e->op)
- {
- case CTPO_CAT_STRING:
- ss_dealloc (&e->string);
- break;
-
- case CTPO_CAT_SRANGE:
- for (size_t i = 0; i < 2; i++)
- ss_dealloc (&e->srange[i]);
- break;
-
- case CTPO_ADD:
- case CTPO_SUB:
- case CTPO_MUL:
- case CTPO_DIV:
- case CTPO_POW:
- case CTPO_NEG:
- for (size_t i = 0; i < 2; i++)
- ctables_pcexpr_destroy (e->subs[i]);
- break;
-
- case CTPO_CONSTANT:
- case CTPO_CAT_NUMBER:
- case CTPO_CAT_NRANGE:
- case CTPO_CAT_MISSING:
- case CTPO_CAT_OTHERNM:
- case CTPO_CAT_SUBTOTAL:
- case CTPO_CAT_TOTAL:
- break;
- }
-
- msg_location_destroy (e->location);
- free (e);
- }
-}
-
-static struct ctables_pcexpr *
-ctables_pcexpr_allocate_binary (enum ctables_pcexpr_op op,
- struct ctables_pcexpr *sub0,
- struct ctables_pcexpr *sub1)
-{
- struct ctables_pcexpr *e = xmalloc (sizeof *e);
- *e = (struct ctables_pcexpr) {
- .op = op,
- .subs = { sub0, sub1 },
- .location = msg_location_merged (sub0->location, sub1->location),
- };
- return e;
-}
-
-/* How to parse an operator. */
-struct operator
- {
- enum token_type token;
- enum ctables_pcexpr_op op;
- };
-
-static const struct operator *
-ctables_pcexpr_match_operator (struct lexer *lexer,
- const struct operator ops[], size_t n_ops)
-{
- for (const struct operator *op = ops; op < ops + n_ops; op++)
- if (lex_token (lexer) == op->token)
- {
- if (op->token != T_NEG_NUM)
- lex_get (lexer);
-
- return op;
- }
-
- return NULL;
-}
-
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_binary_operators__ (
- struct lexer *lexer, struct dictionary *dict,
- const struct operator ops[], size_t n_ops,
- parse_recursively_func *parse_next_level,
- const char *chain_warning, struct ctables_pcexpr *lhs)
-{
- for (int op_count = 0; ; op_count++)
- {
- const struct operator *op
- = ctables_pcexpr_match_operator (lexer, ops, n_ops);
- if (!op)
- {
- if (op_count > 1 && chain_warning)
- msg_at (SW, lhs->location, "%s", chain_warning);
-
- return lhs;
- }
-
- struct ctables_pcexpr *rhs = parse_next_level (lexer, dict);
- if (!rhs)
- {
- ctables_pcexpr_destroy (lhs);
- return NULL;
- }
-
- lhs = ctables_pcexpr_allocate_binary (op->op, lhs, rhs);
- }
-}
-
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_binary_operators (
- struct lexer *lexer, struct dictionary *dict,
- const struct operator ops[], size_t n_ops,
- parse_recursively_func *parse_next_level, const char *chain_warning)
-{
- struct ctables_pcexpr *lhs = parse_next_level (lexer, dict);
- if (!lhs)
- return NULL;
-
- return ctables_pcexpr_parse_binary_operators__ (lexer, dict, ops, n_ops,
- parse_next_level,
- chain_warning, lhs);
-}
-
-static struct ctables_pcexpr *ctables_pcexpr_parse_add (struct lexer *,
- struct dictionary *);
-
-static struct ctables_pcexpr
-ctpo_cat_nrange (double low, double high)
-{
- return (struct ctables_pcexpr) {
- .op = CTPO_CAT_NRANGE,
- .nrange = { low, high },
- };
-}
-
-static struct ctables_pcexpr
-ctpo_cat_srange (struct substring low, struct substring high)
-{
- return (struct ctables_pcexpr) {
- .op = CTPO_CAT_SRANGE,
- .srange = { low, high },
- };
-}
-
-static struct substring
-parse_substring (struct lexer *lexer, struct dictionary *dict)
-{
- struct substring s = recode_substring_pool (
- dict_get_encoding (dict), "UTF-8", lex_tokss (lexer), NULL);
- ss_rtrim (&s, ss_cstr (" "));
- lex_get (lexer);
- return s;
-}
-
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_primary (struct lexer *lexer, struct dictionary *dict)
-{
- int start_ofs = lex_ofs (lexer);
- struct ctables_pcexpr e;
- if (lex_is_number (lexer))
- {
- e = (struct ctables_pcexpr) { .op = CTPO_CONSTANT,
- .number = lex_number (lexer) };
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "MISSING"))
- e = (struct ctables_pcexpr) { .op = CTPO_CAT_MISSING };
- else if (lex_match_id (lexer, "OTHERNM"))
- e = (struct ctables_pcexpr) { .op = CTPO_CAT_OTHERNM };
- else if (lex_match_id (lexer, "TOTAL"))
- e = (struct ctables_pcexpr) { .op = CTPO_CAT_TOTAL };
- else if (lex_match_id (lexer, "SUBTOTAL"))
- {
- size_t subtotal_index = 0;
- if (lex_match (lexer, T_LBRACK))
- {
- if (!lex_force_int_range (lexer, "SUBTOTAL", 1, LONG_MAX))
- return NULL;
- subtotal_index = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RBRACK))
- return NULL;
- }
- e = (struct ctables_pcexpr) { .op = CTPO_CAT_SUBTOTAL,
- .subtotal_index = subtotal_index };
- }
- else if (lex_match (lexer, T_LBRACK))
- {
- if (lex_match_id (lexer, "LO"))
- {
- if (!lex_force_match_id (lexer, "THRU"))
- return false;
-
- if (lex_is_string (lexer))
- {
- struct substring low = { .string = NULL };
- struct substring high = parse_substring (lexer, dict);
- e = ctpo_cat_srange (low, high);
- }
- else
- {
- if (!lex_force_num (lexer))
- return false;
- e = ctpo_cat_nrange (-DBL_MAX, lex_number (lexer));
- lex_get (lexer);
- }
- }
- else if (lex_is_number (lexer))
- {
- double number = lex_number (lexer);
- lex_get (lexer);
- if (lex_match_id (lexer, "THRU"))
- {
- if (lex_match_id (lexer, "HI"))
- e = ctpo_cat_nrange (number, DBL_MAX);
- else
- {
- if (!lex_force_num (lexer))
- return false;
- e = ctpo_cat_nrange (number, lex_number (lexer));
- lex_get (lexer);
- }
- }
- else
- e = (struct ctables_pcexpr) { .op = CTPO_CAT_NUMBER,
- .number = number };
- }
- else if (lex_is_string (lexer))
- {
- struct substring s = parse_substring (lexer, dict);
-
- if (lex_match_id (lexer, "THRU"))
- {
- struct substring high;
-
- if (lex_match_id (lexer, "HI"))
- high = (struct substring) { .string = NULL };
- else
- {
- if (!lex_force_string (lexer))
- {
- ss_dealloc (&s);
- return false;
- }
- high = parse_substring (lexer, dict);
- }
-
- e = ctpo_cat_srange (s, high);
- }
- else
- e = (struct ctables_pcexpr) { .op = CTPO_CAT_STRING, .string = s };
- }
- else
- {
- lex_error (lexer,
- _("Syntax error expecting number or string or range."));
- return NULL;
- }
-
- if (!lex_force_match (lexer, T_RBRACK))
- {
- if (e.op == CTPO_CAT_STRING)
- ss_dealloc (&e.string);
- else if (e.op == CTPO_CAT_SRANGE)
- {
- ss_dealloc (&e.srange[0]);
- ss_dealloc (&e.srange[1]);
- }
- return NULL;
- }
- }
- else if (lex_match (lexer, T_LPAREN))
- {
- struct ctables_pcexpr *ep = ctables_pcexpr_parse_add (lexer, dict);
- if (!ep)
- return NULL;
- if (!lex_force_match (lexer, T_RPAREN))
- {
- ctables_pcexpr_destroy (ep);
- return NULL;
- }
- return ep;
- }
- else
- {
- lex_error (lexer, _("Syntax error in postcompute expression."));
- return NULL;
- }
-
- e.location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1);
- return xmemdup (&e, sizeof e);
-}
-
-static struct ctables_pcexpr *
-ctables_pcexpr_allocate_neg (struct ctables_pcexpr *sub,
- struct lexer *lexer, int start_ofs)
-{
- struct ctables_pcexpr *e = xmalloc (sizeof *e);
- *e = (struct ctables_pcexpr) {
- .op = CTPO_NEG,
- .subs = { sub },
- .location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1),
- };
- return e;
-}
-
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_exp (struct lexer *lexer, struct dictionary *dict)
-{
- static const struct operator op = { T_EXP, CTPO_POW };
-
- const char *chain_warning =
- _("The exponentiation operator (`**') is left-associative: "
- "`a**b**c' equals `(a**b)**c', not `a**(b**c)'. "
- "To disable this warning, insert parentheses.");
-
- if (lex_token (lexer) != T_NEG_NUM || lex_next_token (lexer, 1) != T_EXP)
- return ctables_pcexpr_parse_binary_operators (lexer, dict, &op, 1,
- ctables_pcexpr_parse_primary,
- chain_warning);
-
- /* Special case for situations like "-5**6", which must be parsed as
- -(5**6). */
-
- int start_ofs = lex_ofs (lexer);
- struct ctables_pcexpr *lhs = xmalloc (sizeof *lhs);
- *lhs = (struct ctables_pcexpr) {
- .op = CTPO_CONSTANT,
- .number = -lex_tokval (lexer),
- .location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer)),
- };
- lex_get (lexer);
-
- struct ctables_pcexpr *node = ctables_pcexpr_parse_binary_operators__ (
- lexer, dict, &op, 1,
- ctables_pcexpr_parse_primary, chain_warning, lhs);
- if (!node)
- return NULL;
-
- return ctables_pcexpr_allocate_neg (node, lexer, start_ofs);
-}
-
-/* Parses the unary minus level. */
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_neg (struct lexer *lexer, struct dictionary *dict)
-{
- int start_ofs = lex_ofs (lexer);
- if (!lex_match (lexer, T_DASH))
- return ctables_pcexpr_parse_exp (lexer, dict);
-
- struct ctables_pcexpr *inner = ctables_pcexpr_parse_neg (lexer, dict);
- if (!inner)
- return NULL;
-
- return ctables_pcexpr_allocate_neg (inner, lexer, start_ofs);
-}
-
-/* Parses the multiplication and division level. */
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_mul (struct lexer *lexer, struct dictionary *dict)
-{
- static const struct operator ops[] =
- {
- { T_ASTERISK, CTPO_MUL },
- { T_SLASH, CTPO_DIV },
- };
-
- return ctables_pcexpr_parse_binary_operators (lexer, dict, ops,
- sizeof ops / sizeof *ops,
- ctables_pcexpr_parse_neg, NULL);
-}
-
-/* Parses the addition and subtraction level. */
-static struct ctables_pcexpr *
-ctables_pcexpr_parse_add (struct lexer *lexer, struct dictionary *dict)
-{
- static const struct operator ops[] =
- {
- { T_PLUS, CTPO_ADD },
- { T_DASH, CTPO_SUB },
- { T_NEG_NUM, CTPO_ADD },
- };
-
- return ctables_pcexpr_parse_binary_operators (lexer, dict,
- ops, sizeof ops / sizeof *ops,
- ctables_pcexpr_parse_mul, NULL);
-}
-\f
-/* CTABLES axis expressions. */
-
-/* CTABLES has a number of extra formats that we implement via custom
- currency specifications on an alternate fmt_settings. */
-#define CTEF_NEGPAREN FMT_CCA
-#define CTEF_NEQUAL FMT_CCB
-#define CTEF_PAREN FMT_CCC
-#define CTEF_PCTPAREN FMT_CCD
-
-enum ctables_summary_variant
- {
- CSV_CELL,
- CSV_TOTAL
-#define N_CSVS 2
- };
-
-struct ctables_axis
- {
- enum ctables_axis_op
- {
- /* Terminals. */
- CTAO_VAR,
-
- /* Nonterminals. */
- CTAO_STACK, /* + */
- CTAO_NEST, /* > */
- }
- op;
-
- union
- {
- /* Terminals. */
- struct
- {
- struct variable *var;
- bool scale;
- struct ctables_summary_spec_set specs[N_CSVS];
- };
-
- /* Nonterminals. */
- struct ctables_axis *subs[2];
- };
-
- struct msg_location *loc;
- };
-
-static void
-ctables_axis_destroy (struct ctables_axis *axis)
-{
- if (!axis)
- return;
-
- switch (axis->op)
- {
- case CTAO_VAR:
- for (size_t i = 0; i < N_CSVS; i++)
- ctables_summary_spec_set_uninit (&axis->specs[i]);
- break;
-
- case CTAO_STACK:
- case CTAO_NEST:
- ctables_axis_destroy (axis->subs[0]);
- ctables_axis_destroy (axis->subs[1]);
- break;
- }
- msg_location_destroy (axis->loc);
- free (axis);
-}
-
-static struct ctables_axis *
-ctables_axis_new_nonterminal (enum ctables_axis_op op,
- struct ctables_axis *sub0,
- struct ctables_axis *sub1,
- struct lexer *lexer, int start_ofs)
-{
- struct ctables_axis *axis = xmalloc (sizeof *axis);
- *axis = (struct ctables_axis) {
- .op = op,
- .subs = { sub0, sub1 },
- .loc = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1),
- };
- return axis;
-}
-
-struct ctables_axis_parse_ctx
- {
- struct lexer *lexer;
- struct dictionary *dict;
- };
-
-static struct pivot_value *
-ctables_summary_label (const struct ctables_summary_spec *spec, double cilevel)
-{
- if (!spec->label)
- return ctables_summary_function_label (spec->function, spec->weighting,
- spec->user_area, spec->percentile);
- else
- {
- struct substring in = ss_cstr (spec->label);
- struct substring target = ss_cstr (")CILEVEL");
-
- struct string out = DS_EMPTY_INITIALIZER;
- for (;;)
- {
- size_t chunk = ss_find_substring (in, target);
- ds_put_substring (&out, ss_head (in, chunk));
- ss_advance (&in, chunk);
- if (!in.length)
- return pivot_value_new_user_text_nocopy (ds_steal_cstr (&out));
-
- ss_advance (&in, target.length);
- ds_put_format (&out, "%g", cilevel);
- }
- }
-}
-
-static bool
-add_summary_spec (struct ctables_axis *axis,
- enum ctables_summary_function function,
- enum ctables_weighting weighting,
- enum ctables_area_type area, double percentile,
- const char *label, const struct fmt_spec *format,
- bool is_ctables_format, const struct msg_location *loc,
- enum ctables_summary_variant sv)
-{
- if (axis->op == CTAO_VAR)
- {
- char function_name[128];
- ctables_summary_function_name (function, weighting, area,
- function_name, sizeof function_name);
- const char *var_name = var_get_name (axis->var);
- switch (ctables_function_availability (function))
- {
-#if 0
- case CTFA_MRSETS:
- msg_at (SE, loc, _("Summary function %s applies only to multiple "
- "response sets."), function_name);
- msg_at (SN, axis->loc, _("'%s' is not a multiple response set."),
- var_name);
- return false;
-#endif
-
- case CTFA_SCALE:
- if (!axis->scale && sv != CSV_TOTAL)
- {
- msg_at (SE, loc,
- _("Summary function %s applies only to scale variables."),
- function_name);
- msg_at (SN, axis->loc, _("'%s' is not a scale variable."),
- var_name);
- return false;
- }
- break;
-
- case CTFA_ALL:
- break;
- }
-
- struct ctables_summary_spec_set *set = &axis->specs[sv];
- if (set->n >= set->allocated)
- set->specs = x2nrealloc (set->specs, &set->allocated,
- sizeof *set->specs);
-
- struct ctables_summary_spec *dst = &set->specs[set->n++];
- *dst = (struct ctables_summary_spec) {
- .function = function,
- .weighting = weighting,
- .calc_area = area,
- .user_area = area,
- .percentile = percentile,
- .label = xstrdup_if_nonnull (label),
- .format = (format ? *format
- : ctables_summary_default_format (function, axis->var)),
- .is_ctables_format = is_ctables_format,
- };
- return true;
- }
- else
- {
- for (size_t i = 0; i < 2; i++)
- if (!add_summary_spec (axis->subs[i], function, weighting, area,
- percentile, label, format, is_ctables_format,
- loc, sv))
- return false;
- return true;
- }
-}
-
-static struct ctables_axis *ctables_axis_parse_stack (
- struct ctables_axis_parse_ctx *);
-
-static struct ctables_axis *
-ctables_axis_parse_primary (struct ctables_axis_parse_ctx *ctx)
-{
- if (lex_match (ctx->lexer, T_LPAREN))
- {
- struct ctables_axis *sub = ctables_axis_parse_stack (ctx);
- if (!sub || !lex_force_match (ctx->lexer, T_RPAREN))
- {
- ctables_axis_destroy (sub);
- return NULL;
- }
- return sub;
- }
-
- if (!lex_force_id (ctx->lexer))
- return NULL;
-
- if (lex_tokcstr (ctx->lexer)[0] == '$')
- {
- lex_error (ctx->lexer,
- _("Multiple response set support not implemented."));
- return NULL;
- }
-
- int start_ofs = lex_ofs (ctx->lexer);
- struct variable *var = parse_variable (ctx->lexer, ctx->dict);
- if (!var)
- return NULL;
-
- struct ctables_axis *axis = xmalloc (sizeof *axis);
- *axis = (struct ctables_axis) { .op = CTAO_VAR, .var = var };
-
- axis->scale = (lex_match_phrase (ctx->lexer, "[S]") ? true
- : lex_match_phrase (ctx->lexer, "[C]") ? false
- : var_get_measure (var) == MEASURE_SCALE);
- axis->loc = lex_ofs_location (ctx->lexer, start_ofs,
- lex_ofs (ctx->lexer) - 1);
- if (axis->scale && var_is_alpha (var))
- {
- msg_at (SE, axis->loc, _("Cannot use string variable %s as a scale "
- "variable."),
- var_get_name (var));
- ctables_axis_destroy (axis);
- return NULL;
- }
-
- return axis;
-}
-
-static bool
-has_digit (const char *s)
-{
- return s[strcspn (s, "0123456789")] != '\0';
-}
-
-static bool
-parse_ctables_format_specifier (struct lexer *lexer, struct fmt_spec *format,
- bool *is_ctables_format)
-{
- char type[FMT_TYPE_LEN_MAX + 1];
- if (!parse_abstract_format_specifier__ (lexer, type, &format->w, &format->d))
- return false;
-
- if (!strcasecmp (type, "NEGPAREN"))
- format->type = CTEF_NEGPAREN;
- else if (!strcasecmp (type, "NEQUAL"))
- format->type = CTEF_NEQUAL;
- else if (!strcasecmp (type, "PAREN"))
- format->type = CTEF_PAREN;
- else if (!strcasecmp (type, "PCTPAREN"))
- format->type = CTEF_PCTPAREN;
- else
- {
- *is_ctables_format = false;
- if (!parse_format_specifier (lexer, format))
- return false;
-
- char *error = fmt_check_output__ (format);
- if (!error)
- error = fmt_check_type_compat__ (format, NULL, VAL_NUMERIC);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- return false;
- }
-
- return true;
- }
-
- lex_get (lexer);
- if (format->w < 2)
- {
- lex_next_error (lexer, -1, -1,
- _("Output format %s requires width 2 or greater."), type);
- return false;
- }
- else if (format->d > format->w - 1)
- {
- lex_next_error (lexer, -1, -1, _("Output format %s requires width "
- "greater than decimals."), type);
- return false;
- }
- else
- {
- *is_ctables_format = true;
- return true;
- }
-}
-
-static struct ctables_axis *
-ctables_axis_parse_postfix (struct ctables_axis_parse_ctx *ctx)
-{
- struct ctables_axis *sub = ctables_axis_parse_primary (ctx);
- if (!sub || !lex_match (ctx->lexer, T_LBRACK))
- return sub;
-
- enum ctables_summary_variant sv = CSV_CELL;
- for (;;)
- {
- int start_ofs = lex_ofs (ctx->lexer);
-
- /* Parse function. */
- enum ctables_summary_function function;
- enum ctables_weighting weighting;
- enum ctables_area_type area;
- if (!parse_ctables_summary_function (ctx->lexer, &function, &weighting,
- &area))
- goto error;
-
- /* Parse percentile. */
- double percentile = 0;
- if (function == CTSF_PTILE)
- {
- if (!lex_force_num_range_closed (ctx->lexer, "PTILE", 0, 100))
- goto error;
- percentile = lex_number (ctx->lexer);
- lex_get (ctx->lexer);
- }
-
- /* Parse label. */
- char *label = NULL;
- if (lex_is_string (ctx->lexer))
- {
- label = ss_xstrdup (lex_tokss (ctx->lexer));
- lex_get (ctx->lexer);
- }
-
- /* Parse format. */
- struct fmt_spec format;
- const struct fmt_spec *formatp;
- bool is_ctables_format = false;
- if (lex_token (ctx->lexer) == T_ID
- && has_digit (lex_tokcstr (ctx->lexer)))
- {
- if (!parse_ctables_format_specifier (ctx->lexer, &format,
- &is_ctables_format))
- {
- free (label);
- goto error;
- }
- formatp = &format;
- }
- else
- formatp = NULL;
-
- struct msg_location *loc = lex_ofs_location (ctx->lexer, start_ofs,
- lex_ofs (ctx->lexer) - 1);
- add_summary_spec (sub, function, weighting, area, percentile, label,
- formatp, is_ctables_format, loc, sv);
- free (label);
- msg_location_destroy (loc);
-
- lex_match (ctx->lexer, T_COMMA);
- if (sv == CSV_CELL && lex_match_id (ctx->lexer, "TOTALS"))
- {
- if (!lex_force_match (ctx->lexer, T_LBRACK))
- goto error;
- sv = CSV_TOTAL;
- }
- else if (lex_match (ctx->lexer, T_RBRACK))
- {
- if (sv == CSV_TOTAL && !lex_force_match (ctx->lexer, T_RBRACK))
- goto error;
- return sub;
- }
- }
-
-error:
- ctables_axis_destroy (sub);
- return NULL;
-}
-
-static const struct ctables_axis *
-find_scale (const struct ctables_axis *axis)
-{
- if (!axis)
- return NULL;
- else if (axis->op == CTAO_VAR)
- return axis->scale ? axis : NULL;
- else
- {
- for (size_t i = 0; i < 2; i++)
- {
- const struct ctables_axis *scale = find_scale (axis->subs[i]);
- if (scale)
- return scale;
- }
- return NULL;
- }
-}
-
-static const struct ctables_axis *
-find_categorical_summary_spec (const struct ctables_axis *axis)
-{
- if (!axis)
- return NULL;
- else if (axis->op == CTAO_VAR)
- return !axis->scale && axis->specs[CSV_CELL].n ? axis : NULL;
- else
- {
- for (size_t i = 0; i < 2; i++)
- {
- const struct ctables_axis *sum
- = find_categorical_summary_spec (axis->subs[i]);
- if (sum)
- return sum;
- }
- return NULL;
- }
-}
-
-static struct ctables_axis *
-ctables_axis_parse_nest (struct ctables_axis_parse_ctx *ctx)
-{
- int start_ofs = lex_ofs (ctx->lexer);
- struct ctables_axis *lhs = ctables_axis_parse_postfix (ctx);
- if (!lhs)
- return NULL;
-
- while (lex_match (ctx->lexer, T_GT))
- {
- struct ctables_axis *rhs = ctables_axis_parse_postfix (ctx);
- if (!rhs)
- {
- ctables_axis_destroy (lhs);
- return NULL;
- }
-
- struct ctables_axis *nest = ctables_axis_new_nonterminal (
- CTAO_NEST, lhs, rhs, ctx->lexer, start_ofs);
-
- const struct ctables_axis *outer_scale = find_scale (lhs);
- const struct ctables_axis *inner_scale = find_scale (rhs);
- if (outer_scale && inner_scale)
- {
- msg_at (SE, nest->loc, _("Cannot nest scale variables."));
- msg_at (SN, outer_scale->loc, _("This is an outer scale variable."));
- msg_at (SN, inner_scale->loc, _("This is an inner scale variable."));
- ctables_axis_destroy (nest);
- return NULL;
- }
-
- const struct ctables_axis *outer_sum = find_categorical_summary_spec (lhs);
- if (outer_sum)
- {
- msg_at (SE, nest->loc,
- _("Summaries may only be requested for categorical variables "
- "at the innermost nesting level."));
- msg_at (SN, outer_sum->loc,
- _("This outer categorical variable has a summary."));
- ctables_axis_destroy (nest);
- return NULL;
- }
-
- lhs = nest;
- }
-
- return lhs;
-}
-
-static struct ctables_axis *
-ctables_axis_parse_stack (struct ctables_axis_parse_ctx *ctx)
-{
- int start_ofs = lex_ofs (ctx->lexer);
- struct ctables_axis *lhs = ctables_axis_parse_nest (ctx);
- if (!lhs)
- return NULL;
-
- while (lex_match (ctx->lexer, T_PLUS))
- {
- struct ctables_axis *rhs = ctables_axis_parse_nest (ctx);
- if (!rhs)
- {
- ctables_axis_destroy (lhs);
- return NULL;
- }
-
- lhs = ctables_axis_new_nonterminal (CTAO_STACK, lhs, rhs,
- ctx->lexer, start_ofs);
- }
-
- return lhs;
-}
-
-static bool
-ctables_axis_parse (struct lexer *lexer, struct dictionary *dict,
- struct ctables_axis **axisp)
-{
- *axisp = NULL;
- if (lex_token (lexer) == T_BY
- || lex_token (lexer) == T_SLASH
- || lex_token (lexer) == T_ENDCMD)
- return true;
-
- struct ctables_axis_parse_ctx ctx = {
- .lexer = lexer,
- .dict = dict,
- };
- *axisp = ctables_axis_parse_stack (&ctx);
- return *axisp;
-}
-\f
-/* CTABLES categories. */
-
-struct ctables_categories
- {
- size_t n_refs;
- struct ctables_category *cats;
- size_t n_cats;
- };
-
-struct ctables_category
- {
- enum ctables_category_type
- {
- /* Explicit category lists. */
- CCT_NUMBER,
- CCT_STRING,
- CCT_NRANGE, /* Numerical range. */
- CCT_SRANGE, /* String range. */
- CCT_MISSING,
- CCT_OTHERNM,
- CCT_POSTCOMPUTE,
-
- /* Totals and subtotals. */
- CCT_SUBTOTAL,
- CCT_TOTAL,
-
- /* Implicit category lists. */
- CCT_VALUE,
- CCT_LABEL,
- CCT_FUNCTION,
-
- /* For contributing to TOTALN. */
- CCT_EXCLUDED_MISSING,
- }
- type;
-
- struct ctables_category *subtotal;
-
- bool hide;
-
- union
- {
- double number; /* CCT_NUMBER. */
- struct substring string; /* CCT_STRING, in dictionary encoding. */
- double nrange[2]; /* CCT_NRANGE. */
- struct substring srange[2]; /* CCT_SRANGE. */
-
- struct
- {
- char *total_label; /* CCT_SUBTOTAL, CCT_TOTAL. */
- bool hide_subcategories; /* CCT_SUBTOTAL. */
- };
-
- /* CCT_POSTCOMPUTE. */
- struct
- {
- const struct ctables_postcompute *pc;
- enum fmt_type parse_format;
- };
-
- /* CCT_VALUE, CCT_LABEL, CCT_FUNCTION. */
- struct
- {
- bool include_missing;
- bool sort_ascending;
-
- /* CCT_FUNCTION. */
- enum ctables_summary_function sort_function;
- enum ctables_weighting weighting;
- enum ctables_area_type area;
- struct variable *sort_var;
- double percentile;
- };
- };
-
- /* Source location (sometimes NULL). */
- struct msg_location *location;
- };
-
-static void
-ctables_category_uninit (struct ctables_category *cat)
-{
- if (!cat)
- return;
-
- msg_location_destroy (cat->location);
- switch (cat->type)
- {
- case CCT_NUMBER:
- case CCT_NRANGE:
- case CCT_MISSING:
- case CCT_OTHERNM:
- case CCT_POSTCOMPUTE:
- break;
-
- case CCT_STRING:
- ss_dealloc (&cat->string);
- break;
-
- case CCT_SRANGE:
- ss_dealloc (&cat->srange[0]);
- ss_dealloc (&cat->srange[1]);
- break;
-
- case CCT_SUBTOTAL:
- case CCT_TOTAL:
- free (cat->total_label);
- break;
-
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- break;
-
- case CCT_EXCLUDED_MISSING:
- break;
- }
-}
-
-static bool
-nullable_substring_equal (const struct substring *a,
- const struct substring *b)
-{
- return !a->string ? !b->string : b->string && ss_equals (*a, *b);
-}
-
-static bool
-ctables_category_equal (const struct ctables_category *a,
- const struct ctables_category *b)
-{
- if (a->type != b->type)
- return false;
-
- switch (a->type)
- {
- case CCT_NUMBER:
- return a->number == b->number;
-
- case CCT_STRING:
- return ss_equals (a->string, b->string);
-
- case CCT_NRANGE:
- return a->nrange[0] == b->nrange[0] && a->nrange[1] == b->nrange[1];
-
- case CCT_SRANGE:
- return (nullable_substring_equal (&a->srange[0], &b->srange[0])
- && nullable_substring_equal (&a->srange[1], &b->srange[1]));
-
- case CCT_MISSING:
- case CCT_OTHERNM:
- return true;
-
- case CCT_POSTCOMPUTE:
- return a->pc == b->pc;
-
- case CCT_SUBTOTAL:
- case CCT_TOTAL:
- return !strcmp (a->total_label, b->total_label);
-
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- return (a->include_missing == b->include_missing
- && a->sort_ascending == b->sort_ascending
- && a->sort_function == b->sort_function
- && a->sort_var == b->sort_var
- && a->percentile == b->percentile);
-
- case CCT_EXCLUDED_MISSING:
- return true;
- }
-
- NOT_REACHED ();
-}
-
-static void
-ctables_categories_unref (struct ctables_categories *c)
-{
- if (!c)
- return;
-
- assert (c->n_refs > 0);
- if (--c->n_refs)
- return;
-
- for (size_t i = 0; i < c->n_cats; i++)
- ctables_category_uninit (&c->cats[i]);
- free (c->cats);
- free (c);
-}
-
-static bool
-ctables_categories_equal (const struct ctables_categories *a,
- const struct ctables_categories *b)
-{
- if (a->n_cats != b->n_cats)
- return false;
-
- for (size_t i = 0; i < a->n_cats; i++)
- if (!ctables_category_equal (&a->cats[i], &b->cats[i]))
- return false;
-
- return true;
-}
-
-static struct ctables_category
-cct_nrange (double low, double high)
-{
- return (struct ctables_category) {
- .type = CCT_NRANGE,
- .nrange = { low, high }
- };
-}
-
-static struct ctables_category
-cct_srange (struct substring low, struct substring high)
-{
- return (struct ctables_category) {
- .type = CCT_SRANGE,
- .srange = { low, high }
- };
-}
-
-static bool
-ctables_table_parse_subtotal (struct lexer *lexer, bool hide_subcategories,
- struct ctables_category *cat)
-{
- char *total_label;
- if (lex_match (lexer, T_EQUALS))
- {
- if (!lex_force_string (lexer))
- return false;
-
- total_label = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- else
- total_label = xstrdup (_("Subtotal"));
-
- *cat = (struct ctables_category) {
- .type = CCT_SUBTOTAL,
- .hide_subcategories = hide_subcategories,
- .total_label = total_label
- };
- return true;
-}
-
-static bool
-ctables_table_parse_explicit_category (struct lexer *lexer,
- struct dictionary *dict,
- struct ctables *ct,
- struct ctables_category *cat)
-{
- if (lex_match_id (lexer, "OTHERNM"))
- *cat = (struct ctables_category) { .type = CCT_OTHERNM };
- else if (lex_match_id (lexer, "MISSING"))
- *cat = (struct ctables_category) { .type = CCT_MISSING };
- else if (lex_match_id (lexer, "SUBTOTAL"))
- return ctables_table_parse_subtotal (lexer, false, cat);
- else if (lex_match_id (lexer, "HSUBTOTAL"))
- return ctables_table_parse_subtotal (lexer, true, cat);
- else if (lex_match_id (lexer, "LO"))
- {
- if (!lex_force_match_id (lexer, "THRU"))
- return false;
- if (lex_is_string (lexer))
- {
- struct substring sr0 = { .string = NULL };
- struct substring sr1 = parse_substring (lexer, dict);
- *cat = cct_srange (sr0, sr1);
- }
- else if (lex_force_num (lexer))
- {
- *cat = cct_nrange (-DBL_MAX, lex_number (lexer));
- lex_get (lexer);
- }
- else
- return false;
- }
- else if (lex_is_number (lexer))
- {
- double number = lex_number (lexer);
- lex_get (lexer);
- if (lex_match_id (lexer, "THRU"))
- {
- if (lex_match_id (lexer, "HI"))
- *cat = cct_nrange (number, DBL_MAX);
- else
- {
- if (!lex_force_num (lexer))
- return false;
- *cat = cct_nrange (number, lex_number (lexer));
- lex_get (lexer);
- }
- }
- else
- *cat = (struct ctables_category) {
- .type = CCT_NUMBER,
- .number = number
- };
- }
- else if (lex_is_string (lexer))
- {
- struct substring s = parse_substring (lexer, dict);
- if (lex_match_id (lexer, "THRU"))
- {
- if (lex_match_id (lexer, "HI"))
- {
- struct substring sr1 = { .string = NULL };
- *cat = cct_srange (s, sr1);
- }
- else
- {
- if (!lex_force_string (lexer))
- {
- ss_dealloc (&s);
- return false;
- }
- struct substring sr1 = parse_substring (lexer, dict);
- *cat = cct_srange (s, sr1);
- }
- }
- else
- *cat = (struct ctables_category) { .type = CCT_STRING, .string = s };
- }
- else if (lex_match (lexer, T_AND))
- {
- if (!lex_force_id (lexer))
- return false;
- struct ctables_postcompute *pc = ctables_find_postcompute (
- ct, lex_tokcstr (lexer));
- if (!pc)
- {
- struct msg_location *loc = lex_get_location (lexer, -1, 0);
- msg_at (SE, loc, _("Unknown postcompute &%s."),
- lex_tokcstr (lexer));
- msg_location_destroy (loc);
- return false;
- }
- lex_get (lexer);
-
- *cat = (struct ctables_category) { .type = CCT_POSTCOMPUTE, .pc = pc };
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting category specification."));
- return false;
- }
-
- return true;
-}
-
-static bool
-parse_category_string (struct msg_location *location,
- struct substring s, const struct dictionary *dict,
- enum fmt_type format, double *n)
-{
- union value v;
- char *error = data_in (s, dict_get_encoding (dict), format,
- settings_get_fmt_settings (), &v, 0, NULL);
- if (error)
- {
- msg_at (SE, location,
- _("Failed to parse category specification as format %s: %s."),
- fmt_name (format), error);
- free (error);
- return false;
- }
-
- *n = v.f;
- return true;
-}
-
-static struct ctables_category *
-ctables_find_category_for_postcompute__ (const struct ctables_categories *cats,
- const struct ctables_pcexpr *e)
-{
- struct ctables_category *best = NULL;
- size_t n_subtotals = 0;
- for (size_t i = 0; i < cats->n_cats; i++)
- {
- struct ctables_category *cat = &cats->cats[i];
- switch (e->op)
- {
- case CTPO_CAT_NUMBER:
- if (cat->type == CCT_NUMBER && cat->number == e->number)
- best = cat;
- break;
-
- case CTPO_CAT_STRING:
- if (cat->type == CCT_STRING && ss_equals (cat->string, e->string))
- best = cat;
- break;
-
- case CTPO_CAT_NRANGE:
- if (cat->type == CCT_NRANGE
- && cat->nrange[0] == e->nrange[0]
- && cat->nrange[1] == e->nrange[1])
- best = cat;
- break;
-
- case CTPO_CAT_SRANGE:
- if (cat->type == CCT_SRANGE
- && nullable_substring_equal (&cat->srange[0], &e->srange[0])
- && nullable_substring_equal (&cat->srange[1], &e->srange[1]))
- best = cat;
- break;
-
- case CTPO_CAT_MISSING:
- if (cat->type == CCT_MISSING)
- best = cat;
- break;
-
- case CTPO_CAT_OTHERNM:
- if (cat->type == CCT_OTHERNM)
- best = cat;
- break;
-
- case CTPO_CAT_SUBTOTAL:
- if (cat->type == CCT_SUBTOTAL)
- {
- n_subtotals++;
- if (e->subtotal_index == n_subtotals)
- return cat;
- else if (e->subtotal_index == 0)
- best = cat;
- }
- break;
-
- case CTPO_CAT_TOTAL:
- if (cat->type == CCT_TOTAL)
- return cat;
- break;
-
- case CTPO_CONSTANT:
- case CTPO_ADD:
- case CTPO_SUB:
- case CTPO_MUL:
- case CTPO_DIV:
- case CTPO_POW:
- case CTPO_NEG:
- NOT_REACHED ();
- }
- }
- if (e->op == CTPO_CAT_SUBTOTAL && e->subtotal_index == 0 && n_subtotals > 1)
- return NULL;
- return best;
-}
-
-static struct ctables_category *
-ctables_find_category_for_postcompute (const struct dictionary *dict,
- const struct ctables_categories *cats,
- enum fmt_type parse_format,
- const struct ctables_pcexpr *e)
-{
- if (parse_format != FMT_F)
- {
- if (e->op == CTPO_CAT_STRING)
- {
- double number;
- if (!parse_category_string (e->location, e->string, dict,
- parse_format, &number))
- return NULL;
-
- struct ctables_pcexpr e2 = {
- .op = CTPO_CAT_NUMBER,
- .number = number,
- .location = e->location,
- };
- return ctables_find_category_for_postcompute__ (cats, &e2);
- }
- else if (e->op == CTPO_CAT_SRANGE)
- {
- double nrange[2];
- if (!e->srange[0].string)
- nrange[0] = -DBL_MAX;
- else if (!parse_category_string (e->location, e->srange[0], dict,
- parse_format, &nrange[0]))
- return NULL;
-
- if (!e->srange[1].string)
- nrange[1] = DBL_MAX;
- else if (!parse_category_string (e->location, e->srange[1], dict,
- parse_format, &nrange[1]))
- return NULL;
-
- struct ctables_pcexpr e2 = {
- .op = CTPO_CAT_NRANGE,
- .nrange = { nrange[0], nrange[1] },
- .location = e->location,
- };
- return ctables_find_category_for_postcompute__ (cats, &e2);
- }
- }
- return ctables_find_category_for_postcompute__ (cats, e);
-}
-
-static struct substring
-rtrim_value (const union value *v, const struct variable *var)
-{
- struct substring s = ss_buffer (CHAR_CAST (char *, v->s),
- var_get_width (var));
- ss_rtrim (&s, ss_cstr (" "));
- return s;
-}
-
-static bool
-in_string_range (const union value *v, const struct variable *var,
- const struct substring *srange)
-{
- struct substring s = rtrim_value (v, var);
- return ((!srange[0].string || ss_compare (s, srange[0]) >= 0)
- && (!srange[1].string || ss_compare (s, srange[1]) <= 0));
-}
-
-static const struct ctables_category *
-ctables_categories_match (const struct ctables_categories *c,
- const union value *v, const struct variable *var)
-{
- if (var_is_numeric (var) && v->f == SYSMIS)
- return NULL;
-
- const struct ctables_category *othernm = NULL;
- for (size_t i = c->n_cats; i-- > 0; )
- {
- const struct ctables_category *cat = &c->cats[i];
- switch (cat->type)
- {
- case CCT_NUMBER:
- if (cat->number == v->f)
- return cat;
- break;
-
- case CCT_STRING:
- if (ss_equals (cat->string, rtrim_value (v, var)))
- return cat;
- break;
-
- case CCT_NRANGE:
- if ((cat->nrange[0] == -DBL_MAX || v->f >= cat->nrange[0])
- && (cat->nrange[1] == DBL_MAX || v->f <= cat->nrange[1]))
- return cat;
- break;
-
- case CCT_SRANGE:
- if (in_string_range (v, var, cat->srange))
- return cat;
- break;
-
- case CCT_MISSING:
- if (var_is_value_missing (var, v))
- return cat;
- break;
-
- case CCT_POSTCOMPUTE:
- break;
-
- case CCT_OTHERNM:
- if (!othernm)
- othernm = cat;
- break;
-
- case CCT_SUBTOTAL:
- case CCT_TOTAL:
- break;
-
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- return (cat->include_missing || !var_is_value_missing (var, v) ? cat
- : NULL);
-
- case CCT_EXCLUDED_MISSING:
- break;
- }
- }
-
- return var_is_value_missing (var, v) ? NULL : othernm;
-}
-
-static const struct ctables_category *
-ctables_categories_total (const struct ctables_categories *c)
-{
- const struct ctables_category *first = &c->cats[0];
- const struct ctables_category *last = &c->cats[c->n_cats - 1];
- return (first->type == CCT_TOTAL ? first
- : last->type == CCT_TOTAL ? last
- : NULL);
-}
-
-static void
-ctables_category_format_number (double number, const struct variable *var,
- struct string *s)
-{
- struct pivot_value *pv = pivot_value_new_var_value (
- var, &(union value) { .f = number });
- pivot_value_format (pv, NULL, s);
- pivot_value_destroy (pv);
-}
-
-static void
-ctables_category_format_string (struct substring string,
- const struct variable *var, struct string *out)
-{
- int width = var_get_width (var);
- char *s = xmalloc (width);
- buf_copy_rpad (s, width, string.string, string.length, ' ');
- struct pivot_value *pv = pivot_value_new_var_value (
- var, &(union value) { .s = CHAR_CAST (uint8_t *, s) });
- pivot_value_format (pv, NULL, out);
- pivot_value_destroy (pv);
- free (s);
-}
-
-static bool
-ctables_category_format_label (const struct ctables_category *cat,
- const struct variable *var,
- struct string *s)
-{
- switch (cat->type)
- {
- case CCT_NUMBER:
- ctables_category_format_number (cat->number, var, s);
- return true;
-
- case CCT_STRING:
- ctables_category_format_string (cat->string, var, s);
- return true;
-
- case CCT_NRANGE:
- ctables_category_format_number (cat->nrange[0], var, s);
- ds_put_format (s, " THRU ");
- ctables_category_format_number (cat->nrange[1], var, s);
- return true;
-
- case CCT_SRANGE:
- ctables_category_format_string (cat->srange[0], var, s);
- ds_put_format (s, " THRU ");
- ctables_category_format_string (cat->srange[1], var, s);
- return true;
-
- case CCT_MISSING:
- ds_put_cstr (s, "MISSING");
- return true;
-
- case CCT_OTHERNM:
- ds_put_cstr (s, "OTHERNM");
- return true;
-
- case CCT_POSTCOMPUTE:
- ds_put_format (s, "&%s", cat->pc->name);
- return true;
-
- case CCT_TOTAL:
- case CCT_SUBTOTAL:
- ds_put_cstr (s, cat->total_label);
- return true;
-
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- case CCT_EXCLUDED_MISSING:
- return false;
- }
-
- return false;
-}
-
-static bool
-ctables_recursive_check_postcompute (struct dictionary *dict,
- const struct ctables_pcexpr *e,
- struct ctables_category *pc_cat,
- const struct ctables_categories *cats,
- const struct msg_location *cats_location)
-{
- switch (e->op)
- {
- case CTPO_CAT_NUMBER:
- case CTPO_CAT_STRING:
- case CTPO_CAT_NRANGE:
- case CTPO_CAT_SRANGE:
- case CTPO_CAT_MISSING:
- case CTPO_CAT_OTHERNM:
- case CTPO_CAT_SUBTOTAL:
- case CTPO_CAT_TOTAL:
- {
- struct ctables_category *cat = ctables_find_category_for_postcompute (
- dict, cats, pc_cat->parse_format, e);
- if (!cat)
- {
- if (e->op == CTPO_CAT_SUBTOTAL && e->subtotal_index == 0)
- {
- size_t n_subtotals = 0;
- for (size_t i = 0; i < cats->n_cats; i++)
- n_subtotals += cats->cats[i].type == CCT_SUBTOTAL;
- if (n_subtotals > 1)
- {
- msg_at (SE, cats_location,
- ngettext ("These categories include %zu instance "
- "of SUBTOTAL or HSUBTOTAL, so references "
- "from computed categories must refer to "
- "subtotals by position, "
- "e.g. SUBTOTAL[1].",
- "These categories include %zu instances "
- "of SUBTOTAL or HSUBTOTAL, so references "
- "from computed categories must refer to "
- "subtotals by position, "
- "e.g. SUBTOTAL[1].",
- n_subtotals),
- n_subtotals);
- msg_at (SN, e->location,
- _("This is the reference that lacks a position."));
- return NULL;
- }
- }
-
- msg_at (SE, pc_cat->location,
- _("Computed category &%s references a category not included "
- "in the category list."),
- pc_cat->pc->name);
- msg_at (SN, e->location, _("This is the missing category."));
- if (e->op == CTPO_CAT_SUBTOTAL)
- msg_at (SN, cats_location,
- _("To fix the problem, add subtotals to the "
- "list of categories here."));
- else if (e->op == CTPO_CAT_TOTAL)
- msg (SN, _("To fix the problem, add TOTAL=YES to the variable's "
- "CATEGORIES specification."));
- else
- msg_at (SN, cats_location,
- _("To fix the problem, add the missing category to the "
- "list of categories here."));
- return false;
- }
- if (pc_cat->pc->hide_source_cats)
- cat->hide = true;
- return true;
- }
-
- case CTPO_CONSTANT:
- return true;
-
- case CTPO_ADD:
- case CTPO_SUB:
- case CTPO_MUL:
- case CTPO_DIV:
- case CTPO_POW:
- case CTPO_NEG:
- for (size_t i = 0; i < 2; i++)
- if (e->subs[i] && !ctables_recursive_check_postcompute (
- dict, e->subs[i], pc_cat, cats, cats_location))
- return false;
- return true;
- }
-
- NOT_REACHED ();
-}
-
-static struct pivot_value *
-ctables_postcompute_label (const struct ctables_categories *cats,
- const struct ctables_category *cat,
- const struct variable *var)
-{
- struct substring in = ss_cstr (cat->pc->label);
- struct substring target = ss_cstr (")LABEL[");
-
- struct string out = DS_EMPTY_INITIALIZER;
- for (;;)
- {
- size_t chunk = ss_find_substring (in, target);
- if (chunk == SIZE_MAX)
- {
- if (ds_is_empty (&out))
- return pivot_value_new_user_text (in.string, in.length);
- else
- {
- ds_put_substring (&out, in);
- return pivot_value_new_user_text_nocopy (ds_steal_cstr (&out));
- }
- }
-
- ds_put_substring (&out, ss_head (in, chunk));
- ss_advance (&in, chunk + target.length);
-
- struct substring idx_s;
- if (!ss_get_until (&in, ']', &idx_s))
- goto error;
- char *tail;
- long int idx = strtol (idx_s.string, &tail, 10);
- if (idx < 1 || idx > cats->n_cats || tail != ss_end (idx_s))
- goto error;
-
- struct ctables_category *cat2 = &cats->cats[idx - 1];
- if (!ctables_category_format_label (cat2, var, &out))
- goto error;
- }
-
-error:
- ds_destroy (&out);
- return pivot_value_new_user_text (cat->pc->label, SIZE_MAX);
-}
-
-static struct pivot_value *
-ctables_category_create_value_label (const struct ctables_categories *cats,
- const struct ctables_category *cat,
- const struct variable *var,
- const union value *value)
-{
- return (cat->type == CCT_POSTCOMPUTE && cat->pc->label
- ? ctables_postcompute_label (cats, cat, var)
- : cat->type == CCT_TOTAL || cat->type == CCT_SUBTOTAL
- ? pivot_value_new_user_text (cat->total_label, SIZE_MAX)
- : pivot_value_new_var_value (var, value));
-}
-\f
-/* CTABLES variable nesting and stacking. */
-
-/* A nested sequence of variables, e.g. a > b > c. */
-struct ctables_nest
- {
- struct variable **vars;
- size_t n;
- size_t scale_idx;
- size_t summary_idx;
- size_t *areas[N_CTATS];
- size_t n_areas[N_CTATS];
- size_t group_head;
-
- struct ctables_summary_spec_set specs[N_CSVS];
- };
-
-/* A stack of nestings, e.g. nest1 + nest2 + ... + nestN. */
-struct ctables_stack
- {
- struct ctables_nest *nests;
- size_t n;
- };
-
-static void
-ctables_nest_uninit (struct ctables_nest *nest)
-{
- free (nest->vars);
- for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
- ctables_summary_spec_set_uninit (&nest->specs[sv]);
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- free (nest->areas[at]);
-}
-
-static void
-ctables_stack_uninit (struct ctables_stack *stack)
-{
- if (stack)
- {
- for (size_t i = 0; i < stack->n; i++)
- ctables_nest_uninit (&stack->nests[i]);
- free (stack->nests);
- }
-}
-
-static struct ctables_stack
-nest_fts (struct ctables_stack s0, struct ctables_stack s1)
-{
- if (!s0.n)
- return s1;
- else if (!s1.n)
- return s0;
-
- struct ctables_stack stack = { .nests = xnmalloc (s0.n, s1.n * sizeof *stack.nests) };
- for (size_t i = 0; i < s0.n; i++)
- for (size_t j = 0; j < s1.n; j++)
- {
- const struct ctables_nest *a = &s0.nests[i];
- const struct ctables_nest *b = &s1.nests[j];
-
- size_t allocate = a->n + b->n;
- struct variable **vars = xnmalloc (allocate, sizeof *vars);
- size_t n = 0;
- for (size_t k = 0; k < a->n; k++)
- vars[n++] = a->vars[k];
- for (size_t k = 0; k < b->n; k++)
- vars[n++] = b->vars[k];
- assert (n == allocate);
-
- const struct ctables_nest *summary_src;
- if (!a->specs[CSV_CELL].var)
- summary_src = b;
- else if (!b->specs[CSV_CELL].var)
- summary_src = a;
- else
- NOT_REACHED ();
-
- struct ctables_nest *new = &stack.nests[stack.n++];
- *new = (struct ctables_nest) {
- .vars = vars,
- .scale_idx = (a->scale_idx != SIZE_MAX ? a->scale_idx
- : b->scale_idx != SIZE_MAX ? a->n + b->scale_idx
- : SIZE_MAX),
- .summary_idx = (a->summary_idx != SIZE_MAX ? a->summary_idx
- : b->summary_idx != SIZE_MAX ? a->n + b->summary_idx
- : SIZE_MAX),
- .n = n,
- };
- for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
- ctables_summary_spec_set_clone (&new->specs[sv], &summary_src->specs[sv]);
- }
- ctables_stack_uninit (&s0);
- ctables_stack_uninit (&s1);
- return stack;
-}
-
-static struct ctables_stack
-stack_fts (struct ctables_stack s0, struct ctables_stack s1)
-{
- struct ctables_stack stack = { .nests = xnmalloc (s0.n + s1.n, sizeof *stack.nests) };
- for (size_t i = 0; i < s0.n; i++)
- stack.nests[stack.n++] = s0.nests[i];
- for (size_t i = 0; i < s1.n; i++)
- {
- stack.nests[stack.n] = s1.nests[i];
- stack.nests[stack.n].group_head += s0.n;
- stack.n++;
- }
- assert (stack.n == s0.n + s1.n);
- free (s0.nests);
- free (s1.nests);
- return stack;
-}
-
-static struct ctables_stack
-var_fts (const struct ctables_axis *a)
-{
- struct variable **vars = xmalloc (sizeof *vars);
- *vars = a->var;
-
- bool is_summary = a->specs[CSV_CELL].n || a->scale;
- struct ctables_nest *nest = xmalloc (sizeof *nest);
- *nest = (struct ctables_nest) {
- .vars = vars,
- .n = 1,
- .scale_idx = a->scale ? 0 : SIZE_MAX,
- .summary_idx = is_summary ? 0 : SIZE_MAX,
- };
- if (is_summary)
- for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
- {
- ctables_summary_spec_set_clone (&nest->specs[sv], &a->specs[sv]);
- nest->specs[sv].var = a->var;
- nest->specs[sv].is_scale = a->scale;
- }
- return (struct ctables_stack) { .nests = nest, .n = 1 };
-}
-
-static struct ctables_stack
-enumerate_fts (enum pivot_axis_type axis_type, const struct ctables_axis *a)
-{
- if (!a)
- return (struct ctables_stack) { .n = 0 };
-
- switch (a->op)
- {
- case CTAO_VAR:
- return var_fts (a);
-
- case CTAO_STACK:
- return stack_fts (enumerate_fts (axis_type, a->subs[0]),
- enumerate_fts (axis_type, a->subs[1]));
-
- case CTAO_NEST:
- /* This should consider any of the scale variables found in the result to
- be linked to each other listwise for SMISSING=LISTWISE. */
- return nest_fts (enumerate_fts (axis_type, a->subs[0]),
- enumerate_fts (axis_type, a->subs[1]));
- }
-
- NOT_REACHED ();
-}
-\f
-/* CTABLES summary calculation. */
-
-union ctables_summary
- {
- /* COUNT, VALIDN, TOTALN. */
- double count;
-
- /* MINIMUM, MAXIMUM, RANGE. */
- struct
- {
- double min;
- double max;
- };
-
- /* MEAN, SEMEAN, STDDEV, SUM, VARIANCE, *.SUM. */
- struct moments1 *moments;
-
- /* MEDIAN, MODE, PTILE. */
- struct
- {
- struct casewriter *writer;
- double ovalid;
- double ovalue;
- };
- };
-
-static void
-ctables_summary_init (union ctables_summary *s,
- const struct ctables_summary_spec *ss)
-{
- switch (ss->function)
- {
- case CTSF_COUNT:
- case CTSF_areaPCT_COUNT:
- case CTSF_areaPCT_VALIDN:
- case CTSF_areaPCT_TOTALN:
- case CTSF_MISSING:
- case CTSF_TOTALN:
- case CTSF_VALIDN:
- s->count = 0;
- break;
-
- case CTSF_areaID:
- break;
-
- case CTSF_MAXIMUM:
- case CTSF_MINIMUM:
- case CTSF_RANGE:
- s->min = s->max = SYSMIS;
- break;
-
- case CTSF_MEAN:
- case CTSF_SUM:
- case CTSF_areaPCT_SUM:
- s->moments = moments1_create (MOMENT_MEAN);
- break;
-
- case CTSF_SEMEAN:
- case CTSF_STDDEV:
- case CTSF_VARIANCE:
- s->moments = moments1_create (MOMENT_VARIANCE);
- break;
-
- case CTSF_MEDIAN:
- case CTSF_MODE:
- case CTSF_PTILE:
- {
- struct caseproto *proto = caseproto_create ();
- proto = caseproto_add_width (proto, 0);
- proto = caseproto_add_width (proto, 0);
-
- struct subcase ordering;
- subcase_init (&ordering, 0, 0, SC_ASCEND);
- s->writer = sort_create_writer (&ordering, proto);
- subcase_uninit (&ordering);
- caseproto_unref (proto);
-
- s->ovalid = 0;
- s->ovalue = SYSMIS;
- }
- break;
- }
-}
-
-static void
-ctables_summary_uninit (union ctables_summary *s,
- const struct ctables_summary_spec *ss)
-{
- switch (ss->function)
- {
- case CTSF_COUNT:
- case CTSF_areaPCT_COUNT:
- case CTSF_areaPCT_VALIDN:
- case CTSF_areaPCT_TOTALN:
- case CTSF_MISSING:
- case CTSF_TOTALN:
- case CTSF_VALIDN:
- break;
-
- case CTSF_areaID:
- break;
-
- case CTSF_MAXIMUM:
- case CTSF_MINIMUM:
- case CTSF_RANGE:
- break;
-
- case CTSF_MEAN:
- case CTSF_SEMEAN:
- case CTSF_STDDEV:
- case CTSF_SUM:
- case CTSF_VARIANCE:
- case CTSF_areaPCT_SUM:
- moments1_destroy (s->moments);
- break;
-
- case CTSF_MEDIAN:
- case CTSF_MODE:
- case CTSF_PTILE:
- casewriter_destroy (s->writer);
- break;
- }
-}
-
-static void
-ctables_summary_add (union ctables_summary *s,
- const struct ctables_summary_spec *ss,
- const union value *value,
- bool is_missing, bool is_included,
- double weight)
-{
- /* To determine whether a case is included in a given table for a particular
- kind of summary, consider the following charts for the variable being
- summarized. Only if "yes" appears is the case counted.
-
- Categorical variables: VALIDN other TOTALN
- Valid values in included categories yes yes yes
- Missing values in included categories --- yes yes
- Missing values in excluded categories --- --- yes
- Valid values in excluded categories --- --- ---
-
- Scale variables: VALIDN other TOTALN
- Valid value yes yes yes
- Missing value --- yes yes
-
- Missing values include both user- and system-missing. (The system-missing
- value is always in an excluded category.)
-
- One way to interpret the above table is that scale variables are like
- categorical variables in which all values are in included categories.
- */
- switch (ss->function)
- {
- case CTSF_TOTALN:
- case CTSF_areaPCT_TOTALN:
- s->count += weight;
- break;
-
- case CTSF_COUNT:
- case CTSF_areaPCT_COUNT:
- if (is_included)
- s->count += weight;
- break;
-
- case CTSF_VALIDN:
- case CTSF_areaPCT_VALIDN:
- if (!is_missing)
- s->count += weight;
- break;
-
- case CTSF_areaID:
- break;
-
- case CTSF_MISSING:
- if (is_missing)
- s->count += weight;
- break;
-
- case CTSF_MAXIMUM:
- case CTSF_MINIMUM:
- case CTSF_RANGE:
- if (!is_missing)
- {
- if (s->min == SYSMIS || value->f < s->min)
- s->min = value->f;
- if (s->max == SYSMIS || value->f > s->max)
- s->max = value->f;
- }
- break;
-
- case CTSF_MEAN:
- case CTSF_SEMEAN:
- case CTSF_STDDEV:
- case CTSF_SUM:
- case CTSF_VARIANCE:
- if (!is_missing)
- moments1_add (s->moments, value->f, weight);
- break;
-
- case CTSF_areaPCT_SUM:
- if (!is_missing)
- moments1_add (s->moments, value->f, weight);
- break;
-
- case CTSF_MEDIAN:
- case CTSF_MODE:
- case CTSF_PTILE:
- if (!is_missing)
- {
- s->ovalid += weight;
-
- struct ccase *c = case_create (casewriter_get_proto (s->writer));
- *case_num_rw_idx (c, 0) = value->f;
- *case_num_rw_idx (c, 1) = weight;
- casewriter_write (s->writer, c);
- }
- break;
- }
-}
-
-static double
-ctables_summary_value (struct ctables_area *areas[N_CTATS],
- union ctables_summary *s,
- const struct ctables_summary_spec *ss)
-{
- switch (ss->function)
- {
- case CTSF_COUNT:
- return s->count;
-
- case CTSF_areaID:
- return areas[ss->calc_area]->sequence;
-
- case CTSF_areaPCT_COUNT:
- {
- const struct ctables_area *a = areas[ss->calc_area];
- double a_count = a->count[ss->weighting];
- return a_count ? s->count / a_count * 100 : SYSMIS;
- }
-
- case CTSF_areaPCT_VALIDN:
- {
- const struct ctables_area *a = areas[ss->calc_area];
- double a_valid = a->valid[ss->weighting];
- return a_valid ? s->count / a_valid * 100 : SYSMIS;
- }
-
- case CTSF_areaPCT_TOTALN:
- {
- const struct ctables_area *a = areas[ss->calc_area];
- double a_total = a->total[ss->weighting];
- return a_total ? s->count / a_total * 100 : SYSMIS;
- }
-
- case CTSF_MISSING:
- case CTSF_TOTALN:
- case CTSF_VALIDN:
- return s->count;
-
- case CTSF_MAXIMUM:
- return s->max;
-
- case CTSF_MINIMUM:
- return s->min;
-
- case CTSF_RANGE:
- return s->max != SYSMIS && s->min != SYSMIS ? s->max - s->min : SYSMIS;
-
- case CTSF_MEAN:
- {
- double mean;
- moments1_calculate (s->moments, NULL, &mean, NULL, NULL, NULL);
- return mean;
- }
-
- case CTSF_SEMEAN:
- {
- double weight, variance;
- moments1_calculate (s->moments, &weight, NULL, &variance, NULL, NULL);
- return calc_semean (variance, weight);
- }
-
- case CTSF_STDDEV:
- {
- double variance;
- moments1_calculate (s->moments, NULL, NULL, &variance, NULL, NULL);
- return variance != SYSMIS ? sqrt (variance) : SYSMIS;
- }
-
- case CTSF_SUM:
- {
- double weight, mean;
- moments1_calculate (s->moments, &weight, &mean, NULL, NULL, NULL);
- return weight != SYSMIS && mean != SYSMIS ? weight * mean : SYSMIS;
- }
-
- case CTSF_VARIANCE:
- {
- double variance;
- moments1_calculate (s->moments, NULL, NULL, &variance, NULL, NULL);
- return variance;
- }
-
- case CTSF_areaPCT_SUM:
- {
- double weight, mean;
- moments1_calculate (s->moments, &weight, &mean, NULL, NULL, NULL);
- if (weight == SYSMIS || mean == SYSMIS)
- return SYSMIS;
-
- const struct ctables_area *a = areas[ss->calc_area];
- const struct ctables_sum *sum = &a->sums[ss->sum_var_idx];
- double denom = sum->sum[ss->weighting];
- return denom != 0 ? weight * mean / denom * 100 : SYSMIS;
- }
-
- case CTSF_MEDIAN:
- case CTSF_PTILE:
- if (s->writer)
- {
- struct casereader *reader = casewriter_make_reader (s->writer);
- s->writer = NULL;
-
- struct percentile *ptile = percentile_create (
- ss->function == CTSF_PTILE ? ss->percentile : 0.5, s->ovalid);
- struct order_stats *os = &ptile->parent;
- order_stats_accumulate_idx (&os, 1, reader, 1, 0);
- s->ovalue = percentile_calculate (ptile, PC_HAVERAGE);
- statistic_destroy (&ptile->parent.parent);
- }
- return s->ovalue;
-
- case CTSF_MODE:
- if (s->writer)
- {
- struct casereader *reader = casewriter_make_reader (s->writer);
- s->writer = NULL;
-
- struct mode *mode = mode_create ();
- struct order_stats *os = &mode->parent;
- order_stats_accumulate_idx (&os, 1, reader, 1, 0);
- s->ovalue = mode->mode;
- statistic_destroy (&mode->parent.parent);
- }
- return s->ovalue;
- }
-
- NOT_REACHED ();
-}
-\f
-/* CTABLES occurrences. */
-
-struct ctables_occurrence
- {
- struct hmap_node node;
- union value value;
- };
-
-static void
-ctables_add_occurrence (const struct variable *var,
- const union value *value,
- struct hmap *occurrences)
-{
- int width = var_get_width (var);
- unsigned int hash = value_hash (value, width, 0);
-
- struct ctables_occurrence *o;
- HMAP_FOR_EACH_WITH_HASH (o, struct ctables_occurrence, node, hash,
- occurrences)
- if (value_equal (value, &o->value, width))
- return;
-
- o = xmalloc (sizeof *o);
- value_clone (&o->value, value, width);
- hmap_insert (occurrences, &o->node, hash);
-}
-\f
-enum ctables_vlabel
- {
- CTVL_NONE = SETTINGS_VALUE_SHOW_DEFAULT,
- CTVL_NAME = SETTINGS_VALUE_SHOW_VALUE,
- CTVL_LABEL = SETTINGS_VALUE_SHOW_LABEL,
- CTVL_BOTH = SETTINGS_VALUE_SHOW_BOTH,
- };
-
-struct ctables_cell
- {
- /* In struct ctables_section's 'cells' hmap. Indexed by all the values in
- all the axes (except the scalar variable, if any). */
- struct hmap_node node;
- struct ctables_section *section;
-
- /* The areas that contain this cell. */
- uint32_t omit_areas;
- struct ctables_area *areas[N_CTATS];
-
- bool hide;
-
- bool postcompute;
- enum ctables_summary_variant sv;
-
- struct ctables_cell_axis
- {
- struct ctables_cell_value
- {
- const struct ctables_category *category;
- union value value;
- }
- *cvs;
- int leaf;
- }
- axes[PIVOT_N_AXES];
-
- union ctables_summary *summaries;
- };
-
-struct ctables_section
- {
- /* Settings. */
- struct ctables_table *table;
- struct ctables_nest *nests[PIVOT_N_AXES];
-
- /* Data. */
- struct hmap *occurrences[PIVOT_N_AXES]; /* "struct ctables_occurrence"s. */
- struct hmap cells; /* Contains "struct ctables_cell"s. */
- struct hmap areas[N_CTATS]; /* Contains "struct ctables_area"s. */
- };
-
-static void ctables_section_uninit (struct ctables_section *);
-
-struct ctables_table
- {
- struct ctables *ctables;
- struct ctables_axis *axes[PIVOT_N_AXES];
- struct ctables_stack stacks[PIVOT_N_AXES];
- struct ctables_section *sections;
- size_t n_sections;
- enum pivot_axis_type summary_axis;
- struct ctables_summary_spec_set summary_specs;
- struct variable **sum_vars;
- size_t n_sum_vars;
-
- enum pivot_axis_type slabels_axis;
- bool slabels_visible;
-
- /* The innermost category labels for axis 'a' appear on axis label_axis[a].
-
- Most commonly, label_axis[a] == a, and in particular we always have
- label_axis{PIVOT_AXIS_LAYER] == PIVOT_AXIS_LAYER.
-
- If ROWLABELS or COLLABELS is specified, then one of
- label_axis[PIVOT_AXIS_ROW] or label_axis[PIVOT_AXIS_COLUMN] can be the
- opposite axis or PIVOT_AXIS_LAYER. Only one of them will differ.
-
- If any category labels are moved, then 'clabels_example' is one of the
- variables being moved (and it is otherwise NULL). All of the variables
- being moved have the same width, value labels, and categories, so this
- example variable can be used to find those out.
-
- The remaining members in this group are relevant only if category labels
- are moved.
-
- 'clabels_values_map' holds a "struct ctables_value" for all the values
- that appear in all of the variables in the moved categories. It is
- accumulated as the data is read. Once the data is fully read, its
- sorted values are put into 'clabels_values' and 'n_clabels_values'.
- */
- enum pivot_axis_type label_axis[PIVOT_N_AXES];
- enum pivot_axis_type clabels_from_axis;
- enum pivot_axis_type clabels_to_axis;
- int clabels_start_ofs, clabels_end_ofs;
- const struct variable *clabels_example;
- struct hmap clabels_values_map;
- struct ctables_value **clabels_values;
- size_t n_clabels_values;
-
- /* Indexed by variable dictionary index. */
- struct ctables_categories **categories;
- size_t n_categories;
- bool *show_empty;
-
- double cilevel;
-
- char *caption;
- char *corner;
- char *title;
-
- struct ctables_chisq *chisq;
- struct ctables_pairwise *pairwise;
- };
-
-struct ctables_cell_sort_aux
- {
- const struct ctables_nest *nest;
- enum pivot_axis_type a;
- };
-
-static int
-ctables_cell_compare_3way (const void *a_, const void *b_, const void *aux_)
-{
- const struct ctables_cell_sort_aux *aux = aux_;
- struct ctables_cell *const *ap = a_;
- struct ctables_cell *const *bp = b_;
- const struct ctables_cell *a = *ap;
- const struct ctables_cell *b = *bp;
-
- const struct ctables_nest *nest = aux->nest;
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- const struct variable *var = nest->vars[i];
- const struct ctables_cell_value *a_cv = &a->axes[aux->a].cvs[i];
- const struct ctables_cell_value *b_cv = &b->axes[aux->a].cvs[i];
- if (a_cv->category != b_cv->category)
- return a_cv->category > b_cv->category ? 1 : -1;
-
- const union value *a_val = &a_cv->value;
- const union value *b_val = &b_cv->value;
- switch (a_cv->category->type)
- {
- case CCT_NUMBER:
- case CCT_STRING:
- case CCT_SUBTOTAL:
- case CCT_TOTAL:
- case CCT_POSTCOMPUTE:
- case CCT_EXCLUDED_MISSING:
- /* Must be equal. */
- continue;
-
- case CCT_NRANGE:
- case CCT_SRANGE:
- case CCT_MISSING:
- case CCT_OTHERNM:
- {
- int cmp = value_compare_3way (a_val, b_val, var_get_width (var));
- if (cmp)
- return cmp;
- }
- break;
-
- case CCT_VALUE:
- {
- int cmp = value_compare_3way (a_val, b_val, var_get_width (var));
- if (cmp)
- return a_cv->category->sort_ascending ? cmp : -cmp;
- }
- break;
-
- case CCT_LABEL:
- {
- const char *a_label = var_lookup_value_label (var, a_val);
- const char *b_label = var_lookup_value_label (var, b_val);
- int cmp;
- if (a_label)
- {
- if (!b_label)
- return -1;
- cmp = strcmp (a_label, b_label);
- }
- else
- {
- if (b_label)
- return 1;
- cmp = value_compare_3way (a_val, b_val, var_get_width (var));
- }
- if (cmp)
- return a_cv->category->sort_ascending ? cmp : -cmp;
- }
- break;
-
- case CCT_FUNCTION:
- NOT_REACHED ();
- }
- }
- return 0;
-}
-
-static struct ctables_area *
-ctables_area_insert (struct ctables_cell *cell, enum ctables_area_type area)
-{
- struct ctables_section *s = cell->section;
- size_t hash = 0;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n_areas[area]; i++)
- {
- size_t v_idx = nest->areas[area][i];
- struct ctables_cell_value *cv = &cell->axes[a].cvs[v_idx];
- hash = hash_pointer (cv->category, hash);
- if (cv->category->type != CCT_TOTAL
- && cv->category->type != CCT_SUBTOTAL
- && cv->category->type != CCT_POSTCOMPUTE)
- hash = value_hash (&cv->value,
- var_get_width (nest->vars[v_idx]), hash);
- }
- }
-
- struct ctables_area *a;
- HMAP_FOR_EACH_WITH_HASH (a, struct ctables_area, node, hash, &s->areas[area])
- {
- const struct ctables_cell *df = a->example;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n_areas[area]; i++)
- {
- size_t v_idx = nest->areas[area][i];
- struct ctables_cell_value *cv1 = &df->axes[a].cvs[v_idx];
- struct ctables_cell_value *cv2 = &cell->axes[a].cvs[v_idx];
- if (cv1->category != cv2->category
- || (cv1->category->type != CCT_TOTAL
- && cv1->category->type != CCT_SUBTOTAL
- && cv1->category->type != CCT_POSTCOMPUTE
- && !value_equal (&cv1->value, &cv2->value,
- var_get_width (nest->vars[v_idx]))))
- goto not_equal;
- }
- }
- return a;
-
- not_equal: ;
- }
-
- struct ctables_sum *sums = (s->table->n_sum_vars
- ? xzalloc (s->table->n_sum_vars * sizeof *sums)
- : NULL);
-
- a = xmalloc (sizeof *a);
- *a = (struct ctables_area) { .example = cell, .sums = sums };
- hmap_insert (&s->areas[area], &a->node, hash);
- return a;
-}
-
-static struct ctables_cell *
-ctables_cell_insert__ (struct ctables_section *s, const struct ccase *c,
- const struct ctables_category **cats[PIVOT_N_AXES])
-{
- size_t hash = 0;
- enum ctables_summary_variant sv = CSV_CELL;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- hash = hash_pointer (cats[a][i], hash);
- if (cats[a][i]->type != CCT_TOTAL
- && cats[a][i]->type != CCT_SUBTOTAL
- && cats[a][i]->type != CCT_POSTCOMPUTE)
- hash = value_hash (case_data (c, nest->vars[i]),
- var_get_width (nest->vars[i]), hash);
- else
- sv = CSV_TOTAL;
- }
- }
-
- struct ctables_cell *cell;
- HMAP_FOR_EACH_WITH_HASH (cell, struct ctables_cell, node, hash, &s->cells)
- {
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx
- && (cats[a][i] != cell->axes[a].cvs[i].category
- || (cats[a][i]->type != CCT_TOTAL
- && cats[a][i]->type != CCT_SUBTOTAL
- && cats[a][i]->type != CCT_POSTCOMPUTE
- && !value_equal (case_data (c, nest->vars[i]),
- &cell->axes[a].cvs[i].value,
- var_get_width (nest->vars[i])))))
- goto not_equal;
- }
-
- return cell;
-
- not_equal: ;
- }
-
- cell = xmalloc (sizeof *cell);
- cell->section = s;
- cell->hide = false;
- cell->sv = sv;
- cell->omit_areas = 0;
- cell->postcompute = false;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- cell->axes[a].cvs = (nest->n
- ? xnmalloc (nest->n, sizeof *cell->axes[a].cvs)
- : NULL);
- for (size_t i = 0; i < nest->n; i++)
- {
- const struct ctables_category *cat = cats[a][i];
- const struct variable *var = nest->vars[i];
- const union value *value = case_data (c, var);
- if (i != nest->scale_idx)
- {
- const struct ctables_category *subtotal = cat->subtotal;
- if (cat->hide || (subtotal && subtotal->hide_subcategories))
- cell->hide = true;
-
- if (cat->type == CCT_TOTAL
- || cat->type == CCT_SUBTOTAL
- || cat->type == CCT_POSTCOMPUTE)
- {
- switch (a)
- {
- case PIVOT_AXIS_COLUMN:
- cell->omit_areas |= ((1u << CTAT_TABLE) |
- (1u << CTAT_LAYER) |
- (1u << CTAT_LAYERCOL) |
- (1u << CTAT_SUBTABLE) |
- (1u << CTAT_COL));
- break;
- case PIVOT_AXIS_ROW:
- cell->omit_areas |= ((1u << CTAT_TABLE) |
- (1u << CTAT_LAYER) |
- (1u << CTAT_LAYERROW) |
- (1u << CTAT_SUBTABLE) |
- (1u << CTAT_ROW));
- break;
- case PIVOT_AXIS_LAYER:
- cell->omit_areas |= ((1u << CTAT_TABLE) |
- (1u << CTAT_LAYER));
- break;
- }
- }
- if (cat->type == CCT_POSTCOMPUTE)
- cell->postcompute = true;
- }
-
- cell->axes[a].cvs[i].category = cat;
- value_clone (&cell->axes[a].cvs[i].value, value, var_get_width (var));
- }
- }
-
- const struct ctables_nest *ss = s->nests[s->table->summary_axis];
- const struct ctables_summary_spec_set *specs = &ss->specs[cell->sv];
- cell->summaries = xmalloc (specs->n * sizeof *cell->summaries);
- for (size_t i = 0; i < specs->n; i++)
- ctables_summary_init (&cell->summaries[i], &specs->specs[i]);
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- cell->areas[at] = ctables_area_insert (cell, at);
- hmap_insert (&s->cells, &cell->node, hash);
- return cell;
-}
-
-static void
-add_weight (double dst[N_CTWS], const double src[N_CTWS])
-{
- for (enum ctables_weighting wt = 0; wt < N_CTWS; wt++)
- dst[wt] += src[wt];
-}
-
-static void
-ctables_cell_add__ (struct ctables_section *s, const struct ccase *c,
- const struct ctables_category **cats[PIVOT_N_AXES],
- bool is_included, double weight[N_CTWS])
-{
- struct ctables_cell *cell = ctables_cell_insert__ (s, c, cats);
- const struct ctables_nest *ss = s->nests[s->table->summary_axis];
-
- const struct ctables_summary_spec_set *specs = &ss->specs[cell->sv];
- const union value *value = case_data (c, specs->var);
- bool is_missing = var_is_value_missing (specs->var, value);
- bool is_scale_missing
- = is_missing || (specs->is_scale && is_listwise_missing (specs, c));
-
- for (size_t i = 0; i < specs->n; i++)
- ctables_summary_add (&cell->summaries[i], &specs->specs[i], value,
- is_scale_missing, is_included,
- weight[specs->specs[i].weighting]);
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- if (!(cell->omit_areas && (1u << at)))
- {
- struct ctables_area *a = cell->areas[at];
-
- add_weight (a->total, weight);
- if (is_included)
- add_weight (a->count, weight);
- if (!is_missing)
- {
- add_weight (a->valid, weight);
-
- if (!is_scale_missing)
- for (size_t i = 0; i < s->table->n_sum_vars; i++)
- {
- const struct variable *var = s->table->sum_vars[i];
- double addend = case_num (c, var);
- if (!var_is_num_missing (var, addend))
- for (enum ctables_weighting wt = 0; wt < N_CTWS; wt++)
- a->sums[i].sum[wt] += addend * weight[wt];
- }
- }
- }
-}
-
-static void
-recurse_totals (struct ctables_section *s, const struct ccase *c,
- const struct ctables_category **cats[PIVOT_N_AXES],
- bool is_included, double weight[N_CTWS],
- enum pivot_axis_type start_axis, size_t start_nest)
-{
- for (enum pivot_axis_type a = start_axis; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = start_nest; i < nest->n; i++)
- {
- if (i == nest->scale_idx)
- continue;
-
- const struct variable *var = nest->vars[i];
-
- const struct ctables_category *total = ctables_categories_total (
- s->table->categories[var_get_dict_index (var)]);
- if (total)
- {
- const struct ctables_category *save = cats[a][i];
- cats[a][i] = total;
- ctables_cell_add__ (s, c, cats, is_included, weight);
- recurse_totals (s, c, cats, is_included, weight, a, i + 1);
- cats[a][i] = save;
- }
- }
- start_nest = 0;
- }
-}
-
-static void
-recurse_subtotals (struct ctables_section *s, const struct ccase *c,
- const struct ctables_category **cats[PIVOT_N_AXES],
- bool is_included, double weight[N_CTWS],
- enum pivot_axis_type start_axis, size_t start_nest)
-{
- for (enum pivot_axis_type a = start_axis; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = start_nest; i < nest->n; i++)
- {
- if (i == nest->scale_idx)
- continue;
-
- const struct ctables_category *save = cats[a][i];
- if (save->subtotal)
- {
- cats[a][i] = save->subtotal;
- ctables_cell_add__ (s, c, cats, is_included, weight);
- recurse_subtotals (s, c, cats, is_included, weight, a, i + 1);
- cats[a][i] = save;
- }
- }
- start_nest = 0;
- }
-}
-
-static void
-ctables_cell_insert (struct ctables_section *s, const struct ccase *c,
- double weight[N_CTWS])
-{
- const struct ctables_category *layer_cats[s->nests[PIVOT_AXIS_LAYER]->n];
- const struct ctables_category *row_cats[s->nests[PIVOT_AXIS_ROW]->n];
- const struct ctables_category *column_cats[s->nests[PIVOT_AXIS_COLUMN]->n];
- const struct ctables_category **cats[PIVOT_N_AXES] =
- {
- [PIVOT_AXIS_LAYER] = layer_cats,
- [PIVOT_AXIS_ROW] = row_cats,
- [PIVOT_AXIS_COLUMN] = column_cats,
- };
-
- bool is_included = true;
-
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- const struct variable *var = nest->vars[i];
- const union value *value = case_data (c, var);
-
- cats[a][i] = ctables_categories_match (
- s->table->categories[var_get_dict_index (var)], value, var);
- if (!cats[a][i])
- {
- if (i != nest->summary_idx)
- return;
-
- if (!var_is_value_missing (var, value))
- return;
-
- static const struct ctables_category cct_excluded_missing = {
- .type = CCT_EXCLUDED_MISSING,
- .hide = true,
- };
- cats[a][i] = &cct_excluded_missing;
- is_included = false;
- }
- }
- }
-
- if (is_included)
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- const struct variable *var = nest->vars[i];
- const union value *value = case_data (c, var);
- ctables_add_occurrence (var, value, &s->occurrences[a][i]);
- }
- }
-
- ctables_cell_add__ (s, c, cats, is_included, weight);
- recurse_totals (s, c, cats, is_included, weight, 0, 0);
- recurse_subtotals (s, c, cats, is_included, weight, 0, 0);
-}
-\f
-struct ctables_value
- {
- struct hmap_node node;
- union value value;
- int leaf;
- };
-
-static struct ctables_value *
-ctables_value_find__ (const struct ctables_table *t, const union value *value,
- int width, unsigned int hash)
-{
- struct ctables_value *clv;
- HMAP_FOR_EACH_WITH_HASH (clv, struct ctables_value, node,
- hash, &t->clabels_values_map)
- if (value_equal (value, &clv->value, width))
- return clv;
- return NULL;
-}
-
-static void
-ctables_value_insert (struct ctables_table *t, const union value *value,
- int width)
-{
- unsigned int hash = value_hash (value, width, 0);
- struct ctables_value *clv = ctables_value_find__ (t, value, width, hash);
- if (!clv)
- {
- clv = xmalloc (sizeof *clv);
- value_clone (&clv->value, value, width);
- hmap_insert (&t->clabels_values_map, &clv->node, hash);
- }
-}
-
-static const struct ctables_value *
-ctables_value_find (const struct ctables_cell *cell)
-{
- const struct ctables_section *s = cell->section;
- const struct ctables_table *t = s->table;
- if (!t->clabels_example)
- return NULL;
-
- const struct ctables_nest *clabels_nest = s->nests[t->clabels_from_axis];
- const struct variable *var = clabels_nest->vars[clabels_nest->n - 1];
- const union value *value
- = &cell->axes[t->clabels_from_axis].cvs[clabels_nest->n - 1].value;
- int width = var_get_width (var);
- const struct ctables_value *ctv = ctables_value_find__ (
- t, value, width, value_hash (value, width, 0));
- assert (ctv != NULL);
- return ctv;
-}
-
-static int
-compare_ctables_values_3way (const void *a_, const void *b_, const void *width_)
-{
- const struct ctables_value *const *ap = a_;
- const struct ctables_value *const *bp = b_;
- const struct ctables_value *a = *ap;
- const struct ctables_value *b = *bp;
- const int *width = width_;
- return value_compare_3way (&a->value, &b->value, *width);
-}
-
-static void
-ctables_sort_clabels_values (struct ctables_table *t)
-{
- const struct variable *v0 = t->clabels_example;
- int width = var_get_width (v0);
-
- size_t i0 = var_get_dict_index (v0);
- struct ctables_categories *c0 = t->categories[i0];
- if (t->show_empty[i0])
- {
- const struct val_labs *val_labs = var_get_value_labels (v0);
- for (const struct val_lab *vl = val_labs_first (val_labs); vl;
- vl = val_labs_next (val_labs, vl))
- if (ctables_categories_match (c0, &vl->value, v0))
- ctables_value_insert (t, &vl->value, width);
- }
-
- size_t n = hmap_count (&t->clabels_values_map);
- t->clabels_values = xnmalloc (n, sizeof *t->clabels_values);
-
- struct ctables_value *clv;
- size_t i = 0;
- HMAP_FOR_EACH (clv, struct ctables_value, node, &t->clabels_values_map)
- t->clabels_values[i++] = clv;
- t->n_clabels_values = n;
- assert (i == n);
-
- sort (t->clabels_values, n, sizeof *t->clabels_values,
- compare_ctables_values_3way, &width);
-
- for (size_t i = 0; i < n; i++)
- t->clabels_values[i]->leaf = i;
-}
-\f
-struct ctables
- {
- const struct dictionary *dict;
- struct pivot_table_look *look;
-
- /* For CTEF_* formats. */
- struct fmt_settings ctables_formats;
-
- /* If this is NULL, zeros are displayed using the normal print format.
- Otherwise, this string is displayed. */
- char *zero;
-
- /* If this is NULL, missing values are displayed using the normal print
- format. Otherwise, this string is displayed. */
- char *missing;
-
- /* Indexed by variable dictionary index. */
- enum ctables_vlabel *vlabels;
-
- struct hmap postcomputes; /* Contains "struct ctables_postcompute"s. */
-
- bool mrsets_count_duplicates; /* MRSETS. */
- bool smissing_listwise; /* SMISSING. */
- struct variable *e_weight; /* WEIGHT. */
- int hide_threshold; /* HIDESMALLCOUNTS. */
-
- struct ctables_table **tables;
- size_t n_tables;
- };
-\f
-static double
-ctpo_add (double a, double b)
-{
- return a + b;
-}
-
-static double
-ctpo_sub (double a, double b)
-{
- return a - b;
-}
-
-static double
-ctpo_mul (double a, double b)
-{
- return a * b;
-}
-
-static double
-ctpo_div (double a, double b)
-{
- return b ? a / b : SYSMIS;
-}
-
-static double
-ctpo_pow (double a, double b)
-{
- int save_errno = errno;
- errno = 0;
- double result = pow (a, b);
- if (errno)
- result = SYSMIS;
- errno = save_errno;
- return result;
-}
-
-static double
-ctpo_neg (double a, double b UNUSED)
-{
- return -a;
-}
-
-struct ctables_pcexpr_evaluate_ctx
- {
- const struct ctables_cell *cell;
- const struct ctables_section *section;
- const struct ctables_categories *cats;
- enum pivot_axis_type pc_a;
- size_t pc_a_idx;
- size_t summary_idx;
- enum fmt_type parse_format;
- };
-
-static double ctables_pcexpr_evaluate (
- const struct ctables_pcexpr_evaluate_ctx *, const struct ctables_pcexpr *);
-
-static double
-ctables_pcexpr_evaluate_nonterminal (
- const struct ctables_pcexpr_evaluate_ctx *ctx,
- const struct ctables_pcexpr *e, size_t n_args,
- double evaluate (double, double))
-{
- double args[2] = { 0, 0 };
- for (size_t i = 0; i < n_args; i++)
- {
- args[i] = ctables_pcexpr_evaluate (ctx, e->subs[i]);
- if (!isfinite (args[i]) || args[i] == SYSMIS)
- return SYSMIS;
- }
- return evaluate (args[0], args[1]);
-}
-
-static double
-ctables_pcexpr_evaluate_category (const struct ctables_pcexpr_evaluate_ctx *ctx,
- const struct ctables_cell_value *pc_cv)
-{
- const struct ctables_section *s = ctx->section;
-
- size_t hash = 0;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- const struct ctables_cell_value *cv
- = (a == ctx->pc_a && i == ctx->pc_a_idx ? pc_cv
- : &ctx->cell->axes[a].cvs[i]);
- hash = hash_pointer (cv->category, hash);
- if (cv->category->type != CCT_TOTAL
- && cv->category->type != CCT_SUBTOTAL
- && cv->category->type != CCT_POSTCOMPUTE)
- hash = value_hash (&cv->value,
- var_get_width (nest->vars[i]), hash);
- }
- }
-
- struct ctables_cell *tc;
- HMAP_FOR_EACH_WITH_HASH (tc, struct ctables_cell, node, hash, &s->cells)
- {
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- const struct ctables_cell_value *p_cv
- = (a == ctx->pc_a && i == ctx->pc_a_idx ? pc_cv
- : &ctx->cell->axes[a].cvs[i]);
- const struct ctables_cell_value *t_cv = &tc->axes[a].cvs[i];
- if (p_cv->category != t_cv->category
- || (p_cv->category->type != CCT_TOTAL
- && p_cv->category->type != CCT_SUBTOTAL
- && p_cv->category->type != CCT_POSTCOMPUTE
- && !value_equal (&p_cv->value,
- &t_cv->value,
- var_get_width (nest->vars[i]))))
- goto not_equal;
- }
- }
-
- goto found;
-
- not_equal: ;
- }
- return 0;
-
-found: ;
- const struct ctables_table *t = s->table;
- const struct ctables_nest *specs_nest = s->nests[t->summary_axis];
- const struct ctables_summary_spec_set *specs = &specs_nest->specs[tc->sv];
- return ctables_summary_value (tc->areas, &tc->summaries[ctx->summary_idx],
- &specs->specs[ctx->summary_idx]);
-}
-
-static double
-ctables_pcexpr_evaluate (const struct ctables_pcexpr_evaluate_ctx *ctx,
- const struct ctables_pcexpr *e)
-{
- switch (e->op)
- {
- case CTPO_CONSTANT:
- return e->number;
-
- case CTPO_CAT_NRANGE:
- case CTPO_CAT_SRANGE:
- case CTPO_CAT_MISSING:
- case CTPO_CAT_OTHERNM:
- {
- struct ctables_cell_value cv = {
- .category = ctables_find_category_for_postcompute (ctx->section->table->ctables->dict, ctx->cats, ctx->parse_format, e)
- };
- assert (cv.category != NULL);
-
- struct hmap *occurrences = &ctx->section->occurrences[ctx->pc_a][ctx->pc_a_idx];
- const struct ctables_occurrence *o;
-
- double sum = 0.0;
- const struct variable *var = ctx->section->nests[ctx->pc_a]->vars[ctx->pc_a_idx];
- HMAP_FOR_EACH (o, struct ctables_occurrence, node, occurrences)
- if (ctables_categories_match (ctx->cats, &o->value, var) == cv.category)
- {
- cv.value = o->value;
- sum += ctables_pcexpr_evaluate_category (ctx, &cv);
- }
- return sum;
- }
-
- case CTPO_CAT_NUMBER:
- case CTPO_CAT_SUBTOTAL:
- case CTPO_CAT_TOTAL:
- {
- struct ctables_cell_value cv = {
- .category = ctables_find_category_for_postcompute (ctx->section->table->ctables->dict, ctx->cats, ctx->parse_format, e),
- .value = { .f = e->number },
- };
- assert (cv.category != NULL);
- return ctables_pcexpr_evaluate_category (ctx, &cv);
- }
-
- case CTPO_CAT_STRING:
- {
- int width = var_get_width (ctx->section->nests[ctx->pc_a]->vars[ctx->pc_a_idx]);
- char *s = NULL;
- if (width > e->string.length)
- {
- s = xmalloc (width);
- buf_copy_rpad (s, width, e->string.string, e->string.length, ' ');
- }
-
- const struct ctables_category *category
- = ctables_find_category_for_postcompute (
- ctx->section->table->ctables->dict,
- ctx->cats, ctx->parse_format, e);
- assert (category != NULL);
-
- struct ctables_cell_value cv = { .category = category };
- if (category->type == CCT_NUMBER)
- cv.value.f = category->number;
- else if (category->type == CCT_STRING)
- cv.value.s = CHAR_CAST (uint8_t *, s ? s : e->string.string);
- else
- NOT_REACHED ();
-
- double retval = ctables_pcexpr_evaluate_category (ctx, &cv);
- free (s);
- return retval;
- }
-
- case CTPO_ADD:
- return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_add);
-
- case CTPO_SUB:
- return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_sub);
-
- case CTPO_MUL:
- return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_mul);
-
- case CTPO_DIV:
- return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_div);
-
- case CTPO_POW:
- return ctables_pcexpr_evaluate_nonterminal (ctx, e, 2, ctpo_pow);
-
- case CTPO_NEG:
- return ctables_pcexpr_evaluate_nonterminal (ctx, e, 1, ctpo_neg);
- }
-
- NOT_REACHED ();
-}
-
-static const struct ctables_category *
-ctables_cell_postcompute (const struct ctables_section *s,
- const struct ctables_cell *cell,
- enum pivot_axis_type *pc_a_p,
- size_t *pc_a_idx_p)
-{
- assert (cell->postcompute);
- const struct ctables_category *pc_cat = NULL;
- for (enum pivot_axis_type pc_a = 0; pc_a < PIVOT_N_AXES; pc_a++)
- for (size_t pc_a_idx = 0; pc_a_idx < s->nests[pc_a]->n; pc_a_idx++)
- {
- const struct ctables_cell_value *cv = &cell->axes[pc_a].cvs[pc_a_idx];
- if (cv->category->type == CCT_POSTCOMPUTE)
- {
- if (pc_cat)
- {
- /* Multiple postcomputes cross each other. The value is
- undefined. */
- return NULL;
- }
-
- pc_cat = cv->category;
- if (pc_a_p)
- *pc_a_p = pc_a;
- if (pc_a_idx_p)
- *pc_a_idx_p = pc_a_idx;
- }
- }
-
- assert (pc_cat != NULL);
- return pc_cat;
-}
-
-static double
-ctables_cell_calculate_postcompute (const struct ctables_section *s,
- const struct ctables_cell *cell,
- const struct ctables_summary_spec *ss,
- struct fmt_spec *format,
- bool *is_ctables_format,
- size_t summary_idx)
-{
- enum pivot_axis_type pc_a = 0;
- size_t pc_a_idx = 0;
- const struct ctables_category *pc_cat = ctables_cell_postcompute (
- s, cell, &pc_a, &pc_a_idx);
- if (!pc_cat)
- return SYSMIS;
-
- const struct ctables_postcompute *pc = pc_cat->pc;
- if (pc->specs)
- {
- for (size_t i = 0; i < pc->specs->n; i++)
- {
- const struct ctables_summary_spec *ss2 = &pc->specs->specs[i];
- if (ss->function == ss2->function
- && ss->weighting == ss2->weighting
- && ss->calc_area == ss2->calc_area
- && ss->percentile == ss2->percentile)
- {
- *format = ss2->format;
- *is_ctables_format = ss2->is_ctables_format;
- break;
- }
- }
- }
-
- const struct variable *var = s->nests[pc_a]->vars[pc_a_idx];
- const struct ctables_categories *cats = s->table->categories[
- var_get_dict_index (var)];
- struct ctables_pcexpr_evaluate_ctx ctx = {
- .cell = cell,
- .section = s,
- .cats = cats,
- .pc_a = pc_a,
- .pc_a_idx = pc_a_idx,
- .summary_idx = summary_idx,
- .parse_format = pc_cat->parse_format,
- };
- return ctables_pcexpr_evaluate (&ctx, pc->expr);
-}
-\f
-/* Chi-square test (SIGTEST). */
-struct ctables_chisq
- {
- double alpha;
- bool include_mrsets;
- bool all_visible;
- };
-
-/* Pairwise comparison test (COMPARETEST). */
-struct ctables_pairwise
- {
- enum { PROP, MEAN } type;
- double alpha[2];
- bool include_mrsets;
- bool meansvariance_allcats;
- bool all_visible;
- enum { BONFERRONI = 1, BH } adjust;
- bool merge;
- bool apa_style;
- bool show_sig;
- };
-
-
-
-static bool
-parse_col_width (struct lexer *lexer, const char *name, double *width)
-{
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "DEFAULT"))
- *width = SYSMIS;
- else if (lex_force_num_range_closed (lexer, name, 0, DBL_MAX))
- {
- *width = lex_number (lexer);
- lex_get (lexer);
- }
- else
- return false;
-
- return true;
-}
-
-static bool
-parse_bool (struct lexer *lexer, bool *b)
-{
- if (lex_match_id (lexer, "NO"))
- *b = false;
- else if (lex_match_id (lexer, "YES"))
- *b = true;
- else
- {
- lex_error_expecting (lexer, "YES", "NO");
- return false;
- }
- return true;
-}
-
-static void
-ctables_chisq_destroy (struct ctables_chisq *chisq)
-{
- free (chisq);
-}
-
-static void
-ctables_pairwise_destroy (struct ctables_pairwise *pairwise)
-{
- free (pairwise);
-}
-
-static void
-ctables_table_destroy (struct ctables_table *t)
-{
- if (!t)
- return;
-
- for (size_t i = 0; i < t->n_sections; i++)
- ctables_section_uninit (&t->sections[i]);
- free (t->sections);
-
- for (size_t i = 0; i < t->n_categories; i++)
- ctables_categories_unref (t->categories[i]);
- free (t->categories);
- free (t->show_empty);
-
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- ctables_axis_destroy (t->axes[a]);
- ctables_stack_uninit (&t->stacks[a]);
- }
- free (t->summary_specs.specs);
-
- struct ctables_value *ctv, *next_ctv;
- HMAP_FOR_EACH_SAFE (ctv, next_ctv, struct ctables_value, node,
- &t->clabels_values_map)
- {
- value_destroy (&ctv->value, var_get_width (t->clabels_example));
- hmap_delete (&t->clabels_values_map, &ctv->node);
- free (ctv);
- }
- hmap_destroy (&t->clabels_values_map);
- free (t->clabels_values);
-
- free (t->sum_vars);
- free (t->caption);
- free (t->corner);
- free (t->title);
- ctables_chisq_destroy (t->chisq);
- ctables_pairwise_destroy (t->pairwise);
- free (t);
-}
-
-static void
-ctables_destroy (struct ctables *ct)
-{
- if (!ct)
- return;
-
- struct ctables_postcompute *pc, *next_pc;
- HMAP_FOR_EACH_SAFE (pc, next_pc, struct ctables_postcompute, hmap_node,
- &ct->postcomputes)
- {
- free (pc->name);
- msg_location_destroy (pc->location);
- ctables_pcexpr_destroy (pc->expr);
- free (pc->label);
- if (pc->specs)
- {
- ctables_summary_spec_set_uninit (pc->specs);
- free (pc->specs);
- }
- hmap_delete (&ct->postcomputes, &pc->hmap_node);
- free (pc);
- }
- hmap_destroy (&ct->postcomputes);
-
- fmt_settings_uninit (&ct->ctables_formats);
- pivot_table_look_unref (ct->look);
- free (ct->zero);
- free (ct->missing);
- free (ct->vlabels);
- for (size_t i = 0; i < ct->n_tables; i++)
- ctables_table_destroy (ct->tables[i]);
- free (ct->tables);
- free (ct);
-}
-
-static bool
-all_strings (struct variable **vars, size_t n_vars,
- const struct ctables_category *cat)
-{
- for (size_t j = 0; j < n_vars; j++)
- if (var_is_numeric (vars[j]))
- {
- msg_at (SE, cat->location,
- _("This category specification may be applied only to string "
- "variables, but this subcommand tries to apply it to "
- "numeric variable %s."),
- var_get_name (vars[j]));
- return false;
- }
- return true;
-}
-
-static bool
-ctables_table_parse_categories (struct lexer *lexer, struct dictionary *dict,
- struct ctables *ct, struct ctables_table *t)
-{
- if (!lex_force_match_id (lexer, "VARIABLES"))
- return false;
- lex_match (lexer, T_EQUALS);
-
- struct variable **vars;
- size_t n_vars;
- if (!parse_variables (lexer, dict, &vars, &n_vars, PV_NO_SCRATCH))
- return false;
-
- const struct fmt_spec *common_format = var_get_print_format (vars[0]);
- for (size_t i = 1; i < n_vars; i++)
- {
- const struct fmt_spec *f = var_get_print_format (vars[i]);
- if (f->type != common_format->type)
- {
- common_format = NULL;
- break;
- }
- }
- bool parse_strings
- = (common_format
- && (fmt_get_category (common_format->type)
- & (FMT_CAT_DATE | FMT_CAT_TIME | FMT_CAT_DATE_COMPONENT)));
-
- struct ctables_categories *c = xmalloc (sizeof *c);
- *c = (struct ctables_categories) { .n_refs = 1 };
-
- bool set_categories = false;
-
- size_t allocated_cats = 0;
- int cats_start_ofs = -1;
- int cats_end_ofs = -1;
- if (lex_match (lexer, T_LBRACK))
- {
- set_categories = true;
- cats_start_ofs = lex_ofs (lexer);
- do
- {
- if (c->n_cats >= allocated_cats)
- c->cats = x2nrealloc (c->cats, &allocated_cats, sizeof *c->cats);
-
- int start_ofs = lex_ofs (lexer);
- struct ctables_category *cat = &c->cats[c->n_cats];
- if (!ctables_table_parse_explicit_category (lexer, dict, ct, cat))
- goto error;
- cat->location = lex_ofs_location (lexer, start_ofs, lex_ofs (lexer) - 1);
- c->n_cats++;
-
- lex_match (lexer, T_COMMA);
- }
- while (!lex_match (lexer, T_RBRACK));
- cats_end_ofs = lex_ofs (lexer) - 1;
- }
-
- struct ctables_category cat = {
- .type = CCT_VALUE,
- .include_missing = false,
- .sort_ascending = true,
- };
- bool show_totals = false;
- char *total_label = NULL;
- bool totals_before = false;
- int key_start_ofs = 0;
- int key_end_ofs = 0;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (!c->n_cats && lex_match_id (lexer, "ORDER"))
- {
- set_categories = true;
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "A"))
- cat.sort_ascending = true;
- else if (lex_match_id (lexer, "D"))
- cat.sort_ascending = false;
- else
- {
- lex_error_expecting (lexer, "A", "D");
- goto error;
- }
- }
- else if (!c->n_cats && lex_match_id (lexer, "KEY"))
- {
- set_categories = true;
- key_start_ofs = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "VALUE"))
- cat.type = CCT_VALUE;
- else if (lex_match_id (lexer, "LABEL"))
- cat.type = CCT_LABEL;
- else
- {
- cat.type = CCT_FUNCTION;
- if (!parse_ctables_summary_function (lexer, &cat.sort_function,
- &cat.weighting, &cat.area))
- goto error;
-
- if (lex_match (lexer, T_LPAREN))
- {
- cat.sort_var = parse_variable (lexer, dict);
- if (!cat.sort_var)
- goto error;
-
- if (cat.sort_function == CTSF_PTILE)
- {
- lex_match (lexer, T_COMMA);
- if (!lex_force_num_range_closed (lexer, "PTILE", 0, 100))
- goto error;
- cat.percentile = lex_number (lexer);
- lex_get (lexer);
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (ctables_function_availability (cat.sort_function)
- == CTFA_SCALE)
- {
- bool UNUSED b = lex_force_match (lexer, T_LPAREN);
- goto error;
- }
- }
- key_end_ofs = lex_ofs (lexer) - 1;
-
- if (cat.type == CCT_FUNCTION)
- {
- lex_ofs_error (lexer, key_start_ofs, key_end_ofs,
- _("Data-dependent sorting is not implemented."));
- goto error;
- }
- }
- else if (!c->n_cats && lex_match_id (lexer, "MISSING"))
- {
- set_categories = true;
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "INCLUDE"))
- cat.include_missing = true;
- else if (lex_match_id (lexer, "EXCLUDE"))
- cat.include_missing = false;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "TOTAL"))
- {
- set_categories = true;
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &show_totals))
- goto error;
- }
- else if (lex_match_id (lexer, "LABEL"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
- free (total_label);
- total_label = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "POSITION"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "BEFORE"))
- totals_before = true;
- else if (lex_match_id (lexer, "AFTER"))
- totals_before = false;
- else
- {
- lex_error_expecting (lexer, "BEFORE", "AFTER");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "EMPTY"))
- {
- lex_match (lexer, T_EQUALS);
-
- bool show_empty;
- if (lex_match_id (lexer, "INCLUDE"))
- show_empty = true;
- else if (lex_match_id (lexer, "EXCLUDE"))
- show_empty = false;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
-
- for (size_t i = 0; i < n_vars; i++)
- t->show_empty[var_get_dict_index (vars[i])] = show_empty;
- }
- else
- {
- if (!c->n_cats)
- lex_error_expecting (lexer, "ORDER", "KEY", "MISSING",
- "TOTAL", "LABEL", "POSITION", "EMPTY");
- else
- lex_error_expecting (lexer, "TOTAL", "LABEL", "POSITION", "EMPTY");
- goto error;
- }
- }
-
- if (!c->n_cats)
- {
- if (key_start_ofs)
- cat.location = lex_ofs_location (lexer, key_start_ofs, key_end_ofs);
-
- if (c->n_cats >= allocated_cats)
- c->cats = x2nrealloc (c->cats, &allocated_cats, sizeof *c->cats);
- c->cats[c->n_cats++] = cat;
- }
-
- if (show_totals)
- {
- if (c->n_cats >= allocated_cats)
- c->cats = x2nrealloc (c->cats, &allocated_cats, sizeof *c->cats);
-
- struct ctables_category *totals;
- if (totals_before)
- {
- insert_element (c->cats, c->n_cats, sizeof *c->cats, 0);
- totals = &c->cats[0];
- }
- else
- totals = &c->cats[c->n_cats];
- c->n_cats++;
-
- *totals = (struct ctables_category) {
- .type = CCT_TOTAL,
- .total_label = total_label ? total_label : xstrdup (_("Total")),
- };
- }
-
- struct ctables_category *subtotal = NULL;
- for (size_t i = totals_before ? 0 : c->n_cats;
- totals_before ? i < c->n_cats : i-- > 0;
- totals_before ? i++ : 0)
- {
- struct ctables_category *cat = &c->cats[i];
- switch (cat->type)
- {
- case CCT_NUMBER:
- case CCT_STRING:
- case CCT_NRANGE:
- case CCT_SRANGE:
- case CCT_MISSING:
- case CCT_OTHERNM:
- cat->subtotal = subtotal;
- break;
-
- case CCT_POSTCOMPUTE:
- break;
-
- case CCT_SUBTOTAL:
- subtotal = cat;
- break;
-
- case CCT_TOTAL:
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- case CCT_EXCLUDED_MISSING:
- break;
- }
- }
-
- if (cats_start_ofs != -1)
- {
- for (size_t i = 0; i < c->n_cats; i++)
- {
- struct ctables_category *cat = &c->cats[i];
- switch (cat->type)
- {
- case CCT_POSTCOMPUTE:
- cat->parse_format = parse_strings ? common_format->type : FMT_F;
- struct msg_location *cats_location
- = lex_ofs_location (lexer, cats_start_ofs, cats_end_ofs);
- bool ok = ctables_recursive_check_postcompute (
- dict, cat->pc->expr, cat, c, cats_location);
- msg_location_destroy (cats_location);
- if (!ok)
- goto error;
- break;
-
- case CCT_NUMBER:
- case CCT_NRANGE:
- for (size_t j = 0; j < n_vars; j++)
- if (var_is_alpha (vars[j]))
- {
- msg_at (SE, cat->location,
- _("This category specification may be applied "
- "only to numeric variables, but this "
- "subcommand tries to apply it to string "
- "variable %s."),
- var_get_name (vars[j]));
- goto error;
- }
- break;
-
- case CCT_STRING:
- if (parse_strings)
- {
- double n;
- if (!parse_category_string (cat->location, cat->string, dict,
- common_format->type, &n))
- goto error;
-
- ss_dealloc (&cat->string);
-
- cat->type = CCT_NUMBER;
- cat->number = n;
- }
- else if (!all_strings (vars, n_vars, cat))
- goto error;
- break;
-
- case CCT_SRANGE:
- if (parse_strings)
- {
- double n[2];
-
- if (!cat->srange[0].string)
- n[0] = -DBL_MAX;
- else if (!parse_category_string (cat->location,
- cat->srange[0], dict,
- common_format->type, &n[0]))
- goto error;
-
- if (!cat->srange[1].string)
- n[1] = DBL_MAX;
- else if (!parse_category_string (cat->location,
- cat->srange[1], dict,
- common_format->type, &n[1]))
- goto error;
-
- ss_dealloc (&cat->srange[0]);
- ss_dealloc (&cat->srange[1]);
-
- cat->type = CCT_NRANGE;
- cat->nrange[0] = n[0];
- cat->nrange[1] = n[1];
- }
- else if (!all_strings (vars, n_vars, cat))
- goto error;
- break;
-
- case CCT_MISSING:
- case CCT_OTHERNM:
- case CCT_SUBTOTAL:
- case CCT_TOTAL:
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- case CCT_EXCLUDED_MISSING:
- break;
- }
- }
- }
-
- if (set_categories)
- for (size_t i = 0; i < n_vars; i++)
- {
- struct ctables_categories **cp
- = &t->categories[var_get_dict_index (vars[i])];
- ctables_categories_unref (*cp);
- *cp = c;
- c->n_refs++;
- }
-
- ctables_categories_unref (c);
- free (vars);
- return true;
-
-error:
- ctables_categories_unref (c);
- free (vars);
- return false;
-}
-\f
-
-struct merge_item
- {
- const struct ctables_summary_spec_set *set;
- size_t ofs;
- };
-
-static int
-merge_item_compare_3way (const struct merge_item *a, const struct merge_item *b)
-{
- const struct ctables_summary_spec *as = &a->set->specs[a->ofs];
- const struct ctables_summary_spec *bs = &b->set->specs[b->ofs];
- if (as->function != bs->function)
- return as->function > bs->function ? 1 : -1;
- else if (as->weighting != bs->weighting)
- return as->weighting > bs->weighting ? 1 : -1;
- else if (as->calc_area != bs->calc_area)
- return as->calc_area > bs->calc_area ? 1 : -1;
- else if (as->percentile != bs->percentile)
- return as->percentile < bs->percentile ? 1 : -1;
-
- const char *as_label = as->label ? as->label : "";
- const char *bs_label = bs->label ? bs->label : "";
- return strcmp (as_label, bs_label);
-}
-
-static void
-ctables_table_add_section (struct ctables_table *t, enum pivot_axis_type a,
- size_t ix[PIVOT_N_AXES])
-{
- if (a < PIVOT_N_AXES)
- {
- size_t limit = MAX (t->stacks[a].n, 1);
- for (ix[a] = 0; ix[a] < limit; ix[a]++)
- ctables_table_add_section (t, a + 1, ix);
- }
- else
- {
- struct ctables_section *s = &t->sections[t->n_sections++];
- *s = (struct ctables_section) {
- .table = t,
- .cells = HMAP_INITIALIZER (s->cells),
- };
- for (a = 0; a < PIVOT_N_AXES; a++)
- if (t->stacks[a].n)
- {
- struct ctables_nest *nest = &t->stacks[a].nests[ix[a]];
- s->nests[a] = nest;
- s->occurrences[a] = xnmalloc (nest->n, sizeof *s->occurrences[a]);
- for (size_t i = 0; i < nest->n; i++)
- hmap_init (&s->occurrences[a][i]);
- }
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- hmap_init (&s->areas[at]);
- }
-}
-
-static char *
-ctables_format (double d, const struct fmt_spec *format,
- const struct fmt_settings *settings)
-{
- const union value v = { .f = d };
- char *s = data_out_stretchy (&v, "UTF-8", format, settings, NULL);
-
- /* The custom-currency specifications for NEQUAL, PAREN, and PCTPAREN don't
- produce the results we want for negative numbers, putting the negative
- sign in the wrong spot, before the prefix instead of after it. We can't,
- in fact, produce the desired results using a custom-currency
- specification. Instead, we postprocess the output, moving the negative
- sign into place:
-
- NEQUAL: "-N=3" => "N=-3"
- PAREN: "-(3)" => "(-3)"
- PCTPAREN: "-(3%)" => "(-3%)"
-
- This transformation doesn't affect NEGPAREN. */
- char *minus_src = strchr (s, '-');
- if (minus_src && (minus_src == s || minus_src[-1] != 'E'))
- {
- char *n_equals = strstr (s, "N=");
- char *lparen = strchr (s, '(');
- char *minus_dst = n_equals ? n_equals + 1 : lparen;
- if (minus_dst)
- move_element (s, minus_dst - s + 1, 1, minus_src - s, minus_dst - s);
- }
- return s;
-}
-
-static bool
-all_hidden_vlabels (const struct ctables_table *t, enum pivot_axis_type a)
-{
- for (size_t i = 0; i < t->stacks[a].n; i++)
- {
- struct ctables_nest *nest = &t->stacks[a].nests[i];
- if (nest->n != 1 || nest->scale_idx != 0)
- return false;
-
- enum ctables_vlabel vlabel
- = t->ctables->vlabels[var_get_dict_index (nest->vars[0])];
- if (vlabel != CTVL_NONE)
- return false;
- }
- return true;
-}
-
-static int
-compare_ints_3way (int a, int b)
-{
- return a < b ? -1 : a > b;
-}
-
-static int
-ctables_cell_compare_leaf_3way (const void *a_, const void *b_,
- const void *aux UNUSED)
-{
- struct ctables_cell *const *ap = a_;
- struct ctables_cell *const *bp = b_;
- const struct ctables_cell *a = *ap;
- const struct ctables_cell *b = *bp;
-
- if (a == b)
- {
- assert (a_ == b_);
- return 0;
- }
-
- for (enum pivot_axis_type axis = 0; axis < PIVOT_N_AXES; axis++)
- {
- int cmp = compare_ints_3way (a->axes[axis].leaf, b->axes[axis].leaf);
- if (cmp)
- return cmp;
- }
-
- const struct ctables_value *a_ctv = ctables_value_find (a);
- const struct ctables_value *b_ctv = ctables_value_find (b);
- if (a_ctv && b_ctv)
- {
- int cmp = compare_ints_3way (a_ctv->leaf, b_ctv->leaf);
- if (cmp)
- return cmp;
- }
- else
- assert (!a_ctv && !b_ctv);
- return 0;
-}
-
-static void
-ctables_table_output (struct ctables *ct, struct ctables_table *t)
-{
- struct pivot_table *pt = pivot_table_create__ (
- (t->title
- ? pivot_value_new_user_text (t->title, SIZE_MAX)
- : pivot_value_new_text (N_("Custom Tables"))),
- "Custom Tables");
- if (t->caption)
- pivot_table_set_caption (
- pt, pivot_value_new_user_text (t->caption, SIZE_MAX));
- if (t->corner)
- pivot_table_set_corner_text (
- pt, pivot_value_new_user_text (t->corner, SIZE_MAX));
-
- bool summary_dimension = (t->summary_axis != t->slabels_axis
- || (!t->slabels_visible
- && t->summary_specs.n > 1));
- if (summary_dimension)
- {
- struct pivot_dimension *d = pivot_dimension_create (
- pt, t->slabels_axis, N_("Statistics"));
- const struct ctables_summary_spec_set *specs = &t->summary_specs;
- if (!t->slabels_visible)
- d->hide_all_labels = true;
- for (size_t i = 0; i < specs->n; i++)
- pivot_category_create_leaf (
- d->root, ctables_summary_label (&specs->specs[i], t->cilevel));
- }
-
- bool categories_dimension = t->clabels_example != NULL;
- if (categories_dimension)
- {
- struct pivot_dimension *d = pivot_dimension_create (
- pt, t->label_axis[t->clabels_from_axis],
- t->clabels_from_axis == PIVOT_AXIS_ROW
- ? N_("Row Categories")
- : N_("Column Categories"));
- const struct variable *var = t->clabels_example;
- const struct ctables_categories *c = t->categories[var_get_dict_index (var)];
- for (size_t i = 0; i < t->n_clabels_values; i++)
- {
- const struct ctables_value *value = t->clabels_values[i];
- const struct ctables_category *cat = ctables_categories_match (c, &value->value, var);
- assert (cat != NULL);
- pivot_category_create_leaf (
- d->root, ctables_category_create_value_label (c, cat,
- t->clabels_example,
- &value->value));
- }
- }
-
- pivot_table_set_look (pt, ct->look);
- struct pivot_dimension *d[PIVOT_N_AXES];
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- static const char *names[] = {
- [PIVOT_AXIS_ROW] = N_("Rows"),
- [PIVOT_AXIS_COLUMN] = N_("Columns"),
- [PIVOT_AXIS_LAYER] = N_("Layers"),
- };
- d[a] = (t->axes[a] || a == t->summary_axis
- ? pivot_dimension_create (pt, a, names[a])
- : NULL);
- if (!d[a])
- continue;
-
- assert (t->axes[a]);
-
- for (size_t i = 0; i < t->stacks[a].n; i++)
- {
- struct ctables_nest *nest = &t->stacks[a].nests[i];
- struct ctables_section **sections = xnmalloc (t->n_sections,
- sizeof *sections);
- size_t n_sections = 0;
-
- size_t n_total_cells = 0;
- size_t max_depth = 0;
- for (size_t j = 0; j < t->n_sections; j++)
- if (t->sections[j].nests[a] == nest)
- {
- struct ctables_section *s = &t->sections[j];
- sections[n_sections++] = s;
- n_total_cells += hmap_count (&s->cells);
-
- size_t depth = s->nests[a]->n;
- max_depth = MAX (depth, max_depth);
- }
-
- struct ctables_cell **sorted = xnmalloc (n_total_cells,
- sizeof *sorted);
- size_t n_sorted = 0;
-
- for (size_t j = 0; j < n_sections; j++)
- {
- struct ctables_section *s = sections[j];
-
- struct ctables_cell *cell;
- HMAP_FOR_EACH (cell, struct ctables_cell, node, &s->cells)
- if (!cell->hide)
- sorted[n_sorted++] = cell;
- assert (n_sorted <= n_total_cells);
- }
-
- struct ctables_cell_sort_aux aux = { .nest = nest, .a = a };
- sort (sorted, n_sorted, sizeof *sorted, ctables_cell_compare_3way, &aux);
-
- struct ctables_level
- {
- enum ctables_level_type
- {
- CTL_VAR, /* Variable label for nest->vars[var_idx]. */
- CTL_CATEGORY, /* Category for nest->vars[var_idx]. */
- CTL_SUMMARY, /* Summary functions. */
- }
- type;
-
- enum settings_value_show vlabel; /* CTL_VAR only. */
- size_t var_idx;
- };
- struct ctables_level *levels = xnmalloc (1 + 2 * max_depth, sizeof *levels);
- size_t n_levels = 0;
- for (size_t k = 0; k < nest->n; k++)
- {
- enum ctables_vlabel vlabel = ct->vlabels[var_get_dict_index (nest->vars[k])];
- if (vlabel == CTVL_NONE && nest->scale_idx == k)
- vlabel = CTVL_NAME;
- if (vlabel != CTVL_NONE)
- {
- levels[n_levels++] = (struct ctables_level) {
- .type = CTL_VAR,
- .vlabel = (enum settings_value_show) vlabel,
- .var_idx = k,
- };
- }
-
- if (nest->scale_idx != k
- && (k != nest->n - 1 || t->label_axis[a] == a))
- {
- levels[n_levels++] = (struct ctables_level) {
- .type = CTL_CATEGORY,
- .var_idx = k,
- };
- }
- }
-
- if (!summary_dimension && a == t->slabels_axis)
- {
- levels[n_levels++] = (struct ctables_level) {
- .type = CTL_SUMMARY,
- .var_idx = SIZE_MAX,
- };
- }
-
- /* Pivot categories:
-
- - variable label for nest->vars[0], if vlabel != CTVL_NONE
- - category for nest->vars[0], if nest->scale_idx != 0
- - variable label for nest->vars[1], if vlabel != CTVL_NONE
- - category for nest->vars[1], if nest->scale_idx != 1
- ...
- - variable label for nest->vars[n - 1], if vlabel != CTVL_NONE
- - category for nest->vars[n - 1], if t->label_axis[a] == a && nest->scale_idx != n - 1.
- - summary function, if 'a == t->slabels_axis && a ==
- t->summary_axis'.
-
- Additional dimensions:
-
- - If 'a == t->slabels_axis && a != t->summary_axis', add a summary
- dimension.
- - If 't->label_axis[b] == a' for some 'b != a', add a category
- dimension to 'a'.
- */
-
-
- struct pivot_category **groups = xnmalloc (1 + 2 * max_depth, sizeof *groups);
- int prev_leaf = 0;
- for (size_t j = 0; j < n_sorted; j++)
- {
- struct ctables_cell *cell = sorted[j];
- struct ctables_cell *prev = j > 0 ? sorted[j - 1] : NULL;
-
- size_t n_common = 0;
- if (j > 0)
- {
- for (; n_common < n_levels; n_common++)
- {
- const struct ctables_level *level = &levels[n_common];
- if (level->type == CTL_CATEGORY)
- {
- size_t var_idx = level->var_idx;
- const struct ctables_category *c = cell->axes[a].cvs[var_idx].category;
- if (prev->axes[a].cvs[var_idx].category != c)
- break;
- else if (c->type != CCT_SUBTOTAL
- && c->type != CCT_TOTAL
- && c->type != CCT_POSTCOMPUTE
- && !value_equal (&prev->axes[a].cvs[var_idx].value,
- &cell->axes[a].cvs[var_idx].value,
- var_get_type (nest->vars[var_idx])))
- break;
- }
- }
- }
-
- for (size_t k = n_common; k < n_levels; k++)
- {
- const struct ctables_level *level = &levels[k];
- struct pivot_category *parent = k ? groups[k - 1] : d[a]->root;
- if (level->type == CTL_SUMMARY)
- {
- assert (k == n_levels - 1);
-
- const struct ctables_summary_spec_set *specs = &t->summary_specs;
- for (size_t m = 0; m < specs->n; m++)
- {
- int leaf = pivot_category_create_leaf (
- parent, ctables_summary_label (&specs->specs[m],
- t->cilevel));
- if (!m)
- prev_leaf = leaf;
- }
- }
- else
- {
- const struct variable *var = nest->vars[level->var_idx];
- struct pivot_value *label;
- if (level->type == CTL_VAR)
- {
- label = pivot_value_new_variable (var);
- label->variable.show = level->vlabel;
- }
- else if (level->type == CTL_CATEGORY)
- {
- const struct ctables_cell_value *cv = &cell->axes[a].cvs[level->var_idx];
- label = ctables_category_create_value_label (
- t->categories[var_get_dict_index (var)],
- cv->category, var, &cv->value);
- }
- else
- NOT_REACHED ();
-
- if (k == n_levels - 1)
- prev_leaf = pivot_category_create_leaf (parent, label);
- else
- groups[k] = pivot_category_create_group__ (parent, label);
- }
- }
-
- cell->axes[a].leaf = prev_leaf;
- }
- free (sorted);
- free (groups);
- free (levels);
- free (sections);
-
- }
-
- d[a]->hide_all_labels = all_hidden_vlabels (t, a);
- }
-
- {
- size_t n_total_cells = 0;
- for (size_t j = 0; j < t->n_sections; j++)
- n_total_cells += hmap_count (&t->sections[j].cells);
-
- struct ctables_cell **sorted = xnmalloc (n_total_cells, sizeof *sorted);
- size_t n_sorted = 0;
- for (size_t j = 0; j < t->n_sections; j++)
- {
- const struct ctables_section *s = &t->sections[j];
- struct ctables_cell *cell;
- HMAP_FOR_EACH (cell, struct ctables_cell, node, &s->cells)
- if (!cell->hide)
- sorted[n_sorted++] = cell;
- }
- assert (n_sorted <= n_total_cells);
- sort (sorted, n_sorted, sizeof *sorted, ctables_cell_compare_leaf_3way,
- NULL);
- size_t ids[N_CTATS];
- memset (ids, 0, sizeof ids);
- for (size_t j = 0; j < n_sorted; j++)
- {
- struct ctables_cell *cell = sorted[j];
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- {
- struct ctables_area *area = cell->areas[at];
- if (!area->sequence)
- area->sequence = ++ids[at];
- }
- }
-
- free (sorted);
- }
-
- for (size_t i = 0; i < t->n_sections; i++)
- {
- struct ctables_section *s = &t->sections[i];
-
- struct ctables_cell *cell;
- HMAP_FOR_EACH (cell, struct ctables_cell, node, &s->cells)
- {
- if (cell->hide)
- continue;
-
- const struct ctables_value *ctv = ctables_value_find (cell);
- const struct ctables_nest *specs_nest = s->nests[t->summary_axis];
- const struct ctables_summary_spec_set *specs = &specs_nest->specs[cell->sv];
- for (size_t j = 0; j < specs->n; j++)
- {
- size_t dindexes[5];
- size_t n_dindexes = 0;
-
- if (summary_dimension)
- dindexes[n_dindexes++] = specs->specs[j].axis_idx;
-
- if (ctv)
- dindexes[n_dindexes++] = ctv->leaf;
-
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- if (d[a])
- {
- int leaf = cell->axes[a].leaf;
- if (a == t->summary_axis && !summary_dimension)
- leaf += specs->specs[j].axis_idx;
- dindexes[n_dindexes++] = leaf;
- }
-
- const struct ctables_summary_spec *ss = &specs->specs[j];
-
- struct fmt_spec format = specs->specs[j].format;
- bool is_ctables_format = ss->is_ctables_format;
- double d = (cell->postcompute
- ? ctables_cell_calculate_postcompute (
- s, cell, ss, &format, &is_ctables_format, j)
- : ctables_summary_value (cell->areas,
- &cell->summaries[j], ss));
-
- struct pivot_value *value;
- if (ct->hide_threshold != 0
- && d < ct->hide_threshold
- && ss->function == CTSF_COUNT)
- {
- value = pivot_value_new_user_text_nocopy (
- xasprintf ("<%d", ct->hide_threshold));
- }
- else if (d == 0 && ct->zero)
- value = pivot_value_new_user_text (ct->zero, SIZE_MAX);
- else if (d == SYSMIS && ct->missing)
- value = pivot_value_new_user_text (ct->missing, SIZE_MAX);
- else if (is_ctables_format)
- value = pivot_value_new_user_text_nocopy (
- ctables_format (d, &format, &ct->ctables_formats));
- else
- {
- value = pivot_value_new_number (d);
- value->numeric.format = format;
- }
- /* XXX should text values be right-justified? */
- pivot_table_put (pt, dindexes, n_dindexes, value);
- }
- }
- }
-
- pivot_table_submit (pt);
-}
-
-static bool
-ctables_check_label_position (struct ctables_table *t, struct lexer *lexer,
- enum pivot_axis_type a)
-{
- enum pivot_axis_type label_pos = t->label_axis[a];
- if (label_pos == a)
- return true;
-
- const struct ctables_stack *stack = &t->stacks[a];
- if (!stack->n)
- return true;
-
- const struct ctables_nest *n0 = &stack->nests[0];
- if (n0->n == 0)
- {
- assert (stack->n == 1);
- return true;
- }
-
- const struct variable *v0 = n0->vars[n0->n - 1];
- struct ctables_categories *c0 = t->categories[var_get_dict_index (v0)];
- t->clabels_example = v0;
-
- for (size_t i = 0; i < c0->n_cats; i++)
- if (c0->cats[i].type == CCT_FUNCTION)
- {
- msg (SE, _("Category labels may not be moved to another axis when "
- "sorting by a summary function."));
- lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
- _("This syntax moves category labels to another axis."));
- msg_at (SN, c0->cats[i].location,
- _("This syntax requests sorting by a summary function."));
- return false;
- }
-
- for (size_t i = 0; i < stack->n; i++)
- {
- const struct ctables_nest *ni = &stack->nests[i];
- assert (ni->n > 0);
- const struct variable *vi = ni->vars[ni->n - 1];
- if (n0->n - 1 == ni->scale_idx)
- {
- msg (SE, _("To move category labels from one axis to another, "
- "the variables whose labels are to be moved must be "
- "categorical, but %s is scale."), var_get_name (vi));
- lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
- _("This syntax moves category labels to another axis."));
- return false;
- }
- }
-
- for (size_t i = 1; i < stack->n; i++)
- {
- const struct ctables_nest *ni = &stack->nests[i];
- assert (ni->n > 0);
- const struct variable *vi = ni->vars[ni->n - 1];
- struct ctables_categories *ci = t->categories[var_get_dict_index (vi)];
-
- if (var_get_width (v0) != var_get_width (vi))
- {
- msg (SE, _("To move category labels from one axis to another, "
- "the variables whose labels are to be moved must all "
- "have the same width, but %s has width %d and %s has "
- "width %d."),
- var_get_name (v0), var_get_width (v0),
- var_get_name (vi), var_get_width (vi));
- lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
- _("This syntax moves category labels to another axis."));
- return false;
- }
- if (!val_labs_equal (var_get_value_labels (v0),
- var_get_value_labels (vi)))
- {
- msg (SE, _("To move category labels from one axis to another, "
- "the variables whose labels are to be moved must all "
- "have the same value labels, but %s and %s have "
- "different value labels."),
- var_get_name (v0), var_get_name (vi));
- lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
- _("This syntax moves category labels to another axis."));
- return false;
- }
- if (!ctables_categories_equal (c0, ci))
- {
- msg (SE, _("To move category labels from one axis to another, "
- "the variables whose labels are to be moved must all "
- "have the same category specifications, but %s and %s "
- "have different category specifications."),
- var_get_name (v0), var_get_name (vi));
- lex_ofs_msg (lexer, SN, t->clabels_start_ofs, t->clabels_end_ofs,
- _("This syntax moves category labels to another axis."));
- return false;
- }
- }
-
- return true;
-}
-
-static size_t
-add_sum_var (struct variable *var,
- struct variable ***sum_vars, size_t *n, size_t *allocated)
-{
- for (size_t i = 0; i < *n; i++)
- if (var == (*sum_vars)[i])
- return i;
-
- if (*n >= *allocated)
- *sum_vars = x2nrealloc (*sum_vars, allocated, sizeof **sum_vars);
- (*sum_vars)[*n] = var;
- return (*n)++;
-}
-
-static enum ctables_area_type
-rotate_area (enum ctables_area_type area)
-{
- return area;
- switch (area)
- {
- case CTAT_TABLE:
- case CTAT_LAYER:
- case CTAT_SUBTABLE:
- return area;
-
- case CTAT_LAYERROW:
- return CTAT_LAYERCOL;
-
- case CTAT_LAYERCOL:
- return CTAT_LAYERROW;
-
- case CTAT_ROW:
- return CTAT_COL;
-
- case CTAT_COL:
- return CTAT_ROW;
- }
-
- NOT_REACHED ();
-}
-
-static void
-enumerate_sum_vars (const struct ctables_axis *a,
- struct variable ***sum_vars, size_t *n, size_t *allocated)
-{
- if (!a)
- return;
-
- switch (a->op)
- {
- case CTAO_VAR:
- for (size_t i = 0; i < N_CSVS; i++)
- for (size_t j = 0; j < a->specs[i].n; j++)
- {
- struct ctables_summary_spec *spec = &a->specs[i].specs[j];
- if (spec->function == CTSF_areaPCT_SUM)
- spec->sum_var_idx = add_sum_var (a->var, sum_vars, n, allocated);
- }
- break;
-
- case CTAO_STACK:
- case CTAO_NEST:
- for (size_t i = 0; i < 2; i++)
- enumerate_sum_vars (a->subs[i], sum_vars, n, allocated);
- break;
- }
-}
-
-static bool
-ctables_prepare_table (struct ctables_table *t, struct lexer *lexer)
-{
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- if (t->axes[a])
- {
- t->stacks[a] = enumerate_fts (a, t->axes[a]);
-
- for (size_t j = 0; j < t->stacks[a].n; j++)
- {
- struct ctables_nest *nest = &t->stacks[a].nests[j];
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- {
- nest->areas[at] = xmalloc (nest->n * sizeof *nest->areas[at]);
- nest->n_areas[at] = 0;
-
- enum pivot_axis_type ata, atb;
- if (at == CTAT_ROW || at == CTAT_LAYERROW)
- {
- ata = PIVOT_AXIS_ROW;
- atb = PIVOT_AXIS_COLUMN;
- }
- else if (at == CTAT_COL || at == CTAT_LAYERCOL)
- {
- ata = PIVOT_AXIS_COLUMN;
- atb = PIVOT_AXIS_ROW;
- }
-
- if (at == CTAT_LAYER
- ? a != PIVOT_AXIS_LAYER && t->label_axis[a] == PIVOT_AXIS_LAYER
- : at == CTAT_LAYERCOL || at == CTAT_LAYERROW
- ? a == atb && t->label_axis[a] != a
- : false)
- {
- for (size_t k = nest->n - 1; k < nest->n; k--)
- if (k != nest->scale_idx)
- {
- nest->areas[at][nest->n_areas[at]++] = k;
- break;
- }
- continue;
- }
-
- if (at == CTAT_LAYER ? a != PIVOT_AXIS_LAYER
- : at == CTAT_LAYERROW || at == CTAT_LAYERCOL ? a == atb
- : at == CTAT_TABLE ? true
- : false)
- continue;
-
- for (size_t k = 0; k < nest->n; k++)
- if (k != nest->scale_idx)
- nest->areas[at][nest->n_areas[at]++] = k;
-
- int n_drop;
- switch (at)
- {
- case CTAT_SUBTABLE:
-#define L PIVOT_AXIS_LAYER
- n_drop = (t->clabels_from_axis == L ? a != L
- : t->clabels_to_axis == L ? (t->clabels_from_axis == a ? -1 : a != L)
- : t->clabels_from_axis == a ? 2
- : 0);
-#undef L
- break;
-
- case CTAT_LAYERROW:
- case CTAT_LAYERCOL:
- n_drop = a == ata && t->label_axis[ata] == atb;
- break;
-
- case CTAT_ROW:
- case CTAT_COL:
- n_drop = (a == ata ? t->label_axis[ata] == atb
- : a != atb ? 0
- : t->clabels_from_axis == atb ? -1
- : t->clabels_to_axis != atb ? 1
- : 0);
- break;
-
- case CTAT_LAYER:
- case CTAT_TABLE:
- n_drop = 0;
- break;
- }
-
- if (n_drop < 0)
- {
- size_t n = nest->n_areas[at];
- if (n > 1)
- {
- nest->areas[at][n - 2] = nest->areas[at][n - 1];
- nest->n_areas[at]--;
- }
- }
- else
- {
- for (int i = 0; i < n_drop; i++)
- if (nest->n_areas[at] > 0)
- nest->n_areas[at]--;
- }
- }
- }
- }
- else
- {
- struct ctables_nest *nest = xmalloc (sizeof *nest);
- *nest = (struct ctables_nest) {
- .n = 0,
- .scale_idx = SIZE_MAX,
- .summary_idx = SIZE_MAX
- };
- t->stacks[a] = (struct ctables_stack) { .nests = nest, .n = 1 };
-
- /* There's no point in moving labels away from an axis that has no
- labels, so avoid dealing with the special cases around that. */
- t->label_axis[a] = a;
- }
-
- struct ctables_stack *stack = &t->stacks[t->summary_axis];
- for (size_t i = 0; i < stack->n; i++)
- {
- struct ctables_nest *nest = &stack->nests[i];
- if (!nest->specs[CSV_CELL].n)
- {
- struct ctables_summary_spec_set *ss = &nest->specs[CSV_CELL];
- ss->specs = xmalloc (sizeof *ss->specs);
- ss->n = 1;
-
- enum ctables_summary_function function
- = ss->is_scale ? CTSF_MEAN : CTSF_COUNT;
-
- if (!ss->var)
- {
- nest->summary_idx = nest->n - 1;
- ss->var = nest->vars[nest->summary_idx];
- }
- *ss->specs = (struct ctables_summary_spec) {
- .function = function,
- .weighting = ss->is_scale ? CTW_EFFECTIVE : CTW_DICTIONARY,
- .format = ctables_summary_default_format (function, ss->var),
- };
-
- ctables_summary_spec_set_clone (&nest->specs[CSV_TOTAL],
- &nest->specs[CSV_CELL]);
- }
- else if (!nest->specs[CSV_TOTAL].n)
- ctables_summary_spec_set_clone (&nest->specs[CSV_TOTAL],
- &nest->specs[CSV_CELL]);
-
- if (t->label_axis[PIVOT_AXIS_ROW] == PIVOT_AXIS_COLUMN
- || t->label_axis[PIVOT_AXIS_COLUMN] == PIVOT_AXIS_ROW)
- {
- for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
- for (size_t i = 0; i < nest->specs[sv].n; i++)
- {
- struct ctables_summary_spec *ss = &nest->specs[sv].specs[i];
- const struct ctables_function_info *cfi =
- &ctables_function_info[ss->function];
- if (cfi->is_area)
- ss->calc_area = rotate_area (ss->calc_area);
- }
- }
-
- if (t->ctables->smissing_listwise)
- {
- struct variable **listwise_vars = NULL;
- size_t n = 0;
- size_t allocated = 0;
-
- for (size_t j = nest->group_head; j < stack->n; j++)
- {
- const struct ctables_nest *other_nest = &stack->nests[j];
- if (other_nest->group_head != nest->group_head)
- break;
-
- if (nest != other_nest && other_nest->scale_idx < other_nest->n)
- {
- if (n >= allocated)
- listwise_vars = x2nrealloc (listwise_vars, &allocated,
- sizeof *listwise_vars);
- listwise_vars[n++] = other_nest->vars[other_nest->scale_idx];
- }
- }
- for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
- {
- if (sv > 0)
- listwise_vars = xmemdup (listwise_vars,
- n * sizeof *listwise_vars);
- nest->specs[sv].listwise_vars = listwise_vars;
- nest->specs[sv].n_listwise_vars = n;
- }
- }
- }
-
- struct ctables_summary_spec_set *merged = &t->summary_specs;
- struct merge_item *items = xnmalloc (N_CSVS * stack->n, sizeof *items);
- size_t n_left = 0;
- for (size_t j = 0; j < stack->n; j++)
- {
- const struct ctables_nest *nest = &stack->nests[j];
- if (nest->n)
- for (enum ctables_summary_variant sv = 0; sv < N_CSVS; sv++)
- items[n_left++] = (struct merge_item) { .set = &nest->specs[sv] };
- }
-
- while (n_left > 0)
- {
- struct merge_item min = items[0];
- for (size_t j = 1; j < n_left; j++)
- if (merge_item_compare_3way (&items[j], &min) < 0)
- min = items[j];
-
- if (merged->n >= merged->allocated)
- merged->specs = x2nrealloc (merged->specs, &merged->allocated,
- sizeof *merged->specs);
- merged->specs[merged->n++] = min.set->specs[min.ofs];
-
- for (size_t j = 0; j < n_left; )
- {
- if (merge_item_compare_3way (&items[j], &min) == 0)
- {
- struct merge_item *item = &items[j];
- item->set->specs[item->ofs++].axis_idx = merged->n - 1;
- if (item->ofs >= item->set->n)
- {
- items[j] = items[--n_left];
- continue;
- }
- }
- j++;
- }
- }
- free (items);
-
- size_t allocated_sum_vars = 0;
- enumerate_sum_vars (t->axes[t->summary_axis],
- &t->sum_vars, &t->n_sum_vars, &allocated_sum_vars);
-
- return (ctables_check_label_position (t, lexer, PIVOT_AXIS_ROW)
- && ctables_check_label_position (t, lexer, PIVOT_AXIS_COLUMN));
-}
-
-static void
-ctables_insert_clabels_values (struct ctables_table *t, const struct ccase *c,
- enum pivot_axis_type a)
-{
- struct ctables_stack *stack = &t->stacks[a];
- for (size_t i = 0; i < stack->n; i++)
- {
- const struct ctables_nest *nest = &stack->nests[i];
- const struct variable *var = nest->vars[nest->n - 1];
- const union value *value = case_data (c, var);
-
- if (var_is_numeric (var) && value->f == SYSMIS)
- continue;
-
- if (ctables_categories_match (t->categories [var_get_dict_index (var)],
- value, var))
- ctables_value_insert (t, value, var_get_width (var));
- }
-}
-
-static void
-ctables_add_category_occurrences (const struct variable *var,
- struct hmap *occurrences,
- const struct ctables_categories *cats)
-{
- const struct val_labs *val_labs = var_get_value_labels (var);
-
- for (size_t i = 0; i < cats->n_cats; i++)
- {
- const struct ctables_category *c = &cats->cats[i];
- switch (c->type)
- {
- case CCT_NUMBER:
- ctables_add_occurrence (var, &(const union value) { .f = c->number },
- occurrences);
- break;
-
- case CCT_STRING:
- {
- int width = var_get_width (var);
- union value value;
- value_init (&value, width);
- value_copy_buf_rpad (&value, width,
- CHAR_CAST (uint8_t *, c->string.string),
- c->string.length, ' ');
- ctables_add_occurrence (var, &value, occurrences);
- value_destroy (&value, width);
- }
- break;
-
- case CCT_NRANGE:
- assert (var_is_numeric (var));
- for (const struct val_lab *vl = val_labs_first (val_labs); vl;
- vl = val_labs_next (val_labs, vl))
- if (vl->value.f >= c->nrange[0] && vl->value.f <= c->nrange[1])
- ctables_add_occurrence (var, &vl->value, occurrences);
- break;
-
- case CCT_SRANGE:
- assert (var_is_alpha (var));
- for (const struct val_lab *vl = val_labs_first (val_labs); vl;
- vl = val_labs_next (val_labs, vl))
- if (in_string_range (&vl->value, var, c->srange))
- ctables_add_occurrence (var, &vl->value, occurrences);
- break;
-
- case CCT_MISSING:
- for (const struct val_lab *vl = val_labs_first (val_labs); vl;
- vl = val_labs_next (val_labs, vl))
- if (var_is_value_missing (var, &vl->value))
- ctables_add_occurrence (var, &vl->value, occurrences);
- break;
-
- case CCT_OTHERNM:
- for (const struct val_lab *vl = val_labs_first (val_labs); vl;
- vl = val_labs_next (val_labs, vl))
- ctables_add_occurrence (var, &vl->value, occurrences);
- break;
-
- case CCT_POSTCOMPUTE:
- break;
-
- case CCT_SUBTOTAL:
- case CCT_TOTAL:
- break;
-
- case CCT_VALUE:
- case CCT_LABEL:
- case CCT_FUNCTION:
- for (const struct val_lab *vl = val_labs_first (val_labs); vl;
- vl = val_labs_next (val_labs, vl))
- if (c->include_missing || !var_is_value_missing (var, &vl->value))
- ctables_add_occurrence (var, &vl->value, occurrences);
- break;
-
- case CCT_EXCLUDED_MISSING:
- break;
- }
- }
-}
-
-static void
-ctables_section_recurse_add_empty_categories (
- struct ctables_section *s,
- const struct ctables_category **cats[PIVOT_N_AXES], struct ccase *c,
- enum pivot_axis_type a, size_t a_idx)
-{
- if (a >= PIVOT_N_AXES)
- ctables_cell_insert__ (s, c, cats);
- else if (!s->nests[a] || a_idx >= s->nests[a]->n)
- ctables_section_recurse_add_empty_categories (s, cats, c, a + 1, 0);
- else
- {
- const struct variable *var = s->nests[a]->vars[a_idx];
- const struct ctables_categories *categories = s->table->categories[
- var_get_dict_index (var)];
- int width = var_get_width (var);
- const struct hmap *occurrences = &s->occurrences[a][a_idx];
- const struct ctables_occurrence *o;
- HMAP_FOR_EACH (o, struct ctables_occurrence, node, occurrences)
- {
- union value *value = case_data_rw (c, var);
- value_destroy (value, width);
- value_clone (value, &o->value, width);
- cats[a][a_idx] = ctables_categories_match (categories, value, var);
- assert (cats[a][a_idx] != NULL);
- ctables_section_recurse_add_empty_categories (s, cats, c, a, a_idx + 1);
- }
-
- for (size_t i = 0; i < categories->n_cats; i++)
- {
- const struct ctables_category *cat = &categories->cats[i];
- if (cat->type == CCT_POSTCOMPUTE)
- {
- cats[a][a_idx] = cat;
- ctables_section_recurse_add_empty_categories (s, cats, c, a, a_idx + 1);
- }
- }
- }
-}
-
-static void
-ctables_section_add_empty_categories (struct ctables_section *s)
-{
- bool show_empty = false;
- for (size_t a = 0; a < PIVOT_N_AXES; a++)
- if (s->nests[a])
- for (size_t k = 0; k < s->nests[a]->n; k++)
- if (k != s->nests[a]->scale_idx)
- {
- const struct variable *var = s->nests[a]->vars[k];
- size_t idx = var_get_dict_index (var);
- const struct ctables_categories *cats = s->table->categories[idx];
- if (s->table->show_empty[idx])
- {
- show_empty = true;
- ctables_add_category_occurrences (var, &s->occurrences[a][k], cats);
- }
- }
- if (!show_empty)
- return;
-
- const struct ctables_category *layer_cats[s->nests[PIVOT_AXIS_LAYER]->n];
- const struct ctables_category *row_cats[s->nests[PIVOT_AXIS_ROW]->n];
- const struct ctables_category *column_cats[s->nests[PIVOT_AXIS_COLUMN]->n];
- const struct ctables_category **cats[PIVOT_N_AXES] =
- {
- [PIVOT_AXIS_LAYER] = layer_cats,
- [PIVOT_AXIS_ROW] = row_cats,
- [PIVOT_AXIS_COLUMN] = column_cats,
- };
- struct ccase *c = case_create (dict_get_proto (s->table->ctables->dict));
- ctables_section_recurse_add_empty_categories (s, cats, c, 0, 0);
- case_unref (c);
-}
-
-static void
-ctables_section_clear (struct ctables_section *s)
-{
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- {
- const struct variable *var = nest->vars[i];
- int width = var_get_width (var);
- struct ctables_occurrence *o, *next;
- struct hmap *map = &s->occurrences[a][i];
- HMAP_FOR_EACH_SAFE (o, next, struct ctables_occurrence, node, map)
- {
- value_destroy (&o->value, width);
- hmap_delete (map, &o->node);
- free (o);
- }
- hmap_shrink (map);
- }
- }
-
- struct ctables_cell *cell, *next_cell;
- HMAP_FOR_EACH_SAFE (cell, next_cell, struct ctables_cell, node, &s->cells)
- {
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- const struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- if (i != nest->scale_idx)
- value_destroy (&cell->axes[a].cvs[i].value,
- var_get_width (nest->vars[i]));
- free (cell->axes[a].cvs);
- }
-
- const struct ctables_nest *ss = s->nests[s->table->summary_axis];
- const struct ctables_summary_spec_set *specs = &ss->specs[cell->sv];
- for (size_t i = 0; i < specs->n; i++)
- ctables_summary_uninit (&cell->summaries[i], &specs->specs[i]);
- free (cell->summaries);
-
- hmap_delete (&s->cells, &cell->node);
- free (cell);
- }
- hmap_shrink (&s->cells);
-
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- {
- struct ctables_area *area, *next_area;
- HMAP_FOR_EACH_SAFE (area, next_area, struct ctables_area, node,
- &s->areas[at])
- {
- free (area->sums);
- hmap_delete (&s->areas[at], &area->node);
- free (area);
- }
- hmap_shrink (&s->areas[at]);
- }
-}
-
-static void
-ctables_section_uninit (struct ctables_section *s)
-{
- ctables_section_clear (s);
-
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- struct ctables_nest *nest = s->nests[a];
- for (size_t i = 0; i < nest->n; i++)
- hmap_destroy (&s->occurrences[a][i]);
- free (s->occurrences[a]);
- }
-
- hmap_destroy (&s->cells);
- for (enum ctables_area_type at = 0; at < N_CTATS; at++)
- hmap_destroy (&s->areas[at]);
-}
-
-static void
-ctables_table_clear (struct ctables_table *t)
-{
- for (size_t i = 0; i < t->n_sections; i++)
- ctables_section_clear (&t->sections[i]);
-
- if (t->clabels_example)
- {
- int width = var_get_width (t->clabels_example);
- struct ctables_value *value, *next_value;
- HMAP_FOR_EACH_SAFE (value, next_value, struct ctables_value, node,
- &t->clabels_values_map)
- {
- value_destroy (&value->value, width);
- hmap_delete (&t->clabels_values_map, &value->node);
- free (value);
- }
- hmap_shrink (&t->clabels_values_map);
-
- free (t->clabels_values);
- t->clabels_values = NULL;
- t->n_clabels_values = 0;
- }
-}
-
-static bool
-ctables_execute (struct dataset *ds, struct casereader *input,
- struct ctables *ct)
-{
- for (size_t i = 0; i < ct->n_tables; i++)
- {
- struct ctables_table *t = ct->tables[i];
- t->sections = xnmalloc (MAX (1, t->stacks[PIVOT_AXIS_ROW].n) *
- MAX (1, t->stacks[PIVOT_AXIS_COLUMN].n) *
- MAX (1, t->stacks[PIVOT_AXIS_LAYER].n),
- sizeof *t->sections);
- size_t ix[PIVOT_N_AXES];
- ctables_table_add_section (t, 0, ix);
- }
-
- struct dictionary *dict = dataset_dict (ds);
-
- bool splitting = dict_get_split_type (dict) == SPLIT_SEPARATE;
- struct casegrouper *grouper
- = (splitting
- ? casegrouper_create_splits (input, dict)
- : casegrouper_create_vars (input, NULL, 0));
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- if (splitting)
- output_split_file_values_peek (ds, group);
-
- bool warn_on_invalid = true;
- for (struct ccase *c = casereader_read (group); c;
- case_unref (c), c = casereader_read (group))
- {
- double d_weight = dict_get_rounded_case_weight (dict, c, &warn_on_invalid);
- double e_weight = (ct->e_weight
- ? var_force_valid_weight (ct->e_weight,
- case_num (c, ct->e_weight),
- &warn_on_invalid)
- : d_weight);
- double weight[] = {
- [CTW_DICTIONARY] = d_weight,
- [CTW_EFFECTIVE] = e_weight,
- [CTW_UNWEIGHTED] = 1.0,
- };
-
- for (size_t i = 0; i < ct->n_tables; i++)
- {
- struct ctables_table *t = ct->tables[i];
-
- for (size_t j = 0; j < t->n_sections; j++)
- ctables_cell_insert (&t->sections[j], c, weight);
-
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- if (t->label_axis[a] != a)
- ctables_insert_clabels_values (t, c, a);
- }
- }
- casereader_destroy (group);
-
- for (size_t i = 0; i < ct->n_tables; i++)
- {
- struct ctables_table *t = ct->tables[i];
-
- if (t->clabels_example)
- ctables_sort_clabels_values (t);
-
- for (size_t j = 0; j < t->n_sections; j++)
- ctables_section_add_empty_categories (&t->sections[j]);
-
- ctables_table_output (ct, t);
- ctables_table_clear (t);
- }
- }
- return casegrouper_destroy (grouper);
-}
-\f
-static struct ctables_postcompute *
-ctables_find_postcompute (struct ctables *ct, const char *name)
-{
- struct ctables_postcompute *pc;
- HMAP_FOR_EACH_WITH_HASH (pc, struct ctables_postcompute, hmap_node,
- utf8_hash_case_string (name, 0), &ct->postcomputes)
- if (!utf8_strcasecmp (pc->name, name))
- return pc;
- return NULL;
-}
-
-static bool
-ctables_parse_pcompute (struct lexer *lexer, struct dictionary *dict,
- struct ctables *ct)
-{
- int pcompute_start = lex_ofs (lexer) - 1;
-
- if (!lex_match (lexer, T_AND))
- {
- lex_error_expecting (lexer, "&");
- return false;
- }
- if (!lex_force_id (lexer))
- return false;
-
- char *name = ss_xstrdup (lex_tokss (lexer));
-
- lex_get (lexer);
- if (!lex_force_match_phrase (lexer, "=EXPR("))
- {
- free (name);
- return false;
- }
-
- int expr_start = lex_ofs (lexer);
- struct ctables_pcexpr *expr = ctables_pcexpr_parse_add (lexer, dict);
- int expr_end = lex_ofs (lexer) - 1;
- if (!expr || !lex_force_match (lexer, T_RPAREN))
- {
- ctables_pcexpr_destroy (expr);
- free (name);
- return false;
- }
- int pcompute_end = lex_ofs (lexer) - 1;
-
- struct msg_location *location = lex_ofs_location (lexer, pcompute_start,
- pcompute_end);
-
- struct ctables_postcompute *pc = ctables_find_postcompute (ct, name);
- if (pc)
- {
- msg_at (SW, location, _("New definition of &%s will override the "
- "previous definition."),
- pc->name);
- msg_at (SN, pc->location, _("This is the previous definition."));
-
- ctables_pcexpr_destroy (pc->expr);
- msg_location_destroy (pc->location);
- free (name);
- }
- else
- {
- pc = xmalloc (sizeof *pc);
- *pc = (struct ctables_postcompute) { .name = name };
- hmap_insert (&ct->postcomputes, &pc->hmap_node,
- utf8_hash_case_string (pc->name, 0));
- }
- pc->expr = expr;
- pc->location = location;
- if (!pc->label)
- pc->label = lex_ofs_representation (lexer, expr_start, expr_end);
- return true;
-}
-
-static bool
-ctables_parse_pproperties_format (struct lexer *lexer,
- struct ctables_summary_spec_set *sss)
-{
- *sss = (struct ctables_summary_spec_set) { .n = 0 };
-
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH
- && !(lex_token (lexer) == T_ID
- && (lex_id_match (ss_cstr ("LABEL"), lex_tokss (lexer))
- || lex_id_match (ss_cstr ("HIDESOURCECATS"),
- lex_tokss (lexer)))))
- {
- /* Parse function. */
- enum ctables_summary_function function;
- enum ctables_weighting weighting;
- enum ctables_area_type area;
- if (!parse_ctables_summary_function (lexer, &function, &weighting, &area))
- goto error;
-
- /* Parse percentile. */
- double percentile = 0;
- if (function == CTSF_PTILE)
- {
- if (!lex_force_num_range_closed (lexer, "PTILE", 0, 100))
- goto error;
- percentile = lex_number (lexer);
- lex_get (lexer);
- }
-
- /* Parse format. */
- struct fmt_spec format;
- bool is_ctables_format;
- if (!parse_ctables_format_specifier (lexer, &format, &is_ctables_format))
- goto error;
-
- if (sss->n >= sss->allocated)
- sss->specs = x2nrealloc (sss->specs, &sss->allocated,
- sizeof *sss->specs);
- sss->specs[sss->n++] = (struct ctables_summary_spec) {
- .function = function,
- .weighting = weighting,
- .calc_area = area,
- .user_area = area,
- .percentile = percentile,
- .format = format,
- .is_ctables_format = is_ctables_format,
- };
- }
- return true;
-
-error:
- ctables_summary_spec_set_uninit (sss);
- return false;
-}
-
-static bool
-ctables_parse_pproperties (struct lexer *lexer, struct ctables *ct)
-{
- struct ctables_postcompute **pcs = NULL;
- size_t n_pcs = 0;
- size_t allocated_pcs = 0;
-
- while (lex_match (lexer, T_AND))
- {
- if (!lex_force_id (lexer))
- goto error;
- struct ctables_postcompute *pc
- = ctables_find_postcompute (ct, lex_tokcstr (lexer));
- if (!pc)
- {
- lex_error (lexer, _("Unknown computed category &%s."),
- lex_tokcstr (lexer));
- goto error;
- }
- lex_get (lexer);
-
- if (n_pcs >= allocated_pcs)
- pcs = x2nrealloc (pcs, &allocated_pcs, sizeof *pcs);
- pcs[n_pcs++] = pc;
- }
-
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "LABEL"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- for (size_t i = 0; i < n_pcs; i++)
- {
- free (pcs[i]->label);
- pcs[i]->label = ss_xstrdup (lex_tokss (lexer));
- }
-
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
-
- struct ctables_summary_spec_set sss;
- if (!ctables_parse_pproperties_format (lexer, &sss))
- goto error;
-
- for (size_t i = 0; i < n_pcs; i++)
- {
- if (pcs[i]->specs)
- ctables_summary_spec_set_uninit (pcs[i]->specs);
- else
- pcs[i]->specs = xmalloc (sizeof *pcs[i]->specs);
- ctables_summary_spec_set_clone (pcs[i]->specs, &sss);
- }
- ctables_summary_spec_set_uninit (&sss);
- }
- else if (lex_match_id (lexer, "HIDESOURCECATS"))
- {
- lex_match (lexer, T_EQUALS);
- bool hide_source_cats;
- if (!parse_bool (lexer, &hide_source_cats))
- goto error;
- for (size_t i = 0; i < n_pcs; i++)
- pcs[i]->hide_source_cats = hide_source_cats;
- }
- else
- {
- lex_error_expecting (lexer, "LABEL", "FORMAT", "HIDESOURCECATS");
- goto error;
- }
- }
- free (pcs);
- return true;
-
-error:
- free (pcs);
- return false;
-}
-
-static void
-put_strftime (struct string *out, time_t now, const char *format)
-{
- const struct tm *tm = localtime (&now);
- char value[128];
- strftime (value, sizeof value, format, tm);
- ds_put_cstr (out, value);
-}
-
-static bool
-skip_prefix (struct substring *s, struct substring prefix)
-{
- if (ss_starts_with (*s, prefix))
- {
- ss_advance (s, prefix.length);
- return true;
- }
- else
- return false;
-}
-
-static void
-put_table_expression (struct string *out, struct lexer *lexer,
- struct dictionary *dict, int expr_start, int expr_end)
-{
- size_t nest = 0;
- for (int ofs = expr_start; ofs < expr_end; ofs++)
- {
- const struct token *t = lex_ofs_token (lexer, ofs);
- if (t->type == T_LBRACK)
- nest++;
- else if (t->type == T_RBRACK && nest > 0)
- nest--;
- else if (nest > 0)
- {
- /* Nothing. */
- }
- else if (t->type == T_ID)
- {
- const struct variable *var
- = dict_lookup_var (dict, t->string.string);
- const char *label = var ? var_get_label (var) : NULL;
- ds_put_cstr (out, label ? label : t->string.string);
- }
- else
- {
- if (ofs != expr_start && t->type != T_RPAREN && ds_last (out) != ' ')
- ds_put_byte (out, ' ');
-
- char *repr = lex_ofs_representation (lexer, ofs, ofs);
- ds_put_cstr (out, repr);
- free (repr);
-
- if (ofs + 1 != expr_end && t->type != T_LPAREN)
- ds_put_byte (out, ' ');
- }
- }
-}
-
-static void
-put_title_text (struct string *out, struct substring in, time_t now,
- struct lexer *lexer, struct dictionary *dict,
- int expr_start, int expr_end)
-{
- for (;;)
- {
- size_t chunk = ss_find_byte (in, ')');
- ds_put_substring (out, ss_head (in, chunk));
- ss_advance (&in, chunk);
- if (ss_is_empty (in))
- return;
-
- if (skip_prefix (&in, ss_cstr (")DATE")))
- put_strftime (out, now, "%x");
- else if (skip_prefix (&in, ss_cstr (")TIME")))
- put_strftime (out, now, "%X");
- else if (skip_prefix (&in, ss_cstr (")TABLE")))
- put_table_expression (out, lexer, dict, expr_start, expr_end);
- else
- {
- ds_put_byte (out, ')');
- ss_advance (&in, 1);
- }
- }
-}
-
-int
-cmd_ctables (struct lexer *lexer, struct dataset *ds)
-{
- struct casereader *input = NULL;
-
- struct measure_guesser *mg = measure_guesser_create (ds);
- if (mg)
- {
- input = proc_open (ds);
- measure_guesser_run (mg, input);
- measure_guesser_destroy (mg);
- }
-
- size_t n_vars = dict_get_n_vars (dataset_dict (ds));
- enum ctables_vlabel *vlabels = xnmalloc (n_vars, sizeof *vlabels);
- enum settings_value_show tvars = settings_get_show_variables ();
- for (size_t i = 0; i < n_vars; i++)
- vlabels[i] = (enum ctables_vlabel) tvars;
-
- struct pivot_table_look *look = pivot_table_look_unshare (
- pivot_table_look_ref (pivot_table_look_get_default ()));
- look->omit_empty = false;
-
- struct ctables *ct = xmalloc (sizeof *ct);
- *ct = (struct ctables) {
- .dict = dataset_dict (ds),
- .look = look,
- .ctables_formats = FMT_SETTINGS_INIT,
- .vlabels = vlabels,
- .postcomputes = HMAP_INITIALIZER (ct->postcomputes),
- };
-
- time_t now = time (NULL);
-
- struct ctf
- {
- enum fmt_type type;
- const char *dot_string;
- const char *comma_string;
- };
- static const struct ctf ctfs[4] = {
- { CTEF_NEGPAREN, "(,,,)", "(...)" },
- { CTEF_NEQUAL, "-,N=,,", "-.N=.." },
- { CTEF_PAREN, "-,(,),", "-.(.)." },
- { CTEF_PCTPAREN, "-,(,%),", "-.(.%)." },
- };
- bool is_dot = settings_get_fmt_settings ()->decimal == '.';
- for (size_t i = 0; i < 4; i++)
- {
- const char *s = is_dot ? ctfs[i].dot_string : ctfs[i].comma_string;
- fmt_settings_set_cc (&ct->ctables_formats, ctfs[i].type,
- fmt_number_style_from_string (s));
- }
-
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
-
- while (!lex_match_id (lexer, "TABLE"))
- {
- if (lex_match_id (lexer, "FORMAT"))
- {
- double widths[2] = { SYSMIS, SYSMIS };
- double units_per_inch = 72.0;
-
- int start_ofs = lex_ofs (lexer);
- while (lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "MINCOLWIDTH"))
- {
- if (!parse_col_width (lexer, "MINCOLWIDTH", &widths[0]))
- goto error;
- }
- else if (lex_match_id (lexer, "MAXCOLWIDTH"))
- {
- if (!parse_col_width (lexer, "MAXCOLWIDTH", &widths[1]))
- goto error;
- }
- else if (lex_match_id (lexer, "UNITS"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "POINTS"))
- units_per_inch = 72.0;
- else if (lex_match_id (lexer, "INCHES"))
- units_per_inch = 1.0;
- else if (lex_match_id (lexer, "CM"))
- units_per_inch = 2.54;
- else
- {
- lex_error_expecting (lexer, "POINTS", "INCHES", "CM");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "EMPTY"))
- {
- free (ct->zero);
- ct->zero = NULL;
-
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "ZERO"))
- {
- /* Nothing to do. */
- }
- else if (lex_match_id (lexer, "BLANK"))
- ct->zero = xstrdup ("");
- else if (lex_force_string (lexer))
- {
- ct->zero = ss_xstrdup (lex_tokss (lexer));
- lex_get (lexer);
- }
- else
- goto error;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto error;
-
- free (ct->missing);
- ct->missing = (strcmp (lex_tokcstr (lexer), ".")
- ? ss_xstrdup (lex_tokss (lexer))
- : NULL);
- lex_get (lexer);
- }
- else
- {
- lex_error_expecting (lexer, "MINCOLWIDTH", "MAXCOLWIDTH",
- "UNITS", "EMPTY", "MISSING");
- goto error;
- }
- }
-
- if (widths[0] != SYSMIS && widths[1] != SYSMIS
- && widths[0] > widths[1])
- {
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("MINCOLWIDTH must not be greater than "
- "MAXCOLWIDTH."));
- goto error;
- }
-
- for (size_t i = 0; i < 2; i++)
- if (widths[i] != SYSMIS)
- {
- int *wr = ct->look->width_ranges[TABLE_HORZ];
- wr[i] = widths[i] / units_per_inch * 96.0;
- if (wr[0] > wr[1])
- wr[!i] = wr[i];
- }
- }
- else if (lex_match_id (lexer, "VLABELS"))
- {
- if (!lex_force_match_id (lexer, "VARIABLES"))
- goto error;
- lex_match (lexer, T_EQUALS);
-
- struct variable **vars;
- size_t n_vars;
- if (!parse_variables (lexer, dataset_dict (ds), &vars, &n_vars,
- PV_NO_SCRATCH))
- goto error;
-
- if (!lex_force_match_id (lexer, "DISPLAY"))
- {
- free (vars);
- goto error;
- }
- lex_match (lexer, T_EQUALS);
-
- enum ctables_vlabel vlabel;
- if (lex_match_id (lexer, "DEFAULT"))
- vlabel = (enum ctables_vlabel) settings_get_show_variables ();
- else if (lex_match_id (lexer, "NAME"))
- vlabel = CTVL_NAME;
- else if (lex_match_id (lexer, "LABEL"))
- vlabel = CTVL_LABEL;
- else if (lex_match_id (lexer, "BOTH"))
- vlabel = CTVL_BOTH;
- else if (lex_match_id (lexer, "NONE"))
- vlabel = CTVL_NONE;
- else
- {
- lex_error_expecting (lexer, "DEFAULT", "NAME", "LABEL",
- "BOTH", "NONE");
- free (vars);
- goto error;
- }
-
- for (size_t i = 0; i < n_vars; i++)
- ct->vlabels[var_get_dict_index (vars[i])] = vlabel;
- free (vars);
- }
- else if (lex_match_id (lexer, "MRSETS"))
- {
- if (!lex_force_match_id (lexer, "COUNTDUPLICATES"))
- goto error;
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &ct->mrsets_count_duplicates))
- goto error;
- }
- else if (lex_match_id (lexer, "SMISSING"))
- {
- if (lex_match_id (lexer, "VARIABLE"))
- ct->smissing_listwise = false;
- else if (lex_match_id (lexer, "LISTWISE"))
- ct->smissing_listwise = true;
- else
- {
- lex_error_expecting (lexer, "VARIABLE", "LISTWISE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "PCOMPUTE"))
- {
- if (!ctables_parse_pcompute (lexer, dataset_dict (ds), ct))
- goto error;
- }
- else if (lex_match_id (lexer, "PPROPERTIES"))
- {
- if (!ctables_parse_pproperties (lexer, ct))
- goto error;
- }
- else if (lex_match_id (lexer, "WEIGHT"))
- {
- if (!lex_force_match_id (lexer, "VARIABLE"))
- goto error;
- lex_match (lexer, T_EQUALS);
- ct->e_weight = parse_variable (lexer, dataset_dict (ds));
- if (!ct->e_weight)
- goto error;
- }
- else if (lex_match_id (lexer, "HIDESMALLCOUNTS"))
- {
- if (lex_match_id (lexer, "COUNT"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "HIDESMALLCOUNTS COUNT",
- 2, INT_MAX))
- goto error;
- ct->hide_threshold = lex_integer (lexer);
- lex_get (lexer);
- }
- else if (ct->hide_threshold == 0)
- ct->hide_threshold = 5;
- }
- else
- {
- lex_error_expecting (lexer, "FORMAT", "VLABELS", "MRSETS",
- "SMISSING", "PCOMPUTE", "PPROPERTIES",
- "WEIGHT", "HIDESMALLCOUNTS", "TABLE");
- if (lex_match_id (lexer, "SLABELS")
- || lex_match_id (lexer, "CLABELS")
- || lex_match_id (lexer, "CRITERIA")
- || lex_match_id (lexer, "CATEGORIES")
- || lex_match_id (lexer, "TITLES")
- || lex_match_id (lexer, "SIGTEST")
- || lex_match_id (lexer, "COMPARETEST"))
- lex_next_msg (lexer, SN, -1, -1,
- _("TABLE must appear before this subcommand."));
- goto error;
- }
-
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
- }
-
- size_t allocated_tables = 0;
- do
- {
- if (ct->n_tables >= allocated_tables)
- ct->tables = x2nrealloc (ct->tables, &allocated_tables,
- sizeof *ct->tables);
-
- struct ctables_category *cat = xmalloc (sizeof *cat);
- *cat = (struct ctables_category) {
- .type = CCT_VALUE,
- .include_missing = false,
- .sort_ascending = true,
- };
-
- struct ctables_categories *c = xmalloc (sizeof *c);
- size_t n_vars = dict_get_n_vars (dataset_dict (ds));
- *c = (struct ctables_categories) {
- .n_refs = n_vars,
- .cats = cat,
- .n_cats = 1,
- };
-
- struct ctables_categories **categories = xnmalloc (n_vars,
- sizeof *categories);
- for (size_t i = 0; i < n_vars; i++)
- categories[i] = c;
-
- bool *show_empty = xmalloc (n_vars);
- memset (show_empty, true, n_vars);
-
- struct ctables_table *t = xmalloc (sizeof *t);
- *t = (struct ctables_table) {
- .ctables = ct,
- .slabels_axis = PIVOT_AXIS_COLUMN,
- .slabels_visible = true,
- .clabels_values_map = HMAP_INITIALIZER (t->clabels_values_map),
- .label_axis = {
- [PIVOT_AXIS_ROW] = PIVOT_AXIS_ROW,
- [PIVOT_AXIS_COLUMN] = PIVOT_AXIS_COLUMN,
- [PIVOT_AXIS_LAYER] = PIVOT_AXIS_LAYER,
- },
- .clabels_from_axis = PIVOT_AXIS_LAYER,
- .clabels_to_axis = PIVOT_AXIS_LAYER,
- .categories = categories,
- .n_categories = n_vars,
- .show_empty = show_empty,
- .cilevel = 95,
- };
- ct->tables[ct->n_tables++] = t;
-
- lex_match (lexer, T_EQUALS);
- int expr_start = lex_ofs (lexer);
- if (!ctables_axis_parse (lexer, dataset_dict (ds),
- &t->axes[PIVOT_AXIS_ROW]))
- goto error;
- if (lex_match (lexer, T_BY))
- {
- if (!ctables_axis_parse (lexer, dataset_dict (ds),
- &t->axes[PIVOT_AXIS_COLUMN]))
- goto error;
-
- if (lex_match (lexer, T_BY))
- {
- if (!ctables_axis_parse (lexer, dataset_dict (ds),
- &t->axes[PIVOT_AXIS_LAYER]))
- goto error;
- }
- }
- int expr_end = lex_ofs (lexer);
-
- if (!t->axes[PIVOT_AXIS_ROW] && !t->axes[PIVOT_AXIS_COLUMN]
- && !t->axes[PIVOT_AXIS_LAYER])
- {
- lex_error (lexer, _("At least one variable must be specified."));
- goto error;
- }
-
- const struct ctables_axis *scales[PIVOT_N_AXES];
- size_t n_scales = 0;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- scales[a] = find_scale (t->axes[a]);
- if (scales[a])
- n_scales++;
- }
- if (n_scales > 1)
- {
- msg (SE, _("Scale variables may appear only on one axis."));
- if (scales[PIVOT_AXIS_ROW])
- msg_at (SN, scales[PIVOT_AXIS_ROW]->loc,
- _("This scale variable appears on the rows axis."));
- if (scales[PIVOT_AXIS_COLUMN])
- msg_at (SN, scales[PIVOT_AXIS_COLUMN]->loc,
- _("This scale variable appears on the columns axis."));
- if (scales[PIVOT_AXIS_LAYER])
- msg_at (SN, scales[PIVOT_AXIS_LAYER]->loc,
- _("This scale variable appears on the layer axis."));
- goto error;
- }
-
- const struct ctables_axis *summaries[PIVOT_N_AXES];
- size_t n_summaries = 0;
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- {
- summaries[a] = (scales[a]
- ? scales[a]
- : find_categorical_summary_spec (t->axes[a]));
- if (summaries[a])
- n_summaries++;
- }
- if (n_summaries > 1)
- {
- msg (SE, _("Summaries may appear only on one axis."));
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- if (summaries[a])
- {
- msg_at (SN, summaries[a]->loc,
- a == PIVOT_AXIS_ROW
- ? _("This variable on the rows axis has a summary.")
- : a == PIVOT_AXIS_COLUMN
- ? _("This variable on the columns axis has a summary.")
- : _("This variable on the layers axis has a summary."));
- if (scales[a])
- msg_at (SN, summaries[a]->loc,
- _("This is a scale variable, so it always has a "
- "summary even if the syntax does not explicitly "
- "specify one."));
- }
- goto error;
- }
- for (enum pivot_axis_type a = 0; a < PIVOT_N_AXES; a++)
- if (n_summaries ? summaries[a] : t->axes[a])
- {
- t->summary_axis = a;
- break;
- }
-
- if (lex_token (lexer) == T_ENDCMD)
- {
- if (!ctables_prepare_table (t, lexer))
- goto error;
- break;
- }
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
-
- while (!lex_match_id (lexer, "TABLE") && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "SLABELS"))
- {
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "POSITION"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "COLUMN"))
- t->slabels_axis = PIVOT_AXIS_COLUMN;
- else if (lex_match_id (lexer, "ROW"))
- t->slabels_axis = PIVOT_AXIS_ROW;
- else if (lex_match_id (lexer, "LAYER"))
- t->slabels_axis = PIVOT_AXIS_LAYER;
- else
- {
- lex_error_expecting (lexer, "COLUMN", "ROW", "LAYER");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "VISIBLE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &t->slabels_visible))
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "POSITION", "VISIBLE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "CLABELS"))
- {
- int start_ofs = lex_ofs (lexer) - 1;
- if (lex_match_id (lexer, "AUTO"))
- {
- t->label_axis[PIVOT_AXIS_ROW] = PIVOT_AXIS_ROW;
- t->label_axis[PIVOT_AXIS_COLUMN] = PIVOT_AXIS_COLUMN;
- }
- else if (lex_match_id (lexer, "ROWLABELS"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "OPPOSITE"))
- t->label_axis[PIVOT_AXIS_ROW] = PIVOT_AXIS_COLUMN;
- else if (lex_match_id (lexer, "LAYER"))
- t->label_axis[PIVOT_AXIS_ROW] = PIVOT_AXIS_LAYER;
- else
- {
- lex_error_expecting (lexer, "OPPOSITE", "LAYER");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "COLLABELS"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "OPPOSITE"))
- t->label_axis[PIVOT_AXIS_COLUMN] = PIVOT_AXIS_ROW;
- else if (lex_match_id (lexer, "LAYER"))
- t->label_axis[PIVOT_AXIS_COLUMN] = PIVOT_AXIS_LAYER;
- else
- {
- lex_error_expecting (lexer, "OPPOSITE", "LAYER");
- goto error;
- }
- }
- else
- {
- lex_error_expecting (lexer, "AUTO", "ROWLABELS",
- "COLLABELS");
- goto error;
- }
- int end_ofs = lex_ofs (lexer) - 1;
-
- if (t->label_axis[PIVOT_AXIS_ROW] != PIVOT_AXIS_ROW
- && t->label_axis[PIVOT_AXIS_COLUMN] != PIVOT_AXIS_COLUMN)
- {
- msg (SE, _("ROWLABELS and COLLABELS may not both be "
- "specified."));
-
- lex_ofs_msg (lexer, SN, t->clabels_start_ofs,
- t->clabels_end_ofs,
- _("This is the first specification."));
- lex_ofs_msg (lexer, SN, start_ofs, end_ofs,
- _("This is the second specification."));
- goto error;
- }
-
- t->clabels_start_ofs = start_ofs;
- t->clabels_end_ofs = end_ofs;
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- if (!lex_force_match_id (lexer, "CILEVEL"))
- goto error;
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_num_range_halfopen (lexer, "CILEVEL", 0, 100))
- goto error;
- t->cilevel = lex_number (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "CATEGORIES"))
- {
- if (!ctables_table_parse_categories (lexer, dataset_dict (ds),
- ct, t))
- goto error;
- }
- else if (lex_match_id (lexer, "TITLES"))
- {
- do
- {
- char **textp;
- if (lex_match_id (lexer, "CAPTIONS"))
- textp = &t->caption;
- else if (lex_match_id (lexer, "CORNERS"))
- textp = &t->corner;
- else if (lex_match_id (lexer, "TITLES"))
- textp = &t->title;
- else
- {
- lex_error_expecting (lexer, "CAPTION", "CORNER", "TITLE");
- goto error;
- }
- lex_match (lexer, T_EQUALS);
-
- struct string s = DS_EMPTY_INITIALIZER;
- while (lex_is_string (lexer))
- {
- if (!ds_is_empty (&s))
- ds_put_byte (&s, '\n');
- put_title_text (&s, lex_tokss (lexer), now,
- lexer, dataset_dict (ds),
- expr_start, expr_end);
- lex_get (lexer);
- }
- free (*textp);
- *textp = ds_steal_cstr (&s);
- }
- while (lex_token (lexer) != T_SLASH
- && lex_token (lexer) != T_ENDCMD);
- }
- else if (lex_match_id (lexer, "SIGTEST"))
- {
- int start_ofs = lex_ofs (lexer) - 1;
- if (!t->chisq)
- {
- t->chisq = xmalloc (sizeof *t->chisq);
- *t->chisq = (struct ctables_chisq) {
- .alpha = .05,
- .include_mrsets = true,
- .all_visible = true,
- };
- }
-
- do
- {
- if (lex_match_id (lexer, "TYPE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_match_id (lexer, "CHISQUARE"))
- goto error;
- }
- else if (lex_match_id (lexer, "ALPHA"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_num_range_halfopen (lexer, "ALPHA", 0, 1))
- goto error;
- t->chisq->alpha = lex_number (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "INCLUDEMRSETS"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &t->chisq->include_mrsets))
- goto error;
- }
- else if (lex_match_id (lexer, "CATEGORIES"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "ALLVISIBLE"))
- t->chisq->all_visible = true;
- else if (lex_match_id (lexer, "SUBTOTALS"))
- t->chisq->all_visible = false;
- else
- {
- lex_error_expecting (lexer,
- "ALLVISIBLE", "SUBTOTALS");
- goto error;
- }
- }
- else
- {
- lex_error_expecting (lexer, "TYPE", "ALPHA",
- "INCLUDEMRSETS", "CATEGORIES");
- goto error;
- }
- }
- while (lex_token (lexer) != T_SLASH
- && lex_token (lexer) != T_ENDCMD);
-
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("Support for SIGTEST not yet implemented."));
- goto error;
- }
- else if (lex_match_id (lexer, "COMPARETEST"))
- {
- int start_ofs = lex_ofs (lexer) - 1;
- if (!t->pairwise)
- {
- t->pairwise = xmalloc (sizeof *t->pairwise);
- *t->pairwise = (struct ctables_pairwise) {
- .type = PROP,
- .alpha = { .05, .05 },
- .adjust = BONFERRONI,
- .include_mrsets = true,
- .meansvariance_allcats = true,
- .all_visible = true,
- .merge = false,
- .apa_style = true,
- .show_sig = false,
- };
- }
-
- do
- {
- if (lex_match_id (lexer, "TYPE"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "PROP"))
- t->pairwise->type = PROP;
- else if (lex_match_id (lexer, "MEAN"))
- t->pairwise->type = MEAN;
- else
- {
- lex_error_expecting (lexer, "PROP", "MEAN");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "ALPHA"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_num_range_open (lexer, "ALPHA", 0, 1))
- goto error;
- double a0 = lex_number (lexer);
- lex_get (lexer);
-
- lex_match (lexer, T_COMMA);
- if (lex_is_number (lexer))
- {
- if (!lex_force_num_range_open (lexer, "ALPHA", 0, 1))
- goto error;
- double a1 = lex_number (lexer);
- lex_get (lexer);
-
- t->pairwise->alpha[0] = MIN (a0, a1);
- t->pairwise->alpha[1] = MAX (a0, a1);
- }
- else
- t->pairwise->alpha[0] = t->pairwise->alpha[1] = a0;
- }
- else if (lex_match_id (lexer, "ADJUST"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "BONFERRONI"))
- t->pairwise->adjust = BONFERRONI;
- else if (lex_match_id (lexer, "BH"))
- t->pairwise->adjust = BH;
- else if (lex_match_id (lexer, "NONE"))
- t->pairwise->adjust = 0;
- else
- {
- lex_error_expecting (lexer, "BONFERRONI", "BH",
- "NONE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "INCLUDEMRSETS"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &t->pairwise->include_mrsets))
- goto error;
- }
- else if (lex_match_id (lexer, "MEANSVARIANCE"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "ALLCATS"))
- t->pairwise->meansvariance_allcats = true;
- else if (lex_match_id (lexer, "TESTEDCATS"))
- t->pairwise->meansvariance_allcats = false;
- else
- {
- lex_error_expecting (lexer, "ALLCATS", "TESTEDCATS");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "CATEGORIES"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "ALLVISIBLE"))
- t->pairwise->all_visible = true;
- else if (lex_match_id (lexer, "SUBTOTALS"))
- t->pairwise->all_visible = false;
- else
- {
- lex_error_expecting (lexer, "ALLVISIBLE",
- "SUBTOTALS");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "MERGE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &t->pairwise->merge))
- goto error;
- }
- else if (lex_match_id (lexer, "STYLE"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "APA"))
- t->pairwise->apa_style = true;
- else if (lex_match_id (lexer, "SIMPLE"))
- t->pairwise->apa_style = false;
- else
- {
- lex_error_expecting (lexer, "APA", "SIMPLE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "SHOWSIG"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_bool (lexer, &t->pairwise->show_sig))
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "TYPE", "ALPHA", "ADJUST",
- "INCLUDEMRSETS", "MEANSVARIANCE",
- "CATEGORIES", "MERGE", "STYLE",
- "SHOWSIG");
- goto error;
- }
- }
- while (lex_token (lexer) != T_SLASH
- && lex_token (lexer) != T_ENDCMD);
-
- lex_ofs_error (lexer, start_ofs, lex_ofs (lexer) - 1,
- _("Support for COMPARETEST not yet implemented."));
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "TABLE", "SLABELS", "CLABELS",
- "CRITERIA", "CATEGORIES", "TITLES",
- "SIGTEST", "COMPARETEST");
- if (lex_match_id (lexer, "FORMAT")
- || lex_match_id (lexer, "VLABELS")
- || lex_match_id (lexer, "MRSETS")
- || lex_match_id (lexer, "SMISSING")
- || lex_match_id (lexer, "PCOMPUTE")
- || lex_match_id (lexer, "PPROPERTIES")
- || lex_match_id (lexer, "WEIGHT")
- || lex_match_id (lexer, "HIDESMALLCOUNTS"))
- lex_next_msg (lexer, SN, -1, -1,
- _("This subcommand must appear before TABLE."));
- goto error;
- }
-
- if (!lex_match (lexer, T_SLASH))
- break;
- }
-
- if (t->label_axis[PIVOT_AXIS_ROW] != PIVOT_AXIS_ROW)
- t->clabels_from_axis = PIVOT_AXIS_ROW;
- else if (t->label_axis[PIVOT_AXIS_COLUMN] != PIVOT_AXIS_COLUMN)
- t->clabels_from_axis = PIVOT_AXIS_COLUMN;
- t->clabels_to_axis = t->label_axis[t->clabels_from_axis];
-
- if (!ctables_prepare_table (t, lexer))
- goto error;
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- if (!input)
- input = proc_open (ds);
- bool ok = ctables_execute (ds, input, ct);
- ok = proc_commit (ds) && ok;
-
- ctables_destroy (ct);
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-
-error:
- if (input)
- proc_commit (ds);
- ctables_destroy (ct);
- return CMD_FAILURE;
-}
-
+++ /dev/null
-/* -*- c -*- */
-
-/* Summary functions for all variables. */
-S(CTSF_COUNT, "COUNT", CTFT_UECELL, CTF_COUNT, CTFA_ALL)
-S(CTSF_areaPCT_COUNT, "PCT.COUNT", CTFT_AREA, CTF_PERCENT, CTFA_ALL)
-S(CTSF_areaPCT_VALIDN, "PCT.VALIDN", CTFT_AREA, CTF_PERCENT, CTFA_ALL)
-S(CTSF_areaPCT_TOTALN, "PCT.TOTALN", CTFT_AREA, CTF_PERCENT, CTFA_ALL)
-
-/* Scale variables, totals, and subtotals. */
-S(CTSF_MAXIMUM, "MAXIMUM", CTFT_CELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_MEAN, "MEAN", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_MEDIAN, "MEDIAN", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_MINIMUM, "MINIMUM", CTFT_CELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_MISSING, "MISSING", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_MODE, "MODE", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_PTILE, "PTILE", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_RANGE, "RANGE", CTFT_CELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_SEMEAN, "SEMEAN", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_STDDEV, "STDDEV", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_SUM, "SUM", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_TOTALN, "TOTALN", CTFT_UECELL, CTF_COUNT, CTFA_SCALE)
-S(CTSF_VALIDN, "VALIDN", CTFT_UECELL, CTF_COUNT, CTFA_SCALE)
-S(CTSF_VARIANCE, "VARIANCE", CTFT_UCELL, CTF_GENERAL, CTFA_SCALE)
-S(CTSF_areaPCT_SUM, "PCT.SUM", CTFT_AREA, CTF_PERCENT, CTFA_SCALE)
-
-/* Debugging and troubleshooting. */
-S(CTSF_areaID, "ID", CTFT_AREA, CTF_COUNT, CTFA_ALL)
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-2000, 2009-2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <limits.h>
-#include <math.h>
-#include <stdlib.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/subcase.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "math/moments.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-/* DESCRIPTIVES private data. */
-
-/* Handling of missing values. */
-enum dsc_missing_type
- {
- DSC_VARIABLE, /* Handle missing values on a per-variable basis. */
- DSC_LISTWISE /* Discard entire case if any variable is missing. */
- };
-
-/* Describes properties of a distribution for the purpose of
- calculating a Z-score. */
-struct dsc_z_score
- {
- const struct variable *src_var; /* Variable on which z-score is based. */
- struct variable *z_var; /* New z-score variable. */
- double mean; /* Distribution mean. */
- double std_dev; /* Distribution standard deviation. */
- };
-
-/* DESCRIPTIVES transformation (for calculating Z-scores). */
-struct dsc_trns
- {
- struct dsc_z_score *z_scores; /* Array of Z-scores. */
- size_t n_z_scores; /* Number of Z-scores. */
- const struct variable **vars; /* Variables for listwise missing checks. */
- size_t n_vars; /* Number of variables. */
- enum dsc_missing_type missing_type; /* Treatment of missing values. */
- enum mv_class exclude; /* Classes of missing values to exclude. */
- const struct variable *filter; /* Dictionary FILTER BY variable. */
- struct casereader *z_reader; /* Reader for count, mean, stddev. */
- casenumber count; /* Number left in this SPLIT FILE group.*/
- bool ok;
- };
-
-/* Statistics. Used as bit indexes, so must be 32 or fewer. */
-enum dsc_statistic
- {
- DSC_MEAN = 0, DSC_SEMEAN, DSC_STDDEV, DSC_VARIANCE, DSC_KURTOSIS,
- DSC_SEKURT, DSC_SKEWNESS, DSC_SESKEW, DSC_RANGE, DSC_MIN,
- DSC_MAX, DSC_SUM, DSC_N_STATS,
-
- /* Only valid as sort criteria. */
- DSC_NAME = -2, /* Sort by name. */
- DSC_NONE = -1 /* Unsorted. */
- };
-
-/* Describes one statistic. */
-struct dsc_statistic_info
- {
- const char *identifier; /* Identifier. */
- const char *name; /* Full name. */
- enum moment moment; /* Highest moment needed to calculate. */
- };
-
-/* Table of statistics, indexed by DSC_*. */
-static const struct dsc_statistic_info dsc_info[DSC_N_STATS] =
- {
- {"MEAN", N_("Mean"), MOMENT_MEAN},
- {"SEMEAN", N_("S.E. Mean"), MOMENT_VARIANCE},
- {"STDDEV", N_("Std Dev"), MOMENT_VARIANCE},
- {"VARIANCE", N_("Variance"), MOMENT_VARIANCE},
- {"KURTOSIS", N_("Kurtosis"), MOMENT_KURTOSIS},
- {"SEKURTOSIS", N_("S.E. Kurt"), MOMENT_NONE},
- {"SKEWNESS", N_("Skewness"), MOMENT_SKEWNESS},
- {"SESKEWNESS", N_("S.E. Skew"), MOMENT_NONE},
- {"RANGE", N_("Range"), MOMENT_NONE},
- {"MINIMUM", N_("Minimum"), MOMENT_NONE},
- {"MAXIMUM", N_("Maximum"), MOMENT_NONE},
- {"SUM", N_("Sum"), MOMENT_MEAN},
- };
-
-/* Statistics calculated by default if none are explicitly
- requested. */
-#define DEFAULT_STATS \
- ((1UL << DSC_MEAN) | (1UL << DSC_STDDEV) | (1UL << DSC_MIN) \
- | (1UL << DSC_MAX))
-
-/* A variable specified on DESCRIPTIVES. */
-struct dsc_var
- {
- const struct variable *v; /* Variable to calculate on. */
- char *z_name; /* Name for z-score variable. */
- double valid, missing; /* Valid, missing counts. */
- struct moments *moments; /* Moments. */
- double min, max; /* Maximum and mimimum values. */
- double stats[DSC_N_STATS]; /* All the stats' values. */
- };
-
-/* A DESCRIPTIVES procedure. */
-struct dsc_proc
- {
- /* Per-variable info. */
- struct dictionary *dict; /* Dictionary. */
- struct dsc_var *vars; /* Variables. */
- size_t n_vars; /* Number of variables. */
-
- /* User options. */
- enum dsc_missing_type missing_type; /* Treatment of missing values. */
- enum mv_class exclude; /* Classes of missing values to exclude. */
-
- /* Accumulated results. */
- double missing_listwise; /* Sum of weights of cases missing listwise. */
- double valid; /* Sum of weights of valid cases. */
- bool bad_warn; /* Warn if bad weight found. */
- enum dsc_statistic sort_by_stat; /* Statistic to sort by; -1: name. */
- enum subcase_direction sort_direction;
- unsigned long show_stats; /* Statistics to display. */
- unsigned long calc_stats; /* Statistics to calculate. */
- enum moment max_moment; /* Highest moment needed for stats. */
-
- /* Z scores. */
- struct casewriter *z_writer; /* Mean and stddev per SPLIT FILE group. */
- };
-
-/* Parsing. */
-static enum dsc_statistic match_statistic (struct lexer *);
-static void free_dsc_proc (struct dsc_proc *);
-
-/* Z-score functions. */
-static bool try_name (const struct dictionary *dict,
- struct dsc_proc *dsc, const char *name);
-static char *generate_z_varname (const struct dictionary *dict,
- struct dsc_proc *dsc,
- const char *name, int *n_zs);
-static void dump_z_table (struct dsc_proc *);
-static void setup_z_trns (struct dsc_proc *, struct dataset *);
-
-/* Procedure execution functions. */
-static void calc_descriptives (struct dsc_proc *, struct casereader *,
- struct dataset *);
-static void display (struct dsc_proc *dsc);
-\f
-/* Parser and outline. */
-
-/* Handles DESCRIPTIVES. */
-int
-cmd_descriptives (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- const struct variable **vars = NULL;
- size_t n_vars = 0;
- bool save_z_scores = false;
- int n_zs = 0;
-
- /* Create and initialize dsc. */
- struct dsc_proc *dsc = xmalloc (sizeof *dsc);
- *dsc = (struct dsc_proc) {
- .dict = dict,
- .missing_type = DSC_VARIABLE,
- .exclude = MV_ANY,
- .bad_warn = 1,
- .sort_by_stat = DSC_NONE,
- .sort_direction = SC_ASCEND,
- .show_stats = DEFAULT_STATS,
- .calc_stats = DEFAULT_STATS,
- };
-
- /* Parse DESCRIPTIVES. */
- int z_ofs = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "VARIABLE"))
- dsc->missing_type = DSC_VARIABLE;
- else if (lex_match_id (lexer, "LISTWISE"))
- dsc->missing_type = DSC_LISTWISE;
- else if (lex_match_id (lexer, "INCLUDE"))
- dsc->exclude = MV_SYSTEM;
- else
- {
- lex_error_expecting (lexer, "VARIABLE", "LISTWISE",
- "INCLUDE");
- goto error;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "SAVE"))
- {
- save_z_scores = true;
- z_ofs = lex_ofs (lexer) - 1;
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "LABELS")
- || lex_match_id (lexer, "NOLABELS")
- || lex_match_id (lexer, "INDEX")
- || lex_match_id (lexer, "NOINDEX")
- || lex_match_id (lexer, "LINE")
- || lex_match_id (lexer, "SERIAL"))
- {
- /* Ignore. */
- }
- else
- {
- lex_error_expecting (lexer, "LABELS", "NOLABELS",
- "INDEX", "NOINDEX", "LINE", "SERIAL");
- goto error;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
- dsc->show_stats = 0;
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match (lexer, T_ALL))
- dsc->show_stats |= (1UL << DSC_N_STATS) - 1;
- else if (lex_match_id (lexer, "DEFAULT"))
- dsc->show_stats |= DEFAULT_STATS;
- else
- {
- enum dsc_statistic s = match_statistic (lexer);
- if (s == DSC_NONE)
- goto error;
- dsc->show_stats |= 1UL << s;
- }
- lex_match (lexer, T_COMMA);
- }
- if (dsc->show_stats == 0)
- dsc->show_stats = DEFAULT_STATS;
- }
- else if (lex_match_id (lexer, "SORT"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "NAME"))
- dsc->sort_by_stat = DSC_NAME;
- else
- {
- dsc->sort_by_stat = match_statistic (lexer);
- if (dsc->sort_by_stat == DSC_NONE)
- dsc->sort_by_stat = DSC_MEAN;
- }
- if (lex_match (lexer, T_LPAREN))
- {
- if (lex_match_id (lexer, "A"))
- dsc->sort_direction = SC_ASCEND;
- else if (lex_match_id (lexer, "D"))
- dsc->sort_direction = SC_DESCEND;
- else
- {
- lex_error_expecting (lexer, "A", "D");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
- else if (n_vars == 0)
- {
- lex_match_phrase (lexer, "VARIABLES=");
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (!parse_variables_const (lexer, dict, &vars, &n_vars,
- PV_APPEND | PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
-
- dsc->vars = xnrealloc ((void *)dsc->vars, n_vars, sizeof *dsc->vars);
- for (size_t i = dsc->n_vars; i < n_vars; i++)
- dsc->vars[i] = (struct dsc_var) { .v = vars[i] };
- dsc->n_vars = n_vars;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_id (lexer))
- goto error;
- z_ofs = lex_ofs (lexer);
- if (try_name (dict, dsc, lex_tokcstr (lexer)))
- {
- struct dsc_var *dsc_var = &dsc->vars[dsc->n_vars - 1];
- dsc_var->z_name = xstrdup (lex_tokcstr (lexer));
- n_zs++;
- }
- else
- lex_error (lexer, _("Z-score variable name %s would be "
- "a duplicate variable name."),
- lex_tokcstr (lexer));
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "MISSING", "SAVE", "FORMAT", "STATISTICS",
- "SORT", "VARIABLES");
- goto error;
- }
-
- lex_match (lexer, T_SLASH);
- }
- if (n_vars == 0)
- {
- msg (SE, _("No variables specified."));
- goto error;
- }
-
- /* Construct z-score varnames, show translation table. */
- if (n_zs || save_z_scores)
- {
- if (save_z_scores)
- {
- int n_gens = 0;
-
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dsc_var = &dsc->vars[i];
- if (dsc_var->z_name == NULL)
- {
- const char *name = var_get_name (dsc_var->v);
- dsc_var->z_name = generate_z_varname (dict, dsc, name,
- &n_gens);
- if (dsc_var->z_name == NULL)
- goto error;
-
- n_zs++;
- }
- }
- }
-
- /* It would be better to handle Z scores correctly (however we define
- that) when TEMPORARY is in effect, but in the meantime this at least
- prevents a use-after-free error. See bug #38786. */
- if (proc_make_temporary_transformations_permanent (ds))
- lex_ofs_msg (lexer, SW, z_ofs, z_ofs,
- _("DESCRIPTIVES with Z scores ignores TEMPORARY. "
- "Temporary transformations will be made permanent."));
-
- struct caseproto *proto = caseproto_create ();
- for (size_t i = 0; i < 1 + 2 * n_zs; i++)
- proto = caseproto_add_width (proto, 0);
- dsc->z_writer = autopaging_writer_create (proto);
- caseproto_unref (proto);
-
- dump_z_table (dsc);
- }
-
- /* Figure out statistics to display. */
- if (dsc->show_stats & (1UL << DSC_SKEWNESS))
- dsc->show_stats |= 1UL << DSC_SESKEW;
- if (dsc->show_stats & (1UL << DSC_KURTOSIS))
- dsc->show_stats |= 1UL << DSC_SEKURT;
-
- /* Figure out which statistics to calculate. */
- dsc->calc_stats = dsc->show_stats;
- if (n_zs > 0)
- dsc->calc_stats |= (1UL << DSC_MEAN) | (1UL << DSC_STDDEV);
- if (dsc->sort_by_stat >= 0)
- dsc->calc_stats |= 1UL << dsc->sort_by_stat;
- if (dsc->show_stats & (1UL << DSC_SESKEW))
- dsc->calc_stats |= 1UL << DSC_SKEWNESS;
- if (dsc->show_stats & (1UL << DSC_SEKURT))
- dsc->calc_stats |= 1UL << DSC_KURTOSIS;
-
- /* Figure out maximum moment needed and allocate moments for
- the variables. */
- dsc->max_moment = MOMENT_NONE;
- for (size_t i = 0; i < DSC_N_STATS; i++)
- if (dsc->calc_stats & (1UL << i) && dsc_info[i].moment > dsc->max_moment)
- dsc->max_moment = dsc_info[i].moment;
- if (dsc->max_moment != MOMENT_NONE)
- for (size_t i = 0; i < dsc->n_vars; i++)
- dsc->vars[i].moments = moments_create (dsc->max_moment);
-
- /* Data pass. */
- struct casegrouper *grouper = casegrouper_create_splits (proc_open_filtering (
- ds, false), dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- calc_descriptives (dsc, group, ds);
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- /* Z-scoring! */
- if (ok && n_zs)
- setup_z_trns (dsc, ds);
-
- /* Done. */
- free (vars);
- free_dsc_proc (dsc);
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-
- error:
- free (vars);
- free_dsc_proc (dsc);
- return CMD_FAILURE;
-}
-
-/* Returns the statistic named by the current token and skips past the token.
- Returns DSC_NONE if no statistic is given (e.g., subcommand with no
- specifiers). Emits an error if the current token ID does not name a
- statistic. */
-static enum dsc_statistic
-match_statistic (struct lexer *lexer)
-{
- if (lex_token (lexer) == T_ID)
- {
- for (enum dsc_statistic stat = 0; stat < DSC_N_STATS; stat++)
- if (lex_match_id (lexer, dsc_info[stat].identifier))
- return stat;
-
- const char *stat_names[DSC_N_STATS];
- for (enum dsc_statistic stat = 0; stat < DSC_N_STATS; stat++)
- stat_names[stat] = dsc_info[stat].identifier;
- lex_error_expecting_array (lexer, stat_names,
- sizeof stat_names / sizeof *stat_names);
- lex_get (lexer);
- }
-
- return DSC_NONE;
-}
-
-/* Frees DSC. */
-static void
-free_dsc_proc (struct dsc_proc *dsc)
-{
- if (dsc == NULL)
- return;
-
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dsc_var = &dsc->vars[i];
- free (dsc_var->z_name);
- moments_destroy (dsc_var->moments);
- }
- casewriter_destroy (dsc->z_writer);
- free (dsc->vars);
- free (dsc);
-}
-\f
-/* Z scores. */
-
-/* Returns false if NAME is a duplicate of any existing variable name or
- of any previously-declared z-var name; otherwise returns true. */
-static bool
-try_name (const struct dictionary *dict, struct dsc_proc *dsc,
- const char *name)
-{
- if (dict_lookup_var (dict, name) != NULL)
- return false;
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dsc_var = &dsc->vars[i];
- if (dsc_var->z_name != NULL && !utf8_strcasecmp (dsc_var->z_name, name))
- return false;
- }
- return true;
-}
-
-/* Generates a name for a Z-score variable based on a variable
- named VAR_NAME, given that *Z_CNT generated variable names are
- known to already exist. If successful, returns the new name
- as a dynamically allocated string. On failure, returns NULL. */
-static char *
-generate_z_varname (const struct dictionary *dict, struct dsc_proc *dsc,
- const char *var_name, int *n_zs)
-{
- /* Try a name based on the original variable name. */
- char *z_name = xasprintf ("Z%s", var_name);
- char *trunc_name = utf8_encoding_trunc (z_name, dict_get_encoding (dict),
- ID_MAX_LEN);
- free (z_name);
- if (try_name (dict, dsc, trunc_name))
- return trunc_name;
- free (trunc_name);
-
- /* Generate a synthetic name. */
- for (;;)
- {
- char name[16];
-
- (*n_zs)++;
-
- if (*n_zs <= 99)
- sprintf (name, "ZSC%03d", *n_zs);
- else if (*n_zs <= 108)
- sprintf (name, "STDZ%02d", *n_zs - 99);
- else if (*n_zs <= 117)
- sprintf (name, "ZZZZ%02d", *n_zs - 108);
- else if (*n_zs <= 126)
- sprintf (name, "ZQZQ%02d", *n_zs - 117);
- else
- {
- msg (SE, _("Ran out of generic names for Z-score variables. "
- "There are only 126 generic names: ZSC001-ZSC099, "
- "STDZ01-STDZ09, ZZZZ01-ZZZZ09, ZQZQ01-ZQZQ09."));
- return NULL;
- }
-
- if (try_name (dict, dsc, name))
- return xstrdup (name);
- }
- NOT_REACHED();
-}
-
-/* Outputs a table describing the mapping between source
- variables and Z-score variables. */
-static void
-dump_z_table (struct dsc_proc *dsc)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Mapping of Variables to Z-scores"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Names"),
- N_("Source"), N_("Target"));
-
- struct pivot_dimension *names = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
- names->hide_all_labels = true;
-
- for (size_t i = 0; i < dsc->n_vars; i++)
- if (dsc->vars[i].z_name != NULL)
- {
- int row = pivot_category_create_leaf (names->root,
- pivot_value_new_number (i));
-
- pivot_table_put2 (table, 0, row,
- pivot_value_new_variable (dsc->vars[i].v));
- pivot_table_put2 (table, 1, row,
- pivot_value_new_user_text (dsc->vars[i].z_name, -1));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-descriptives_set_all_sysmis_zscores (const struct dsc_trns *t, struct ccase *c)
-{
- for (const struct dsc_z_score *z = t->z_scores;
- z < t->z_scores + t->n_z_scores; z++)
- *case_num_rw (c, z->z_var) = SYSMIS;
-}
-
-/* Transformation function to calculate Z-scores. Will return SYSMIS if any of
- the following are true: 1) mean or standard deviation is SYSMIS 2) score is
- SYSMIS 3) score is user missing and they were not included in the original
- analyis. 4) any of the variables in the original analysis were missing
- (either system or user-missing values that weren't included).
-*/
-static enum trns_result
-descriptives_trns_proc (void *trns_, struct ccase **c,
- casenumber case_idx UNUSED)
-{
- struct dsc_trns *t = trns_;
-
- *c = case_unshare (*c);
-
- if (t->filter)
- {
- double f = case_num (*c, t->filter);
- if (f == 0.0 || var_is_num_missing (t->filter, f))
- {
- descriptives_set_all_sysmis_zscores (t, *c);
- return TRNS_CONTINUE;
- }
- }
-
- if (t->count <= 0)
- {
- struct ccase *z_case = casereader_read (t->z_reader);
- if (z_case)
- {
- size_t z_idx = 0;
-
- t->count = case_num_idx (z_case, z_idx++);
- for (struct dsc_z_score *z = t->z_scores;
- z < t->z_scores + t->n_z_scores; z++)
- {
- z->mean = case_num_idx (z_case, z_idx++);
- z->std_dev = case_num_idx (z_case, z_idx++);
- }
- case_unref (z_case);
- }
- else
- {
- if (t->ok)
- {
- msg (SE, _("Internal error processing Z scores. "
- "Please report this to %s."),
- PACKAGE_BUGREPORT);
- t->ok = false;
- }
- descriptives_set_all_sysmis_zscores (t, *c);
- return TRNS_CONTINUE;
- }
- }
- t->count--;
-
- if (t->missing_type == DSC_LISTWISE)
- {
- assert (t->vars != NULL);
- for (const struct variable **vars = t->vars; vars < t->vars + t->n_vars;
- vars++)
- {
- double score = case_num (*c, *vars);
- if (var_is_num_missing (*vars, score) & t->exclude)
- {
- descriptives_set_all_sysmis_zscores (t, *c);
- return TRNS_CONTINUE;
- }
- }
- }
-
- for (struct dsc_z_score *z = t->z_scores; z < t->z_scores + t->n_z_scores;
- z++)
- {
- double input = case_num (*c, z->src_var);
- double *output = case_num_rw (*c, z->z_var);
-
- if (z->mean == SYSMIS || z->std_dev == SYSMIS
- || var_is_num_missing (z->src_var, input) & t->exclude)
- *output = SYSMIS;
- else
- *output = (input - z->mean) / z->std_dev;
- }
- return TRNS_CONTINUE;
-}
-
-/* Frees a descriptives_trns struct. */
-static bool
-descriptives_trns_free (void *trns_)
-{
- struct dsc_trns *t = trns_;
- bool ok = t->ok && !casereader_error (t->z_reader);
-
- free (t->z_scores);
- casereader_destroy (t->z_reader);
- assert ((t->missing_type != DSC_LISTWISE) != (t->vars != NULL));
- free (t->vars);
- free (t);
-
- return ok;
-}
-
-static const struct trns_class descriptives_trns_class = {
- .name = "DESCRIPTIVES (Z scores)",
- .execute = descriptives_trns_proc,
- .destroy = descriptives_trns_free,
-};
-
-/* Sets up a transformation to calculate Z scores. */
-static void
-setup_z_trns (struct dsc_proc *dsc, struct dataset *ds)
-{
- size_t n = 0;
- for (size_t i = 0; i < dsc->n_vars; i++)
- if (dsc->vars[i].z_name != NULL)
- n++;
-
- struct dsc_trns *t = xmalloc (sizeof *t);
- *t = (struct dsc_trns) {
- .z_scores = xmalloc (n * sizeof *t->z_scores),
- .n_z_scores = n,
- .missing_type = dsc->missing_type,
- .exclude = dsc->exclude,
- .filter = dict_get_filter (dataset_dict (ds)),
- .z_reader = casewriter_make_reader (dsc->z_writer),
- .ok = true,
- };
- if (t->missing_type == DSC_LISTWISE)
- {
- t->n_vars = dsc->n_vars;
- t->vars = xnmalloc (t->n_vars, sizeof *t->vars);
- for (size_t i = 0; i < t->n_vars; i++)
- t->vars[i] = dsc->vars[i].v;
- }
- dsc->z_writer = NULL;
-
- n = 0;
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dv = &dsc->vars[i];
- if (dv->z_name != NULL)
- {
- struct variable *dst_var = dict_create_var_assert (dataset_dict (ds),
- dv->z_name, 0);
-
- char *label = xasprintf (_("Z-score of %s"), var_to_string (dv->v));
- var_set_label (dst_var, label);
- free (label);
-
- struct dsc_z_score *z = &t->z_scores[n++];
- *z = (struct dsc_z_score) {
- .src_var = dv->v,
- .z_var = dst_var,
- };
- }
- }
-
- add_transformation (ds, &descriptives_trns_class, t);
-}
-\f
-/* Statistical calculation. */
-
-static bool listwise_missing (struct dsc_proc *dsc, const struct ccase *c);
-
-/* Calculates and displays descriptive statistics for the cases
- in CF. */
-static void
-calc_descriptives (struct dsc_proc *dsc, struct casereader *group,
- struct dataset *ds)
-{
- output_split_file_values_peek (ds, group);
- group = casereader_create_filter_weight (group, dataset_dict (ds),
- NULL, NULL);
-
- struct casereader *pass1 = group;
- struct casereader *pass2 = (dsc->max_moment <= MOMENT_MEAN ? NULL
- : casereader_clone (pass1));
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dv = &dsc->vars[i];
-
- dv->valid = dv->missing = 0.0;
- if (dv->moments != NULL)
- moments_clear (dv->moments);
- dv->min = DBL_MAX;
- dv->max = -DBL_MAX;
- }
- dsc->missing_listwise = 0.;
- dsc->valid = 0.;
-
- /* First pass to handle most of the work. */
- casenumber count = 0;
- const struct variable *filter = dict_get_filter (dataset_dict (ds));
- struct ccase *c;
- for (; (c = casereader_read (pass1)) != NULL; case_unref (c))
- {
- double weight = dict_get_case_weight (dataset_dict (ds), c, NULL);
-
- if (filter)
- {
- double f = case_num (c, filter);
- if (f == 0.0 || var_is_num_missing (filter, f))
- continue;
- }
-
- /* Check for missing values. */
- if (listwise_missing (dsc, c))
- {
- dsc->missing_listwise += weight;
- if (dsc->missing_type == DSC_LISTWISE)
- continue;
- }
- dsc->valid += weight;
-
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dv = &dsc->vars[i];
- double x = case_num (c, dv->v);
-
- if (var_is_num_missing (dv->v, x) & dsc->exclude)
- {
- dv->missing += weight;
- continue;
- }
-
- if (dv->moments != NULL)
- moments_pass_one (dv->moments, x, weight);
-
- if (x < dv->min)
- dv->min = x;
- if (x > dv->max)
- dv->max = x;
- }
-
- count++;
- }
- if (!casereader_destroy (pass1))
- {
- casereader_destroy (pass2);
- return;
- }
-
- /* Second pass for higher-order moments. */
- if (dsc->max_moment > MOMENT_MEAN)
- {
- for (; (c = casereader_read (pass2)) != NULL; case_unref (c))
- {
- double weight = dict_get_case_weight (dataset_dict (ds), c, NULL);
-
- if (filter)
- {
- double f = case_num (c, filter);
- if (f == 0.0 || var_is_num_missing (filter, f))
- continue;
- }
-
- /* Check for missing values. */
- if (dsc->missing_type == DSC_LISTWISE && listwise_missing (dsc, c))
- continue;
-
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dv = &dsc->vars[i];
- double x = case_num (c, dv->v);
-
- if (var_is_num_missing (dv->v, x) & dsc->exclude)
- continue;
-
- if (dv->moments != NULL)
- moments_pass_two (dv->moments, x, weight);
- }
- }
- if (!casereader_destroy (pass2))
- return;
- }
-
- /* Calculate results. */
- size_t z_idx = 0;
- if (dsc->z_writer && count > 0)
- {
- c = case_create (casewriter_get_proto (dsc->z_writer));
- *case_num_rw_idx (c, z_idx++) = count;
- }
- else
- c = NULL;
-
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dv = &dsc->vars[i];
-
- for (size_t j = 0; j < DSC_N_STATS; j++)
- dv->stats[j] = SYSMIS;
-
- double W = dsc->valid - dv->missing;
- dv->valid = W;
-
- if (dv->moments != NULL)
- moments_calculate (dv->moments, NULL,
- &dv->stats[DSC_MEAN], &dv->stats[DSC_VARIANCE],
- &dv->stats[DSC_SKEWNESS], &dv->stats[DSC_KURTOSIS]);
- if (dsc->calc_stats & (1UL << DSC_SEMEAN)
- && dv->stats[DSC_VARIANCE] != SYSMIS && W > 0.)
- dv->stats[DSC_SEMEAN] = sqrt (dv->stats[DSC_VARIANCE]) / sqrt (W);
- if (dsc->calc_stats & (1UL << DSC_STDDEV)
- && dv->stats[DSC_VARIANCE] != SYSMIS)
- dv->stats[DSC_STDDEV] = sqrt (dv->stats[DSC_VARIANCE]);
- if (dsc->calc_stats & (1UL << DSC_SEKURT))
- if (dv->stats[DSC_KURTOSIS] != SYSMIS)
- dv->stats[DSC_SEKURT] = calc_sekurt (W);
- if (dsc->calc_stats & (1UL << DSC_SESKEW)
- && dv->stats[DSC_SKEWNESS] != SYSMIS)
- dv->stats[DSC_SESKEW] = calc_seskew (W);
- dv->stats[DSC_RANGE] = ((dv->min == DBL_MAX || dv->max == -DBL_MAX)
- ? SYSMIS : dv->max - dv->min);
- dv->stats[DSC_MIN] = dv->min == DBL_MAX ? SYSMIS : dv->min;
- dv->stats[DSC_MAX] = dv->max == -DBL_MAX ? SYSMIS : dv->max;
- if (dsc->calc_stats & (1UL << DSC_SUM))
- dv->stats[DSC_SUM] = W * dv->stats[DSC_MEAN];
-
- if (dv->z_name && c != NULL)
- {
- *case_num_rw_idx (c, z_idx++) = dv->stats[DSC_MEAN];
- *case_num_rw_idx (c, z_idx++) = dv->stats[DSC_STDDEV];
- }
- }
-
- if (c != NULL)
- casewriter_write (dsc->z_writer, c);
-
- /* Output results. */
- display (dsc);
-}
-
-/* Returns true if any of the descriptives variables in DSC's
- variable list have missing values in case C, false otherwise. */
-static bool
-listwise_missing (struct dsc_proc *dsc, const struct ccase *c)
-{
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- struct dsc_var *dv = &dsc->vars[i];
- double x = case_num (c, dv->v);
-
- if (var_is_num_missing (dv->v, x) & dsc->exclude)
- return true;
- }
- return false;
-}
-\f
-/* Statistical display. */
-
-static algo_compare_func descriptives_compare_dsc_vars;
-
-/* Displays a table of descriptive statistics for DSC. */
-static void
-display (struct dsc_proc *dsc)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Descriptive Statistics"));
- pivot_table_set_weight_var (table, dict_get_weight (dsc->dict));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- pivot_category_create_leaf_rc (
- statistics->root, pivot_value_new_text (N_("N")), PIVOT_RC_COUNT);
- for (int i = 0; i < DSC_N_STATS; i++)
- if (dsc->show_stats & (1UL << i))
- pivot_category_create_leaf (statistics->root,
- pivot_value_new_text (dsc_info[i].name));
-
- if (dsc->sort_by_stat != DSC_NONE)
- sort (dsc->vars, dsc->n_vars, sizeof *dsc->vars,
- descriptives_compare_dsc_vars, dsc);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
- for (size_t i = 0; i < dsc->n_vars; i++)
- {
- const struct dsc_var *dv = &dsc->vars[i];
-
- int row = pivot_category_create_leaf (variables->root,
- pivot_value_new_variable (dv->v));
-
- int column = 0;
- pivot_table_put2 (table, column++, row,
- pivot_value_new_number (dv->valid));
-
- for (int j = 0; j < DSC_N_STATS; j++)
- if (dsc->show_stats & (1UL << j))
- {
- union value v = { .f = dv->stats[j] };
- struct pivot_value *pv = (j == DSC_MIN || j == DSC_MAX
- ? pivot_value_new_var_value (dv->v, &v)
- : pivot_value_new_number (dv->stats[j]));
- pivot_table_put2 (table, column++, row, pv);
- }
- }
-
- int row = pivot_category_create_leaves (
- variables->root, N_("Valid N (listwise)"), N_("Missing N (listwise)"));
- pivot_table_put2 (table, 0, row, pivot_value_new_number (dsc->valid));
- pivot_table_put2 (table, 0, row + 1,
- pivot_value_new_number (dsc->missing_listwise));
- pivot_table_submit (table);
-}
-
-/* Compares `struct dsc_var's A and B according to the ordering
- specified by CMD. */
-static int
-descriptives_compare_dsc_vars (const void *a_, const void *b_, const void *dsc_)
-{
- const struct dsc_var *a = a_;
- const struct dsc_var *b = b_;
- const struct dsc_proc *dsc = dsc_;
-
- int result;
-
- if (dsc->sort_by_stat == DSC_NAME)
- result = utf8_strcasecmp (var_get_name (a->v), var_get_name (b->v));
- else
- {
- double as = a->stats[dsc->sort_by_stat];
- double bs = b->stats[dsc->sort_by_stat];
-
- result = as < bs ? -1 : as > bs;
- }
-
- if (dsc->sort_direction == SC_DESCEND)
- result = -result;
-
- return result;
-}
+++ /dev/null
-/*
- PSPP - a program for statistical analysis.
- Copyright (C) 2012, 2013, 2016, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include <config.h>
-
-#include <math.h>
-#include <gsl/gsl_cdf.h>
-
-#include "data/casegrouper.h"
-#include "data/caseproto.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-#include "math/box-whisker.h"
-#include "math/categoricals.h"
-#include "math/chart-geometry.h"
-#include "math/histogram.h"
-#include "math/interaction.h"
-#include "math/moments.h"
-#include "math/np.h"
-#include "math/order-stats.h"
-#include "math/percentiles.h"
-#include "math/shapiro-wilk.h"
-#include "math/sort.h"
-#include "math/trimmed-mean.h"
-#include "math/tukey-hinges.h"
-#include "output/charts/boxplot.h"
-#include "output/charts/np-plot.h"
-#include "output/charts/plot-hist.h"
-#include "output/charts/spreadlevel-plot.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-static void
-append_value_name (const struct variable *var, const union value *val, struct string *str)
-{
- var_append_value_name (var, val, str);
- if (var_is_value_missing (var, val))
- ds_put_cstr (str, _(" (missing)"));
-}
-
-enum bp_mode
- {
- BP_GROUPS,
- BP_VARIABLES
- };
-
-/* Indices for the ex_proto member (below) */
-enum
- {
- EX_VAL, /* value */
- EX_ID, /* identity */
- EX_WT /* weight */
- };
-
-
-struct examine
-{
- struct pool *pool;
-
- /* A caseproto used to contain the data subsets under examination,
- see (enum above) */
- struct caseproto *ex_proto;
-
- size_t n_dep_vars;
- const struct variable **dep_vars;
-
- size_t n_iacts;
- struct interaction **iacts;
-
- enum mv_class dep_excl;
- enum mv_class fctr_excl;
-
- const struct dictionary *dict;
-
- struct categoricals *cats;
-
- /* how many extremities to display */
- int disp_extremes;
- int calc_extremes;
- bool descriptives;
-
- double conf;
-
- bool missing_pw;
-
- /* The case index of the ID value (or -1) if not applicable */
- size_t id_idx;
- int id_width;
-
- enum pc_alg pc_alg;
- double *ptiles;
- size_t n_percentiles;
-
- bool plot_histogram;
- bool plot_boxplot;
- bool plot_npplot;
- bool plot_spreadlevel;
- float sl_power;
-
- enum bp_mode boxplot_mode;
-
- const struct variable *id_var;
-
- const struct variable *wv;
-};
-
-struct extremity
-{
- /* The value of this extremity */
- double val;
-
- /* Either the casenumber or the value of the variable specified
- by the /ID subcommand which corresponds to this extremity */
- union value identity;
-};
-
-struct exploratory_stats
-{
- double missing;
- double non_missing;
-
- struct moments *mom;
-
- /* Most operations need a sorted reader/writer */
- struct casewriter *sorted_writer;
- struct casereader *sorted_reader;
-
- struct extremity *minima;
- struct extremity *maxima;
-
- /*
- Minimum should alway equal mimima[0].val.
- Likewise, maximum should alway equal maxima[0].val.
- This redundancy exists as an optimisation effort.
- Some statistics (eg histogram) require early calculation
- of the min and max
- */
- double minimum;
- double maximum;
-
- struct trimmed_mean *trimmed_mean;
- struct percentile *quartiles[3];
- struct percentile **percentiles;
- struct shapiro_wilk *shapiro_wilk;
-
- struct tukey_hinges *hinges;
-
- /* The data for the NP Plots */
- struct np *np;
-
- struct histogram *histogram;
-
- /* The data for the box plots */
- struct box_whisker *box_whisker;
-
- /* Total weight */
- double cc;
-
- /* The minimum weight */
- double cmin;
-};
-
-static void
-show_boxplot_grouped (const struct examine *cmd, int iact_idx)
-{
- int v;
-
- const struct interaction *iact = cmd->iacts[iact_idx];
- const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- double y_min = DBL_MAX;
- double y_max = -DBL_MAX;
- int grp;
- struct boxplot *boxplot;
- struct string title;
- ds_init_empty (&title);
-
- if (iact->n_vars > 0)
- {
- struct string istr;
- ds_init_empty (&istr);
- interaction_to_string (iact, &istr);
- ds_put_format (&title, _("Boxplot of %s vs. %s"),
- var_to_string (cmd->dep_vars[v]),
- ds_cstr (&istr));
- ds_destroy (&istr);
- }
- else
- ds_put_format (&title, _("Boxplot of %s"), var_to_string (cmd->dep_vars[v]));
-
- for (grp = 0; grp < n_cats; ++grp)
- {
- const struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- if (y_min > es[v].minimum)
- y_min = es[v].minimum;
-
- if (y_max < es[v].maximum)
- y_max = es[v].maximum;
- }
-
- boxplot = boxplot_create (y_min, y_max, ds_cstr (&title));
-
- ds_destroy (&title);
-
- for (grp = 0; grp < n_cats; ++grp)
- {
- int ivar_idx;
- struct string label;
-
- const struct ccase *c =
- categoricals_get_case_by_category_real (cmd->cats, iact_idx, grp);
-
- struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- ds_init_empty (&label);
- for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
- {
- struct string l;
- const struct variable *ivar = iact->vars[ivar_idx];
- const union value *val = case_data (c, ivar);
- ds_init_empty (&l);
-
- append_value_name (ivar, val, &l);
- ds_ltrim (&l, ss_cstr (" "));
-
- ds_put_substring (&label, l.ss);
- if (ivar_idx < iact->n_vars - 1)
- ds_put_cstr (&label, "; ");
-
- ds_destroy (&l);
- }
-
- boxplot_add_box (boxplot, es[v].box_whisker, ds_cstr (&label));
- es[v].box_whisker = NULL;
-
- ds_destroy (&label);
- }
-
- boxplot_submit (boxplot);
- }
-}
-
-static void
-show_boxplot_variabled (const struct examine *cmd, int iact_idx)
-{
- int grp;
- const struct interaction *iact = cmd->iacts[iact_idx];
- const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
-
- for (grp = 0; grp < n_cats; ++grp)
- {
- struct boxplot *boxplot;
- int v;
- double y_min = DBL_MAX;
- double y_max = -DBL_MAX;
-
- const struct ccase *c =
- categoricals_get_case_by_category_real (cmd->cats, iact_idx, grp);
-
- struct string title;
- ds_init_empty (&title);
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- const struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- if (y_min > es[v].minimum)
- y_min = es[v].minimum;
-
- if (y_max < es[v].maximum)
- y_max = es[v].maximum;
- }
-
- if (iact->n_vars == 0)
- ds_put_format (&title, _("Boxplot"));
- else
- {
- int ivar_idx;
- struct string label;
- ds_init_empty (&label);
- for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
- {
- const struct variable *ivar = iact->vars[ivar_idx];
- const union value *val = case_data (c, ivar);
-
- ds_put_cstr (&label, var_to_string (ivar));
- ds_put_cstr (&label, " = ");
- append_value_name (ivar, val, &label);
- ds_put_cstr (&label, "; ");
- }
-
- ds_put_format (&title, _("Boxplot of %s"),
- ds_cstr (&label));
-
- ds_destroy (&label);
- }
-
- boxplot = boxplot_create (y_min, y_max, ds_cstr (&title));
-
- ds_destroy (&title);
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- boxplot_add_box (boxplot, es[v].box_whisker,
- var_to_string (cmd->dep_vars[v]));
- es[v].box_whisker = NULL;
- }
-
- boxplot_submit (boxplot);
- }
-}
-
-
-static void
-show_npplot (const struct examine *cmd, int iact_idx)
-{
- const struct interaction *iact = cmd->iacts[iact_idx];
- const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
-
- int v;
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- int grp;
- for (grp = 0; grp < n_cats; ++grp)
- {
- struct chart *npp, *dnpp;
- struct casereader *reader;
- struct np *np;
-
- int ivar_idx;
- const struct ccase *c =
- categoricals_get_case_by_category_real (cmd->cats,
- iact_idx, grp);
-
- const struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- struct string label;
- ds_init_cstr (&label,
- var_to_string (cmd->dep_vars[v]));
-
- if (iact->n_vars > 0)
- {
- ds_put_cstr (&label, " (");
- for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
- {
- const struct variable *ivar = iact->vars[ivar_idx];
- const union value *val = case_data (c, ivar);
-
- ds_put_cstr (&label, var_to_string (ivar));
- ds_put_cstr (&label, " = ");
- append_value_name (ivar, val, &label);
- ds_put_cstr (&label, "; ");
-
- }
- ds_put_cstr (&label, ")");
- }
-
- np = es[v].np;
- reader = casewriter_make_reader (np->writer);
- np->writer = NULL;
-
- npp = np_plot_create (np, reader, ds_cstr (&label));
- dnpp = dnp_plot_create (np, reader, ds_cstr (&label));
-
- if (npp == NULL || dnpp == NULL)
- {
- msg (MW, _("Not creating NP plot because data set is empty."));
- chart_unref (npp);
- chart_unref (dnpp);
- }
- else
- {
- chart_submit (npp);
- chart_submit (dnpp);
- }
- casereader_destroy (reader);
-
- ds_destroy (&label);
- }
- }
-}
-
-static void
-show_spreadlevel (const struct examine *cmd, int iact_idx)
-{
- const struct interaction *iact = cmd->iacts[iact_idx];
- const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
-
- int v;
-
- /* Spreadlevel when there are no levels is not useful */
- if (iact->n_vars == 0)
- return;
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- int grp;
- struct chart *sl;
-
- struct string label;
- ds_init_cstr (&label,
- var_to_string (cmd->dep_vars[v]));
-
- if (iact->n_vars > 0)
- {
- ds_put_cstr (&label, " (");
- interaction_to_string (iact, &label);
- ds_put_cstr (&label, ")");
- }
-
- sl = spreadlevel_plot_create (ds_cstr (&label), cmd->sl_power);
-
- for (grp = 0; grp < n_cats; ++grp)
- {
- const struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- double median = percentile_calculate (es[v].quartiles[1], cmd->pc_alg);
-
- double iqr = percentile_calculate (es[v].quartiles[2], cmd->pc_alg) -
- percentile_calculate (es[v].quartiles[0], cmd->pc_alg);
-
- spreadlevel_plot_add (sl, iqr, median);
- }
-
- if (sl == NULL)
- msg (MW, _("Not creating spreadlevel chart for %s"), ds_cstr (&label));
- else
- chart_submit (sl);
-
- ds_destroy (&label);
- }
-}
-
-
-static void
-show_histogram (const struct examine *cmd, int iact_idx)
-{
- const struct interaction *iact = cmd->iacts[iact_idx];
- const size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
-
- int v;
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- int grp;
- for (grp = 0; grp < n_cats; ++grp)
- {
- double n, mean, var;
- int ivar_idx;
- const struct ccase *c =
- categoricals_get_case_by_category_real (cmd->cats,
- iact_idx, grp);
-
- const struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, iact_idx, grp);
-
- struct string label;
-
- if (es[v].histogram == NULL)
- continue;
-
- ds_init_cstr (&label,
- var_to_string (cmd->dep_vars[v]));
-
- if (iact->n_vars > 0)
- {
- ds_put_cstr (&label, " (");
- for (ivar_idx = 0; ivar_idx < iact->n_vars; ++ivar_idx)
- {
- const struct variable *ivar = iact->vars[ivar_idx];
- const union value *val = case_data (c, ivar);
-
- ds_put_cstr (&label, var_to_string (ivar));
- ds_put_cstr (&label, " = ");
- append_value_name (ivar, val, &label);
- ds_put_cstr (&label, "; ");
-
- }
- ds_put_cstr (&label, ")");
- }
-
-
- moments_calculate (es[v].mom, &n, &mean, &var, NULL, NULL);
-
- chart_submit
- (histogram_chart_create (es[v].histogram->gsl_hist,
- ds_cstr (&label), n, mean,
- sqrt (var), false));
-
-
- ds_destroy (&label);
- }
- }
-}
-
-static struct pivot_value *
-new_value_with_missing_footnote (const struct variable *var,
- const union value *value,
- struct pivot_footnote *missing_footnote)
-{
- struct pivot_value *pv = pivot_value_new_var_value (var, value);
- if (var_is_value_missing (var, value) == MV_USER)
- pivot_value_add_footnote (pv, missing_footnote);
- return pv;
-}
-
-static void
-create_interaction_dimensions (struct pivot_table *table,
- const struct categoricals *cats,
- const struct interaction *iact,
- struct pivot_footnote *missing_footnote)
-{
- for (size_t i = iact->n_vars; i-- > 0;)
- {
- const struct variable *var = iact->vars[i];
- struct pivot_dimension *d = pivot_dimension_create__ (
- table, PIVOT_AXIS_ROW, pivot_value_new_variable (var));
- d->root->show_label = true;
-
- size_t n;
- union value *values = categoricals_get_var_values (cats, var, &n);
- for (size_t j = 0; j < n; j++)
- pivot_category_create_leaf (
- d->root, new_value_with_missing_footnote (var, &values[j],
- missing_footnote));
- }
-}
-
-static struct pivot_footnote *
-create_missing_footnote (struct pivot_table *table)
-{
- return pivot_table_create_footnote (
- table, pivot_value_new_text (N_("User-missing value.")));
-}
-
-static void
-percentiles_report (const struct examine *cmd, int iact_idx)
-{
- struct pivot_table *table = pivot_table_create (N_("Percentiles"));
-
- struct pivot_dimension *percentiles = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Percentiles"));
- percentiles->root->show_label = true;
- for (int i = 0; i < cmd->n_percentiles; ++i)
- pivot_category_create_leaf (
- percentiles->root,
- pivot_value_new_user_text_nocopy (xasprintf ("%g", cmd->ptiles[i])));
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Weighted Average"), N_("Tukey's Hinges"));
-
- const struct interaction *iact = cmd->iacts[iact_idx];
- struct pivot_footnote *missing_footnote = create_missing_footnote (table);
- create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
-
- struct pivot_dimension *dep_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
-
- size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- {
- indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
- dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
-
- for (size_t i = 0; i < n_cats; ++i)
- {
- for (size_t j = 0; j < iact->n_vars; j++)
- {
- int idx = categoricals_get_value_index_by_category_real (
- cmd->cats, iact_idx, i, j);
- indexes[table->n_dimensions - 2 - j] = idx;
- }
-
- const struct exploratory_stats *ess
- = categoricals_get_user_data_by_category_real (
- cmd->cats, iact_idx, i);
- const struct exploratory_stats *es = ess + v;
-
- double hinges[3];
- tukey_hinges_calculate (es->hinges, hinges);
-
- for (size_t pc_idx = 0; pc_idx < cmd->n_percentiles; ++pc_idx)
- {
- indexes[0] = pc_idx;
-
- indexes[1] = 0;
- double value = percentile_calculate (es->percentiles[pc_idx],
- cmd->pc_alg);
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (value));
-
- double hinge = (cmd->ptiles[pc_idx] == 25.0 ? hinges[0]
- : cmd->ptiles[pc_idx] == 50.0 ? hinges[1]
- : cmd->ptiles[pc_idx] == 75.0 ? hinges[2]
- : SYSMIS);
- if (hinge != SYSMIS)
- {
- indexes[1] = 1;
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (hinge));
- }
- }
- }
-
- }
- free (indexes);
-
- pivot_table_submit (table);
-}
-
-static void
-normality_report (const struct examine *cmd, int iact_idx)
-{
- struct pivot_table *table = pivot_table_create (N_("Tests of Normality"));
-
- struct pivot_dimension *test =
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Shapiro-Wilk"),
- N_("Statistic"),
- N_("df"), PIVOT_RC_COUNT,
- N_("Sig."));
-
- test->root->show_label = true;
-
- const struct interaction *iact = cmd->iacts[iact_idx];
- struct pivot_footnote *missing_footnote = create_missing_footnote (table);
- create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
-
- struct pivot_dimension *dep_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
-
- size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- {
- indexes[table->n_dimensions - 1] =
- pivot_category_create_leaf (dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
-
- for (size_t i = 0; i < n_cats; ++i)
- {
- indexes[1] = i;
-
- const struct exploratory_stats *es
- = categoricals_get_user_data_by_category_real (
- cmd->cats, iact_idx, i);
-
- struct shapiro_wilk *sw = es[v].shapiro_wilk;
-
- if (sw == NULL)
- continue;
-
- double w = shapiro_wilk_calculate (sw);
-
- int j = 0;
- indexes[0] = j;
-
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (w));
-
- indexes[0] = ++j;
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (sw->n));
-
- indexes[0] = ++j;
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (shapiro_wilk_significance (sw->n, w)));
- }
- }
-
- free (indexes);
-
- pivot_table_submit (table);
-}
-
-
-static void
-descriptives_report (const struct examine *cmd, int iact_idx)
-{
- struct pivot_table *table = pivot_table_create (N_("Descriptives"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Aspect"),
- N_("Statistic"), N_("Std. Error"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"), N_("Mean"));
- struct pivot_category *interval = pivot_category_create_group__ (
- statistics->root,
- pivot_value_new_text_format (N_("%g%% Confidence Interval for Mean"),
- cmd->conf * 100.0));
- pivot_category_create_leaves (interval, N_("Lower Bound"),
- N_("Upper Bound"));
- pivot_category_create_leaves (
- statistics->root, N_("5% Trimmed Mean"), N_("Median"), N_("Variance"),
- N_("Std. Deviation"), N_("Minimum"), N_("Maximum"), N_("Range"),
- N_("Interquartile Range"), N_("Skewness"), N_("Kurtosis"));
-
- const struct interaction *iact = cmd->iacts[iact_idx];
- struct pivot_footnote *missing_footnote = create_missing_footnote (table);
- create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
-
- struct pivot_dimension *dep_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
-
- size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- {
- indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
- dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
-
- for (size_t i = 0; i < n_cats; ++i)
- {
- for (size_t j = 0; j < iact->n_vars; j++)
- {
- int idx = categoricals_get_value_index_by_category_real (
- cmd->cats, iact_idx, i, j);
- indexes[table->n_dimensions - 2 - j] = idx;
- }
-
- const struct exploratory_stats *ess
- = categoricals_get_user_data_by_category_real (cmd->cats,
- iact_idx, i);
- const struct exploratory_stats *es = ess + v;
-
- double m0, m1, m2, m3, m4;
- moments_calculate (es->mom, &m0, &m1, &m2, &m3, &m4);
- double tval = gsl_cdf_tdist_Qinv ((1.0 - cmd->conf) / 2.0, m0 - 1.0);
-
- struct entry
- {
- int stat_idx;
- int aspect_idx;
- double x;
- }
- entries[] = {
- { 0, 0, m1 },
- { 0, 1, calc_semean (m2, m0) },
- { 1, 0, m1 - tval * calc_semean (m2, m0) },
- { 2, 0, m1 + tval * calc_semean (m2, m0) },
- { 3, 0, trimmed_mean_calculate (es->trimmed_mean) },
- { 4, 0, percentile_calculate (es->quartiles[1], cmd->pc_alg) },
- { 5, 0, m2 },
- { 6, 0, sqrt (m2) },
- { 7, 0, es->minima[0].val },
- { 8, 0, es->maxima[0].val },
- { 9, 0, es->maxima[0].val - es->minima[0].val },
- { 10, 0, (percentile_calculate (es->quartiles[2], cmd->pc_alg) -
- percentile_calculate (es->quartiles[0], cmd->pc_alg)) },
- { 11, 0, m3 },
- { 11, 1, calc_seskew (m0) },
- { 12, 0, m4 },
- { 12, 1, calc_sekurt (m0) },
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- {
- const struct entry *e = &entries[j];
- indexes[0] = e->aspect_idx;
- indexes[1] = e->stat_idx;
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (e->x));
- }
- }
- }
-
- free (indexes);
-
- pivot_table_submit (table);
-}
-
-
-static void
-extremes_report (const struct examine *cmd, int iact_idx)
-{
- struct pivot_table *table = pivot_table_create (N_("Extreme Values"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- pivot_category_create_leaf (statistics->root,
- (cmd->id_var
- ? pivot_value_new_variable (cmd->id_var)
- : pivot_value_new_text (N_("Case Number"))));
- pivot_category_create_leaves (statistics->root, N_("Value"));
-
- struct pivot_dimension *order = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Order"));
- for (size_t i = 0; i < cmd->disp_extremes; i++)
- pivot_category_create_leaf (order->root, pivot_value_new_integer (i + 1));
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW,
- /* TRANSLATORS: This is a noun, not an adjective. */
- N_("Extreme"),
- N_("Highest"), N_("Lowest"));
-
- const struct interaction *iact = cmd->iacts[iact_idx];
- struct pivot_footnote *missing_footnote = create_missing_footnote (table);
- create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
-
- struct pivot_dimension *dep_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
-
- size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- {
- indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
- dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
-
- for (size_t i = 0; i < n_cats; ++i)
- {
- for (size_t j = 0; j < iact->n_vars; j++)
- {
- int idx = categoricals_get_value_index_by_category_real (
- cmd->cats, iact_idx, i, j);
- indexes[table->n_dimensions - 2 - j] = idx;
- }
-
- const struct exploratory_stats *ess
- = categoricals_get_user_data_by_category_real (cmd->cats,
- iact_idx, i);
- const struct exploratory_stats *es = ess + v;
-
- for (int e = 0; e < cmd->disp_extremes; ++e)
- {
- indexes[1] = e;
-
- for (size_t j = 0; j < 2; j++)
- {
- const struct extremity *extremity
- = j ? &es->minima[e] : &es->maxima[e];
- indexes[2] = j;
-
- indexes[0] = 0;
- pivot_table_put (
- table, indexes, table->n_dimensions,
- (cmd->id_var
- ? new_value_with_missing_footnote (cmd->id_var,
- &extremity->identity,
- missing_footnote)
- : pivot_value_new_integer (extremity->identity.f)));
-
- indexes[0] = 1;
- union value val = { .f = extremity->val };
- pivot_table_put (
- table, indexes, table->n_dimensions,
- new_value_with_missing_footnote (cmd->dep_vars[v], &val,
- missing_footnote));
- }
- }
- }
- }
- free (indexes);
-
- pivot_table_submit (table);
-}
-
-
-static void
-summary_report (const struct examine *cmd, int iact_idx)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Case Processing Summary"));
- pivot_table_set_weight_var (table, dict_get_weight (cmd->dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Percent"), PIVOT_RC_PERCENT);
- struct pivot_dimension *cases = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Cases"), N_("Valid"), N_("Missing"),
- N_("Total"));
- cases->root->show_label = true;
-
- const struct interaction *iact = cmd->iacts[iact_idx];
- struct pivot_footnote *missing_footnote = create_missing_footnote (table);
- create_interaction_dimensions (table, cmd->cats, iact, missing_footnote);
-
- struct pivot_dimension *dep_dim = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- size_t *indexes = xnmalloc (table->n_dimensions, sizeof *indexes);
-
- size_t n_cats = categoricals_n_count (cmd->cats, iact_idx);
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- {
- indexes[table->n_dimensions - 1] = pivot_category_create_leaf (
- dep_dim->root, pivot_value_new_variable (cmd->dep_vars[v]));
-
- for (size_t i = 0; i < n_cats; ++i)
- {
- for (size_t j = 0; j < iact->n_vars; j++)
- {
- int idx = categoricals_get_value_index_by_category_real (
- cmd->cats, iact_idx, i, j);
- indexes[table->n_dimensions - 2 - j] = idx;
- }
-
- const struct exploratory_stats *es
- = categoricals_get_user_data_by_category_real (
- cmd->cats, iact_idx, i);
-
- double total = es[v].missing + es[v].non_missing;
- struct entry
- {
- int stat_idx;
- int case_idx;
- double x;
- }
- entries[] = {
- { 0, 0, es[v].non_missing },
- { 1, 0, 100.0 * es[v].non_missing / total },
- { 0, 1, es[v].missing },
- { 1, 1, 100.0 * es[v].missing / total },
- { 0, 2, total },
- { 1, 2, 100.0 },
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- {
- const struct entry *e = &entries[j];
- indexes[0] = e->stat_idx;
- indexes[1] = e->case_idx;
- pivot_table_put (table, indexes, table->n_dimensions,
- pivot_value_new_number (e->x));
- }
- }
- }
-
- free (indexes);
-
- pivot_table_submit (table);
-}
-
-/* Attempt to parse an interaction from LEXER */
-static struct interaction *
-parse_interaction (struct lexer *lexer, struct examine *ex)
-{
- const struct variable *v;
- if (!lex_match_variable (lexer, ex->dict, &v))
- return NULL;
-
- struct interaction *iact = interaction_create (v);
- while (lex_match (lexer, T_BY))
- {
- if (!lex_match_variable (lexer, ex->dict, &v))
- {
- interaction_destroy (iact);
- return NULL;
- }
- interaction_add_variable (iact, v);
- }
- lex_match (lexer, T_COMMA);
- return iact;
-}
-
-
-static void *
-create_n (const void *aux1, void *aux2 UNUSED)
-{
- int v;
-
- const struct examine *examine = aux1;
- struct exploratory_stats *es = pool_calloc (examine->pool, examine->n_dep_vars, sizeof (*es));
- struct subcase ordering;
- subcase_init (&ordering, 0, 0, SC_ASCEND);
-
- for (v = 0; v < examine->n_dep_vars; v++)
- {
- es[v].sorted_writer = sort_create_writer (&ordering, examine->ex_proto);
- es[v].sorted_reader = NULL;
-
- es[v].mom = moments_create (MOMENT_KURTOSIS);
- es[v].cmin = DBL_MAX;
-
- es[v].maximum = -DBL_MAX;
- es[v].minimum = DBL_MAX;
- }
-
- subcase_uninit (&ordering);
- return es;
-}
-
-static void
-update_n (const void *aux1, void *aux2 UNUSED, void *user_data,
- const struct ccase *c, double weight)
-{
- int v;
- const struct examine *examine = aux1;
- struct exploratory_stats *es = user_data;
-
- bool this_case_is_missing = false;
- /* LISTWISE missing must be dealt with here */
- if (!examine->missing_pw)
- {
- for (v = 0; v < examine->n_dep_vars; v++)
- {
- const struct variable *var = examine->dep_vars[v];
-
- if (var_is_value_missing (var, case_data (c, var))
- & examine->dep_excl)
- {
- es[v].missing += weight;
- this_case_is_missing = true;
- }
- }
- }
-
- if (this_case_is_missing)
- return;
-
- for (v = 0; v < examine->n_dep_vars; v++)
- {
- struct ccase *outcase;
- const struct variable *var = examine->dep_vars[v];
- const double x = case_num (c, var);
-
- if (var_is_value_missing (var, case_data (c, var)) & examine->dep_excl)
- {
- es[v].missing += weight;
- continue;
- }
-
- outcase = case_create (examine->ex_proto);
-
- if (x > es[v].maximum)
- es[v].maximum = x;
-
- if (x < es[v].minimum)
- es[v].minimum = x;
-
- es[v].non_missing += weight;
-
- moments_pass_one (es[v].mom, x, weight);
-
- /* Save the value and the ID to the writer */
- assert (examine->id_idx != -1);
- *case_num_rw_idx (outcase, EX_VAL) = x;
- value_copy (case_data_rw_idx (outcase, EX_ID),
- case_data_idx (c, examine->id_idx), examine->id_width);
-
- *case_num_rw_idx (outcase, EX_WT) = weight;
-
- es[v].cc += weight;
-
- if (es[v].cmin > weight)
- es[v].cmin = weight;
-
- casewriter_write (es[v].sorted_writer, outcase);
- }
-}
-
-static void
-calculate_n (const void *aux1, void *aux2 UNUSED, void *user_data)
-{
- int v;
- const struct examine *examine = aux1;
- struct exploratory_stats *es = user_data;
-
- for (v = 0; v < examine->n_dep_vars; v++)
- {
- int i;
- casenumber imin = 0;
- casenumber imax;
- struct casereader *reader;
- struct ccase *c;
-
- if (examine->plot_histogram && es[v].non_missing > 0)
- {
- /* Sturges Rule */
- double bin_width = fabs (es[v].minimum - es[v].maximum)
- / (1 + log2 (es[v].cc));
-
- es[v].histogram =
- histogram_create (bin_width, es[v].minimum, es[v].maximum);
- }
-
- es[v].sorted_reader = casewriter_make_reader (es[v].sorted_writer);
- es[v].sorted_writer = NULL;
-
- imax = casereader_get_n_cases (es[v].sorted_reader);
-
- es[v].maxima = pool_calloc (examine->pool, examine->calc_extremes, sizeof (*es[v].maxima));
- es[v].minima = pool_calloc (examine->pool, examine->calc_extremes, sizeof (*es[v].minima));
- for (i = 0; i < examine->calc_extremes; ++i)
- {
- value_init_pool (examine->pool, &es[v].maxima[i].identity, examine->id_width);
- value_init_pool (examine->pool, &es[v].minima[i].identity, examine->id_width);
- }
-
- bool warn = true;
- for (reader = casereader_clone (es[v].sorted_reader);
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- const double val = case_num_idx (c, EX_VAL);
- double wt = case_num_idx (c, EX_WT);
- wt = var_force_valid_weight (examine->wv, wt, &warn);
-
- moments_pass_two (es[v].mom, val, wt);
-
- if (es[v].histogram)
- histogram_add (es[v].histogram, val, wt);
-
- if (imin < examine->calc_extremes)
- {
- int x;
- for (x = imin; x < examine->calc_extremes; ++x)
- {
- struct extremity *min = &es[v].minima[x];
- min->val = val;
- value_copy (&min->identity, case_data_idx (c, EX_ID), examine->id_width);
- }
- imin ++;
- }
-
- imax --;
- if (imax < examine->calc_extremes)
- {
- int x;
-
- for (x = imax; x < imax + 1; ++x)
- {
- struct extremity *max;
-
- if (x >= examine->calc_extremes)
- break;
-
- max = &es[v].maxima[x];
- max->val = val;
- value_copy (&max->identity, case_data_idx (c, EX_ID), examine->id_width);
- }
- }
- }
- casereader_destroy (reader);
-
- if (examine->calc_extremes > 0 && es[v].non_missing > 0)
- {
- assert (es[v].minima[0].val == es[v].minimum);
- assert (es[v].maxima[0].val == es[v].maximum);
- }
-
- {
- const int n_os = 5 + examine->n_percentiles;
- es[v].percentiles = pool_calloc (examine->pool, examine->n_percentiles, sizeof (*es[v].percentiles));
-
- es[v].trimmed_mean = trimmed_mean_create (es[v].cc, 0.05);
- es[v].shapiro_wilk = NULL;
-
- struct order_stats **os = XCALLOC (n_os, struct order_stats *);
- os[0] = &es[v].trimmed_mean->parent;
-
- es[v].quartiles[0] = percentile_create (0.25, es[v].cc);
- es[v].quartiles[1] = percentile_create (0.5, es[v].cc);
- es[v].quartiles[2] = percentile_create (0.75, es[v].cc);
-
- os[1] = &es[v].quartiles[0]->parent;
- os[2] = &es[v].quartiles[1]->parent;
- os[3] = &es[v].quartiles[2]->parent;
-
- es[v].hinges = tukey_hinges_create (es[v].cc, es[v].cmin);
- os[4] = &es[v].hinges->parent;
-
- for (i = 0; i < examine->n_percentiles; ++i)
- {
- es[v].percentiles[i] = percentile_create (examine->ptiles[i] / 100.00, es[v].cc);
- os[5 + i] = &es[v].percentiles[i]->parent;
- }
-
- order_stats_accumulate_idx (os, n_os,
- casereader_clone (es[v].sorted_reader),
- EX_WT, EX_VAL);
-
- free (os);
- }
-
- if (examine->plot_boxplot)
- {
- struct order_stats *os;
-
- es[v].box_whisker = box_whisker_create (es[v].hinges,
- EX_ID, examine->id_var);
-
- os = &es[v].box_whisker->parent;
- order_stats_accumulate_idx (&os, 1,
- casereader_clone (es[v].sorted_reader),
- EX_WT, EX_VAL);
- }
-
- if (examine->plot_boxplot || examine->plot_histogram
- || examine->plot_npplot || examine->plot_spreadlevel)
- {
- double mean;
-
- moments_calculate (es[v].mom, NULL, &mean, NULL, NULL, NULL);
-
- es[v].shapiro_wilk = shapiro_wilk_create (es[v].non_missing, mean);
-
- if (es[v].shapiro_wilk)
- {
- struct order_stats *os = &es[v].shapiro_wilk->parent;
- order_stats_accumulate_idx (&os, 1,
- casereader_clone (es[v].sorted_reader),
- EX_WT, EX_VAL);
- }
- }
-
- if (examine->plot_npplot)
- {
- double n, mean, var;
- struct order_stats *os;
-
- moments_calculate (es[v].mom, &n, &mean, &var, NULL, NULL);
-
- es[v].np = np_create (n, mean, var);
-
- os = &es[v].np->parent;
-
- order_stats_accumulate_idx (&os, 1,
- casereader_clone (es[v].sorted_reader),
- EX_WT, EX_VAL);
- }
-
- }
-}
-
-static void
-cleanup_exploratory_stats (struct examine *cmd)
-{
- int i;
- for (i = 0; i < cmd->n_iacts; ++i)
- {
- int v;
- const size_t n_cats = categoricals_n_count (cmd->cats, i);
-
- for (v = 0; v < cmd->n_dep_vars; ++v)
- {
- int grp;
- for (grp = 0; grp < n_cats; ++grp)
- {
- int q;
- const struct exploratory_stats *es =
- categoricals_get_user_data_by_category_real (cmd->cats, i, grp);
-
- struct order_stats *os = &es[v].hinges->parent;
- struct statistic *stat = &os->parent;
- stat->destroy (stat);
-
- for (q = 0; q < 3; q++)
- {
- os = &es[v].quartiles[q]->parent;
- stat = &os->parent;
- stat->destroy (stat);
- }
-
- for (q = 0; q < cmd->n_percentiles; q++)
- {
- os = &es[v].percentiles[q]->parent;
- stat = &os->parent;
- stat->destroy (stat);
- }
-
- if (es[v].shapiro_wilk)
- {
- stat = &es[v].shapiro_wilk->parent.parent;
- stat->destroy (stat);
- }
-
- os = &es[v].trimmed_mean->parent;
- stat = &os->parent;
- stat->destroy (stat);
-
- os = &es[v].np->parent;
- if (os)
- {
- stat = &os->parent;
- stat->destroy (stat);
- }
-
- statistic_destroy (&es[v].histogram->parent);
- moments_destroy (es[v].mom);
-
- if (es[v].box_whisker)
- {
- stat = &es[v].box_whisker->parent.parent;
- stat->destroy (stat);
- }
-
- casereader_destroy (es[v].sorted_reader);
- }
- }
- }
-}
-
-
-static void
-run_examine (struct examine *cmd, struct casereader *input)
-{
- int i;
- struct ccase *c;
- struct casereader *reader;
-
- struct payload payload;
- payload.create = create_n;
- payload.update = update_n;
- payload.calculate = calculate_n;
- payload.destroy = NULL;
-
- cmd->wv = dict_get_weight (cmd->dict);
-
- cmd->cats
- = categoricals_create (cmd->iacts, cmd->n_iacts, cmd->wv, cmd->fctr_excl);
-
- categoricals_set_payload (cmd->cats, &payload, cmd, NULL);
-
- if (cmd->id_var == NULL)
- {
- struct ccase *c = casereader_peek (input, 0);
-
- cmd->id_idx = case_get_n_values (c);
- input = casereader_create_arithmetic_sequence (input, 1.0, 1.0);
-
- case_unref (c);
- }
-
- for (reader = input;
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- categoricals_update (cmd->cats, c);
- }
- casereader_destroy (reader);
- categoricals_done (cmd->cats);
-
- for (i = 0; i < cmd->n_iacts; ++i)
- {
- summary_report (cmd, i);
-
- const size_t n_cats = categoricals_n_count (cmd->cats, i);
- if (n_cats == 0)
- continue;
-
- if (cmd->disp_extremes > 0)
- extremes_report (cmd, i);
-
- if (cmd->n_percentiles > 0)
- percentiles_report (cmd, i);
-
- if (cmd->plot_boxplot)
- {
- switch (cmd->boxplot_mode)
- {
- case BP_GROUPS:
- show_boxplot_grouped (cmd, i);
- break;
- case BP_VARIABLES:
- show_boxplot_variabled (cmd, i);
- break;
- default:
- NOT_REACHED ();
- break;
- }
- }
-
- if (cmd->plot_histogram)
- show_histogram (cmd, i);
-
- if (cmd->plot_npplot)
- show_npplot (cmd, i);
-
- if (cmd->plot_spreadlevel)
- show_spreadlevel (cmd, i);
-
- if (cmd->descriptives)
- descriptives_report (cmd, i);
-
- if (cmd->plot_histogram || cmd->plot_npplot
- || cmd->plot_spreadlevel || cmd->plot_boxplot)
- normality_report (cmd, i);
- }
-
- cleanup_exploratory_stats (cmd);
- categoricals_destroy (cmd->cats);
-}
-
-static void
-add_interaction (struct examine *examine, struct interaction *iact,
- size_t *allocated_iacts)
-{
- if (examine->n_iacts >= *allocated_iacts)
- examine->iacts = pool_2nrealloc (examine->pool, examine->iacts,
- allocated_iacts, sizeof *examine->iacts);
- examine->iacts[examine->n_iacts++] = iact;
-}
-
-int
-cmd_examine (struct lexer *lexer, struct dataset *ds)
-{
- bool nototals_seen = false;
- bool totals_seen = false;
-
- bool percentiles_seen = false;
-
- size_t allocated_iacts = 0;
- struct examine examine = {
- .pool = pool_create (),
- .dict = dataset_dict (ds),
-
- .conf = 0.95,
- .pc_alg = PC_HAVERAGE,
- .id_idx = -1,
- .boxplot_mode = BP_GROUPS,
-
- .ex_proto = caseproto_create (),
-
- .dep_excl = MV_ANY,
- .fctr_excl = MV_ANY,
- };
-
- /* Allocate space for the first interaction.
- This is interaction is an empty one (for the totals).
- If no totals are requested, we will simply ignore this
- interaction.
- */
- add_interaction (&examine, interaction_create (NULL), &allocated_iacts);
-
- /* Accept an optional, completely pointless "/VARIABLES=" */
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "VARIABLES") && !lex_force_match (lexer, T_EQUALS))
- goto error;
-
- if (!parse_variables_const (lexer, examine.dict,
- &examine.dep_vars, &examine.n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
-
- if (lex_match (lexer, T_BY))
- {
- for (;;)
- {
- struct interaction *iact = parse_interaction (lexer, &examine);
- if (!iact)
- break;
-
- add_interaction (&examine, iact, &allocated_iacts);
- }
- }
-
- int nototals_ofs = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "DESCRIPTIVES"))
- examine.descriptives = true;
- else if (lex_match_id (lexer, "EXTREME"))
- {
- int extr = 5;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_int_range (lexer, "EXTREME", 0, INT_MAX))
- goto error;
- extr = lex_integer (lexer);
-
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- examine.disp_extremes = extr;
- }
- else if (lex_match_id (lexer, "NONE"))
- {
- }
- else if (lex_match (lexer, T_ALL))
- {
- if (examine.disp_extremes == 0)
- examine.disp_extremes = 5;
- }
- else
- {
- lex_error_expecting (lexer, "DESCRIPTIVES", "EXTREME",
- "NONE", "ALL");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "PERCENTILES"))
- {
- percentiles_seen = true;
- if (lex_match (lexer, T_LPAREN))
- {
- size_t allocated_percentiles = examine.n_percentiles;
- while (lex_is_number (lexer))
- {
- if (!lex_force_num_range_open (lexer, "PERCENTILES", 0, 100))
- goto error;
- double p = lex_number (lexer);
-
- if (examine.n_percentiles >= allocated_percentiles)
- examine.ptiles = x2nrealloc (examine.ptiles,
- &allocated_percentiles,
- sizeof *examine.ptiles);
- examine.ptiles[examine.n_percentiles++] = p;
-
- lex_get (lexer);
- lex_match (lexer, T_COMMA);
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
-
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "HAVERAGE"))
- examine.pc_alg = PC_HAVERAGE;
- else if (lex_match_id (lexer, "WAVERAGE"))
- examine.pc_alg = PC_WAVERAGE;
- else if (lex_match_id (lexer, "ROUND"))
- examine.pc_alg = PC_ROUND;
- else if (lex_match_id (lexer, "EMPIRICAL"))
- examine.pc_alg = PC_EMPIRICAL;
- else if (lex_match_id (lexer, "AEMPIRICAL"))
- examine.pc_alg = PC_AEMPIRICAL;
- else if (lex_match_id (lexer, "NONE"))
- examine.pc_alg = PC_NONE;
- else
- {
- lex_error_expecting (lexer, "HAVERAGE", "WAVERAGE",
- "ROUND", "EMPIRICAL", "AEMPIRICAL",
- "NONE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "TOTAL"))
- totals_seen = true;
- else if (lex_match_id (lexer, "NOTOTAL"))
- {
- nototals_seen = true;
- nototals_ofs = lex_ofs (lexer) - 1;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "LISTWISE"))
- examine.missing_pw = false;
- else if (lex_match_id (lexer, "PAIRWISE"))
- examine.missing_pw = true;
- else if (lex_match_id (lexer, "EXCLUDE"))
- examine.dep_excl = MV_ANY;
- else if (lex_match_id (lexer, "INCLUDE"))
- examine.dep_excl = MV_SYSTEM;
- else if (lex_match_id (lexer, "REPORT"))
- examine.fctr_excl = 0;
- else if (lex_match_id (lexer, "NOREPORT"))
- examine.fctr_excl = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "LISTWISE", "PAIRWISE",
- "EXCLUDE", "INCLUDE", "REPORT",
- "NOREPORT");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "COMPARE"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "VARIABLES"))
- examine.boxplot_mode = BP_VARIABLES;
- else if (lex_match_id (lexer, "GROUPS"))
- examine.boxplot_mode = BP_GROUPS;
- else
- {
- lex_error_expecting (lexer, "VARIABLES", "GROUPS");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "PLOT"))
- {
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "BOXPLOT"))
- examine.plot_boxplot = true;
- else if (lex_match_id (lexer, "NPPLOT"))
- examine.plot_npplot = true;
- else if (lex_match_id (lexer, "HISTOGRAM"))
- examine.plot_histogram = true;
- else if (lex_match_id (lexer, "SPREADLEVEL"))
- {
- examine.plot_spreadlevel = true;
- examine.sl_power = 0;
- if (lex_match (lexer, T_LPAREN) && lex_force_num (lexer))
- {
- examine.sl_power = lex_number (lexer);
-
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
- else if (lex_match_id (lexer, "NONE"))
- examine.plot_boxplot = examine.plot_npplot
- = examine.plot_histogram = examine.plot_spreadlevel = false;
- else if (lex_match (lexer, T_ALL))
- examine.plot_boxplot = examine.plot_npplot
- = examine.plot_histogram = examine.plot_spreadlevel = true;
- else
- {
- lex_error_expecting (lexer, "BOXPLOT", "NPPLOT",
- "HISTOGRAM", "SPREADLEVEL",
- "NONE", "ALL");
- goto error;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "CINTERVAL"))
- {
- if (!lex_force_num (lexer))
- goto error;
-
- examine.conf = lex_number (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "ID"))
- {
- lex_match (lexer, T_EQUALS);
-
- examine.id_var = parse_variable_const (lexer, examine.dict);
- if (!examine.id_var)
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "STATISTICS", "PERCENTILES",
- "TOTAL", "NOTOTAL", "MISSING", "COMPARE",
- "PLOT", "CINTERVAL", "ID");
- goto error;
- }
- }
-
-
- if (totals_seen && nototals_seen)
- {
- lex_ofs_error (lexer, nototals_ofs, nototals_ofs,
- _("%s and %s are mutually exclusive."),
- "TOTAL", "NOTOTAL");
- goto error;
- }
-
- /* If totals have been requested or if there are no factors
- in this analysis, then the totals need to be included. */
- if (nototals_seen && examine.n_iacts > 1)
- {
- interaction_destroy (examine.iacts[0]);
- examine.iacts++;
- examine.n_iacts--;
- }
-
- if (examine.id_var)
- {
- examine.id_idx = var_get_case_index (examine.id_var);
- examine.id_width = var_get_width (examine.id_var);
- }
-
- examine.ex_proto = caseproto_add_width (examine.ex_proto, 0); /* value */
- examine.ex_proto = caseproto_add_width (examine.ex_proto, examine.id_width); /* id */
- examine.ex_proto = caseproto_add_width (examine.ex_proto, 0); /* weight */
-
- if (examine.disp_extremes > 0)
- examine.calc_extremes = examine.disp_extremes;
-
- if (examine.descriptives && examine.calc_extremes == 0)
- {
- /* Descriptives always displays the max and min */
- examine.calc_extremes = 1;
- }
-
- if (percentiles_seen && examine.n_percentiles == 0)
- {
- examine.n_percentiles = 7;
- examine.ptiles = xmalloc (examine.n_percentiles * sizeof *examine.ptiles);
-
- examine.ptiles[0] = 5;
- examine.ptiles[1] = 10;
- examine.ptiles[2] = 25;
- examine.ptiles[3] = 50;
- examine.ptiles[4] = 75;
- examine.ptiles[5] = 90;
- examine.ptiles[6] = 95;
- }
-
- assert (examine.calc_extremes >= examine.disp_extremes);
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), examine.dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- run_examine (&examine, group);
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- caseproto_unref (examine.ex_proto);
-
- for (size_t i = 0; i < examine.n_iacts; ++i)
- interaction_destroy (examine.iacts[i]);
- free (examine.ptiles);
- free (examine.dep_vars);
- pool_destroy (examine.pool);
-
- return CMD_SUCCESS;
-
- error:
- caseproto_unref (examine.ex_proto);
- for (size_t i = 0; i < examine.n_iacts; ++i)
- interaction_destroy (examine.iacts[i]);
- free (examine.dep_vars);
- free (examine.ptiles);
- pool_destroy (examine.pool);
-
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009, 2010, 2011, 2012, 2014, 2015,
- 2016, 2017 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_vector.h>
-#include <gsl/gsl_linalg.h>
-#include <gsl/gsl_matrix.h>
-#include <gsl/gsl_eigen.h>
-#include <gsl/gsl_blas.h>
-#include <gsl/gsl_sort_vector.h>
-#include <gsl/gsl_cdf.h>
-
-#include "data/any-reader.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "language/data-io/file-handle.h"
-#include "language/data-io/matrix-reader.h"
-#include "libpspp/cast.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/correlation.h"
-#include "math/covariance.h"
-#include "math/moments.h"
-#include "output/charts/scree.h"
-#include "output/pivot-table.h"
-
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-enum method
- {
- METHOD_CORR,
- METHOD_COV
- };
-
-enum missing_type
- {
- MISS_LISTWISE,
- MISS_PAIRWISE,
- MISS_MEANSUB,
- };
-
-enum extraction_method
- {
- EXTRACTION_PC,
- EXTRACTION_PAF,
- };
-
-enum plot_opts
- {
- PLOT_SCREE = 0x0001,
- PLOT_ROTATION = 0x0002
- };
-
-enum print_opts
- {
- PRINT_UNIVARIATE = 1 << 0,
- PRINT_DETERMINANT = 1 << 1,
- PRINT_INV = 1 << 2,
- PRINT_AIC = 1 << 3,
- PRINT_SIG = 1 << 4,
- PRINT_COVARIANCE = 1 << 5,
- PRINT_CORRELATION = 1 << 6,
- PRINT_ROTATION = 1 << 7,
- PRINT_EXTRACTION = 1 << 8,
- PRINT_INITIAL = 1 << 9,
- PRINT_KMO = 1 << 10,
- PRINT_REPR = 1 << 11,
- PRINT_FSCORE = 1 << 12
- };
-
-enum rotation_type
- {
- ROT_VARIMAX = 0,
- ROT_EQUAMAX,
- ROT_QUARTIMAX,
- ROT_PROMAX,
- ROT_NONE
- };
-
-typedef void (*rotation_coefficients) (double *x, double *y,
- double a, double b, double c, double d,
- const gsl_matrix *loadings);
-
-
-static void
-varimax_coefficients (double *x, double *y,
- double a, double b, double c, double d,
- const gsl_matrix *loadings)
-{
- *x = d - 2 * a * b / loadings->size1;
- *y = c - (a * a - b * b) / loadings->size1;
-}
-
-static void
-equamax_coefficients (double *x, double *y,
- double a, double b, double c, double d,
- const gsl_matrix *loadings)
-{
- *x = d - loadings->size2 * a * b / loadings->size1;
- *y = c - loadings->size2 * (a * a - b * b) / (2 * loadings->size1);
-}
-
-static void
-quartimax_coefficients (double *x, double *y,
- double a UNUSED, double b UNUSED, double c, double d,
- const gsl_matrix *loadings UNUSED)
-{
- *x = d;
- *y = c;
-}
-
-static const rotation_coefficients rotation_coeff[] = {
- varimax_coefficients,
- equamax_coefficients,
- quartimax_coefficients,
- varimax_coefficients /* PROMAX is identical to VARIMAX */
-};
-
-
-/* return diag (C'C) ^ {-0.5} */
-static gsl_matrix *
-diag_rcp_sqrt (const gsl_matrix *C)
-{
- gsl_matrix *d = gsl_matrix_calloc (C->size1, C->size2);
- gsl_matrix *r = gsl_matrix_calloc (C->size1, C->size2);
-
- assert (C->size1 == C->size2);
-
- gsl_linalg_matmult_mod (C, GSL_LINALG_MOD_TRANSPOSE,
- C, GSL_LINALG_MOD_NONE,
- d);
-
- for (int j = 0; j < d->size2; ++j)
- {
- double e = gsl_matrix_get (d, j, j);
- e = 1.0 / sqrt (e);
- gsl_matrix_set (r, j, j, e);
- }
-
- gsl_matrix_free (d);
-
- return r;
-}
-
-
-
-/* return diag ((C'C)^-1) ^ {-0.5} */
-static gsl_matrix *
-diag_rcp_inv_sqrt (const gsl_matrix *CCinv)
-{
- gsl_matrix *r = gsl_matrix_calloc (CCinv->size1, CCinv->size2);
-
- assert (CCinv->size1 == CCinv->size2);
-
- for (int j = 0; j < CCinv->size2; ++j)
- {
- double e = gsl_matrix_get (CCinv, j, j);
- e = 1.0 / sqrt (e);
- gsl_matrix_set (r, j, j, e);
- }
-
- return r;
-}
-
-
-
-
-
-struct cmd_factor
-{
- size_t n_vars;
- const struct variable **vars;
-
- const struct variable *wv;
-
- enum method method;
- enum missing_type missing_type;
- enum mv_class exclude;
- enum print_opts print;
- enum extraction_method extraction;
- enum plot_opts plot;
- enum rotation_type rotation;
- int rotation_iterations;
- int promax_power;
-
- /* Extraction Criteria */
- int n_factors;
- double min_eigen;
- double econverge;
- int extraction_iterations;
-
- double rconverge;
-
- /* Format */
- double blank;
- bool sort;
-};
-
-
-struct idata
-{
- /* Intermediate values used in calculation */
- struct matrix_material mm;
-
- gsl_matrix *analysis_matrix; /* A pointer to either mm.corr or mm.cov */
-
- gsl_vector *eval; /* The eigenvalues */
- gsl_matrix *evec; /* The eigenvectors */
-
- int n_extractions;
-
- gsl_vector *msr; /* Multiple Squared Regressions */
-
- double detR; /* The determinant of the correlation matrix */
-
- gsl_matrix *ai_cov; /* The anti-image covariance matrix */
- gsl_matrix *ai_cor; /* The anti-image correlation matrix */
- struct covariance *cvm;
-};
-
-static struct idata *
-idata_alloc (size_t n_vars)
-{
- struct idata *id = XZALLOC (struct idata);
-
- id->n_extractions = 0;
- id->msr = gsl_vector_alloc (n_vars);
-
- id->eval = gsl_vector_alloc (n_vars);
- id->evec = gsl_matrix_alloc (n_vars, n_vars);
-
- return id;
-}
-
-static void
-idata_free (struct idata *id)
-{
- gsl_vector_free (id->msr);
- gsl_vector_free (id->eval);
- gsl_matrix_free (id->evec);
- gsl_matrix_free (id->ai_cov);
- gsl_matrix_free (id->ai_cor);
-
- free (id);
-}
-
-/* Return the sum of squares of all the elements in row J excluding column J */
-static double
-ssq_row_od_n (const gsl_matrix *m, int j)
-{
- assert (m->size1 == m->size2);
- assert (j < m->size1);
-
- double ss = 0;
- for (int i = 0; i < m->size1; ++i)
- if (i != j)
- ss += pow2 (gsl_matrix_get (m, i, j));
- return ss;
-}
-
-/* Return the sum of squares of all the elements excluding row N */
-static double
-ssq_od_n (const gsl_matrix *m, int n)
-{
- assert (m->size1 == m->size2);
- assert (n < m->size1);
-
- double ss = 0;
- for (int i = 0; i < m->size1; ++i)
- for (int j = 0; j < m->size2; ++j)
- if (i != j)
- ss += pow2 (gsl_matrix_get (m, i, j));
- return ss;
-}
-
-
-static gsl_matrix *
-anti_image_corr (const gsl_matrix *m, const struct idata *idata)
-{
- assert (m->size1 == m->size2);
-
- gsl_matrix *a = gsl_matrix_alloc (m->size1, m->size2);
- for (int i = 0; i < m->size1; ++i)
- for (int j = 0; j < m->size2; ++j)
- {
- double *p = gsl_matrix_ptr (a, i, j);
- *p = gsl_matrix_get (m, i, j);
- *p /= sqrt (gsl_matrix_get (m, i, i) *
- gsl_matrix_get (m, j, j));
- }
-
- for (int i = 0; i < m->size1; ++i)
- {
- double r = ssq_row_od_n (idata->mm.corr, i);
- double u = ssq_row_od_n (a, i);
- gsl_matrix_set (a, i, i, r / (r + u));
- }
-
- return a;
-}
-
-static gsl_matrix *
-anti_image_cov (const gsl_matrix *m)
-{
- assert (m->size1 == m->size2);
-
- gsl_matrix *a = gsl_matrix_alloc (m->size1, m->size2);
- for (int i = 0; i < m->size1; ++i)
- for (int j = 0; j < m->size2; ++j)
- {
- double *p = gsl_matrix_ptr (a, i, j);
- *p = gsl_matrix_get (m, i, j);
- *p /= gsl_matrix_get (m, i, i);
- *p /= gsl_matrix_get (m, j, j);
- }
-
- return a;
-}
-
-#if 0
-static void
-dump_matrix (const gsl_matrix *m)
-{
- for (int i = 0; i < m->size1; ++i)
- {
- for (int j = 0; j < m->size2; ++j)
- printf ("%02f ", gsl_matrix_get (m, i, j));
- printf ("\n");
- }
-}
-
-static void
-dump_matrix_permute (const gsl_matrix *m, const gsl_permutation *p)
-{
- for (int i = 0; i < m->size1; ++i)
- {
- for (int j = 0; j < m->size2; ++j)
- printf ("%02f ", gsl_matrix_get (m, gsl_permutation_get (p, i), j));
- printf ("\n");
- }
-}
-
-
-static void
-dump_vector (const gsl_vector *v)
-{
- for (size_t i = 0; i < v->size; ++i)
- printf ("%02f\n", gsl_vector_get (v, i));
- printf ("\n");
-}
-#endif
-
-
-static int
-n_extracted_factors (const struct cmd_factor *factor, struct idata *idata)
-{
- /* If there is a cached value, then return that. */
- if (idata->n_extractions != 0)
- return idata->n_extractions;
-
- /* Otherwise, if the number of factors has been explicitly requested,
- use that. */
- if (factor->n_factors > 0)
- {
- idata->n_extractions = factor->n_factors;
- goto finish;
- }
-
- /* Use the MIN_EIGEN setting. */
- for (int i = 0; i < idata->eval->size; ++i)
- {
- double evali = fabs (gsl_vector_get (idata->eval, i));
-
- idata->n_extractions = i;
-
- if (evali < factor->min_eigen)
- goto finish;
- }
-
- finish:
- return idata->n_extractions;
-}
-
-
-/* Returns a newly allocated matrix identical to M.
- It is the callers responsibility to free the returned value.
-*/
-static gsl_matrix *
-matrix_dup (const gsl_matrix *m)
-{
- gsl_matrix *n = gsl_matrix_alloc (m->size1, m->size2);
- gsl_matrix_memcpy (n, m);
- return n;
-}
-
-
-struct smr_workspace
-{
- /* Copy of the subject */
- gsl_matrix *m;
-
- gsl_matrix *inverse;
-
- gsl_permutation *perm;
-
- gsl_matrix *result1;
- gsl_matrix *result2;
-};
-
-
-static struct smr_workspace *ws_create (const gsl_matrix *input)
-{
- struct smr_workspace *ws = xmalloc (sizeof (*ws));
-
- ws->m = gsl_matrix_alloc (input->size1, input->size2);
- ws->inverse = gsl_matrix_calloc (input->size1 - 1, input->size2 - 1);
- ws->perm = gsl_permutation_alloc (input->size1 - 1);
- ws->result1 = gsl_matrix_calloc (input->size1 - 1, 1);
- ws->result2 = gsl_matrix_calloc (1, 1);
-
- return ws;
-}
-
-static void
-ws_destroy (struct smr_workspace *ws)
-{
- gsl_matrix_free (ws->result2);
- gsl_matrix_free (ws->result1);
- gsl_permutation_free (ws->perm);
- gsl_matrix_free (ws->inverse);
- gsl_matrix_free (ws->m);
-
- free (ws);
-}
-
-
-/*
- Return the square of the regression coefficient for VAR regressed against all other variables.
- */
-static double
-squared_multiple_correlation (const gsl_matrix *corr, int var, struct smr_workspace *ws)
-{
- /* For an explanation of what this is doing, see
- http://www.visualstatistics.net/Visual%20Statistics%20Multimedia/multiple_regression_analysis.htm
- */
-
- gsl_matrix_memcpy (ws->m, corr);
-
- gsl_matrix_swap_rows (ws->m, 0, var);
- gsl_matrix_swap_columns (ws->m, 0, var);
-
- gsl_matrix_view rxx = gsl_matrix_submatrix (ws->m, 1, 1, ws->m->size1 - 1, ws->m->size1 - 1);
-
- int signum = 0;
- gsl_linalg_LU_decomp (&rxx.matrix, ws->perm, &signum);
-
- gsl_linalg_LU_invert (&rxx.matrix, ws->perm, ws->inverse);
-
- gsl_matrix_const_view rxy = gsl_matrix_const_submatrix (ws->m, 1, 0, ws->m->size1 - 1, 1);
- gsl_matrix_const_view ryx = gsl_matrix_const_submatrix (ws->m, 0, 1, 1, ws->m->size1 - 1);
-
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans,
- 1.0, ws->inverse, &rxy.matrix, 0.0, ws->result1);
-
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans,
- 1.0, &ryx.matrix, ws->result1, 0.0, ws->result2);
-
- return gsl_matrix_get (ws->result2, 0, 0);
-}
-
-
-
-static double the_communality (const gsl_matrix *evec, const gsl_vector *eval, int n, int n_factors);
-
-
-struct factor_matrix_workspace
-{
- size_t n_factors;
- gsl_eigen_symmv_workspace *eigen_ws;
-
- gsl_vector *eval;
- gsl_matrix *evec;
-
- gsl_matrix *gamma;
-
- gsl_matrix *r;
-};
-
-static struct factor_matrix_workspace *
-factor_matrix_workspace_alloc (size_t n, size_t nf)
-{
- struct factor_matrix_workspace *ws = xmalloc (sizeof (*ws));
-
- ws->n_factors = nf;
- ws->gamma = gsl_matrix_calloc (nf, nf);
- ws->eigen_ws = gsl_eigen_symmv_alloc (n);
- ws->eval = gsl_vector_alloc (n);
- ws->evec = gsl_matrix_alloc (n, n);
- ws->r = gsl_matrix_alloc (n, n);
-
- return ws;
-}
-
-static void
-factor_matrix_workspace_free (struct factor_matrix_workspace *ws)
-{
- gsl_eigen_symmv_free (ws->eigen_ws);
- gsl_vector_free (ws->eval);
- gsl_matrix_free (ws->evec);
- gsl_matrix_free (ws->gamma);
- gsl_matrix_free (ws->r);
- free (ws);
-}
-
-/*
- Shift P left by OFFSET places, and overwrite TARGET
- with the shifted result.
- Positions in TARGET less than OFFSET are unchanged.
-*/
-static void
-perm_shift_apply (gsl_permutation *target, const gsl_permutation *p,
- size_t offset)
-{
- assert (target->size == p->size);
- assert (offset <= target->size);
-
- for (size_t i = 0; i < target->size - offset; ++i)
- target->data[i] = p->data [i + offset];
-}
-
-
-/*
- Indirectly sort the rows of matrix INPUT, storing the sort order in PERM.
- The sort criteria are as follows:
-
- Rows are sorted on the first column, until the absolute value of an
- element in a subsequent column is greater than that of the first
- column. Thereafter, rows will be sorted on the second column,
- until the absolute value of an element in a subsequent column
- exceeds that of the second column ...
-*/
-static void
-sort_matrix_indirect (const gsl_matrix *input, gsl_permutation *perm)
-{
- assert (perm->size == input->size1);
-
- const size_t n = perm->size;
- const size_t m = input->size2;
- gsl_permutation *p = gsl_permutation_alloc (n);
-
- /* Copy INPUT into MAT, discarding the sign */
- gsl_matrix *mat = gsl_matrix_alloc (n, m);
- for (int i = 0; i < mat->size1; ++i)
- for (int j = 0; j < mat->size2; ++j)
- gsl_matrix_set (mat, i, j, fabs (gsl_matrix_get (input, i, j)));
-
- int column_n = 0;
- int row_n = 0;
- while (column_n < m && row_n < n)
- {
- gsl_vector_const_view columni = gsl_matrix_const_column (mat, column_n);
- gsl_sort_vector_index (p, &columni.vector);
-
- int i;
- for (i = 0; i < n; ++i)
- {
- gsl_vector_view row = gsl_matrix_row (mat, p->data[n - 1 - i]);
- size_t maxindex = gsl_vector_max_index (&row.vector);
-
- if (maxindex > column_n)
- break;
-
- /* All subsequent elements of this row, are of no interest.
- So set them all to a highly negative value */
- for (int j = column_n + 1; j < row.vector.size; ++j)
- gsl_vector_set (&row.vector, j, -DBL_MAX);
- }
-
- perm_shift_apply (perm, p, row_n);
- row_n += i;
-
- column_n++;
- }
-
- gsl_permutation_free (p);
- gsl_matrix_free (mat);
-
- assert (0 == gsl_permutation_valid (perm));
-
- /* We want the biggest value to be first */
- gsl_permutation_reverse (perm);
-}
-
-
-static void
-drot_go (double phi, double *l0, double *l1)
-{
- double r0 = cos (phi) * *l0 + sin (phi) * *l1;
- double r1 = - sin (phi) * *l0 + cos (phi) * *l1;
-
- *l0 = r0;
- *l1 = r1;
-}
-
-
-static gsl_matrix *
-clone_matrix (const gsl_matrix *m)
-{
- gsl_matrix *c = gsl_matrix_calloc (m->size1, m->size2);
-
- for (int j = 0; j < c->size1; ++j)
- for (int k = 0; k < c->size2; ++k)
- gsl_matrix_set (c, j, k, gsl_matrix_get (m, j, k));
-
- return c;
-}
-
-
-static double
-initial_sv (const gsl_matrix *fm)
-{
- double sv = 0.0;
- for (int j = 0; j < fm->size2; ++j)
- {
- double l4s = 0;
- double l2s = 0;
-
- for (int k = j + 1; k < fm->size2; ++k)
- {
- double lambda = gsl_matrix_get (fm, k, j);
- double lambda_sq = lambda * lambda;
- double lambda_4 = lambda_sq * lambda_sq;
-
- l4s += lambda_4;
- l2s += lambda_sq;
- }
- sv += (fm->size1 * l4s - (l2s * l2s)) / (fm->size1 * fm->size1);
- }
- return sv;
-}
-
-static void
-rotate (const struct cmd_factor *cf, const gsl_matrix *unrot,
- const gsl_vector *communalities,
- gsl_matrix *result,
- gsl_vector *rotated_loadings,
- gsl_matrix *pattern_matrix,
- gsl_matrix *factor_correlation_matrix)
-{
- /* First get a normalised version of UNROT */
- gsl_matrix *normalised = gsl_matrix_calloc (unrot->size1, unrot->size2);
- gsl_matrix *h_sqrt = gsl_matrix_calloc (communalities->size, communalities->size);
- gsl_matrix *h_sqrt_inv;
-
- /* H is the diagonal matrix containing the absolute values of the communalities */
- for (int i = 0; i < communalities->size; ++i)
- {
- double *ptr = gsl_matrix_ptr (h_sqrt, i, i);
- *ptr = fabs (gsl_vector_get (communalities, i));
- }
-
- /* Take the square root of the communalities */
- gsl_linalg_cholesky_decomp (h_sqrt);
-
- /* Save a copy of h_sqrt and invert it */
- h_sqrt_inv = clone_matrix (h_sqrt);
- gsl_linalg_cholesky_decomp (h_sqrt_inv);
- gsl_linalg_cholesky_invert (h_sqrt_inv);
-
- /* normalised vertion is H^{1/2} x UNROT */
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, h_sqrt_inv, unrot, 0.0, normalised);
-
- gsl_matrix_free (h_sqrt_inv);
-
- /* Now perform the rotation iterations */
- double prev_sv = initial_sv (normalised);
- for (int i = 0; i < cf->rotation_iterations; ++i)
- {
- double sv = 0.0;
- for (int j = 0; j < normalised->size2; ++j)
- {
- /* These variables relate to the convergence criterium */
- double l4s = 0;
- double l2s = 0;
-
- for (int k = j + 1; k < normalised->size2; ++k)
- {
- double a = 0.0;
- double b = 0.0;
- double c = 0.0;
- double d = 0.0;
- for (int p = 0; p < normalised->size1; ++p)
- {
- double jv = gsl_matrix_get (normalised, p, j);
- double kv = gsl_matrix_get (normalised, p, k);
-
- double u = jv * jv - kv * kv;
- double v = 2 * jv * kv;
- a += u;
- b += v;
- c += u * u - v * v;
- d += 2 * u * v;
- }
-
- double x, y;
- rotation_coeff [cf->rotation] (&x, &y, a, b, c, d, normalised);
- double phi = atan2 (x, y) / 4.0;
-
- /* Don't bother rotating if the angle is small */
- if (fabs (sin (phi)) <= pow (10.0, -15.0))
- continue;
-
- for (int p = 0; p < normalised->size1; ++p)
- {
- double *lambda0 = gsl_matrix_ptr (normalised, p, j);
- double *lambda1 = gsl_matrix_ptr (normalised, p, k);
- drot_go (phi, lambda0, lambda1);
- }
-
- /* Calculate the convergence criterium */
- double lambda = gsl_matrix_get (normalised, k, j);
- double lambda_sq = lambda * lambda;
- double lambda_4 = lambda_sq * lambda_sq;
-
- l4s += lambda_4;
- l2s += lambda_sq;
- }
- sv += (normalised->size1 * l4s - (l2s * l2s)) / (normalised->size1 * normalised->size1);
- }
-
- if (fabs (sv - prev_sv) <= cf->rconverge)
- break;
-
- prev_sv = sv;
- }
-
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0,
- h_sqrt, normalised, 0.0, result);
-
- gsl_matrix_free (h_sqrt);
- gsl_matrix_free (normalised);
-
- if (cf->rotation == ROT_PROMAX)
- {
- /* general purpose m by m matrix, where m is the number of factors */
- gsl_matrix *mm1 = gsl_matrix_calloc (unrot->size2, unrot->size2);
- gsl_matrix *mm2 = gsl_matrix_calloc (unrot->size2, unrot->size2);
-
- /* general purpose m by p matrix, where p is the number of variables */
- gsl_matrix *mp1 = gsl_matrix_calloc (unrot->size2, unrot->size1);
-
- gsl_matrix *pm1 = gsl_matrix_calloc (unrot->size1, unrot->size2);
-
- gsl_permutation *perm = gsl_permutation_alloc (unrot->size2);
-
-
- /* The following variables follow the notation by SPSS Statistical
- Algorithms page 342. */
- gsl_matrix *L = gsl_matrix_calloc (unrot->size2, unrot->size2);
- gsl_matrix *P = clone_matrix (result);
-
- /* Vector of length p containing (indexed by i)
- \Sum^m_j {\lambda^2_{ij}} */
- gsl_vector *rssq = gsl_vector_calloc (unrot->size1);
-
- for (int i = 0; i < P->size1; ++i)
- {
- double sum = 0;
- for (int j = 0; j < P->size2; ++j)
- sum += gsl_matrix_get (result, i, j) * gsl_matrix_get (result, i, j);
- gsl_vector_set (rssq, i, sqrt (sum));
- }
-
- for (int i = 0; i < P->size1; ++i)
- {
- for (int j = 0; j < P->size2; ++j)
- {
- double l = gsl_matrix_get (result, i, j);
- double r = gsl_vector_get (rssq, i);
- gsl_matrix_set (P, i, j, pow (fabs (l / r), cf->promax_power + 1) * r / l);
- }
- }
-
- gsl_vector_free (rssq);
-
- gsl_linalg_matmult_mod (result,
- GSL_LINALG_MOD_TRANSPOSE,
- result,
- GSL_LINALG_MOD_NONE,
- mm1);
-
- int signum;
- gsl_linalg_LU_decomp (mm1, perm, &signum);
- gsl_linalg_LU_invert (mm1, perm, mm2);
-
- gsl_linalg_matmult_mod (mm2, GSL_LINALG_MOD_NONE,
- result, GSL_LINALG_MOD_TRANSPOSE,
- mp1);
-
- gsl_linalg_matmult_mod (mp1, GSL_LINALG_MOD_NONE,
- P, GSL_LINALG_MOD_NONE,
- L);
-
- gsl_matrix *D = diag_rcp_sqrt (L);
- gsl_matrix *Q = gsl_matrix_calloc (unrot->size2, unrot->size2);
-
- gsl_linalg_matmult_mod (L, GSL_LINALG_MOD_NONE,
- D, GSL_LINALG_MOD_NONE,
- Q);
-
- gsl_matrix *QQinv = gsl_matrix_calloc (unrot->size2, unrot->size2);
-
- gsl_linalg_matmult_mod (Q, GSL_LINALG_MOD_TRANSPOSE,
- Q, GSL_LINALG_MOD_NONE,
- QQinv);
-
- gsl_linalg_cholesky_decomp (QQinv);
- gsl_linalg_cholesky_invert (QQinv);
-
-
- gsl_matrix *C = diag_rcp_inv_sqrt (QQinv);
- gsl_matrix *Cinv = clone_matrix (C);
-
- gsl_linalg_cholesky_decomp (Cinv);
- gsl_linalg_cholesky_invert (Cinv);
-
-
- gsl_linalg_matmult_mod (result, GSL_LINALG_MOD_NONE,
- Q, GSL_LINALG_MOD_NONE,
- pm1);
-
- gsl_linalg_matmult_mod (pm1, GSL_LINALG_MOD_NONE,
- Cinv, GSL_LINALG_MOD_NONE,
- pattern_matrix);
-
-
- gsl_linalg_matmult_mod (C, GSL_LINALG_MOD_NONE,
- QQinv, GSL_LINALG_MOD_NONE,
- mm1);
-
- gsl_linalg_matmult_mod (mm1, GSL_LINALG_MOD_NONE,
- C, GSL_LINALG_MOD_TRANSPOSE,
- factor_correlation_matrix);
-
- gsl_linalg_matmult_mod (pattern_matrix, GSL_LINALG_MOD_NONE,
- factor_correlation_matrix, GSL_LINALG_MOD_NONE,
- pm1);
-
- gsl_matrix_memcpy (result, pm1);
-
-
- gsl_matrix_free (QQinv);
- gsl_matrix_free (C);
- gsl_matrix_free (Cinv);
-
- gsl_matrix_free (D);
- gsl_matrix_free (Q);
- gsl_matrix_free (L);
- gsl_matrix_free (P);
-
- gsl_permutation_free (perm);
-
- gsl_matrix_free (mm1);
- gsl_matrix_free (mm2);
- gsl_matrix_free (mp1);
- gsl_matrix_free (pm1);
- }
-
-
- /* reflect negative sums and populate the rotated loadings vector*/
- for (int i = 0; i < result->size2; ++i)
- {
- double ssq = 0.0;
- double sum = 0.0;
- for (int j = 0; j < result->size1; ++j)
- {
- double s = gsl_matrix_get (result, j, i);
- ssq += s * s;
- sum += s;
- }
-
- gsl_vector_set (rotated_loadings, i, ssq);
-
- if (sum < 0)
- for (int j = 0; j < result->size1; ++j)
- {
- double *lambda = gsl_matrix_ptr (result, j, i);
- *lambda = - *lambda;
- }
- }
-}
-
-/*
- Get an approximation for the factor matrix into FACTORS, and the communalities into COMMUNALITIES.
- R is the matrix to be analysed.
- WS is a pointer to a structure which must have been initialised with factor_matrix_workspace_init.
- */
-static void
-iterate_factor_matrix (const gsl_matrix *r, gsl_vector *communalities, gsl_matrix *factors,
- struct factor_matrix_workspace *ws)
-{
- assert (r->size1 == r->size2);
- assert (r->size1 == communalities->size);
-
- assert (factors->size1 == r->size1);
- assert (factors->size2 == ws->n_factors);
-
- gsl_matrix_memcpy (ws->r, r);
-
- /* Apply Communalities to diagonal of correlation matrix */
- for (size_t i = 0; i < communalities->size; ++i)
- {
- double *x = gsl_matrix_ptr (ws->r, i, i);
- *x = gsl_vector_get (communalities, i);
- }
-
- gsl_eigen_symmv (ws->r, ws->eval, ws->evec, ws->eigen_ws);
-
- gsl_matrix_view mv = gsl_matrix_submatrix (ws->evec, 0, 0, ws->evec->size1, ws->n_factors);
-
- /* Gamma is the diagonal matrix containing the absolute values of the eigenvalues */
- for (size_t i = 0; i < ws->n_factors; ++i)
- {
- double *ptr = gsl_matrix_ptr (ws->gamma, i, i);
- *ptr = fabs (gsl_vector_get (ws->eval, i));
- }
-
- /* Take the square root of gamma */
- gsl_linalg_cholesky_decomp (ws->gamma);
-
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, &mv.matrix, ws->gamma, 0.0, factors);
-
- for (size_t i = 0; i < r->size1; ++i)
- {
- double h = the_communality (ws->evec, ws->eval, i, ws->n_factors);
- gsl_vector_set (communalities, i, h);
- }
-}
-
-
-
-static bool run_factor (struct dataset *ds, const struct cmd_factor *factor);
-
-static void do_factor_by_matrix (const struct cmd_factor *factor, struct idata *idata);
-
-
-
-int
-cmd_factor (struct lexer *lexer, struct dataset *ds)
-{
- int n_iterations = 25;
-
- struct cmd_factor factor = {
- .n_vars = 0,
- .vars = NULL,
- .method = METHOD_CORR,
- .missing_type = MISS_LISTWISE,
- .exclude = MV_ANY,
- .print = PRINT_INITIAL | PRINT_EXTRACTION | PRINT_ROTATION,
- .extraction = EXTRACTION_PC,
- .n_factors = 0,
- .min_eigen = SYSMIS,
- .extraction_iterations = 25,
- .rotation_iterations = 25,
- .econverge = 0.001,
-
- .blank = 0,
- .sort = false,
- .plot = 0,
- .rotation = ROT_VARIMAX,
- .wv = NULL,
-
- .rconverge = 0.0001,
- };
-
- lex_match (lexer, T_SLASH);
-
- struct dictionary *dict = NULL;
- struct matrix_reader *mr = NULL;
- struct casereader *matrix_reader = NULL;
-
- int vars_start, vars_end;
- if (lex_match_id (lexer, "VARIABLES"))
- {
- lex_match (lexer, T_EQUALS);
- dict = dataset_dict (ds);
- factor.wv = dict_get_weight (dict);
-
- vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, dict, &factor.vars, &factor.n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- vars_end = lex_ofs (lexer) - 1;
- }
- else if (lex_match_id (lexer, "MATRIX"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_match_phrase (lexer, "IN("))
- goto error;
- if (!lex_match_id (lexer, "CORR") && !lex_match_id (lexer, "COV"))
- {
- lex_error (lexer, _("Matrix input for %s must be either COV or CORR"),
- "FACTOR");
- goto error;
- }
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
- vars_start = lex_ofs (lexer);
- if (lex_match (lexer, T_ASTERISK))
- {
- dict = dataset_dict (ds);
- matrix_reader = casereader_clone (dataset_source (ds));
- }
- else
- {
- struct file_handle *fh = fh_parse (lexer, FH_REF_FILE, NULL);
- if (fh == NULL)
- goto error;
-
- matrix_reader = any_reader_open_and_decode (fh, NULL, &dict, NULL);
-
- if (!(matrix_reader && dict))
- goto error;
- }
- vars_end = lex_ofs (lexer) - 1;
-
- if (!lex_force_match (lexer, T_RPAREN))
- {
- casereader_destroy (matrix_reader);
- goto error;
- }
-
- mr = matrix_reader_create (dict, matrix_reader);
- factor.vars = xmemdup (mr->cvars, mr->n_cvars * sizeof *mr->cvars);
- factor.n_vars = mr->n_cvars;
- }
- else
- goto error;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "ANALYSIS"))
- {
- struct const_var_set *vs;
- const struct variable **vars;
- size_t n_vars;
-
- lex_match (lexer, T_EQUALS);
-
- vars_start = lex_ofs (lexer);
- vs = const_var_set_create_from_array (factor.vars, factor.n_vars);
- vars_end = lex_ofs (lexer) - 1;
- bool ok = parse_const_var_set_vars (lexer, vs, &vars, &n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC);
- const_var_set_destroy (vs);
-
- if (!ok)
- goto error;
-
- free (factor.vars);
- factor.vars = vars;
- factor.n_vars = n_vars;
-
- if (mr)
- {
- free (mr->cvars);
- mr->cvars = xmemdup (vars, n_vars * sizeof *vars);
- mr->n_cvars = n_vars;
- }
- }
- else if (lex_match_id (lexer, "PLOT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "EIGEN"))
- {
- factor.plot |= PLOT_SCREE;
- }
-#if FACTOR_FULLY_IMPLEMENTED
- else if (lex_match_id (lexer, "ROTATION"))
- {
- }
-#endif
- else
- {
- lex_error_expecting (lexer, "EIGEN"
-#if FACTOR_FULLY_IMPLEMENTED
- , "ROTATION"
-#endif
- );
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "METHOD"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "COVARIANCE"))
- factor.method = METHOD_COV;
- else if (lex_match_id (lexer, "CORRELATION"))
- factor.method = METHOD_CORR;
- else
- {
- lex_error_expecting (lexer, "COVARIANCE", "CORRELATION");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "ROTATION"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- /* VARIMAX and DEFAULT are defaults */
- if (lex_match_id (lexer, "VARIMAX") || lex_match_id (lexer, "DEFAULT"))
- factor.rotation = ROT_VARIMAX;
- else if (lex_match_id (lexer, "EQUAMAX"))
- factor.rotation = ROT_EQUAMAX;
- else if (lex_match_id (lexer, "QUARTIMAX"))
- factor.rotation = ROT_QUARTIMAX;
- else if (lex_match_id (lexer, "PROMAX"))
- {
- factor.promax_power = 5;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_int (lexer))
- goto error;
- factor.promax_power = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- factor.rotation = ROT_PROMAX;
- }
- else if (lex_match_id (lexer, "NOROTATE"))
- factor.rotation = ROT_NONE;
- else
- {
- lex_error_expecting (lexer, "DEFAULT", "VARIMAX", "EQUAMAX",
- "QUARTIMAX", "PROMAX", "NOROTATE");
- goto error;
- }
- }
- factor.rotation_iterations = n_iterations;
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "FACTORS"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int (lexer))
- goto error;
- factor.n_factors = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "MINEIGEN"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- factor.min_eigen = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "ECONVERGE"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- factor.econverge = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "RCONVERGE"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- factor.rconverge = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "ITERATE"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "ITERATE", 0, INT_MAX))
- goto error;
- n_iterations = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "DEFAULT"))
- {
- factor.n_factors = 0;
- factor.min_eigen = 1;
- n_iterations = 25;
- }
- else
- {
- lex_error_expecting (lexer, "FACTORS", "MINEIGEN",
- "ECONVERGE", "RCONVERGE", "ITERATE",
- "DEFAULT");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "EXTRACTION"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "PAF"))
- factor.extraction = EXTRACTION_PAF;
- else if (lex_match_id (lexer, "PC"))
- factor.extraction = EXTRACTION_PC;
- else if (lex_match_id (lexer, "PA1"))
- factor.extraction = EXTRACTION_PC;
- else if (lex_match_id (lexer, "DEFAULT"))
- factor.extraction = EXTRACTION_PC;
- else
- {
- lex_error_expecting (lexer, "PAF", "PC", "PA1", "DEFAULT");
- goto error;
- }
- }
- factor.extraction_iterations = n_iterations;
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "SORT"))
- factor.sort = true;
- else if (lex_match_id (lexer, "BLANK"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- factor.blank = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "DEFAULT"))
- {
- factor.blank = 0;
- factor.sort = false;
- }
- else
- {
- lex_error_expecting (lexer, "SORT", "BLANK", "DEFAULT");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "PRINT"))
- {
- factor.print = 0;
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "UNIVARIATE"))
- factor.print |= PRINT_UNIVARIATE;
- else if (lex_match_id (lexer, "DET"))
- factor.print |= PRINT_DETERMINANT;
-#if FACTOR_FULLY_IMPLEMENTED
- else if (lex_match_id (lexer, "INV"))
- {
- }
-#endif
- else if (lex_match_id (lexer, "AIC"))
- factor.print |= PRINT_AIC;
- else if (lex_match_id (lexer, "SIG"))
- factor.print |= PRINT_SIG;
- else if (lex_match_id (lexer, "CORRELATION"))
- factor.print |= PRINT_CORRELATION;
- else if (lex_match_id (lexer, "COVARIANCE"))
- factor.print |= PRINT_COVARIANCE;
- else if (lex_match_id (lexer, "ROTATION"))
- factor.print |= PRINT_ROTATION;
- else if (lex_match_id (lexer, "EXTRACTION"))
- factor.print |= PRINT_EXTRACTION;
- else if (lex_match_id (lexer, "INITIAL"))
- factor.print |= PRINT_INITIAL;
- else if (lex_match_id (lexer, "KMO"))
- factor.print |= PRINT_KMO;
-#if FACTOR_FULLY_IMPLEMENTED
- else if (lex_match_id (lexer, "REPR"))
- {
- }
- else if (lex_match_id (lexer, "FSCORE"))
- {
- }
-#endif
- else if (lex_match (lexer, T_ALL))
- factor.print = -1;
- else if (lex_match_id (lexer, "DEFAULT"))
- {
- factor.print |= PRINT_INITIAL;
- factor.print |= PRINT_EXTRACTION;
- factor.print |= PRINT_ROTATION;
- }
- else
- {
- lex_error_expecting (lexer, "UNIVARIATE", "DET", "AIC", "SIG",
- "CORRELATION", "COVARIANCE", "ROTATION",
- "EXTRACTION", "INITIAL", "KMO", "ALL",
- "DEFAULT");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- factor.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- factor.exclude = MV_ANY;
- else if (lex_match_id (lexer, "LISTWISE"))
- factor.missing_type = MISS_LISTWISE;
- else if (lex_match_id (lexer, "PAIRWISE"))
- factor.missing_type = MISS_PAIRWISE;
- else if (lex_match_id (lexer, "MEANSUB"))
- factor.missing_type = MISS_MEANSUB;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE", "LISTWISE",
- "PAIRRWISE", "MEANSUB");
- goto error;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "ANALYSIS", "PLOT", "METHOD", "ROTATION",
- "CRITERIA", "EXTRACTION", "FORMAT", "PRINT",
- "MISSING");
- goto error;
- }
- }
-
- if (factor.rotation == ROT_NONE)
- factor.print &= ~PRINT_ROTATION;
-
- assert (factor.n_vars > 0);
- if (factor.n_vars < 2)
- lex_ofs_msg (lexer, SW, vars_start, vars_end,
- _("Factor analysis on a single variable is not useful."));
-
- if (matrix_reader)
- {
- struct idata *id = idata_alloc (factor.n_vars);
-
- while (matrix_reader_next (&id->mm, mr, NULL))
- {
- do_factor_by_matrix (&factor, id);
-
- gsl_matrix_free (id->ai_cov);
- id->ai_cov = NULL;
- gsl_matrix_free (id->ai_cor);
- id->ai_cor = NULL;
-
- matrix_material_uninit (&id->mm);
- }
-
- idata_free (id);
- }
- else
- if (!run_factor (ds, &factor))
- goto error;
-
- matrix_reader_destroy (mr);
- free (factor.vars);
- return CMD_SUCCESS;
-
-error:
- matrix_reader_destroy (mr);
- free (factor.vars);
- return CMD_FAILURE;
-}
-
-static void do_factor (const struct cmd_factor *factor, struct casereader *group);
-
-
-static bool
-run_factor (struct dataset *ds, const struct cmd_factor *factor)
-{
- struct dictionary *dict = dataset_dict (ds);
- bool ok;
- struct casereader *group;
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
-
- while (casegrouper_get_next_group (grouper, &group))
- {
- if (factor->missing_type == MISS_LISTWISE)
- group = casereader_create_filter_missing (group, factor->vars, factor->n_vars,
- factor->exclude,
- NULL, NULL);
- do_factor (factor, group);
- }
-
- ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- return ok;
-}
-
-
-/* Return the communality of variable N, calculated to N_FACTORS */
-static double
-the_communality (const gsl_matrix *evec, const gsl_vector *eval, int n, int n_factors)
-{
- assert (n >= 0);
- assert (n < eval->size);
- assert (n < evec->size1);
- assert (n_factors <= eval->size);
-
- double comm = 0;
- for (size_t i = 0; i < n_factors; ++i)
- {
- double evali = fabs (gsl_vector_get (eval, i));
-
- double eveci = gsl_matrix_get (evec, n, i);
-
- comm += pow2 (eveci) * evali;
- }
-
- return comm;
-}
-
-/* Return the communality of variable N, calculated to N_FACTORS */
-static double
-communality (const struct idata *idata, int n, int n_factors)
-{
- return the_communality (idata->evec, idata->eval, n, n_factors);
-}
-
-
-static void
-show_scree (const struct cmd_factor *f, const struct idata *idata)
-{
- struct scree *s;
- const char *label;
-
- if (!(f->plot & PLOT_SCREE))
- return;
-
-
- label = f->extraction == EXTRACTION_PC ? _("Component Number") : _("Factor Number");
-
- s = scree_create (idata->eval, label);
-
- scree_submit (s);
-}
-
-static void
-show_communalities (const struct cmd_factor * factor,
- const gsl_vector *initial, const gsl_vector *extracted)
-{
- if (!(factor->print & (PRINT_INITIAL | PRINT_EXTRACTION)))
- return;
-
- struct pivot_table *table = pivot_table_create (N_("Communalities"));
-
- struct pivot_dimension *communalities = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Communalities"));
- if (factor->print & PRINT_INITIAL)
- pivot_category_create_leaves (communalities->root, N_("Initial"));
- if (factor->print & PRINT_EXTRACTION)
- pivot_category_create_leaves (communalities->root, N_("Extraction"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0; i < factor->n_vars; ++i)
- {
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (factor->vars[i]));
-
- int col = 0;
- if (factor->print & PRINT_INITIAL)
- pivot_table_put2 (table, col++, row, pivot_value_new_number (
- gsl_vector_get (initial, i)));
- if (factor->print & PRINT_EXTRACTION)
- pivot_table_put2 (table, col++, row, pivot_value_new_number (
- gsl_vector_get (extracted, i)));
- }
-
- pivot_table_submit (table);
-}
-
-static struct pivot_dimension *
-create_numeric_dimension (struct pivot_table *table,
- enum pivot_axis_type axis_type, const char *name,
- size_t n, bool show_label)
-{
- struct pivot_dimension *d = pivot_dimension_create (table, axis_type, name);
- d->root->show_label = show_label;
- for (int i = 0; i < n; ++i)
- pivot_category_create_leaf (d->root, pivot_value_new_integer (i + 1));
- return d;
-}
-
-static void
-show_factor_matrix (const struct cmd_factor *factor, const struct idata *idata, const char *title, const gsl_matrix *fm)
-{
- struct pivot_table *table = pivot_table_create (title);
-
- const int n_factors = idata->n_extractions;
- create_numeric_dimension (
- table, PIVOT_AXIS_COLUMN,
- factor->extraction == EXTRACTION_PC ? N_("Component") : N_("Factor"),
- n_factors, true);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- /* Initialise to the identity permutation */
- gsl_permutation *perm = gsl_permutation_calloc (factor->n_vars);
-
- if (factor->sort)
- sort_matrix_indirect (fm, perm);
-
- for (size_t i = 0; i < factor->n_vars; ++i)
- {
- const int matrix_row = perm->data[i];
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (factor->vars[matrix_row]));
-
- for (size_t j = 0; j < n_factors; ++j)
- {
- double x = gsl_matrix_get (fm, matrix_row, j);
- if (fabs (x) < factor->blank)
- continue;
-
- pivot_table_put2 (table, j, var_idx, pivot_value_new_number (x));
- }
- }
-
- gsl_permutation_free (perm);
-
- pivot_table_submit (table);
-}
-
-static void
-put_variance (struct pivot_table *table, int row, int phase_idx,
- double lambda, double percent, double cum)
-{
- double entries[] = { lambda, percent, cum };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put3 (table, i, phase_idx, row,
- pivot_value_new_number (entries[i]));
-}
-
-static void
-show_explained_variance (const struct cmd_factor * factor,
- const struct idata *idata,
- const gsl_vector *initial_eigenvalues,
- const gsl_vector *extracted_eigenvalues,
- const gsl_vector *rotated_loadings)
-{
- if (!(factor->print & (PRINT_INITIAL | PRINT_EXTRACTION | PRINT_ROTATION)))
- return;
-
- struct pivot_table *table = pivot_table_create (
- N_("Total Variance Explained"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Total"), PIVOT_RC_OTHER,
- /* xgettext:no-c-format */
- N_("% of Variance"), PIVOT_RC_PERCENT,
- /* xgettext:no-c-format */
- N_("Cumulative %"), PIVOT_RC_PERCENT);
-
- struct pivot_dimension *phase = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Phase"));
- if (factor->print & PRINT_INITIAL)
- pivot_category_create_leaves (phase->root, N_("Initial Eigenvalues"));
-
- if (factor->print & PRINT_EXTRACTION)
- pivot_category_create_leaves (phase->root,
- N_("Extraction Sums of Squared Loadings"));
-
- if (factor->print & PRINT_ROTATION)
- pivot_category_create_leaves (phase->root,
- N_("Rotation Sums of Squared Loadings"));
-
- struct pivot_dimension *components = pivot_dimension_create (
- table, PIVOT_AXIS_ROW,
- factor->extraction == EXTRACTION_PC ? N_("Component") : N_("Factor"));
-
- double i_total = 0.0;
- for (size_t i = 0; i < initial_eigenvalues->size; ++i)
- i_total += gsl_vector_get (initial_eigenvalues, i);
-
- double e_total = (factor->extraction == EXTRACTION_PAF
- ? factor->n_vars
- : i_total);
-
- double i_cum = 0.0;
- double e_cum = 0.0;
- double r_cum = 0.0;
- for (size_t i = 0; i < factor->n_vars; ++i)
- {
- const double i_lambda = gsl_vector_get (initial_eigenvalues, i);
- double i_percent = 100.0 * i_lambda / i_total;
- i_cum += i_percent;
-
- const double e_lambda = gsl_vector_get (extracted_eigenvalues, i);
- double e_percent = 100.0 * e_lambda / e_total;
- e_cum += e_percent;
-
- int row = pivot_category_create_leaf (
- components->root, pivot_value_new_integer (i + 1));
-
- int phase_idx = 0;
-
- /* Initial Eigenvalues */
- if (factor->print & PRINT_INITIAL)
- put_variance (table, row, phase_idx++, i_lambda, i_percent, i_cum);
-
- if (i < idata->n_extractions)
- {
- if (factor->print & PRINT_EXTRACTION)
- put_variance (table, row, phase_idx++, e_lambda, e_percent, e_cum);
-
- if (rotated_loadings != NULL && factor->print & PRINT_ROTATION)
- {
- double r_lambda = gsl_vector_get (rotated_loadings, i);
- double r_percent = 100.0 * r_lambda / e_total;
- if (factor->rotation == ROT_PROMAX)
- r_lambda = r_percent = SYSMIS;
-
- r_cum += r_percent;
- put_variance (table, row, phase_idx++, r_lambda, r_percent,
- r_cum);
- }
- }
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_factor_correlation (const struct cmd_factor * factor, const gsl_matrix *fcm)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Factor Correlation Matrix"));
-
- create_numeric_dimension (
- table, PIVOT_AXIS_ROW,
- factor->extraction == EXTRACTION_PC ? N_("Component") : N_("Factor"),
- fcm->size2, true);
-
- create_numeric_dimension (table, PIVOT_AXIS_COLUMN, N_("Factor 2"),
- fcm->size1, false);
-
- for (size_t i = 0; i < fcm->size1; ++i)
- for (size_t j = 0; j < fcm->size2; ++j)
- pivot_table_put2 (table, j, i,
- pivot_value_new_number (gsl_matrix_get (fcm, i, j)));
-
- pivot_table_submit (table);
-}
-
-static void
-add_var_dims (struct pivot_table *table, const struct cmd_factor *factor)
-{
- for (int i = 0; i < 2; i++)
- {
- struct pivot_dimension *d = pivot_dimension_create (
- table, i ? PIVOT_AXIS_ROW : PIVOT_AXIS_COLUMN,
- N_("Variables"));
-
- for (size_t j = 0; j < factor->n_vars; j++)
- pivot_category_create_leaf (
- d->root, pivot_value_new_variable (factor->vars[j]));
- }
-}
-
-static void
-show_aic (const struct cmd_factor *factor, const struct idata *idata)
-{
- if ((factor->print & PRINT_AIC) == 0)
- return;
-
- struct pivot_table *table = pivot_table_create (N_("Anti-Image Matrices"));
-
- add_var_dims (table, factor);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Anti-image Covariance"),
- N_("Anti-image Correlation"));
-
- for (size_t i = 0; i < factor->n_vars; ++i)
- for (size_t j = 0; j < factor->n_vars; ++j)
- {
- double cov = gsl_matrix_get (idata->ai_cov, i, j);
- pivot_table_put3 (table, i, j, 0, pivot_value_new_number (cov));
-
- double corr = gsl_matrix_get (idata->ai_cor, i, j);
- pivot_table_put3 (table, i, j, 1, pivot_value_new_number (corr));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_correlation_matrix (const struct cmd_factor *factor, const struct idata *idata)
-{
- if (!(factor->print & (PRINT_CORRELATION | PRINT_SIG | PRINT_DETERMINANT)))
- return;
-
- struct pivot_table *table = pivot_table_create (N_("Correlation Matrix"));
-
- if (factor->print & (PRINT_CORRELATION | PRINT_SIG))
- {
- add_var_dims (table, factor);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"));
- if (factor->print & PRINT_CORRELATION)
- pivot_category_create_leaves (statistics->root, N_("Correlation"),
- PIVOT_RC_CORRELATION);
- if (factor->print & PRINT_SIG)
- pivot_category_create_leaves (statistics->root, N_("Sig. (1-tailed)"),
- PIVOT_RC_SIGNIFICANCE);
-
- int stat_idx = 0;
- if (factor->print & PRINT_CORRELATION)
- {
- for (int i = 0; i < factor->n_vars; ++i)
- for (int j = 0; j < factor->n_vars; ++j)
- {
- double corr = gsl_matrix_get (idata->mm.corr, i, j);
- pivot_table_put3 (table, j, i, stat_idx,
- pivot_value_new_number (corr));
- }
- stat_idx++;
- }
-
- if (factor->print & PRINT_SIG)
- {
- for (int i = 0; i < factor->n_vars; ++i)
- for (int j = 0; j < factor->n_vars; ++j)
- if (i != j)
- {
- double rho = gsl_matrix_get (idata->mm.corr, i, j);
- double w = gsl_matrix_get (idata->mm.n, i, j);
- double sig = significance_of_correlation (rho, w);
- pivot_table_put3 (table, j, i, stat_idx,
- pivot_value_new_number (sig));
- }
- stat_idx++;
- }
- }
-
- if (factor->print & PRINT_DETERMINANT)
- {
- struct pivot_value *caption = pivot_value_new_user_text_nocopy (
- xasprintf ("%s: %.2f", _("Determinant"), idata->detR));
- pivot_table_set_caption (table, caption);
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_covariance_matrix (const struct cmd_factor *factor, const struct idata *idata)
-{
- if (!(factor->print & PRINT_COVARIANCE))
- return;
-
- struct pivot_table *table = pivot_table_create (N_("Covariance Matrix"));
- add_var_dims (table, factor);
-
- for (int i = 0; i < factor->n_vars; ++i)
- for (int j = 0; j < factor->n_vars; ++j)
- {
- double cov = gsl_matrix_get (idata->mm.cov, i, j);
- pivot_table_put2 (table, j, i, pivot_value_new_number (cov));
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-do_factor (const struct cmd_factor *factor, struct casereader *r)
-{
- struct ccase *c;
- struct idata *idata = idata_alloc (factor->n_vars);
-
- idata->cvm = covariance_1pass_create (factor->n_vars, factor->vars,
- factor->wv, factor->exclude, true);
-
- for (; (c = casereader_read (r)); case_unref (c))
- {
- covariance_accumulate (idata->cvm, c);
- }
-
- idata->mm.cov = covariance_calculate (idata->cvm);
-
- if (idata->mm.cov == NULL)
- {
- msg (MW, _("The dataset contains no complete observations. No analysis will be performed."));
- covariance_destroy (idata->cvm);
- goto finish;
- }
-
- idata->mm.var_matrix = covariance_moments (idata->cvm, MOMENT_VARIANCE);
- idata->mm.mean_matrix = covariance_moments (idata->cvm, MOMENT_MEAN);
- idata->mm.n = covariance_moments (idata->cvm, MOMENT_NONE);
-
- do_factor_by_matrix (factor, idata);
-
- finish:
- gsl_matrix_free (idata->mm.corr);
- gsl_matrix_free (idata->mm.cov);
-
- idata_free (idata);
- casereader_destroy (r);
-}
-
-static void
-do_factor_by_matrix (const struct cmd_factor *factor, struct idata *idata)
-{
- if (!idata->mm.cov && !(idata->mm.corr && idata->mm.var_matrix))
- {
- msg (ME, _("The dataset has no covariance matrix or a "
- "correlation matrix along with standard deviations."));
- return;
- }
-
- if (idata->mm.cov && !idata->mm.corr)
- idata->mm.corr = correlation_from_covariance (idata->mm.cov, idata->mm.var_matrix);
- if (idata->mm.corr && !idata->mm.cov)
- idata->mm.cov = covariance_from_correlation (idata->mm.corr, idata->mm.var_matrix);
- if (factor->method == METHOD_CORR)
- idata->analysis_matrix = idata->mm.corr;
- else
- idata->analysis_matrix = idata->mm.cov;
-
- gsl_matrix *r_inv;
- r_inv = clone_matrix (idata->mm.corr);
- gsl_linalg_cholesky_decomp (r_inv);
- gsl_linalg_cholesky_invert (r_inv);
-
- idata->ai_cov = anti_image_cov (r_inv);
- idata->ai_cor = anti_image_corr (r_inv, idata);
-
- double sum_ssq_r = 0;
- double sum_ssq_a = 0;
- for (int i = 0; i < r_inv->size1; ++i)
- {
- sum_ssq_r += ssq_od_n (idata->mm.corr, i);
- sum_ssq_a += ssq_od_n (idata->ai_cor, i);
- }
-
- gsl_matrix_free (r_inv);
-
- if (factor->print & PRINT_DETERMINANT
- || factor->print & PRINT_KMO)
- {
- int sign = 0;
-
- const int size = idata->mm.corr->size1;
- gsl_permutation *p = gsl_permutation_calloc (size);
- gsl_matrix *tmp = gsl_matrix_calloc (size, size);
- gsl_matrix_memcpy (tmp, idata->mm.corr);
-
- gsl_linalg_LU_decomp (tmp, p, &sign);
- idata->detR = gsl_linalg_LU_det (tmp, sign);
- gsl_permutation_free (p);
- gsl_matrix_free (tmp);
- }
-
- if (factor->print & PRINT_UNIVARIATE
- && idata->mm.n && idata->mm.mean_matrix && idata->mm.var_matrix)
- {
- struct pivot_table *table = pivot_table_create (
- N_("Descriptive Statistics"));
- pivot_table_set_weight_var (table, factor->wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("Analysis N"), PIVOT_RC_COUNT);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0; i < factor->n_vars; ++i)
- {
- const struct variable *v = factor->vars[i];
-
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (v));
-
- double entries[] = {
- gsl_matrix_get (idata->mm.mean_matrix, i, i),
- sqrt (gsl_matrix_get (idata->mm.var_matrix, i, i)),
- gsl_matrix_get (idata->mm.n, i, i),
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, row,
- pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
- }
-
- if (factor->print & PRINT_KMO && idata->mm.n)
- {
- struct pivot_table *table = pivot_table_create (
- N_("KMO and Bartlett's Test"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Kaiser-Meyer-Olkin Measure of Sampling Adequacy"), PIVOT_RC_OTHER);
- pivot_category_create_group (
- statistics->root, N_("Bartlett's Test of Sphericity"),
- N_("Approx. Chi-Square"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
-
- /* The literature doesn't say what to do for the value of W when
- missing values are involved. The best thing I can think of
- is to take the mean average. */
- double w = 0;
- for (int i = 0; i < idata->mm.n->size1; ++i)
- w += gsl_matrix_get (idata->mm.n, i, i);
- w /= idata->mm.n->size1;
-
- double xsq = ((w - 1 - (2 * factor->n_vars + 5) / 6.0)
- * -log (idata->detR));
- double df = factor->n_vars * (factor->n_vars - 1) / 2;
- double entries[] = {
- sum_ssq_r / (sum_ssq_r + sum_ssq_a),
- xsq,
- df,
- gsl_cdf_chisq_Q (xsq, df)
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
-
- pivot_table_submit (table);
- }
-
- show_correlation_matrix (factor, idata);
- show_covariance_matrix (factor, idata);
- if (idata->cvm)
- covariance_destroy (idata->cvm);
-
- {
- gsl_matrix *am = matrix_dup (idata->analysis_matrix);
- gsl_eigen_symmv_workspace *workspace = gsl_eigen_symmv_alloc (factor->n_vars);
-
- gsl_eigen_symmv (am, idata->eval, idata->evec, workspace);
-
- gsl_eigen_symmv_free (workspace);
- gsl_matrix_free (am);
- }
-
- gsl_eigen_symmv_sort (idata->eval, idata->evec, GSL_EIGEN_SORT_ABS_DESC);
-
- idata->n_extractions = n_extracted_factors (factor, idata);
-
- if (idata->n_extractions == 0)
- {
- msg (MW, _("The %s criteria result in zero factors extracted. Therefore no analysis will be performed."), "FACTOR");
- return;
- }
-
- if (idata->n_extractions > factor->n_vars)
- {
- msg (MW,
- _("The %s criteria result in more factors than variables, which is not meaningful. No analysis will be performed."),
- "FACTOR");
- return;
- }
-
- {
- gsl_matrix *rotated_factors = NULL;
- gsl_matrix *pattern_matrix = NULL;
- gsl_matrix *fcm = NULL;
- gsl_vector *rotated_loadings = NULL;
-
- const gsl_vector *extracted_eigenvalues = NULL;
- gsl_vector *initial_communalities = gsl_vector_alloc (factor->n_vars);
- gsl_vector *extracted_communalities = gsl_vector_alloc (factor->n_vars);
- struct factor_matrix_workspace *fmw = factor_matrix_workspace_alloc (idata->msr->size, idata->n_extractions);
- gsl_matrix *factor_matrix = gsl_matrix_calloc (factor->n_vars, fmw->n_factors);
-
- if (factor->extraction == EXTRACTION_PAF)
- {
- gsl_vector *diff = gsl_vector_alloc (idata->msr->size);
- struct smr_workspace *ws = ws_create (idata->analysis_matrix);
-
- for (size_t i = 0; i < factor->n_vars; ++i)
- {
- double r2 = squared_multiple_correlation (idata->analysis_matrix, i, ws);
-
- gsl_vector_set (idata->msr, i, r2);
- }
- ws_destroy (ws);
-
- gsl_vector_memcpy (initial_communalities, idata->msr);
-
- for (size_t i = 0; i < factor->extraction_iterations; ++i)
- {
- double min, max;
- gsl_vector_memcpy (diff, idata->msr);
-
- iterate_factor_matrix (idata->analysis_matrix, idata->msr, factor_matrix, fmw);
-
- gsl_vector_sub (diff, idata->msr);
-
- gsl_vector_minmax (diff, &min, &max);
-
- if (fabs (min) < factor->econverge && fabs (max) < factor->econverge)
- break;
- }
- gsl_vector_free (diff);
-
-
-
- gsl_vector_memcpy (extracted_communalities, idata->msr);
- extracted_eigenvalues = fmw->eval;
- }
- else if (factor->extraction == EXTRACTION_PC)
- {
- for (size_t i = 0; i < factor->n_vars; ++i)
- gsl_vector_set (initial_communalities, i, communality (idata, i, factor->n_vars));
-
- gsl_vector_memcpy (extracted_communalities, initial_communalities);
-
- iterate_factor_matrix (idata->analysis_matrix, extracted_communalities, factor_matrix, fmw);
-
-
- extracted_eigenvalues = idata->eval;
- }
-
-
- show_aic (factor, idata);
- show_communalities (factor, initial_communalities, extracted_communalities);
-
- if (factor->rotation != ROT_NONE)
- {
- rotated_factors = gsl_matrix_calloc (factor_matrix->size1, factor_matrix->size2);
- rotated_loadings = gsl_vector_calloc (factor_matrix->size2);
- if (factor->rotation == ROT_PROMAX)
- {
- pattern_matrix = gsl_matrix_calloc (factor_matrix->size1, factor_matrix->size2);
- fcm = gsl_matrix_calloc (factor_matrix->size2, factor_matrix->size2);
- }
-
-
- rotate (factor, factor_matrix, extracted_communalities, rotated_factors, rotated_loadings, pattern_matrix, fcm);
- }
-
- show_explained_variance (factor, idata, idata->eval, extracted_eigenvalues, rotated_loadings);
-
- factor_matrix_workspace_free (fmw);
-
- show_scree (factor, idata);
-
- show_factor_matrix (factor, idata,
- (factor->extraction == EXTRACTION_PC
- ? N_("Component Matrix") : N_("Factor Matrix")),
- factor_matrix);
-
- if (factor->rotation == ROT_PROMAX)
- {
- show_factor_matrix (factor, idata, N_("Pattern Matrix"),
- pattern_matrix);
- gsl_matrix_free (pattern_matrix);
- }
-
- if (factor->rotation != ROT_NONE)
- {
- show_factor_matrix (factor, idata,
- (factor->rotation == ROT_PROMAX
- ? N_("Structure Matrix")
- : factor->extraction == EXTRACTION_PC
- ? N_("Rotated Component Matrix")
- : N_("Rotated Factor Matrix")),
- rotated_factors);
-
- gsl_matrix_free (rotated_factors);
- }
-
- if (factor->rotation == ROT_PROMAX)
- {
- show_factor_correlation (factor, fcm);
- gsl_matrix_free (fcm);
- }
-
- gsl_matrix_free (factor_matrix);
- gsl_vector_free (rotated_loadings);
- gsl_vector_free (initial_communalities);
- gsl_vector_free (extracted_communalities);
- }
-}
-
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2013 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <errno.h>
-#include <float.h>
-#include <limits.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/casereader-provider.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/settings.h"
-#include "data/short-names.h"
-#include "data/value.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-#include "data/data-in.h"
-#include "data/data-out.h"
-
-#include "gl/intprops.h"
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* List of variable names. */
-struct var_names
- {
- const char **names;
- size_t n_names, allocated_names;
- };
-
-static void var_names_init (struct var_names *);
-static void var_names_add (struct pool *, struct var_names *, const char *);
-
-/* Represents a FLIP input program. */
-struct flip_pgm
- {
- struct pool *pool; /* Pool containing FLIP data. */
- size_t n_vars; /* Pre-flip number of variables. */
- int n_cases; /* Pre-flip number of cases. */
-
- struct variable *new_names_var; /* Variable with new variable names. */
- const char *encoding; /* Variable names' encoding. */
- struct var_names old_names; /* Variable names before FLIP. */
- struct var_names new_names; /* Variable names after FLIP. */
-
- FILE *file; /* Temporary file containing data. */
- size_t cases_read; /* Number of cases already read. */
- bool error; /* Error reading temporary file? */
- };
-
-static const struct casereader_class flip_casereader_class;
-
-static void destroy_flip_pgm (struct flip_pgm *);
-static bool flip_file (struct flip_pgm *);
-static void make_new_var (struct dictionary *, const char *name);
-
-/* Parses and executes FLIP. */
-int
-cmd_flip (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *old_dict = dataset_dict (ds);
- struct dictionary *new_dict = NULL;
- const struct variable **vars;
- struct flip_pgm *flip;
- struct casereader *input, *reader;
- struct ccase *c;
- size_t i;
- bool ok;
-
- if (proc_make_temporary_transformations_permanent (ds))
- lex_ofs_msg (lexer, SW, 0, lex_ofs (lexer) - 1,
- _("%s ignores %s. "
- "Temporary transformations will be made permanent."),
- "FLIP", "TEMPORARY");
-
- flip = pool_create_container (struct flip_pgm, pool);
- flip->n_vars = 0;
- flip->n_cases = 0;
- flip->new_names_var = NULL;
- var_names_init (&flip->old_names);
- var_names_init (&flip->new_names);
- flip->file = NULL;
- flip->cases_read = 0;
- flip->error = false;
-
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "VARIABLES"))
- {
- lex_match (lexer, T_EQUALS);
- if (!parse_variables_const (lexer, old_dict, &vars, &flip->n_vars,
- PV_NO_DUPLICATE))
- goto error;
- lex_match (lexer, T_SLASH);
- }
- else
- dict_get_vars (old_dict, &vars, &flip->n_vars, DC_SYSTEM);
- pool_register (flip->pool, free, vars);
-
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "NEWNAMES"))
- {
- lex_match (lexer, T_EQUALS);
- flip->new_names_var = parse_variable (lexer, old_dict);
- if (!flip->new_names_var)
- goto error;
- }
- else
- flip->new_names_var = dict_lookup_var (old_dict, "CASE_LBL");
-
- if (flip->new_names_var)
- {
- for (i = 0; i < flip->n_vars; i++)
- if (vars[i] == flip->new_names_var)
- {
- remove_element (vars, flip->n_vars, sizeof *vars, i);
- flip->n_vars--;
- break;
- }
- }
- if (flip->n_vars <= 0)
- goto error;
-
- flip->file = pool_create_temp_file (flip->pool);
- if (flip->file == NULL)
- {
- msg (SE, _("Could not create temporary file for %s."), "FLIP");
- goto error;
- }
-
- /* Save old variable names for use as values of CASE_LBL
- variable in flipped file. */
- for (i = 0; i < flip->n_vars; i++)
- var_names_add (flip->pool, &flip->old_names,
- pool_strdup (flip->pool, var_get_name (vars[i])));
-
- /* Read the active dataset into a flip_sink. */
- proc_discard_output (ds);
-
- /* Save old dictionary. */
- new_dict = dict_clone (old_dict);
- flip->encoding = dict_get_encoding (new_dict);
- dict_clear (new_dict);
-
- input = proc_open_filtering (ds, false);
- while ((c = casereader_read (input)) != NULL)
- {
- flip->n_cases++;
- for (i = 0; i < flip->n_vars; i++)
- {
- const struct variable *v = vars[i];
- double out = var_is_numeric (v) ? case_num (c, v) : SYSMIS;
- fwrite (&out, sizeof out, 1, flip->file);
- }
- if (flip->new_names_var != NULL)
- {
- const union value *value = case_data (c, flip->new_names_var);
- const char *name;
- if (var_is_numeric (flip->new_names_var))
- {
- double f = value->f;
- name = (f == SYSMIS ? "VSYSMIS"
- : f < INT_MIN ? "VNEGINF"
- : f > INT_MAX ? "VPOSINF"
- : pool_asprintf (flip->pool, "V%d", (int) f));
- }
- else
- {
- name = data_out_pool (value, dict_get_encoding (old_dict),
- var_get_write_format (flip->new_names_var),
- settings_get_fmt_settings (), flip->pool);
- }
- var_names_add (flip->pool, &flip->new_names, name);
- }
- case_unref (c);
- }
- ok = casereader_destroy (input);
- ok = proc_commit (ds) && ok;
-
- /* Flip the data we read. */
- if (!ok || !flip_file (flip))
- {
- dataset_clear (ds);
- goto error;
- }
-
- /* Flip the dictionary. */
- dict_create_var_assert (new_dict, "CASE_LBL", 8);
- for (i = 0; i < flip->n_cases; i++)
- if (flip->new_names.n_names)
- make_new_var (new_dict, flip->new_names.names[i]);
- else
- {
- char s[3 + INT_STRLEN_BOUND (i) + 1];
- sprintf (s, "VAR%03zu", i);
- dict_create_var_assert (new_dict, s, 0);
- }
-
- /* Set up flipped data for reading. */
- reader = casereader_create_sequential (NULL, dict_get_proto (new_dict),
- flip->n_vars,
- &flip_casereader_class, flip);
- dataset_set_dict (ds, new_dict);
- dataset_set_source (ds, reader);
- return CMD_SUCCESS;
-
- error:
- dict_unref (new_dict);
- destroy_flip_pgm (flip);
- return CMD_CASCADING_FAILURE;
-}
-
-/* Destroys FLIP. */
-static void
-destroy_flip_pgm (struct flip_pgm *flip)
-{
- if (flip != NULL)
- pool_destroy (flip->pool);
-}
-
-/* Make a new variable with base name NAME, which is bowdlerized and
- mangled until acceptable. */
-static void
-make_new_var (struct dictionary *dict, const char *name_)
-{
- char *name = xstrdup (name_);
- char *cp;
-
- /* Trim trailing spaces. */
- cp = strchr (name, '\0');
- while (cp > name && isspace ((unsigned char) cp[-1]))
- *--cp = '\0';
-
- /* Fix invalid characters. */
- for (cp = name; *cp && cp < name + ID_MAX_LEN; cp++)
- if (cp == name)
- {
- if (!lex_is_id1 (*cp) || *cp == '$')
- *cp = 'V';
- }
- else
- {
- if (!lex_is_idn (*cp))
- *cp = '_';
- }
- *cp = '\0';
-
- if (strlen (name) == 0)
- {
- free (name);
- name = xstrdup ("v");
- }
-
- /* Use the mangled name, if it is available, or add numeric
- extensions until we find one that is. */
- if (!id_is_plausible (name) || !dict_create_var (dict, name, 0))
- {
- int len = strlen (name);
- int i;
- for (i = 1; ; i++)
- {
- char n[ID_MAX_LEN + 1];
- int ofs = MIN (ID_MAX_LEN - 1 - intlog10 (i), len);
- strncpy (n, name, ofs);
- sprintf (&n[ofs], "%d", i);
-
- if (id_is_plausible (n) && dict_create_var (dict, n, 0))
- break;
- }
- }
- free (name);
-}
-
-/* Transposes the external file into a new file. */
-static bool
-flip_file (struct flip_pgm *flip)
-{
- size_t case_bytes;
- size_t case_capacity;
- size_t case_idx;
- double *input_buf, *output_buf;
- FILE *input_file, *output_file;
-
- /* Allocate memory for many cases. */
- case_bytes = flip->n_vars * sizeof *input_buf;
- case_capacity = settings_get_workspace () / case_bytes;
- if (case_capacity > flip->n_cases * 2)
- case_capacity = flip->n_cases * 2;
- if (case_capacity < 2)
- case_capacity = 2;
- for (;;)
- {
- size_t bytes = case_bytes * case_capacity;
- if (case_capacity > 2)
- input_buf = malloc (bytes);
- else
- input_buf = xmalloc (bytes);
- if (input_buf != NULL)
- break;
-
- case_capacity /= 2;
- if (case_capacity < 2)
- case_capacity = 2;
- }
- pool_register (flip->pool, free, input_buf);
-
- /* Use half the allocated memory for input_buf, half for
- output_buf. */
- case_capacity /= 2;
- output_buf = input_buf + flip->n_vars * case_capacity;
-
- input_file = flip->file;
- if (fseeko (input_file, 0, SEEK_SET) != 0)
- {
- msg (SE, _("Error rewinding %s file: %s."), "FLIP", strerror (errno));
- return false;
- }
-
- output_file = pool_create_temp_file (flip->pool);
- if (output_file == NULL)
- {
- msg (SE, _("Error creating %s source file."), "FLIP");
- return false;
- }
-
- for (case_idx = 0; case_idx < flip->n_cases;)
- {
- unsigned long read_cases = MIN (flip->n_cases - case_idx,
- case_capacity);
- size_t i;
-
- if (read_cases != fread (input_buf, case_bytes, read_cases, input_file))
- {
- if (ferror (input_file))
- msg (SE, _("Error reading %s file: %s."), "FLIP", strerror (errno));
- else
- msg (SE, _("Unexpected end of file reading %s file."), "FLIP");
- return false;
- }
-
- for (i = 0; i < flip->n_vars; i++)
- {
- unsigned long j;
-
- for (j = 0; j < read_cases; j++)
- output_buf[j] = input_buf[i + j * flip->n_vars];
-
- if (fseeko (output_file,
- sizeof *input_buf * (case_idx
- + (off_t) i * flip->n_cases),
- SEEK_SET) != 0)
- {
- msg (SE, _("Error seeking %s source file: %s."), "FLIP",
- strerror (errno));
- return false;
- }
-
- if (fwrite (output_buf, sizeof *output_buf, read_cases, output_file)
- != read_cases)
- {
- msg (SE, _("Error writing %s source file: %s."), "FLIP",
- strerror (errno));
- return false;
- }
- }
-
- case_idx += read_cases;
- }
-
- pool_fclose_temp_file (flip->pool, input_file);
- pool_unregister (flip->pool, input_buf);
- free (input_buf);
-
- if (fseeko (output_file, 0, SEEK_SET) != 0)
- {
- msg (SE, _("Error rewinding %s source file: %s."), "FLIP", strerror (errno));
- return false;
- }
- flip->file = output_file;
-
- return true;
-}
-
-/* Reads and returns one case.
- Returns a null pointer at end of file or if an I/O error occurred. */
-static struct ccase *
-flip_casereader_read (struct casereader *reader, void *flip_)
-{
- struct flip_pgm *flip = flip_;
- struct ccase *c;
- size_t i;
-
- if (flip->error || flip->cases_read >= flip->n_vars)
- return false;
-
- c = case_create (casereader_get_proto (reader));
- data_in (ss_cstr (flip->old_names.names[flip->cases_read]), flip->encoding,
- FMT_A, settings_get_fmt_settings (), case_data_rw_idx (c, 0),
- 8, flip->encoding);
-
- for (i = 0; i < flip->n_cases; i++)
- {
- double in;
- if (fread (&in, sizeof in, 1, flip->file) != 1)
- {
- case_unref (c);
- if (ferror (flip->file))
- msg (SE, _("Error reading %s temporary file: %s."), "FLIP",
- strerror (errno));
- else if (feof (flip->file))
- msg (SE, _("Unexpected end of file reading %s temporary file."), "FLIP");
- else
- NOT_REACHED ();
- flip->error = true;
- return NULL;
- }
- *case_num_rw_idx (c, i + 1) = in;
- }
-
- flip->cases_read++;
-
- return c;
-}
-
-/* Destroys the source.
- Returns true if successful read, false if an I/O occurred
- during destruction or previously. */
-static void
-flip_casereader_destroy (struct casereader *reader, void *flip_)
-{
- struct flip_pgm *flip = flip_;
- if (flip->error)
- casereader_force_error (reader);
- destroy_flip_pgm (flip);
-}
-
-static const struct casereader_class flip_casereader_class =
- {
- flip_casereader_read,
- flip_casereader_destroy,
- NULL,
- NULL,
- };
-\f
-static void
-var_names_init (struct var_names *vn)
-{
- vn->names = NULL;
- vn->n_names = 0;
- vn->allocated_names = 0;
-}
-
-static void
-var_names_add (struct pool *pool, struct var_names *vn, const char *name)
-{
- if (vn->n_names >= vn->allocated_names)
- vn->names = pool_2nrealloc (pool, vn->names, &vn->allocated_names,
- sizeof *vn->names);
- vn->names[vn->n_names++] = name;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2009, 2010 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/freq.h"
-
-#include <stdlib.h>
-
-#include "data/variable.h"
-#include "data/value.h"
-#include "libpspp/array.h"
-#include "libpspp/compiler.h"
-
-struct freq *
-freq_clone (const struct freq *in, int values, int *widths)
-{
- int i;
- struct freq *f = xmalloc (sizeof (struct freq) +
- (sizeof (union value) * (values - 1)));
-
- f->node = in->node;
- f->count = in->count;
- for (i = 0; i < values; ++i)
- {
- value_init (&f->values[i], widths[i]);
- value_copy (&f->values[i], &in->values[i], widths[i]);
- }
-
- return f;
-}
-
-void
-freq_destroy (struct freq *f, int values, int *widths)
-{
- int i;
- for (i = 0; i < values; ++i)
- {
- value_destroy (&f->values[i], widths[i]);
- }
-
- free (f);
-}
-
-
-
-void
-freq_hmap_destroy (struct hmap *hmap, int width)
-{
- struct freq *f, *next;
-
- HMAP_FOR_EACH_SAFE (f, next, struct freq, node, hmap)
- {
- value_destroy (&f->values[0], width);
- hmap_delete (hmap, &f->node);
- free (f);
- }
- hmap_destroy (hmap);
-}
-
-struct freq *
-freq_hmap_search (struct hmap *hmap,
- const union value *value, int width, size_t hash)
-{
- struct freq *f;
-
- HMAP_FOR_EACH_WITH_HASH (f, struct freq, node, hash, hmap)
- if (value_equal (value, &f->values[0], width))
- return f;
-
- return NULL;
-}
-
-struct freq *
-freq_hmap_insert (struct hmap *hmap,
- const union value *value, int width, size_t hash)
-{
- struct freq *f = xmalloc (sizeof *f);
- value_clone (&f->values[0], value, width);
- f->count = 0;
- hmap_insert (hmap, &f->node, hash);
- return f;
-}
-
-int
-compare_freq_ptr_3way (const void *a_, const void *b_, const void *width_)
-{
- const struct freq *const *ap = a_;
- const struct freq *const *bp = b_;
- const int *widthp = width_;
-
- return value_compare_3way (&(*ap)->values[0], &(*bp)->values[0], *widthp);
-}
-
-struct freq **
-freq_hmap_sort (struct hmap *hmap, int width)
-{
- size_t n_entries = hmap_count (hmap);
- struct freq **entries;
- struct freq *f;
- size_t i;
-
- entries = xnmalloc (n_entries, sizeof *entries);
- i = 0;
- HMAP_FOR_EACH (f, struct freq, node, hmap)
- entries[i++] = f;
- assert (i == n_entries);
-
- sort (entries, n_entries, sizeof *entries, compare_freq_ptr_3way, &width);
-
- return entries;
-}
-
-struct freq *
-freq_hmap_extract (struct hmap *hmap)
-{
- struct freq *freqs, *f;
- size_t n_freqs;
- size_t i;
-
- n_freqs = hmap_count (hmap);
- freqs = xnmalloc (n_freqs, sizeof *freqs);
- i = 0;
- HMAP_FOR_EACH (f, struct freq, node, hmap)
- freqs[i++] = *f;
- assert (i == n_freqs);
-
- return freqs;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2010, 2015 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef LANGUAGE_STATS_FREQ_H
-#define LANGUAGE_STATS_FREQ_H 1
-
-#include "data/value.h"
-#include "libpspp/hmap.h"
-
-/* Frequency table entry. */
-struct freq
- {
- struct hmap_node node; /* Element in hash table. */
- double count; /* The number of occurrences of the value. */
- union value values[1]; /* The value. */
- };
-
-
-struct freq *freq_clone (const struct freq *, int values, int *widths);
-void freq_destroy (struct freq *f, int values, int *widths);
-
-
-static inline size_t
-table_entry_size (size_t n_values)
-{
- return (offsetof (struct freq, values)
- + n_values * sizeof (union value));
-}
-
-
-int compare_freq_ptr_3way (const void *a_, const void *b_, const void *width_);
-
-
-
-void freq_hmap_destroy (struct hmap *, int width);
-
-struct freq *freq_hmap_search (struct hmap *, const union value *, int width,
- size_t hash);
-struct freq *freq_hmap_insert (struct hmap *, const union value *, int width,
- size_t hash);
-
-struct freq **freq_hmap_sort (struct hmap *, int width);
-struct freq *freq_hmap_extract (struct hmap *);
-
-
-
-
-#endif /* language/stats/freq.h */
+++ /dev/null
-/*
- PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2007, 2009, 2010, 2011, 2014, 2015 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include <config.h>
-#include <stdlib.h>
-#include <gsl/gsl_histogram.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "data/settings.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-
-#include "language/dictionary/split-file.h"
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "language/stats/freq.h"
-
-#include "libpspp/array.h"
-#include "libpspp/bit-vector.h"
-#include "libpspp/compiler.h"
-#include "libpspp/hmap.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-
-#include "math/histogram.h"
-#include "math/moments.h"
-#include "math/chart-geometry.h"
-
-
-#include "output/charts/barchart.h"
-#include "output/charts/piechart.h"
-#include "output/charts/plot-hist.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-/* Percentiles to calculate. */
-
-struct percentile
- {
- double p; /* The percentile to calculate, between 0 and 1. */
- bool show; /* True to show this percentile in the statistics box. */
- };
-
-static int
-percentile_compare_3way (const void *a_, const void *b_)
-{
- const struct percentile *a = a_;
- const struct percentile *b = b_;
-
- return a->p < b->p ? -1 : a->p > b->p;
-}
-
-enum
- {
- FRQ_FREQ,
- FRQ_PERCENT
- };
-
-enum sortprops
- {
- FRQ_AFREQ,
- FRQ_DFREQ,
- FRQ_AVALUE,
- FRQ_DVALUE
- };
-
-#define STATISTICS \
- S(FRQ_ST_MEAN, "MEAN", N_("Mean")) \
- S(FRQ_ST_SEMEAN, "SEMEAN", N_("S.E. Mean")) \
- S(FRQ_ST_MEDIAN, "MEDIAN", N_("Median")) \
- S(FRQ_ST_MODE, "MODE", N_("Mode")) \
- S(FRQ_ST_STDDEV, "STDDEV", N_("Std Dev")) \
- S(FRQ_ST_VARIANCE, "VARIANCE", N_("Variance")) \
- S(FRQ_ST_KURTOSIS, "KURTOSIS", N_("Kurtosis")) \
- S(FRQ_ST_SEKURTOSIS, "SEKURTOSIS",N_("S.E. Kurt")) \
- S(FRQ_ST_SKEWNESS, "SKEWNESS", N_("Skewness")) \
- S(FRQ_ST_SESKEWNESS, "SESKEWNESS",N_("S.E. Skew")) \
- S(FRQ_ST_RANGE, "RANGE", N_("Range")) \
- S(FRQ_ST_MINIMUM, "MINIMUM", N_("Minimum")) \
- S(FRQ_ST_MAXIMUM, "MAXIMUM", N_("Maximum")) \
- S(FRQ_ST_SUM, "SUM", N_("Sum"))
-
-enum frq_statistic
- {
-#define S(ENUM, KEYWORD, NAME) ENUM,
-STATISTICS
-#undef S
- };
-
-enum {
-#define S(ENUM, KEYWORD, NAME) +1
- FRQ_ST_count = STATISTICS,
-#undef S
-};
-
-static const char *st_keywords[FRQ_ST_count] = {
-#define S(ENUM, KEYWORD, NAME) KEYWORD,
- STATISTICS
-#undef S
-};
-
-static const char *st_names[FRQ_ST_count] = {
-#define S(ENUM, KEYWORD, NAME) NAME,
- STATISTICS
-#undef S
-};
-
-struct freq_tab
- {
- struct hmap data; /* Hash table for accumulating counts. */
- struct freq *valid; /* Valid freqs. */
- size_t n_valid; /* Number of total freqs. */
- const struct dictionary *dict; /* Source of entries in the table. */
-
- struct freq *missing; /* Missing freqs. */
- size_t n_missing; /* Number of missing freqs. */
-
- /* Statistics. */
- double total_cases; /* Sum of weights of all cases. */
- double valid_cases; /* Sum of weights of valid cases. */
- };
-
-struct frq_chart
- {
- double x_min; /* X axis minimum value. */
- double x_max; /* X axis maximum value. */
- int y_scale; /* Y axis scale: FRQ_FREQ or FRQ_PERCENT. */
-
- /* Histograms only. */
- double y_max; /* Y axis maximum value. */
- bool draw_normal; /* Whether to draw normal curve. */
-
- /* Pie charts only. */
- bool include_missing; /* Whether to include missing values. */
- };
-
-/* Per-variable frequency data. */
-struct var_freqs
- {
- const struct variable *var;
-
- /* Freqency table. */
- struct freq_tab tab; /* Frequencies table to use. */
-
- /* Statistics. */
- double stat[FRQ_ST_count];
- double *percentiles;
-
- /* Variable attributes. */
- int width;
- };
-
-struct frq_proc
- {
- struct var_freqs *vars;
- size_t n_vars;
-
- /* Percentiles to calculate and possibly display. */
- struct percentile *percentiles;
- size_t median_idx;
- size_t n_percentiles;
-
- /* Frequency table display. */
- long int max_categories; /* Maximum categories to show. */
- int sort; /* FRQ_AVALUE or FRQ_DVALUE
- or FRQ_AFREQ or FRQ_DFREQ. */
-
- /* Statistics. */
- unsigned long stats;
-
- /* Histogram and pie chart settings. */
- struct frq_chart *hist, *pie, *bar;
-
- bool warn;
- };
-
-
-struct freq_compare_aux
- {
- bool by_freq;
- bool ascending_freq;
-
- int width;
- bool ascending_value;
- };
-
-static void calc_stats (const struct frq_proc *,
- const struct var_freqs *, double d[FRQ_ST_count]);
-
-static void do_piechart(const struct frq_chart *pie,
- const struct variable *var,
- const struct freq_tab *frq_tab);
-
-static void do_barchart(const struct frq_chart *bar,
- const struct variable **var,
- const struct freq_tab *frq_tab);
-
-static struct frq_stats_table *frq_stats_table_submit (
- struct frq_stats_table *, const struct frq_proc *,
- const struct dictionary *, const struct variable *wv,
- const struct ccase *example);
-static void frq_stats_table_destroy (struct frq_stats_table *);
-
-static int
-compare_freq (const void *a_, const void *b_, const void *aux_)
-{
- const struct freq_compare_aux *aux = aux_;
- const struct freq *a = a_;
- const struct freq *b = b_;
-
- if (aux->by_freq && a->count != b->count)
- {
- int cmp = a->count > b->count ? 1 : -1;
- return aux->ascending_freq ? cmp : -cmp;
- }
- else
- {
- int cmp = value_compare_3way (a->values, b->values, aux->width);
- return aux->ascending_value ? cmp : -cmp;
- }
-}
-
-/* Create a gsl_histogram from a freq_tab */
-static struct histogram *freq_tab_to_hist (const struct frq_proc *,
- const struct var_freqs *);
-
-static void
-put_freq_row (struct pivot_table *table, int var_idx,
- double frequency, double percent,
- double valid_percent, double cum_percent)
-{
- double entries[] = { frequency, percent, valid_percent, cum_percent };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- if (entries[i] != SYSMIS)
- pivot_table_put2 (table, i, var_idx,
- pivot_value_new_number (entries[i]));
-}
-
-/* Displays a full frequency table for variable V. */
-static void
-dump_freq_table (const struct var_freqs *vf, const struct variable *wv)
-{
- const struct freq_tab *ft = &vf->tab;
-
- struct pivot_table *table = pivot_table_create__ (pivot_value_new_variable (
- vf->var), "Frequencies");
- pivot_table_set_weight_var (table, wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Frequency"), PIVOT_RC_COUNT,
- N_("Percent"), PIVOT_RC_PERCENT,
- N_("Valid Percent"), PIVOT_RC_PERCENT,
- N_("Cumulative Percent"), PIVOT_RC_PERCENT);
-
- struct pivot_dimension *variable = pivot_dimension_create__ (
- table, PIVOT_AXIS_ROW, pivot_value_new_variable (vf->var));
-
- double cum_freq = 0.0;
- double cum_percent = 0.0;
- struct pivot_category *valid = NULL;
- for (const struct freq *f = ft->valid; f < ft->missing; f++)
- {
- cum_freq += f->count;
- double valid_percent = f->count / ft->valid_cases * 100.0;
- cum_percent += valid_percent;
-
- if (!valid)
- valid = pivot_category_create_group (variable->root, N_("Valid"));
- int var_idx = pivot_category_create_leaf (
- valid, pivot_value_new_var_value (vf->var, &f->values[0]));
- put_freq_row (table, var_idx, f->count,
- f->count / ft->total_cases * 100.0,
- valid_percent, cum_percent);
- }
-
- struct pivot_category *missing = NULL;
- size_t n_categories = ft->n_valid + ft->n_missing;
- for (const struct freq *f = ft->missing; f < &ft->valid[n_categories]; f++)
- {
- cum_freq += f->count;
-
- if (!missing)
- missing = pivot_category_create_group (variable->root, N_("Missing"));
- int var_idx = pivot_category_create_leaf (
- missing, pivot_value_new_var_value (vf->var, &f->values[0]));
- put_freq_row (table, var_idx, f->count,
- f->count / ft->total_cases * 100.0, SYSMIS, SYSMIS);
- }
-
- int var_idx = pivot_category_create_leaf (
- variable->root, pivot_value_new_text (N_("Total")));
- put_freq_row (table, var_idx, cum_freq, cum_percent, SYSMIS, SYSMIS);
-
- pivot_table_submit (table);
-}
-\f
-/* Statistical display. */
-
-static double
-calc_percentile (double p, double valid_cases, double x1, double x2)
-{
- double s, dummy;
-
- s = (settings_get_algorithm () != COMPATIBLE
- ? modf ((valid_cases - 1) * p, &dummy)
- : modf ((valid_cases + 1) * p - 1, &dummy));
-
- return x1 + (x2 - x1) * s;
-}
-
-/* Calculates all of the percentiles for VF within FRQ. */
-static void
-calc_percentiles (const struct frq_proc *frq, struct var_freqs *vf)
-{
- if (!frq->n_percentiles)
- return;
-
- if (!vf->percentiles)
- vf->percentiles = xnmalloc (frq->n_percentiles, sizeof *vf->percentiles);
-
- const struct freq_tab *ft = &vf->tab;
- const double W = ft->valid_cases;
- size_t idx = 0;
-
- double rank = 0;
- for (const struct freq *f = ft->valid; f < ft->missing; f++)
- {
- rank += f->count;
- for (; idx < frq->n_percentiles; idx++)
- {
- struct percentile *pc = &frq->percentiles[idx];
- double tp;
-
- tp = (settings_get_algorithm () == ENHANCED
- ? (W - 1) * pc->p
- : (W + 1) * pc->p - 1);
-
- if (rank <= tp)
- break;
-
- if (tp + 1 < rank || f + 1 >= ft->missing)
- vf->percentiles[idx] = f->values[0].f;
- else
- vf->percentiles[idx] = calc_percentile (pc->p, W, f->values[0].f,
- f[1].values[0].f);
- }
- }
- for (; idx < frq->n_percentiles; idx++)
- vf->percentiles[idx] = (ft->n_valid > 0
- ? ft->valid[ft->n_valid - 1].values[0].f
- : SYSMIS);
-}
-
-/* Returns true iff the value in struct freq F is non-missing
- for variable V. */
-static bool
-not_missing (const void *f_, const void *v_)
-{
- const struct freq *f = f_;
- const struct variable *v = v_;
-
- return !var_is_value_missing (v, f->values);
-}
-
-/* Summarizes the frequency table data for variable V. */
-static void
-postprocess_freq_tab (const struct frq_proc *frq, struct var_freqs *vf)
-{
- struct freq_tab *ft = &vf->tab;
-
- /* Extract data from hash table. */
- size_t count = hmap_count (&ft->data);
- struct freq *freqs = freq_hmap_extract (&ft->data);
-
- /* Put data into ft. */
- ft->valid = freqs;
- ft->n_valid = partition (freqs, count, sizeof *freqs, not_missing, vf->var);
- ft->missing = freqs + ft->n_valid;
- ft->n_missing = count - ft->n_valid;
-
- /* Sort data. */
- struct freq_compare_aux aux = {
- .by_freq = frq->sort == FRQ_AFREQ || frq->sort == FRQ_DFREQ,
- .ascending_freq = frq->sort != FRQ_DFREQ,
- .width = vf->width,
- .ascending_value = frq->sort != FRQ_DVALUE,
- };
- sort (ft->valid, ft->n_valid, sizeof *ft->valid, compare_freq, &aux);
- sort (ft->missing, ft->n_missing, sizeof *ft->missing, compare_freq, &aux);
-
- /* Summary statistics. */
- ft->valid_cases = 0.0;
- for (size_t i = 0; i < ft->n_valid; ++i)
- ft->valid_cases += ft->valid[i].count;
-
- ft->total_cases = ft->valid_cases;
- for (size_t i = 0; i < ft->n_missing; ++i)
- ft->total_cases += ft->missing[i].count;
-}
-
-/* Add data from case C to the frequency table. */
-static void
-calc (struct frq_proc *frq, const struct ccase *c, const struct dataset *ds)
-{
- double weight = dict_get_case_weight (dataset_dict (ds), c, &frq->warn);
- for (size_t i = 0; i < frq->n_vars; i++)
- {
- struct var_freqs *vf = &frq->vars[i];
- const union value *value = case_data (c, vf->var);
- size_t hash = value_hash (value, vf->width, 0);
- struct freq *f;
-
- f = freq_hmap_search (&vf->tab.data, value, vf->width, hash);
- if (f == NULL)
- f = freq_hmap_insert (&vf->tab.data, value, vf->width, hash);
-
- f->count += weight;
- }
-}
-
-static void
-output_splits_once (bool *need_splits, const struct dataset *ds,
- const struct ccase *c)
-{
- if (*need_splits)
- {
- output_split_file_values (ds, c);
- *need_splits = false;
- }
-}
-
-/* Finishes up with the variables after frequencies have been
- calculated. Displays statistics, percentiles, ... */
-static struct frq_stats_table *
-postcalc (struct frq_proc *frq, const struct dataset *ds,
- struct ccase *example, struct frq_stats_table *fst)
-{
- const struct dictionary *dict = dataset_dict (ds);
- const struct variable *wv = dict_get_weight (dict);
-
- for (size_t i = 0; i < frq->n_vars; i++)
- {
- struct var_freqs *vf = &frq->vars[i];
- postprocess_freq_tab (frq, vf);
- calc_percentiles (frq, vf);
- }
-
- enum split_type st = dict_get_split_type (dict);
- bool need_splits = true;
- if (frq->stats)
- {
- if (st != SPLIT_LAYERED)
- output_splits_once (&need_splits, ds, example);
- fst = frq_stats_table_submit (fst, frq, dict, wv, example);
- }
-
- for (size_t i = 0; i < frq->n_vars; i++)
- {
- struct var_freqs *vf = &frq->vars[i];
-
- /* Frequencies tables. */
- if (vf->tab.n_valid + vf->tab.n_missing <= frq->max_categories)
- {
- output_splits_once (&need_splits, ds, example);
- dump_freq_table (vf, wv);
- }
-
- if (frq->hist && var_is_numeric (vf->var) && vf->tab.n_valid > 0)
- {
- double d[FRQ_ST_count];
- struct histogram *histogram;
-
- calc_stats (frq, vf, d);
-
- histogram = freq_tab_to_hist (frq, vf);
-
- if (histogram)
- {
- output_splits_once (&need_splits, ds, example);
- chart_submit (histogram_chart_create (
- histogram->gsl_hist, var_to_string(vf->var),
- vf->tab.valid_cases,
- d[FRQ_ST_MEAN],
- d[FRQ_ST_STDDEV],
- frq->hist->draw_normal));
-
- statistic_destroy (&histogram->parent);
- }
- }
-
- if (frq->pie)
- {
- output_splits_once (&need_splits, ds, example);
- do_piechart(frq->pie, vf->var, &vf->tab);
- }
-
- if (frq->bar)
- {
- output_splits_once (&need_splits, ds, example);
- do_barchart(frq->bar, &vf->var, &vf->tab);
- }
-
- free (vf->tab.valid);
- freq_hmap_destroy (&vf->tab.data, vf->width);
- }
-
- return fst;
-}
-
-static void
-frq_run (struct frq_proc *frq, struct dataset *ds)
-{
- struct frq_stats_table *fst = NULL;
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds),
- dataset_dict (ds));
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- for (size_t i = 0; i < frq->n_vars; i++)
- hmap_init (&frq->vars[i].tab.data);
-
- struct ccase *example = casereader_peek (group, 0);
-
- struct ccase *c;
- for (; (c = casereader_read (group)) != NULL; case_unref (c))
- calc (frq, c, ds);
- fst = postcalc (frq, ds, example, fst);
- casereader_destroy (group);
-
- case_unref (example);
- }
- frq_stats_table_destroy (fst);
- casegrouper_destroy (grouper);
- proc_commit (ds);
-}
-
-static void
-add_percentile (struct frq_proc *frq, double p, bool show,
- size_t *allocated_percentiles)
-{
- if (frq->n_percentiles >= *allocated_percentiles)
- frq->percentiles = x2nrealloc (frq->percentiles, allocated_percentiles,
- sizeof *frq->percentiles);
- frq->percentiles[frq->n_percentiles++] = (struct percentile) {
- .p = p,
- .show = show,
- };
-}
-
-int
-cmd_frequencies (struct lexer *lexer, struct dataset *ds)
-{
- bool ok = false;
- const struct variable **vars = NULL;
-
- size_t allocated_percentiles = 0;
-
- const unsigned long DEFAULT_STATS = (BIT_INDEX (FRQ_ST_MEAN)
- | BIT_INDEX (FRQ_ST_STDDEV)
- | BIT_INDEX (FRQ_ST_MINIMUM)
- | BIT_INDEX (FRQ_ST_MAXIMUM));
- struct frq_proc frq = {
- .sort = FRQ_AVALUE,
- .stats = DEFAULT_STATS,
- .max_categories = LONG_MAX,
- .median_idx = SIZE_MAX,
- .warn = true,
- };
-
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "VARIABLES") && !lex_force_match (lexer, T_EQUALS))
- goto done;
-
- if (!parse_variables_const (lexer, dataset_dict (ds),
- &vars, &frq.n_vars, PV_NO_DUPLICATE))
- goto done;
-
- frq.vars = xcalloc (frq.n_vars, sizeof *frq.vars);
- for (size_t i = 0; i < frq.n_vars; ++i)
- {
- frq.vars[i].var = vars[i];
- frq.vars[i].width = var_get_width (vars[i]);
- }
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
- frq.stats = 0;
-
- int ofs = lex_ofs (lexer);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- for (int s = 0; s < FRQ_ST_count; s++)
- if (lex_match_id (lexer, st_keywords[s]))
- {
- frq.stats |= 1 << s;
- goto next;
- }
-
- if (lex_match_id (lexer, "DEFAULT"))
- frq.stats = DEFAULT_STATS;
- else if (lex_match (lexer, T_ALL))
- frq.stats = (1 << FRQ_ST_count) - 1;
- else if (lex_match_id (lexer, "NONE"))
- frq.stats = 0;
- else
- {
-#define S(ENUM, KEYWORD, NAME) KEYWORD,
- lex_error_expecting (lexer,
- STATISTICS
- "DEFAULT", "ALL", "NONE");
-#undef S
- goto done;
- }
-
- next:;
- }
-
- if (lex_ofs (lexer) == ofs)
- frq.stats = DEFAULT_STATS;
- }
- else if (lex_match_id (lexer, "PERCENTILES"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (!lex_force_num_range_closed (lexer, "PERCENTILES", 0, 100))
- goto done;
- add_percentile (&frq, lex_number (lexer) / 100.0, true,
- &allocated_percentiles);
- lex_get (lexer);
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "TABLE"))
- {
- }
- else if (lex_match_id (lexer, "NOTABLE"))
- frq.max_categories = 0;
- else if (lex_match_id (lexer, "LIMIT"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "LIMIT", 0, INT_MAX))
- goto done;
-
- frq.max_categories = lex_integer (lexer);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else if (lex_match_id (lexer, "AVALUE"))
- frq.sort = FRQ_AVALUE;
- else if (lex_match_id (lexer, "DVALUE"))
- frq.sort = FRQ_DVALUE;
- else if (lex_match_id (lexer, "AFREQ"))
- frq.sort = FRQ_AFREQ;
- else if (lex_match_id (lexer, "DFREQ"))
- frq.sort = FRQ_DFREQ;
- else
- {
- lex_error_expecting (lexer, "TABLE", "NOTABLE",
- "LIMIT", "AVALUE", "DVALUE",
- "AFREQ", "DFREQ");
- goto done;
- }
- }
- }
- else if (lex_match_id (lexer, "NTILES"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_int_range (lexer, "NTILES", 0, INT_MAX))
- goto done;
-
- int n = lex_integer (lexer);
- lex_get (lexer);
- for (int i = 0; i < n + 1; ++i)
- add_percentile (&frq, i / (double) n, true, &allocated_percentiles);
- }
- else if (lex_match_id (lexer, "ALGORITHM"))
- {
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "COMPATIBLE"))
- settings_set_cmd_algorithm (COMPATIBLE);
- else if (lex_match_id (lexer, "ENHANCED"))
- settings_set_cmd_algorithm (ENHANCED);
- else
- {
- lex_error_expecting (lexer, "COMPATIBLE", "ENHANCED");
- goto done;
- }
- }
- else if (lex_match_id (lexer, "HISTOGRAM"))
- {
- double hi_min = -DBL_MAX;
- double hi_max = DBL_MAX;
- int hi_scale = FRQ_FREQ;
- int hi_freq = INT_MIN;
- int hi_pcnt = INT_MIN;
- bool hi_draw_normal = false;
-
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "NORMAL"))
- hi_draw_normal = true;
- else if (lex_match_id (lexer, "NONORMAL"))
- hi_draw_normal = false;
- else if (lex_match_id (lexer, "FREQ"))
- {
- hi_scale = FRQ_FREQ;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_int_range (lexer, "FREQ", 1, INT_MAX))
- goto done;
- hi_freq = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- }
- else if (lex_match_id (lexer, "PERCENT"))
- {
- hi_scale = FRQ_PERCENT;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_int_range (lexer, "PERCENT", 1, INT_MAX))
- goto done;
- hi_pcnt = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- }
- else if (lex_match_id (lexer, "MINIMUM"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "MINIMUM",
- -DBL_MAX, hi_max))
- goto done;
- hi_min = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else if (lex_match_id (lexer, "MAXIMUM"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "MAXIMUM",
- hi_min, DBL_MAX))
- goto done;
- hi_max = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else
- {
- lex_error_expecting (lexer, "NORMAL", "NONORMAL",
- "FREQ", "PERCENT", "MINIMUM", "MAXIMUM");
- goto done;
- }
- }
-
- free (frq.hist);
- frq.hist = xmalloc (sizeof *frq.hist);
- *frq.hist = (struct frq_chart) {
- .x_min = hi_min,
- .x_max = hi_max,
- .y_scale = hi_scale,
- .y_max = hi_scale == FRQ_FREQ ? hi_freq : hi_pcnt,
- .draw_normal = hi_draw_normal,
- .include_missing = false,
- };
-
- add_percentile (&frq, .25, false, &allocated_percentiles);
- add_percentile (&frq, .75, false, &allocated_percentiles);
- }
- else if (lex_match_id (lexer, "PIECHART"))
- {
- double pie_min = -DBL_MAX;
- double pie_max = DBL_MAX;
- bool pie_missing = true;
-
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "MINIMUM"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "MINIMUM",
- -DBL_MAX, pie_max))
- goto done;
- pie_min = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else if (lex_match_id (lexer, "MAXIMUM"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "MAXIMUM",
- pie_min, DBL_MAX))
- goto done;
- pie_max = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else if (lex_match_id (lexer, "MISSING"))
- pie_missing = true;
- else if (lex_match_id (lexer, "NOMISSING"))
- pie_missing = false;
- else
- {
- lex_error_expecting (lexer, "MINIMUM", "MAXIMUM",
- "MISSING", "NOMISSING");
- goto done;
- }
- }
-
- free (frq.pie);
- frq.pie = xmalloc (sizeof *frq.pie);
- *frq.pie = (struct frq_chart) {
- .x_min = pie_min,
- .x_max = pie_max,
- .include_missing = pie_missing,
- };
- }
- else if (lex_match_id (lexer, "BARCHART"))
- {
- double bar_min = -DBL_MAX;
- double bar_max = DBL_MAX;
- bool bar_freq = true;
-
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "MINIMUM"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "MINIMUM",
- -DBL_MAX, bar_max))
- goto done;
- bar_min = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else if (lex_match_id (lexer, "MAXIMUM"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "MAXIMUM",
- bar_min, DBL_MAX))
- goto done;
- bar_max = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- else if (lex_match_id (lexer, "FREQ"))
- {
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num_range_open (lexer, "FREQ", 0, DBL_MAX))
- goto done;
- /* XXX TODO */
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- bar_freq = true;
- }
- else if (lex_match_id (lexer, "PERCENT"))
- {
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num_range_open (lexer, "PERCENT",
- 0, DBL_MAX))
- goto done;
- /* XXX TODO */
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto done;
- }
- bar_freq = false;
- }
- else
- {
- lex_error_expecting (lexer, "MINIMUM", "MAXIMUM",
- "FREQ", "PERCENT");
- goto done;
- }
- }
-
- free (frq.bar);
- frq.bar = xmalloc (sizeof *frq.bar);
- *frq.bar = (struct frq_chart) {
- .x_min = bar_min,
- .x_max = bar_max,
- .include_missing = false,
- .y_scale = bar_freq ? FRQ_FREQ : FRQ_PERCENT,
- };
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "EXCLUDE"))
- {
- /* XXX TODO */
- }
- else if (lex_match_id (lexer, "INCLUDE"))
- {
- /* XXX TODO */
- }
- else
- {
- lex_error_expecting (lexer, "EXCLUDE", "INCLUDE");
- goto done;
- }
- }
- }
- else if (lex_match_id (lexer, "ORDER"))
- {
- lex_match (lexer, T_EQUALS);
- /* XXX TODO */
- if (!lex_match_id (lexer, "ANALYSIS")
- && !lex_match_id (lexer, "VARIABLE"))
- {
- lex_error_expecting (lexer, "ANALYSIS", "VARIABLE");
- goto done;
- }
- }
- else
- {
- lex_error_expecting (lexer, "STATISTICS", "PERCENTILES", "FORMAT",
- "NTILES", "ALGORITHM", "HISTOGRAM", "PIECHART",
- "BARCHART", "MISSING", "ORDER");
- goto done;
- }
- }
-
- if (frq.stats & BIT_INDEX (FRQ_ST_MEDIAN))
- add_percentile (&frq, .5, false, &allocated_percentiles);
-
- if (frq.n_percentiles > 0)
- {
- qsort (frq.percentiles, frq.n_percentiles, sizeof *frq.percentiles,
- percentile_compare_3way);
-
- /* Combine equal percentiles. */
- size_t o = 1;
- for (int i = 1; i < frq.n_percentiles; ++i)
- {
- struct percentile *prev = &frq.percentiles[o - 1];
- struct percentile *this = &frq.percentiles[i];
- if (this->p != prev->p)
- frq.percentiles[o++] = *this;
- else if (this->show)
- prev->show = true;
- }
- frq.n_percentiles = o;
-
- for (size_t i = 0; i < frq.n_percentiles; i++)
- if (frq.percentiles[i].p == 0.5)
- {
- frq.median_idx = i;
- break;
- }
- }
-
- frq_run (&frq, ds);
- ok = true;
-
-done:
- free (vars);
- for (size_t i = 0; i < frq.n_vars; i++)
- free (frq.vars[i].percentiles);
- free (frq.vars);
- free (frq.bar);
- free (frq.pie);
- free (frq.hist);
- free (frq.percentiles);
-
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-}
-
-static double
-calculate_iqr (const struct frq_proc *frq, const struct var_freqs *vf)
-{
- double q1 = SYSMIS;
- double q3 = SYSMIS;
-
- /* This cannot work unless the 25th and 75th percentile are calculated */
- assert (frq->n_percentiles >= 2);
- for (int i = 0; i < frq->n_percentiles; i++)
- {
- struct percentile *pc = &frq->percentiles[i];
-
- if (fabs (0.25 - pc->p) < DBL_EPSILON)
- q1 = vf->percentiles[i];
- else if (fabs (0.75 - pc->p) < DBL_EPSILON)
- q3 = vf->percentiles[i];
- }
-
- return q1 == SYSMIS || q3 == SYSMIS ? SYSMIS : q3 - q1;
-}
-
-static bool
-chart_includes_value (const struct frq_chart *chart,
- const struct variable *var,
- const union value *value)
-{
- if (!chart->include_missing && var_is_value_missing (var, value))
- return false;
-
- if (var_is_numeric (var)
- && ((chart->x_min != SYSMIS && value->f < chart->x_min)
- || (chart->x_max != SYSMIS && value->f > chart->x_max)))
- return false;
-
- return true;
-}
-
-/* Create a gsl_histogram from a freq_tab */
-static struct histogram *
-freq_tab_to_hist (const struct frq_proc *frq, const struct var_freqs *vf)
-{
- /* Find out the extremes of the x value, within the range to be included in
- the histogram, and sum the total frequency of those values. */
- double x_min = DBL_MAX;
- double x_max = -DBL_MAX;
- double valid_freq = 0;
- for (size_t i = 0; i < vf->tab.n_valid; i++)
- {
- const struct freq *f = &vf->tab.valid[i];
- if (chart_includes_value (frq->hist, vf->var, f->values))
- {
- x_min = MIN (x_min, f->values[0].f);
- x_max = MAX (x_max, f->values[0].f);
- valid_freq += f->count;
- }
- }
-
- if (valid_freq <= 0)
- return NULL;
-
- double iqr = calculate_iqr (frq, vf);
-
- double bin_width =
- (iqr > 0
- ? 2 * iqr / pow (valid_freq, 1.0 / 3.0) /* Freedman-Diaconis. */
- : (x_max - x_min) / (1 + log2 (valid_freq))); /* Sturges */
-
- struct histogram *histogram = histogram_create (bin_width, x_min, x_max);
- if (histogram == NULL)
- return NULL;
-
- for (size_t i = 0; i < vf->tab.n_valid; i++)
- {
- const struct freq *f = &vf->tab.valid[i];
- if (chart_includes_value (frq->hist, vf->var, f->values))
- histogram_add (histogram, f->values[0].f, f->count);
- }
-
- return histogram;
-}
-
-
-/* Allocate an array of struct freqs and fill them from the data in FRQ_TAB,
- according to the parameters of CATCHART
- N_SLICES will contain the number of slices allocated.
- The caller is responsible for freeing slices
-*/
-static struct freq *
-pick_cat_counts (const struct frq_chart *catchart,
- const struct freq_tab *frq_tab,
- int *n_slicesp)
-{
- int n_slices = 0;
- struct freq *slices = xnmalloc (frq_tab->n_valid + frq_tab->n_missing, sizeof *slices);
-
- for (size_t i = 0; i < frq_tab->n_valid; i++)
- {
- struct freq *f = &frq_tab->valid[i];
- if (f->count >= catchart->x_min && f->count <= catchart->x_max)
- slices[n_slices++] = *f;
- }
-
-
- if (catchart->include_missing)
- {
- for (size_t i = 0; i < frq_tab->n_missing; i++)
- {
- const struct freq *f = &frq_tab->missing[i];
- slices[n_slices].count += f->count;
-
- if (i == 0)
- slices[n_slices].values[0] = f->values[0];
- }
-
- if (frq_tab->n_missing > 0)
- n_slices++;
- }
-
- *n_slicesp = n_slices;
- return slices;
-}
-
-
-/* Allocate an array of struct freqs and fill them from the data in FRQ_TAB,
- according to the parameters of CATCHART
- N_SLICES will contain the number of slices allocated.
- The caller is responsible for freeing slices
-*/
-static struct freq **
-pick_cat_counts_ptr (const struct frq_chart *catchart,
- const struct freq_tab *frq_tab,
- int *n_slicesp)
-{
- int n_slices = 0;
- struct freq **slices = xnmalloc (frq_tab->n_valid + frq_tab->n_missing, sizeof *slices);
-
- for (size_t i = 0; i < frq_tab->n_valid; i++)
- {
- struct freq *f = &frq_tab->valid[i];
- if (f->count >= catchart->x_min && f->count <= catchart->x_max)
- slices[n_slices++] = f;
- }
-
- if (catchart->include_missing)
- for (size_t i = 0; i < frq_tab->n_missing; i++)
- {
- const struct freq *f = &frq_tab->missing[i];
- if (i == 0)
- {
- slices[n_slices] = xmalloc (sizeof *slices[n_slices]);
- slices[n_slices]->values[0] = f->values[0];
- }
-
- slices[n_slices]->count += f->count;
- }
-
- *n_slicesp = n_slices;
- return slices;
-}
-
-static void
-do_piechart (const struct frq_chart *pie, const struct variable *var,
- const struct freq_tab *frq_tab)
-{
- int n_slices;
- struct freq *slices = pick_cat_counts (pie, frq_tab, &n_slices);
-
- if (n_slices < 2)
- msg (SW, _("Omitting pie chart for %s, which has only %d unique values."),
- var_get_name (var), n_slices);
- else if (n_slices > 50)
- msg (SW, _("Omitting pie chart for %s, which has over 50 unique values."),
- var_get_name (var));
- else
- chart_submit (piechart_create (var, slices, n_slices));
-
- free (slices);
-}
-
-static void
-do_barchart (const struct frq_chart *bar, const struct variable **var,
- const struct freq_tab *frq_tab)
-{
- int n_slices;
- struct freq **slices = pick_cat_counts_ptr (bar, frq_tab, &n_slices);
-
- if (n_slices < 1)
- msg (SW, _("Omitting bar chart, which has no values."));
- else
- chart_submit (barchart_create (
- var, 1,
- bar->y_scale == FRQ_FREQ ? _("Count") : _("Percent"),
- bar->y_scale == FRQ_PERCENT,
- slices, n_slices));
- free (slices);
-}
-
-/* Calculates all the pertinent statistics for VF, putting them in array
- D[]. */
-static void
-calc_stats (const struct frq_proc *frq, const struct var_freqs *vf,
- double d[FRQ_ST_count])
-{
- const struct freq_tab *ft = &vf->tab;
-
- /* Calculate the mode. If there is more than one mode, we take the
- smallest. */
- int most_often = -1;
- double X_mode = SYSMIS;
- for (const struct freq *f = ft->valid; f < ft->missing; f++)
- if (most_often < f->count)
- {
- most_often = f->count;
- X_mode = f->values[0].f;
- }
-
- /* Calculate moments. */
- struct moments *m = moments_create (MOMENT_KURTOSIS);
- for (const struct freq *f = ft->valid; f < ft->missing; f++)
- moments_pass_one (m, f->values[0].f, f->count);
- for (const struct freq *f = ft->valid; f < ft->missing; f++)
- moments_pass_two (m, f->values[0].f, f->count);
- moments_calculate (m, NULL, &d[FRQ_ST_MEAN], &d[FRQ_ST_VARIANCE],
- &d[FRQ_ST_SKEWNESS], &d[FRQ_ST_KURTOSIS]);
- moments_destroy (m);
-
- /* Formulae below are taken from _SPSS Statistical Algorithms_. */
- double W = ft->valid_cases;
- if (ft->n_valid > 0)
- {
- d[FRQ_ST_MINIMUM] = ft->valid[0].values[0].f;
- d[FRQ_ST_MAXIMUM] = ft->valid[ft->n_valid - 1].values[0].f;
- d[FRQ_ST_RANGE] = d[FRQ_ST_MAXIMUM] - d[FRQ_ST_MINIMUM];
- }
- else
- {
- d[FRQ_ST_MINIMUM] = SYSMIS;
- d[FRQ_ST_MAXIMUM] = SYSMIS;
- d[FRQ_ST_RANGE] = SYSMIS;
- }
- d[FRQ_ST_MODE] = X_mode;
- d[FRQ_ST_SUM] = d[FRQ_ST_MEAN] * W;
- d[FRQ_ST_STDDEV] = sqrt (d[FRQ_ST_VARIANCE]);
- d[FRQ_ST_SEMEAN] = d[FRQ_ST_STDDEV] / sqrt (W);
- d[FRQ_ST_SESKEWNESS] = calc_seskew (W);
- d[FRQ_ST_SEKURTOSIS] = calc_sekurt (W);
- d[FRQ_ST_MEDIAN] = (frq->median_idx != SIZE_MAX
- ? vf->percentiles[frq->median_idx]
- : SYSMIS);
-}
-
-static bool
-all_string_variables (const struct frq_proc *frq)
-{
- for (size_t i = 0; i < frq->n_vars; i++)
- if (var_is_numeric (frq->vars[i].var))
- return false;
-
- return true;
-}
-\f
-struct frq_stats_table
- {
- struct pivot_table *table;
- struct pivot_splits *splits;
- };
-
-/* Displays a table of all the statistics requested. */
-static struct frq_stats_table *
-frq_stats_table_create (const struct frq_proc *frq,
- const struct dictionary *dict,
- const struct variable *wv)
-{
- if (all_string_variables (frq))
- return NULL;
-
- struct pivot_table *table = pivot_table_create (N_("Statistics"));
- pivot_table_set_weight_var (table, wv);
-
- struct pivot_dimension *variables
- = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Variables"));
- for (size_t i = 0; i < frq->n_vars; i++)
- if (!var_is_alpha (frq->vars[i].var))
- pivot_category_create_leaf (variables->root,
- pivot_value_new_variable (frq->vars[i].var));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"));
- struct pivot_category *n = pivot_category_create_group (
- statistics->root, N_("N"));
- pivot_category_create_leaves (n,
- N_("Valid"), PIVOT_RC_COUNT,
- N_("Missing"), PIVOT_RC_COUNT);
- for (int i = 0; i < FRQ_ST_count; i++)
- if (frq->stats & BIT_INDEX (i))
- pivot_category_create_leaf (statistics->root,
- pivot_value_new_text (st_names[i]));
- struct pivot_category *percentiles = NULL;
- for (size_t i = 0; i < frq->n_percentiles; i++)
- {
- const struct percentile *pc = &frq->percentiles[i];
-
- if (!pc->show)
- continue;
-
- if (!percentiles)
- percentiles = pivot_category_create_group (
- statistics->root, N_("Percentiles"));
- pivot_category_create_leaf (percentiles, pivot_value_new_integer (
- pc->p * 100.0));
- }
-
- struct pivot_splits *splits = pivot_splits_create (table, PIVOT_AXIS_COLUMN,
- dict);
-
- struct frq_stats_table *fst = xmalloc (sizeof *fst);
- *fst = (struct frq_stats_table) { .table = table, .splits = splits };
- return fst;
-}
-
-static struct frq_stats_table *
-frq_stats_table_submit (struct frq_stats_table *fst,
- const struct frq_proc *frq,
- const struct dictionary *dict,
- const struct variable *wv,
- const struct ccase *example)
-{
- if (!fst)
- {
- fst = frq_stats_table_create (frq, dict, wv);
- if (!fst)
- return NULL;
- }
- pivot_splits_new_split (fst->splits, example);
-
- int var_idx = 0;
- for (size_t i = 0; i < frq->n_vars; i++)
- {
- struct var_freqs *vf = &frq->vars[i];
- if (var_is_alpha (vf->var))
- continue;
-
- const struct freq_tab *ft = &vf->tab;
-
- int row = 0;
- pivot_splits_put2 (fst->splits, fst->table, var_idx, row++,
- pivot_value_new_number (ft->valid_cases));
- pivot_splits_put2 (fst->splits, fst->table, var_idx, row++,
- pivot_value_new_number (
- ft->total_cases - ft->valid_cases));
-
- double stat_values[FRQ_ST_count];
- calc_stats (frq, vf, stat_values);
- for (int j = 0; j < FRQ_ST_count; j++)
- {
- if (!(frq->stats & BIT_INDEX (j)))
- continue;
-
- union value v = { .f = vf->tab.n_valid ? stat_values[j] : SYSMIS };
- struct pivot_value *pv
- = (j == FRQ_ST_MODE || j == FRQ_ST_MINIMUM || j == FRQ_ST_MAXIMUM
- ? pivot_value_new_var_value (vf->var, &v)
- : pivot_value_new_number (v.f));
- pivot_splits_put2 (fst->splits, fst->table, var_idx, row++, pv);
- }
-
- for (size_t j = 0; j < frq->n_percentiles; j++)
- {
- const struct percentile *pc = &frq->percentiles[j];
- if (!pc->show)
- continue;
-
- union value v = {
- .f = vf->tab.n_valid ? vf->percentiles[j] : SYSMIS
- };
- pivot_splits_put2 (fst->splits, fst->table, var_idx, row++,
- pivot_value_new_var_value (vf->var, &v));
- }
-
- var_idx++;
- }
-
- if (!fst->splits)
- {
- frq_stats_table_destroy (fst);
- return NULL;
- }
- return fst;
-}
-
-static void
-frq_stats_table_destroy (struct frq_stats_table *fst)
-{
- if (!fst)
- return;
-
- pivot_table_submit (fst->table);
- pivot_splits_destroy (fst->splits);
- free (fst);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis. -*-c-*-
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include <config.h>
-
-#include "language/stats/friedman.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct friedman
-{
- double *rank_sum;
- double cc;
- double chi_sq;
- double w;
- const struct dictionary *dict;
-};
-
-static void show_ranks_box (const struct one_sample_test *ost,
- const struct friedman *fr);
-
-static void show_sig_box (const struct one_sample_test *ost,
- const struct friedman *fr);
-
-struct datum
-{
- long posn;
- double x;
-};
-
-static int
-cmp_x (const void *a_, const void *b_)
-{
- const struct datum *a = a_;
- const struct datum *b = b_;
-
- if (a->x < b->x)
- return -1;
-
- return (a->x > b->x);
-}
-
-static int
-cmp_posn (const void *a_, const void *b_)
-{
- const struct datum *a = a_;
- const struct datum *b = b_;
-
- if (a->posn < b->posn)
- return -1;
-
- return (a->posn > b->posn);
-}
-
-void
-friedman_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- double numerator = 0.0;
- double denominator = 0.0;
- int v;
- struct ccase *c;
- const struct dictionary *dict = dataset_dict (ds);
- const struct variable *weight = dict_get_weight (dict);
-
- struct one_sample_test *ost = UP_CAST (test, struct one_sample_test, parent);
- struct friedman_test *ft = UP_CAST (ost, struct friedman_test, parent);
- bool warn = true;
-
- double sigma_t = 0.0;
- struct datum *row = XCALLOC (ost->n_vars, struct datum);
- double rsq;
- struct friedman fr;
- fr.rank_sum = xcalloc (ost->n_vars, sizeof *fr.rank_sum);
- fr.cc = 0.0;
- fr.dict = dict;
- for (v = 0; v < ost->n_vars; ++v)
- {
- row[v].posn = v;
- fr.rank_sum[v] = 0.0;
- }
-
- input = casereader_create_filter_weight (input, dict, &warn, NULL);
- input = casereader_create_filter_missing (input,
- ost->vars, ost->n_vars,
- exclude, 0, 0);
-
- for (; (c = casereader_read (input)); case_unref (c))
- {
- double prev_x = SYSMIS;
- int run_length = 0;
-
- const double w = weight ? case_num (c, weight) : 1.0;
-
- fr.cc += w;
-
- for (v = 0; v < ost->n_vars; ++v)
- {
- const struct variable *var = ost->vars[v];
- const union value *val = case_data (c, var);
- row[v].x = val->f;
- }
-
- qsort (row, ost->n_vars, sizeof *row, cmp_x);
- for (v = 0; v < ost->n_vars; ++v)
- {
- double x = row[v].x;
- /* Replace value by the Rank */
- if (prev_x == x)
- {
- /* Deal with ties */
- int i;
- run_length++;
- for (i = v - run_length; i < v; ++i)
- {
- row[i].x *= run_length ;
- row[i].x += v + 1;
- row[i].x /= run_length + 1;
- }
- row[v].x = row[v-1].x;
- }
- else
- {
- row[v].x = v + 1;
- if (run_length > 0)
- {
- double t = run_length + 1;
- sigma_t += w * (pow3 (t) - t);
- }
- run_length = 0;
- }
- prev_x = x;
- }
- if (run_length > 0)
- {
- double t = run_length + 1;
- sigma_t += w * (pow3 (t) - t);
- }
-
- qsort (row, ost->n_vars, sizeof *row, cmp_posn);
-
- for (v = 0; v < ost->n_vars; ++v)
- fr.rank_sum[v] += row[v].x * w;
- }
- casereader_destroy (input);
- free (row);
-
-
- for (v = 0; v < ost->n_vars; ++v)
- {
- numerator += pow2 (fr.rank_sum[v]);
- }
-
- rsq = numerator;
-
- numerator *= 12.0 / (fr.cc * ost->n_vars * (ost->n_vars + 1));
- numerator -= 3 * fr.cc * (ost->n_vars + 1);
-
- denominator = 1 - sigma_t / (fr.cc * ost->n_vars * (pow2 (ost->n_vars) - 1));
-
- fr.chi_sq = numerator / denominator;
-
- if (ft->kendalls_w)
- {
- fr.w = 12 * rsq ;
- fr.w -= 3 * pow2 (fr.cc) *
- ost->n_vars * pow2 (ost->n_vars + 1);
-
- fr.w /= pow2 (fr.cc) * (pow3 (ost->n_vars) - ost->n_vars)
- - fr.cc * sigma_t;
- }
- else
- fr.w = SYSMIS;
-
- show_ranks_box (ost, &fr);
- show_sig_box (ost, &fr);
-
- free (fr.rank_sum);
-}
-
-\f
-
-
-static void
-show_ranks_box (const struct one_sample_test *ost, const struct friedman *fr)
-{
- struct pivot_table *table = pivot_table_create (N_("Ranks"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Mean Rank"),
- N_("Mean Rank"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t i = 0 ; i < ost->n_vars ; ++i)
- {
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (ost->vars[i]));
-
- pivot_table_put2 (table, 0, row,
- pivot_value_new_number (fr->rank_sum[i] / fr->cc));
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-show_sig_box (const struct one_sample_test *ost, const struct friedman *fr)
-{
- const struct friedman_test *ft = UP_CAST (ost, const struct friedman_test, parent);
-
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
- pivot_table_set_weight_var (table, dict_get_weight (fr->dict));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT);
- if (ft->kendalls_w)
- pivot_category_create_leaves (statistics->root, N_("Kendall's W"),
- PIVOT_RC_OTHER);
- pivot_category_create_leaves (statistics->root,
- N_("Chi-Square"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- double entries[5];
- int n = 0;
-
- entries[n++] = fr->cc;
- if (ft->kendalls_w)
- entries[n++] = fr->w;
- entries[n++] = fr->chi_sq;
- entries[n++] = ost->n_vars - 1;
- entries[n++] = gsl_cdf_chisq_Q (fr->chi_sq, ost->n_vars - 1);
- assert (n <= sizeof entries / sizeof *entries);
-
- for (size_t i = 0; i < n; i++)
- pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !friedman_h
-#define friedman_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-struct friedman_test
-{
- struct one_sample_test parent;
-
- /* Calculate and display the Kendall W statistic */
- bool kendalls_w;
-};
-
-
-void friedman_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool,
- double);
-
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011, 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_matrix.h>
-#include <gsl/gsl_combination.h>
-#include <math.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/ll.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/taint.h"
-#include "linreg/sweep.h"
-#include "math/categoricals.h"
-#include "math/covariance.h"
-#include "math/interaction.h"
-#include "math/moments.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-struct glm_spec
- {
- const struct variable **dep_vars;
- size_t n_dep_vars;
-
- const struct variable **factor_vars;
- size_t n_factor_vars;
-
- struct interaction **interactions;
- size_t n_interactions;
-
- enum mv_class exclude;
-
- const struct variable *wv; /* The weight variable */
-
- const struct dictionary *dict;
-
- int ss_type;
- bool intercept;
-
- double alpha;
-
- bool dump_coding;
- };
-
-struct glm_workspace
- {
- double total_ssq;
- struct moments *totals;
-
- struct categoricals *cats;
-
- /*
- Sums of squares due to different variables. Element 0 is the SSE
- for the entire model. For i > 0, element i is the SS due to
- variable i.
- */
- gsl_vector *ssq;
- };
-
-/* Default design: all possible interactions */
-static void
-design_full (struct glm_spec *glm)
-{
- size_t n = (1 << glm->n_factor_vars) - 1;
- glm->interactions = xnmalloc (n, sizeof *glm->interactions);
-
- /* All subsets, with exception of the empty set, of [0, glm->n_factor_vars) */
- for (size_t sz = 1; sz <= glm->n_factor_vars; ++sz)
- {
- gsl_combination *c = gsl_combination_calloc (glm->n_factor_vars, sz);
-
- do
- {
- struct interaction *iact = interaction_create (NULL);
- for (int e = 0; e < gsl_combination_k (c); ++e)
- interaction_add_variable (
- iact, glm->factor_vars [gsl_combination_get (c, e)]);
-
- glm->interactions[glm->n_interactions++] = iact;
- }
- while (gsl_combination_next (c) == GSL_SUCCESS);
-
- gsl_combination_free (c);
- }
- assert (glm->n_interactions == n);
-}
-
-static void output_glm (const struct glm_spec *,
- const struct glm_workspace *ws);
-static void run_glm (struct glm_spec *cmd, struct casereader *input,
- const struct dataset *ds);
-
-static struct interaction *parse_design_term (struct lexer *,
- const struct dictionary *);
-
-int
-cmd_glm (struct lexer *lexer, struct dataset *ds)
-{
- struct const_var_set *factors = NULL;
- bool design = false;
- struct dictionary *dict = dataset_dict (ds);
- struct glm_spec glm = {
- .dict = dict,
- .exclude = MV_ANY,
- .intercept = true,
- .wv = dict_get_weight (dict),
- .alpha = 0.05,
- .ss_type = 3,
- };
-
- int dep_vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, glm.dict,
- &glm.dep_vars, &glm.n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- int dep_vars_end = lex_ofs (lexer) - 1;
-
- if (!lex_force_match (lexer, T_BY))
- goto error;
-
- if (!parse_variables_const (lexer, glm.dict,
- &glm.factor_vars, &glm.n_factor_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
-
- if (glm.n_dep_vars > 1)
- {
- lex_ofs_error (lexer, dep_vars_start, dep_vars_end,
- _("Multivariate analysis is not yet implemented."));
- goto error;
- }
-
- factors = const_var_set_create_from_array (glm.factor_vars, glm.n_factor_vars);
-
- size_t allocated_interactions = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- glm.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- glm.exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "INTERCEPT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- glm.intercept = true;
- else if (lex_match_id (lexer, "EXCLUDE"))
- glm.intercept = false;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_match_phrase (lexer, "ALPHA(")
- || !lex_force_num (lexer))
- goto error;
- glm.alpha = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "METHOD"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_match_phrase (lexer, "SSTYPE(")
- || !lex_force_int_range (lexer, "SSTYPE", 1, 3))
- goto error;
-
- glm.ss_type = lex_integer (lexer);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "DESIGN"))
- {
- lex_match (lexer, T_EQUALS);
-
- do
- {
- struct interaction *iact = parse_design_term (lexer, glm.dict);
- if (!iact)
- goto error;
-
- if (glm.n_interactions >= allocated_interactions)
- glm.interactions = x2nrealloc (glm.interactions,
- &allocated_interactions,
- sizeof *glm.interactions);
- glm.interactions[glm.n_interactions++] = iact;
-
- lex_match (lexer, T_COMMA);
- }
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH);
-
- if (glm.n_interactions > 0)
- design = true;
- }
- else if (lex_match_id (lexer, "SHOWCODES"))
- {
- /* Undocumented debug option */
- glm.dump_coding = true;
- }
- else
- {
- lex_error_expecting (lexer, "MISSING", "INTERCEPT", "CRITERIA",
- "METHOD", "DESIGN");
- goto error;
- }
- }
-
- if (!design)
- design_full (&glm);
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), glm.dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- run_glm (&glm, group, ds);
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- const_var_set_destroy (factors);
- free (glm.factor_vars);
- for (size_t i = 0; i < glm.n_interactions; ++i)
- interaction_destroy (glm.interactions[i]);
-
- free (glm.interactions);
- free (glm.dep_vars);
-
- return CMD_SUCCESS;
-
-error:
- const_var_set_destroy (factors);
- free (glm.factor_vars);
- for (size_t i = 0; i < glm.n_interactions; ++i)
- interaction_destroy (glm.interactions[i]);
-
- free (glm.interactions);
- free (glm.dep_vars);
-
- return CMD_FAILURE;
-}
-
-static inline bool
-not_dropped (size_t j, const bool *ff)
-{
- return !ff[j];
-}
-
-static void
-fill_submatrix (const gsl_matrix * cov, gsl_matrix * submatrix, bool *dropped_f)
-{
- size_t i;
- size_t j;
- size_t n = 0;
- size_t m = 0;
-
- for (i = 0; i < cov->size1; i++)
- {
- if (not_dropped (i, dropped_f))
- {
- m = 0;
- for (j = 0; j < cov->size2; j++)
- {
- if (not_dropped (j, dropped_f))
- {
- gsl_matrix_set (submatrix, n, m,
- gsl_matrix_get (cov, i, j));
- m++;
- }
- }
- n++;
- }
- }
-}
-
-
-/*
- Type 1 sums of squares.
- Populate SSQ with the Type 1 sums of squares according to COV
- */
-static void
-ssq_type1 (struct covariance *cov, gsl_vector *ssq, const struct glm_spec *cmd)
-{
- const gsl_matrix *cm = covariance_calculate_unnormalized (cov);
- size_t i;
- size_t k;
- bool *model_dropped = XCALLOC (covariance_dim (cov), bool);
- bool *submodel_dropped = XCALLOC (covariance_dim (cov), bool);
- const struct categoricals *cats = covariance_get_categoricals (cov);
-
- size_t n_dropped_model = 0;
- size_t n_dropped_submodel = 0;
-
- for (i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
- {
- n_dropped_model++;
- n_dropped_submodel++;
- model_dropped[i] = true;
- submodel_dropped[i] = true;
- }
-
- for (k = 0; k < cmd->n_interactions; k++)
- {
- gsl_matrix *model_cov = NULL;
- gsl_matrix *submodel_cov = NULL;
-
- n_dropped_submodel = n_dropped_model;
- for (i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
- submodel_dropped[i] = model_dropped[i];
-
- for (i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
- {
- const struct interaction * x =
- categoricals_get_interaction_by_subscript (cats, i - cmd->n_dep_vars);
-
- if (x == cmd->interactions [k])
- {
- model_dropped[i] = false;
- n_dropped_model--;
- }
- }
-
- model_cov = gsl_matrix_alloc (cm->size1 - n_dropped_model, cm->size2 - n_dropped_model);
- submodel_cov = gsl_matrix_alloc (cm->size1 - n_dropped_submodel, cm->size2 - n_dropped_submodel);
-
- fill_submatrix (cm, model_cov, model_dropped);
- fill_submatrix (cm, submodel_cov, submodel_dropped);
-
- reg_sweep (model_cov, 0);
- reg_sweep (submodel_cov, 0);
-
- gsl_vector_set (ssq, k + 1,
- gsl_matrix_get (submodel_cov, 0, 0) - gsl_matrix_get (model_cov, 0, 0)
- );
-
- gsl_matrix_free (model_cov);
- gsl_matrix_free (submodel_cov);
- }
-
- free (model_dropped);
- free (submodel_dropped);
-}
-
-/*
- Type 2 sums of squares.
- Populate SSQ with the Type 2 sums of squares according to COV
- */
-static void
-ssq_type2 (struct covariance *cov, gsl_vector *ssq, const struct glm_spec *cmd)
-{
- const gsl_matrix *cm = covariance_calculate_unnormalized (cov);
- bool *model_dropped = XCALLOC (covariance_dim (cov), bool);
- bool *submodel_dropped = XCALLOC (covariance_dim (cov), bool);
- const struct categoricals *cats = covariance_get_categoricals (cov);
-
- for (size_t k = 0; k < cmd->n_interactions; k++)
- {
- gsl_matrix *model_cov = NULL;
- gsl_matrix *submodel_cov = NULL;
- size_t n_dropped_model = 0;
- size_t n_dropped_submodel = 0;
- for (size_t i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
- {
- const struct interaction * x =
- categoricals_get_interaction_by_subscript (cats, i - cmd->n_dep_vars);
-
- model_dropped[i] = false;
- submodel_dropped[i] = false;
- if (interaction_is_subset (cmd->interactions [k], x))
- {
- assert (n_dropped_submodel < covariance_dim (cov));
- n_dropped_submodel++;
- submodel_dropped[i] = true;
-
- if (cmd->interactions [k]->n_vars < x->n_vars)
- {
- assert (n_dropped_model < covariance_dim (cov));
- n_dropped_model++;
- model_dropped[i] = true;
- }
- }
- }
-
- model_cov = gsl_matrix_alloc (cm->size1 - n_dropped_model, cm->size2 - n_dropped_model);
- submodel_cov = gsl_matrix_alloc (cm->size1 - n_dropped_submodel, cm->size2 - n_dropped_submodel);
-
- fill_submatrix (cm, model_cov, model_dropped);
- fill_submatrix (cm, submodel_cov, submodel_dropped);
-
- reg_sweep (model_cov, 0);
- reg_sweep (submodel_cov, 0);
-
- gsl_vector_set (ssq, k + 1,
- gsl_matrix_get (submodel_cov, 0, 0) - gsl_matrix_get (model_cov, 0, 0)
- );
-
- gsl_matrix_free (model_cov);
- gsl_matrix_free (submodel_cov);
- }
-
- free (model_dropped);
- free (submodel_dropped);
-}
-
-/*
- Type 3 sums of squares.
- Populate SSQ with the Type 2 sums of squares according to COV
- */
-static void
-ssq_type3 (struct covariance *cov, gsl_vector *ssq, const struct glm_spec *cmd)
-{
- const gsl_matrix *cm = covariance_calculate_unnormalized (cov);
- bool *model_dropped = XCALLOC (covariance_dim (cov), bool);
- bool *submodel_dropped = XCALLOC (covariance_dim (cov), bool);
- const struct categoricals *cats = covariance_get_categoricals (cov);
-
- gsl_matrix *submodel_cov = gsl_matrix_alloc (cm->size1, cm->size2);
- fill_submatrix (cm, submodel_cov, submodel_dropped);
- reg_sweep (submodel_cov, 0);
- double ss0 = gsl_matrix_get (submodel_cov, 0, 0);
- gsl_matrix_free (submodel_cov);
- free (submodel_dropped);
-
- for (size_t k = 0; k < cmd->n_interactions; k++)
- {
- size_t n_dropped_model = 0;
- for (size_t i = cmd->n_dep_vars; i < covariance_dim (cov); i++)
- {
- const struct interaction * x =
- categoricals_get_interaction_by_subscript (cats, i - cmd->n_dep_vars);
-
- model_dropped[i] = false;
-
- if (cmd->interactions [k] == x)
- {
- assert (n_dropped_model < covariance_dim (cov));
- n_dropped_model++;
- model_dropped[i] = true;
- }
- }
-
- gsl_matrix *model_cov = gsl_matrix_alloc (cm->size1 - n_dropped_model,
- cm->size2 - n_dropped_model);
-
- fill_submatrix (cm, model_cov, model_dropped);
-
- reg_sweep (model_cov, 0);
-
- gsl_vector_set (ssq, k + 1, gsl_matrix_get (model_cov, 0, 0) - ss0);
-
- gsl_matrix_free (model_cov);
- }
- free (model_dropped);
-}
-
-static void
-run_glm (struct glm_spec *cmd, struct casereader *input,
- const struct dataset *ds)
-{
- bool warn_bad_weight = true;
- struct dictionary *dict = dataset_dict (ds);
-
-
- input = casereader_create_filter_missing (input,
- cmd->dep_vars, cmd->n_dep_vars,
- cmd->exclude,
- NULL, NULL);
-
- input = casereader_create_filter_missing (input,
- cmd->factor_vars, cmd->n_factor_vars,
- cmd->exclude,
- NULL, NULL);
-
- struct glm_workspace ws = {
- .cats = categoricals_create (cmd->interactions, cmd->n_interactions,
- cmd->wv, MV_ANY)
- };
-
- struct covariance *cov = covariance_2pass_create (
- cmd->n_dep_vars, cmd->dep_vars, ws.cats, cmd->wv, cmd->exclude, true);
-
- output_split_file_values_peek (ds, input);
-
- struct taint *taint = taint_clone (casereader_get_taint (input));
-
- ws.totals = moments_create (MOMENT_VARIANCE);
-
- struct casereader *reader = casereader_clone (input);
- struct ccase *c;
- for (; (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- double weight = dict_get_case_weight (dict, c, &warn_bad_weight);
-
- for (int v = 0; v < cmd->n_dep_vars; ++v)
- moments_pass_one (ws.totals, case_num (c, cmd->dep_vars[v]), weight);
-
- covariance_accumulate_pass1 (cov, c);
- }
- casereader_destroy (reader);
-
- if (cmd->dump_coding)
- reader = casereader_clone (input);
- else
- reader = input;
-
- for (; (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- double weight = dict_get_case_weight (dict, c, &warn_bad_weight);
-
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- moments_pass_two (ws.totals, case_num (c, cmd->dep_vars[v]), weight);
-
- covariance_accumulate_pass2 (cov, c);
- }
- casereader_destroy (reader);
-
-
- if (cmd->dump_coding)
- {
- struct pivot_table *t = covariance_dump_enc_header (cov);
- for (reader = input;
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- covariance_dump_enc (cov, c, t);
- }
-
- pivot_table_submit (t);
- }
-
- {
- const gsl_matrix *ucm = covariance_calculate_unnormalized (cov);
- gsl_matrix *cm = gsl_matrix_alloc (ucm->size1, ucm->size2);
- gsl_matrix_memcpy (cm, ucm);
-
- // dump_matrix (cm);
-
- ws.total_ssq = gsl_matrix_get (cm, 0, 0);
-
- reg_sweep (cm, 0);
-
- /*
- Store the overall SSE.
- */
- ws.ssq = gsl_vector_alloc (cm->size1);
- gsl_vector_set (ws.ssq, 0, gsl_matrix_get (cm, 0, 0));
- switch (cmd->ss_type)
- {
- case 1:
- ssq_type1 (cov, ws.ssq, cmd);
- break;
- case 2:
- ssq_type2 (cov, ws.ssq, cmd);
- break;
- case 3:
- ssq_type3 (cov, ws.ssq, cmd);
- break;
- default:
- NOT_REACHED ();
- break;
- }
- // dump_matrix (cm);
- gsl_matrix_free (cm);
- }
-
- if (!taint_has_tainted_successor (taint))
- output_glm (cmd, &ws);
-
- gsl_vector_free (ws.ssq);
-
- covariance_destroy (cov);
- moments_destroy (ws.totals);
-
- taint_destroy (taint);
-}
-
-static void
-put_glm_row (struct pivot_table *table, int row,
- double a, double b, double c, double d, double e)
-{
- double entries[] = { a, b, c, d, e };
-
- for (size_t col = 0; col < sizeof entries / sizeof *entries; col++)
- if (entries[col] != SYSMIS)
- pivot_table_put2 (table, col, row,
- pivot_value_new_number (entries[col]));
-}
-
-static void
-output_glm (const struct glm_spec *cmd, const struct glm_workspace *ws)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Tests of Between-Subjects Effects"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- (cmd->ss_type == 1 ? N_("Type I Sum Of Squares")
- : cmd->ss_type == 2 ? N_("Type II Sum Of Squares")
- : N_("Type III Sum Of Squares")), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_COUNT,
- N_("Mean Square"), PIVOT_RC_OTHER,
- N_("F"), PIVOT_RC_OTHER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *source = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Source"),
- cmd->intercept ? N_("Corrected Model") : N_("Model"));
-
- double n_total, mean;
- moments_calculate (ws->totals, &n_total, &mean, NULL, NULL, NULL);
-
- double df_corr = 1.0 + categoricals_df_total (ws->cats);
-
- double mse = gsl_vector_get (ws->ssq, 0) / (n_total - df_corr);
- double intercept_ssq = pow2 (mean * n_total) / n_total;
- if (cmd->intercept)
- {
- int row = pivot_category_create_leaf (
- source->root, pivot_value_new_text (N_("Intercept")));
-
- /* The intercept for unbalanced models is of limited use and
- nobody knows how to calculate it properly */
- if (categoricals_isbalanced (ws->cats))
- {
- const double df = 1.0;
- const double F = intercept_ssq / df / mse;
- put_glm_row (table, row, intercept_ssq, 1.0, intercept_ssq / df,
- F, gsl_cdf_fdist_Q (F, df, n_total - df_corr));
- }
- }
-
- double ssq_effects = 0.0;
- for (int f = 0; f < cmd->n_interactions; ++f)
- {
- double df = categoricals_df (ws->cats, f);
- double ssq = gsl_vector_get (ws->ssq, f + 1);
- ssq_effects += ssq;
- if (!cmd->intercept)
- {
- df++;
- ssq += intercept_ssq;
- }
- double F = ssq / df / mse;
-
- struct string str = DS_EMPTY_INITIALIZER;
- interaction_to_string (cmd->interactions[f], &str);
- int row = pivot_category_create_leaf (
- source->root, pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
-
- put_glm_row (table, row, ssq, df, ssq / df, F,
- gsl_cdf_fdist_Q (F, df, n_total - df_corr));
- }
-
- {
- /* Model / Corrected Model */
- double df = df_corr;
- double ssq = ws->total_ssq - gsl_vector_get (ws->ssq, 0);
- if (cmd->intercept)
- df--;
- else
- ssq += intercept_ssq;
- double F = ssq / df / mse;
- put_glm_row (table, 0, ssq, df, ssq / df, F,
- gsl_cdf_fdist_Q (F, df, n_total - df_corr));
- }
-
- {
- int row = pivot_category_create_leaf (source->root,
- pivot_value_new_text (N_("Error")));
- const double df = n_total - df_corr;
- const double ssq = gsl_vector_get (ws->ssq, 0);
- const double mse = ssq / df;
- put_glm_row (table, row, ssq, df, mse, SYSMIS, SYSMIS);
- }
-
- {
- int row = pivot_category_create_leaf (source->root,
- pivot_value_new_text (N_("Total")));
- put_glm_row (table, row, ws->total_ssq + intercept_ssq, n_total,
- SYSMIS, SYSMIS, SYSMIS);
- }
-
- if (cmd->intercept)
- {
- int row = pivot_category_create_leaf (
- source->root, pivot_value_new_text (N_("Corrected Total")));
- put_glm_row (table, row, ws->total_ssq, n_total - 1.0, SYSMIS,
- SYSMIS, SYSMIS);
- }
-
- pivot_table_submit (table);
-}
-
-#if 0
-static void
-dump_matrix (const gsl_matrix * m)
-{
- size_t i, j;
- for (i = 0; i < m->size1; ++i)
- {
- for (j = 0; j < m->size2; ++j)
- {
- double x = gsl_matrix_get (m, i, j);
- printf ("%.3f ", x);
- }
- printf ("\n");
- }
- printf ("\n");
-}
-#endif
-
-
-\f
-static struct interaction *
-parse_design_term (struct lexer *lexer, const struct dictionary *dict)
-{
- struct interaction *iact = interaction_create (NULL);
- do
- {
- struct variable *var = parse_variable (lexer, dict);
- if (!var)
- goto error;
- interaction_add_variable (iact, var);
-
- if (lex_match (lexer, T_LPAREN) || lex_match_id (lexer, "WITHIN"))
- {
- lex_next_error (lexer, -1, -1,
- "Nested variables are not yet implemented.");
- goto error;
- }
- }
- while (lex_match (lexer, T_ASTERISK));
-
- return iact;
-
-error:
- interaction_destroy (iact);
- return NULL;
-}
+++ /dev/null
-/*
- PSPP - a program for statistical analysis.
- Copyright (C) 2012, 2013, 2015, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-/*
- * This module implements the graph command
- */
-
-#include <config.h>
-
-#include <math.h>
-#include "gl/xalloc.h"
-#include <gsl/gsl_cdf.h>
-
-#include "libpspp/assertion.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/caseproto.h"
-#include "data/subcase.h"
-
-
-#include "data/format.h"
-
-#include "math/chart-geometry.h"
-#include "math/histogram.h"
-#include "math/moments.h"
-#include "math/sort.h"
-#include "math/order-stats.h"
-#include "output/charts/plot-hist.h"
-#include "output/charts/scatterplot.h"
-#include "output/charts/barchart.h"
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "language/stats/freq.h"
-#include "language/stats/chart-category.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-enum chart_type
- {
- CT_NONE,
- CT_BAR,
- CT_LINE,
- CT_PIE,
- CT_ERRORBAR,
- CT_HILO,
- CT_HISTOGRAM,
- CT_SCATTERPLOT,
- CT_PARETO
- };
-
-enum scatter_type
- {
- ST_BIVARIATE,
- ST_OVERLAY,
- ST_MATRIX,
- ST_XYZ
- };
-
-enum bar_type
- {
- CBT_SIMPLE,
- CBT_GROUPED,
- CBT_STACKED,
- CBT_RANGE
- };
-
-
-/* Variable index for histogram case */
-enum
- {
- HG_IDX_X,
- HG_IDX_WT
- };
-
-struct exploratory_stats
-{
- double missing;
- double non_missing;
-
- struct moments *mom;
-
- double minimum;
- double maximum;
-
- /* Total weight */
- double cc;
-
- /* The minimum weight */
- double cmin;
-};
-
-
-struct graph
-{
- struct pool *pool;
-
- size_t n_dep_vars;
- const struct variable **dep_vars;
- struct exploratory_stats *es;
-
- enum mv_class dep_excl;
- enum mv_class fctr_excl;
-
- const struct dictionary *dict;
-
- bool missing_pw;
-
- /* ------------ Graph ---------------- */
- bool normal; /* For histograms, draw the normal curve */
-
- enum chart_type chart_type;
- enum scatter_type scatter_type;
- enum bar_type bar_type;
- const struct variable *by_var[2];
- size_t n_by_vars;
-
- struct subcase ordering; /* Ordering for aggregation */
- int agr; /* Index into ag_func */
-
- /* A caseproto that contains the plot data */
- struct caseproto *gr_proto;
-};
-
-
-
-
-static double
-calc_mom1 (double acc, double x, double w)
-{
- return acc + x * w;
-}
-
-static double
-calc_mom0 (double acc, double x UNUSED, double w)
-{
- return acc + w;
-}
-
-static double
-pre_low_extreme (void)
-{
- return -DBL_MAX;
-}
-
-static double
-calc_max (double acc, double x, double w UNUSED)
-{
- return (acc > x) ? acc : x;
-}
-
-static double
-pre_high_extreme (void)
-{
- return DBL_MAX;
-}
-
-static double
-calc_min (double acc, double x, double w UNUSED)
-{
- return (acc < x) ? acc : x;
-}
-
-static double
-post_normalise (double acc, double cc)
-{
- return acc / cc;
-}
-
-static double
-post_percentage (double acc, double ccc)
-{
- return acc / ccc * 100.0;
-}
-
-const struct ag_func ag_func[] =
- {
- {"COUNT", N_("Count"), 0, 0, NULL, calc_mom0, 0, 0},
- {"PCT", N_("Percentage"), 0, 0, NULL, calc_mom0, 0, post_percentage},
- {"CUFREQ", N_("Cumulative Count"), 0, 1, NULL, calc_mom0, 0, 0},
- {"CUPCT", N_("Cumulative Percent"), 0, 1, NULL, calc_mom0, 0,
- post_percentage},
-
- {"MEAN", N_("Mean"), 1, 0, NULL, calc_mom1, post_normalise, 0},
- {"SUM", N_("Sum"), 1, 0, NULL, calc_mom1, 0, 0},
- {"MAXIMUM", N_("Maximum"), 1, 0, pre_low_extreme, calc_max, 0, 0},
- {"MINIMUM", N_("Minimum"), 1, 0, pre_high_extreme, calc_min, 0, 0},
- };
-
-const int N_AG_FUNCS = sizeof (ag_func) / sizeof (ag_func[0]);
-
-static bool
-parse_function_name (struct lexer *lexer, int *agr)
-{
- for (size_t i = 0; i < N_AG_FUNCS; ++i)
- {
- if (lex_match_id (lexer, ag_func[i].name))
- {
- *agr = i;
- return true;
- }
- }
-
- const char *ag_func_names[N_AG_FUNCS];
- for (size_t i = 0; i < N_AG_FUNCS; ++i)
- ag_func_names[i] = ag_func[i].name;
- lex_error_expecting_array (lexer, ag_func_names, N_AG_FUNCS);
- return false;
-}
-
-static bool
-parse_function (struct lexer *lexer, struct graph *graph)
-{
- if (!parse_function_name (lexer, &graph->agr))
- return false;
-
- size_t arity = ag_func[graph->agr].arity;
- graph->n_dep_vars = arity;
- if (arity > 0)
- {
- if (!lex_force_match (lexer, T_LPAREN))
- return false;
-
- graph->dep_vars = xcalloc (graph->n_dep_vars, sizeof (graph->dep_vars));
- for (int v = 0; v < arity; ++v)
- {
- graph->dep_vars[v] = parse_variable (lexer, graph->dict);
- if (!graph->dep_vars[v])
- return false;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
-
- if (!lex_force_match (lexer, T_BY))
- return false;
-
- graph->by_var[0] = parse_variable (lexer, graph->dict);
- if (!graph->by_var[0])
- return false;
- subcase_add_var (&graph->ordering, graph->by_var[0], SC_ASCEND);
- graph->n_by_vars++;
-
- if (lex_match (lexer, T_BY))
- {
- graph->by_var[1] = parse_variable (lexer, graph->dict);
- if (!graph->by_var[1])
- return false;
- subcase_add_var (&graph->ordering, graph->by_var[1], SC_ASCEND);
- graph->n_by_vars++;
- }
-
- return true;
-}
-
-static void
-show_scatterplot (const struct graph *cmd, struct casereader *input)
-{
- struct scatterplot_chart *scatterplot;
- bool byvar_overflow = false;
-
- char *title = (cmd->n_by_vars > 0
- ? xasprintf (_("%s vs. %s by %s"),
- var_to_string (cmd->dep_vars[1]),
- var_to_string (cmd->dep_vars[0]),
- var_to_string (cmd->by_var[0]))
- : xasprintf (_("%s vs. %s"),
- var_to_string (cmd->dep_vars[1]),
- var_to_string (cmd->dep_vars[0])));;
-
- scatterplot = scatterplot_create (input,
- var_to_string(cmd->dep_vars[0]),
- var_to_string(cmd->dep_vars[1]),
- (cmd->n_by_vars > 0) ? cmd->by_var[0]
- : NULL,
- &byvar_overflow,
- title,
- cmd->es[0].minimum, cmd->es[0].maximum,
- cmd->es[1].minimum, cmd->es[1].maximum);
- scatterplot_chart_submit (scatterplot);
- free (title);
-
- if (byvar_overflow)
- msg (MW, _("Maximum number of scatterplot categories reached. "
- "Your BY variable has too many distinct values. "
- "The coloring of the plot will not be correct."));
-}
-
-static void
-show_histogr (const struct graph *cmd, struct casereader *input)
-{
- struct histogram *histogram;
-
- if (cmd->es[0].cc <= 0)
- {
- casereader_destroy (input);
- return;
- }
-
- /* Sturges Rule */
- double bin_width = fabs (cmd->es[0].minimum - cmd->es[0].maximum)
- / (1 + log2 (cmd->es[0].cc));
- histogram = histogram_create (bin_width,
- cmd->es[0].minimum, cmd->es[0].maximum);
- if (!histogram)
- {
- casereader_destroy (input);
- return;
- }
-
- struct ccase *c;
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- {
- const double x = case_num_idx (c, HG_IDX_X);
- const double weight = case_num_idx (c, HG_IDX_WT);
- moments_pass_two (cmd->es[0].mom, x, weight);
- histogram_add (histogram, x, weight);
- }
- casereader_destroy (input);
-
- const char *label = var_to_string (cmd->dep_vars[0]);
- double n, mean, var;
- moments_calculate (cmd->es[0].mom, &n, &mean, &var, NULL, NULL);
- chart_submit (histogram_chart_create (histogram->gsl_hist, label, n, mean,
- sqrt (var), cmd->normal));
-
- statistic_destroy (&histogram->parent);
-}
-
-static void
-cleanup_exploratory_stats (struct graph *cmd)
-{
- for (size_t v = 0; v < cmd->n_dep_vars; ++v)
- moments_destroy (cmd->es[v].mom);
-}
-
-static bool
-any_categorical_missing (const struct graph *cmd, const struct ccase *c)
-{
- for (size_t v = 0; v < cmd->n_by_vars; ++v)
- if (var_is_value_missing (cmd->by_var[v], case_data (c, cmd->by_var[v]))
- & cmd->fctr_excl)
- return true;
- return false;
-}
-
-static struct freq *
-find_fcol (struct hmap *columns, const union value *value, size_t hash,
- int width)
-{
- struct freq *fcol;
- HMAP_FOR_EACH_WITH_HASH (fcol, struct freq, node, hash, columns)
- if (value_equal (value, &fcol->values[0], width))
- return fcol;
- return NULL;
-}
-
-static void
-run_barchart (struct graph *cmd, struct casereader *input)
-{
- double ccc = 0.0;
-
- if (cmd->missing_pw == false)
- input = casereader_create_filter_missing (input,
- cmd->dep_vars,
- cmd->n_dep_vars,
- cmd->dep_excl,
- NULL,
- NULL);
-
-
- input = sort_execute (input, &cmd->ordering);
-
- struct freq **cells = NULL;
- size_t n_cells = 0;
- size_t allocated_cells = 0;
-
- struct hmap columns = HMAP_INITIALIZER (columns);
- assert (cmd->n_by_vars <= 2);
- struct casegrouper *grouper = casegrouper_create_vars (input, cmd->by_var,
- cmd->n_by_vars);
- struct casereader *group;
- for (; casegrouper_get_next_group (grouper, &group);
- casereader_destroy (group))
- {
- struct ccase *c = casereader_peek (group, 0);
- if (any_categorical_missing (cmd, c))
- {
- case_unref (c);
- continue;
- }
-
- if (n_cells >= allocated_cells)
- cells = x2nrealloc (cells, &allocated_cells, sizeof *cells);
- cells[n_cells++] = xzalloc (table_entry_size (cmd->n_by_vars));
-
- if (ag_func[cmd->agr].cumulative && n_cells >= 2)
- cells[n_cells - 1]->count = cells[n_cells - 2]->count;
- else
- cells[n_cells - 1]->count = 0;
- if (ag_func[cmd->agr].pre)
- cells[n_cells - 1]->count = ag_func[cmd->agr].pre();
-
- if (cmd->n_by_vars > 1)
- {
- const union value *vv = case_data (c, cmd->by_var[1]);
- const double weight = dict_get_case_weight (cmd->dict, c, NULL);
- int v1_width = var_get_width (cmd->by_var[1]);
- size_t hash = value_hash (vv, v1_width, 0);
-
- struct freq *fcol = find_fcol (&columns, vv, hash, v1_width);
- if (!fcol)
- {
- fcol = xzalloc (sizeof *fcol);
- value_clone (&fcol->values[0], vv, v1_width);
- hmap_insert (&columns, &fcol->node, hash);
- }
- fcol->count += weight;
- }
-
- for (size_t v = 0; v < cmd->n_by_vars; ++v)
- value_clone (&cells[n_cells - 1]->values[v],
- case_data (c, cmd->by_var[v]),
- var_get_width (cmd->by_var[v]));
- case_unref (c);
-
- double cc = 0;
- for (; (c = casereader_read (group)) != NULL; case_unref (c))
- {
- const double weight = dict_get_case_weight (cmd->dict, c, NULL);
- const double x = (cmd->n_dep_vars > 0
- ? case_num (c, cmd->dep_vars[0]) : SYSMIS);
-
- cc += weight;
- cells[n_cells - 1]->count
- = ag_func[cmd->agr].calc (cells[n_cells - 1]->count, x, weight);
- }
-
- if (ag_func[cmd->agr].post)
- cells[n_cells - 1]->count
- = ag_func[cmd->agr].post (cells[n_cells - 1]->count, cc);
-
- ccc += cc;
- }
-
- casegrouper_destroy (grouper);
-
- for (int i = 0; i < n_cells; ++i)
- {
- if (ag_func[cmd->agr].ppost)
- {
- struct freq *cell = cells[i];
- if (cmd->n_by_vars > 1)
- {
- const union value *vv = &cell->values[1];
-
- int v1_width = var_get_width (cmd->by_var[1]);
- size_t hash = value_hash (vv, v1_width, 0);
-
- struct freq *fcol = find_fcol (&columns, vv, hash, v1_width);
- cell->count = ag_func[cmd->agr].ppost (cell->count, fcol->count);
- }
- else
- cell->count = ag_func[cmd->agr].ppost (cell->count, ccc);
- }
- }
-
- if (cmd->n_by_vars > 1)
- {
- struct freq *cell, *next;
- HMAP_FOR_EACH_SAFE (cell, next, struct freq, node, &columns)
- {
- value_destroy (cell->values, var_get_width (cmd->by_var[1]));
- free (cell);
- }
- }
- hmap_destroy (&columns);
-
- char *label = (cmd->n_dep_vars > 0
- ? xasprintf (_("%s of %s"),
- ag_func[cmd->agr].description,
- var_get_name (cmd->dep_vars[0]))
- : xstrdup (ag_func[cmd->agr].description));
- chart_submit (barchart_create (cmd->by_var, cmd->n_by_vars, label, false,
- cells, n_cells));
- free (label);
-
- for (int i = 0; i < n_cells; ++i)
- free (cells[i]);
-
- free (cells);
-}
-
-static void
-run_graph (struct graph *cmd, struct casereader *input)
-{
- cmd->es = pool_nmalloc (cmd->pool, cmd->n_dep_vars, sizeof *cmd->es);
- for (int v = 0; v < cmd->n_dep_vars; v++)
- cmd->es[v] = (struct exploratory_stats) {
- .mom = moments_create (MOMENT_KURTOSIS),
- .cmin = DBL_MAX,
- .maximum = -DBL_MAX,
- .minimum = DBL_MAX,
- };
-
- /* Always remove cases listwise. This is correct for the histogram because
- there is only one variable and a simple bivariate scatterplot. */
- input = casereader_create_filter_missing (input,
- cmd->dep_vars,
- cmd->n_dep_vars,
- cmd->dep_excl,
- NULL,
- NULL);
-
- struct casewriter *writer = autopaging_writer_create (cmd->gr_proto);
-
- /* The case data is copied to a new writer.
- The setup of the case depends on the chart type.
-
- For Scatterplot:
- - x is assumed in dep_vars[0].
- - y is assumed in dep_vars[1].
-
- For Histogram:
- - x is assumed in dep_vars[0]. */
- assert (SP_IDX_X == 0 && SP_IDX_Y == 1 && HG_IDX_X == 0);
-
- struct ccase *c;
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- {
- struct ccase *outcase = case_create (cmd->gr_proto);
- const double weight = dict_get_case_weight (cmd->dict, c, NULL);
- if (cmd->chart_type == CT_HISTOGRAM)
- *case_num_rw_idx (outcase, HG_IDX_WT) = weight;
- if (cmd->chart_type == CT_SCATTERPLOT && cmd->n_by_vars > 0)
- value_copy (case_data_rw_idx (outcase, SP_IDX_BY),
- case_data (c, cmd->by_var[0]),
- var_get_width (cmd->by_var[0]));
- for (int v = 0; v < cmd->n_dep_vars; v++)
- {
- const struct variable *var = cmd->dep_vars[v];
- const double x = case_num (c, var);
-
- if (var_is_value_missing (var, case_data (c, var)) & cmd->dep_excl)
- {
- cmd->es[v].missing += weight;
- continue;
- }
-
- /* Magically v value fits to SP_IDX_X, SP_IDX_Y, HG_IDX_X. */
- *case_num_rw_idx (outcase, v) = x;
-
- if (x > cmd->es[v].maximum)
- cmd->es[v].maximum = x;
-
- if (x < cmd->es[v].minimum)
- cmd->es[v].minimum = x;
-
- cmd->es[v].non_missing += weight;
-
- moments_pass_one (cmd->es[v].mom, x, weight);
-
- cmd->es[v].cc += weight;
-
- if (cmd->es[v].cmin > weight)
- cmd->es[v].cmin = weight;
- }
- casewriter_write (writer, outcase);
- }
-
- struct casereader *reader = casewriter_make_reader (writer);
- switch (cmd->chart_type)
- {
- case CT_HISTOGRAM:
- show_histogr (cmd,reader);
- break;
-
- case CT_SCATTERPLOT:
- show_scatterplot (cmd,reader);
- break;
-
- case CT_NONE:
- case CT_BAR:
- case CT_LINE:
- case CT_PIE:
- case CT_ERRORBAR:
- case CT_HILO:
- case CT_PARETO:
- NOT_REACHED ();
- }
-
- casereader_destroy (input);
- cleanup_exploratory_stats (cmd);
-}
-
-int
-cmd_graph (struct lexer *lexer, struct dataset *ds)
-{
- struct graph graph = {
- .missing_pw = false,
-
- .pool = pool_create (),
-
- .dep_excl = MV_ANY,
- .fctr_excl = MV_ANY,
-
- .dict = dataset_dict (ds),
-
- .chart_type = CT_NONE,
- .scatter_type = ST_BIVARIATE,
- .gr_proto = caseproto_create (),
- .ordering = SUBCASE_EMPTY_INITIALIZER,
- };
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "HISTOGRAM"))
- {
- if (graph.chart_type != CT_NONE)
- {
- lex_next_error (lexer, -1, -1,
- _("Only one chart type is allowed."));
- goto error;
- }
- graph.normal = false;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_match_phrase (lexer, "NORMAL)"))
- goto error;
-
- graph.normal = true;
- }
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
- graph.chart_type = CT_HISTOGRAM;
- int vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, graph.dict,
- &graph.dep_vars, &graph.n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- if (graph.n_dep_vars > 1)
- {
- lex_ofs_error (lexer, vars_start, lex_ofs (lexer) - 1,
- _("Only one variable is allowed."));
- goto error;
- }
- }
- else if (lex_match_id (lexer, "BAR"))
- {
- if (graph.chart_type != CT_NONE)
- {
- lex_next_error (lexer, -1, -1,
- _("Only one chart type is allowed."));
- goto error;
- }
- graph.chart_type = CT_BAR;
- graph.bar_type = CBT_SIMPLE;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (lex_match_id (lexer, "SIMPLE"))
- {
- /* This is the default anyway */
- }
- else if (lex_match_id (lexer, "GROUPED"))
- {
- graph.bar_type = CBT_GROUPED;
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."), "GROUPED");
- goto error;
- }
- else if (lex_match_id (lexer, "STACKED"))
- {
- graph.bar_type = CBT_STACKED;
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."), "STACKED");
- goto error;
- }
- else if (lex_match_id (lexer, "RANGE"))
- {
- graph.bar_type = CBT_RANGE;
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."), "RANGE");
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "SIMPLE", "GROUPED",
- "STACKED", "RANGE");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
-
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- if (!parse_function (lexer, &graph))
- goto error;
- }
- else if (lex_match_id (lexer, "SCATTERPLOT"))
- {
- if (graph.chart_type != CT_NONE)
- {
- lex_next_error (lexer, -1, -1,
- _("Only one chart type is allowed."));
- goto error;
- }
- graph.chart_type = CT_SCATTERPLOT;
- if (lex_match (lexer, T_LPAREN))
- {
- if (lex_match_id (lexer, "BIVARIATE"))
- {
- /* This is the default anyway */
- }
- else if (lex_match_id (lexer, "OVERLAY"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"OVERLAY");
- goto error;
- }
- else if (lex_match_id (lexer, "MATRIX"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"MATRIX");
- goto error;
- }
- else if (lex_match_id (lexer, "XYZ"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"XYZ");
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "BIVARIATE", "OVERLAY",
- "MATRIX", "XYZ");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- int vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, graph.dict,
- &graph.dep_vars, &graph.n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
-
- if (graph.scatter_type == ST_BIVARIATE && graph.n_dep_vars != 1)
- {
- lex_ofs_error (lexer, vars_start, lex_ofs (lexer) - 1,
- _("Only one variable is allowed."));
- goto error;
- }
-
- if (!lex_force_match (lexer, T_WITH))
- goto error;
-
- vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, graph.dict,
- &graph.dep_vars, &graph.n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC | PV_APPEND))
- goto error;
-
- if (graph.scatter_type == ST_BIVARIATE && graph.n_dep_vars != 2)
- {
- lex_ofs_error (lexer, vars_start, lex_ofs (lexer) - 1,
- _("Only one variable is allowed."));
- goto error;
- }
-
- if (lex_match (lexer, T_BY))
- {
- const struct variable *v = NULL;
- if (!lex_match_variable (lexer,graph.dict,&v))
- {
- lex_error (lexer, _("Syntax error expecting variable name."));
- goto error;
- }
- graph.by_var[0] = v;
- graph.n_by_vars = 1;
- }
- }
- else if (lex_match_id (lexer, "LINE"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"LINE");
- goto error;
- }
- else if (lex_match_id (lexer, "PIE"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"PIE");
- goto error;
- }
- else if (lex_match_id (lexer, "ERRORBAR"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"ERRORBAR");
- goto error;
- }
- else if (lex_match_id (lexer, "PARETO"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"PARETO");
- goto error;
- }
- else if (lex_match_id (lexer, "TITLE"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"TITLE");
- goto error;
- }
- else if (lex_match_id (lexer, "SUBTITLE"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"SUBTITLE");
- goto error;
- }
- else if (lex_match_id (lexer, "FOOTNOTE"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not yet implemented."),"FOOTNOTE");
- goto error;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "LISTWISE"))
- graph.missing_pw = false;
- else if (lex_match_id (lexer, "VARIABLE"))
- graph.missing_pw = true;
- else if (lex_match_id (lexer, "EXCLUDE"))
- graph.dep_excl = MV_ANY;
- else if (lex_match_id (lexer, "INCLUDE"))
- graph.dep_excl = MV_SYSTEM;
- else if (lex_match_id (lexer, "REPORT"))
- graph.fctr_excl = 0;
- else if (lex_match_id (lexer, "NOREPORT"))
- graph.fctr_excl = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "LISTWISE", "VARIABLE",
- "EXCLUDE", "INCLUDE",
- "REPORT", "NOREPORT");
- goto error;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "HISTOGRAM", "BAR", "SCATTERPLOT", "LINE",
- "PIE", "ERRORBAR", "PARETO", "TITLE", "SUBTITLE",
- "FOOTNOTE", "MISSING");
- goto error;
- }
- }
-
- switch (graph.chart_type)
- {
- case CT_SCATTERPLOT:
- /* See scatterplot.h for the setup of the case prototype */
-
- /* x value - SP_IDX_X*/
- graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
-
- /* y value - SP_IDX_Y*/
- graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
- /* The by_var contains the plot categories for the different xy
- plot colors */
- if (graph.n_by_vars > 0) /* SP_IDX_BY */
- graph.gr_proto = caseproto_add_width (graph.gr_proto,
- var_get_width(graph.by_var[0]));
- break;
-
- case CT_HISTOGRAM:
- /* x value */
- graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
- /* weight value */
- graph.gr_proto = caseproto_add_width (graph.gr_proto, 0);
- break;
-
- case CT_BAR:
- break;
-
- case CT_NONE:
- lex_error_expecting (lexer, "HISTOGRAM", "SCATTERPLOT", "BAR");
- goto error;
-
- default:
- NOT_REACHED ();
- break;
- }
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), graph.dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- if (graph.chart_type == CT_BAR)
- run_barchart (&graph, group);
- else
- run_graph (&graph, group);
- }
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- subcase_uninit (&graph.ordering);
- free (graph.dep_vars);
- pool_destroy (graph.pool);
- caseproto_unref (graph.gr_proto);
-
- return CMD_SUCCESS;
-
- error:
- subcase_uninit (&graph.ordering);
- caseproto_unref (graph.gr_proto);
- free (graph.dep_vars);
- pool_destroy (graph.pool);
-
- return CMD_FAILURE;
-}
+++ /dev/null
-/* Pspp - a program for statistical analysis.
- Copyright (C) 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-
-#include <config.h>
-
-#include "jonckheere-terpstra.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "libpspp/assertion.h"
-#include "libpspp/hmap.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/sort.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-/* Returns true iff the independent variable lies in the
- between val1 and val2. Regardless of which is the greater value.
-*/
-static bool
-include_func_bi (const struct ccase *c, void *aux)
-{
- const struct n_sample_test *nst = aux;
- const union value *bigger = NULL;
- const union value *smaller = NULL;
-
- if (0 > value_compare_3way (&nst->val1, &nst->val2, var_get_width (nst->indep_var)))
- {
- bigger = &nst->val2;
- smaller = &nst->val1;
- }
- else
- {
- smaller = &nst->val2;
- bigger = &nst->val1;
- }
-
- if (0 < value_compare_3way (smaller, case_data (c, nst->indep_var), var_get_width (nst->indep_var)))
- return false;
-
- if (0 > value_compare_3way (bigger, case_data (c, nst->indep_var), var_get_width (nst->indep_var)))
- return false;
-
- return true;
-}
-
-struct group_data
-{
- /* The total of the caseweights in the group */
- double cc;
-
- /* A casereader containing the group data.
- This casereader contains just two values:
- 0: The raw value of the data
- 1: The cumulative caseweight
- */
- struct casereader *reader;
-};
-
-
-static double
-u (const struct group_data *grp0, const struct group_data *grp1)
-{
- struct ccase *c0;
-
- struct casereader *r0 = casereader_clone (grp0->reader);
- double usum = 0;
- double prev_cc0 = 0.0;
- for (; (c0 = casereader_read (r0)); case_unref (c0))
- {
- struct ccase *c1;
- struct casereader *r1 = casereader_clone (grp1->reader);
- double x0 = case_num_idx (c0, 0);
- double cc0 = case_num_idx (c0, 1);
- double w0 = cc0 - prev_cc0;
-
- double prev_cc1 = 0;
-
- for (; (c1 = casereader_read (r1)); case_unref (c1))
- {
- double x1 = case_num_idx (c1, 0);
- double cc1 = case_num_idx (c1, 1);
-
- if (x0 > x1)
- {
- /* Do nothing */
- }
- else if (x0 < x1)
- {
- usum += w0 * (grp1->cc - prev_cc1);
- case_unref (c1);
- break;
- }
- else
- {
-#if 1
- usum += w0 * ((grp1->cc - prev_cc1) / 2.0);
-#else
- usum += w0 * (grp1->cc - (prev_cc1 + cc1) / 2.0);
-#endif
- case_unref (c1);
- break;
- }
-
- prev_cc1 = cc1;
- }
- casereader_destroy (r1);
- prev_cc0 = cc0;
- }
- casereader_destroy (r0);
-
- return usum;
-}
-
-
-typedef double func_f (double e_l);
-
-/*
- These 3 functions are used repeatedly in the calculation of the
- variance of the JT statistic.
- Having them explicitly defined makes the variance calculation
- a lot simpler.
-*/
-static double
-ff1 (double e)
-{
- return e * (e - 1) * (2*e + 5);
-}
-
-static double
-ff2 (double e)
-{
- return e * (e - 1) * (e - 2);
-}
-
-static double
-ff3 (double e)
-{
- return e * (e - 1) ;
-}
-
-static func_f *mff[3] =
- {
- ff1, ff2, ff3
- };
-
-
-/*
- This function does the following:
- It creates an ordered set of *distinct* values from IR.
- For each case in that set, it calls f[0..N] passing it the caseweight.
- It returns the sum of f[j] in result[j].
-
- result and f must be allocated prior to calling this function.
- */
-static
-void variance_calculation (struct casereader *ir, const struct variable *var,
- const struct dictionary *dict,
- func_f **f, double *result, size_t n)
-{
- int i;
- struct casereader *r = casereader_clone (ir);
- struct ccase *c;
- const struct variable *wv = dict_get_weight (dict);
- const int w_idx = wv ?
- var_get_case_index (wv) :
- caseproto_get_n_widths (casereader_get_proto (r)) ;
-
- r = sort_execute_1var (r, var);
-
- r = casereader_create_distinct (r, var, dict_get_weight (dict));
-
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = case_num_idx (c, w_idx);
-
- for (i = 0; i < n; ++i)
- result[i] += f[i] (w);
- }
-
- casereader_destroy (r);
-}
-
-struct jt
-{
- int levels;
- double n;
- double obs;
- double mean;
- double stddev;
-};
-
-static void show_jt (const struct n_sample_test *, const struct jt *,
- const struct fmt_spec *wfmt);
-
-
-void
-jonckheere_terpstra_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- int v;
- bool warn = true;
- const struct dictionary *dict = dataset_dict (ds);
- const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test, parent);
-
- struct caseproto *proto = caseproto_create ();
- proto = caseproto_add_width (proto, 0);
- proto = caseproto_add_width (proto, 0);
-
- /* If the independent variable is missing, then we ignore the case */
- input = casereader_create_filter_missing (input,
- &nst->indep_var, 1,
- exclude,
- NULL, NULL);
-
- /* Remove cases with invalid weigths */
- input = casereader_create_filter_weight (input, dict, &warn, NULL);
-
- /* Remove all those cases which are outside the range (val1, val2) */
- input = casereader_create_filter_func (input, include_func_bi, NULL,
- CONST_CAST (struct n_sample_test *, nst), NULL);
-
- /* Sort the data by the independent variable */
- input = sort_execute_1var (input, nst->indep_var);
-
- for (v = 0; v < nst->n_vars ; ++v)
- {
- struct jt jt;
- double variance;
- int g0;
- double nn = 0;
- int i;
- double sums[3] = {0,0,0};
- double e_sum[3] = {0,0,0};
-
- struct group_data *grp = NULL;
- double ccsq_sum = 0;
-
- struct casegrouper *grouper;
- struct casereader *group;
- struct casereader *vreader= casereader_clone (input);
-
- /* Get a few values into e_sum - we'll be needing these later */
- variance_calculation (vreader, nst->vars[v], dict, mff, e_sum, 3);
-
- grouper =
- casegrouper_create_vars (vreader, &nst->indep_var, 1);
-
- jt.obs = 0;
- jt.levels = 0;
- jt.n = 0;
- for (; casegrouper_get_next_group (grouper, &group);
- casereader_destroy (group))
- {
- struct casewriter *writer = autopaging_writer_create (proto);
- struct ccase *c;
- double cc = 0;
-
- group = sort_execute_1var (group, nst->vars[v]);
- for (; (c = casereader_read (group)); case_unref (c))
- {
- struct ccase *c_out = case_create (proto);
-
- *case_num_rw_idx (c_out, 0) = case_num (c, nst->vars[v]);
-
- cc += dict_get_case_weight (dict, c, &warn);
- *case_num_rw_idx (c_out, 1) = cc;
- casewriter_write (writer, c_out);
- }
-
- grp = xrealloc (grp, sizeof *grp * (jt.levels + 1));
-
- grp[jt.levels].reader = casewriter_make_reader (writer);
- grp[jt.levels].cc = cc;
-
- jt.levels++;
- jt.n += cc;
- ccsq_sum += pow2 (cc);
- }
-
- casegrouper_destroy (grouper);
-
- for (g0 = 0; g0 < jt.levels; ++g0)
- {
- int g1;
- for (g1 = g0 +1 ; g1 < jt.levels; ++g1)
- {
- double uu = u (&grp[g0], &grp[g1]);
- jt.obs += uu;
- }
- nn += pow2 (grp[g0].cc) * (2 * grp[g0].cc + 3);
-
- for (i = 0; i < 3; ++i)
- sums[i] += mff[i] (grp[g0].cc);
-
- casereader_destroy (grp[g0].reader);
- }
-
- free (grp);
-
- variance = (mff[0](jt.n) - sums[0] - e_sum[0]) / 72.0;
- variance += sums[1] * e_sum[1] / (36.0 * mff[1] (jt.n));
- variance += sums[2] * e_sum[2] / (8.0 * mff[2] (jt.n));
-
- jt.stddev = sqrt (variance);
-
- jt.mean = (pow2 (jt.n) - ccsq_sum) / 4.0;
-
- show_jt (nst, &jt, dict_get_weight_format (dict));
- }
-
- casereader_destroy (input);
- caseproto_unref (proto);
-}
-\f
-static void
-show_jt (const struct n_sample_test *nst, const struct jt *jt,
- const struct fmt_spec *wfmt)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Jonckheere-Terpstra Test"));
- pivot_table_set_weight_format (table, wfmt);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- pivot_category_create_leaf_rc (
- statistics->root,
- pivot_value_new_text_format (N_("Number of levels in %s"),
- var_to_string (nst->indep_var)),
- PIVOT_RC_INTEGER);
- pivot_category_create_leaves (
- statistics->root,
- N_("N"), PIVOT_RC_COUNT,
- N_("Observed J-T Statistic"), PIVOT_RC_OTHER,
- N_("Mean J-T Statistic"), PIVOT_RC_OTHER,
- N_("Std. Deviation of J-T Statistic"), PIVOT_RC_OTHER,
- N_("Std. J-T Statistic"), PIVOT_RC_OTHER,
- N_("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t i = 0; i < nst->n_vars; ++i)
- {
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (nst->vars[i]));
-
- double std_jt = (jt[0].obs - jt[0].mean) / jt[0].stddev;
- double sig = (2.0 * (std_jt > 0
- ? gsl_cdf_ugaussian_Q (std_jt)
- : gsl_cdf_ugaussian_P (std_jt)));
- double entries[] = {
- jt[0].levels,
- jt[0].n,
- jt[0].obs,
- jt[0].mean,
- jt[0].stddev,
- std_jt,
- sig,
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, row, pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !jonckheere_terpstra_h
-#define jonckheere_terpstra_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "data/case.h"
-#include "language/stats/npar.h"
-
-struct jonckheere_terpstra_test
-{
- struct two_sample_test parent;
-};
-
-struct casereader;
-struct dataset;
-
-void jonckheere_terpstra_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer
- );
-
-#endif
+++ /dev/null
-/* Pspp - a program for statistical analysis.
- Copyright (C) 2010, 2011, 2022 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-
-#include <config.h>
-
-#include "kruskal-wallis.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "libpspp/assertion.h"
-#include "libpspp/hmap.h"
-#include "libpspp/bt.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/sort.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-/* Returns true iff the independent variable lies between nst->val1 and nst->val2 */
-static bool
-include_func (const struct ccase *c, void *aux)
-{
- const struct n_sample_test *nst = aux;
-
- const union value *smaller = 0;
- const union value *larger = 0;
- int x = value_compare_3way (&nst->val1, &nst->val2, var_get_width (nst->indep_var));
- if (x < 0)
- {
- smaller = &nst->val1;
- larger = &nst->val2;
- }
- else
- {
- smaller = &nst->val2;
- larger = &nst->val1;
- }
-
- if (0 < value_compare_3way (smaller, case_data (c, nst->indep_var),
- var_get_width (nst->indep_var)))
- return false;
-
- if (0 > value_compare_3way (larger, case_data (c, nst->indep_var),
- var_get_width (nst->indep_var)))
- return false;
-
- return true;
-}
-
-
-struct rank_entry
-{
- struct hmap_node node;
- struct bt_node btn;
- union value group;
-
- double sum_of_ranks;
- double n;
-};
-
-
-static int
-compare_rank_entries_3way (const struct bt_node *a,
- const struct bt_node *b,
- const void *aux)
-{
- const struct variable *var = aux;
- const struct rank_entry *rea = BT_DATA (a, struct rank_entry, btn);
- const struct rank_entry *reb = BT_DATA (b, struct rank_entry, btn);
-
- return value_compare_3way (&rea->group, &reb->group, var_get_width (var));
-}
-
-
-/* Return the entry with the key GROUP or null if there is no such entry */
-static struct rank_entry *
-find_rank_entry (const struct hmap *map, const union value *group, size_t width)
-{
- struct rank_entry *re = NULL;
- size_t hash = value_hash (group, width, 0);
-
- HMAP_FOR_EACH_WITH_HASH (re, struct rank_entry, node, hash, map)
- {
- if (0 == value_compare_3way (group, &re->group, width))
- return re;
- }
-
- return re;
-}
-
-/* Calculates the adjustment necessary for tie compensation */
-static void
-distinct_callback (double v UNUSED, casenumber t, double w UNUSED, void *aux)
-{
- double *tiebreaker = aux;
-
- *tiebreaker += pow3 (t) - t;
-}
-
-
-struct kw
-{
- struct hmap map;
- double h;
-};
-
-static void show_ranks_box (const struct n_sample_test *, const struct kw *);
-static void show_sig_box (const struct n_sample_test *, const struct kw *);
-
-void
-kruskal_wallis_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- int i;
- struct ccase *c;
- bool warn = true;
- const struct dictionary *dict = dataset_dict (ds);
- const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test, parent);
- const struct caseproto *proto ;
- size_t rank_idx ;
-
- int total_n_groups = 0.0;
-
- struct kw *kw = XCALLOC (nst->n_vars, struct kw);
-
- /* If the independent variable is missing, then we ignore the case */
- input = casereader_create_filter_missing (input,
- &nst->indep_var, 1,
- exclude,
- NULL, NULL);
-
- input = casereader_create_filter_weight (input, dict, &warn, NULL);
-
- /* Remove all those cases which are outside the range (val1, val2) */
- input = casereader_create_filter_func (input, include_func, NULL,
- CONST_CAST (struct n_sample_test *, nst), NULL);
-
- proto = casereader_get_proto (input);
- rank_idx = caseproto_get_n_widths (proto);
-
- /* Rank cases by the v value */
- for (i = 0; i < nst->n_vars; ++i)
- {
- double tiebreaker = 0.0;
- bool warn = true;
- enum rank_error rerr = 0;
- struct casereader *rr;
- struct casereader *r = casereader_clone (input);
-
- r = sort_execute_1var (r, nst->vars[i]);
-
- /* Ignore missings in the test variable */
- r = casereader_create_filter_missing (r, &nst->vars[i], 1,
- exclude,
- NULL, NULL);
-
- rr = casereader_create_append_rank (r,
- nst->vars[i],
- dict_get_weight (dict),
- &rerr,
- distinct_callback, &tiebreaker);
-
- hmap_init (&kw[i].map);
- for (; (c = casereader_read (rr)); case_unref (c))
- {
- const union value *group = case_data (c, nst->indep_var);
- const size_t group_var_width = var_get_width (nst->indep_var);
- struct rank_entry *rank = find_rank_entry (&kw[i].map, group, group_var_width);
-
- if (NULL == rank)
- {
- rank = xzalloc (sizeof *rank);
- value_clone (&rank->group, group, group_var_width);
-
- hmap_insert (&kw[i].map, &rank->node,
- value_hash (&rank->group, group_var_width, 0));
- }
-
- rank->sum_of_ranks += case_num_idx (c, rank_idx);
- rank->n += dict_get_case_weight (dict, c, &warn);
-
- /* If this assertion fires, then either the data wasn't sorted or some other
- problem occurred */
- assert (rerr == 0);
- }
-
- casereader_destroy (rr);
-
- /* Calculate the value of h */
- {
- struct rank_entry *mre;
- double n = 0.0;
-
- HMAP_FOR_EACH (mre, struct rank_entry, node, &kw[i].map)
- {
- kw[i].h += pow2 (mre->sum_of_ranks) / mre->n;
- n += mre->n;
-
- total_n_groups ++;
- }
- kw[i].h *= 12 / (n * (n + 1));
- kw[i].h -= 3 * (n + 1) ;
-
- kw[i].h /= 1 - tiebreaker/ (pow3 (n) - n);
- }
- }
-
- casereader_destroy (input);
-
- show_ranks_box (nst, kw);
- show_sig_box (nst, kw);
-
- /* Cleanup allocated memory */
- for (i = 0 ; i < nst->n_vars; ++i)
- {
- struct rank_entry *mre, *next;
- HMAP_FOR_EACH_SAFE (mre, next, struct rank_entry, node, &kw[i].map)
- {
- hmap_delete (&kw[i].map, &mre->node);
- free (mre);
- }
- hmap_destroy (&kw[i].map);
- }
-
- free (kw);
-}
-\f
-static void
-show_ranks_box (const struct n_sample_test *nst, const struct kw *kw)
-{
- struct pivot_table *table = pivot_table_create (N_("Ranks"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_INTEGER,
- N_("Mean Rank"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0 ; i < nst->n_vars ; ++i)
- {
- /* Sort the rank entries, by iteratin the hash and putting the entries
- into a binary tree. */
- struct bt bt = BT_INITIALIZER(compare_rank_entries_3way, nst->vars[i]);
- struct rank_entry *re_x;
- HMAP_FOR_EACH (re_x, struct rank_entry, node, &kw[i].map)
- bt_insert (&bt, &re_x->btn);
-
- /* Report the rank entries in sorted order. */
- struct pivot_category *group = pivot_category_create_group__ (
- variables->root, pivot_value_new_variable (nst->vars[i]));
- int tot = 0;
- const struct rank_entry *re;
- BT_FOR_EACH (re, struct rank_entry, btn, &bt)
- {
- struct string str = DS_EMPTY_INITIALIZER;
- var_append_value_name (nst->indep_var, &re->group, &str);
- int row = pivot_category_create_leaf (
- group, pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
-
- double entries[] = { re->n, re->sum_of_ranks / re->n };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, row,
- pivot_value_new_number (entries[j]));
-
- tot += re->n;
- }
-
- int row = pivot_category_create_leaves (group, N_("Total"));
- pivot_table_put2 (table, 0, row, pivot_value_new_number (tot));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_sig_box (const struct n_sample_test *nst, const struct kw *kw)
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Chi-Square"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Variables"));
-
- for (size_t i = 0 ; i < nst->n_vars; ++i)
- {
- int col = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (nst->vars[i]));
-
- double df = hmap_count (&kw[i].map) - 1;
- double sig = gsl_cdf_chisq_Q (kw[i].h, df);
- double entries[] = { kw[i].h, df, sig };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, col, pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011, 2022 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !kruskal_wallis_h
-#define kruskal_wallis_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "data/case.h"
-#include "language/stats/npar.h"
-
-struct kruskal_wallis_test
-{
- struct n_sample_test parent;
-};
-
-struct casereader;
-struct dataset;
-
-void kruskal_wallis_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer
- );
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/ks-one-sample.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-#include <stdlib.h>
-
-
-#include "math/sort.h"
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value-labels.h"
-#include "data/variable.h"
-#include "language/stats/freq.h"
-#include "language/stats/npar.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/compiler.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-/* The per test variable statistics */
-struct ks
-{
- double obs_cc;
-
- double test_min ;
- double test_max;
- double mu;
- double sigma;
-
- double diff_pos;
- double diff_neg;
-
- double ssq;
- double sum;
-};
-
-typedef double theoretical (const struct ks *ks, double x);
-typedef theoretical *theoreticalfp;
-
-static double
-theoretical_uniform (const struct ks *ks, double x)
-{
- return gsl_cdf_flat_P (x, ks->test_min, ks->test_max);
-}
-
-static double
-theoretical_normal (const struct ks *ks, double x)
-{
- return gsl_cdf_gaussian_P (x - ks->mu, ks->sigma);
-}
-
-static double
-theoretical_poisson (const struct ks *ks, double x)
-{
- return gsl_cdf_poisson_P (x, ks->mu);
-}
-
-static double
-theoretical_exponential (const struct ks *ks, double x)
-{
- return gsl_cdf_exponential_P (x, 1/ks->mu);
-}
-
-
-static const theoreticalfp theoreticalf[4] =
-{
- theoretical_normal,
- theoretical_uniform,
- theoretical_poisson,
- theoretical_exponential
-};
-
-/*
- Return the assymptotic approximation to the significance of Z
- */
-static double
-ks_asymp_sig (double z)
-{
- if (z < 0.27)
- return 1;
-
- if (z >= 3.1)
- return 0;
-
- if (z < 1)
- {
- double q = exp (-1.233701 * pow (z, -2));
- return 1 - 2.506628 * (q + pow (q, 9) + pow (q, 25))/ z ;
- }
- else
- {
- double q = exp (-2 * z * z);
- return 2 * (q - pow (q, 4) + pow (q, 9) - pow (q, 16))/ z ;
- }
-}
-
-static void show_results (const struct ks *, const struct ks_one_sample_test *, const struct fmt_spec *);
-
-
-void
-ks_one_sample_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool x UNUSED, double y UNUSED)
-{
- const struct dictionary *dict = dataset_dict (ds);
- const struct ks_one_sample_test *kst = UP_CAST (test, const struct ks_one_sample_test, parent.parent);
- const struct one_sample_test *ost = &kst->parent;
- struct ccase *c;
- const struct fmt_spec *wfmt = dict_get_weight_format (dict);
- bool warn = true;
- int v;
- struct casereader *r = casereader_clone (input);
-
- struct ks *ks = XCALLOC (ost->n_vars, struct ks);
-
- for (v = 0; v < ost->n_vars; ++v)
- {
- ks[v].obs_cc = 0;
- ks[v].test_min = DBL_MAX;
- ks[v].test_max = -DBL_MAX;
- ks[v].diff_pos = -DBL_MAX;
- ks[v].diff_neg = DBL_MAX;
- ks[v].sum = 0;
- ks[v].ssq = 0;
- }
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- const double weight = dict_get_case_weight (dict, c, &warn);
-
- for (v = 0; v < ost->n_vars; ++v)
- {
- const struct variable *var = ost->vars[v];
- const union value *val = case_data (c, var);
-
- if (var_is_value_missing (var, val) & exclude)
- continue;
-
- minimize (&ks[v].test_min, val->f);
- maximize (&ks[v].test_max, val->f);
-
- ks[v].obs_cc += weight;
- ks[v].sum += val->f;
- ks[v].ssq += pow2 (val->f);
- }
- }
- casereader_destroy (r);
-
- for (v = 0; v < ost->n_vars; ++v)
- {
- const struct variable *var = ost->vars[v];
- double cc = 0;
- double prev_empirical = 0;
-
- switch (kst->dist)
- {
- case KS_UNIFORM:
- if (kst->p[0] != SYSMIS)
- ks[v].test_min = kst->p[0];
-
- if (kst->p[1] != SYSMIS)
- ks[v].test_max = kst->p[1];
- break;
- case KS_NORMAL:
- if (kst->p[0] != SYSMIS)
- ks[v].mu = kst->p[0];
- else
- ks[v].mu = ks[v].sum / ks[v].obs_cc;
-
- if (kst->p[1] != SYSMIS)
- ks[v].sigma = kst->p[1];
- else
- {
- ks[v].sigma = ks[v].ssq - pow2 (ks[v].sum) / ks[v].obs_cc;
- ks[v].sigma /= ks[v].obs_cc - 1;
- ks[v].sigma = sqrt (ks[v].sigma);
- }
-
- break;
- case KS_POISSON:
- case KS_EXPONENTIAL:
- if (kst->p[0] != SYSMIS)
- ks[v].mu = ks[v].sigma = kst->p[0];
- else
- ks[v].mu = ks[v].sigma = ks[v].sum / ks[v].obs_cc;
- break;
- default:
- NOT_REACHED ();
- }
-
- r = sort_execute_1var (casereader_clone (input), var);
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- double theoretical, empirical;
- double d, dp;
- const double weight = dict_get_case_weight (dict, c, &warn);
- const union value *val = case_data (c, var);
-
- if (var_is_value_missing (var, val) & exclude)
- continue;
-
- cc += weight;
-
- empirical = cc / ks[v].obs_cc;
-
- theoretical = theoreticalf[kst->dist] (&ks[v], val->f);
-
- d = empirical - theoretical;
- dp = prev_empirical - theoretical;
-
- if (d > 0)
- maximize (&ks[v].diff_pos, d);
- else
- minimize (&ks[v].diff_neg, d);
-
- if (dp > 0)
- maximize (&ks[v].diff_pos, dp);
- else
- minimize (&ks[v].diff_neg, dp);
-
- prev_empirical = empirical;
- }
-
- casereader_destroy (r);
- }
-
- show_results (ks, kst, wfmt);
-
- free (ks);
- casereader_destroy (input);
-}
-
-
-static void
-show_results (const struct ks *ks,
- const struct ks_one_sample_test *kst,
- const struct fmt_spec *wfmt)
-{
- struct pivot_table *table = pivot_table_create (
- N_("One-Sample Kolmogorov-Smirnov Test"));
- pivot_table_set_weight_format (table, wfmt);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT);
-
- switch (kst->dist)
- {
- case KS_UNIFORM:
- pivot_category_create_group (statistics->root, N_("Uniform Parameters"),
- N_("Minimum"), N_("Maximum"));
- break;
-
- case KS_NORMAL:
- pivot_category_create_group (statistics->root, N_("Normal Parameters"),
- N_("Mean"), N_("Std. Deviation"));
- break;
-
- case KS_POISSON:
- pivot_category_create_group (statistics->root, N_("Poisson Parameters"),
- N_("Lambda"));
- break;
-
- case KS_EXPONENTIAL:
- pivot_category_create_group (statistics->root,
- N_("Exponential Parameters"), N_("Scale"));
- break;
-
- default:
- NOT_REACHED ();
- }
-
- pivot_category_create_group (
- statistics->root, N_("Most Extreme Differences"),
- N_("Absolute"), N_("Positive"), N_("Negative"));
-
- pivot_category_create_leaves (
- statistics->root, N_("Kolmogorov-Smirnov Z"),
- _("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Variables"));
-
- for (size_t i = 0; i < kst->parent.n_vars; ++i)
- {
- int col = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (kst->parent.vars[i]));
-
- double values[10];
- size_t n = 0;
-
- values[n++] = ks[i].obs_cc;
-
- switch (kst->dist)
- {
- case KS_UNIFORM:
- values[n++] = ks[i].test_min;
- values[n++] = ks[i].test_max;
- break;
-
- case KS_NORMAL:
- values[n++] = ks[i].mu;
- values[n++] = ks[i].sigma;
- break;
-
- case KS_POISSON:
- case KS_EXPONENTIAL:
- values[n++] = ks[i].mu;
- break;
-
- default:
- NOT_REACHED ();
- }
-
- double abs = ks[i].diff_pos;
- maximize (&abs, -ks[i].diff_neg);
-
- double z = sqrt (ks[i].obs_cc) * abs;
-
- values[n++] = abs;
- values[n++] = ks[i].diff_pos;
- values[n++] = ks[i].diff_neg;
- values[n++] = z;
- values[n++] = ks_asymp_sig (z);
-
- for (size_t j = 0; j < n; j++)
- pivot_table_put2 (table, j, col, pivot_value_new_number (values[j]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !ks_one_sample_h
-#define ks_one_sample_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-enum dist
- {
- KS_NORMAL,
- KS_UNIFORM,
- KS_POISSON,
- KS_EXPONENTIAL
- };
-
-struct ks_one_sample_test
-{
- struct one_sample_test parent;
-
- double p[2];
- enum dist dist;
-};
-
-struct casereader;
-struct dataset;
-
-
-void ks_one_sample_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool, double);
-
-#endif
+++ /dev/null
-/* pspp - a program for statistical analysis.
- Copyright (C) 2012 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-
-/*
- References:
- 1. "Coding Logistic Regression with Newton-Raphson", James McCaffrey
- http://msdn.microsoft.com/en-us/magazine/jj618304.aspx
-
- 2. "SPSS Statistical Algorithms" Chapter LOGISTIC REGRESSION Algorithms
-
-
- The Newton Raphson method finds successive approximations to $\bf b$ where
- approximation ${\bf b}_t$ is (hopefully) better than the previous ${\bf b}_{t-1}$.
-
- $ {\bf b}_t = {\bf b}_{t -1} + ({\bf X}^T{\bf W}_{t-1}{\bf X})^{-1}{\bf X}^T({\bf y} - {\bf \pi}_{t-1})$
- where:
-
- $\bf X$ is the $n \times p$ design matrix, $n$ being the number of cases,
- $p$ the number of parameters, \par
- $\bf W$ is the diagonal matrix whose diagonal elements are
- $\hat{\pi}_0(1 - \hat{\pi}_0), \, \hat{\pi}_1(1 - \hat{\pi}_2)\dots \hat{\pi}_{n-1}(1 - \hat{\pi}_{n-1})$
- \par
-
-*/
-
-#include <config.h>
-
-#include <gsl/gsl_blas.h>
-
-#include <gsl/gsl_linalg.h>
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_matrix.h>
-#include <gsl/gsl_vector.h>
-#include <math.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmap.h"
-#include "libpspp/ll.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/categoricals.h"
-#include "math/interaction.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-
-
-#define PRINT_EACH_STEP 0x01
-#define PRINT_SUMMARY 0x02
-#define PRINT_CORR 0x04
-#define PRINT_ITER 0x08
-#define PRINT_GOODFIT 0x10
-#define PRINT_CI 0x20
-
-
-#define PRINT_DEFAULT (PRINT_SUMMARY | PRINT_EACH_STEP)
-
-/*
- The constant parameters of the procedure.
- That is, those which are set by the user.
-*/
-struct lr_spec
-{
- /* The dependent variable */
- const struct variable *dep_var;
-
- /* The predictor variables (excluding categorical ones) */
- const struct variable **predictor_vars;
- size_t n_predictor_vars;
-
- /* The categorical predictors */
- struct interaction **cat_predictors;
- size_t n_cat_predictors;
-
-
- /* The union of the categorical and non-categorical variables */
- const struct variable **indep_vars;
- size_t n_indep_vars;
-
-
- /* Which classes of missing vars are to be excluded */
- enum mv_class exclude;
-
- /* The weight variable */
- const struct variable *wv;
-
- /* The dictionary of the dataset */
- const struct dictionary *dict;
-
- /* True iff the constant (intercept) is to be included in the model */
- bool constant;
-
- /* Ths maximum number of iterations */
- int max_iter;
-
- /* Other iteration limiting conditions */
- double bcon;
- double min_epsilon;
- double lcon;
-
- /* The confidence interval (in percent) */
- int confidence;
-
- /* What results should be presented */
- unsigned int print;
-
- /* Inverse logit of the cut point */
- double ilogit_cut_point;
-};
-
-
-/* The results and intermediate result of the procedure.
- These are mutated as the procedure runs. Used for
- temporary variables etc.
-*/
-struct lr_result
-{
- /* Used to indicate if a pass should flag a warning when
- invalid (ie negative or missing) weight values are encountered */
- bool warn_bad_weight;
-
- /* The two values of the dependent variable. */
- union value y0;
- union value y1;
-
-
- /* The sum of caseweights */
- double cc;
-
- /* The number of missing and nonmissing cases */
- casenumber n_missing;
- casenumber n_nonmissing;
-
-
- gsl_matrix *hessian;
-
- /* The categoricals and their payload. Null if the analysis has no
- categorical predictors */
- struct categoricals *cats;
- struct payload cp;
-
-
- /* The estimates of the predictor coefficients */
- gsl_vector *beta_hat;
-
- /* The predicted classifications:
- True Negative, True Positive, False Negative, False Positive */
- double tn, tp, fn, fp;
-};
-
-
-/*
- Convert INPUT into a dichotomous scalar, according to how the dependent variable's
- values are mapped.
- For simple cases, this is a 1:1 mapping
- The return value is always either 0 or 1
-*/
-static double
-map_dependent_var (const struct lr_spec *cmd, const struct lr_result *res, const union value *input)
-{
- const int width = var_get_width (cmd->dep_var);
- if (value_equal (input, &res->y0, width))
- return 0;
-
- if (value_equal (input, &res->y1, width))
- return 1;
-
- /* This should never happen. If it does, then y0 and/or y1 have probably not been set */
- NOT_REACHED ();
-
- return SYSMIS;
-}
-
-static void output_classification_table (const struct lr_spec *cmd, const struct lr_result *res);
-
-static void output_categories (const struct lr_spec *cmd, const struct lr_result *res);
-
-static void output_depvarmap (const struct lr_spec *cmd, const struct lr_result *);
-
-static void output_variables (const struct lr_spec *cmd,
- const struct lr_result *);
-
-static void output_model_summary (const struct lr_result *,
- double initial_likelihood, double likelihood);
-
-static void case_processing_summary (const struct lr_result *);
-
-
-/* Return the value of case C corresponding to the INDEX'th entry in the
- model */
-static double
-predictor_value (const struct ccase *c,
- const struct variable **x, size_t n_x,
- const struct categoricals *cats,
- size_t index)
-{
- /* Values of the scalar predictor variables */
- if (index < n_x)
- return case_num (c, x[index]);
-
- /* Coded values of categorical predictor variables (or interactions) */
- if (cats && index - n_x < categoricals_df_total (cats))
- {
- double x = categoricals_get_dummy_code_for_case (cats, index - n_x, c);
- return x;
- }
-
- /* The constant term */
- return 1.0;
-}
-
-
-/*
- Return the probability beta_hat (that is the estimator logit(y))
- corresponding to the coefficient estimator for case C
-*/
-static double
-pi_hat (const struct lr_spec *cmd,
- const struct lr_result *res,
- const struct variable **x, size_t n_x,
- const struct ccase *c)
-{
- int v0;
- double pi = 0;
- size_t n_coeffs = res->beta_hat->size;
-
- if (cmd->constant)
- {
- pi += gsl_vector_get (res->beta_hat, res->beta_hat->size - 1);
- n_coeffs--;
- }
-
- for (v0 = 0; v0 < n_coeffs; ++v0)
- {
- pi += gsl_vector_get (res->beta_hat, v0) *
- predictor_value (c, x, n_x, res->cats, v0);
- }
-
- pi = 1.0 / (1.0 + exp(-pi));
-
- return pi;
-}
-
-
-/*
- Calculates the Hessian matrix X' V X,
- where: X is the n by N_X matrix comprising the n cases in INPUT
- V is a diagonal matrix { (pi_hat_0)(1 - pi_hat_0), (pi_hat_1)(1 - pi_hat_1), ... (pi_hat_{N-1})(1 - pi_hat_{N-1})}
- (the partial derivative of the predicted values)
-
- If ALL predicted values derivatives are close to zero or one, then CONVERGED
- will be set to true.
-*/
-static void
-hessian (const struct lr_spec *cmd,
- struct lr_result *res,
- struct casereader *input,
- const struct variable **x, size_t n_x,
- bool *converged)
-{
- struct casereader *reader;
- struct ccase *c;
-
- double max_w = -DBL_MAX;
-
- gsl_matrix_set_zero (res->hessian);
-
- for (reader = casereader_clone (input);
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- int v0, v1;
- double pi = pi_hat (cmd, res, x, n_x, c);
-
- double weight = dict_get_case_weight (cmd->dict, c, &res->warn_bad_weight);
- double w = pi * (1 - pi);
- if (w > max_w)
- max_w = w;
- w *= weight;
-
- for (v0 = 0; v0 < res->beta_hat->size; ++v0)
- {
- double in0 = predictor_value (c, x, n_x, res->cats, v0);
- for (v1 = 0; v1 < res->beta_hat->size; ++v1)
- {
- double in1 = predictor_value (c, x, n_x, res->cats, v1);
- double *o = gsl_matrix_ptr (res->hessian, v0, v1);
- *o += in0 * w * in1;
- }
- }
- }
- casereader_destroy (reader);
-
- if (max_w < cmd->min_epsilon)
- {
- *converged = true;
- msg (MN, _("All predicted values are either 1 or 0"));
- }
-}
-
-
-/* Calculates the value X' (y - pi)
- where X is the design model,
- y is the vector of observed independent variables
- pi is the vector of estimates for y
-
- Side effects:
- the likelihood is stored in LIKELIHOOD;
- the predicted values are placed in the respective tn, fn, tp fp values in RES
-*/
-static gsl_vector *
-xt_times_y_pi (const struct lr_spec *cmd,
- struct lr_result *res,
- struct casereader *input,
- const struct variable **x, size_t n_x,
- const struct variable *y_var,
- double *llikelihood)
-{
- struct casereader *reader;
- struct ccase *c;
- gsl_vector *output = gsl_vector_calloc (res->beta_hat->size);
-
- *llikelihood = 0.0;
- res->tn = res->tp = res->fn = res->fp = 0;
- for (reader = casereader_clone (input);
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- double pred_y = 0;
- int v0;
- double pi = pi_hat (cmd, res, x, n_x, c);
- double weight = dict_get_case_weight (cmd->dict, c, &res->warn_bad_weight);
-
-
- double y = map_dependent_var (cmd, res, case_data (c, y_var));
-
- *llikelihood += (weight * y) * log (pi) + log (1 - pi) * weight * (1 - y);
-
- for (v0 = 0; v0 < res->beta_hat->size; ++v0)
- {
- double in0 = predictor_value (c, x, n_x, res->cats, v0);
- double *o = gsl_vector_ptr (output, v0);
- *o += in0 * (y - pi) * weight;
- pred_y += gsl_vector_get (res->beta_hat, v0) * in0;
- }
-
- /* Count the number of cases which would be correctly/incorrectly classified by this
- estimated model */
- if (pred_y <= cmd->ilogit_cut_point)
- {
- if (y == 0)
- res->tn += weight;
- else
- res->fn += weight;
- }
- else
- {
- if (y == 0)
- res->fp += weight;
- else
- res->tp += weight;
- }
- }
-
- casereader_destroy (reader);
-
- return output;
-}
-
-\f
-
-/* "payload" functions for the categoricals.
- The only function is to accumulate the frequency of each
- category.
- */
-
-static void *
-frq_create (const void *aux1 UNUSED, void *aux2 UNUSED)
-{
- return xzalloc (sizeof (double));
-}
-
-static void
-frq_update (const void *aux1 UNUSED, void *aux2 UNUSED,
- void *ud, const struct ccase *c UNUSED , double weight)
-{
- double *freq = ud;
- *freq += weight;
-}
-
-static void
-frq_destroy (const void *aux1 UNUSED, void *aux2 UNUSED, void *user_data)
-{
- free (user_data);
-}
-
-\f
-
-/*
- Makes an initial pass though the data, doing the following:
-
- * Checks that the dependent variable is dichotomous,
- * Creates and initialises the categoricals,
- * Accumulates summary results,
- * Calculates necessary initial values.
- * Creates an initial value for \hat\beta the vector of beta_hats of \beta
-
- Returns true if successful
-*/
-static bool
-initial_pass (const struct lr_spec *cmd, struct lr_result *res, struct casereader *input)
-{
- const int width = var_get_width (cmd->dep_var);
-
- struct ccase *c;
- struct casereader *reader;
-
- double sum;
- double sumA = 0.0;
- double sumB = 0.0;
-
- bool v0set = false;
- bool v1set = false;
-
- size_t n_coefficients = cmd->n_predictor_vars;
- if (cmd->constant)
- n_coefficients++;
-
- /* Create categoricals if appropriate */
- if (cmd->n_cat_predictors > 0)
- {
- res->cp.create = frq_create;
- res->cp.update = frq_update;
- res->cp.calculate = NULL;
- res->cp.destroy = frq_destroy;
-
- res->cats = categoricals_create (cmd->cat_predictors, cmd->n_cat_predictors,
- cmd->wv, MV_ANY);
-
- categoricals_set_payload (res->cats, &res->cp, cmd, res);
- }
-
- res->cc = 0;
- for (reader = casereader_clone (input);
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- int v;
- bool missing = false;
- double weight = dict_get_case_weight (cmd->dict, c, &res->warn_bad_weight);
- const union value *depval = case_data (c, cmd->dep_var);
-
- if (var_is_value_missing (cmd->dep_var, depval) & cmd->exclude)
- {
- missing = true;
- }
- else
- for (v = 0; v < cmd->n_indep_vars; ++v)
- {
- const union value *val = case_data (c, cmd->indep_vars[v]);
- if (var_is_value_missing (cmd->indep_vars[v], val) & cmd->exclude)
- {
- missing = true;
- break;
- }
- }
-
- /* Accumulate the missing and non-missing counts */
- if (missing)
- {
- res->n_missing++;
- continue;
- }
- res->n_nonmissing++;
-
- /* Find the values of the dependent variable */
- if (!v0set)
- {
- value_clone (&res->y0, depval, width);
- v0set = true;
- }
- else if (!v1set)
- {
- if (!value_equal (&res->y0, depval, width))
- {
- value_clone (&res->y1, depval, width);
- v1set = true;
- }
- }
- else
- {
- if (!value_equal (&res->y0, depval, width)
- &&
- !value_equal (&res->y1, depval, width)
- )
- {
- msg (ME, _("Dependent variable's values are not dichotomous."));
- case_unref (c);
- goto error;
- }
- }
-
- if (v0set && value_equal (&res->y0, depval, width))
- sumA += weight;
-
- if (v1set && value_equal (&res->y1, depval, width))
- sumB += weight;
-
-
- res->cc += weight;
-
- categoricals_update (res->cats, c);
- }
- casereader_destroy (reader);
-
- categoricals_done (res->cats);
-
- sum = sumB;
-
- /* Ensure that Y0 is less than Y1. Otherwise the mapping gets
- inverted, which is confusing to users */
- if (var_is_numeric (cmd->dep_var) && value_compare_3way (&res->y0, &res->y1, width) > 0)
- {
- union value tmp;
- value_clone (&tmp, &res->y0, width);
- value_copy (&res->y0, &res->y1, width);
- value_copy (&res->y1, &tmp, width);
- value_destroy (&tmp, width);
- sum = sumA;
- }
-
- n_coefficients += categoricals_df_total (res->cats);
- res->beta_hat = gsl_vector_calloc (n_coefficients);
-
- if (cmd->constant)
- {
- double mean = sum / res->cc;
- gsl_vector_set (res->beta_hat, res->beta_hat->size - 1, log (mean / (1 - mean)));
- }
-
- return true;
-
- error:
- casereader_destroy (reader);
- return false;
-}
-
-
-
-/* Start of the logistic regression routine proper */
-static bool
-run_lr (const struct lr_spec *cmd, struct casereader *input,
- const struct dataset *ds UNUSED)
-{
- int i;
-
- bool converged = false;
-
- /* Set the log likelihoods to a sentinel value */
- double log_likelihood = SYSMIS;
- double prev_log_likelihood = SYSMIS;
- double initial_log_likelihood = SYSMIS;
-
- struct lr_result work;
- work.n_missing = 0;
- work.n_nonmissing = 0;
- work.warn_bad_weight = true;
- work.cats = NULL;
- work.beta_hat = NULL;
- work.hessian = NULL;
-
- /* Get the initial estimates of \beta and their standard errors.
- And perform other auxiliary initialisation. */
- if (!initial_pass (cmd, &work, input))
- goto error;
-
- for (i = 0; i < cmd->n_cat_predictors; ++i)
- {
- if (1 >= categoricals_n_count (work.cats, i))
- {
- struct string str;
- ds_init_empty (&str);
-
- interaction_to_string (cmd->cat_predictors[i], &str);
-
- msg (ME, _("Category %s does not have at least two distinct values. Logistic regression will not be run."),
- ds_cstr(&str));
- ds_destroy (&str);
- goto error;
- }
- }
-
- output_depvarmap (cmd, &work);
-
- case_processing_summary (&work);
-
-
- input = casereader_create_filter_missing (input,
- cmd->indep_vars,
- cmd->n_indep_vars,
- cmd->exclude,
- NULL,
- NULL);
-
- input = casereader_create_filter_missing (input,
- &cmd->dep_var,
- 1,
- cmd->exclude,
- NULL,
- NULL);
-
- work.hessian = gsl_matrix_calloc (work.beta_hat->size, work.beta_hat->size);
-
- /* Start the Newton Raphson iteration process... */
- for(i = 0; i < cmd->max_iter; ++i)
- {
- double min, max;
- gsl_vector *v;
-
-
- hessian (cmd, &work, input,
- cmd->predictor_vars, cmd->n_predictor_vars,
- &converged);
-
- gsl_linalg_cholesky_decomp (work.hessian);
- gsl_linalg_cholesky_invert (work.hessian);
-
- v = xt_times_y_pi (cmd, &work, input,
- cmd->predictor_vars, cmd->n_predictor_vars,
- cmd->dep_var,
- &log_likelihood);
-
- {
- /* delta = M.v */
- gsl_vector *delta = gsl_vector_alloc (v->size);
- gsl_blas_dgemv (CblasNoTrans, 1.0, work.hessian, v, 0, delta);
- gsl_vector_free (v);
-
-
- gsl_vector_add (work.beta_hat, delta);
-
- gsl_vector_minmax (delta, &min, &max);
-
- if (fabs (min) < cmd->bcon && fabs (max) < cmd->bcon)
- {
- msg (MN, _("Estimation terminated at iteration number %d because parameter estimates changed by less than %g"),
- i + 1, cmd->bcon);
- converged = true;
- }
-
- gsl_vector_free (delta);
- }
-
- if (i > 0)
- {
- if (-log_likelihood > -(1.0 - cmd->lcon) * prev_log_likelihood)
- {
- msg (MN, _("Estimation terminated at iteration number %d because Log Likelihood decreased by less than %g%%"), i + 1, 100 * cmd->lcon);
- converged = true;
- }
- }
- if (i == 0)
- initial_log_likelihood = log_likelihood;
- prev_log_likelihood = log_likelihood;
-
- if (converged)
- break;
- }
-
-
-
- if (!converged)
- msg (MW, _("Estimation terminated at iteration number %d because maximum iterations has been reached"), i);
-
-
- output_model_summary (&work, initial_log_likelihood, log_likelihood);
-
- if (work.cats)
- output_categories (cmd, &work);
-
- output_classification_table (cmd, &work);
- output_variables (cmd, &work);
-
- casereader_destroy (input);
- gsl_matrix_free (work.hessian);
- gsl_vector_free (work.beta_hat);
- categoricals_destroy (work.cats);
-
- return true;
-
- error:
- casereader_destroy (input);
- gsl_matrix_free (work.hessian);
- gsl_vector_free (work.beta_hat);
- categoricals_destroy (work.cats);
-
- return false;
-}
-
-struct variable_node
-{
- struct hmap_node node; /* Node in hash map. */
- const struct variable *var; /* The variable */
-};
-
-static struct variable_node *
-lookup_variable (const struct hmap *map, const struct variable *var, unsigned int hash)
-{
- struct variable_node *vn;
- HMAP_FOR_EACH_WITH_HASH (vn, struct variable_node, node, hash, map)
- if (vn->var == var)
- return vn;
-
- return NULL;
-}
-
-static void
-insert_variable (struct hmap *map, const struct variable *var, unsigned int hash)
-{
- if (!lookup_variable (map, var, hash))
- {
- struct variable_node *vn = xmalloc (sizeof *vn);
- *vn = (struct variable_node) { .var = var };
- hmap_insert (map, &vn->node, hash);
- }
-}
-
-/* Parse the LOGISTIC REGRESSION command syntax */
-int
-cmd_logistic (struct lexer *lexer, struct dataset *ds)
-{
- /* Temporary location for the predictor variables.
- These may or may not include the categorical predictors */
- const struct variable **pred_vars = NULL;
- size_t n_pred_vars = 0;
- double cp = 0.5;
-
- struct dictionary *dict = dataset_dict (ds);
- struct lr_spec lr = {
- .dict = dict,
- .exclude = MV_ANY,
- .wv = dict_get_weight (dict),
- .max_iter = 20,
- .lcon = 0.0000,
- .bcon = 0.001,
- .min_epsilon = 0.00000001,
- .constant = true,
- .confidence = 95,
- .print = PRINT_DEFAULT,
- };
-
- if (lex_match_id (lexer, "VARIABLES"))
- lex_match (lexer, T_EQUALS);
-
- lr.dep_var = parse_variable_const (lexer, lr.dict);
- if (!lr.dep_var)
- goto error;
-
- if (!lex_force_match (lexer, T_WITH))
- goto error;
-
- if (!parse_variables_const (lexer, lr.dict, &pred_vars, &n_pred_vars,
- PV_NO_DUPLICATE))
- goto error;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- lr.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- lr.exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "ORIGIN"))
- lr.constant = false;
- else if (lex_match_id (lexer, "NOORIGIN"))
- lr.constant = true;
- else if (lex_match_id (lexer, "NOCONST"))
- lr.constant = false;
- else if (lex_match_id (lexer, "EXTERNAL"))
- {
- /* This is for compatibility. It does nothing */
- }
- else if (lex_match_id (lexer, "CATEGORICAL"))
- {
- lex_match (lexer, T_EQUALS);
- struct variable **cats;
- size_t n_cats;
- if (!parse_variables (lexer, lr.dict, &cats, &n_cats, PV_NO_DUPLICATE))
- goto error;
-
- lr.cat_predictors = xrealloc (lr.cat_predictors,
- sizeof *lr.cat_predictors
- * (n_cats + lr.n_cat_predictors));
- for (size_t i = 0; i < n_cats; i++)
- lr.cat_predictors[lr.n_cat_predictors++] = interaction_create (cats[i]);
- free (cats);
- }
- else if (lex_match_id (lexer, "PRINT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "DEFAULT"))
- lr.print |= PRINT_DEFAULT;
- else if (lex_match_id (lexer, "SUMMARY"))
- lr.print |= PRINT_SUMMARY;
-#if 0
- else if (lex_match_id (lexer, "CORR"))
- lr.print |= PRINT_CORR;
- else if (lex_match_id (lexer, "ITER"))
- lr.print |= PRINT_ITER;
- else if (lex_match_id (lexer, "GOODFIT"))
- lr.print |= PRINT_GOODFIT;
-#endif
- else if (lex_match_id (lexer, "CI"))
- {
- lr.print |= PRINT_CI;
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- lr.confidence = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "ALL"))
- lr.print = ~0x0000;
- else
- {
- lex_error_expecting (lexer, "DEFAULT", "SUMMARY",
-#if 0
- "CORR", "ITER", "GOODFIT",
-#endif
- "CI", "ALL");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "BCON"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- lr.bcon = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "ITERATE"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "ITERATE", 0, INT_MAX))
- goto error;
- lr.max_iter = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "LCON"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- lr.lcon = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "EPS"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- lr.min_epsilon = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "CUT"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_closed (lexer, "CUT", 0, 1))
- goto error;
-
- cp = lex_number (lexer);
-
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "BCON", "ITERATE", "LCON", "EPS",
- "CUT");
- goto error;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "MISSING", "ORIGIN", "NOORIGIN",
- "NOCONST", "EXTERNAL", "CATEGORICAL",
- "PRINT", "CRITERIA");
- goto error;
- }
- }
-
- lr.ilogit_cut_point = - log (1/cp - 1);
-
- /* Copy the predictor variables from the temporary location into the
- final one, dropping any categorical variables which appear there.
- FIXME: This is O(NxM).
- */
- struct hmap allvars = HMAP_INITIALIZER (allvars);
- size_t allocated_predictor_vars = 0;
- for (size_t v = 0; v < n_pred_vars; ++v)
- {
- bool drop = false;
- const struct variable *var = pred_vars[v];
-
- unsigned int hash = hash_pointer (var, 0);
- insert_variable (&allvars, var, hash);
-
- for (size_t cv = 0; cv < lr.n_cat_predictors; ++cv)
- {
- const struct interaction *iact = lr.cat_predictors[cv];
- for (size_t iv = 0; iv < iact->n_vars; ++iv)
- {
- const struct variable *ivar = iact->vars[iv];
- unsigned int hash = hash_pointer (ivar, 0);
- insert_variable (&allvars, ivar, hash);
-
- if (var == ivar)
- drop = true;
- }
- }
-
- if (drop)
- continue;
-
- if (lr.n_predictor_vars >= allocated_predictor_vars)
- lr.predictor_vars = x2nrealloc (lr.predictor_vars,
- &allocated_predictor_vars,
- sizeof *lr.predictor_vars);
- lr.predictor_vars[lr.n_predictor_vars++] = var;
- }
-
- lr.n_indep_vars = hmap_count (&allvars);
- lr.indep_vars = xmalloc (lr.n_indep_vars * sizeof *lr.indep_vars);
-
- /* Interate over each variable and push it into the array */
- size_t x = 0;
- struct variable_node *vn, *next;
- HMAP_FOR_EACH_SAFE (vn, next, struct variable_node, node, &allvars)
- {
- lr.indep_vars[x++] = vn->var;
- hmap_delete (&allvars, &vn->node);
- free (vn);
- }
- assert (x == lr.n_indep_vars);
- hmap_destroy (&allvars);
-
- /* Run logistical regression for each split group. */
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), lr.dict);
- struct casereader *group;
- bool ok = true;
- while (casegrouper_get_next_group (grouper, &group))
- ok = run_lr (&lr, group, ds) && ok;
- ok = casegrouper_destroy (grouper) && ok;
- ok = proc_commit (ds) && ok;
-
- for (size_t i = 0; i < lr.n_cat_predictors; ++i)
- interaction_destroy (lr.cat_predictors[i]);
- free (lr.predictor_vars);
- free (lr.cat_predictors);
- free (lr.indep_vars);
- free (pred_vars);
-
- return CMD_SUCCESS;
-
- error:
- for (size_t i = 0; i < lr.n_cat_predictors; ++i)
- interaction_destroy (lr.cat_predictors[i]);
- free (lr.predictor_vars);
- free (lr.cat_predictors);
- free (lr.indep_vars);
- free (pred_vars);
-
- return CMD_FAILURE;
-}
-
-
-\f
-
-/* Show the Dependent Variable Encoding box.
- This indicates how the dependent variable
- is mapped to the internal zero/one values.
-*/
-static void
-output_depvarmap (const struct lr_spec *cmd, const struct lr_result *res)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Dependent Variable Encoding"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Mapping"),
- N_("Internal Value"));
-
- struct pivot_dimension *original = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Original Value"));
- original->root->show_label = true;
-
- for (int i = 0; i < 2; i++)
- {
- const union value *v = i ? &res->y1 : &res->y0;
- int orig_idx = pivot_category_create_leaf (
- original->root, pivot_value_new_var_value (cmd->dep_var, v));
- pivot_table_put2 (table, 0, orig_idx, pivot_value_new_number (
- map_dependent_var (cmd, res, v)));
- }
-
- pivot_table_submit (table);
-}
-
-
-/* Show the Variables in the Equation box */
-static void
-output_variables (const struct lr_spec *cmd,
- const struct lr_result *res)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Variables in the Equation"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("B"), PIVOT_RC_OTHER,
- N_("S.E."), PIVOT_RC_OTHER,
- N_("Wald"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE,
- N_("Exp(B)"), PIVOT_RC_OTHER);
- if (cmd->print & PRINT_CI)
- {
- struct pivot_category *group = pivot_category_create_group__ (
- statistics->root,
- pivot_value_new_text_format (N_("%d%% CI for Exp(B)"),
- cmd->confidence));
- pivot_category_create_leaves (group, N_("Lower"), N_("Upper"));
- }
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
- struct pivot_category *step1 = pivot_category_create_group (
- variables->root, N_("Step 1"));
-
- int ivar = 0;
- int idx_correction = 0;
- int i = 0;
-
- int nr = cmd->n_predictor_vars;
- if (cmd->constant)
- nr++;
- if (res->cats)
- nr += categoricals_df_total (res->cats) + cmd->n_cat_predictors;
-
- for (int row = 0; row < nr; row++)
- {
- const int idx = row - idx_correction;
-
- int var_idx;
- if (idx < cmd->n_predictor_vars)
- var_idx = pivot_category_create_leaf (
- step1, pivot_value_new_variable (cmd->predictor_vars[idx]));
- else if (i < cmd->n_cat_predictors)
- {
- const struct interaction *cat_predictors = cmd->cat_predictors[i];
- struct string str = DS_EMPTY_INITIALIZER;
- interaction_to_string (cat_predictors, &str);
- if (ivar != 0)
- ds_put_format (&str, "(%d)", ivar);
- var_idx = pivot_category_create_leaf (
- step1, pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
-
- int df = categoricals_df (res->cats, i);
- bool summary = ivar == 0;
- if (summary)
- {
- /* Calculate the Wald statistic,
- which is \beta' C^-1 \beta .
- where \beta is the vector of the coefficient estimates comprising this
- categorial variable. and C is the corresponding submatrix of the
- hessian matrix.
- */
- gsl_matrix_const_view mv =
- gsl_matrix_const_submatrix (res->hessian, idx, idx, df, df);
- gsl_matrix *subhessian = gsl_matrix_alloc (mv.matrix.size1, mv.matrix.size2);
- gsl_vector_const_view vv = gsl_vector_const_subvector (res->beta_hat, idx, df);
- gsl_vector *temp = gsl_vector_alloc (df);
-
- gsl_matrix_memcpy (subhessian, &mv.matrix);
- gsl_linalg_cholesky_decomp (subhessian);
- gsl_linalg_cholesky_invert (subhessian);
-
- gsl_blas_dgemv (CblasTrans, 1.0, subhessian, &vv.vector, 0, temp);
- double wald;
- gsl_blas_ddot (temp, &vv.vector, &wald);
-
- double entries[] = { wald, df, gsl_cdf_chisq_Q (wald, df) };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j + 2, var_idx,
- pivot_value_new_number (entries[j]));
-
- idx_correction++;
- gsl_matrix_free (subhessian);
- gsl_vector_free (temp);
- }
-
- if (ivar++ == df)
- {
- ++i; /* next interaction */
- ivar = 0;
- }
-
- if (summary)
- continue;
- }
- else
- var_idx = pivot_category_create_leaves (step1, N_("Constant"));
-
- double b = gsl_vector_get (res->beta_hat, idx);
- double sigma2 = gsl_matrix_get (res->hessian, idx, idx);
- double wald = pow2 (b) / sigma2;
- double df = 1;
- double wc = (gsl_cdf_ugaussian_Pinv (0.5 + cmd->confidence / 200.0)
- * sqrt (sigma2));
- bool show_ci = cmd->print & PRINT_CI && row < nr - cmd->constant;
-
- double entries[] = {
- b,
- sqrt (sigma2),
- wald,
- df,
- gsl_cdf_chisq_Q (wald, df),
- exp (b),
- show_ci ? exp (b - wc) : SYSMIS,
- show_ci ? exp (b + wc) : SYSMIS,
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- if (entries[j] != SYSMIS)
- pivot_table_put2 (table, j, var_idx,
- pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
-}
-
-
-/* Show the model summary box */
-static void
-output_model_summary (const struct lr_result *res,
- double initial_log_likelihood, double log_likelihood)
-{
- struct pivot_table *table = pivot_table_create (N_("Model Summary"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("-2 Log likelihood"), PIVOT_RC_OTHER,
- N_("Cox & Snell R Square"), PIVOT_RC_OTHER,
- N_("Nagelkerke R Square"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *step = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Step"));
- step->root->show_label = true;
- pivot_category_create_leaf (step->root, pivot_value_new_integer (1));
-
- double cox = (1.0 - exp ((initial_log_likelihood - log_likelihood)
- * (2 / res->cc)));
- double entries[] = {
- -2 * log_likelihood,
- cox,
- cox / (1.0 - exp(initial_log_likelihood * (2 / res->cc)))
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, 0, pivot_value_new_number (entries[i]));
-
- pivot_table_submit (table);
-}
-
-/* Show the case processing summary box */
-static void
-case_processing_summary (const struct lr_result *res)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Case Processing Summary"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Percent"), PIVOT_RC_PERCENT);
-
- struct pivot_dimension *cases = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Unweighted Cases"),
- N_("Included in Analysis"), N_("Missing Cases"), N_("Total"));
- cases->root->show_label = true;
-
- double total = res->n_nonmissing + res->n_missing;
- struct entry
- {
- int stat_idx;
- int case_idx;
- double x;
- }
- entries[] = {
- { 0, 0, res->n_nonmissing },
- { 0, 1, res->n_missing },
- { 0, 2, total },
- { 1, 0, 100.0 * res->n_nonmissing / total },
- { 1, 1, 100.0 * res->n_missing / total },
- { 1, 2, 100.0 },
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, entries[i].stat_idx, entries[i].case_idx,
- pivot_value_new_number (entries[i].x));
-
- pivot_table_submit (table);
-}
-
-static void
-output_categories (const struct lr_spec *cmd, const struct lr_result *res)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Categorical Variables' Codings"));
- pivot_table_set_weight_var (table, dict_get_weight (cmd->dict));
-
- int max_df = 0;
- int total_cats = 0;
- for (int i = 0; i < cmd->n_cat_predictors; ++i)
- {
- size_t n = categoricals_n_count (res->cats, i);
- size_t df = categoricals_df (res->cats, i);
- if (max_df < df)
- max_df = df;
- total_cats += n;
- }
-
- struct pivot_dimension *codings = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Codings"),
- N_("Frequency"), PIVOT_RC_COUNT);
- struct pivot_category *coding_group = pivot_category_create_group (
- codings->root, N_("Parameter coding"));
- for (int i = 0; i < max_df; ++i)
- pivot_category_create_leaf_rc (
- coding_group,
- pivot_value_new_user_text_nocopy (xasprintf ("(%d)", i + 1)),
- PIVOT_RC_INTEGER);
-
- struct pivot_dimension *categories = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Categories"));
-
- int cumulative_df = 0;
- for (int v = 0; v < cmd->n_cat_predictors; ++v)
- {
- int cat;
- const struct interaction *cat_predictors = cmd->cat_predictors[v];
- int df = categoricals_df (res->cats, v);
-
- struct string str = DS_EMPTY_INITIALIZER;
- interaction_to_string (cat_predictors, &str);
- struct pivot_category *var_group = pivot_category_create_group__ (
- categories->root,
- pivot_value_new_user_text_nocopy (ds_steal_cstr (&str)));
-
- for (cat = 0; cat < categoricals_n_count (res->cats, v); ++cat)
- {
- const struct ccase *c = categoricals_get_case_by_category_real (
- res->cats, v, cat);
- struct string label = DS_EMPTY_INITIALIZER;
- for (int x = 0; x < cat_predictors->n_vars; ++x)
- {
- if (!ds_is_empty (&label))
- ds_put_byte (&label, ' ');
-
- const union value *val = case_data (c, cat_predictors->vars[x]);
- var_append_value_name (cat_predictors->vars[x], val, &label);
- }
- int cat_idx = pivot_category_create_leaf (
- var_group,
- pivot_value_new_user_text_nocopy (ds_steal_cstr (&label)));
-
- double *freq = categoricals_get_user_data_by_category_real (
- res->cats, v, cat);
- pivot_table_put2 (table, 0, cat_idx, pivot_value_new_number (*freq));
-
- for (int x = 0; x < df; ++x)
- pivot_table_put2 (table, x + 1, cat_idx,
- pivot_value_new_number (cat == x));
- }
- cumulative_df += df;
- }
-
- pivot_table_submit (table);
-}
-
-static void
-create_classification_dimension (const struct lr_spec *cmd,
- const struct lr_result *res,
- struct pivot_table *table,
- enum pivot_axis_type axis_type,
- const char *label, const char *total)
-{
- struct pivot_dimension *d = pivot_dimension_create (
- table, axis_type, label);
- d->root->show_label = true;
- struct pivot_category *pred_group = pivot_category_create_group__ (
- d->root, pivot_value_new_variable (cmd->dep_var));
- for (int i = 0; i < 2; i++)
- {
- const union value *y = i ? &res->y1 : &res->y0;
- pivot_category_create_leaf_rc (
- pred_group, pivot_value_new_var_value (cmd->dep_var, y),
- PIVOT_RC_COUNT);
- }
- pivot_category_create_leaves (d->root, total, PIVOT_RC_PERCENT);
-}
-
-static void
-output_classification_table (const struct lr_spec *cmd, const struct lr_result *res)
-{
- struct pivot_table *table = pivot_table_create (N_("Classification Table"));
- pivot_table_set_weight_var (table, cmd->wv);
-
- create_classification_dimension (cmd, res, table, PIVOT_AXIS_COLUMN,
- N_("Predicted"), N_("Percentage Correct"));
- create_classification_dimension (cmd, res, table, PIVOT_AXIS_ROW,
- N_("Observed"), N_("Overall Percentage"));
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Step"), N_("Step 1"));
-
- struct entry
- {
- int pred_idx;
- int obs_idx;
- double x;
- }
- entries[] = {
- { 0, 0, res->tn },
- { 0, 1, res->fn },
- { 1, 0, res->fp },
- { 1, 1, res->tp },
- { 2, 0, 100 * res->tn / (res->tn + res->fp) },
- { 2, 1, 100 * res->tp / (res->tp + res->fn) },
- { 2, 2,
- 100 * (res->tp + res->tn) / (res->tp + res->tn + res->fp + res->fn)},
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put3 (table, e->pred_idx, e->obs_idx, 0,
- pivot_value_new_number (e->x));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/mann-whitney.h"
-
-#include <gsl/gsl_cdf.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "libpspp/cast.h"
-#include "libpspp/misc.h"
-#include "math/sort.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-/* Calculates the adjustment necessary for tie compensation */
-static void
-distinct_callback (double v UNUSED, casenumber t, double w UNUSED, void *aux)
-{
- double *tiebreaker = aux;
-
- *tiebreaker += (pow3 (t) - t) / 12.0;
-}
-
-struct mw
-{
- double rank_sum[2];
- double n[2];
-
- double u; /* The Mann-Whitney U statistic */
- double w; /* The Wilcoxon Rank Sum W statistic */
- double z;
-};
-
-static void show_ranks_box (const struct n_sample_test *, const struct mw *);
-static void show_statistics_box (const struct n_sample_test *,
- const struct mw *);
-
-
-
-static bool
-belongs_to_test (const struct ccase *c, void *aux)
-{
- const struct n_sample_test *nst = aux;
-
- const union value *group = case_data (c, nst->indep_var);
- const size_t group_var_width = var_get_width (nst->indep_var);
-
- if (value_equal (group, &nst->val1, group_var_width))
- return true;
-
- if (value_equal (group, &nst->val2, group_var_width))
- return true;
-
- return false;
-}
-
-
-
-void
-mann_whitney_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- int i;
- const struct dictionary *dict = dataset_dict (ds);
- const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test, parent);
-
- const struct caseproto *proto = casereader_get_proto (input);
- size_t rank_idx = caseproto_get_n_widths (proto);
-
- struct mw *mw = XCALLOC (nst->n_vars, struct mw);
-
- for (i = 0; i < nst->n_vars; ++i)
- {
- double tiebreaker = 0.0;
- bool warn = true;
- enum rank_error rerr = 0;
- struct casereader *rr;
- struct ccase *c;
- const struct variable *var = nst->vars[i];
-
- struct casereader *reader =
- casereader_create_filter_func (casereader_clone (input),
- belongs_to_test,
- NULL,
- CONST_CAST (struct n_sample_test *, nst),
- NULL);
-
- reader = casereader_create_filter_missing (reader, &var, 1,
- exclude,
- NULL, NULL);
-
- reader = sort_execute_1var (reader, var);
-
- rr = casereader_create_append_rank (reader, var,
- dict_get_weight (dict),
- &rerr,
- distinct_callback, &tiebreaker);
-
- for (; (c = casereader_read (rr)); case_unref (c))
- {
- const union value *group = case_data (c, nst->indep_var);
- const size_t group_var_width = var_get_width (nst->indep_var);
- const double rank = case_num_idx (c, rank_idx);
-
- if (value_equal (group, &nst->val1, group_var_width))
- {
- mw[i].rank_sum[0] += rank;
- mw[i].n[0] += dict_get_case_weight (dict, c, &warn);
- }
- else if (value_equal (group, &nst->val2, group_var_width))
- {
- mw[i].rank_sum[1] += rank;
- mw[i].n[1] += dict_get_case_weight (dict, c, &warn);
- }
- }
- casereader_destroy (rr);
-
- {
- double n;
- double denominator;
- struct mw *mwv = &mw[i];
-
- mwv->u = mwv->n[0] * mwv->n[1] ;
- mwv->u += mwv->n[0] * (mwv->n[0] + 1) / 2.0;
- mwv->u -= mwv->rank_sum[0];
-
- mwv->w = mwv->rank_sum[1];
- if (mwv->u > mwv->n[0] * mwv->n[1] / 2.0)
- {
- mwv->u = mwv->n[0] * mwv->n[1] - mwv->u;
- mwv->w = mwv->rank_sum[0];
- }
- mwv->z = mwv->u - mwv->n[0] * mwv->n[1] / 2.0;
- n = mwv->n[0] + mwv->n[1];
- denominator = pow3(n) - n;
- denominator /= 12;
- denominator -= tiebreaker;
- denominator *= mwv->n[0] * mwv->n[1];
- denominator /= n * (n - 1);
-
- mwv->z /= sqrt (denominator);
- }
- }
- casereader_destroy (input);
-
- show_ranks_box (nst, mw);
- show_statistics_box (nst, mw);
-
- free (mw);
-}
-\f
-static void
-show_ranks_box (const struct n_sample_test *nst, const struct mw *mwv)
-{
- struct pivot_table *table = pivot_table_create (N_("Ranks"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Mean Rank"), PIVOT_RC_OTHER,
- N_("Sum of Ranks"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *indep = pivot_dimension_create__ (
- table, PIVOT_AXIS_ROW, pivot_value_new_variable (nst->indep_var));
- pivot_category_create_leaf (indep->root,
- pivot_value_new_var_value (nst->indep_var,
- &nst->val1));
- pivot_category_create_leaf (indep->root,
- pivot_value_new_var_value (nst->indep_var,
- &nst->val2));
- pivot_category_create_leaves (indep->root, N_("Total"));
-
- struct pivot_dimension *dep = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- for (size_t i = 0 ; i < nst->n_vars ; ++i)
- {
- const struct mw *mw = &mwv[i];
-
- int dep_idx = pivot_category_create_leaf (
- dep->root, pivot_value_new_variable (nst->vars[i]));
-
- struct entry
- {
- int stat_idx;
- int indep_idx;
- double x;
- }
- entries[] = {
- /* N. */
- { 0, 0, mw->n[0] },
- { 0, 1, mw->n[1] },
- { 0, 2, mw->n[0] + mw->n[1] },
-
- /* Mean Rank. */
- { 1, 0, mw->rank_sum[0] / mw->n[0] },
- { 1, 1, mw->rank_sum[1] / mw->n[1] },
-
- /* Sum of Ranks. */
- { 2, 0, mw->rank_sum[0] },
- { 2, 1, mw->rank_sum[1] },
- };
-
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- {
- const struct entry *e = &entries[j];
- pivot_table_put3 (table, e->stat_idx, e->indep_idx, dep_idx,
- pivot_value_new_number (e->x));
- }
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_statistics_box (const struct n_sample_test *nst, const struct mw *mwv)
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- _("Mann-Whitney U"), PIVOT_RC_OTHER,
- _("Wilcoxon W"), PIVOT_RC_OTHER,
- _("Z"), PIVOT_RC_OTHER,
- _("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0 ; i < nst->n_vars ; ++i)
- {
- const struct mw *mw = &mwv[i];
-
- int row = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (nst->vars[i]));
-
- double entries[] = {
- mw->u,
- mw->w,
- mw->z,
- 2.0 * gsl_cdf_ugaussian_P (mw->z),
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !mann_whitney_h
-#define mann_whitney_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-
-struct mann_whitney_test
-{
- struct two_sample_test parent;
-};
-
-struct casereader;
-struct dataset;
-
-void mann_whitney_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer
- );
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2021 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_blas.h>
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_eigen.h>
-#include <gsl/gsl_linalg.h>
-#include <gsl/gsl_matrix.h>
-#include <gsl/gsl_permutation.h>
-#include <gsl/gsl_randist.h>
-#include <gsl/gsl_vector.h>
-#include <limits.h>
-#include <math.h>
-#include <uniwidth.h>
-
-#include "data/any-reader.h"
-#include "data/any-writer.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/data-in.h"
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/file-handle-def.h"
-#include "language/command.h"
-#include "language/data-io/data-reader.h"
-#include "language/data-io/data-writer.h"
-#include "language/data-io/file-handle.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/hmap.h"
-#include "libpspp/i18n.h"
-#include "libpspp/intern.h"
-#include "libpspp/misc.h"
-#include "libpspp/str.h"
-#include "libpspp/string-array.h"
-#include "libpspp/stringi-set.h"
-#include "libpspp/u8-line.h"
-#include "math/distributions.h"
-#include "math/random.h"
-#include "output/driver.h"
-#include "output/output-item.h"
-#include "output/pivot-table.h"
-
-#include "gl/c-ctype.h"
-#include "gl/c-strcase.h"
-#include "gl/ftoastr.h"
-#include "gl/minmax.h"
-#include "gl/xsize.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-struct matrix_state;
-
-/* A variable in the matrix language. */
-struct matrix_var
- {
- struct hmap_node hmap_node; /* In matrix_state's 'vars' hmap. */
- char *name; /* UTF-8. */
- gsl_matrix *value; /* NULL, if the variable is uninitialized. */
- };
-
-/* All the MSAVE commands within a matrix program share common configuration,
- provided by the first MSAVE command within the program. This structure
- encapsulates this configuration. */
-struct msave_common
- {
- /* Common configuration for all MSAVEs. */
- struct msg_location *location; /* Range of lines for first MSAVE. */
- struct file_handle *outfile; /* Output file for all the MSAVEs. */
- struct msg_location *outfile_location;
- struct string_array variables; /* VARIABLES subcommand. */
- struct msg_location *variables_location;
- struct string_array fnames; /* FNAMES subcommand. */
- struct msg_location *fnames_location;
- struct string_array snames; /* SNAMES subcommand. */
- struct msg_location *snames_location;
-
- /* Collects and owns factors and splits. The individual msave_command
- structs point to these but do not own them. (This is because factors
- and splits can be carried over from one MSAVE to the next, so it's
- easiest to just take the most recent.) */
- struct matrix_expr **factors;
- size_t n_factors, allocated_factors;
- struct matrix_expr **splits;
- size_t n_splits, allocated_splits;
-
- /* Execution state. */
- struct dictionary *dict;
- struct casewriter *writer;
- };
-
-/* A file used by one or more READ commands. */
-struct read_file
- {
- /* Parse state. */
- struct file_handle *file;
-
- /* Execution state. */
- struct dfm_reader *reader;
- char *encoding;
- };
-
-static struct read_file *read_file_create (struct matrix_state *,
- struct file_handle *);
-static struct dfm_reader *read_file_open (struct read_file *);
-
-/* A file used by one or more WRITE comamnds. */
-struct write_file
- {
- /* Parse state. */
- struct file_handle *file;
-
- /* Execution state. */
- struct dfm_writer *writer;
- char *encoding;
- struct u8_line *held; /* Output held by a previous WRITE with /HOLD. */
- };
-
-static struct write_file *write_file_create (struct matrix_state *,
- struct file_handle *);
-static struct dfm_writer *write_file_open (struct write_file *);
-static void write_file_destroy (struct write_file *);
-
-/* A file used by one or more SAVE commands. */
-struct save_file
- {
- /* Parse state. */
- struct file_handle *file;
- struct dataset *dataset;
- struct string_array variables;
- struct matrix_expr *names;
- struct stringi_set strings;
-
- /* Execution state. */
- bool error;
- struct casewriter *writer;
- struct dictionary *dict;
- struct msg_location *location;
- };
-
-/* State of an entire matrix program. */
-struct matrix_state
- {
- /* State passed into MATRIX from outside. */
- struct dataset *dataset;
- struct session *session;
- struct lexer *lexer;
-
- /* Matrix program's own state. */
- struct hmap vars; /* Dictionary of matrix variables. */
- bool in_loop; /* True if parsing within a LOOP. */
-
- /* MSAVE. */
- struct msave_common *msave_common;
-
- /* READ. */
- struct file_handle *prev_read_file;
- struct read_file **read_files;
- size_t n_read_files;
-
- /* WRITE. */
- struct file_handle *prev_write_file;
- struct write_file **write_files;
- size_t n_write_files;
-
- /* SAVE. */
- struct file_handle *prev_save_file;
- struct save_file **save_files;
- size_t n_save_files;
- };
-
-/* Finds and returns the variable with the given NAME (case-insensitive) within
- S, if there is one, or a null pointer if there is not. */
-static struct matrix_var *
-matrix_var_lookup (struct matrix_state *s, struct substring name)
-{
- struct matrix_var *var;
-
- HMAP_FOR_EACH_WITH_HASH (var, struct matrix_var, hmap_node,
- utf8_hash_case_substring (name, 0), &s->vars)
- if (!utf8_sscasecmp (ss_cstr (var->name), name))
- return var;
-
- return NULL;
-}
-
-/* Creates and returns a new variable named NAME within S. There must not
- already be a variable with the same (case-insensitive) name. The variable
- is created uninitialized. */
-static struct matrix_var *
-matrix_var_create (struct matrix_state *s, struct substring name)
-{
- struct matrix_var *var = xmalloc (sizeof *var);
- *var = (struct matrix_var) { .name = ss_xstrdup (name) };
- hmap_insert (&s->vars, &var->hmap_node, utf8_hash_case_substring (name, 0));
- return var;
-}
-
-/* Replaces VAR's value by VALUE. Takes ownership of VALUE. */
-static void
-matrix_var_set (struct matrix_var *var, gsl_matrix *value)
-{
- gsl_matrix_free (var->value);
- var->value = value;
-}
-\f
-/* Matrix function catalog. */
-
-/* The third argument to F() is a "prototype". For most prototypes, the first
- letter (before the _) represents the return type and each other letter
- (after the _) is an argument type. The types are:
-
- - "m": A matrix of unrestricted dimensions.
-
- - "d": A scalar.
-
- - "v": A row or column vector.
-
- - "e": Primarily for the first argument, this is a matrix with
- unrestricted dimensions treated elementwise. Each element in the matrix
- is passed to the implementation function separately.
-
- - "n": This gets passed the "const struct matrix_expr *" that represents
- the expression. This allows the evaluation function to grab the source
- location of arguments so that it can report accurate error locations.
- This type doesn't correspond to an argument passed in by the user.
-
- The fourth argument is an optional constraints string. For this purpose the
- first argument is named "a", the second "b", and so on. The following kinds
- of constraints are supported. For matrix arguments, the constraints are
- applied to each value in the matrix separately:
-
- - "a(0,1)" or "a[0,1]": 0 < a < 1 or 0 <= a <= 1, respectively. Any
- integer may substitute for 0 and 1. Half-open constraints (] and [) are
- also supported.
-
- - "ai": Restrict a to integer values.
-
- - "a>0", "a<0", "a>=0", "a<=0", "a!=0".
-
- - "a<b", "a>b", "a<=b", "a>=b", "b!=0".
-*/
-#define MATRIX_FUNCTIONS \
- F(ABS, "ABS", m_e, NULL) \
- F(ALL, "ALL", d_m, NULL) \
- F(ANY, "ANY", d_m, NULL) \
- F(ARSIN, "ARSIN", m_e, "a[-1,1]") \
- F(ARTAN, "ARTAN", m_e, NULL) \
- F(BLOCK, "BLOCK", m_any, NULL) \
- F(CHOL, "CHOL", m_mn, NULL) \
- F(CMIN, "CMIN", m_m, NULL) \
- F(CMAX, "CMAX", m_m, NULL) \
- F(COS, "COS", m_e, NULL) \
- F(CSSQ, "CSSQ", m_m, NULL) \
- F(CSUM, "CSUM", m_m, NULL) \
- F(DESIGN, "DESIGN", m_mn, NULL) \
- F(DET, "DET", d_m, NULL) \
- F(DIAG, "DIAG", m_m, NULL) \
- F(EVAL, "EVAL", m_mn, NULL) \
- F(EXP, "EXP", m_e, NULL) \
- F(GINV, "GINV", m_m, NULL) \
- F(GRADE, "GRADE", m_m, NULL) \
- F(GSCH, "GSCH", m_mn, NULL) \
- F(IDENT, "IDENT", IDENT, NULL) \
- F(INV, "INV", m_m, NULL) \
- F(KRONEKER, "KRONEKER", m_mm, NULL) \
- F(LG10, "LG10", m_e, "a>0") \
- F(LN, "LN", m_e, "a>0") \
- F(MAGIC, "MAGIC", m_d, "ai>=3") \
- F(MAKE, "MAKE", m_ddd, "ai>=0 bi>=0") \
- F(MDIAG, "MDIAG", m_v, NULL) \
- F(MMAX, "MMAX", d_m, NULL) \
- F(MMIN, "MMIN", d_m, NULL) \
- F(MOD, "MOD", m_md, "b!=0") \
- F(MSSQ, "MSSQ", d_m, NULL) \
- F(MSUM, "MSUM", d_m, NULL) \
- F(NCOL, "NCOL", d_m, NULL) \
- F(NROW, "NROW", d_m, NULL) \
- F(RANK, "RANK", d_m, NULL) \
- F(RESHAPE, "RESHAPE", m_mddn, NULL) \
- F(RMAX, "RMAX", m_m, NULL) \
- F(RMIN, "RMIN", m_m, NULL) \
- F(RND, "RND", m_e, NULL) \
- F(RNKORDER, "RNKORDER", m_m, NULL) \
- F(RSSQ, "RSSQ", m_m, NULL) \
- F(RSUM, "RSUM", m_m, NULL) \
- F(SIN, "SIN", m_e, NULL) \
- F(SOLVE, "SOLVE", m_mmn, NULL) \
- F(SQRT, "SQRT", m_e, "a>=0") \
- F(SSCP, "SSCP", m_m, NULL) \
- F(SVAL, "SVAL", m_m, NULL) \
- F(SWEEP, "SWEEP", m_mdn, NULL) \
- F(T, "T", m_m, NULL) \
- F(TRACE, "TRACE", d_m, NULL) \
- F(TRANSPOS, "TRANSPOS", m_m, NULL) \
- F(TRUNC, "TRUNC", m_e, NULL) \
- F(UNIFORM, "UNIFORM", m_ddn, "ai>=0 bi>=0") \
- F(PDF_BETA, "PDF.BETA", m_edd, "a[0,1] b>0 c>0") \
- F(CDF_BETA, "CDF.BETA", m_edd, "a[0,1] b>0 c>0") \
- F(IDF_BETA, "IDF.BETA", m_edd, "a[0,1] b>0 c>0") \
- F(RV_BETA, "RV.BETA", d_dd, "a>0 b>0") \
- F(NCDF_BETA, "NCDF.BETA", m_eddd, "a>=0 b>0 c>0 d>0") \
- F(NPDF_BETA, "NCDF.BETA", m_eddd, "a>=0 b>0 c>0 d>0") \
- F(CDF_BVNOR, "CDF.BVNOR", m_eed, "c[-1,1]") \
- F(PDF_BVNOR, "PDF.BVNOR", m_eed, "c[-1,1]") \
- F(CDF_CAUCHY, "CDF.CAUCHY", m_edd, "c>0") \
- F(IDF_CAUCHY, "IDF.CAUCHY", m_edd, "a(0,1) c>0") \
- F(PDF_CAUCHY, "PDF.CAUCHY", m_edd, "c>0") \
- F(RV_CAUCHY, "RV.CAUCHY", d_dd, "b>0") \
- F(CDF_CHISQ, "CDF.CHISQ", m_ed, "a>=0 b>0") \
- F(CHICDF, "CHICDF", m_ed, "a>=0 b>0") \
- F(IDF_CHISQ, "IDF.CHISQ", m_ed, "a[0,1) b>0") \
- F(PDF_CHISQ, "PDF.CHISQ", m_ed, "a>=0 b>0") \
- F(RV_CHISQ, "RV.CHISQ", d_d, "a>0") \
- F(SIG_CHISQ, "SIG.CHISQ", m_ed, "a>=0 b>0") \
- F(CDF_EXP, "CDF.EXP", m_ed, "a>=0 b>=0") \
- F(IDF_EXP, "IDF.EXP", m_ed, "a[0,1) b>0") \
- F(PDF_EXP, "PDF.EXP", m_ed, "a>=0 b>0") \
- F(RV_EXP, "RV.EXP", d_d, "a>0") \
- F(PDF_XPOWER, "PDF.XPOWER", m_edd, "b>0 c>=0") \
- F(RV_XPOWER, "RV.XPOWER", d_dd, "a>0 c>=0") \
- F(CDF_F, "CDF.F", m_edd, "a>=0 b>0 c>0") \
- F(FCDF, "FCDF", m_edd, "a>=0 b>0 c>0") \
- F(IDF_F, "IDF.F", m_edd, "a[0,1) b>0 c>0") \
- F(PDF_F, "PDF.F", m_edd, "a>=0 b>0 c>0") \
- F(RV_F, "RV.F", d_dd, "a>0 b>0") \
- F(SIG_F, "SIG.F", m_edd, "a>=0 b>0 c>0") \
- F(CDF_GAMMA, "CDF.GAMMA", m_edd, "a>=0 b>0 c>0") \
- F(IDF_GAMMA, "IDF.GAMMA", m_edd, "a[0,1] b>0 c>0") \
- F(PDF_GAMMA, "PDF.GAMMA", m_edd, "a>=0 b>0 c>0") \
- F(RV_GAMMA, "RV.GAMMA", d_dd, "a>0 b>0") \
- F(PDF_LANDAU, "PDF.LANDAU", m_e, NULL) \
- F(RV_LANDAU, "RV.LANDAU", d_none, NULL) \
- F(CDF_LAPLACE, "CDF.LAPLACE", m_edd, "c>0") \
- F(IDF_LAPLACE, "IDF.LAPLACE", m_edd, "a(0,1) c>0") \
- F(PDF_LAPLACE, "PDF.LAPLACE", m_edd, "c>0") \
- F(RV_LAPLACE, "RV.LAPLACE", d_dd, "b>0") \
- F(RV_LEVY, "RV.LEVY", d_dd, "b(0,2]") \
- F(RV_LVSKEW, "RV.LVSKEW", d_ddd, "b(0,2] c[-1,1]") \
- F(CDF_LOGISTIC, "CDF.LOGISTIC", m_edd, "c>0") \
- F(IDF_LOGISTIC, "IDF.LOGISTIC", m_edd, "a(0,1) c>0") \
- F(PDF_LOGISTIC, "PDF.LOGISTIC", m_edd, "c>0") \
- F(RV_LOGISTIC, "RV.LOGISTIC", d_dd, "b>0") \
- F(CDF_LNORMAL, "CDF.LNORMAL", m_edd, "a>=0 b>0 c>0") \
- F(IDF_LNORMAL, "IDF.LNORMAL", m_edd, "a[0,1) b>0 c>0") \
- F(PDF_LNORMAL, "PDF.LNORMAL", m_edd, "a>=0 b>0 c>0") \
- F(RV_LNORMAL, "RV.LNORMAL", d_dd, "a>0 b>0") \
- F(CDF_NORMAL, "CDF.NORMAL", m_edd, "c>0") \
- F(IDF_NORMAL, "IDF.NORMAL", m_edd, "a(0,1) c>0") \
- F(PDF_NORMAL, "PDF.NORMAL", m_edd, "c>0") \
- F(RV_NORMAL, "RV.NORMAL", d_dd, "b>0") \
- F(CDFNORM, "CDFNORM", m_e, NULL) \
- F(PROBIT, "PROBIT", m_e, "a(0,1)") \
- F(NORMAL, "NORMAL", m_e, "a>0") \
- F(PDF_NTAIL, "PDF.NTAIL", m_edd, "b>0 c>0") \
- F(RV_NTAIL, "RV.NTAIL", d_dd, "a>0 b>0") \
- F(CDF_PARETO, "CDF.PARETO", m_edd, "a>=b b>0 c>0") \
- F(IDF_PARETO, "IDF.PARETO", m_edd, "a[0,1) b>0 c>0") \
- F(PDF_PARETO, "PDF.PARETO", m_edd, "a>=b b>0 c>0") \
- F(RV_PARETO, "RV.PARETO", d_dd, "a>0 b>0") \
- F(CDF_RAYLEIGH, "CDF.RAYLEIGH", m_ed, "b>0") \
- F(IDF_RAYLEIGH, "IDF.RAYLEIGH", m_ed, "a[0,1] b>0") \
- F(PDF_RAYLEIGH, "PDF.RAYLEIGH", m_ed, "b>0") \
- F(RV_RAYLEIGH, "RV.RAYLEIGH", d_d, "a>0") \
- F(PDF_RTAIL, "PDF.RTAIL", m_edd, NULL) \
- F(RV_RTAIL, "RV.RTAIL", d_dd, NULL) \
- F(CDF_T, "CDF.T", m_ed, "b>0") \
- F(TCDF, "TCDF", m_ed, "b>0") \
- F(IDF_T, "IDF.T", m_ed, "a(0,1) b>0") \
- F(PDF_T, "PDF.T", m_ed, "b>0") \
- F(RV_T, "RV.T", d_d, "a>0") \
- F(CDF_T1G, "CDF.T1G", m_edd, NULL) \
- F(IDF_T1G, "IDF.T1G", m_edd, "a(0,1)") \
- F(PDF_T1G, "PDF.T1G", m_edd, NULL) \
- F(RV_T1G, "RV.T1G", d_dd, NULL) \
- F(CDF_T2G, "CDF.T2G", m_edd, NULL) \
- F(IDF_T2G, "IDF.T2G", m_edd, "a(0,1)") \
- F(PDF_T2G, "PDF.T2G", m_edd, NULL) \
- F(RV_T2G, "RV.T2G", d_dd, NULL) \
- F(CDF_UNIFORM, "CDF.UNIFORM", m_edd, "a<=c b<=c") \
- F(IDF_UNIFORM, "IDF.UNIFORM", m_edd, "a[0,1] b<=c") \
- F(PDF_UNIFORM, "PDF.UNIFORM", m_edd, "a<=c b<=c") \
- F(RV_UNIFORM, "RV.UNIFORM", d_dd, "a<=b") \
- F(CDF_WEIBULL, "CDF.WEIBULL", m_edd, "a>=0 b>0 c>0") \
- F(IDF_WEIBULL, "IDF.WEIBULL", m_edd, "a[0,1) b>0 c>0") \
- F(PDF_WEIBULL, "PDF.WEIBULL", m_edd, "a>=0 b>0 c>0") \
- F(RV_WEIBULL, "RV.WEIBULL", d_dd, "a>0 b>0") \
- F(CDF_BERNOULLI, "CDF.BERNOULLI", m_ed, "ai[0,1] b[0,1]") \
- F(PDF_BERNOULLI, "PDF.BERNOULLI", m_ed, "ai[0,1] b[0,1]") \
- F(RV_BERNOULLI, "RV.BERNOULLI", d_d, "a[0,1]") \
- F(CDF_BINOM, "CDF.BINOM", m_edd, "bi>0 c[0,1]") \
- F(PDF_BINOM, "PDF.BINOM", m_edd, "ai>=0<=b bi>0 c[0,1]") \
- F(RV_BINOM, "RV.BINOM", d_dd, "ai>0 b[0,1]") \
- F(CDF_GEOM, "CDF.GEOM", m_ed, "ai>=1 b[0,1]") \
- F(PDF_GEOM, "PDF.GEOM", m_ed, "ai>=1 b[0,1]") \
- F(RV_GEOM, "RV.GEOM", d_d, "a[0,1]") \
- F(CDF_HYPER, "CDF.HYPER", m_eddd, "ai>=0<=d bi>0 ci>0<=b di>0<=b") \
- F(PDF_HYPER, "PDF.HYPER", m_eddd, "ai>=0<=d bi>0 ci>0<=b di>0<=b") \
- F(RV_HYPER, "RV.HYPER", d_ddd, "ai>0 bi>0<=a ci>0<=a") \
- F(PDF_LOG, "PDF.LOG", m_ed, "a>=1 b(0,1]") \
- F(RV_LOG, "RV.LOG", d_d, "a(0,1]") \
- F(CDF_NEGBIN, "CDF.NEGBIN", m_edd, "a>=1 bi c(0,1]") \
- F(PDF_NEGBIN, "PDF.NEGBIN", m_edd, "a>=1 bi c(0,1]") \
- F(RV_NEGBIN, "RV.NEGBIN", d_dd, "ai b(0,1]") \
- F(CDF_POISSON, "CDF.POISSON", m_ed, "ai>=0 b>0") \
- F(PDF_POISSON, "PDF.POISSON", m_ed, "ai>=0 b>0") \
- F(RV_POISSON, "RV.POISSON", d_d, "a>0")
-
-/* Properties of a matrix function.
-
- These come straight from the macro invocations above. */
-struct matrix_function_properties
- {
- const char *name;
- const char *constraints;
- };
-
-/* Minimum and maximum argument counts for each matrix function prototype. */
-enum { IDENT_MIN_ARGS = 1, IDENT_MAX_ARGS = 2 };
-enum { d_d_MIN_ARGS = 1, d_d_MAX_ARGS = 1 };
-enum { d_dd_MIN_ARGS = 2, d_dd_MAX_ARGS = 2 };
-enum { d_ddd_MIN_ARGS = 3, d_ddd_MAX_ARGS = 3 };
-enum { d_m_MIN_ARGS = 1, d_m_MAX_ARGS = 1 };
-enum { d_none_MIN_ARGS = 0, d_none_MAX_ARGS = 0 };
-enum { m_any_MIN_ARGS = 1, m_any_MAX_ARGS = INT_MAX };
-enum { m_d_MIN_ARGS = 1, m_d_MAX_ARGS = 1 };
-enum { m_ddd_MIN_ARGS = 3, m_ddd_MAX_ARGS = 3 };
-enum { m_ddn_MIN_ARGS = 2, m_ddn_MAX_ARGS = 2 };
-enum { m_e_MIN_ARGS = 1, m_e_MAX_ARGS = 1 };
-enum { m_ed_MIN_ARGS = 2, m_ed_MAX_ARGS = 2 };
-enum { m_edd_MIN_ARGS = 3, m_edd_MAX_ARGS = 3 };
-enum { m_eddd_MIN_ARGS = 4, m_eddd_MAX_ARGS = 4 };
-enum { m_eed_MIN_ARGS = 3, m_eed_MAX_ARGS = 3 };
-enum { m_m_MIN_ARGS = 1, m_m_MAX_ARGS = 1 };
-enum { m_md_MIN_ARGS = 2, m_md_MAX_ARGS = 2 };
-enum { m_mddn_MIN_ARGS = 3, m_mddn_MAX_ARGS = 3 };
-enum { m_mdn_MIN_ARGS = 2, m_mdn_MAX_ARGS = 2 };
-enum { m_mm_MIN_ARGS = 2, m_mm_MAX_ARGS = 2 };
-enum { m_mmn_MIN_ARGS = 2, m_mmn_MAX_ARGS = 2 };
-enum { m_mn_MIN_ARGS = 1, m_mn_MAX_ARGS = 1 };
-enum { m_v_MIN_ARGS = 1, m_v_MAX_ARGS = 1 };
-
-/* C function prototype for each matrix function prototype. */
-typedef double matrix_proto_d_none (void);
-typedef double matrix_proto_d_d (double);
-typedef double matrix_proto_d_dd (double, double);
-typedef double matrix_proto_d_dd (double, double);
-typedef double matrix_proto_d_ddd (double, double, double);
-typedef gsl_matrix *matrix_proto_m_d (double);
-typedef gsl_matrix *matrix_proto_m_ddd (double, double, double);
-typedef gsl_matrix *matrix_proto_m_ddn (double, double,
- const struct matrix_expr *);
-typedef gsl_matrix *matrix_proto_m_m (gsl_matrix *);
-typedef gsl_matrix *matrix_proto_m_mn (gsl_matrix *,
- const struct matrix_expr *);
-typedef double matrix_proto_m_e (double);
-typedef gsl_matrix *matrix_proto_m_md (gsl_matrix *, double);
-typedef gsl_matrix *matrix_proto_m_mdn (gsl_matrix *, double,
- const struct matrix_expr *);
-typedef double matrix_proto_m_ed (double, double);
-typedef gsl_matrix *matrix_proto_m_mddn (gsl_matrix *, double, double,
- const struct matrix_expr *);
-typedef double matrix_proto_m_edd (double, double, double);
-typedef double matrix_proto_m_eddd (double, double, double, double);
-typedef double matrix_proto_m_eed (double, double, double);
-typedef gsl_matrix *matrix_proto_m_mm (gsl_matrix *, gsl_matrix *);
-typedef gsl_matrix *matrix_proto_m_mmn (gsl_matrix *, gsl_matrix *,
- const struct matrix_expr *);
-typedef gsl_matrix *matrix_proto_m_v (gsl_vector *);
-typedef double matrix_proto_d_m (gsl_matrix *);
-typedef gsl_matrix *matrix_proto_m_any (gsl_matrix *[], size_t n);
-typedef gsl_matrix *matrix_proto_IDENT (double, double);
-
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
- static matrix_proto_##PROTO matrix_eval_##ENUM;
-MATRIX_FUNCTIONS
-#undef F
-\f
-/* Matrix expression data structure and parsing. */
-
-/* A node in a matrix expression. */
-struct matrix_expr
- {
- enum matrix_op
- {
- /* Functions. */
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) MOP_F_##ENUM,
- MATRIX_FUNCTIONS
-#undef F
-
- /* Elementwise and scalar arithmetic. */
- MOP_NEGATE, /* unary - */
- MOP_ADD_ELEMS, /* + */
- MOP_SUB_ELEMS, /* - */
- MOP_MUL_ELEMS, /* &* */
- MOP_DIV_ELEMS, /* / and &/ */
- MOP_EXP_ELEMS, /* &** */
- MOP_SEQ, /* a:b */
- MOP_SEQ_BY, /* a:b:c */
-
- /* Matrix arithmetic. */
- MOP_MUL_MAT, /* * */
- MOP_EXP_MAT, /* ** */
-
- /* Relational. */
- MOP_GT, /* > */
- MOP_GE, /* >= */
- MOP_LT, /* < */
- MOP_LE, /* <= */
- MOP_EQ, /* = */
- MOP_NE, /* <> */
-
- /* Logical. */
- MOP_NOT, /* NOT */
- MOP_AND, /* AND */
- MOP_OR, /* OR */
- MOP_XOR, /* XOR */
-
- /* {}. */
- MOP_PASTE_HORZ, /* a, b, c, ... */
- MOP_PASTE_VERT, /* a; b; c; ... */
- MOP_EMPTY, /* {} */
-
- /* Sub-matrices. */
- MOP_VEC_INDEX, /* x(y) */
- MOP_VEC_ALL, /* x(:) */
- MOP_MAT_INDEX, /* x(y,z) */
- MOP_ROW_INDEX, /* x(y,:) */
- MOP_COL_INDEX, /* x(:,z) */
-
- /* Literals. */
- MOP_NUMBER,
- MOP_VARIABLE,
-
- /* Oddball stuff. */
- MOP_EOF, /* EOF('file') */
- }
- op;
-
- union
- {
- /* Nonterminal expression nodes. */
- struct
- {
- struct matrix_expr **subs;
- size_t n_subs;
- };
-
- /* Terminal expression nodes. */
- double number; /* MOP_NUMBER. */
- struct matrix_var *variable; /* MOP_VARIABLE. */
- struct read_file *eof; /* MOP_EOF. */
- };
-
- /* The syntax location corresponding to this expression node, for use in
- error messages. This is always nonnull for terminal expression nodes.
- For most others, it is null because it can be computed lazily if and
- when it is needed.
-
- Use matrix_expr_location() instead of using this member directly, so
- that it gets computed lazily if needed. */
- struct msg_location *location;
- };
-
-static void
-matrix_expr_location__ (const struct matrix_expr *e,
- const struct msg_location **minp,
- const struct msg_location **maxp)
-{
- struct msg_location *loc = e->location;
- if (loc)
- {
- const struct msg_location *min = *minp;
- if (loc->start.line
- && (!min
- || loc->start.line < min->start.line
- || (loc->start.line == min->start.line
- && loc->start.column < min->start.column)))
- *minp = loc;
-
- const struct msg_location *max = *maxp;
- if (loc->end.line
- && (!max
- || loc->end.line > max->end.line
- || (loc->end.line == max->end.line
- && loc->end.column > max->end.column)))
- *maxp = loc;
-
- return;
- }
-
- assert (e->op != MOP_NUMBER && e->op != MOP_VARIABLE && e->op != MOP_EOF);
- for (size_t i = 0; i < e->n_subs; i++)
- matrix_expr_location__ (e->subs[i], minp, maxp);
-}
-
-/* Returns the source code location corresponding to expression E, computing it
- lazily if needed. */
-static const struct msg_location *
-matrix_expr_location (const struct matrix_expr *e_)
-{
- struct matrix_expr *e = CONST_CAST (struct matrix_expr *, e_);
- if (!e)
- return NULL;
-
- if (!e->location)
- {
- const struct msg_location *min = NULL;
- const struct msg_location *max = NULL;
- matrix_expr_location__ (e, &min, &max);
- if (min && max)
- {
- e->location = msg_location_dup (min);
- e->location->end = max->end;
- }
- }
- return e->location;
-}
-
-/* Sets e->location to the tokens in S's lexer from offset START_OFS to the
- token before the current one. Has no effect if E already has a location or
- if E is null. */
-static void
-matrix_expr_add_location (struct matrix_state *s, int start_ofs,
- struct matrix_expr *e)
-{
- if (e && !e->location)
- e->location = lex_ofs_location (s->lexer, start_ofs,
- lex_ofs (s->lexer) - 1);
-}
-
-/* Frees E and all the data and sub-expressions that it references. */
-static void
-matrix_expr_destroy (struct matrix_expr *e)
-{
- if (!e)
- return;
-
- switch (e->op)
- {
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) case MOP_F_##ENUM:
-MATRIX_FUNCTIONS
-#undef F
- case MOP_NEGATE:
- case MOP_ADD_ELEMS:
- case MOP_SUB_ELEMS:
- case MOP_MUL_ELEMS:
- case MOP_DIV_ELEMS:
- case MOP_EXP_ELEMS:
- case MOP_SEQ:
- case MOP_SEQ_BY:
- case MOP_MUL_MAT:
- case MOP_EXP_MAT:
- case MOP_GT:
- case MOP_GE:
- case MOP_LT:
- case MOP_LE:
- case MOP_EQ:
- case MOP_NE:
- case MOP_NOT:
- case MOP_AND:
- case MOP_OR:
- case MOP_XOR:
- case MOP_EMPTY:
- case MOP_PASTE_HORZ:
- case MOP_PASTE_VERT:
- case MOP_VEC_INDEX:
- case MOP_VEC_ALL:
- case MOP_MAT_INDEX:
- case MOP_ROW_INDEX:
- case MOP_COL_INDEX:
- for (size_t i = 0; i < e->n_subs; i++)
- matrix_expr_destroy (e->subs[i]);
- free (e->subs);
- break;
-
- case MOP_NUMBER:
- case MOP_VARIABLE:
- case MOP_EOF:
- break;
- }
- msg_location_destroy (e->location);
- free (e);
-}
-
-/* Creates and returns a new matrix_expr with type OP, which must be a
- nonterminal type. Initializes the new matrix_expr with the N_SUBS
- expressions in SUBS as subexpressions. */
-static struct matrix_expr *
-matrix_expr_create_subs (enum matrix_op op, struct matrix_expr **subs,
- size_t n_subs)
-{
- struct matrix_expr *e = xmalloc (sizeof *e);
- *e = (struct matrix_expr) {
- .op = op,
- .subs = xmemdup (subs, n_subs * sizeof *subs),
- .n_subs = n_subs
- };
- return e;
-}
-
-static struct matrix_expr *
-matrix_expr_create_0 (enum matrix_op op)
-{
- struct matrix_expr *sub;
- return matrix_expr_create_subs (op, &sub, 0);
-}
-
-static struct matrix_expr *
-matrix_expr_create_1 (enum matrix_op op, struct matrix_expr *sub)
-{
- return matrix_expr_create_subs (op, &sub, 1);
-}
-
-static struct matrix_expr *
-matrix_expr_create_2 (enum matrix_op op,
- struct matrix_expr *sub0, struct matrix_expr *sub1)
-{
- struct matrix_expr *subs[] = { sub0, sub1 };
- return matrix_expr_create_subs (op, subs, sizeof subs / sizeof *subs);
-}
-
-static struct matrix_expr *
-matrix_expr_create_3 (enum matrix_op op, struct matrix_expr *sub0,
- struct matrix_expr *sub1, struct matrix_expr *sub2)
-{
- struct matrix_expr *subs[] = { sub0, sub1, sub2 };
- return matrix_expr_create_subs (op, subs, sizeof subs / sizeof *subs);
-}
-
-/* Creates and returns a new MOP_NUMBER expression node to contain NUMBER. */
-static struct matrix_expr *
-matrix_expr_create_number (double number)
-{
- struct matrix_expr *e = xmalloc (sizeof *e);
- *e = (struct matrix_expr) {
- .op = MOP_NUMBER,
- .number = number,
- };
- return e;
-}
-
-static struct matrix_expr *matrix_expr_parse (struct matrix_state *);
-
-/* A binary operator for matrix_parse_binary_operator(). */
-struct matrix_operator_syntax
- {
- /* Exactly one of these specifies the operator syntax. */
- enum token_type token; /* A token, e.g. T_ASTERISK. */
- const char *id; /* An identifier, e.g. "XOR". */
- const char *phrase; /* A token phrase, e.g. "&**". */
-
- /* The matrix operator corresponding to the syntax. */
- enum matrix_op op;
- };
-
-static bool
-matrix_operator_syntax_match (struct lexer *lexer,
- const struct matrix_operator_syntax *syntax,
- size_t n_syntax, enum matrix_op *op)
-{
- const struct matrix_operator_syntax *end = &syntax[n_syntax];
- for (const struct matrix_operator_syntax *syn = syntax; syn < end; syn++)
- if (syn->id ? lex_match_id (lexer, syn->id)
- : syn->phrase ? lex_match_phrase (lexer, syn->phrase)
- : lex_match (lexer, syn->token))
- {
- *op = syn->op;
- return true;
- }
- return false;
-}
-
-/* Parses a binary operator level in the recursive descent parser, returning a
- matrix expression if successful or a null pointer otherwise. PARSE_NEXT
- must be the function to parse the next level of precedence. The N_SYNTAX
- elements of SYNTAX must specify the syntax and matrix_expr node type to
- parse at this level. */
-static struct matrix_expr *
-matrix_parse_binary_operator (
- struct matrix_state *s,
- struct matrix_expr *(*parse_next) (struct matrix_state *),
- const struct matrix_operator_syntax *syntax, size_t n_syntax)
-{
- struct matrix_expr *lhs = parse_next (s);
- if (!lhs)
- return NULL;
-
- for (;;)
- {
- enum matrix_op op;
- if (!matrix_operator_syntax_match (s->lexer, syntax, n_syntax, &op))
- return lhs;
-
- struct matrix_expr *rhs = parse_next (s);
- if (!rhs)
- {
- matrix_expr_destroy (lhs);
- return NULL;
- }
- lhs = matrix_expr_create_2 (op, lhs, rhs);
- }
-}
-
-/* Parses a comma-separated list of expressions within {}, transforming them
- into MOP_PASTE_HORZ operators. Returns the new expression or NULL on
- error. */
-static struct matrix_expr *
-matrix_parse_curly_comma (struct matrix_state *s)
-{
- static const struct matrix_operator_syntax op = {
- .token = T_COMMA, .op = MOP_PASTE_HORZ
- };
- return matrix_parse_binary_operator (s, matrix_expr_parse, &op, 1);
-}
-
-/* Parses a semicolon-separated list of expressions within {}, transforming
- them into MOP_PASTE_VERT operators. Returns the new expression or NULL on
- error. */
-static struct matrix_expr *
-matrix_parse_curly_semi (struct matrix_state *s)
-{
- if (lex_token (s->lexer) == T_RCURLY)
- {
- /* {} is a special case for a 0×0 matrix. */
- return matrix_expr_create_0 (MOP_EMPTY);
- }
-
- static const struct matrix_operator_syntax op = {
- .token = T_SEMICOLON, .op = MOP_PASTE_VERT
- };
- return matrix_parse_binary_operator (s, matrix_parse_curly_comma, &op, 1);
-}
-
-struct matrix_function
- {
- const char *name;
- enum matrix_op op;
- size_t min_args, max_args;
- };
-
-static struct matrix_expr *matrix_expr_parse (struct matrix_state *);
-
-static bool
-word_matches (const char **test, const char **name)
-{
- size_t test_len = strcspn (*test, ".");
- size_t name_len = strcspn (*name, ".");
- if (test_len == name_len)
- {
- if (buf_compare_case (*test, *name, test_len))
- return false;
- }
- else if (test_len < 3 || test_len > name_len)
- return false;
- else
- {
- if (buf_compare_case (*test, *name, test_len))
- return false;
- }
-
- *test += test_len;
- *name += name_len;
- if (**test != **name)
- return false;
-
- if (**test == '.')
- {
- (*test)++;
- (*name)++;
- }
- return true;
-}
-
-/* Returns 0 if TOKEN and FUNC do not match,
- 1 if TOKEN is an acceptable abbreviation for FUNC,
- 2 if TOKEN equals FUNC. */
-static int
-compare_function_names (const char *token_, const char *func_)
-{
- const char *token = token_;
- const char *func = func_;
- while (*token || *func)
- if (!word_matches (&token, &func))
- return 0;
- return !c_strcasecmp (token_, func_) ? 2 : 1;
-}
-
-static const struct matrix_function *
-matrix_parse_function_name (const char *token)
-{
- static const struct matrix_function functions[] =
- {
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
- { STRING, MOP_F_##ENUM, PROTO##_MIN_ARGS, PROTO##_MAX_ARGS },
- MATRIX_FUNCTIONS
-#undef F
- };
- enum { N_FUNCTIONS = sizeof functions / sizeof *functions };
-
- for (size_t i = 0; i < N_FUNCTIONS; i++)
- {
- if (compare_function_names (token, functions[i].name) > 0)
- return &functions[i];
- }
- return NULL;
-}
-
-static bool
-matrix_parse_function (struct matrix_state *s, const char *token,
- struct matrix_expr **exprp)
-{
- *exprp = NULL;
- if (lex_next_token (s->lexer, 1) != T_LPAREN)
- return false;
-
- int start_ofs = lex_ofs (s->lexer);
- if (lex_match_id (s->lexer, "EOF"))
- {
- lex_get (s->lexer);
- struct file_handle *fh = fh_parse (s->lexer, FH_REF_FILE, s->session);
- if (!fh)
- return true;
-
- if (!lex_force_match (s->lexer, T_RPAREN))
- {
- fh_unref (fh);
- return true;
- }
-
- struct read_file *rf = read_file_create (s, fh);
-
- struct matrix_expr *e = xmalloc (sizeof *e);
- *e = (struct matrix_expr) { .op = MOP_EOF, .eof = rf };
- matrix_expr_add_location (s, start_ofs, e);
- *exprp = e;
- return true;
- }
-
- const struct matrix_function *f = matrix_parse_function_name (token);
- if (!f)
- return false;
-
- struct matrix_expr *e = xmalloc (sizeof *e);
- *e = (struct matrix_expr) { .op = f->op };
-
- lex_get_n (s->lexer, 2);
- if (lex_token (s->lexer) != T_RPAREN)
- {
- size_t allocated_subs = 0;
- do
- {
- struct matrix_expr *sub = matrix_expr_parse (s);
- if (!sub)
- goto error;
-
- if (e->n_subs >= allocated_subs)
- e->subs = x2nrealloc (e->subs, &allocated_subs, sizeof *e->subs);
- e->subs[e->n_subs++] = sub;
- }
- while (lex_match (s->lexer, T_COMMA));
- }
- if (!lex_force_match (s->lexer, T_RPAREN))
- goto error;
-
- if (e->n_subs < f->min_args || e->n_subs > f->max_args)
- {
- if (f->min_args == f->max_args)
- msg_at (SE, e->location,
- ngettext ("Matrix function %s requires %zu argument.",
- "Matrix function %s requires %zu arguments.",
- f->min_args),
- f->name, f->min_args);
- else if (f->min_args == 1 && f->max_args == 2)
- msg_at (SE, e->location,
- ngettext ("Matrix function %s requires 1 or 2 arguments, "
- "but %zu was provided.",
- "Matrix function %s requires 1 or 2 arguments, "
- "but %zu were provided.",
- e->n_subs),
- f->name, e->n_subs);
- else if (f->min_args == 1 && f->max_args == INT_MAX)
- msg_at (SE, e->location,
- _("Matrix function %s requires at least one argument."),
- f->name);
- else
- NOT_REACHED ();
-
- goto error;
- }
-
- matrix_expr_add_location (s, start_ofs, e);
-
- *exprp = e;
- return true;
-
-error:
- matrix_expr_destroy (e);
- return true;
-}
-
-static struct matrix_expr *
-matrix_parse_primary__ (struct matrix_state *s)
-{
- if (lex_is_number (s->lexer))
- {
- double number = lex_number (s->lexer);
- lex_get (s->lexer);
-
- return matrix_expr_create_number (number);
- }
- else if (lex_is_string (s->lexer))
- {
- char string[sizeof (double)];
- buf_copy_str_rpad (string, sizeof string, lex_tokcstr (s->lexer), ' ');
- lex_get (s->lexer);
-
- double number;
- memcpy (&number, string, sizeof number);
-
- return matrix_expr_create_number (number);
- }
- else if (lex_match (s->lexer, T_LPAREN))
- {
- struct matrix_expr *e = matrix_expr_parse (s);
- if (!e || !lex_force_match (s->lexer, T_RPAREN))
- {
- matrix_expr_destroy (e);
- return NULL;
- }
- return e;
- }
- else if (lex_match (s->lexer, T_LCURLY))
- {
- struct matrix_expr *e = matrix_parse_curly_semi (s);
- if (!e || !lex_force_match (s->lexer, T_RCURLY))
- {
- matrix_expr_destroy (e);
- return NULL;
- }
- return e;
- }
- else if (lex_token (s->lexer) == T_ID)
- {
- struct matrix_expr *retval;
- if (matrix_parse_function (s, lex_tokcstr (s->lexer), &retval))
- return retval;
-
- struct matrix_var *var = matrix_var_lookup (s, lex_tokss (s->lexer));
- if (!var)
- {
- lex_error (s->lexer, _("Unknown variable %s."),
- lex_tokcstr (s->lexer));
- return NULL;
- }
- lex_get (s->lexer);
-
- struct matrix_expr *e = xmalloc (sizeof *e);
- *e = (struct matrix_expr) { .op = MOP_VARIABLE, .variable = var };
- return e;
- }
- else if (lex_token (s->lexer) == T_ALL)
- {
- struct matrix_expr *retval;
- if (matrix_parse_function (s, "ALL", &retval))
- return retval;
- }
-
- lex_error (s->lexer, _("Syntax error expecting matrix expression."));
- return NULL;
-}
-
-static struct matrix_expr *
-matrix_parse_primary (struct matrix_state *s)
-{
- int start_ofs = lex_ofs (s->lexer);
- struct matrix_expr *e = matrix_parse_primary__ (s);
- matrix_expr_add_location (s, start_ofs, e);
- return e;
-}
-
-static struct matrix_expr *matrix_parse_postfix (struct matrix_state *);
-
-static bool
-matrix_parse_index_expr (struct matrix_state *s,
- struct matrix_expr **indexp,
- struct msg_location **locationp)
-{
- if (lex_match (s->lexer, T_COLON))
- {
- if (locationp)
- *locationp = lex_get_location (s->lexer, -1, -1);
- *indexp = NULL;
- return true;
- }
- else
- {
- *indexp = matrix_expr_parse (s);
- if (locationp && *indexp)
- *locationp = msg_location_dup (matrix_expr_location (*indexp));
- return *indexp != NULL;
- }
-}
-
-static struct matrix_expr *
-matrix_parse_postfix (struct matrix_state *s)
-{
- struct matrix_expr *lhs = matrix_parse_primary (s);
- if (!lhs || !lex_match (s->lexer, T_LPAREN))
- return lhs;
-
- struct matrix_expr *i0;
- if (!matrix_parse_index_expr (s, &i0, NULL))
- {
- matrix_expr_destroy (lhs);
- return NULL;
- }
- if (lex_match (s->lexer, T_RPAREN))
- return (i0
- ? matrix_expr_create_2 (MOP_VEC_INDEX, lhs, i0)
- : matrix_expr_create_1 (MOP_VEC_ALL, lhs));
- else if (lex_match (s->lexer, T_COMMA))
- {
- struct matrix_expr *i1;
- if (!matrix_parse_index_expr (s, &i1, NULL)
- || !lex_force_match (s->lexer, T_RPAREN))
- {
- matrix_expr_destroy (lhs);
- matrix_expr_destroy (i0);
- matrix_expr_destroy (i1);
- return NULL;
- }
- return (i0 && i1 ? matrix_expr_create_3 (MOP_MAT_INDEX, lhs, i0, i1)
- : i0 ? matrix_expr_create_2 (MOP_ROW_INDEX, lhs, i0)
- : i1 ? matrix_expr_create_2 (MOP_COL_INDEX, lhs, i1)
- : lhs);
- }
- else
- {
- lex_error_expecting (s->lexer, "`)'", "`,'");
- return NULL;
- }
-}
-
-static struct matrix_expr *
-matrix_parse_unary (struct matrix_state *s)
-{
- int start_ofs = lex_ofs (s->lexer);
-
- struct matrix_expr *e;
- if (lex_match (s->lexer, T_DASH))
- {
- struct matrix_expr *sub = matrix_parse_unary (s);
- if (!sub)
- return NULL;
- e = matrix_expr_create_1 (MOP_NEGATE, sub);
- }
- else if (lex_match (s->lexer, T_PLUS))
- {
- e = matrix_parse_unary (s);
- if (!e)
- return NULL;
- }
- else
- return matrix_parse_postfix (s);
-
- matrix_expr_add_location (s, start_ofs, e);
- e->location->start = lex_ofs_start_point (s->lexer, start_ofs);
- return e;
-}
-
-static struct matrix_expr *
-matrix_parse_seq (struct matrix_state *s)
-{
- struct matrix_expr *start = matrix_parse_unary (s);
- if (!start || !lex_match (s->lexer, T_COLON))
- return start;
-
- struct matrix_expr *end = matrix_parse_unary (s);
- if (!end)
- {
- matrix_expr_destroy (start);
- return NULL;
- }
-
- if (lex_match (s->lexer, T_COLON))
- {
- struct matrix_expr *increment = matrix_parse_unary (s);
- if (!increment)
- {
- matrix_expr_destroy (start);
- matrix_expr_destroy (end);
- return NULL;
- }
- return matrix_expr_create_3 (MOP_SEQ_BY, start, end, increment);
- }
- else
- return matrix_expr_create_2 (MOP_SEQ, start, end);
-}
-
-static struct matrix_expr *
-matrix_parse_exp (struct matrix_state *s)
-{
- static const struct matrix_operator_syntax syntax[] = {
- { .token = T_EXP, .op = MOP_EXP_MAT },
- { .phrase = "&**", .op = MOP_EXP_ELEMS },
- };
- size_t n_syntax = sizeof syntax / sizeof *syntax;
-
- return matrix_parse_binary_operator (s, matrix_parse_seq, syntax, n_syntax);
-}
-
-static struct matrix_expr *
-matrix_parse_mul_div (struct matrix_state *s)
-{
- static const struct matrix_operator_syntax syntax[] = {
- { .token = T_ASTERISK, .op = MOP_MUL_MAT },
- { .token = T_SLASH, .op = MOP_DIV_ELEMS },
- { .phrase = "&*", .op = MOP_MUL_ELEMS },
- { .phrase = "&/", .op = MOP_DIV_ELEMS },
- };
- size_t n_syntax = sizeof syntax / sizeof *syntax;
-
- return matrix_parse_binary_operator (s, matrix_parse_exp, syntax, n_syntax);
-}
-
-static struct matrix_expr *
-matrix_parse_add_sub (struct matrix_state *s)
-{
- struct matrix_expr *lhs = matrix_parse_mul_div (s);
- if (!lhs)
- return NULL;
-
- for (;;)
- {
- enum matrix_op op;
- if (lex_match (s->lexer, T_PLUS))
- op = MOP_ADD_ELEMS;
- else if (lex_match (s->lexer, T_DASH))
- op = MOP_SUB_ELEMS;
- else if (lex_token (s->lexer) == T_NEG_NUM)
- op = MOP_ADD_ELEMS;
- else
- return lhs;
-
- struct matrix_expr *rhs = matrix_parse_mul_div (s);
- if (!rhs)
- {
- matrix_expr_destroy (lhs);
- return NULL;
- }
- lhs = matrix_expr_create_2 (op, lhs, rhs);
- }
-}
-
-static struct matrix_expr *
-matrix_parse_relational (struct matrix_state *s)
-{
- static const struct matrix_operator_syntax syntax[] = {
- { .token = T_GT, .op = MOP_GT },
- { .token = T_GE, .op = MOP_GE },
- { .token = T_LT, .op = MOP_LT },
- { .token = T_LE, .op = MOP_LE },
- { .token = T_EQUALS, .op = MOP_EQ },
- { .token = T_EQ, .op = MOP_EQ },
- { .token = T_NE, .op = MOP_NE },
- };
- size_t n_syntax = sizeof syntax / sizeof *syntax;
-
- return matrix_parse_binary_operator (s, matrix_parse_add_sub,
- syntax, n_syntax);
-}
-
-static struct matrix_expr *
-matrix_parse_not (struct matrix_state *s)
-{
- int start_ofs = lex_ofs (s->lexer);
- if (lex_match (s->lexer, T_NOT))
- {
- struct matrix_expr *sub = matrix_parse_not (s);
- if (!sub)
- return NULL;
-
- struct matrix_expr *e = matrix_expr_create_1 (MOP_NOT, sub);
- matrix_expr_add_location (s, start_ofs, e);
- e->location->start = lex_ofs_start_point (s->lexer, start_ofs);
- return e;
- }
- else
- return matrix_parse_relational (s);
-}
-
-static struct matrix_expr *
-matrix_parse_and (struct matrix_state *s)
-{
- static const struct matrix_operator_syntax op = {
- .token = T_AND, .op = MOP_AND
- };
-
- return matrix_parse_binary_operator (s, matrix_parse_not, &op, 1);
-}
-
-static struct matrix_expr *
-matrix_expr_parse__ (struct matrix_state *s)
-{
- static const struct matrix_operator_syntax syntax[] = {
- { .token = T_OR, .op = MOP_OR },
- { .id = "XOR", .op = MOP_XOR },
- };
- size_t n_syntax = sizeof syntax / sizeof *syntax;
-
- return matrix_parse_binary_operator (s, matrix_parse_and, syntax, n_syntax);
-}
-
-static struct matrix_expr *
-matrix_expr_parse (struct matrix_state *s)
-{
- int start_ofs = lex_ofs (s->lexer);
- struct matrix_expr *e = matrix_expr_parse__ (s);
- matrix_expr_add_location (s, start_ofs, e);
- return e;
-}
-\f
-/* Matrix expression evaluation. */
-
-/* Iterates over all the elements in matrix M, setting Y and X to the row and
- column indexes, respectively, and pointing D to the entry at each
- position. */
-#define MATRIX_FOR_ALL_ELEMENTS(D, Y, X, M) \
- for (size_t Y = 0; Y < (M)->size1; Y++) \
- for (size_t X = 0; X < (M)->size2; X++) \
- for (double *D = gsl_matrix_ptr ((M), Y, X); D; D = NULL)
-
-static bool
-is_vector (const gsl_matrix *m)
-{
- return m->size1 <= 1 || m->size2 <= 1;
-}
-
-static gsl_vector
-to_vector (gsl_matrix *m)
-{
- return (m->size1 == 1
- ? gsl_matrix_row (m, 0).vector
- : gsl_matrix_column (m, 0).vector);
-}
-
-static double
-matrix_eval_ABS (double d)
-{
- return fabs (d);
-}
-
-static double
-matrix_eval_ALL (gsl_matrix *m)
-{
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- if (*d == 0.0)
- return 0.0;
- return 1.0;
-}
-
-static double
-matrix_eval_ANY (gsl_matrix *m)
-{
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- if (*d != 0.0)
- return 1.0;
- return 0.0;
-}
-
-static double
-matrix_eval_ARSIN (double d)
-{
- return asin (d);
-}
-
-static double
-matrix_eval_ARTAN (double d)
-{
- return atan (d);
-}
-
-static gsl_matrix *
-matrix_eval_BLOCK (gsl_matrix *m[], size_t n)
-{
- size_t r = 0;
- size_t c = 0;
- for (size_t i = 0; i < n; i++)
- {
- r += m[i]->size1;
- c += m[i]->size2;
- }
- gsl_matrix *block = gsl_matrix_calloc (r, c);
- r = c = 0;
- for (size_t i = 0; i < n; i++)
- {
- for (size_t y = 0; y < m[i]->size1; y++)
- for (size_t x = 0; x < m[i]->size2; x++)
- gsl_matrix_set (block, r + y, c + x, gsl_matrix_get (m[i], y, x));
- r += m[i]->size1;
- c += m[i]->size2;
- }
- return block;
-}
-
-static gsl_matrix *
-matrix_eval_CHOL (gsl_matrix *m, const struct matrix_expr *e)
-{
- if (!gsl_linalg_cholesky_decomp1 (m))
- {
- for (size_t y = 0; y < m->size1; y++)
- for (size_t x = y + 1; x < m->size2; x++)
- gsl_matrix_set (m, y, x, gsl_matrix_get (m, x, y));
-
- for (size_t y = 0; y < m->size1; y++)
- for (size_t x = 0; x < y; x++)
- gsl_matrix_set (m, y, x, 0);
- return m;
- }
- else
- {
- msg_at (SE, e->subs[0]->location,
- _("Input to CHOL function is not positive-definite."));
- return NULL;
- }
-}
-
-static gsl_matrix *
-matrix_eval_col_extremum (gsl_matrix *m, bool min)
-{
- if (m->size1 <= 1)
- return m;
- else if (!m->size2)
- return gsl_matrix_alloc (1, 0);
-
- gsl_matrix *cext = gsl_matrix_alloc (1, m->size2);
- for (size_t x = 0; x < m->size2; x++)
- {
- double ext = gsl_matrix_get (m, 0, x);
- for (size_t y = 1; y < m->size1; y++)
- {
- double value = gsl_matrix_get (m, y, x);
- if (min ? value < ext : value > ext)
- ext = value;
- }
- gsl_matrix_set (cext, 0, x, ext);
- }
- return cext;
-}
-
-static gsl_matrix *
-matrix_eval_CMAX (gsl_matrix *m)
-{
- return matrix_eval_col_extremum (m, false);
-}
-
-static gsl_matrix *
-matrix_eval_CMIN (gsl_matrix *m)
-{
- return matrix_eval_col_extremum (m, true);
-}
-
-static double
-matrix_eval_COS (double d)
-{
- return cos (d);
-}
-
-static gsl_matrix *
-matrix_eval_col_sum (gsl_matrix *m, bool square)
-{
- if (m->size1 == 0)
- return m;
- else if (!m->size2)
- return gsl_matrix_alloc (1, 0);
-
- gsl_matrix *result = gsl_matrix_alloc (1, m->size2);
- for (size_t x = 0; x < m->size2; x++)
- {
- double sum = 0;
- for (size_t y = 0; y < m->size1; y++)
- {
- double d = gsl_matrix_get (m, y, x);
- sum += square ? pow2 (d) : d;
- }
- gsl_matrix_set (result, 0, x, sum);
- }
- return result;
-}
-
-static gsl_matrix *
-matrix_eval_CSSQ (gsl_matrix *m)
-{
- return matrix_eval_col_sum (m, true);
-}
-
-static gsl_matrix *
-matrix_eval_CSUM (gsl_matrix *m)
-{
- return matrix_eval_col_sum (m, false);
-}
-
-static int
-compare_double_3way (const void *a_, const void *b_)
-{
- const double *a = a_;
- const double *b = b_;
- return *a < *b ? -1 : *a > *b;
-}
-
-static gsl_matrix *
-matrix_eval_DESIGN (gsl_matrix *m, const struct matrix_expr *e)
-{
- double *tmp = xmalloc (m->size1 * m->size2 * sizeof *tmp);
- gsl_matrix m2 = gsl_matrix_view_array (tmp, m->size2, m->size1).matrix;
- gsl_matrix_transpose_memcpy (&m2, m);
-
- for (size_t y = 0; y < m2.size1; y++)
- qsort (tmp + y * m2.size2, m2.size2, sizeof *tmp, compare_double_3way);
-
- size_t *n = xcalloc (m2.size1, sizeof *n);
- size_t n_total = 0;
- for (size_t i = 0; i < m2.size1; i++)
- {
- double *row = tmp + m2.size2 * i;
- for (size_t j = 0; j < m2.size2; )
- {
- size_t k;
- for (k = j + 1; k < m2.size2; k++)
- if (row[j] != row[k])
- break;
- row[n[i]++] = row[j];
- j = k;
- }
-
- if (n[i] <= 1)
- msg_at (MW, e->subs[0]->location,
- _("Column %zu in DESIGN argument has constant value."), i + 1);
- else
- n_total += n[i];
- }
-
- gsl_matrix *result = gsl_matrix_alloc (m->size1, n_total);
- size_t x = 0;
- for (size_t i = 0; i < m->size2; i++)
- {
- if (n[i] <= 1)
- continue;
-
- const double *unique = tmp + m2.size2 * i;
- for (size_t j = 0; j < n[i]; j++, x++)
- {
- double value = unique[j];
- for (size_t y = 0; y < m->size1; y++)
- gsl_matrix_set (result, y, x, gsl_matrix_get (m, y, i) == value);
- }
- }
-
- free (n);
- free (tmp);
-
- return result;
-}
-
-static double
-matrix_eval_DET (gsl_matrix *m)
-{
- gsl_permutation *p = gsl_permutation_alloc (m->size1);
- int signum;
- gsl_linalg_LU_decomp (m, p, &signum);
- gsl_permutation_free (p);
- return gsl_linalg_LU_det (m, signum);
-}
-
-static gsl_matrix *
-matrix_eval_DIAG (gsl_matrix *m)
-{
- gsl_matrix *diag = gsl_matrix_alloc (MIN (m->size1, m->size2), 1);
- for (size_t i = 0; i < diag->size1; i++)
- gsl_matrix_set (diag, i, 0, gsl_matrix_get (m, i, i));
- return diag;
-}
-
-static bool
-is_symmetric (const gsl_matrix *m)
-{
- if (m->size1 != m->size2)
- return false;
-
- for (size_t y = 0; y < m->size1; y++)
- for (size_t x = 0; x < y; x++)
- if (gsl_matrix_get (m, y, x) != gsl_matrix_get (m, x, y))
- return false;
-
- return true;
-}
-
-static int
-compare_double_desc (const void *a_, const void *b_)
-{
- const double *a = a_;
- const double *b = b_;
- return *a > *b ? -1 : *a < *b;
-}
-
-static gsl_matrix *
-matrix_eval_EVAL (gsl_matrix *m, const struct matrix_expr *e)
-{
- if (!is_symmetric (m))
- {
- msg_at (SE, e->subs[0]->location,
- _("Argument of EVAL must be symmetric."));
- return NULL;
- }
-
- gsl_eigen_symm_workspace *w = gsl_eigen_symm_alloc (m->size1);
- gsl_matrix *eval = gsl_matrix_alloc (m->size1, 1);
- gsl_vector v_eval = to_vector (eval);
- gsl_eigen_symm (m, &v_eval, w);
- gsl_eigen_symm_free (w);
-
- assert (v_eval.stride == 1);
- qsort (v_eval.data, v_eval.size, sizeof *v_eval.data, compare_double_desc);
-
- return eval;
-}
-
-static double
-matrix_eval_EXP (double d)
-{
- return exp (d);
-}
-
-/* From https://gist.github.com/turingbirds/5e99656e08dbe1324c99, where it was
- marked as:
-
- Charl Linssen <charl@itfromb.it>
- Feb 2016
- PUBLIC DOMAIN */
-static gsl_matrix *
-matrix_eval_GINV (gsl_matrix *A)
-{
- size_t n = A->size1;
- size_t m = A->size2;
- bool swap = m > n;
- gsl_matrix *tmp_mat = NULL;
- if (swap)
- {
- /* libgsl SVD can only handle the case m <= n, so transpose matrix. */
- tmp_mat = gsl_matrix_alloc (m, n);
- gsl_matrix_transpose_memcpy (tmp_mat, A);
- A = tmp_mat;
- size_t i = m;
- m = n;
- n = i;
- }
-
- /* Do SVD. */
- gsl_matrix *V = gsl_matrix_alloc (m, m);
- gsl_vector *u = gsl_vector_alloc (m);
-
- gsl_vector *tmp_vec = gsl_vector_alloc (m);
- gsl_linalg_SV_decomp (A, V, u, tmp_vec);
- gsl_vector_free (tmp_vec);
-
- /* Compute Σ⁻¹. */
- gsl_matrix *Sigma_pinv = gsl_matrix_alloc (m, n);
- gsl_matrix_set_zero (Sigma_pinv);
- double cutoff = 1e-15 * gsl_vector_max (u);
-
- for (size_t i = 0; i < m; ++i)
- {
- double x = gsl_vector_get (u, i);
- gsl_matrix_set (Sigma_pinv, i, i, x > cutoff ? 1.0 / x : 0);
- }
-
- /* libgsl SVD yields "thin" SVD. Pad to full matrix by adding zeros. */
- gsl_matrix *U = gsl_matrix_calloc (n, n);
- for (size_t i = 0; i < n; i++)
- for (size_t j = 0; j < m; j++)
- gsl_matrix_set (U, i, j, gsl_matrix_get (A, i, j));
-
- /* Two dot products to obtain pseudoinverse. */
- gsl_matrix *tmp_mat2 = gsl_matrix_alloc (m, n);
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1., V, Sigma_pinv, 0., tmp_mat2);
-
- gsl_matrix *A_pinv;
- if (swap)
- {
- A_pinv = gsl_matrix_alloc (n, m);
- gsl_blas_dgemm (CblasNoTrans, CblasTrans, 1., U, tmp_mat2, 0., A_pinv);
- }
- else
- {
- A_pinv = gsl_matrix_alloc (m, n);
- gsl_blas_dgemm (CblasNoTrans, CblasTrans, 1., tmp_mat2, U, 0., A_pinv);
- }
-
- gsl_matrix_free (tmp_mat);
- gsl_matrix_free (tmp_mat2);
- gsl_matrix_free (U);
- gsl_matrix_free (Sigma_pinv);
- gsl_vector_free (u);
- gsl_matrix_free (V);
-
- return A_pinv;
-}
-
-struct grade
- {
- size_t y, x;
- double value;
- };
-
-static int
-grade_compare_3way (const void *a_, const void *b_)
-{
- const struct grade *a = a_;
- const struct grade *b = b_;
-
- return (a->value < b->value ? -1
- : a->value > b->value ? 1
- : a->y < b->y ? -1
- : a->y > b->y ? 1
- : a->x < b->x ? -1
- : a->x > b->x);
-}
-
-static gsl_matrix *
-matrix_eval_GRADE (gsl_matrix *m)
-{
- size_t n = m->size1 * m->size2;
- struct grade *grades = xmalloc (n * sizeof *grades);
-
- size_t i = 0;
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- grades[i++] = (struct grade) { .y = y, .x = x, .value = *d };
- qsort (grades, n, sizeof *grades, grade_compare_3way);
-
- for (size_t i = 0; i < n; i++)
- gsl_matrix_set (m, grades[i].y, grades[i].x, i + 1);
-
- free (grades);
-
- return m;
-}
-
-static double
-dot (gsl_vector *a, gsl_vector *b)
-{
- double result = 0.0;
- for (size_t i = 0; i < a->size; i++)
- result += gsl_vector_get (a, i) * gsl_vector_get (b, i);
- return result;
-}
-
-static double
-norm2 (gsl_vector *v)
-{
- double result = 0.0;
- for (size_t i = 0; i < v->size; i++)
- result += pow2 (gsl_vector_get (v, i));
- return result;
-}
-
-static double
-norm (gsl_vector *v)
-{
- return sqrt (norm2 (v));
-}
-
-static gsl_matrix *
-matrix_eval_GSCH (gsl_matrix *v, const struct matrix_expr *e)
-{
- if (v->size2 < v->size1)
- {
- msg_at (SE, e->subs[0]->location,
- _("GSCH requires its argument to have at least as many columns "
- "as rows, but it has dimensions %zu×%zu."),
- v->size1, v->size2);
- return NULL;
- }
- if (!v->size1 || !v->size2)
- return v;
-
- gsl_matrix *u = gsl_matrix_calloc (v->size1, v->size2);
- size_t ux = 0;
- for (size_t vx = 0; vx < v->size2; vx++)
- {
- gsl_vector u_i = gsl_matrix_column (u, ux).vector;
- gsl_vector v_i = gsl_matrix_column (v, vx).vector;
-
- gsl_vector_memcpy (&u_i, &v_i);
- for (size_t j = 0; j < ux; j++)
- {
- gsl_vector u_j = gsl_matrix_column (u, j).vector;
- double scale = dot (&u_j, &u_i) / norm2 (&u_j);
- for (size_t k = 0; k < u_i.size; k++)
- gsl_vector_set (&u_i, k, (gsl_vector_get (&u_i, k)
- - scale * gsl_vector_get (&u_j, k)));
- }
-
- double len = norm (&u_i);
- if (len > 1e-15)
- {
- gsl_vector_scale (&u_i, 1.0 / len);
- if (++ux >= v->size1)
- break;
- }
- }
-
- if (ux < v->size1)
- {
- msg_at (SE, e->subs[0]->location,
- _("%zu×%zu argument to GSCH contains only "
- "%zu linearly independent columns."),
- v->size1, v->size2, ux);
- gsl_matrix_free (u);
- return NULL;
- }
-
- u->size2 = v->size1;
- return u;
-}
-
-static gsl_matrix *
-matrix_eval_IDENT (double s1, double s2)
-{
- gsl_matrix *m = gsl_matrix_alloc (s1, s2);
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- *d = x == y;
- return m;
-}
-
-/* Inverts X, storing the inverse into INVERSE. As a side effect, replaces X
- by its LU decomposition. */
-static void
-invert_matrix (gsl_matrix *x, gsl_matrix *inverse)
-{
- gsl_permutation *p = gsl_permutation_alloc (x->size1);
- int signum;
- gsl_linalg_LU_decomp (x, p, &signum);
- gsl_linalg_LU_invert (x, p, inverse);
- gsl_permutation_free (p);
-}
-
-static gsl_matrix *
-matrix_eval_INV (gsl_matrix *src)
-{
- gsl_matrix *dst = gsl_matrix_alloc (src->size1, src->size2);
- invert_matrix (src, dst);
- return dst;
-}
-
-static gsl_matrix *
-matrix_eval_KRONEKER (gsl_matrix *a, gsl_matrix *b)
-{
- gsl_matrix *k = gsl_matrix_alloc (a->size1 * b->size1,
- a->size2 * b->size2);
- size_t y = 0;
- for (size_t ar = 0; ar < a->size1; ar++)
- for (size_t br = 0; br < b->size1; br++, y++)
- {
- size_t x = 0;
- for (size_t ac = 0; ac < a->size2; ac++)
- for (size_t bc = 0; bc < b->size2; bc++, x++)
- {
- double av = gsl_matrix_get (a, ar, ac);
- double bv = gsl_matrix_get (b, br, bc);
- gsl_matrix_set (k, y, x, av * bv);
- }
- }
- return k;
-}
-
-static double
-matrix_eval_LG10 (double d)
-{
- return log10 (d);
-}
-
-static double
-matrix_eval_LN (double d)
-{
- return log (d);
-}
-
-static void
-matrix_eval_MAGIC_odd (gsl_matrix *m, size_t n)
-{
- /* Siamese method: https://en.wikipedia.org/wiki/Siamese_method. */
- size_t y = 0;
- size_t x = n / 2;
- for (size_t i = 1; i <= n * n; i++)
- {
- gsl_matrix_set (m, y, x, i);
-
- size_t y1 = !y ? n - 1 : y - 1;
- size_t x1 = x + 1 >= n ? 0 : x + 1;
- if (gsl_matrix_get (m, y1, x1) == 0)
- {
- y = y1;
- x = x1;
- }
- else
- y = y + 1 >= n ? 0 : y + 1;
- }
-}
-
-static void
-magic_exchange (gsl_matrix *m, size_t y1, size_t x1, size_t y2, size_t x2)
-{
- double a = gsl_matrix_get (m, y1, x1);
- double b = gsl_matrix_get (m, y2, x2);
- gsl_matrix_set (m, y1, x1, b);
- gsl_matrix_set (m, y2, x2, a);
-}
-
-static void
-matrix_eval_MAGIC_doubly_even (gsl_matrix *m, size_t n)
-{
- size_t x, y;
-
- /* A. Umar, "On the Construction of Even Order Magic Squares",
- https://arxiv.org/ftp/arxiv/papers/1202/1202.0948.pdf. */
- x = y = 0;
- for (size_t i = 1; i <= n * n / 2; i++)
- {
- gsl_matrix_set (m, y, x, i);
- if (++y >= n)
- {
- y = 0;
- x++;
- }
- }
-
- x = n - 1;
- y = 0;
- for (size_t i = n * n; i > n * n / 2; i--)
- {
- gsl_matrix_set (m, y, x, i);
- if (++y >= n)
- {
- y = 0;
- x--;
- }
- }
-
- for (size_t y = 0; y < n; y++)
- for (size_t x = 0; x < n / 2; x++)
- {
- unsigned int d = gsl_matrix_get (m, y, x);
- if (d % 2 != (y < n / 2))
- magic_exchange (m, y, x, y, n - x - 1);
- }
-
- size_t y1 = n / 2;
- size_t y2 = n - 1;
- size_t x1 = n / 2 - 1;
- size_t x2 = n / 2;
- magic_exchange (m, y1, x1, y2, x1);
- magic_exchange (m, y1, x2, y2, x2);
-}
-
-static void
-matrix_eval_MAGIC_singly_even (gsl_matrix *m, size_t n)
-{
- /* A. Umar, "On the Construction of Even Order Magic Squares",
- https://arxiv.org/ftp/arxiv/papers/1202/1202.0948.pdf. */
- size_t x, y;
-
- x = y = 0;
- for (size_t i = 1; ; i++)
- {
- gsl_matrix_set (m, y, x, i);
- if (++y == n / 2 - 1)
- y += 2;
- else if (y >= n)
- {
- y = 0;
- if (++x >= n / 2)
- break;
- }
- }
-
- x = n - 1;
- y = 0;
- for (size_t i = n * n; ; i--)
- {
- gsl_matrix_set (m, y, x, i);
- if (++y == n / 2 - 1)
- y += 2;
- else if (y >= n)
- {
- y = 0;
- if (--x < n / 2)
- break;
- }
- }
- for (size_t y = 0; y < n; y++)
- if (y != n / 2 - 1 && y != n / 2)
- for (size_t x = 0; x < n / 2; x++)
- {
- unsigned int d = gsl_matrix_get (m, y, x);
- if (d % 2 != (y < n / 2))
- magic_exchange (m, y, x, y, n - x - 1);
- }
-
- size_t a0 = (n * n - 2 * n) / 2 + 1;
- for (size_t i = 0; i < n / 2; i++)
- {
- size_t a = a0 + i;
- gsl_matrix_set (m, n / 2, i, a);
- gsl_matrix_set (m, n / 2 - 1, i, (n * n + 1) - a);
- }
- for (size_t i = 0; i < n / 2; i++)
- {
- size_t a = a0 + i + n / 2;
- gsl_matrix_set (m, n / 2 - 1, n - i - 1, a);
- gsl_matrix_set (m, n / 2, n - i - 1, (n * n + 1) - a);
- }
- for (size_t x = 1; x < n / 2; x += 2)
- magic_exchange (m, n / 2, x, n / 2 - 1, x);
- for (size_t x = n / 2 + 2; x <= n - 3; x += 2)
- magic_exchange (m, n / 2, x, n / 2 - 1, x);
- size_t x1 = n / 2 - 2;
- size_t x2 = n / 2 + 1;
- size_t y1 = n / 2 - 2;
- size_t y2 = n / 2 + 1;
- magic_exchange (m, y1, x1, y2, x1);
- magic_exchange (m, y1, x2, y2, x2);
-}
-
-static gsl_matrix *
-matrix_eval_MAGIC (double n_)
-{
- size_t n = n_;
-
- gsl_matrix *m = gsl_matrix_calloc (n, n);
- if (n % 2)
- matrix_eval_MAGIC_odd (m, n);
- else if (n % 4)
- matrix_eval_MAGIC_singly_even (m, n);
- else
- matrix_eval_MAGIC_doubly_even (m, n);
- return m;
-}
-
-static gsl_matrix *
-matrix_eval_MAKE (double r, double c, double value)
-{
- gsl_matrix *m = gsl_matrix_alloc (r, c);
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- *d = value;
- return m;
-}
-
-static gsl_matrix *
-matrix_eval_MDIAG (gsl_vector *v)
-{
- gsl_matrix *m = gsl_matrix_calloc (v->size, v->size);
- gsl_vector diagonal = gsl_matrix_diagonal (m).vector;
- gsl_vector_memcpy (&diagonal, v);
- return m;
-}
-
-static double
-matrix_eval_MMAX (gsl_matrix *m)
-{
- return gsl_matrix_max (m);
-}
-
-static double
-matrix_eval_MMIN (gsl_matrix *m)
-{
- return gsl_matrix_min (m);
-}
-
-static gsl_matrix *
-matrix_eval_MOD (gsl_matrix *m, double divisor)
-{
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- *d = fmod (*d, divisor);
- return m;
-}
-
-static double
-matrix_eval_MSSQ (gsl_matrix *m)
-{
- double mssq = 0.0;
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- mssq += *d * *d;
- return mssq;
-}
-
-static double
-matrix_eval_MSUM (gsl_matrix *m)
-{
- double msum = 0.0;
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- msum += *d;
- return msum;
-}
-
-static double
-matrix_eval_NCOL (gsl_matrix *m)
-{
- return m->size2;
-}
-
-static double
-matrix_eval_NROW (gsl_matrix *m)
-{
- return m->size1;
-}
-
-static double
-matrix_eval_RANK (gsl_matrix *m)
-{
- gsl_vector *tau = gsl_vector_alloc (MIN (m->size1, m->size2));
- gsl_linalg_QR_decomp (m, tau);
- gsl_vector_free (tau);
-
- return gsl_linalg_QRPT_rank (m, -1);
-}
-
-static gsl_matrix *
-matrix_eval_RESHAPE (gsl_matrix *m, double r_, double c_,
- const struct matrix_expr *e)
-{
- bool r_ok = r_ >= 0 && r_ < SIZE_MAX;
- bool c_ok = c_ >= 0 && c_ < SIZE_MAX;
- if (!r_ok || !c_ok)
- {
- msg_at (SE,
- !r_ok ? e->subs[1]->location : e->subs[2]->location,
- _("Arguments 2 and 3 to RESHAPE must be integers."));
- return NULL;
- }
- size_t r = r_;
- size_t c = c_;
- if (size_overflow_p (xtimes (r, xmax (c, 1))) || c * r != m->size1 * m->size2)
- {
- struct msg_location *loc = msg_location_dup (e->subs[1]->location);
- loc->end = e->subs[2]->location->end;
- msg_at (SE, loc, _("Product of RESHAPE size arguments (%zu×%zu = %zu) "
- "differs from product of matrix dimensions "
- "(%zu×%zu = %zu)."),
- r, c, r * c,
- m->size1, m->size2, m->size1 * m->size2);
- msg_location_destroy (loc);
- return NULL;
- }
-
- gsl_matrix *dst = gsl_matrix_alloc (r, c);
- size_t y1 = 0;
- size_t x1 = 0;
- MATRIX_FOR_ALL_ELEMENTS (d, y2, x2, m)
- {
- gsl_matrix_set (dst, y1, x1, *d);
- if (++x1 >= c)
- {
- x1 = 0;
- y1++;
- }
- }
- return dst;
-}
-
-static gsl_matrix *
-matrix_eval_row_extremum (gsl_matrix *m, bool min)
-{
- if (m->size2 <= 1)
- return m;
- else if (!m->size1)
- return gsl_matrix_alloc (0, 1);
-
- gsl_matrix *rext = gsl_matrix_alloc (m->size1, 1);
- for (size_t y = 0; y < m->size1; y++)
- {
- double ext = gsl_matrix_get (m, y, 0);
- for (size_t x = 1; x < m->size2; x++)
- {
- double value = gsl_matrix_get (m, y, x);
- if (min ? value < ext : value > ext)
- ext = value;
- }
- gsl_matrix_set (rext, y, 0, ext);
- }
- return rext;
-}
-
-static gsl_matrix *
-matrix_eval_RMAX (gsl_matrix *m)
-{
- return matrix_eval_row_extremum (m, false);
-}
-
-static gsl_matrix *
-matrix_eval_RMIN (gsl_matrix *m)
-{
- return matrix_eval_row_extremum (m, true);
-}
-
-static double
-matrix_eval_RND (double d)
-{
- return rint (d);
-}
-
-struct rank
- {
- size_t y, x;
- double value;
- };
-
-static int
-rank_compare_3way (const void *a_, const void *b_)
-{
- const struct rank *a = a_;
- const struct rank *b = b_;
-
- return a->value < b->value ? -1 : a->value > b->value;
-}
-
-static gsl_matrix *
-matrix_eval_RNKORDER (gsl_matrix *m)
-{
- size_t n = m->size1 * m->size2;
- struct rank *ranks = xmalloc (n * sizeof *ranks);
- size_t i = 0;
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- ranks[i++] = (struct rank) { .y = y, .x = x, .value = *d };
- qsort (ranks, n, sizeof *ranks, rank_compare_3way);
-
- for (size_t i = 0; i < n; )
- {
- size_t j;
- for (j = i + 1; j < n; j++)
- if (ranks[i].value != ranks[j].value)
- break;
-
- double rank = (i + j + 1.0) / 2.0;
- for (size_t k = i; k < j; k++)
- gsl_matrix_set (m, ranks[k].y, ranks[k].x, rank);
-
- i = j;
- }
-
- free (ranks);
-
- return m;
-}
-
-static gsl_matrix *
-matrix_eval_row_sum (gsl_matrix *m, bool square)
-{
- if (m->size1 == 0)
- return m;
- else if (!m->size1)
- return gsl_matrix_alloc (0, 1);
-
- gsl_matrix *result = gsl_matrix_alloc (m->size1, 1);
- for (size_t y = 0; y < m->size1; y++)
- {
- double sum = 0;
- for (size_t x = 0; x < m->size2; x++)
- {
- double d = gsl_matrix_get (m, y, x);
- sum += square ? pow2 (d) : d;
- }
- gsl_matrix_set (result, y, 0, sum);
- }
- return result;
-}
-
-static gsl_matrix *
-matrix_eval_RSSQ (gsl_matrix *m)
-{
- return matrix_eval_row_sum (m, true);
-}
-
-static gsl_matrix *
-matrix_eval_RSUM (gsl_matrix *m)
-{
- return matrix_eval_row_sum (m, false);
-}
-
-static double
-matrix_eval_SIN (double d)
-{
- return sin (d);
-}
-
-static gsl_matrix *
-matrix_eval_SOLVE (gsl_matrix *m1, gsl_matrix *m2, const struct matrix_expr *e)
-{
- if (m1->size1 != m2->size1)
- {
- struct msg_location *loc = msg_location_dup (e->subs[0]->location);
- loc->end = e->subs[1]->location->end;
-
- msg_at (SE, e->location,
- _("SOLVE arguments must have the same number of rows."));
- msg_at (SN, e->subs[0]->location,
- _("Argument 1 has dimensions %zu×%zu."), m1->size1, m1->size2);
- msg_at (SN, e->subs[1]->location,
- _("Argument 2 has dimensions %zu×%zu."), m2->size1, m2->size2);
-
- msg_location_destroy (loc);
- return NULL;
- }
-
- gsl_matrix *x = gsl_matrix_alloc (m2->size1, m2->size2);
- gsl_permutation *p = gsl_permutation_alloc (m1->size1);
- int signum;
- gsl_linalg_LU_decomp (m1, p, &signum);
- for (size_t i = 0; i < m2->size2; i++)
- {
- gsl_vector bi = gsl_matrix_column (m2, i).vector;
- gsl_vector xi = gsl_matrix_column (x, i).vector;
- gsl_linalg_LU_solve (m1, p, &bi, &xi);
- }
- gsl_permutation_free (p);
- return x;
-}
-
-static double
-matrix_eval_SQRT (double d)
-{
- return sqrt (d);
-}
-
-static gsl_matrix *
-matrix_eval_SSCP (gsl_matrix *m)
-{
- gsl_matrix *sscp = gsl_matrix_alloc (m->size2, m->size2);
- gsl_blas_dgemm (CblasTrans, CblasNoTrans, 1.0, m, m, 0.0, sscp);
- return sscp;
-}
-
-static gsl_matrix *
-matrix_eval_SVAL (gsl_matrix *m)
-{
- gsl_matrix *tmp_mat = NULL;
- if (m->size2 > m->size1)
- {
- tmp_mat = gsl_matrix_alloc (m->size2, m->size1);
- gsl_matrix_transpose_memcpy (tmp_mat, m);
- m = tmp_mat;
- }
-
- /* Do SVD. */
- gsl_matrix *V = gsl_matrix_alloc (m->size2, m->size2);
- gsl_vector *S = gsl_vector_alloc (m->size2);
- gsl_vector *work = gsl_vector_alloc (m->size2);
- gsl_linalg_SV_decomp (m, V, S, work);
-
- gsl_matrix *vals = gsl_matrix_alloc (m->size2, 1);
- for (size_t i = 0; i < m->size2; i++)
- gsl_matrix_set (vals, i, 0, gsl_vector_get (S, i));
-
- gsl_matrix_free (V);
- gsl_vector_free (S);
- gsl_vector_free (work);
- gsl_matrix_free (tmp_mat);
-
- return vals;
-}
-
-static gsl_matrix *
-matrix_eval_SWEEP (gsl_matrix *m, double d, const struct matrix_expr *e)
-{
- if (d < 1 || d > SIZE_MAX)
- {
- msg_at (SE, e->subs[1]->location,
- _("Scalar argument to SWEEP must be integer."));
- return NULL;
- }
- size_t k = d - 1;
- if (k >= MIN (m->size1, m->size2))
- {
- msg_at (SE, e->subs[1]->location,
- _("Scalar argument to SWEEP must be integer less than or "
- "equal to the smaller of the matrix argument's rows and "
- "columns."));
- return NULL;
- }
-
- double m_kk = gsl_matrix_get (m, k, k);
- if (fabs (m_kk) > 1e-19)
- {
- gsl_matrix *a = gsl_matrix_alloc (m->size1, m->size2);
- MATRIX_FOR_ALL_ELEMENTS (a_ij, i, j, a)
- {
- double m_ij = gsl_matrix_get (m, i, j);
- double m_ik = gsl_matrix_get (m, i, k);
- double m_kj = gsl_matrix_get (m, k, j);
- *a_ij = (i != k && j != k ? m_ij * m_kk - m_ik * m_kj
- : i != k ? -m_ik
- : j != k ? m_kj
- : 1.0) / m_kk;
- }
- return a;
- }
- else
- {
- for (size_t i = 0; i < m->size1; i++)
- {
- gsl_matrix_set (m, i, k, 0);
- gsl_matrix_set (m, k, i, 0);
- }
- return m;
- }
-}
-
-static double
-matrix_eval_TRACE (gsl_matrix *m)
-{
- double sum = 0;
- size_t n = MIN (m->size1, m->size2);
- for (size_t i = 0; i < n; i++)
- sum += gsl_matrix_get (m, i, i);
- return sum;
-}
-
-static gsl_matrix *
-matrix_eval_T (gsl_matrix *m)
-{
- return matrix_eval_TRANSPOS (m);
-}
-
-static gsl_matrix *
-matrix_eval_TRANSPOS (gsl_matrix *m)
-{
- if (m->size1 == m->size2)
- {
- gsl_matrix_transpose (m);
- return m;
- }
- else
- {
- gsl_matrix *t = gsl_matrix_alloc (m->size2, m->size1);
- gsl_matrix_transpose_memcpy (t, m);
- return t;
- }
-}
-
-static double
-matrix_eval_TRUNC (double d)
-{
- return trunc (d);
-}
-
-static gsl_matrix *
-matrix_eval_UNIFORM (double r_, double c_, const struct matrix_expr *e)
-{
- size_t r = r_;
- size_t c = c_;
- if (size_overflow_p (xtimes (r, xmax (c, 1))))
- {
- struct msg_location *loc = msg_location_dup (e->subs[0]->location);
- loc->end = e->subs[1]->location->end;
-
- msg_at (SE, loc,
- _("Product of arguments to UNIFORM exceeds memory size."));
-
- msg_location_destroy (loc);
- return NULL;
- }
-
- gsl_matrix *m = gsl_matrix_alloc (r, c);
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, m)
- *d = gsl_ran_flat (get_rng (), 0, 1);
- return m;
-}
-
-static double
-matrix_eval_PDF_BETA (double x, double a, double b)
-{
- return gsl_ran_beta_pdf (x, a, b);
-}
-
-static double
-matrix_eval_CDF_BETA (double x, double a, double b)
-{
- return gsl_cdf_beta_P (x, a, b);
-}
-
-static double
-matrix_eval_IDF_BETA (double P, double a, double b)
-{
- return gsl_cdf_beta_Pinv (P, a, b);
-}
-
-static double
-matrix_eval_RV_BETA (double a, double b)
-{
- return gsl_ran_beta (get_rng (), a, b);
-}
-
-static double
-matrix_eval_NCDF_BETA (double x, double a, double b, double lambda)
-{
- return ncdf_beta (x, a, b, lambda);
-}
-
-static double
-matrix_eval_NPDF_BETA (double x, double a, double b, double lambda)
-{
- return npdf_beta (x, a, b, lambda);
-}
-
-static double
-matrix_eval_CDF_BVNOR (double x0, double x1, double r)
-{
- return cdf_bvnor (x0, x1, r);
-}
-
-static double
-matrix_eval_PDF_BVNOR (double x0, double x1, double r)
-{
- return gsl_ran_bivariate_gaussian_pdf (x0, x1, 1, 1, r);
-}
-
-static double
-matrix_eval_CDF_CAUCHY (double x, double a, double b)
-{
- return gsl_cdf_cauchy_P ((x - a) / b, 1);
-}
-
-static double
-matrix_eval_IDF_CAUCHY (double P, double a, double b)
-{
- return a + b * gsl_cdf_cauchy_Pinv (P, 1);
-}
-
-static double
-matrix_eval_PDF_CAUCHY (double x, double a, double b)
-{
- return gsl_ran_cauchy_pdf ((x - a) / b, 1) / b;
-}
-
-static double
-matrix_eval_RV_CAUCHY (double a, double b)
-{
- return a + b * gsl_ran_cauchy (get_rng (), 1);
-}
-
-static double
-matrix_eval_CDF_CHISQ (double x, double df)
-{
- return gsl_cdf_chisq_P (x, df);
-}
-
-static double
-matrix_eval_CHICDF (double x, double df)
-{
- return matrix_eval_CDF_CHISQ (x, df);
-}
-
-static double
-matrix_eval_IDF_CHISQ (double P, double df)
-{
- return gsl_cdf_chisq_Pinv (P, df);
-}
-
-static double
-matrix_eval_PDF_CHISQ (double x, double df)
-{
- return gsl_ran_chisq_pdf (x, df);
-}
-
-static double
-matrix_eval_RV_CHISQ (double df)
-{
- return gsl_ran_chisq (get_rng (), df);
-}
-
-static double
-matrix_eval_SIG_CHISQ (double x, double df)
-{
- return gsl_cdf_chisq_Q (x, df);
-}
-
-static double
-matrix_eval_CDF_EXP (double x, double a)
-{
- return gsl_cdf_exponential_P (x, 1. / a);
-}
-
-static double
-matrix_eval_IDF_EXP (double P, double a)
-{
- return gsl_cdf_exponential_Pinv (P, 1. / a);
-}
-
-static double
-matrix_eval_PDF_EXP (double x, double a)
-{
- return gsl_ran_exponential_pdf (x, 1. / a);
-}
-
-static double
-matrix_eval_RV_EXP (double a)
-{
- return gsl_ran_exponential (get_rng (), 1. / a);
-}
-
-static double
-matrix_eval_PDF_XPOWER (double x, double a, double b)
-{
- return gsl_ran_exppow_pdf (x, a, b);
-}
-
-static double
-matrix_eval_RV_XPOWER (double a, double b)
-{
- return gsl_ran_exppow (get_rng (), a, b);
-}
-
-static double
-matrix_eval_CDF_F (double x, double df1, double df2)
-{
- return gsl_cdf_fdist_P (x, df1, df2);
-}
-
-static double
-matrix_eval_FCDF (double x, double df1, double df2)
-{
- return matrix_eval_CDF_F (x, df1, df2);
-}
-
-static double
-matrix_eval_IDF_F (double P, double df1, double df2)
-{
- return idf_fdist (P, df1, df2);
-}
-
-static double
-matrix_eval_RV_F (double df1, double df2)
-{
- return gsl_ran_fdist (get_rng (), df1, df2);
-}
-
-static double
-matrix_eval_PDF_F (double x, double df1, double df2)
-{
- return gsl_ran_fdist_pdf (x, df1, df2);
-}
-
-static double
-matrix_eval_SIG_F (double x, double df1, double df2)
-{
- return gsl_cdf_fdist_Q (x, df1, df2);
-}
-
-static double
-matrix_eval_CDF_GAMMA (double x, double a, double b)
-{
- return gsl_cdf_gamma_P (x, a, 1. / b);
-}
-
-static double
-matrix_eval_IDF_GAMMA (double P, double a, double b)
-{
- return gsl_cdf_gamma_Pinv (P, a, 1. / b);
-}
-
-static double
-matrix_eval_PDF_GAMMA (double x, double a, double b)
-{
- return gsl_ran_gamma_pdf (x, a, 1. / b);
-}
-
-static double
-matrix_eval_RV_GAMMA (double a, double b)
-{
- return gsl_ran_gamma (get_rng (), a, 1. / b);
-}
-
-static double
-matrix_eval_PDF_LANDAU (double x)
-{
- return gsl_ran_landau_pdf (x);
-}
-
-static double
-matrix_eval_RV_LANDAU (void)
-{
- return gsl_ran_landau (get_rng ());
-}
-
-static double
-matrix_eval_CDF_LAPLACE (double x, double a, double b)
-{
- return gsl_cdf_laplace_P ((x - a) / b, 1);
-}
-
-static double
-matrix_eval_IDF_LAPLACE (double P, double a, double b)
-{
- return a + b * gsl_cdf_laplace_Pinv (P, 1);
-}
-
-static double
-matrix_eval_PDF_LAPLACE (double x, double a, double b)
-{
- return gsl_ran_laplace_pdf ((x - a) / b, 1);
-}
-
-static double
-matrix_eval_RV_LAPLACE (double a, double b)
-{
- return a + b * gsl_ran_laplace (get_rng (), 1);
-}
-
-static double
-matrix_eval_RV_LEVY (double c, double alpha)
-{
- return gsl_ran_levy (get_rng (), c, alpha);
-}
-
-static double
-matrix_eval_RV_LVSKEW (double c, double alpha, double beta)
-{
- return gsl_ran_levy_skew (get_rng (), c, alpha, beta);
-}
-
-static double
-matrix_eval_CDF_LOGISTIC (double x, double a, double b)
-{
- return gsl_cdf_logistic_P ((x - a) / b, 1);
-}
-
-static double
-matrix_eval_IDF_LOGISTIC (double P, double a, double b)
-{
- return a + b * gsl_cdf_logistic_Pinv (P, 1);
-}
-
-static double
-matrix_eval_PDF_LOGISTIC (double x, double a, double b)
-{
- return gsl_ran_logistic_pdf ((x - a) / b, 1) / b;
-}
-
-static double
-matrix_eval_RV_LOGISTIC (double a, double b)
-{
- return a + b * gsl_ran_logistic (get_rng (), 1);
-}
-
-static double
-matrix_eval_CDF_LNORMAL (double x, double m, double s)
-{
- return gsl_cdf_lognormal_P (x, log (m), s);
-}
-
-static double
-matrix_eval_IDF_LNORMAL (double P, double m, double s)
-{
- return gsl_cdf_lognormal_Pinv (P, log (m), s);;
-}
-
-static double
-matrix_eval_PDF_LNORMAL (double x, double m, double s)
-{
- return gsl_ran_lognormal_pdf (x, log (m), s);
-}
-
-static double
-matrix_eval_RV_LNORMAL (double m, double s)
-{
- return gsl_ran_lognormal (get_rng (), log (m), s);
-}
-
-static double
-matrix_eval_CDF_NORMAL (double x, double u, double s)
-{
- return gsl_cdf_gaussian_P (x - u, s);
-}
-
-static double
-matrix_eval_IDF_NORMAL (double P, double u, double s)
-{
- return u + gsl_cdf_gaussian_Pinv (P, s);
-}
-
-static double
-matrix_eval_PDF_NORMAL (double x, double u, double s)
-{
- return gsl_ran_gaussian_pdf ((x - u) / s, 1) / s;
-}
-
-static double
-matrix_eval_RV_NORMAL (double u, double s)
-{
- return u + gsl_ran_gaussian (get_rng (), s);
-}
-
-static double
-matrix_eval_CDFNORM (double x)
-{
- return gsl_cdf_ugaussian_P (x);
-}
-
-static double
-matrix_eval_PROBIT (double P)
-{
- return gsl_cdf_ugaussian_Pinv (P);
-}
-
-static double
-matrix_eval_NORMAL (double s)
-{
- return gsl_ran_gaussian (get_rng (), s);
-}
-
-static double
-matrix_eval_PDF_NTAIL (double x, double a, double sigma)
-{
- return gsl_ran_gaussian_tail_pdf (x, a, sigma);;
-}
-
-static double
-matrix_eval_RV_NTAIL (double a, double sigma)
-{
- return gsl_ran_gaussian_tail (get_rng (), a, sigma);
-}
-
-static double
-matrix_eval_CDF_PARETO (double x, double a, double b)
-{
- return gsl_cdf_pareto_P (x, b, a);
-}
-
-static double
-matrix_eval_IDF_PARETO (double P, double a, double b)
-{
- return gsl_cdf_pareto_Pinv (P, b, a);
-}
-
-static double
-matrix_eval_PDF_PARETO (double x, double a, double b)
-{
- return gsl_ran_pareto_pdf (x, b, a);
-}
-
-static double
-matrix_eval_RV_PARETO (double a, double b)
-{
- return gsl_ran_pareto (get_rng (), b, a);
-}
-
-static double
-matrix_eval_CDF_RAYLEIGH (double x, double sigma)
-{
- return gsl_cdf_rayleigh_P (x, sigma);
-}
-
-static double
-matrix_eval_IDF_RAYLEIGH (double P, double sigma)
-{
- return gsl_cdf_rayleigh_Pinv (P, sigma);
-}
-
-static double
-matrix_eval_PDF_RAYLEIGH (double x, double sigma)
-{
- return gsl_ran_rayleigh_pdf (x, sigma);
-}
-
-static double
-matrix_eval_RV_RAYLEIGH (double sigma)
-{
- return gsl_ran_rayleigh (get_rng (), sigma);
-}
-
-static double
-matrix_eval_PDF_RTAIL (double x, double a, double sigma)
-{
- return gsl_ran_rayleigh_tail_pdf (x, a, sigma);
-}
-
-static double
-matrix_eval_RV_RTAIL (double a, double sigma)
-{
- return gsl_ran_rayleigh_tail (get_rng (), a, sigma);
-}
-
-static double
-matrix_eval_CDF_T (double x, double df)
-{
- return gsl_cdf_tdist_P (x, df);
-}
-
-static double
-matrix_eval_TCDF (double x, double df)
-{
- return matrix_eval_CDF_T (x, df);
-}
-
-static double
-matrix_eval_IDF_T (double P, double df)
-{
- return gsl_cdf_tdist_Pinv (P, df);
-}
-
-static double
-matrix_eval_PDF_T (double x, double df)
-{
- return gsl_ran_tdist_pdf (x, df);
-}
-
-static double
-matrix_eval_RV_T (double df)
-{
- return gsl_ran_tdist (get_rng (), df);
-}
-
-static double
-matrix_eval_CDF_T1G (double x, double a, double b)
-{
- return gsl_cdf_gumbel1_P (x, a, b);
-}
-
-static double
-matrix_eval_IDF_T1G (double P, double a, double b)
-{
- return gsl_cdf_gumbel1_Pinv (P, a, b);
-}
-
-static double
-matrix_eval_PDF_T1G (double x, double a, double b)
-{
- return gsl_ran_gumbel1_pdf (x, a, b);
-}
-
-static double
-matrix_eval_RV_T1G (double a, double b)
-{
- return gsl_ran_gumbel1 (get_rng (), a, b);
-}
-
-static double
-matrix_eval_CDF_T2G (double x, double a, double b)
-{
- return gsl_cdf_gumbel1_P (x, a, b);
-}
-
-static double
-matrix_eval_IDF_T2G (double P, double a, double b)
-{
- return gsl_cdf_gumbel1_Pinv (P, a, b);
-}
-
-static double
-matrix_eval_PDF_T2G (double x, double a, double b)
-{
- return gsl_ran_gumbel1_pdf (x, a, b);
-}
-
-static double
-matrix_eval_RV_T2G (double a, double b)
-{
- return gsl_ran_gumbel1 (get_rng (), a, b);
-}
-
-static double
-matrix_eval_CDF_UNIFORM (double x, double a, double b)
-{
- return gsl_cdf_flat_P (x, a, b);
-}
-
-static double
-matrix_eval_IDF_UNIFORM (double P, double a, double b)
-{
- return gsl_cdf_flat_Pinv (P, a, b);
-}
-
-static double
-matrix_eval_PDF_UNIFORM (double x, double a, double b)
-{
- return gsl_ran_flat_pdf (x, a, b);
-}
-
-static double
-matrix_eval_RV_UNIFORM (double a, double b)
-{
- return gsl_ran_flat (get_rng (), a, b);
-}
-
-static double
-matrix_eval_CDF_WEIBULL (double x, double a, double b)
-{
- return gsl_cdf_weibull_P (x, a, b);
-}
-
-static double
-matrix_eval_IDF_WEIBULL (double P, double a, double b)
-{
- return gsl_cdf_weibull_Pinv (P, a, b);
-}
-
-static double
-matrix_eval_PDF_WEIBULL (double x, double a, double b)
-{
- return gsl_ran_weibull_pdf (x, a, b);
-}
-
-static double
-matrix_eval_RV_WEIBULL (double a, double b)
-{
- return gsl_ran_weibull (get_rng (), a, b);
-}
-
-static double
-matrix_eval_CDF_BERNOULLI (double k, double p)
-{
- return k ? 1 : 1 - p;
-}
-
-static double
-matrix_eval_PDF_BERNOULLI (double k, double p)
-{
- return gsl_ran_bernoulli_pdf (k, p);
-}
-
-static double
-matrix_eval_RV_BERNOULLI (double p)
-{
- return gsl_ran_bernoulli (get_rng (), p);
-}
-
-static double
-matrix_eval_CDF_BINOM (double k, double n, double p)
-{
- return gsl_cdf_binomial_P (k, p, n);
-}
-
-static double
-matrix_eval_PDF_BINOM (double k, double n, double p)
-{
- return gsl_ran_binomial_pdf (k, p, n);
-}
-
-static double
-matrix_eval_RV_BINOM (double n, double p)
-{
- return gsl_ran_binomial (get_rng (), p, n);
-}
-
-static double
-matrix_eval_CDF_GEOM (double k, double p)
-{
- return gsl_cdf_geometric_P (k, p);
-}
-
-static double
-matrix_eval_PDF_GEOM (double k, double p)
-{
- return gsl_ran_geometric_pdf (k, p);
-}
-
-static double
-matrix_eval_RV_GEOM (double p)
-{
- return gsl_ran_geometric (get_rng (), p);
-}
-
-static double
-matrix_eval_CDF_HYPER (double k, double a, double b, double c)
-{
- return gsl_cdf_hypergeometric_P (k, c, a - c, b);
-}
-
-static double
-matrix_eval_PDF_HYPER (double k, double a, double b, double c)
-{
- return gsl_ran_hypergeometric_pdf (k, c, a - c, b);
-}
-
-static double
-matrix_eval_RV_HYPER (double a, double b, double c)
-{
- return gsl_ran_hypergeometric (get_rng (), c, a - c, b);
-}
-
-static double
-matrix_eval_PDF_LOG (double k, double p)
-{
- return gsl_ran_logarithmic_pdf (k, p);
-}
-
-static double
-matrix_eval_RV_LOG (double p)
-{
- return gsl_ran_logarithmic (get_rng (), p);
-}
-
-static double
-matrix_eval_CDF_NEGBIN (double k, double n, double p)
-{
- return gsl_cdf_negative_binomial_P (k, p, n);
-}
-
-static double
-matrix_eval_PDF_NEGBIN (double k, double n, double p)
-{
- return gsl_ran_negative_binomial_pdf (k, p, n);
-}
-
-static double
-matrix_eval_RV_NEGBIN (double n, double p)
-{
- return gsl_ran_negative_binomial (get_rng (), p, n);
-}
-
-static double
-matrix_eval_CDF_POISSON (double k, double mu)
-{
- return gsl_cdf_poisson_P (k, mu);
-}
-
-static double
-matrix_eval_PDF_POISSON (double k, double mu)
-{
- return gsl_ran_poisson_pdf (k, mu);
-}
-
-static double
-matrix_eval_RV_POISSON (double mu)
-{
- return gsl_ran_poisson (get_rng (), mu);
-}
-
-static double
-matrix_op_eval (enum matrix_op op, double a, double b)
-{
- switch (op)
- {
- case MOP_ADD_ELEMS: return a + b;
- case MOP_SUB_ELEMS: return a - b;
- case MOP_MUL_ELEMS: return a * b;
- case MOP_DIV_ELEMS: return a / b;
- case MOP_EXP_ELEMS: return pow (a, b);
- case MOP_GT: return a > b;
- case MOP_GE: return a >= b;
- case MOP_LT: return a < b;
- case MOP_LE: return a <= b;
- case MOP_EQ: return a == b;
- case MOP_NE: return a != b;
- case MOP_AND: return (a > 0) && (b > 0);
- case MOP_OR: return (a > 0) || (b > 0);
- case MOP_XOR: return (a > 0) != (b > 0);
-
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) case MOP_F_##ENUM:
- MATRIX_FUNCTIONS
-#undef F
- case MOP_NEGATE:
- case MOP_SEQ:
- case MOP_SEQ_BY:
- case MOP_MUL_MAT:
- case MOP_EXP_MAT:
- case MOP_NOT:
- case MOP_PASTE_HORZ:
- case MOP_PASTE_VERT:
- case MOP_EMPTY:
- case MOP_VEC_INDEX:
- case MOP_VEC_ALL:
- case MOP_MAT_INDEX:
- case MOP_ROW_INDEX:
- case MOP_COL_INDEX:
- case MOP_NUMBER:
- case MOP_VARIABLE:
- case MOP_EOF:
- NOT_REACHED ();
- }
- NOT_REACHED ();
-}
-
-static const char *
-matrix_op_name (enum matrix_op op)
-{
- switch (op)
- {
- case MOP_ADD_ELEMS: return "+";
- case MOP_SUB_ELEMS: return "-";
- case MOP_MUL_ELEMS: return "&*";
- case MOP_DIV_ELEMS: return "&/";
- case MOP_EXP_ELEMS: return "&**";
- case MOP_GT: return ">";
- case MOP_GE: return ">=";
- case MOP_LT: return "<";
- case MOP_LE: return "<=";
- case MOP_EQ: return "=";
- case MOP_NE: return "<>";
- case MOP_AND: return "AND";
- case MOP_OR: return "OR";
- case MOP_XOR: return "XOR";
-
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) case MOP_F_##ENUM:
- MATRIX_FUNCTIONS
-#undef F
- case MOP_NEGATE:
- case MOP_SEQ:
- case MOP_SEQ_BY:
- case MOP_MUL_MAT:
- case MOP_EXP_MAT:
- case MOP_NOT:
- case MOP_PASTE_HORZ:
- case MOP_PASTE_VERT:
- case MOP_EMPTY:
- case MOP_VEC_INDEX:
- case MOP_VEC_ALL:
- case MOP_MAT_INDEX:
- case MOP_ROW_INDEX:
- case MOP_COL_INDEX:
- case MOP_NUMBER:
- case MOP_VARIABLE:
- case MOP_EOF:
- NOT_REACHED ();
- }
- NOT_REACHED ();
-}
-
-static bool
-is_scalar (const gsl_matrix *m)
-{
- return m->size1 == 1 && m->size2 == 1;
-}
-
-static double
-to_scalar (const gsl_matrix *m)
-{
- assert (is_scalar (m));
- return gsl_matrix_get (m, 0, 0);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_elementwise (const struct matrix_expr *e,
- enum matrix_op op,
- gsl_matrix *a, gsl_matrix *b)
-{
- if (is_scalar (b))
- {
- double be = to_scalar (b);
- for (size_t r = 0; r < a->size1; r++)
- for (size_t c = 0; c < a->size2; c++)
- {
- double *ae = gsl_matrix_ptr (a, r, c);
- *ae = matrix_op_eval (op, *ae, be);
- }
- return a;
- }
- else if (is_scalar (a))
- {
- double ae = to_scalar (a);
- for (size_t r = 0; r < b->size1; r++)
- for (size_t c = 0; c < b->size2; c++)
- {
- double *be = gsl_matrix_ptr (b, r, c);
- *be = matrix_op_eval (op, ae, *be);
- }
- return b;
- }
- else if (a->size1 == b->size1 && a->size2 == b->size2)
- {
- for (size_t r = 0; r < a->size1; r++)
- for (size_t c = 0; c < a->size2; c++)
- {
- double *ae = gsl_matrix_ptr (a, r, c);
- double be = gsl_matrix_get (b, r, c);
- *ae = matrix_op_eval (op, *ae, be);
- }
- return a;
- }
- else
- {
- msg_at (SE, matrix_expr_location (e),
- _("The operands of %s must have the same dimensions or one "
- "must be a scalar."),
- matrix_op_name (op));
- msg_at (SN, matrix_expr_location (e->subs[0]),
- _("The left-hand operand is a %zu×%zu matrix."),
- a->size1, a->size2);
- msg_at (SN, matrix_expr_location (e->subs[1]),
- _("The right-hand operand is a %zu×%zu matrix."),
- b->size1, b->size2);
- return NULL;
- }
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_mul_mat (const struct matrix_expr *e,
- gsl_matrix *a, gsl_matrix *b)
-{
- if (is_scalar (a) || is_scalar (b))
- return matrix_expr_evaluate_elementwise (e, MOP_MUL_ELEMS, a, b);
-
- if (a->size2 != b->size1)
- {
- msg_at (SE, e->location,
- _("Matrices not conformable for multiplication."));
- msg_at (SN, matrix_expr_location (e->subs[0]),
- _("The left-hand operand is a %zu×%zu matrix."),
- a->size1, a->size2);
- msg_at (SN, matrix_expr_location (e->subs[1]),
- _("The right-hand operand is a %zu×%zu matrix."),
- b->size1, b->size2);
- return NULL;
- }
-
- gsl_matrix *c = gsl_matrix_alloc (a->size1, b->size2);
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, a, b, 0.0, c);
- return c;
-}
-
-static void
-swap_matrix (gsl_matrix **a, gsl_matrix **b)
-{
- gsl_matrix *tmp = *a;
- *a = *b;
- *b = tmp;
-}
-
-static void
-mul_matrix (gsl_matrix **z, const gsl_matrix *x, const gsl_matrix *y,
- gsl_matrix **tmp)
-{
- gsl_blas_dgemm (CblasNoTrans, CblasNoTrans, 1.0, x, y, 0.0, *tmp);
- swap_matrix (z, tmp);
-}
-
-static void
-square_matrix (gsl_matrix **x, gsl_matrix **tmp)
-{
- mul_matrix (x, *x, *x, tmp);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_exp_mat (const struct matrix_expr *e,
- gsl_matrix *x_, gsl_matrix *b)
-{
- gsl_matrix *x = x_;
- if (x->size1 != x->size2)
- {
- msg_at (SE, matrix_expr_location (e->subs[0]),
- _("Matrix exponentation with ** requires a square matrix on "
- "the left-hand size, not one with dimensions %zu×%zu."),
- x->size1, x->size2);
- return NULL;
- }
- if (!is_scalar (b))
- {
- msg_at (SE, matrix_expr_location (e->subs[1]),
- _("Matrix exponentiation with ** requires a scalar on the "
- "right-hand side, not a matrix with dimensions %zu×%zu."),
- b->size1, b->size2);
- return NULL;
- }
- double bf = to_scalar (b);
- if (bf != floor (bf) || bf <= LONG_MIN || bf > LONG_MAX)
- {
- msg_at (SE, matrix_expr_location (e->subs[1]),
- _("Exponent %.1f in matrix exponentiation is non-integer "
- "or outside the valid range."), bf);
- return NULL;
- }
- long int bl = bf;
-
- gsl_matrix *y_ = gsl_matrix_alloc (x->size1, x->size2);
- gsl_matrix *y = y_;
- gsl_matrix_set_identity (y);
- if (bl == 0)
- return y;
-
- gsl_matrix *t_ = gsl_matrix_alloc (x->size1, x->size2);
- gsl_matrix *t = t_;
- for (unsigned long int n = labs (bl); n > 1; n /= 2)
- if (n & 1)
- {
- mul_matrix (&y, x, y, &t);
- square_matrix (&x, &t);
- }
- else
- square_matrix (&x, &t);
-
- mul_matrix (&y, x, y, &t);
- if (bf < 0)
- {
- invert_matrix (y, x);
- swap_matrix (&x, &y);
- }
-
- /* Garbage collection.
-
- There are three matrices: 'x_', 'y_', and 't_', and 'x', 'y', and 't' are
- a permutation of them. We are returning one of them; that one must not be
- destroyed. We must not destroy 'x_' because the caller owns it. */
- if (y != y_)
- gsl_matrix_free (y_);
- if (y != t_)
- gsl_matrix_free (t_);
-
- return y;
-}
-
-static void
-note_operand_size (const gsl_matrix *m, const struct matrix_expr *e)
-{
- msg_at (SN, matrix_expr_location (e),
- _("This operand is a %zu×%zu matrix."), m->size1, m->size2);
-}
-
-static void
-note_nonscalar (const gsl_matrix *m, const struct matrix_expr *e)
-{
- if (!is_scalar (m))
- note_operand_size (m, e);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_seq (const struct matrix_expr *e,
- gsl_matrix *start_, gsl_matrix *end_,
- gsl_matrix *by_)
-{
- if (!is_scalar (start_) || !is_scalar (end_) || (by_ && !is_scalar (by_)))
- {
- msg_at (SE, matrix_expr_location (e),
- _("All operands of : operator must be scalars."));
-
- note_nonscalar (start_, e->subs[0]);
- note_nonscalar (end_, e->subs[1]);
- if (by_)
- note_nonscalar (by_, e->subs[2]);
- return NULL;
- }
-
- long int start = to_scalar (start_);
- long int end = to_scalar (end_);
- long int by = by_ ? to_scalar (by_) : 1;
-
- if (!by)
- {
- msg_at (SE, matrix_expr_location (e->subs[2]),
- _("The increment operand to : must be nonzero."));
- return NULL;
- }
-
- long int n = (end >= start && by > 0 ? (end - start + by) / by
- : end <= start && by < 0 ? (start - end - by) / -by
- : 0);
- gsl_matrix *m = gsl_matrix_alloc (1, n);
- for (long int i = 0; i < n; i++)
- gsl_matrix_set (m, 0, i, start + i * by);
- return m;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_not (gsl_matrix *a)
-{
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, a)
- *d = !(*d > 0);
- return a;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_paste_horz (const struct matrix_expr *e,
- gsl_matrix *a, gsl_matrix *b)
-{
- if (a->size1 != b->size1)
- {
- if (!a->size1 || !a->size2)
- return b;
- else if (!b->size1 || !b->size2)
- return a;
-
- msg_at (SE, matrix_expr_location (e),
- _("This expression tries to horizontally join matrices with "
- "differing numbers of rows."));
- note_operand_size (a, e->subs[0]);
- note_operand_size (b, e->subs[1]);
- return NULL;
- }
-
- gsl_matrix *c = gsl_matrix_alloc (a->size1, a->size2 + b->size2);
- for (size_t y = 0; y < a->size1; y++)
- {
- for (size_t x = 0; x < a->size2; x++)
- gsl_matrix_set (c, y, x, gsl_matrix_get (a, y, x));
- for (size_t x = 0; x < b->size2; x++)
- gsl_matrix_set (c, y, x + a->size2, gsl_matrix_get (b, y, x));
- }
- return c;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_paste_vert (const struct matrix_expr *e,
- gsl_matrix *a, gsl_matrix *b)
-{
- if (a->size2 != b->size2)
- {
- if (!a->size1 || !a->size2)
- return b;
- else if (!b->size1 || !b->size2)
- return a;
-
- msg_at (SE, matrix_expr_location (e),
- _("This expression tries to vertically join matrices with "
- "differing numbers of columns."));
- note_operand_size (a, e->subs[0]);
- note_operand_size (b, e->subs[1]);
- return NULL;
- }
-
- gsl_matrix *c = gsl_matrix_alloc (a->size1 + b->size1, a->size2);
- for (size_t x = 0; x < a->size2; x++)
- {
- for (size_t y = 0; y < a->size1; y++)
- gsl_matrix_set (c, y, x, gsl_matrix_get (a, y, x));
- for (size_t y = 0; y < b->size1; y++)
- gsl_matrix_set (c, y + a->size1, x, gsl_matrix_get (b, y, x));
- }
- return c;
-}
-
-static gsl_vector *
-matrix_to_vector (gsl_matrix *m)
-{
- assert (m->owner);
- gsl_vector v = to_vector (m);
- assert (v.block == m->block || !v.block);
- assert (!v.owner);
- v.owner = 1;
- m->owner = 0;
- gsl_matrix_free (m);
- return xmemdup (&v, sizeof v);
-}
-
-enum index_type {
- IV_ROW,
- IV_COLUMN,
- IV_VECTOR
-};
-
-struct index_vector
- {
- size_t *indexes;
- size_t n;
- };
-#define INDEX_VECTOR_INIT (struct index_vector) { .n = 0 }
-
-static void
-index_vector_uninit (struct index_vector *iv)
-{
- if (iv)
- free (iv->indexes);
-}
-
-static bool
-matrix_normalize_index_vector (const gsl_matrix *m,
- const struct matrix_expr *me, size_t size,
- enum index_type index_type, size_t other_size,
- struct index_vector *iv)
-{
- if (m)
- {
- if (!is_vector (m))
- {
- switch (index_type)
- {
- case IV_VECTOR:
- msg_at (SE, matrix_expr_location (me),
- _("Vector index must be scalar or vector, not a "
- "%zu×%zu matrix."),
- m->size1, m->size2);
- break;
-
- case IV_ROW:
- msg_at (SE, matrix_expr_location (me),
- _("Matrix row index must be scalar or vector, not a "
- "%zu×%zu matrix."),
- m->size1, m->size2);
- break;
-
- case IV_COLUMN:
- msg_at (SE, matrix_expr_location (me),
- _("Matrix column index must be scalar or vector, not a "
- "%zu×%zu matrix."),
- m->size1, m->size2);
- break;
- }
- return false;
- }
-
- gsl_vector v = to_vector (CONST_CAST (gsl_matrix *, m));
- *iv = (struct index_vector) {
- .indexes = xnmalloc (v.size, sizeof *iv->indexes),
- .n = v.size,
- };
- for (size_t i = 0; i < v.size; i++)
- {
- double index = gsl_vector_get (&v, i);
- if (index < 1 || index >= size + 1)
- {
- switch (index_type)
- {
- case IV_VECTOR:
- msg_at (SE, matrix_expr_location (me),
- _("Index %g is out of range for vector "
- "with %zu elements."), index, size);
- break;
-
- case IV_ROW:
- msg_at (SE, matrix_expr_location (me),
- _("%g is not a valid row index for "
- "a %zu×%zu matrix."),
- index, size, other_size);
- break;
-
- case IV_COLUMN:
- msg_at (SE, matrix_expr_location (me),
- _("%g is not a valid column index for "
- "a %zu×%zu matrix."),
- index, other_size, size);
- break;
- }
-
- index_vector_uninit (iv);
- return false;
- }
- iv->indexes[i] = index - 1;
- }
- return true;
- }
- else
- {
- *iv = (struct index_vector) {
- .indexes = xnmalloc (size, sizeof *iv->indexes),
- .n = size,
- };
- for (size_t i = 0; i < size; i++)
- iv->indexes[i] = i;
- return true;
- }
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_vec_all (const struct matrix_expr *e,
- gsl_matrix *sm)
-{
- if (!is_vector (sm))
- {
- msg_at (SE, matrix_expr_location (e->subs[0]),
- _("Vector index operator may not be applied to "
- "a %zu×%zu matrix."),
- sm->size1, sm->size2);
- return NULL;
- }
-
- return sm;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_vec_index (const struct matrix_expr *e,
- gsl_matrix *sm, gsl_matrix *im)
-{
- if (!matrix_expr_evaluate_vec_all (e, sm))
- return NULL;
-
- gsl_vector sv = to_vector (sm);
- struct index_vector iv;
- if (!matrix_normalize_index_vector (im, e->subs[1],
- sv.size, IV_VECTOR, 0, &iv))
- return NULL;
-
- gsl_matrix *dm = gsl_matrix_alloc (sm->size1 == 1 ? 1 : iv.n,
- sm->size1 == 1 ? iv.n : 1);
- gsl_vector dv = to_vector (dm);
- for (size_t dx = 0; dx < iv.n; dx++)
- {
- size_t sx = iv.indexes[dx];
- gsl_vector_set (&dv, dx, gsl_vector_get (&sv, sx));
- }
- index_vector_uninit (&iv);
-
- return dm;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_mat_index (gsl_matrix *sm,
- gsl_matrix *im0, const struct matrix_expr *eim0,
- gsl_matrix *im1, const struct matrix_expr *eim1)
-{
- struct index_vector iv0;
- if (!matrix_normalize_index_vector (im0, eim0, sm->size1,
- IV_ROW, sm->size2, &iv0))
- return NULL;
-
- struct index_vector iv1;
- if (!matrix_normalize_index_vector (im1, eim1, sm->size2,
- IV_COLUMN, sm->size1, &iv1))
- {
- index_vector_uninit (&iv0);
- return NULL;
- }
-
- gsl_matrix *dm = gsl_matrix_alloc (iv0.n, iv1.n);
- for (size_t dy = 0; dy < iv0.n; dy++)
- {
- size_t sy = iv0.indexes[dy];
-
- for (size_t dx = 0; dx < iv1.n; dx++)
- {
- size_t sx = iv1.indexes[dx];
- gsl_matrix_set (dm, dy, dx, gsl_matrix_get (sm, sy, sx));
- }
- }
- index_vector_uninit (&iv0);
- index_vector_uninit (&iv1);
- return dm;
-}
-
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
- static gsl_matrix *matrix_expr_evaluate_##PROTO ( \
- const struct matrix_function_properties *, gsl_matrix *[], \
- const struct matrix_expr *, matrix_proto_##PROTO *);
-MATRIX_FUNCTIONS
-#undef F
-
-static bool
-check_scalar_arg (const char *name, gsl_matrix *subs[],
- const struct matrix_expr *e, size_t index)
-{
- if (!is_scalar (subs[index]))
- {
- msg_at (SE, matrix_expr_location (e->subs[index]),
- _("Function %s argument %zu must be a scalar, "
- "not a %zu×%zu matrix."),
- name, index + 1, subs[index]->size1, subs[index]->size2);
- return false;
- }
- return true;
-}
-
-static bool
-check_vector_arg (const char *name, gsl_matrix *subs[],
- const struct matrix_expr *e, size_t index)
-{
- if (!is_vector (subs[index]))
- {
- msg_at (SE, matrix_expr_location (e->subs[index]),
- _("Function %s argument %zu must be a vector, "
- "not a %zu×%zu matrix."),
- name, index + 1, subs[index]->size1, subs[index]->size2);
- return false;
- }
- return true;
-}
-
-static bool
-to_scalar_args (const char *name, gsl_matrix *subs[],
- const struct matrix_expr *e, double d[])
-{
- for (size_t i = 0; i < e->n_subs; i++)
- {
- if (!check_scalar_arg (name, subs, e, i))
- return false;
- d[i] = to_scalar (subs[i]);
- }
- return true;
-}
-
-static int
-parse_constraint_value (const char **constraintsp)
-{
- char *tail;
- long retval = strtol (*constraintsp, &tail, 10);
- assert (tail > *constraintsp);
- *constraintsp = tail;
- return retval;
-}
-
-enum matrix_argument_relop
- {
- MRR_GT, /* > */
- MRR_GE, /* >= */
- MRR_LT, /* < */
- MRR_LE, /* <= */
- MRR_NE, /* <> */
- };
-
-static void
-argument_inequality_error (
- const struct matrix_function_properties *props, const struct matrix_expr *e,
- size_t ai, gsl_matrix *a, size_t y, size_t x,
- size_t bi, double b,
- enum matrix_argument_relop relop)
-{
- const struct msg_location *loc = matrix_expr_location (e);
- switch (relop)
- {
- case MRR_GE:
- msg_at (ME, loc, _("Argument %zu to matrix function %s must be greater "
- "than or equal to argument %zu."),
- ai + 1, props->name, bi + 1);
- break;
-
- case MRR_GT:
- msg_at (ME, loc, _("Argument %zu to matrix function %s must be greater "
- "than argument %zu."),
- ai + 1, props->name, bi + 1);
- break;
-
- case MRR_LE:
- msg_at (ME, loc, _("Argument %zu to matrix function %s must be less than "
- "or equal to argument %zu."),
- ai + 1, props->name, bi + 1);
- break;
-
- case MRR_LT:
- msg_at (ME, loc, _("Argument %zu to matrix function %s must be less than "
- "argument %zu."),
- ai + 1, props->name, bi + 1);
- break;
-
- case MRR_NE:
- msg_at (ME, loc, _("Argument %zu to matrix function %s must not be equal "
- "to argument %zu."),
- ai + 1, props->name, bi + 1);
- break;
- }
-
- const struct msg_location *a_loc = matrix_expr_location (e->subs[ai]);
- if (is_scalar (a))
- msg_at (SN, a_loc, _("Argument %zu is %g."),
- ai + 1, gsl_matrix_get (a, y, x));
- else
- msg_at (SN, a_loc, _("Row %zu, column %zu of argument %zu is %g."),
- y + 1, x + 1, ai + 1, gsl_matrix_get (a, y, x));
-
- msg_at (SN, matrix_expr_location (e->subs[bi]),
- _("Argument %zu is %g."), bi + 1, b);
-}
-
-static void
-argument_value_error (
- const struct matrix_function_properties *props, const struct matrix_expr *e,
- size_t ai, gsl_matrix *a, size_t y, size_t x,
- double b,
- enum matrix_argument_relop relop)
-{
- const struct msg_location *loc = matrix_expr_location (e);
- switch (relop)
- {
- case MRR_GE:
- msg_at (SE, loc, _("Argument %zu to matrix function %s must be greater "
- "than or equal to %g."),
- ai + 1, props->name, b);
- break;
-
- case MRR_GT:
- msg_at (SE, loc, _("Argument %zu to matrix function %s must be greater "
- "than %g."),
- ai + 1, props->name, b);
- break;
-
- case MRR_LE:
- msg_at (SE, loc, _("Argument %zu to matrix function %s must be less than "
- "or equal to %g."),
- ai + 1, props->name, b);
- break;
-
- case MRR_LT:
- msg_at (SE, loc, _("Argument %zu to matrix function %s must be less than "
- "%g."),
- ai + 1, props->name, b);
- break;
-
- case MRR_NE:
- msg_at (SE, loc, _("Argument %zu to matrix function %s must not be equal "
- "to %g."),
- ai + 1, props->name, b);
- break;
- }
-
- const struct msg_location *a_loc = matrix_expr_location (e->subs[ai]);
- if (is_scalar (a))
- {
- if (relop != MRR_NE)
- msg_at (SN, a_loc, _("Argument %zu is %g."),
- ai + 1, gsl_matrix_get (a, y, x));
- }
- else
- msg_at (SN, a_loc, _("Row %zu, column %zu of argument %zu is %g."),
- y + 1, x + 1, ai + 1, gsl_matrix_get (a, y, x));
-}
-
-static bool
-matrix_argument_relop_is_satisfied (double a, double b,
- enum matrix_argument_relop relop)
-{
- switch (relop)
- {
- case MRR_GE: return a >= b;
- case MRR_GT: return a > b;
- case MRR_LE: return a <= b;
- case MRR_LT: return a < b;
- case MRR_NE: return a != b;
- }
-
- NOT_REACHED ();
-}
-
-static enum matrix_argument_relop
-matrix_argument_relop_flip (enum matrix_argument_relop relop)
-{
- switch (relop)
- {
- case MRR_GE: return MRR_LE;
- case MRR_GT: return MRR_LT;
- case MRR_LE: return MRR_GE;
- case MRR_LT: return MRR_GT;
- case MRR_NE: return MRR_NE;
- }
-
- NOT_REACHED ();
-}
-
-static bool
-check_constraints (const struct matrix_function_properties *props,
- gsl_matrix *args[], const struct matrix_expr *e)
-{
- size_t n_args = e->n_subs;
- const char *constraints = props->constraints;
- if (!constraints)
- return true;
-
- size_t arg_index = SIZE_MAX;
- while (*constraints)
- {
- if (*constraints >= 'a' && *constraints <= 'd')
- {
- arg_index = *constraints++ - 'a';
- assert (arg_index < n_args);
- }
- else if (*constraints == '[' || *constraints == '(')
- {
- assert (arg_index < n_args);
- bool open_lower = *constraints++ == '(';
- int minimum = parse_constraint_value (&constraints);
- assert (*constraints == ',');
- constraints++;
- int maximum = parse_constraint_value (&constraints);
- assert (*constraints == ']' || *constraints == ')');
- bool open_upper = *constraints++ == ')';
-
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, args[arg_index])
- if ((open_lower ? *d <= minimum : *d < minimum)
- || (open_upper ? *d >= maximum : *d > maximum))
- {
- if (!is_scalar (args[arg_index]))
- msg_at (SE, matrix_expr_location (e->subs[arg_index]),
- _("Row %zu, column %zu of argument %zu to matrix "
- "function %s is %g, which is outside "
- "the valid range %c%d,%d%c."),
- y + 1, x + 1, arg_index + 1, props->name, *d,
- open_lower ? '(' : '[',
- minimum, maximum,
- open_upper ? ')' : ']');
- else
- msg_at (SE, matrix_expr_location (e->subs[arg_index]),
- _("Argument %zu to matrix function %s is %g, "
- "which is outside the valid range %c%d,%d%c."),
- arg_index + 1, props->name, *d,
- open_lower ? '(' : '[',
- minimum, maximum,
- open_upper ? ')' : ']');
- return false;
- }
- }
- else if (*constraints == 'i')
- {
- constraints++;
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, args[arg_index])
- if (*d != floor (*d))
- {
- if (!is_scalar (args[arg_index]))
- msg_at (SE, matrix_expr_location (e->subs[arg_index]),
- _("Argument %zu to matrix function %s, which must be "
- "integer, contains non-integer value %g in "
- "row %zu, column %zu."),
- arg_index + 1, props->name, *d, y + 1, x + 1);
- else
- msg_at (SE, matrix_expr_location (e->subs[arg_index]),
- _("Argument %zu to matrix function %s, which must be "
- "integer, has non-integer value %g."),
- arg_index + 1, props->name, *d);
- return false;
- }
- }
- else if (*constraints == '>'
- || *constraints == '<'
- || *constraints == '!')
- {
- enum matrix_argument_relop relop;
- switch (*constraints++)
- {
- case '>':
- if (*constraints == '=')
- {
- constraints++;
- relop = MRR_GE;
- }
- else
- relop = MRR_GT;
- break;
-
- case '<':
- if (*constraints == '=')
- {
- constraints++;
- relop = MRR_LE;
- }
- else
- relop = MRR_LT;
- break;
-
- case '!':
- assert (*constraints == '=');
- constraints++;
- relop = MRR_NE;
- break;
-
- default:
- NOT_REACHED ();
- }
-
- if (*constraints >= 'a' && *constraints <= 'd')
- {
- size_t a_index = arg_index;
- size_t b_index = *constraints - 'a';
- assert (a_index < n_args);
- assert (b_index < n_args);
-
- /* We only support one of the two arguments being non-scalar.
- It's easier to support only the first one being non-scalar, so
- flip things around if it's the other way. */
- if (!is_scalar (args[b_index]))
- {
- assert (is_scalar (args[a_index]));
- size_t tmp_index = a_index;
- a_index = b_index;
- b_index = tmp_index;
- relop = matrix_argument_relop_flip (relop);
- }
-
- double b = to_scalar (args[b_index]);
- MATRIX_FOR_ALL_ELEMENTS (a, y, x, args[a_index])
- if (!matrix_argument_relop_is_satisfied (*a, b, relop))
- {
- argument_inequality_error (
- props, e,
- a_index, args[a_index], y, x,
- b_index, b,
- relop);
- return false;
- }
- }
- else
- {
- int comparand = parse_constraint_value (&constraints);
-
- MATRIX_FOR_ALL_ELEMENTS (d, y, x, args[arg_index])
- if (!matrix_argument_relop_is_satisfied (*d, comparand, relop))
- {
- argument_value_error (
- props, e,
- arg_index, args[arg_index], y, x,
- comparand,
- relop);
- return false;
- }
- }
- }
- else
- {
- assert (*constraints == ' ');
- constraints++;
- arg_index = SIZE_MAX;
- }
- }
- return true;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_d_none (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_d_none *f)
-{
- assert (e->n_subs == 0);
-
- if (!check_constraints (props, subs, e))
- return NULL;
-
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, f ());
- return m;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_d_d (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_d_d *f)
-{
- assert (e->n_subs == 1);
-
- double d;
- if (!to_scalar_args (props->name, subs, e, &d)
- || !check_constraints (props, subs, e))
- return NULL;
-
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, f (d));
- return m;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_d_dd (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_d_dd *f)
-{
- assert (e->n_subs == 2);
-
- double d[2];
- if (!to_scalar_args (props->name, subs, e, d)
- && !check_constraints (props, subs, e))
- return NULL;
-
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, f (d[0], d[1]));
- return m;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_d_ddd (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_d_ddd *f)
-{
- assert (e->n_subs == 3);
-
- double d[3];
- if (!to_scalar_args (props->name, subs, e, d)
- || !check_constraints (props, subs, e))
- return NULL;
-
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, f (d[0], d[1], d[2]));
- return m;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_d (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_d *f)
-{
- assert (e->n_subs == 1);
-
- double d;
- return (to_scalar_args (props->name, subs, e, &d)
- && check_constraints (props, subs, e)
- ? f(d)
- : NULL);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_ddd (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_ddd *f)
-{
- assert (e->n_subs == 3);
-
- double d[3];
- return (to_scalar_args (props->name, subs, e, d)
- && check_constraints (props, subs, e)
- ? f(d[0], d[1], d[2])
- : NULL);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_ddn (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_ddn *f)
-{
- assert (e->n_subs == 2);
-
- double d[2];
- return (to_scalar_args (props->name, subs, e, d)
- && check_constraints (props, subs, e)
- ? f(d[0], d[1], e)
- : NULL);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_m (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_m *f)
-{
- assert (e->n_subs == 1);
- return check_constraints (props, subs, e) ? f (subs[0]) : NULL;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_mn (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_mn *f)
-{
- assert (e->n_subs == 1);
- return check_constraints (props, subs, e) ? f (subs[0], e) : NULL;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_e (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_e *f)
-{
- assert (e->n_subs == 1);
-
- if (!check_constraints (props, subs, e))
- return NULL;
-
- MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
- *a = f (*a);
- return subs[0];
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_md (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_md *f)
-{
- assert (e->n_subs == 2);
- return (check_scalar_arg (props->name, subs, e, 1)
- && check_constraints (props, subs, e)
- ? f (subs[0], to_scalar (subs[1]))
- : NULL);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_mdn (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_mdn *f)
-{
- assert (e->n_subs == 2);
- return (check_scalar_arg (props->name, subs, e, 1)
- && check_constraints (props, subs, e)
- ? f (subs[0], to_scalar (subs[1]), e)
- : NULL);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_ed (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_ed *f)
-{
- assert (e->n_subs == 2);
- if (!check_scalar_arg (props->name, subs, e, 1)
- || !check_constraints (props, subs, e))
- return NULL;
-
- double b = to_scalar (subs[1]);
- MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
- *a = f (*a, b);
- return subs[0];
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_mddn (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_mddn *f)
-{
- assert (e->n_subs == 3);
- if (!check_scalar_arg (props->name, subs, e, 1)
- || !check_scalar_arg (props->name, subs, e, 2)
- || !check_constraints (props, subs, e))
- return NULL;
- return f (subs[0], to_scalar (subs[1]), to_scalar (subs[2]), e);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_edd (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_edd *f)
-{
- assert (e->n_subs == 3);
- if (!check_scalar_arg (props->name, subs, e, 1)
- || !check_scalar_arg (props->name, subs, e, 2)
- || !check_constraints (props, subs, e))
- return NULL;
-
- double b = to_scalar (subs[1]);
- double c = to_scalar (subs[2]);
- MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
- *a = f (*a, b, c);
- return subs[0];
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_eddd (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_eddd *f)
-{
- assert (e->n_subs == 4);
- for (size_t i = 1; i < 4; i++)
- if (!check_scalar_arg (props->name, subs, e, i))
- return NULL;
-
- if (!check_constraints (props, subs, e))
- return NULL;
-
- double b = to_scalar (subs[1]);
- double c = to_scalar (subs[2]);
- double d = to_scalar (subs[3]);
- MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
- *a = f (*a, b, c, d);
- return subs[0];
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_eed (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_eed *f)
-{
- assert (e->n_subs == 3);
- if (!check_scalar_arg (props->name, subs, e, 2))
- return NULL;
-
- if (!is_scalar (subs[0]) && !is_scalar (subs[1])
- && (subs[0]->size1 != subs[1]->size1 || subs[0]->size2 != subs[1]->size2))
- {
- struct msg_location *loc = msg_location_dup (e->subs[0]->location);
- loc->end = e->subs[1]->location->end;
-
- msg_at (ME, loc,
- _("Arguments 1 and 2 to %s have dimensions %zu×%zu and "
- "%zu×%zu, but %s requires these arguments either to have "
- "the same dimensions or for one of them to be a scalar."),
- props->name,
- subs[0]->size1, subs[0]->size2,
- subs[1]->size1, subs[1]->size2,
- props->name);
-
- msg_location_destroy (loc);
- return NULL;
- }
-
- if (!check_constraints (props, subs, e))
- return NULL;
-
- double c = to_scalar (subs[2]);
-
- if (is_scalar (subs[0]))
- {
- double a = to_scalar (subs[0]);
- MATRIX_FOR_ALL_ELEMENTS (b, y, x, subs[1])
- *b = f (a, *b, c);
- return subs[1];
- }
- else
- {
- double b = to_scalar (subs[1]);
- MATRIX_FOR_ALL_ELEMENTS (a, y, x, subs[0])
- *a = f (*a, b, c);
- return subs[0];
- }
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_mm (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_mm *f)
-{
- assert (e->n_subs == 2);
- return check_constraints (props, subs, e) ? f (subs[0], subs[1]) : NULL;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_mmn (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_mmn *f)
-{
- assert (e->n_subs == 2);
- return check_constraints (props, subs, e) ? f (subs[0], subs[1], e) : NULL;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_v (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_v *f)
-{
- assert (e->n_subs == 1);
- if (!check_vector_arg (props->name, subs, e, 0)
- || !check_constraints (props, subs, e))
- return NULL;
- gsl_vector v = to_vector (subs[0]);
- return f (&v);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_d_m (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_d_m *f)
-{
- assert (e->n_subs == 1);
-
- if (!check_constraints (props, subs, e))
- return NULL;
-
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, f (subs[0]));
- return m;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_m_any (const struct matrix_function_properties *props,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_m_any *f)
-{
- return check_constraints (props, subs, e) ? f (subs, e->n_subs) : NULL;
-}
-
-static gsl_matrix *
-matrix_expr_evaluate_IDENT (const struct matrix_function_properties *props_ UNUSED,
- gsl_matrix *subs[], const struct matrix_expr *e,
- matrix_proto_IDENT *f)
-{
- static const struct matrix_function_properties p1 = {
- .name = "IDENT",
- .constraints = "ai>=0"
- };
- static const struct matrix_function_properties p2 = {
- .name = "IDENT",
- .constraints = "ai>=0 bi>=0"
- };
- const struct matrix_function_properties *props = e->n_subs == 1 ? &p1 : &p2;
-
- assert (e->n_subs <= 2);
-
- double d[2];
- return (to_scalar_args (props->name, subs, e, d)
- && check_constraints (props, subs, e)
- ? f (d[0], d[e->n_subs - 1])
- : NULL);
-}
-
-static gsl_matrix *
-matrix_expr_evaluate (const struct matrix_expr *e)
-{
- if (e->op == MOP_NUMBER)
- {
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, e->number);
- return m;
- }
- else if (e->op == MOP_VARIABLE)
- {
- const gsl_matrix *src = e->variable->value;
- if (!src)
- {
- msg_at (SE, e->location,
- _("Uninitialized variable %s used in expression."),
- e->variable->name);
- return NULL;
- }
-
- gsl_matrix *dst = gsl_matrix_alloc (src->size1, src->size2);
- gsl_matrix_memcpy (dst, src);
- return dst;
- }
- else if (e->op == MOP_EOF)
- {
- struct dfm_reader *reader = read_file_open (e->eof);
- gsl_matrix *m = gsl_matrix_alloc (1, 1);
- gsl_matrix_set (m, 0, 0, !reader || dfm_eof (reader));
- return m;
- }
-
- enum { N_LOCAL = 3 };
- gsl_matrix *local_subs[N_LOCAL];
- gsl_matrix **subs = (e->n_subs < N_LOCAL
- ? local_subs
- : xmalloc (e->n_subs * sizeof *subs));
-
- for (size_t i = 0; i < e->n_subs; i++)
- {
- subs[i] = matrix_expr_evaluate (e->subs[i]);
- if (!subs[i])
- {
- for (size_t j = 0; j < i; j++)
- gsl_matrix_free (subs[j]);
- if (subs != local_subs)
- free (subs);
- return NULL;
- }
- }
-
- gsl_matrix *result = NULL;
- switch (e->op)
- {
-#define F(ENUM, STRING, PROTO, CONSTRAINTS) \
- case MOP_F_##ENUM: \
- { \
- static const struct matrix_function_properties props = { \
- .name = STRING, \
- .constraints = CONSTRAINTS, \
- }; \
- result = matrix_expr_evaluate_##PROTO (&props, subs, e, \
- matrix_eval_##ENUM); \
- } \
- break;
- MATRIX_FUNCTIONS
-#undef F
-
- case MOP_NEGATE:
- gsl_matrix_scale (subs[0], -1.0);
- result = subs[0];
- break;
-
- case MOP_ADD_ELEMS:
- case MOP_SUB_ELEMS:
- case MOP_MUL_ELEMS:
- case MOP_DIV_ELEMS:
- case MOP_EXP_ELEMS:
- case MOP_GT:
- case MOP_GE:
- case MOP_LT:
- case MOP_LE:
- case MOP_EQ:
- case MOP_NE:
- case MOP_AND:
- case MOP_OR:
- case MOP_XOR:
- result = matrix_expr_evaluate_elementwise (e, e->op, subs[0], subs[1]);
- break;
-
- case MOP_NOT:
- result = matrix_expr_evaluate_not (subs[0]);
- break;
-
- case MOP_SEQ:
- result = matrix_expr_evaluate_seq (e, subs[0], subs[1], NULL);
- break;
-
- case MOP_SEQ_BY:
- result = matrix_expr_evaluate_seq (e, subs[0], subs[1], subs[2]);
- break;
-
- case MOP_MUL_MAT:
- result = matrix_expr_evaluate_mul_mat (e, subs[0], subs[1]);
- break;
-
- case MOP_EXP_MAT:
- result = matrix_expr_evaluate_exp_mat (e, subs[0], subs[1]);
- break;
-
- case MOP_PASTE_HORZ:
- result = matrix_expr_evaluate_paste_horz (e, subs[0], subs[1]);
- break;
-
- case MOP_PASTE_VERT:
- result = matrix_expr_evaluate_paste_vert (e, subs[0], subs[1]);
- break;
-
- case MOP_EMPTY:
- result = gsl_matrix_alloc (0, 0);
- break;
-
- case MOP_VEC_INDEX:
- result = matrix_expr_evaluate_vec_index (e, subs[0], subs[1]);
- break;
-
- case MOP_VEC_ALL:
- result = matrix_expr_evaluate_vec_all (e, subs[0]);
- break;
-
- case MOP_MAT_INDEX:
- result = matrix_expr_evaluate_mat_index (subs[0],
- subs[1], e->subs[1],
- subs[2], e->subs[2]);
- break;
-
- case MOP_ROW_INDEX:
- result = matrix_expr_evaluate_mat_index (subs[0],
- subs[1], e->subs[1],
- NULL, NULL);
- break;
-
- case MOP_COL_INDEX:
- result = matrix_expr_evaluate_mat_index (subs[0],
- NULL, NULL,
- subs[1], e->subs[1]);
- break;
-
- case MOP_NUMBER:
- case MOP_VARIABLE:
- case MOP_EOF:
- NOT_REACHED ();
- }
-
- for (size_t i = 0; i < e->n_subs; i++)
- if (subs[i] != result)
- gsl_matrix_free (subs[i]);
- if (subs != local_subs)
- free (subs);
- return result;
-}
-
-static bool
-matrix_expr_evaluate_scalar (const struct matrix_expr *e, const char *context,
- double *d)
-{
- gsl_matrix *m = matrix_expr_evaluate (e);
- if (!m)
- return false;
-
- if (!is_scalar (m))
- {
- msg_at (SE, matrix_expr_location (e),
- _("Expression for %s must evaluate to scalar, "
- "not a %zu×%zu matrix."),
- context, m->size1, m->size2);
- gsl_matrix_free (m);
- return false;
- }
-
- *d = to_scalar (m);
- gsl_matrix_free (m);
- return true;
-}
-
-static bool
-matrix_expr_evaluate_integer (const struct matrix_expr *e, const char *context,
- long int *integer)
-{
- double d;
- if (!matrix_expr_evaluate_scalar (e, context, &d))
- return false;
-
- d = trunc (d);
- if (d < LONG_MIN || d > LONG_MAX)
- {
- msg_at (SE, matrix_expr_location (e),
- _("Expression for %s is outside the integer range."), context);
- return false;
- }
- *integer = d;
- return true;
-}
-\f
-/* Matrix lvalues.
-
- An lvalue is an expression that can appear on the left side of a COMPUTE
- command and in other contexts that assign values.
-
- An lvalue is parsed once, with matrix_lvalue_parse(). It can then be
- evaluated (with matrix_lvalue_evaluate()) and assigned (with
- matrix_lvalue_assign()).
-
- There are three kinds of lvalues:
-
- - A variable name. A variable used as an lvalue need not be initialized,
- since the assignment will initialize.
-
- - A subvector, e.g. "var(index0)". The variable must be initialized and
- must have the form of a vector (it must have 1 column or 1 row).
-
- - A submatrix, e.g. "var(index0, index1)". The variable must be
- initialized. */
-struct matrix_lvalue
- {
- struct matrix_var *var; /* Destination variable. */
- struct matrix_expr *indexes[2]; /* Index expressions, if any. */
- size_t n_indexes; /* Number of indexes. */
-
- struct msg_location *var_location; /* Variable name. */
- struct msg_location *full_location; /* Variable name plus indexing. */
- struct msg_location *index_locations[2]; /* Index expressions. */
- };
-
-/* Frees LVALUE. */
-static void
-matrix_lvalue_destroy (struct matrix_lvalue *lvalue)
-{
- if (lvalue)
- {
- msg_location_destroy (lvalue->var_location);
- msg_location_destroy (lvalue->full_location);
- for (size_t i = 0; i < lvalue->n_indexes; i++)
- {
- matrix_expr_destroy (lvalue->indexes[i]);
- msg_location_destroy (lvalue->index_locations[i]);
- }
- free (lvalue);
- }
-}
-
-/* Parses and returns an lvalue at the current position in S's lexer. Returns
- null on parse failure. On success, the caller must eventually free the
- lvalue. */
-static struct matrix_lvalue *
-matrix_lvalue_parse (struct matrix_state *s)
-{
- if (!lex_force_id (s->lexer))
- return NULL;
-
- struct matrix_lvalue *lvalue = xzalloc (sizeof *lvalue);
- int start_ofs = lex_ofs (s->lexer);
- lvalue->var_location = lex_get_location (s->lexer, 0, 0);
- lvalue->var = matrix_var_lookup (s, lex_tokss (s->lexer));
- if (lex_next_token (s->lexer, 1) == T_LPAREN)
- {
- if (!lvalue->var)
- {
- lex_error (s->lexer, _("Undefined variable %s."),
- lex_tokcstr (s->lexer));
- goto error;
- }
-
- lex_get_n (s->lexer, 2);
-
- if (!matrix_parse_index_expr (s, &lvalue->indexes[0],
- &lvalue->index_locations[0]))
- goto error;
- lvalue->n_indexes++;
-
- if (lex_match (s->lexer, T_COMMA))
- {
- if (!matrix_parse_index_expr (s, &lvalue->indexes[1],
- &lvalue->index_locations[1]))
- goto error;
- lvalue->n_indexes++;
- }
- if (!lex_force_match (s->lexer, T_RPAREN))
- goto error;
-
- lvalue->full_location = lex_ofs_location (s->lexer, start_ofs,
- lex_ofs (s->lexer) - 1);
- }
- else
- {
- if (!lvalue->var)
- lvalue->var = matrix_var_create (s, lex_tokss (s->lexer));
- lex_get (s->lexer);
- }
- return lvalue;
-
-error:
- matrix_lvalue_destroy (lvalue);
- return NULL;
-}
-
-static bool
-matrix_lvalue_evaluate_vector (struct matrix_expr *e, size_t size,
- enum index_type index_type, size_t other_size,
- struct index_vector *iv)
-{
- gsl_matrix *m;
- if (e)
- {
- m = matrix_expr_evaluate (e);
- if (!m)
- return false;
- }
- else
- m = NULL;
-
- bool ok = matrix_normalize_index_vector (m, e, size, index_type,
- other_size, iv);
- gsl_matrix_free (m);
- return ok;
-}
-
-/* Evaluates the indexes in LVALUE into IV0 and IV1, owned by the caller.
- Returns true if successful, false if evaluating the expressions failed or if
- LVALUE otherwise can't be used for an assignment.
-
- On success, the caller retains ownership of the index vectors, which are
- suitable for passing to matrix_lvalue_assign(). If not used for that
- purpose then they need to eventually be freed (with
- index_vector_uninit()). */
-static bool
-matrix_lvalue_evaluate (struct matrix_lvalue *lvalue,
- struct index_vector *iv0,
- struct index_vector *iv1)
-{
- *iv0 = INDEX_VECTOR_INIT;
- *iv1 = INDEX_VECTOR_INIT;
- if (!lvalue->n_indexes)
- return true;
-
- /* Validate destination matrix exists and has the right shape. */
- gsl_matrix *dm = lvalue->var->value;
- if (!dm)
- {
- msg_at (SE, lvalue->var_location,
- _("Undefined variable %s."), lvalue->var->name);
- return false;
- }
- else if (dm->size1 == 0 || dm->size2 == 0)
- {
- msg_at (SE, lvalue->full_location, _("Cannot index %zu×%zu matrix %s."),
- dm->size1, dm->size2, lvalue->var->name);
- return false;
- }
- else if (lvalue->n_indexes == 1)
- {
- if (!is_vector (dm))
- {
- msg_at (SE, lvalue->full_location,
- _("Can't use vector indexing on %zu×%zu matrix %s."),
- dm->size1, dm->size2, lvalue->var->name);
- return false;
- }
- return matrix_lvalue_evaluate_vector (lvalue->indexes[0],
- MAX (dm->size1, dm->size2),
- IV_VECTOR, 0, iv0);
- }
- else
- {
- assert (lvalue->n_indexes == 2);
- if (!matrix_lvalue_evaluate_vector (lvalue->indexes[0], dm->size1,
- IV_ROW, dm->size2, iv0))
- return false;
-
- if (!matrix_lvalue_evaluate_vector (lvalue->indexes[1], dm->size2,
- IV_COLUMN, dm->size1, iv1))
- {
- index_vector_uninit (iv0);
- return false;
- }
- return true;
- }
-}
-
-static bool
-matrix_lvalue_assign_vector (struct matrix_lvalue *lvalue,
- struct index_vector *iv,
- gsl_matrix *sm, const struct msg_location *lsm)
-{
- /* Convert source matrix 'sm' to source vector 'sv'. */
- if (!is_vector (sm))
- {
- msg_at (SE, lvalue->full_location,
- _("Only an %zu-element vector may be assigned to this "
- "%zu-element subvector of %s."),
- iv->n, iv->n, lvalue->var->name);
- msg_at (SE, lsm,
- _("The source is an %zu×%zu matrix."),
- sm->size1, sm->size2);
- return false;
- }
- gsl_vector sv = to_vector (sm);
- if (iv->n != sv.size)
- {
- msg_at (SE, lvalue->full_location,
- _("Only an %zu-element vector may be assigned to this "
- "%zu-element subvector of %s."),
- iv->n, iv->n, lvalue->var->name);
- msg_at (SE, lsm, ngettext ("The source vector has %zu element.",
- "The source vector has %zu elements.",
- sv.size),
- sv.size);
- return false;
- }
-
- /* Assign elements. */
- gsl_vector dv = to_vector (lvalue->var->value);
- for (size_t x = 0; x < iv->n; x++)
- gsl_vector_set (&dv, iv->indexes[x], gsl_vector_get (&sv, x));
- return true;
-}
-
-static bool
-matrix_lvalue_assign_matrix (struct matrix_lvalue *lvalue,
- struct index_vector *iv0,
- struct index_vector *iv1,
- gsl_matrix *sm, const struct msg_location *lsm)
-{
- gsl_matrix *dm = lvalue->var->value;
-
- /* Convert source matrix 'sm' to source vector 'sv'. */
- bool wrong_rows = iv0->n != sm->size1;
- bool wrong_columns = iv1->n != sm->size2;
- if (wrong_rows || wrong_columns)
- {
- if (wrong_rows && wrong_columns)
- msg_at (SE, lvalue->full_location,
- _("Numbers of indexes for assigning to %s differ from the "
- "size of the source matrix."),
- lvalue->var->name);
- else if (wrong_rows)
- msg_at (SE, lvalue->full_location,
- _("Number of row indexes for assigning to %s differs from "
- "number of rows in source matrix."),
- lvalue->var->name);
- else
- msg_at (SE, lvalue->full_location,
- _("Number of column indexes for assigning to %s differs from "
- "number of columns in source matrix."),
- lvalue->var->name);
-
- if (wrong_rows)
- {
- if (lvalue->indexes[0])
- msg_at (SN, lvalue->index_locations[0],
- ngettext ("There is %zu row index.",
- "There are %zu row indexes.",
- iv0->n),
- iv0->n);
- else
- msg_at (SN, lvalue->index_locations[0],
- ngettext ("Destination matrix %s has %zu row.",
- "Destination matrix %s has %zu rows.",
- iv0->n),
- lvalue->var->name, iv0->n);
- }
-
- if (wrong_columns)
- {
- if (lvalue->indexes[1])
- msg_at (SN, lvalue->index_locations[1],
- ngettext ("There is %zu column index.",
- "There are %zu column indexes.",
- iv1->n),
- iv1->n);
- else
- msg_at (SN, lvalue->index_locations[1],
- ngettext ("Destination matrix %s has %zu column.",
- "Destination matrix %s has %zu columns.",
- iv1->n),
- lvalue->var->name, iv1->n);
- }
-
- msg_at (SN, lsm, _("The source matrix is %zu×%zu."),
- sm->size1, sm->size2);
- return false;
- }
-
- /* Assign elements. */
- for (size_t y = 0; y < iv0->n; y++)
- {
- size_t f0 = iv0->indexes[y];
- for (size_t x = 0; x < iv1->n; x++)
- {
- size_t f1 = iv1->indexes[x];
- gsl_matrix_set (dm, f0, f1, gsl_matrix_get (sm, y, x));
- }
- }
- return true;
-}
-
-/* Assigns rvalue SM to LVALUE, whose index vectors IV0 and IV1 were previously
- obtained with matrix_lvalue_evaluate(). Returns true if successful, false
- on error. Always takes ownership of M. LSM should be the source location
- to use for errors related to SM. */
-static bool
-matrix_lvalue_assign (struct matrix_lvalue *lvalue,
- struct index_vector *iv0, struct index_vector *iv1,
- gsl_matrix *sm, const struct msg_location *lsm)
-{
- if (!lvalue->n_indexes)
- {
- gsl_matrix_free (lvalue->var->value);
- lvalue->var->value = sm;
- return true;
- }
- else
- {
- bool ok = (lvalue->n_indexes == 1
- ? matrix_lvalue_assign_vector (lvalue, iv0, sm, lsm)
- : matrix_lvalue_assign_matrix (lvalue, iv0, iv1, sm, lsm));
- index_vector_uninit (iv0);
- index_vector_uninit (iv1);
- gsl_matrix_free (sm);
- return ok;
- }
-}
-
-/* Evaluates and then assigns SM to LVALUE. Always takes ownership of M. LSM
- should be the source location to use for errors related to SM.*/
-static bool
-matrix_lvalue_evaluate_and_assign (struct matrix_lvalue *lvalue,
- gsl_matrix *sm,
- const struct msg_location *lsm)
-{
- struct index_vector iv0, iv1;
- if (!matrix_lvalue_evaluate (lvalue, &iv0, &iv1))
- {
- gsl_matrix_free (sm);
- return false;
- }
-
- return matrix_lvalue_assign (lvalue, &iv0, &iv1, sm, lsm);
-}
-\f
-/* Matrix command data structure. */
-
-/* An array of matrix commands. */
-struct matrix_commands
- {
- struct matrix_command **commands;
- size_t n;
- };
-
-static bool matrix_commands_parse (struct matrix_state *,
- struct matrix_commands *,
- const char *command_name,
- const char *stop1, const char *stop2);
-static void matrix_commands_uninit (struct matrix_commands *);
-
-/* A single matrix command. */
-struct matrix_command
- {
- /* The type of command. */
- enum matrix_command_type
- {
- MCMD_COMPUTE,
- MCMD_PRINT,
- MCMD_DO_IF,
- MCMD_LOOP,
- MCMD_BREAK,
- MCMD_DISPLAY,
- MCMD_RELEASE,
- MCMD_SAVE,
- MCMD_READ,
- MCMD_WRITE,
- MCMD_GET,
- MCMD_MSAVE,
- MCMD_MGET,
- MCMD_EIGEN,
- MCMD_SETDIAG,
- MCMD_SVD,
- }
- type;
-
- /* Source lines for this command. */
- struct msg_location *location;
-
- union
- {
- struct matrix_compute
- {
- struct matrix_lvalue *lvalue;
- struct matrix_expr *rvalue;
- }
- compute;
-
- struct matrix_print
- {
- struct matrix_expr *expression;
- bool use_default_format;
- struct fmt_spec format;
- char *title;
- int space; /* -1 means NEWPAGE. */
-
- struct print_labels
- {
- struct string_array literals; /* CLABELS/RLABELS. */
- struct matrix_expr *expr; /* CNAMES/RNAMES. */
- }
- *rlabels, *clabels;
- }
- print;
-
- struct matrix_do_if
- {
- struct do_if_clause
- {
- struct matrix_expr *condition;
- struct matrix_commands commands;
- }
- *clauses;
- size_t n_clauses;
- }
- do_if;
-
- struct matrix_loop
- {
- /* Index. */
- struct matrix_var *var;
- struct matrix_expr *start;
- struct matrix_expr *end;
- struct matrix_expr *increment;
-
- /* Loop conditions. */
- struct matrix_expr *top_condition;
- struct matrix_expr *bottom_condition;
-
- /* Commands. */
- struct matrix_commands commands;
- }
- loop;
-
- struct matrix_display
- {
- struct matrix_state *state;
- }
- display;
-
- struct matrix_release
- {
- struct matrix_var **vars;
- size_t n_vars;
- }
- release;
-
- struct matrix_save
- {
- struct matrix_expr *expression;
- struct save_file *sf;
- }
- save;
-
- struct matrix_read
- {
- struct read_file *rf;
- struct matrix_lvalue *dst;
- struct matrix_expr *size;
- int c1, c2;
- enum fmt_type format;
- int w;
- bool symmetric;
- bool reread;
- }
- read;
-
- struct matrix_write
- {
- struct write_file *wf;
- struct matrix_expr *expression;
- int c1, c2;
-
- /* If this is nonnull, WRITE uses this format.
-
- If this is NULL, WRITE uses free-field format with as many
- digits of precision as needed. */
- struct fmt_spec *format;
-
- bool triangular;
- bool hold;
- }
- write;
-
- struct matrix_get
- {
- struct lexer *lexer;
- struct matrix_lvalue *dst;
- struct dataset *dataset;
- struct file_handle *file;
- char *encoding;
- struct var_syntax *vars;
- size_t n_vars;
- struct matrix_var *names;
-
- /* Treatment of missing values. */
- struct
- {
- enum
- {
- MGET_ERROR, /* Flag error on command. */
- MGET_ACCEPT, /* Accept without change, user-missing only. */
- MGET_OMIT, /* Drop this case. */
- MGET_RECODE /* Recode to 'substitute'. */
- }
- treatment;
- double substitute; /* MGET_RECODE only. */
- }
- user, system;
- }
- get;
-
- struct matrix_msave
- {
- struct msave_common *common;
- struct matrix_expr *expr;
- const char *rowtype;
- const struct matrix_expr *factors;
- const struct matrix_expr *splits;
- }
- msave;
-
- struct matrix_mget
- {
- struct matrix_state *state;
- struct file_handle *file;
- char *encoding;
- struct stringi_set rowtypes;
- }
- mget;
-
- struct matrix_eigen
- {
- struct matrix_expr *expr;
- struct matrix_var *evec;
- struct matrix_var *eval;
- }
- eigen;
-
- struct matrix_setdiag
- {
- struct matrix_var *dst;
- struct matrix_expr *expr;
- }
- setdiag;
-
- struct matrix_svd
- {
- struct matrix_expr *expr;
- struct matrix_var *u;
- struct matrix_var *s;
- struct matrix_var *v;
- }
- svd;
- };
- };
-
-static struct matrix_command *matrix_command_parse (struct matrix_state *);
-static bool matrix_command_execute (struct matrix_command *);
-static void matrix_command_destroy (struct matrix_command *);
-\f
-/* COMPUTE. */
-
-static struct matrix_command *
-matrix_compute_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_COMPUTE,
- .compute = { .lvalue = NULL },
- };
-
- struct matrix_compute *compute = &cmd->compute;
- compute->lvalue = matrix_lvalue_parse (s);
- if (!compute->lvalue)
- goto error;
-
- if (!lex_force_match (s->lexer, T_EQUALS))
- goto error;
-
- compute->rvalue = matrix_expr_parse (s);
- if (!compute->rvalue)
- goto error;
-
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_compute_execute (struct matrix_command *cmd)
-{
- struct matrix_compute *compute = &cmd->compute;
- gsl_matrix *value = matrix_expr_evaluate (compute->rvalue);
- if (!value)
- return;
-
- matrix_lvalue_evaluate_and_assign (compute->lvalue, value,
- matrix_expr_location (compute->rvalue));
-}
-\f
-/* PRINT. */
-
-static void
-print_labels_destroy (struct print_labels *labels)
-{
- if (labels)
- {
- string_array_destroy (&labels->literals);
- matrix_expr_destroy (labels->expr);
- free (labels);
- }
-}
-
-static void
-parse_literal_print_labels (struct matrix_state *s,
- struct print_labels **labelsp)
-{
- lex_match (s->lexer, T_EQUALS);
- print_labels_destroy (*labelsp);
- *labelsp = xzalloc (sizeof **labelsp);
- while (lex_token (s->lexer) != T_SLASH
- && lex_token (s->lexer) != T_ENDCMD
- && lex_token (s->lexer) != T_STOP)
- {
- struct string label = DS_EMPTY_INITIALIZER;
- while (lex_token (s->lexer) != T_COMMA
- && lex_token (s->lexer) != T_SLASH
- && lex_token (s->lexer) != T_ENDCMD
- && lex_token (s->lexer) != T_STOP)
- {
- if (!ds_is_empty (&label))
- ds_put_byte (&label, ' ');
-
- if (lex_token (s->lexer) == T_STRING)
- ds_put_cstr (&label, lex_tokcstr (s->lexer));
- else
- {
- char *rep = lex_next_representation (s->lexer, 0, 0);
- ds_put_cstr (&label, rep);
- free (rep);
- }
- lex_get (s->lexer);
- }
- string_array_append_nocopy (&(*labelsp)->literals,
- ds_steal_cstr (&label));
-
- if (!lex_match (s->lexer, T_COMMA))
- break;
- }
-}
-
-static bool
-parse_expr_print_labels (struct matrix_state *s, struct print_labels **labelsp)
-{
- lex_match (s->lexer, T_EQUALS);
- struct matrix_expr *e = matrix_parse_exp (s);
- if (!e)
- return false;
-
- print_labels_destroy (*labelsp);
- *labelsp = xzalloc (sizeof **labelsp);
- (*labelsp)->expr = e;
- return true;
-}
-
-static struct matrix_command *
-matrix_print_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_PRINT,
- .print = {
- .use_default_format = true,
- }
- };
-
- if (lex_token (s->lexer) != T_SLASH && lex_token (s->lexer) != T_ENDCMD)
- {
- int start_ofs = lex_ofs (s->lexer);
- cmd->print.expression = matrix_parse_exp (s);
- if (!cmd->print.expression)
- goto error;
- cmd->print.title = lex_ofs_representation (s->lexer, start_ofs,
- lex_ofs (s->lexer) - 1);
- }
-
- while (lex_match (s->lexer, T_SLASH))
- {
- if (lex_match_id (s->lexer, "FORMAT"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!parse_format_specifier (s->lexer, &cmd->print.format))
- goto error;
- cmd->print.use_default_format = false;
- }
- else if (lex_match_id (s->lexer, "TITLE"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!lex_force_string (s->lexer))
- goto error;
- free (cmd->print.title);
- cmd->print.title = ss_xstrdup (lex_tokss (s->lexer));
- lex_get (s->lexer);
- }
- else if (lex_match_id (s->lexer, "SPACE"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (lex_match_id (s->lexer, "NEWPAGE"))
- cmd->print.space = -1;
- else if (lex_force_int_range (s->lexer, "SPACE", 1, INT_MAX))
- {
- cmd->print.space = lex_integer (s->lexer);
- lex_get (s->lexer);
- }
- else
- goto error;
- }
- else if (lex_match_id (s->lexer, "RLABELS"))
- parse_literal_print_labels (s, &cmd->print.rlabels);
- else if (lex_match_id (s->lexer, "CLABELS"))
- parse_literal_print_labels (s, &cmd->print.clabels);
- else if (lex_match_id (s->lexer, "RNAMES"))
- {
- if (!parse_expr_print_labels (s, &cmd->print.rlabels))
- goto error;
- }
- else if (lex_match_id (s->lexer, "CNAMES"))
- {
- if (!parse_expr_print_labels (s, &cmd->print.clabels))
- goto error;
- }
- else
- {
- lex_error_expecting (s->lexer, "FORMAT", "TITLE", "SPACE",
- "RLABELS", "CLABELS", "RNAMES", "CNAMES");
- goto error;
- }
-
- }
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static bool
-matrix_is_integer (const gsl_matrix *m)
-{
- for (size_t y = 0; y < m->size1; y++)
- for (size_t x = 0; x < m->size2; x++)
- {
- double d = gsl_matrix_get (m, y, x);
- if (d != floor (d))
- return false;
- }
- return true;
-}
-
-static double
-matrix_max_magnitude (const gsl_matrix *m)
-{
- double max = 0.0;
- for (size_t y = 0; y < m->size1; y++)
- for (size_t x = 0; x < m->size2; x++)
- {
- double d = fabs (gsl_matrix_get (m, y, x));
- if (d > max)
- max = d;
- }
- return max;
-}
-
-static bool
-format_fits (struct fmt_spec format, double x)
-{
- char *s = data_out (&(union value) { .f = x }, NULL,
- &format, settings_get_fmt_settings ());
- bool fits = *s != '*' && !strchr (s, 'E');
- free (s);
- return fits;
-}
-
-static struct fmt_spec
-default_format (const gsl_matrix *m, int *log_scale)
-{
- *log_scale = 0;
-
- double max = matrix_max_magnitude (m);
-
- if (matrix_is_integer (m))
- for (int w = 1; w <= 12; w++)
- {
- struct fmt_spec format = { .type = FMT_F, .w = w };
- if (format_fits (format, -max))
- return format;
- }
-
- if (max >= 1e9 || max <= 1e-4)
- {
- char s[64];
- snprintf (s, sizeof s, "%.1e", max);
-
- const char *e = strchr (s, 'e');
- if (e)
- *log_scale = atoi (e + 1);
- }
-
- return (struct fmt_spec) { .type = FMT_F, .w = 13, .d = 10 };
-}
-
-static char *
-trimmed_string (double d)
-{
- struct substring s = ss_buffer ((char *) &d, sizeof d);
- ss_rtrim (&s, ss_cstr (" "));
- return ss_xstrdup (s);
-}
-
-static struct string_array *
-print_labels_get (const struct print_labels *labels, size_t n,
- const char *prefix, bool truncate)
-{
- if (!labels)
- return NULL;
-
- struct string_array *out = xzalloc (sizeof *out);
- if (labels->literals.n)
- string_array_clone (out, &labels->literals);
- else if (labels->expr)
- {
- gsl_matrix *m = matrix_expr_evaluate (labels->expr);
- if (m && is_vector (m))
- {
- gsl_vector v = to_vector (m);
- for (size_t i = 0; i < v.size; i++)
- string_array_append_nocopy (out, trimmed_string (
- gsl_vector_get (&v, i)));
- }
- gsl_matrix_free (m);
- }
-
- while (out->n < n)
- {
- if (labels->expr)
- string_array_append_nocopy (out, xasprintf ("%s%zu", prefix,
- out->n + 1));
- else
- string_array_append (out, "");
- }
- while (out->n > n)
- string_array_delete (out, out->n - 1);
-
- if (truncate)
- for (size_t i = 0; i < out->n; i++)
- {
- char *s = out->strings[i];
- s[strnlen (s, 8)] = '\0';
- }
-
- return out;
-}
-
-static void
-matrix_print_space (int space)
-{
- if (space < 0)
- output_item_submit (page_break_item_create ());
- for (int i = 0; i < space; i++)
- output_log ("%s", "");
-}
-
-static void
-matrix_print_text (const struct matrix_print *print, const gsl_matrix *m,
- struct fmt_spec format, int log_scale)
-{
- matrix_print_space (print->space);
- if (print->title)
- output_log ("%s", print->title);
- if (log_scale != 0)
- output_log (" 10 ** %d X", log_scale);
-
- struct string_array *clabels = print_labels_get (print->clabels,
- m->size2, "col", true);
- if (clabels && format.w < 8)
- format.w = 8;
- struct string_array *rlabels = print_labels_get (print->rlabels,
- m->size1, "row", true);
-
- if (clabels)
- {
- struct string line = DS_EMPTY_INITIALIZER;
- if (rlabels)
- ds_put_byte_multiple (&line, ' ', 8);
- for (size_t x = 0; x < m->size2; x++)
- ds_put_format (&line, " %*s", format.w, clabels->strings[x]);
- output_log_nocopy (ds_steal_cstr (&line));
- }
-
- double scale = pow (10.0, log_scale);
- bool numeric = fmt_is_numeric (format.type);
- for (size_t y = 0; y < m->size1; y++)
- {
- struct string line = DS_EMPTY_INITIALIZER;
- if (rlabels)
- ds_put_format (&line, "%-8s", rlabels->strings[y]);
-
- for (size_t x = 0; x < m->size2; x++)
- {
- double f = gsl_matrix_get (m, y, x);
- char *s = (numeric
- ? data_out (&(union value) { .f = f / scale}, NULL,
- &format, settings_get_fmt_settings ())
- : trimmed_string (f));
- ds_put_format (&line, " %s", s);
- free (s);
- }
- output_log_nocopy (ds_steal_cstr (&line));
- }
-
- string_array_destroy (rlabels);
- free (rlabels);
- string_array_destroy (clabels);
- free (clabels);
-}
-
-static void
-create_print_dimension (struct pivot_table *table, enum pivot_axis_type axis,
- const struct print_labels *print_labels, size_t n,
- const char *name, const char *prefix)
-{
- struct string_array *labels = print_labels_get (print_labels, n, prefix,
- false);
- struct pivot_dimension *d = pivot_dimension_create (table, axis, name);
- for (size_t i = 0; i < n; i++)
- pivot_category_create_leaf (
- d->root, (labels
- ? pivot_value_new_user_text (labels->strings[i], SIZE_MAX)
- : pivot_value_new_integer (i + 1)));
- if (!labels)
- d->hide_all_labels = true;
- string_array_destroy (labels);
- free (labels);
-}
-
-static void
-matrix_print_tables (const struct matrix_print *print, const gsl_matrix *m,
- struct fmt_spec format, int log_scale)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_user_text (print->title ? print->title : "", SIZE_MAX),
- "Matrix Print");
-
- create_print_dimension (table, PIVOT_AXIS_ROW, print->rlabels, m->size1,
- N_("Rows"), "row");
- create_print_dimension (table, PIVOT_AXIS_COLUMN, print->clabels, m->size2,
- N_("Columns"), "col");
-
- struct pivot_footnote *footnote = NULL;
- if (log_scale != 0)
- {
- char *s = xasprintf ("× 10**%d", log_scale);
- footnote = pivot_table_create_footnote (
- table, pivot_value_new_user_text_nocopy (s));
- }
-
- double scale = pow (10.0, log_scale);
- bool numeric = fmt_is_numeric (format.type);
- for (size_t y = 0; y < m->size1; y++)
- for (size_t x = 0; x < m->size2; x++)
- {
- double f = gsl_matrix_get (m, y, x);
- struct pivot_value *v;
- if (numeric)
- {
- v = pivot_value_new_number (f / scale);
- v->numeric.format = format;
- }
- else
- v = pivot_value_new_user_text_nocopy (trimmed_string (f));
- if (footnote)
- pivot_value_add_footnote (v, footnote);
- pivot_table_put2 (table, y, x, v);
- }
-
- pivot_table_submit (table);
-}
-
-static void
-matrix_print_execute (const struct matrix_print *print)
-{
- if (print->expression)
- {
- gsl_matrix *m = matrix_expr_evaluate (print->expression);
- if (!m)
- return;
-
- int log_scale = 0;
- struct fmt_spec format = (print->use_default_format
- ? default_format (m, &log_scale)
- : print->format);
-
- if (settings_get_mdisplay () == SETTINGS_MDISPLAY_TEXT)
- matrix_print_text (print, m, format, log_scale);
- else
- matrix_print_tables (print, m, format, log_scale);
-
- gsl_matrix_free (m);
- }
- else
- {
- matrix_print_space (print->space);
- if (print->title)
- output_log ("%s", print->title);
- }
-}
-\f
-/* DO IF. */
-
-static bool
-matrix_do_if_clause_parse (struct matrix_state *s,
- struct matrix_do_if *ifc,
- bool parse_condition,
- size_t *allocated_clauses)
-{
- if (ifc->n_clauses >= *allocated_clauses)
- ifc->clauses = x2nrealloc (ifc->clauses, allocated_clauses,
- sizeof *ifc->clauses);
- struct do_if_clause *c = &ifc->clauses[ifc->n_clauses++];
- *c = (struct do_if_clause) { .condition = NULL };
-
- if (parse_condition)
- {
- c->condition = matrix_expr_parse (s);
- if (!c->condition)
- return false;
- }
-
- return matrix_commands_parse (s, &c->commands, "DO IF", "ELSE", "END IF");
-}
-
-static struct matrix_command *
-matrix_do_if_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_DO_IF,
- .do_if = { .n_clauses = 0 }
- };
-
- size_t allocated_clauses = 0;
- do
- {
- if (!matrix_do_if_clause_parse (s, &cmd->do_if, true, &allocated_clauses))
- goto error;
- }
- while (lex_match_phrase (s->lexer, "ELSE IF"));
-
- if (lex_match_id (s->lexer, "ELSE")
- && !matrix_do_if_clause_parse (s, &cmd->do_if, false, &allocated_clauses))
- goto error;
-
- if (!lex_match_phrase (s->lexer, "END IF"))
- NOT_REACHED ();
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static bool
-matrix_do_if_execute (struct matrix_do_if *cmd)
-{
- for (size_t i = 0; i < cmd->n_clauses; i++)
- {
- struct do_if_clause *c = &cmd->clauses[i];
- if (c->condition)
- {
- double d;
- if (!matrix_expr_evaluate_scalar (c->condition,
- i ? "ELSE IF" : "DO IF",
- &d) || d <= 0)
- continue;
- }
-
- for (size_t j = 0; j < c->commands.n; j++)
- if (!matrix_command_execute (c->commands.commands[j]))
- return false;
- return true;
- }
- return true;
-}
-\f
-/* LOOP. */
-
-static struct matrix_command *
-matrix_loop_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) { .type = MCMD_LOOP, .loop = { .var = NULL } };
-
- struct matrix_loop *loop = &cmd->loop;
- if (lex_token (s->lexer) == T_ID && lex_next_token (s->lexer, 1) == T_EQUALS)
- {
- struct substring name = lex_tokss (s->lexer);
- loop->var = matrix_var_lookup (s, name);
- if (!loop->var)
- loop->var = matrix_var_create (s, name);
-
- lex_get (s->lexer);
- lex_get (s->lexer);
-
- loop->start = matrix_expr_parse (s);
- if (!loop->start || !lex_force_match (s->lexer, T_TO))
- goto error;
-
- loop->end = matrix_expr_parse (s);
- if (!loop->end)
- goto error;
-
- if (lex_match (s->lexer, T_BY))
- {
- loop->increment = matrix_expr_parse (s);
- if (!loop->increment)
- goto error;
- }
- }
-
- if (lex_match_id (s->lexer, "IF"))
- {
- loop->top_condition = matrix_expr_parse (s);
- if (!loop->top_condition)
- goto error;
- }
-
- bool was_in_loop = s->in_loop;
- s->in_loop = true;
- bool ok = matrix_commands_parse (s, &loop->commands, "LOOP",
- "END LOOP", NULL);
- s->in_loop = was_in_loop;
- if (!ok)
- goto error;
-
- if (!lex_match_phrase (s->lexer, "END LOOP"))
- NOT_REACHED ();
-
- if (lex_match_id (s->lexer, "IF"))
- {
- loop->bottom_condition = matrix_expr_parse (s);
- if (!loop->bottom_condition)
- goto error;
- }
-
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_loop_execute (struct matrix_loop *cmd)
-{
- long int value = 0;
- long int end = 0;
- long int increment = 1;
- if (cmd->var)
- {
- if (!matrix_expr_evaluate_integer (cmd->start, "LOOP", &value)
- || !matrix_expr_evaluate_integer (cmd->end, "TO", &end)
- || (cmd->increment
- && !matrix_expr_evaluate_integer (cmd->increment, "BY",
- &increment)))
- return;
-
- if (increment > 0 ? value > end
- : increment < 0 ? value < end
- : true)
- return;
- }
-
- int mxloops = settings_get_mxloops ();
- for (int i = 0; i < mxloops; i++)
- {
- if (cmd->var)
- {
- if (cmd->var->value
- && (cmd->var->value->size1 != 1 || cmd->var->value->size2 != 1))
- {
- gsl_matrix_free (cmd->var->value);
- cmd->var->value = NULL;
- }
- if (!cmd->var->value)
- cmd->var->value = gsl_matrix_alloc (1, 1);
-
- gsl_matrix_set (cmd->var->value, 0, 0, value);
- }
-
- if (cmd->top_condition)
- {
- double d;
- if (!matrix_expr_evaluate_scalar (cmd->top_condition, "LOOP IF",
- &d) || d <= 0)
- return;
- }
-
- for (size_t j = 0; j < cmd->commands.n; j++)
- if (!matrix_command_execute (cmd->commands.commands[j]))
- return;
-
- if (cmd->bottom_condition)
- {
- double d;
- if (!matrix_expr_evaluate_scalar (cmd->bottom_condition,
- "END LOOP IF", &d) || d > 0)
- return;
- }
-
- if (cmd->var)
- {
- value += increment;
- if (increment > 0 ? value > end : value < end)
- return;
- }
- }
-}
-\f
-/* BREAK. */
-
-static struct matrix_command *
-matrix_break_parse (struct matrix_state *s)
-{
- if (!s->in_loop)
- {
- lex_next_error (s->lexer, -1, -1, _("BREAK not inside LOOP."));
- return NULL;
- }
-
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) { .type = MCMD_BREAK };
- return cmd;
-}
-\f
-/* DISPLAY. */
-
-static struct matrix_command *
-matrix_display_parse (struct matrix_state *s)
-{
- while (lex_token (s->lexer) != T_ENDCMD)
- {
- if (!lex_match_id (s->lexer, "DICTIONARY")
- && !lex_match_id (s->lexer, "STATUS"))
- {
- lex_error_expecting (s->lexer, "DICTIONARY", "STATUS");
- return NULL;
- }
- }
-
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) { .type = MCMD_DISPLAY, .display = { s } };
- return cmd;
-}
-
-static int
-compare_matrix_var_pointers (const void *a_, const void *b_)
-{
- const struct matrix_var *const *ap = a_;
- const struct matrix_var *const *bp = b_;
- const struct matrix_var *a = *ap;
- const struct matrix_var *b = *bp;
- return strcmp (a->name, b->name);
-}
-
-static void
-matrix_display_execute (struct matrix_display *cmd)
-{
- const struct matrix_state *s = cmd->state;
-
- struct pivot_table *table = pivot_table_create (N_("Matrix Variables"));
- struct pivot_dimension *attr_dimension
- = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Attribute"));
- pivot_category_create_group (attr_dimension->root, N_("Dimension"),
- N_("Rows"), N_("Columns"));
- pivot_category_create_leaves (attr_dimension->root, N_("Size (kB)"));
-
- const struct matrix_var **vars = xmalloc (hmap_count (&s->vars) * sizeof *vars);
- size_t n_vars = 0;
-
- const struct matrix_var *var;
- HMAP_FOR_EACH (var, struct matrix_var, hmap_node, &s->vars)
- if (var->value)
- vars[n_vars++] = var;
- qsort (vars, n_vars, sizeof *vars, compare_matrix_var_pointers);
-
- struct pivot_dimension *rows = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
- for (size_t i = 0; i < n_vars; i++)
- {
- const struct matrix_var *var = vars[i];
- pivot_category_create_leaf (
- rows->root, pivot_value_new_user_text (var->name, SIZE_MAX));
-
- size_t r = var->value->size1;
- size_t c = var->value->size2;
- double values[] = { r, c, r * c * sizeof (double) / 1024 };
- for (size_t j = 0; j < sizeof values / sizeof *values; j++)
- pivot_table_put2 (table, j, i, pivot_value_new_integer (values[j]));
- }
- free (vars);
- pivot_table_submit (table);
-}
-\f
-/* RELEASE. */
-
-static struct matrix_command *
-matrix_release_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) { .type = MCMD_RELEASE };
-
- size_t allocated_vars = 0;
- while (lex_token (s->lexer) == T_ID)
- {
- struct matrix_var *var = matrix_var_lookup (s, lex_tokss (s->lexer));
- if (var)
- {
- if (cmd->release.n_vars >= allocated_vars)
- cmd->release.vars = x2nrealloc (cmd->release.vars, &allocated_vars,
- sizeof *cmd->release.vars);
- cmd->release.vars[cmd->release.n_vars++] = var;
- }
- else
- lex_error (s->lexer, _("Syntax error expecting variable name."));
- lex_get (s->lexer);
-
- if (!lex_match (s->lexer, T_COMMA))
- break;
- }
-
- return cmd;
-}
-
-static void
-matrix_release_execute (struct matrix_release *cmd)
-{
- for (size_t i = 0; i < cmd->n_vars; i++)
- {
- struct matrix_var *var = cmd->vars[i];
- gsl_matrix_free (var->value);
- var->value = NULL;
- }
-}
-\f
-/* SAVE. */
-
-static struct save_file *
-save_file_create (struct matrix_state *s, struct file_handle *fh,
- struct string_array *variables,
- struct matrix_expr *names,
- struct stringi_set *strings)
-{
- for (size_t i = 0; i < s->n_save_files; i++)
- {
- struct save_file *sf = s->save_files[i];
- if (fh_equal (sf->file, fh))
- {
- fh_unref (fh);
-
- string_array_destroy (variables);
- matrix_expr_destroy (names);
- stringi_set_destroy (strings);
-
- return sf;
- }
- }
-
- struct save_file *sf = xmalloc (sizeof *sf);
- *sf = (struct save_file) {
- .file = fh,
- .dataset = s->dataset,
- .variables = *variables,
- .names = names,
- .strings = STRINGI_SET_INITIALIZER (sf->strings),
- };
-
- stringi_set_swap (&sf->strings, strings);
- stringi_set_destroy (strings);
-
- s->save_files = xrealloc (s->save_files,
- (s->n_save_files + 1) * sizeof *s->save_files);
- s->save_files[s->n_save_files++] = sf;
- return sf;
-}
-
-static struct casewriter *
-save_file_open (struct save_file *sf, gsl_matrix *m,
- const struct msg_location *save_location)
-{
- if (sf->writer || sf->error)
- {
- if (sf->writer)
- {
- size_t n_variables = caseproto_get_n_widths (
- casewriter_get_proto (sf->writer));
- if (m->size2 != n_variables)
- {
- const char *file_name = (sf->file == fh_inline_file () ? "*"
- : fh_get_name (sf->file));
- msg_at (SE, save_location,
- _("Cannot save %zu×%zu matrix to %s because the "
- "first SAVE to %s in this matrix program wrote a "
- "%zu-column matrix."),
- m->size1, m->size2, file_name, file_name, n_variables);
- msg_at (SE, sf->location,
- _("This is the location of the first SAVE to %s."),
- file_name);
- return NULL;
- }
- }
- return sf->writer;
- }
-
- bool ok = true;
- struct dictionary *dict = dict_create (get_default_encoding ());
-
- /* Fill 'names' with user-specified names if there were any, either from
- sf->variables or sf->names. */
- struct string_array names = { .n = 0 };
- if (sf->variables.n)
- string_array_clone (&names, &sf->variables);
- else if (sf->names)
- {
- gsl_matrix *nm = matrix_expr_evaluate (sf->names);
- if (nm && is_vector (nm))
- {
- gsl_vector nv = to_vector (nm);
- for (size_t i = 0; i < nv.size; i++)
- {
- char *name = trimmed_string (gsl_vector_get (&nv, i));
- char *error = dict_id_is_valid__ (dict, name);
- if (!error)
- string_array_append_nocopy (&names, name);
- else
- {
- msg_at (SE, save_location, "%s", error);
- free (error);
- ok = false;
- }
- }
- }
- gsl_matrix_free (nm);
- }
-
- struct stringi_set strings;
- stringi_set_clone (&strings, &sf->strings);
-
- for (size_t i = 0; dict_get_n_vars (dict) < m->size2; i++)
- {
- char tmp_name[64];
- const char *name;
- if (i < names.n)
- name = names.strings[i];
- else
- {
- snprintf (tmp_name, sizeof tmp_name, "COL%zu", i + 1);
- name = tmp_name;
- }
-
- int width = stringi_set_delete (&strings, name) ? 8 : 0;
- struct variable *var = dict_create_var (dict, name, width);
- if (!var)
- {
- msg_at (ME, save_location,
- _("Duplicate variable name %s in SAVE statement."), name);
- ok = false;
- }
- }
-
- if (!stringi_set_is_empty (&strings))
- {
- size_t count = stringi_set_count (&strings);
- const char *example = stringi_set_node_get_string (
- stringi_set_first (&strings));
-
- if (count == 1)
- msg_at (ME, save_location,
- _("The SAVE command STRINGS subcommand specifies an unknown "
- "variable %s."), example);
- else
- msg_at (ME, save_location,
- ngettext ("The SAVE command STRINGS subcommand specifies %zu "
- "unknown variable: %s.",
- "The SAVE command STRINGS subcommand specifies %zu "
- "unknown variables, including %s.",
- count),
- count, example);
- ok = false;
- }
- stringi_set_destroy (&strings);
- string_array_destroy (&names);
-
- if (!ok)
- {
- dict_unref (dict);
- sf->error = true;
- return NULL;
- }
-
- if (sf->file == fh_inline_file ())
- sf->writer = autopaging_writer_create (dict_get_proto (dict));
- else
- sf->writer = any_writer_open (sf->file, dict);
- if (sf->writer)
- {
- sf->dict = dict;
- sf->location = msg_location_dup (save_location);
- }
- else
- {
- dict_unref (dict);
- sf->error = true;
- }
- return sf->writer;
-}
-
-static void
-save_file_destroy (struct save_file *sf)
-{
- if (sf)
- {
- if (sf->file == fh_inline_file () && sf->writer && sf->dict)
- {
- dataset_set_dict (sf->dataset, sf->dict);
- dataset_set_source (sf->dataset, casewriter_make_reader (sf->writer));
- }
- else
- {
- casewriter_destroy (sf->writer);
- dict_unref (sf->dict);
- }
- fh_unref (sf->file);
- string_array_destroy (&sf->variables);
- matrix_expr_destroy (sf->names);
- stringi_set_destroy (&sf->strings);
- msg_location_destroy (sf->location);
- free (sf);
- }
-}
-
-static struct matrix_command *
-matrix_save_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_SAVE,
- .save = { .expression = NULL },
- };
-
- struct file_handle *fh = NULL;
- struct matrix_save *save = &cmd->save;
-
- struct string_array variables = STRING_ARRAY_INITIALIZER;
- struct matrix_expr *names = NULL;
- struct stringi_set strings = STRINGI_SET_INITIALIZER (strings);
-
- save->expression = matrix_parse_exp (s);
- if (!save->expression)
- goto error;
-
- int names_start = 0;
- int names_end = 0;
- while (lex_match (s->lexer, T_SLASH))
- {
- if (lex_match_id (s->lexer, "OUTFILE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- fh_unref (fh);
- fh = (lex_match (s->lexer, T_ASTERISK)
- ? fh_inline_file ()
- : fh_parse (s->lexer, FH_REF_FILE, s->session));
- if (!fh)
- goto error;
- }
- else if (lex_match_id (s->lexer, "VARIABLES"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- char **names;
- size_t n;
- struct dictionary *d = dict_create (get_default_encoding ());
- bool ok = parse_DATA_LIST_vars (s->lexer, d, &names, &n,
- PV_NO_SCRATCH | PV_NO_DUPLICATE);
- dict_unref (d);
- if (!ok)
- goto error;
-
- string_array_clear (&variables);
- variables = (struct string_array) {
- .strings = names,
- .n = n,
- .allocated = n,
- };
- }
- else if (lex_match_id (s->lexer, "NAMES"))
- {
- lex_match (s->lexer, T_EQUALS);
- matrix_expr_destroy (names);
- names_start = lex_ofs (s->lexer);
- names = matrix_parse_exp (s);
- names_end = lex_ofs (s->lexer) - 1;
- if (!names)
- goto error;
- }
- else if (lex_match_id (s->lexer, "STRINGS"))
- {
- lex_match (s->lexer, T_EQUALS);
- while (lex_token (s->lexer) == T_ID)
- {
- stringi_set_insert (&strings, lex_tokcstr (s->lexer));
- lex_get (s->lexer);
- if (!lex_match (s->lexer, T_COMMA))
- break;
- }
- }
- else
- {
- lex_error_expecting (s->lexer, "OUTFILE", "VARIABLES", "NAMES",
- "STRINGS");
- goto error;
- }
- }
-
- if (!fh)
- {
- if (s->prev_save_file)
- fh = fh_ref (s->prev_save_file);
- else
- {
- lex_sbc_missing (s->lexer, "OUTFILE");
- goto error;
- }
- }
- fh_unref (s->prev_save_file);
- s->prev_save_file = fh_ref (fh);
-
- if (variables.n && names)
- {
- lex_ofs_msg (s->lexer, SW, names_start, names_end,
- _("Ignoring NAMES because VARIABLES was also specified."));
- matrix_expr_destroy (names);
- names = NULL;
- }
-
- save->sf = save_file_create (s, fh, &variables, names, &strings);
- return cmd;
-
-error:
- string_array_destroy (&variables);
- matrix_expr_destroy (names);
- stringi_set_destroy (&strings);
- fh_unref (fh);
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_save_execute (const struct matrix_command *cmd)
-{
- const struct matrix_save *save = &cmd->save;
-
- gsl_matrix *m = matrix_expr_evaluate (save->expression);
- if (!m)
- return;
-
- struct casewriter *writer = save_file_open (save->sf, m, cmd->location);
- if (!writer)
- {
- gsl_matrix_free (m);
- return;
- }
-
- const struct caseproto *proto = casewriter_get_proto (writer);
- for (size_t y = 0; y < m->size1; y++)
- {
- struct ccase *c = case_create (proto);
- for (size_t x = 0; x < m->size2; x++)
- {
- double d = gsl_matrix_get (m, y, x);
- int width = caseproto_get_width (proto, x);
- union value *value = case_data_rw_idx (c, x);
- if (width == 0)
- value->f = d;
- else
- memcpy (value->s, &d, width);
- }
- casewriter_write (writer, c);
- }
- gsl_matrix_free (m);
-}
-\f
-/* READ. */
-
-static struct read_file *
-read_file_create (struct matrix_state *s, struct file_handle *fh)
-{
- for (size_t i = 0; i < s->n_read_files; i++)
- {
- struct read_file *rf = s->read_files[i];
- if (rf->file == fh)
- {
- fh_unref (fh);
- return rf;
- }
- }
-
- struct read_file *rf = xmalloc (sizeof *rf);
- *rf = (struct read_file) { .file = fh };
-
- s->read_files = xrealloc (s->read_files,
- (s->n_read_files + 1) * sizeof *s->read_files);
- s->read_files[s->n_read_files++] = rf;
- return rf;
-}
-
-static struct dfm_reader *
-read_file_open (struct read_file *rf)
-{
- if (!rf->reader)
- rf->reader = dfm_open_reader (rf->file, NULL, rf->encoding);
- return rf->reader;
-}
-
-static void
-read_file_destroy (struct read_file *rf)
-{
- if (rf)
- {
- fh_unref (rf->file);
- dfm_close_reader (rf->reader);
- free (rf->encoding);
- free (rf);
- }
-}
-
-static struct matrix_command *
-matrix_read_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_READ,
- .read = { .format = FMT_F },
- };
-
- struct file_handle *fh = NULL;
- char *encoding = NULL;
- struct matrix_read *read = &cmd->read;
- read->dst = matrix_lvalue_parse (s);
- if (!read->dst)
- goto error;
-
- int by_ofs = 0;
- int format_ofs = 0;
- int record_width_start = 0, record_width_end = 0;
-
- int by = 0;
- int repetitions = 0;
- int record_width = 0;
- bool seen_format = false;
- while (lex_match (s->lexer, T_SLASH))
- {
- if (lex_match_id (s->lexer, "FILE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- fh_unref (fh);
- fh = fh_parse (s->lexer, FH_REF_FILE, NULL);
- if (!fh)
- goto error;
- }
- else if (lex_match_id (s->lexer, "ENCODING"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!lex_force_string (s->lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (s->lexer));
-
- lex_get (s->lexer);
- }
- else if (lex_match_id (s->lexer, "FIELD"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- record_width_start = lex_ofs (s->lexer);
- if (!lex_force_int_range (s->lexer, "FIELD", 1, INT_MAX))
- goto error;
- read->c1 = lex_integer (s->lexer);
- lex_get (s->lexer);
- if (!lex_force_match (s->lexer, T_TO)
- || !lex_force_int_range (s->lexer, "TO", read->c1, INT_MAX))
- goto error;
- read->c2 = lex_integer (s->lexer) + 1;
- record_width_end = lex_ofs (s->lexer);
- lex_get (s->lexer);
-
- record_width = read->c2 - read->c1;
- if (lex_match (s->lexer, T_BY))
- {
- if (!lex_force_int_range (s->lexer, "BY", 1,
- read->c2 - read->c1))
- goto error;
- by = lex_integer (s->lexer);
- by_ofs = lex_ofs (s->lexer);
- int field_end = lex_ofs (s->lexer);
- lex_get (s->lexer);
-
- if (record_width % by)
- {
- lex_ofs_error (
- s->lexer, record_width_start, field_end,
- _("Field width %d does not evenly divide record width %d."),
- by, record_width);
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates the record width."));
- lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
- _("This syntax specifies the field width."));
- goto error;
- }
- }
- else
- by = 0;
- }
- else if (lex_match_id (s->lexer, "SIZE"))
- {
- lex_match (s->lexer, T_EQUALS);
- matrix_expr_destroy (read->size);
- read->size = matrix_parse_exp (s);
- if (!read->size)
- goto error;
- }
- else if (lex_match_id (s->lexer, "MODE"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (lex_match_id (s->lexer, "RECTANGULAR"))
- read->symmetric = false;
- else if (lex_match_id (s->lexer, "SYMMETRIC"))
- read->symmetric = true;
- else
- {
- lex_error_expecting (s->lexer, "RECTANGULAR", "SYMMETRIC");
- goto error;
- }
- }
- else if (lex_match_id (s->lexer, "REREAD"))
- read->reread = true;
- else if (lex_match_id (s->lexer, "FORMAT"))
- {
- if (seen_format)
- {
- lex_sbc_only_once (s->lexer, "FORMAT");
- goto error;
- }
- seen_format = true;
-
- lex_match (s->lexer, T_EQUALS);
-
- if (lex_token (s->lexer) != T_STRING && !lex_force_id (s->lexer))
- goto error;
-
- format_ofs = lex_ofs (s->lexer);
- const char *p = lex_tokcstr (s->lexer);
- if (c_isdigit (p[0]))
- {
- repetitions = atoi (p);
- p += strspn (p, "0123456789");
- if (!fmt_from_name (p, &read->format))
- {
- lex_error (s->lexer, _("Unknown format %s."), p);
- goto error;
- }
- lex_get (s->lexer);
- }
- else if (fmt_from_name (p, &read->format))
- lex_get (s->lexer);
- else
- {
- struct fmt_spec format;
- if (!parse_format_specifier (s->lexer, &format))
- goto error;
- read->format = format.type;
- read->w = format.w;
- }
- }
- else
- {
- lex_error_expecting (s->lexer, "FILE", "FIELD", "MODE",
- "REREAD", "FORMAT");
- goto error;
- }
- }
-
- if (!read->c1)
- {
- lex_sbc_missing (s->lexer, "FIELD");
- goto error;
- }
-
- if (!read->dst->n_indexes && !read->size)
- {
- msg (SE, _("SIZE is required for reading data into a full matrix "
- "(as opposed to a submatrix)."));
- msg_at (SN, read->dst->var_location,
- _("This expression designates a full matrix."));
- goto error;
- }
-
- if (!fh)
- {
- if (s->prev_read_file)
- fh = fh_ref (s->prev_read_file);
- else
- {
- lex_sbc_missing (s->lexer, "FILE");
- goto error;
- }
- }
- fh_unref (s->prev_read_file);
- s->prev_read_file = fh_ref (fh);
-
- read->rf = read_file_create (s, fh);
- fh = NULL;
- if (encoding)
- {
- free (read->rf->encoding);
- read->rf->encoding = encoding;
- encoding = NULL;
- }
-
- /* Field width may be specified in multiple ways:
-
- 1. BY on FIELD.
- 2. The format on FORMAT.
- 3. The repetition factor on FORMAT.
-
- (2) and (3) are mutually exclusive.
-
- If more than one of these is present, they must agree. If none of them is
- present, then free-field format is used.
- */
- if (repetitions > record_width)
- {
- msg (SE, _("%d repetitions cannot fit in record width %d."),
- repetitions, record_width);
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- _("This syntax designates the number of repetitions."));
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates the record width."));
- goto error;
- }
- int w = (repetitions ? record_width / repetitions
- : read->w ? read->w
- : by);
- if (by && w != by)
- {
- msg (SE, _("This command specifies two different field widths."));
- if (repetitions)
- {
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- ngettext ("This syntax specifies %d repetition.",
- "This syntax specifies %d repetitions.",
- repetitions),
- repetitions);
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates record width %d, "
- "which divided by %d repetitions implies "
- "field width %d."),
- record_width, repetitions, w);
- }
- else
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- _("This syntax specifies field width %d."), w);
-
- lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
- _("This syntax specifies field width %d."), by);
- goto error;
- }
- read->w = w;
- return cmd;
-
-error:
- fh_unref (fh);
- matrix_command_destroy (cmd);
- free (encoding);
- return NULL;
-}
-
-static void
-parse_error (const struct dfm_reader *reader, enum fmt_type format,
- struct substring data, size_t y, size_t x,
- int first_column, int last_column, char *error)
-{
- int line_number = dfm_get_line_number (reader);
- struct msg_location location = {
- .file_name = intern_new (dfm_get_file_name (reader)),
- .start = { .line = line_number, .column = first_column },
- .end = { .line = line_number, .column = last_column },
- };
- msg_at (DW, &location, _("Error reading \"%.*s\" as format %s "
- "for matrix row %zu, column %zu: %s"),
- (int) data.length, data.string, fmt_name (format),
- y + 1, x + 1, error);
- msg_location_uninit (&location);
- free (error);
-}
-
-static void
-matrix_read_set_field (struct matrix_read *read, struct dfm_reader *reader,
- gsl_matrix *m, struct substring p, size_t y, size_t x,
- const char *line_start)
-{
- const char *input_encoding = dfm_reader_get_encoding (reader);
- char *error;
- double f;
- if (fmt_is_numeric (read->format))
- {
- union value v;
- error = data_in (p, input_encoding, read->format,
- settings_get_fmt_settings (), &v, 0, NULL);
- if (!error && v.f == SYSMIS)
- error = xstrdup (_("Matrix data may not contain missing value."));
- f = v.f;
- }
- else
- {
- uint8_t s[sizeof (double)];
- union value v = { .s = s };
- error = data_in (p, input_encoding, read->format,
- settings_get_fmt_settings (), &v, sizeof s, "UTF-8");
- memcpy (&f, s, sizeof f);
- }
-
- if (error)
- {
- int c1 = utf8_count_columns (line_start, p.string - line_start) + 1;
- int nc = ss_utf8_count_columns (p);
- int c2 = c1 + MAX (1, nc) - 1;
- parse_error (reader, read->format, p, y, x, c1, c2, error);
- }
- else
- {
- gsl_matrix_set (m, y, x, f);
- if (read->symmetric && x != y)
- gsl_matrix_set (m, x, y, f);
- }
-}
-
-static bool
-matrix_read_line (struct matrix_command *cmd, struct dfm_reader *reader,
- struct substring *line, const char **startp)
-{
- struct matrix_read *read = &cmd->read;
- if (dfm_eof (reader))
- {
- msg_at (SE, cmd->location,
- _("Unexpected end of file reading matrix data."));
- return false;
- }
- dfm_expand_tabs (reader);
- struct substring record = dfm_get_record (reader);
- /* XXX need to recode record into UTF-8 */
- *startp = record.string;
- *line = ss_utf8_columns (record, read->c1 - 1, read->c2 - read->c1);
- return true;
-}
-
-static void
-matrix_read (struct matrix_command *cmd, struct dfm_reader *reader,
- gsl_matrix *m)
-{
- struct matrix_read *read = &cmd->read;
- for (size_t y = 0; y < m->size1; y++)
- {
- size_t nx = read->symmetric ? y + 1 : m->size2;
-
- struct substring line = ss_empty ();
- const char *line_start = line.string;
- for (size_t x = 0; x < nx; x++)
- {
- struct substring p;
- if (!read->w)
- {
- for (;;)
- {
- ss_ltrim (&line, ss_cstr (" ,"));
- if (!ss_is_empty (line))
- break;
- if (!matrix_read_line (cmd, reader, &line, &line_start))
- return;
- dfm_forward_record (reader);
- }
-
- ss_get_bytes (&line, ss_cspan (line, ss_cstr (" ,")), &p);
- }
- else
- {
- if (!matrix_read_line (cmd, reader, &line, &line_start))
- return;
- size_t fields_per_line = (read->c2 - read->c1) / read->w;
- int f = x % fields_per_line;
- if (f == fields_per_line - 1)
- dfm_forward_record (reader);
-
- p = ss_substr (line, read->w * f, read->w);
- }
-
- matrix_read_set_field (read, reader, m, p, y, x, line_start);
- }
-
- if (read->w)
- dfm_forward_record (reader);
- else
- {
- ss_ltrim (&line, ss_cstr (" ,"));
- if (!ss_is_empty (line))
- {
- int line_number = dfm_get_line_number (reader);
- int c1 = utf8_count_columns (line_start,
- line.string - line_start) + 1;
- int c2 = c1 + ss_utf8_count_columns (line) - 1;
- struct msg_location location = {
- .file_name = intern_new (dfm_get_file_name (reader)),
- .start = { .line = line_number, .column = c1 },
- .end = { .line = line_number, .column = c2 },
- };
- msg_at (DW, &location,
- _("Trailing garbage following data for matrix row %zu."),
- y + 1);
- msg_location_uninit (&location);
- }
- }
- }
-}
-
-static void
-matrix_read_execute (struct matrix_command *cmd)
-{
- struct matrix_read *read = &cmd->read;
- struct index_vector iv0, iv1;
- if (!matrix_lvalue_evaluate (read->dst, &iv0, &iv1))
- return;
-
- size_t size[2] = { SIZE_MAX, SIZE_MAX };
- if (read->size)
- {
- gsl_matrix *m = matrix_expr_evaluate (read->size);
- if (!m)
- return;
-
- if (!is_vector (m))
- {
- msg_at (SE, matrix_expr_location (read->size),
- _("SIZE must evaluate to a scalar or a 2-element vector, "
- "not a %zu×%zu matrix."), m->size1, m->size2);
- gsl_matrix_free (m);
- index_vector_uninit (&iv0);
- index_vector_uninit (&iv1);
- return;
- }
-
- gsl_vector v = to_vector (m);
- double d[2];
- if (v.size == 1)
- {
- d[0] = gsl_vector_get (&v, 0);
- d[1] = 1;
- }
- else if (v.size == 2)
- {
- d[0] = gsl_vector_get (&v, 0);
- d[1] = gsl_vector_get (&v, 1);
- }
- else
- {
- msg_at (SE, matrix_expr_location (read->size),
- _("SIZE must evaluate to a scalar or a 2-element vector, "
- "not a %zu×%zu matrix."),
- m->size1, m->size2),
- gsl_matrix_free (m);
- index_vector_uninit (&iv0);
- index_vector_uninit (&iv1);
- return;
- }
- gsl_matrix_free (m);
-
- if (d[0] < 0 || d[0] > SIZE_MAX || d[1] < 0 || d[1] > SIZE_MAX)
- {
- msg_at (SE, matrix_expr_location (read->size),
- _("Matrix dimensions %g×%g specified on SIZE "
- "are outside valid range."),
- d[0], d[1]);
- index_vector_uninit (&iv0);
- index_vector_uninit (&iv1);
- return;
- }
-
- size[0] = d[0];
- size[1] = d[1];
- }
-
- if (read->dst->n_indexes)
- {
- size_t submatrix_size[2];
- if (read->dst->n_indexes == 2)
- {
- submatrix_size[0] = iv0.n;
- submatrix_size[1] = iv1.n;
- }
- else if (read->dst->var->value->size1 == 1)
- {
- submatrix_size[0] = 1;
- submatrix_size[1] = iv0.n;
- }
- else
- {
- submatrix_size[0] = iv0.n;
- submatrix_size[1] = 1;
- }
-
- if (read->size)
- {
- if (size[0] != submatrix_size[0] || size[1] != submatrix_size[1])
- {
- msg_at (SE, cmd->location,
- _("Dimensions specified on SIZE differ from dimensions "
- "of destination submatrix."));
- msg_at (SN, matrix_expr_location (read->size),
- _("SIZE specifies dimensions %zu×%zu."),
- size[0], size[1]);
- msg_at (SN, read->dst->full_location,
- _("Destination submatrix has dimensions %zu×%zu."),
- submatrix_size[0], submatrix_size[1]);
- index_vector_uninit (&iv0);
- index_vector_uninit (&iv1);
- return;
- }
- }
- else
- {
- size[0] = submatrix_size[0];
- size[1] = submatrix_size[1];
- }
- }
-
- struct dfm_reader *reader = read_file_open (read->rf);
- if (read->reread)
- dfm_reread_record (reader, 1);
-
- if (read->symmetric && size[0] != size[1])
- {
- msg_at (SE, cmd->location,
- _("Cannot read non-square %zu×%zu matrix "
- "using READ with MODE=SYMMETRIC."),
- size[0], size[1]);
- index_vector_uninit (&iv0);
- index_vector_uninit (&iv1);
- return;
- }
- gsl_matrix *tmp = gsl_matrix_calloc (size[0], size[1]);
- matrix_read (cmd, reader, tmp);
- matrix_lvalue_assign (read->dst, &iv0, &iv1, tmp, cmd->location);
-}
-\f
-/* WRITE. */
-
-static struct write_file *
-write_file_create (struct matrix_state *s, struct file_handle *fh)
-{
- for (size_t i = 0; i < s->n_write_files; i++)
- {
- struct write_file *wf = s->write_files[i];
- if (wf->file == fh)
- {
- fh_unref (fh);
- return wf;
- }
- }
-
- struct write_file *wf = xmalloc (sizeof *wf);
- *wf = (struct write_file) { .file = fh };
-
- s->write_files = xrealloc (s->write_files,
- (s->n_write_files + 1) * sizeof *s->write_files);
- s->write_files[s->n_write_files++] = wf;
- return wf;
-}
-
-static struct dfm_writer *
-write_file_open (struct write_file *wf)
-{
- if (!wf->writer)
- wf->writer = dfm_open_writer (wf->file, wf->encoding);
- return wf->writer;
-}
-
-static void
-write_file_destroy (struct write_file *wf)
-{
- if (wf)
- {
- if (wf->held)
- {
- dfm_put_record_utf8 (wf->writer, wf->held->s.ss.string,
- wf->held->s.ss.length);
- u8_line_destroy (wf->held);
- free (wf->held);
- }
-
- fh_unref (wf->file);
- dfm_close_writer (wf->writer);
- free (wf->encoding);
- free (wf);
- }
-}
-
-static struct matrix_command *
-matrix_write_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_WRITE,
- };
-
- struct file_handle *fh = NULL;
- char *encoding = NULL;
- struct matrix_write *write = &cmd->write;
- write->expression = matrix_parse_exp (s);
- if (!write->expression)
- goto error;
-
- int by_ofs = 0;
- int format_ofs = 0;
- int record_width_start = 0, record_width_end = 0;
-
- int by = 0;
- int repetitions = 0;
- int record_width = 0;
- enum fmt_type format = FMT_F;
- bool has_format = false;
- while (lex_match (s->lexer, T_SLASH))
- {
- if (lex_match_id (s->lexer, "OUTFILE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- fh_unref (fh);
- fh = fh_parse (s->lexer, FH_REF_FILE, NULL);
- if (!fh)
- goto error;
- }
- else if (lex_match_id (s->lexer, "ENCODING"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!lex_force_string (s->lexer))
- goto error;
-
- free (encoding);
- encoding = ss_xstrdup (lex_tokss (s->lexer));
-
- lex_get (s->lexer);
- }
- else if (lex_match_id (s->lexer, "FIELD"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- record_width_start = lex_ofs (s->lexer);
-
- if (!lex_force_int_range (s->lexer, "FIELD", 1, INT_MAX))
- goto error;
- write->c1 = lex_integer (s->lexer);
- lex_get (s->lexer);
- if (!lex_force_match (s->lexer, T_TO)
- || !lex_force_int_range (s->lexer, "TO", write->c1, INT_MAX))
- goto error;
- write->c2 = lex_integer (s->lexer) + 1;
- record_width_end = lex_ofs (s->lexer);
- lex_get (s->lexer);
-
- record_width = write->c2 - write->c1;
- if (lex_match (s->lexer, T_BY))
- {
- if (!lex_force_int_range (s->lexer, "BY", 1,
- write->c2 - write->c1))
- goto error;
- by_ofs = lex_ofs (s->lexer);
- int field_end = lex_ofs (s->lexer);
- by = lex_integer (s->lexer);
- lex_get (s->lexer);
-
- if (record_width % by)
- {
- lex_ofs_error (
- s->lexer, record_width_start, field_end,
- _("Field width %d does not evenly divide record width %d."),
- by, record_width);
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates the record width."));
- lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
- _("This syntax specifies the field width."));
- goto error;
- }
- }
- else
- by = 0;
- }
- else if (lex_match_id (s->lexer, "MODE"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (lex_match_id (s->lexer, "RECTANGULAR"))
- write->triangular = false;
- else if (lex_match_id (s->lexer, "TRIANGULAR"))
- write->triangular = true;
- else
- {
- lex_error_expecting (s->lexer, "RECTANGULAR", "TRIANGULAR");
- goto error;
- }
- }
- else if (lex_match_id (s->lexer, "HOLD"))
- write->hold = true;
- else if (lex_match_id (s->lexer, "FORMAT"))
- {
- if (has_format || write->format)
- {
- lex_sbc_only_once (s->lexer, "FORMAT");
- goto error;
- }
-
- lex_match (s->lexer, T_EQUALS);
-
- if (lex_token (s->lexer) != T_STRING && !lex_force_id (s->lexer))
- goto error;
-
- format_ofs = lex_ofs (s->lexer);
- const char *p = lex_tokcstr (s->lexer);
- if (c_isdigit (p[0]))
- {
- repetitions = atoi (p);
- p += strspn (p, "0123456789");
- if (!fmt_from_name (p, &format))
- {
- lex_error (s->lexer, _("Unknown format %s."), p);
- goto error;
- }
- has_format = true;
- lex_get (s->lexer);
- }
- else if (fmt_from_name (p, &format))
- {
- has_format = true;
- lex_get (s->lexer);
- }
- else
- {
- struct fmt_spec spec;
- if (!parse_format_specifier (s->lexer, &spec))
- goto error;
- write->format = xmemdup (&spec, sizeof spec);
- }
- }
- else
- {
- lex_error_expecting (s->lexer, "OUTFILE", "FIELD", "MODE",
- "HOLD", "FORMAT");
- goto error;
- }
- }
-
- if (!write->c1)
- {
- lex_sbc_missing (s->lexer, "FIELD");
- goto error;
- }
-
- if (!fh)
- {
- if (s->prev_write_file)
- fh = fh_ref (s->prev_write_file);
- else
- {
- lex_sbc_missing (s->lexer, "OUTFILE");
- goto error;
- }
- }
- fh_unref (s->prev_write_file);
- s->prev_write_file = fh_ref (fh);
-
- write->wf = write_file_create (s, fh);
- fh = NULL;
- if (encoding)
- {
- free (write->wf->encoding);
- write->wf->encoding = encoding;
- encoding = NULL;
- }
-
- /* Field width may be specified in multiple ways:
-
- 1. BY on FIELD.
- 2. The format on FORMAT.
- 3. The repetition factor on FORMAT.
-
- (2) and (3) are mutually exclusive.
-
- If more than one of these is present, they must agree. If none of them is
- present, then free-field format is used.
- */
- if (repetitions > record_width)
- {
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- _("This syntax designates the number of repetitions."));
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates the record width."));
- goto error;
- }
- int w = (repetitions ? record_width / repetitions
- : write->format ? write->format->w
- : by);
- if (by && w != by)
- {
- msg (SE, _("This command specifies two different field widths."));
- if (repetitions)
- {
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- ngettext ("This syntax specifies %d repetition.",
- "This syntax specifies %d repetitions.",
- repetitions),
- repetitions);
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates record width %d, "
- "which divided by %d repetitions implies "
- "field width %d."),
- record_width, repetitions, w);
- }
- else
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- _("This syntax specifies field width %d."), w);
-
- lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
- _("This syntax specifies field width %d."), by);
- goto error;
- }
- if (w && !write->format)
- {
- write->format = xmalloc (sizeof *write->format);
- *write->format = (struct fmt_spec) { .type = format, .w = w };
-
- char *error = fmt_check_output__ (write->format);
- if (error)
- {
- msg (SE, "%s", error);
- free (error);
-
- if (has_format)
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- _("This syntax specifies format %s."),
- fmt_name (format));
-
- if (repetitions)
- {
- lex_ofs_msg (s->lexer, SN, format_ofs, format_ofs,
- ngettext ("This syntax specifies %d repetition.",
- "This syntax specifies %d repetitions.",
- repetitions),
- repetitions);
- lex_ofs_msg (s->lexer, SN, record_width_start, record_width_end,
- _("This syntax designates record width %d, "
- "which divided by %d repetitions implies "
- "field width %d."),
- record_width, repetitions, w);
- }
-
- if (by)
- lex_ofs_msg (s->lexer, SN, by_ofs, by_ofs,
- _("This syntax specifies field width %d."), by);
-
- goto error;
- }
- }
-
- if (write->format && fmt_var_width (write->format) > sizeof (double))
- {
- char fs[FMT_STRING_LEN_MAX + 1];
- fmt_to_string (write->format, fs);
- lex_ofs_error (s->lexer, format_ofs, format_ofs,
- _("Format %s is too wide for %zu-byte matrix elements."),
- fs, sizeof (double));
- goto error;
- }
-
- return cmd;
-
-error:
- fh_unref (fh);
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_write_execute (struct matrix_write *write)
-{
- gsl_matrix *m = matrix_expr_evaluate (write->expression);
- if (!m)
- return;
-
- if (write->triangular && m->size1 != m->size2)
- {
- msg_at (SE, matrix_expr_location (write->expression),
- _("WRITE with MODE=TRIANGULAR requires a square matrix but "
- "the matrix to be written has dimensions %zu×%zu."),
- m->size1, m->size2);
- gsl_matrix_free (m);
- return;
- }
-
- struct dfm_writer *writer = write_file_open (write->wf);
- if (!writer || !m->size1)
- {
- gsl_matrix_free (m);
- return;
- }
-
- const struct fmt_settings *settings = settings_get_fmt_settings ();
- struct u8_line *line = write->wf->held;
- for (size_t y = 0; y < m->size1; y++)
- {
- if (!line)
- {
- line = xmalloc (sizeof *line);
- u8_line_init (line);
- }
- size_t nx = write->triangular ? y + 1 : m->size2;
- int x0 = write->c1;
- for (size_t x = 0; x < nx; x++)
- {
- char *s;
- double f = gsl_matrix_get (m, y, x);
- if (write->format)
- {
- union value v;
- if (fmt_is_numeric (write->format->type))
- v.f = f;
- else
- v.s = (uint8_t *) &f;
- s = data_out (&v, NULL, write->format, settings);
- }
- else
- {
- s = xmalloc (DBL_BUFSIZE_BOUND);
- if (c_dtoastr (s, DBL_BUFSIZE_BOUND, FTOASTR_UPPER_E, 0, f)
- >= DBL_BUFSIZE_BOUND)
- abort ();
- }
- size_t len = strlen (s);
- int width = u8_width (CHAR_CAST (const uint8_t *, s), len, UTF8);
- if (width + x0 > write->c2)
- {
- dfm_put_record_utf8 (writer, line->s.ss.string,
- line->s.ss.length);
- u8_line_clear (line);
- x0 = write->c1;
- }
- u8_line_put (line, x0, x0 + width, s, len);
- free (s);
-
- x0 += write->format ? write->format->w : width + 1;
- }
-
- if (y + 1 >= m->size1 && write->hold)
- break;
- dfm_put_record_utf8 (writer, line->s.ss.string, line->s.ss.length);
- u8_line_clear (line);
- }
- if (!write->hold)
- {
- u8_line_destroy (line);
- free (line);
- line = NULL;
- }
- write->wf->held = line;
-
- gsl_matrix_free (m);
-}
-\f
-/* GET. */
-
-static struct matrix_command *
-matrix_get_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_GET,
- .get = {
- .lexer = s->lexer,
- .dataset = s->dataset,
- .user = { .treatment = MGET_ERROR },
- .system = { .treatment = MGET_ERROR },
- }
- };
-
- struct matrix_get *get = &cmd->get;
- get->dst = matrix_lvalue_parse (s);
- if (!get->dst)
- goto error;
-
- while (lex_match (s->lexer, T_SLASH))
- {
- if (lex_match_id (s->lexer, "FILE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- fh_unref (get->file);
- if (lex_match (s->lexer, T_ASTERISK))
- get->file = NULL;
- else
- {
- get->file = fh_parse (s->lexer, FH_REF_FILE, s->session);
- if (!get->file)
- goto error;
- }
- }
- else if (lex_match_id (s->lexer, "ENCODING"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!lex_force_string (s->lexer))
- goto error;
-
- free (get->encoding);
- get->encoding = ss_xstrdup (lex_tokss (s->lexer));
-
- lex_get (s->lexer);
- }
- else if (lex_match_id (s->lexer, "VARIABLES"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- if (get->n_vars)
- {
- lex_sbc_only_once (s->lexer, "VARIABLES");
- goto error;
- }
-
- if (!var_syntax_parse (s->lexer, &get->vars, &get->n_vars))
- goto error;
- }
- else if (lex_match_id (s->lexer, "NAMES"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!lex_force_id (s->lexer))
- goto error;
-
- struct substring name = lex_tokss (s->lexer);
- get->names = matrix_var_lookup (s, name);
- if (!get->names)
- get->names = matrix_var_create (s, name);
- lex_get (s->lexer);
- }
- else if (lex_match_id (s->lexer, "MISSING"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (lex_match_id (s->lexer, "ACCEPT"))
- get->user.treatment = MGET_ACCEPT;
- else if (lex_match_id (s->lexer, "OMIT"))
- get->user.treatment = MGET_OMIT;
- else if (lex_is_number (s->lexer))
- {
- get->user.treatment = MGET_RECODE;
- get->user.substitute = lex_number (s->lexer);
- lex_get (s->lexer);
- }
- else
- {
- lex_error (s->lexer, _("Syntax error expecting ACCEPT or OMIT or "
- "a number for MISSING."));
- goto error;
- }
- }
- else if (lex_match_id (s->lexer, "SYSMIS"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (lex_match_id (s->lexer, "OMIT"))
- get->system.treatment = MGET_OMIT;
- else if (lex_is_number (s->lexer))
- {
- get->system.treatment = MGET_RECODE;
- get->system.substitute = lex_number (s->lexer);
- lex_get (s->lexer);
- }
- else
- {
- lex_error (s->lexer, _("Syntax error expecting OMIT or a number "
- "for SYSMIS."));
- goto error;
- }
- }
- else
- {
- lex_error_expecting (s->lexer, "FILE", "VARIABLES", "NAMES",
- "MISSING", "SYSMIS");
- goto error;
- }
- }
-
- if (get->user.treatment != MGET_ACCEPT)
- get->system.treatment = MGET_ERROR;
-
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_get_execute__ (struct matrix_command *cmd, struct casereader *reader,
- const struct dictionary *dict)
-{
- struct matrix_get *get = &cmd->get;
- struct variable **vars;
- size_t n_vars = 0;
-
- if (get->n_vars)
- {
- if (!var_syntax_evaluate (get->lexer, get->vars, get->n_vars, dict,
- &vars, &n_vars, PV_NUMERIC))
- return;
- }
- else
- {
- n_vars = dict_get_n_vars (dict);
- vars = xnmalloc (n_vars, sizeof *vars);
- for (size_t i = 0; i < n_vars; i++)
- {
- struct variable *var = dict_get_var (dict, i);
- if (!var_is_numeric (var))
- {
- msg_at (SE, cmd->location, _("Variable %s is not numeric."),
- var_get_name (var));
- free (vars);
- return;
- }
- vars[i] = var;
- }
- }
-
- if (get->names)
- {
- gsl_matrix *names = gsl_matrix_alloc (n_vars, 1);
- for (size_t i = 0; i < n_vars; i++)
- {
- char s[sizeof (double)];
- double f;
- buf_copy_str_rpad (s, sizeof s, var_get_name (vars[i]), ' ');
- memcpy (&f, s, sizeof f);
- gsl_matrix_set (names, i, 0, f);
- }
-
- gsl_matrix_free (get->names->value);
- get->names->value = names;
- }
-
- size_t n_rows = 0;
- gsl_matrix *m = gsl_matrix_alloc (4, n_vars);
- long long int casenum = 1;
- bool error = false;
- for (struct ccase *c = casereader_read (reader); c;
- c = casereader_read (reader), casenum++)
- {
- if (n_rows >= m->size1)
- {
- gsl_matrix *p = gsl_matrix_alloc (m->size1 * 2, n_vars);
- for (size_t y = 0; y < n_rows; y++)
- for (size_t x = 0; x < n_vars; x++)
- gsl_matrix_set (p, y, x, gsl_matrix_get (m, y, x));
- gsl_matrix_free (m);
- m = p;
- }
-
- bool keep = true;
- for (size_t x = 0; x < n_vars; x++)
- {
- const struct variable *var = vars[x];
- double d = case_num (c, var);
- if (d == SYSMIS)
- {
- if (get->system.treatment == MGET_RECODE)
- d = get->system.substitute;
- else if (get->system.treatment == MGET_OMIT)
- keep = false;
- else
- {
- msg_at (SE, cmd->location, _("Variable %s in case %lld "
- "is system-missing."),
- var_get_name (var), casenum);
- error = true;
- }
- }
- else if (var_is_num_missing (var, d) == MV_USER)
- {
- if (get->user.treatment == MGET_RECODE)
- d = get->user.substitute;
- else if (get->user.treatment == MGET_OMIT)
- keep = false;
- else if (get->user.treatment != MGET_ACCEPT)
- {
- msg_at (SE, cmd->location,
- _("Variable %s in case %lld has user-missing "
- "value %g."),
- var_get_name (var), casenum, d);
- error = true;
- }
- }
- gsl_matrix_set (m, n_rows, x, d);
- }
- case_unref (c);
- if (error)
- break;
- if (keep)
- n_rows++;
- }
- if (!error)
- {
- m->size1 = n_rows;
- matrix_lvalue_evaluate_and_assign (get->dst, m, cmd->location);
- }
- else
- gsl_matrix_free (m);
- free (vars);
-}
-
-static bool
-matrix_open_casereader (const struct matrix_command *cmd,
- const char *command_name, struct file_handle *file,
- const char *encoding, struct dataset *dataset,
- struct casereader **readerp, struct dictionary **dictp)
-{
- if (file)
- {
- *readerp = any_reader_open_and_decode (file, encoding, dictp, NULL);
- return *readerp != NULL;
- }
- else
- {
- if (dict_get_n_vars (dataset_dict (dataset)) == 0)
- {
- msg_at (SE, cmd->location,
- _("The %s command cannot read an empty active file."),
- command_name);
- return false;
- }
- *readerp = proc_open (dataset);
- *dictp = dict_ref (dataset_dict (dataset));
- return true;
- }
-}
-
-static void
-matrix_close_casereader (struct file_handle *file, struct dataset *dataset,
- struct casereader *reader, struct dictionary *dict)
-{
- dict_unref (dict);
- casereader_destroy (reader);
- if (!file)
- proc_commit (dataset);
-}
-
-static void
-matrix_get_execute (struct matrix_command *cmd)
-{
- struct matrix_get *get = &cmd->get;
- struct casereader *r;
- struct dictionary *d;
- if (matrix_open_casereader (cmd, "GET", get->file, get->encoding,
- get->dataset, &r, &d))
- {
- matrix_get_execute__ (cmd, r, d);
- matrix_close_casereader (get->file, get->dataset, r, d);
- }
-}
-\f
-/* MSAVE. */
-
-static bool
-variables_changed (const char *keyword,
- const struct string_array *new_vars,
- const struct msg_location *new_vars_location,
- const struct msg_location *new_location,
- const struct string_array *old_vars,
- const struct msg_location *old_vars_location,
- const struct msg_location *old_location)
-{
- if (new_vars->n)
- {
- if (!old_vars->n)
- {
- msg_at (SE, new_location,
- _("%s may only be specified on MSAVE if it was specified "
- "on the first MSAVE within MATRIX."), keyword);
- msg_at (SN, old_location,
- _("The first MSAVE in MATRIX did not specify %s."),
- keyword);
- msg_at (SN, new_vars_location,
- _("This is the specification of %s on a later MSAVE."),
- keyword);
- return true;
- }
- if (!string_array_equal_case (old_vars, new_vars))
- {
- msg_at (SE, new_location,
- _("%s must specify the same variables on each MSAVE "
- "within a given MATRIX."), keyword);
- msg_at (SE, old_vars_location,
- _("This is the specification of %s on the first MSAVE."),
- keyword);
- msg_at (SE, new_vars_location,
- _("This is a different specification of %s on a later MSAVE."),
- keyword);
- return true;
- }
- }
- return false;
-}
-
-static bool
-msave_common_changed (const struct msave_common *old,
- const struct msave_common *new)
-{
- if (new->outfile && !fh_equal (old->outfile, new->outfile))
- {
- msg (SE, _("OUTFILE must name the same file on each MSAVE "
- "within a single MATRIX command."));
- msg_at (SN, old->outfile_location,
- _("This is the OUTFILE on the first MSAVE command."));
- msg_at (SN, new->outfile_location,
- _("This is the OUTFILE on a later MSAVE command."));
- return false;
- }
-
- if (!variables_changed ("VARIABLES",
- &new->variables, new->variables_location, new->location,
- &old->variables, old->variables_location, old->location)
- && !variables_changed ("FNAMES",
- &new->fnames, new->fnames_location, new->location,
- &old->fnames, old->fnames_location, old->location)
- && !variables_changed ("SNAMES",
- &new->snames, new->snames_location, new->location,
- &old->snames, old->snames_location, old->location))
- return false;
-
- return true;
-}
-
-static void
-msave_common_destroy (struct msave_common *common)
-{
- if (common)
- {
- msg_location_destroy (common->location);
- fh_unref (common->outfile);
- msg_location_destroy (common->outfile_location);
- string_array_destroy (&common->variables);
- msg_location_destroy (common->variables_location);
- string_array_destroy (&common->fnames);
- msg_location_destroy (common->fnames_location);
- string_array_destroy (&common->snames);
- msg_location_destroy (common->snames_location);
-
- for (size_t i = 0; i < common->n_factors; i++)
- matrix_expr_destroy (common->factors[i]);
- free (common->factors);
-
- for (size_t i = 0; i < common->n_splits; i++)
- matrix_expr_destroy (common->splits[i]);
- free (common->splits);
-
- dict_unref (common->dict);
- casewriter_destroy (common->writer);
-
- free (common);
- }
-}
-
-static const char *
-match_rowtype (struct lexer *lexer)
-{
- static const char *rowtypes[] = {
- "COV", "CORR", "MEAN", "STDDEV", "N", "COUNT"
- };
- size_t n_rowtypes = sizeof rowtypes / sizeof *rowtypes;
-
- for (size_t i = 0; i < n_rowtypes; i++)
- if (lex_match_id (lexer, rowtypes[i]))
- return rowtypes[i];
-
- lex_error_expecting_array (lexer, rowtypes, n_rowtypes);
- return NULL;
-}
-
-static bool
-parse_var_names (struct lexer *lexer, struct string_array *sa,
- struct msg_location **locationp)
-{
- lex_match (lexer, T_EQUALS);
-
- string_array_clear (sa);
- msg_location_destroy (*locationp);
- *locationp = NULL;
-
- struct dictionary *dict = dict_create (get_default_encoding ());
- char **names;
- size_t n_names;
- int start_ofs = lex_ofs (lexer);
- bool ok = parse_DATA_LIST_vars (lexer, dict, &names, &n_names,
- PV_NO_DUPLICATE | PV_NO_SCRATCH);
- int end_ofs = lex_ofs (lexer) - 1;
- dict_unref (dict);
-
- if (ok)
- {
- for (size_t i = 0; i < n_names; i++)
- if (ss_equals_case (ss_cstr (names[i]), ss_cstr ("ROWTYPE_"))
- || ss_equals_case (ss_cstr (names[i]), ss_cstr ("VARNAME_")))
- {
- lex_ofs_error (lexer, start_ofs, end_ofs,
- _("Variable name %s is reserved."), names[i]);
- for (size_t j = 0; j < n_names; j++)
- free (names[i]);
- free (names);
- return false;
- }
-
- sa->strings = names;
- sa->n = sa->allocated = n_names;
- *locationp = lex_ofs_location (lexer, start_ofs, end_ofs);
- }
- return ok;
-}
-
-static struct matrix_command *
-matrix_msave_parse (struct matrix_state *s)
-{
- int start_ofs = lex_ofs (s->lexer);
-
- struct msave_common *common = xmalloc (sizeof *common);
- *common = (struct msave_common) { .outfile = NULL };
-
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) { .type = MCMD_MSAVE, .msave = { .expr = NULL } };
-
- struct matrix_expr *splits = NULL;
- struct matrix_expr *factors = NULL;
-
- struct matrix_msave *msave = &cmd->msave;
- msave->expr = matrix_parse_exp (s);
- if (!msave->expr)
- goto error;
-
- while (lex_match (s->lexer, T_SLASH))
- {
- if (lex_match_id (s->lexer, "TYPE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- msave->rowtype = match_rowtype (s->lexer);
- if (!msave->rowtype)
- goto error;
- }
- else if (lex_match_id (s->lexer, "OUTFILE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- fh_unref (common->outfile);
- int start_ofs = lex_ofs (s->lexer);
- common->outfile = fh_parse (s->lexer, FH_REF_FILE, NULL);
- if (!common->outfile)
- goto error;
- msg_location_destroy (common->outfile_location);
- common->outfile_location = lex_ofs_location (s->lexer, start_ofs,
- lex_ofs (s->lexer) - 1);
- }
- else if (lex_match_id (s->lexer, "VARIABLES"))
- {
- if (!parse_var_names (s->lexer, &common->variables,
- &common->variables_location))
- goto error;
- }
- else if (lex_match_id (s->lexer, "FNAMES"))
- {
- if (!parse_var_names (s->lexer, &common->fnames,
- &common->fnames_location))
- goto error;
- }
- else if (lex_match_id (s->lexer, "SNAMES"))
- {
- if (!parse_var_names (s->lexer, &common->snames,
- &common->snames_location))
- goto error;
- }
- else if (lex_match_id (s->lexer, "SPLIT"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- matrix_expr_destroy (splits);
- splits = matrix_parse_exp (s);
- if (!splits)
- goto error;
- }
- else if (lex_match_id (s->lexer, "FACTOR"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- matrix_expr_destroy (factors);
- factors = matrix_parse_exp (s);
- if (!factors)
- goto error;
- }
- else
- {
- lex_error_expecting (s->lexer, "TYPE", "OUTFILE", "VARIABLES",
- "FNAMES", "SNAMES", "SPLIT", "FACTOR");
- goto error;
- }
- }
- if (!msave->rowtype)
- {
- lex_sbc_missing (s->lexer, "TYPE");
- goto error;
- }
-
- if (!s->msave_common)
- {
- if (common->fnames.n && !factors)
- {
- msg_at (SE, common->fnames_location, _("FNAMES requires FACTOR."));
- goto error;
- }
- if (common->snames.n && !splits)
- {
- msg_at (SE, common->snames_location, _("SNAMES requires SPLIT."));
- goto error;
- }
- if (!common->outfile)
- {
- lex_sbc_missing (s->lexer, "OUTFILE");
- goto error;
- }
- common->location = lex_ofs_location (s->lexer, start_ofs,
- lex_ofs (s->lexer));
- msg_location_remove_columns (common->location);
- s->msave_common = common;
- }
- else
- {
- if (msave_common_changed (s->msave_common, common))
- goto error;
- msave_common_destroy (common);
- }
- msave->common = s->msave_common;
-
- struct msave_common *c = s->msave_common;
- if (factors)
- {
- if (c->n_factors >= c->allocated_factors)
- c->factors = x2nrealloc (c->factors, &c->allocated_factors,
- sizeof *c->factors);
- c->factors[c->n_factors++] = factors;
- }
- if (c->n_factors > 0)
- msave->factors = c->factors[c->n_factors - 1];
-
- if (splits)
- {
- if (c->n_splits >= c->allocated_splits)
- c->splits = x2nrealloc (c->splits, &c->allocated_splits,
- sizeof *c->splits);
- c->splits[c->n_splits++] = splits;
- }
- if (c->n_splits > 0)
- msave->splits = c->splits[c->n_splits - 1];
-
- return cmd;
-
-error:
- matrix_expr_destroy (splits);
- matrix_expr_destroy (factors);
- msave_common_destroy (common);
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static gsl_vector *
-matrix_expr_evaluate_vector (const struct matrix_expr *e, const char *name)
-{
- gsl_matrix *m = matrix_expr_evaluate (e);
- if (!m)
- return NULL;
-
- if (!is_vector (m))
- {
- msg_at (SE, matrix_expr_location (e),
- _("%s expression must evaluate to vector, "
- "not a %zu×%zu matrix."),
- name, m->size1, m->size2);
- gsl_matrix_free (m);
- return NULL;
- }
-
- return matrix_to_vector (m);
-}
-
-static const char *
-msave_add_vars (struct dictionary *d, const struct string_array *vars)
-{
- for (size_t i = 0; i < vars->n; i++)
- if (!dict_create_var (d, vars->strings[i], 0))
- return vars->strings[i];
- return NULL;
-}
-
-static struct dictionary *
-msave_create_dict (const struct msave_common *common)
-{
- struct dictionary *dict = dict_create (get_default_encoding ());
-
- const char *dup_split = msave_add_vars (dict, &common->snames);
- if (dup_split)
- {
- /* Should not be possible because the parser ensures that the names are
- unique. */
- NOT_REACHED ();
- }
-
- dict_create_var_assert (dict, "ROWTYPE_", 8);
-
- const char *dup_factor = msave_add_vars (dict, &common->fnames);
- if (dup_factor)
- {
- msg_at (SE, common->fnames_location,
- _("Duplicate or invalid FACTOR variable name %s."),
- dup_factor);
- goto error;
- }
-
- dict_create_var_assert (dict, "VARNAME_", 8);
-
- const char *dup_var = msave_add_vars (dict, &common->variables);
- if (dup_var)
- {
- msg_at (SE, common->variables_location,
- _("Duplicate or invalid variable name %s."),
- dup_var);
- goto error;
- }
-
- return dict;
-
-error:
- dict_unref (dict);
- return NULL;
-}
-
-static void
-matrix_msave_execute (struct matrix_command *cmd)
-{
- struct matrix_msave *msave = &cmd->msave;
- struct msave_common *common = msave->common;
- gsl_matrix *m = NULL;
- gsl_vector *factors = NULL;
- gsl_vector *splits = NULL;
-
- m = matrix_expr_evaluate (msave->expr);
- if (!m)
- goto error;
-
- if (!common->variables.n)
- for (size_t i = 0; i < m->size2; i++)
- string_array_append_nocopy (&common->variables,
- xasprintf ("COL%zu", i + 1));
- else if (m->size2 != common->variables.n)
- {
- msg_at (SE, matrix_expr_location (msave->expr),
- _("Matrix on MSAVE has %zu columns but there are %zu variables."),
- m->size2, common->variables.n);
- goto error;
- }
-
- if (msave->factors)
- {
- factors = matrix_expr_evaluate_vector (msave->factors, "FACTOR");
- if (!factors)
- goto error;
-
- if (!common->fnames.n)
- for (size_t i = 0; i < factors->size; i++)
- string_array_append_nocopy (&common->fnames,
- xasprintf ("FAC%zu", i + 1));
- else if (factors->size != common->fnames.n)
- {
- msg_at (SE, matrix_expr_location (msave->factors),
- _("There are %zu factor variables, "
- "but %zu factor values were supplied."),
- common->fnames.n, factors->size);
- goto error;
- }
- }
-
- if (msave->splits)
- {
- splits = matrix_expr_evaluate_vector (msave->splits, "SPLIT");
- if (!splits)
- goto error;
-
- if (!common->snames.n)
- for (size_t i = 0; i < splits->size; i++)
- string_array_append_nocopy (&common->snames,
- xasprintf ("SPL%zu", i + 1));
- else if (splits->size != common->snames.n)
- {
- msg_at (SE, matrix_expr_location (msave->splits),
- _("There are %zu split variables, "
- "but %zu split values were supplied."),
- common->snames.n, splits->size);
- goto error;
- }
- }
-
- if (!common->writer)
- {
- struct dictionary *dict = msave_create_dict (common);
- if (!dict)
- goto error;
-
- common->writer = any_writer_open (common->outfile, dict);
- if (!common->writer)
- {
- dict_unref (dict);
- goto error;
- }
-
- common->dict = dict;
- }
-
- bool matrix = (!strcmp (msave->rowtype, "COV")
- || !strcmp (msave->rowtype, "CORR"));
- for (size_t y = 0; y < m->size1; y++)
- {
- struct ccase *c = case_create (dict_get_proto (common->dict));
- size_t idx = 0;
-
- /* Split variables */
- if (splits)
- for (size_t i = 0; i < splits->size; i++)
- case_data_rw_idx (c, idx++)->f = gsl_vector_get (splits, i);
-
- /* ROWTYPE_. */
- buf_copy_str_rpad (CHAR_CAST (char *, case_data_rw_idx (c, idx++)->s), 8,
- msave->rowtype, ' ');
-
- /* Factors. */
- if (factors)
- for (size_t i = 0; i < factors->size; i++)
- *case_num_rw_idx (c, idx++) = gsl_vector_get (factors, i);
-
- /* VARNAME_. */
- const char *varname_ = (matrix && y < common->variables.n
- ? common->variables.strings[y]
- : "");
- buf_copy_str_rpad (CHAR_CAST (char *, case_data_rw_idx (c, idx++)->s), 8,
- varname_, ' ');
-
- /* Continuous variables. */
- for (size_t x = 0; x < m->size2; x++)
- case_data_rw_idx (c, idx++)->f = gsl_matrix_get (m, y, x);
- casewriter_write (common->writer, c);
- }
-
-error:
- gsl_matrix_free (m);
- gsl_vector_free (factors);
- gsl_vector_free (splits);
-}
-\f
-/* MGET. */
-
-static struct matrix_command *
-matrix_mget_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_MGET,
- .mget = {
- .state = s,
- .rowtypes = STRINGI_SET_INITIALIZER (cmd->mget.rowtypes),
- },
- };
-
- struct matrix_mget *mget = &cmd->mget;
-
- lex_match (s->lexer, T_SLASH);
- while (lex_token (s->lexer) != T_ENDCMD)
- {
- if (lex_match_id (s->lexer, "FILE"))
- {
- lex_match (s->lexer, T_EQUALS);
-
- fh_unref (mget->file);
- mget->file = fh_parse (s->lexer, FH_REF_FILE, s->session);
- if (!mget->file)
- goto error;
- }
- else if (lex_match_id (s->lexer, "ENCODING"))
- {
- lex_match (s->lexer, T_EQUALS);
- if (!lex_force_string (s->lexer))
- goto error;
-
- free (mget->encoding);
- mget->encoding = ss_xstrdup (lex_tokss (s->lexer));
-
- lex_get (s->lexer);
- }
- else if (lex_match_id (s->lexer, "TYPE"))
- {
- lex_match (s->lexer, T_EQUALS);
- while (lex_token (s->lexer) != T_SLASH
- && lex_token (s->lexer) != T_ENDCMD)
- {
- const char *rowtype = match_rowtype (s->lexer);
- if (!rowtype)
- goto error;
-
- stringi_set_insert (&mget->rowtypes, rowtype);
- }
- }
- else
- {
- lex_error_expecting (s->lexer, "FILE", "TYPE");
- goto error;
- }
- lex_match (s->lexer, T_SLASH);
- }
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static const struct variable *
-get_a8_var (const struct msg_location *loc,
- const struct dictionary *d, const char *name)
-{
- const struct variable *v = dict_lookup_var (d, name);
- if (!v)
- {
- msg_at (SE, loc, _("Matrix data file lacks %s variable."), name);
- return NULL;
- }
- if (var_get_width (v) != 8)
- {
- msg_at (SE, loc, _("%s variable in matrix data file must be "
- "8-byte string, but it has width %d."),
- name, var_get_width (v));
- return NULL;
- }
- return v;
-}
-
-static bool
-var_changed (const struct ccase *ca, const struct ccase *cb,
- const struct variable *var)
-{
- return (ca && cb
- ? !value_equal (case_data (ca, var), case_data (cb, var),
- var_get_width (var))
- : ca || cb);
-}
-
-static bool
-vars_changed (const struct ccase *ca, const struct ccase *cb,
- const struct dictionary *d,
- size_t first_var, size_t n_vars)
-{
- for (size_t i = 0; i < n_vars; i++)
- {
- const struct variable *v = dict_get_var (d, first_var + i);
- if (var_changed (ca, cb, v))
- return true;
- }
- return false;
-}
-
-static bool
-vars_all_missing (const struct ccase *c, const struct dictionary *d,
- size_t first_var, size_t n_vars)
-{
- for (size_t i = 0; i < n_vars; i++)
- if (case_num (c, dict_get_var (d, first_var + i)) != SYSMIS)
- return false;
- return true;
-}
-
-static void
-matrix_mget_commit_var (struct ccase **rows, size_t n_rows,
- const struct dictionary *d,
- const struct variable *rowtype_var,
- const struct stringi_set *accepted_rowtypes,
- struct matrix_state *s,
- size_t ss, size_t sn, size_t si,
- size_t fs, size_t fn, size_t fi,
- size_t cs, size_t cn,
- struct pivot_table *pt,
- struct pivot_dimension *var_dimension)
-{
- if (!n_rows)
- goto exit;
-
- /* Is this a matrix for pooled data, either where there are no factor
- variables or the factor variables are missing? */
- bool pooled = !fn || vars_all_missing (rows[0], d, fs, fn);
-
- struct substring rowtype = case_ss (rows[0], rowtype_var);
- ss_rtrim (&rowtype, ss_cstr (" "));
- if (!stringi_set_is_empty (accepted_rowtypes)
- && !stringi_set_contains_len (accepted_rowtypes,
- rowtype.string, rowtype.length))
- goto exit;
-
- const char *prefix = (ss_equals_case (rowtype, ss_cstr ("COV")) ? "CV"
- : ss_equals_case (rowtype, ss_cstr ("CORR")) ? "CR"
- : ss_equals_case (rowtype, ss_cstr ("MEAN")) ? "MN"
- : ss_equals_case (rowtype, ss_cstr ("STDDEV")) ? "SD"
- : ss_equals_case (rowtype, ss_cstr ("N")) ? "NC"
- : ss_equals_case (rowtype, ss_cstr ("COUNT")) ? "CN"
- : NULL);
- if (!prefix)
- {
- msg (SE, _("Matrix data file contains unknown ROWTYPE_ \"%.*s\"."),
- (int) rowtype.length, rowtype.string);
- goto exit;
- }
-
- struct string name = DS_EMPTY_INITIALIZER;
- ds_put_cstr (&name, prefix);
- if (!pooled)
- ds_put_format (&name, "F%zu", fi);
- if (si > 0)
- ds_put_format (&name, "S%zu", si);
-
- struct matrix_var *mv = matrix_var_lookup (s, ds_ss (&name));
- if (!mv)
- mv = matrix_var_create (s, ds_ss (&name));
- else if (mv->value)
- {
- msg (SW, _("Matrix data file contains variable with existing name %s."),
- ds_cstr (&name));
- goto exit_free_name;
- }
-
- gsl_matrix *m = gsl_matrix_alloc (n_rows, cn);
- size_t n_missing = 0;
- for (size_t y = 0; y < n_rows; y++)
- {
- for (size_t x = 0; x < cn; x++)
- {
- struct variable *var = dict_get_var (d, cs + x);
- double value = case_num (rows[y], var);
- if (var_is_num_missing (var, value))
- {
- n_missing++;
- value = 0.0;
- }
- gsl_matrix_set (m, y, x, value);
- }
- }
-
- int var_index = pivot_category_create_leaf (
- var_dimension->root, pivot_value_new_user_text (ds_cstr (&name), SIZE_MAX));
- double values[] = { n_rows, cn };
- for (size_t j = 0; j < sn; j++)
- {
- struct variable *var = dict_get_var (d, ss + j);
- const union value *value = case_data (rows[0], var);
- pivot_table_put2 (pt, j, var_index,
- pivot_value_new_var_value (var, value));
- }
- for (size_t j = 0; j < fn; j++)
- {
- struct variable *var = dict_get_var (d, fs + j);
- const union value sysmis = { .f = SYSMIS };
- const union value *value = pooled ? &sysmis : case_data (rows[0], var);
- pivot_table_put2 (pt, j + sn, var_index,
- pivot_value_new_var_value (var, value));
- }
- for (size_t j = 0; j < sizeof values / sizeof *values; j++)
- pivot_table_put2 (pt, j + sn + fn, var_index,
- pivot_value_new_integer (values[j]));
-
- if (n_missing)
- msg (SE, ngettext ("Matrix data file variable %s contains a missing "
- "value, which was treated as zero.",
- "Matrix data file variable %s contains %zu missing "
- "values, which were treated as zero.", n_missing),
- ds_cstr (&name), n_missing);
- mv->value = m;
-
-exit_free_name:
- ds_destroy (&name);
-
-exit:
- for (size_t y = 0; y < n_rows; y++)
- case_unref (rows[y]);
-}
-
-static void
-matrix_mget_execute__ (struct matrix_command *cmd, struct casereader *r,
- const struct dictionary *d)
-{
- struct matrix_mget *mget = &cmd->mget;
- const struct msg_location *loc = cmd->location;
- const struct variable *rowtype_ = get_a8_var (loc, d, "ROWTYPE_");
- const struct variable *varname_ = get_a8_var (loc, d, "VARNAME_");
- if (!rowtype_ || !varname_)
- return;
-
- if (var_get_dict_index (rowtype_) >= var_get_dict_index (varname_))
- {
- msg_at (SE, loc,
- _("ROWTYPE_ must precede VARNAME_ in matrix data file."));
- return;
- }
- if (var_get_dict_index (varname_) + 1 >= dict_get_n_vars (d))
- {
- msg_at (SE, loc, _("Matrix data file contains no continuous variables."));
- return;
- }
-
- for (size_t i = 0; i < dict_get_n_vars (d); i++)
- {
- const struct variable *v = dict_get_var (d, i);
- if (v != rowtype_ && v != varname_ && var_get_width (v) != 0)
- {
- msg_at (SE, loc,
- _("Matrix data file contains unexpected string variable %s."),
- var_get_name (v));
- return;
- }
- }
-
- /* SPLIT variables. */
- size_t ss = 0;
- size_t sn = var_get_dict_index (rowtype_);
- struct ccase *sc = NULL;
- size_t si = 0;
-
- /* FACTOR variables. */
- size_t fs = var_get_dict_index (rowtype_) + 1;
- size_t fn = var_get_dict_index (varname_) - var_get_dict_index (rowtype_) - 1;
- struct ccase *fc = NULL;
- size_t fi = 0;
-
- /* Continuous variables. */
- size_t cs = var_get_dict_index (varname_) + 1;
- size_t cn = dict_get_n_vars (d) - cs;
- struct ccase *cc = NULL;
-
- /* Pivot table. */
- struct pivot_table *pt = pivot_table_create (
- N_("Matrix Variables Created by MGET"));
- struct pivot_dimension *attr_dimension = pivot_dimension_create (
- pt, PIVOT_AXIS_COLUMN, N_("Attribute"));
- struct pivot_dimension *var_dimension = pivot_dimension_create (
- pt, PIVOT_AXIS_ROW, N_("Variable"));
- if (sn > 0)
- {
- struct pivot_category *splits = pivot_category_create_group (
- attr_dimension->root, N_("Split Values"));
- for (size_t i = 0; i < sn; i++)
- pivot_category_create_leaf (splits, pivot_value_new_variable (
- dict_get_var (d, ss + i)));
- }
- if (fn > 0)
- {
- struct pivot_category *factors = pivot_category_create_group (
- attr_dimension->root, N_("Factors"));
- for (size_t i = 0; i < fn; i++)
- pivot_category_create_leaf (factors, pivot_value_new_variable (
- dict_get_var (d, fs + i)));
- }
- pivot_category_create_group (attr_dimension->root, N_("Dimensions"),
- N_("Rows"), N_("Columns"));
-
- /* Matrix. */
- struct ccase **rows = NULL;
- size_t allocated_rows = 0;
- size_t n_rows = 0;
-
- struct ccase *c;
- while ((c = casereader_read (r)) != NULL)
- {
- bool row_has_factors = fn && !vars_all_missing (c, d, fs, fn);
-
- enum
- {
- SPLITS_CHANGED,
- FACTORS_CHANGED,
- ROWTYPE_CHANGED,
- NOTHING_CHANGED
- }
- change
- = (sn && (!sc || vars_changed (sc, c, d, ss, sn)) ? SPLITS_CHANGED
- : fn && (!fc || vars_changed (fc, c, d, fs, fn)) ? FACTORS_CHANGED
- : !cc || var_changed (cc, c, rowtype_) ? ROWTYPE_CHANGED
- : NOTHING_CHANGED);
-
- if (change != NOTHING_CHANGED)
- {
- matrix_mget_commit_var (rows, n_rows, d,
- rowtype_, &mget->rowtypes,
- mget->state,
- ss, sn, si,
- fs, fn, fi,
- cs, cn,
- pt, var_dimension);
- n_rows = 0;
- case_unref (cc);
- cc = case_ref (c);
- }
-
- if (n_rows >= allocated_rows)
- rows = x2nrealloc (rows, &allocated_rows, sizeof *rows);
- rows[n_rows++] = c;
-
- if (change == SPLITS_CHANGED)
- {
- si++;
- case_unref (sc);
- sc = case_ref (c);
-
- /* Reset the factor number, if there are factors. */
- if (fn)
- {
- fi = 0;
- if (row_has_factors)
- fi++;
- case_unref (fc);
- fc = case_ref (c);
- }
- }
- else if (change == FACTORS_CHANGED)
- {
- if (row_has_factors)
- fi++;
- case_unref (fc);
- fc = case_ref (c);
- }
- }
- matrix_mget_commit_var (rows, n_rows, d,
- rowtype_, &mget->rowtypes,
- mget->state,
- ss, sn, si,
- fs, fn, fi,
- cs, cn,
- pt, var_dimension);
- free (rows);
-
- case_unref (sc);
- case_unref (fc);
- case_unref (cc);
-
- if (var_dimension->n_leaves)
- pivot_table_submit (pt);
- else
- pivot_table_unref (pt);
-}
-
-static void
-matrix_mget_execute (struct matrix_command *cmd)
-{
- struct matrix_mget *mget = &cmd->mget;
- struct casereader *r;
- struct dictionary *d;
- if (matrix_open_casereader (cmd, "MGET", mget->file, mget->encoding,
- mget->state->dataset, &r, &d))
- {
- matrix_mget_execute__ (cmd, r, d);
- matrix_close_casereader (mget->file, mget->state->dataset, r, d);
- }
-}
-\f
-/* CALL EIGEN. */
-
-static bool
-matrix_parse_dst_var (struct matrix_state *s, struct matrix_var **varp)
-{
- if (!lex_force_id (s->lexer))
- return false;
-
- *varp = matrix_var_lookup (s, lex_tokss (s->lexer));
- if (!*varp)
- *varp = matrix_var_create (s, lex_tokss (s->lexer));
- lex_get (s->lexer);
- return true;
-}
-
-static struct matrix_command *
-matrix_eigen_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_EIGEN,
- .eigen = { .expr = NULL }
- };
-
- struct matrix_eigen *eigen = &cmd->eigen;
- if (!lex_force_match (s->lexer, T_LPAREN))
- goto error;
- eigen->expr = matrix_expr_parse (s);
- if (!eigen->expr
- || !lex_force_match (s->lexer, T_COMMA)
- || !matrix_parse_dst_var (s, &eigen->evec)
- || !lex_force_match (s->lexer, T_COMMA)
- || !matrix_parse_dst_var (s, &eigen->eval)
- || !lex_force_match (s->lexer, T_RPAREN))
- goto error;
-
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_eigen_execute (struct matrix_command *cmd)
-{
- struct matrix_eigen *eigen = &cmd->eigen;
- gsl_matrix *A = matrix_expr_evaluate (eigen->expr);
- if (!A)
- return;
- if (!is_symmetric (A))
- {
- msg_at (SE, cmd->location, _("Argument of EIGEN must be symmetric."));
- gsl_matrix_free (A);
- return;
- }
-
- gsl_eigen_symmv_workspace *w = gsl_eigen_symmv_alloc (A->size1);
- gsl_matrix *eval = gsl_matrix_alloc (A->size1, 1);
- gsl_vector v_eval = to_vector (eval);
- gsl_matrix *evec = gsl_matrix_alloc (A->size1, A->size2);
- gsl_eigen_symmv (A, &v_eval, evec, w);
- gsl_eigen_symmv_free (w);
-
- gsl_eigen_symmv_sort (&v_eval, evec, GSL_EIGEN_SORT_VAL_DESC);
-
- gsl_matrix_free (A);
-
- gsl_matrix_free (eigen->eval->value);
- eigen->eval->value = eval;
-
- gsl_matrix_free (eigen->evec->value);
- eigen->evec->value = evec;
-}
-\f
-/* CALL SETDIAG. */
-
-static struct matrix_command *
-matrix_setdiag_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_SETDIAG,
- .setdiag = { .dst = NULL }
- };
-
- struct matrix_setdiag *setdiag = &cmd->setdiag;
- if (!lex_force_match (s->lexer, T_LPAREN) || !lex_force_id (s->lexer))
- goto error;
-
- setdiag->dst = matrix_var_lookup (s, lex_tokss (s->lexer));
- if (!setdiag->dst)
- {
- lex_error (s->lexer, _("Unknown variable %s."), lex_tokcstr (s->lexer));
- goto error;
- }
- lex_get (s->lexer);
-
- if (!lex_force_match (s->lexer, T_COMMA))
- goto error;
-
- setdiag->expr = matrix_expr_parse (s);
- if (!setdiag->expr)
- goto error;
-
- if (!lex_force_match (s->lexer, T_RPAREN))
- goto error;
-
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_setdiag_execute (struct matrix_command *cmd)
-{
- struct matrix_setdiag *setdiag = &cmd->setdiag;
- gsl_matrix *dst = setdiag->dst->value;
- if (!dst)
- {
- msg_at (SE, cmd->location,
- _("SETDIAG destination matrix %s is uninitialized."),
- setdiag->dst->name);
- return;
- }
-
- gsl_matrix *src = matrix_expr_evaluate (setdiag->expr);
- if (!src)
- return;
-
- size_t n = MIN (dst->size1, dst->size2);
- if (is_scalar (src))
- {
- double d = to_scalar (src);
- for (size_t i = 0; i < n; i++)
- gsl_matrix_set (dst, i, i, d);
- }
- else if (is_vector (src))
- {
- gsl_vector v = to_vector (src);
- for (size_t i = 0; i < n && i < v.size; i++)
- gsl_matrix_set (dst, i, i, gsl_vector_get (&v, i));
- }
- else
- msg_at (SE, matrix_expr_location (setdiag->expr),
- _("SETDIAG argument 2 must be a scalar or a vector, "
- "not a %zu×%zu matrix."),
- src->size1, src->size2);
- gsl_matrix_free (src);
-}
-\f
-/* CALL SVD. */
-
-static struct matrix_command *
-matrix_svd_parse (struct matrix_state *s)
-{
- struct matrix_command *cmd = xmalloc (sizeof *cmd);
- *cmd = (struct matrix_command) {
- .type = MCMD_SVD,
- .svd = { .expr = NULL }
- };
-
- struct matrix_svd *svd = &cmd->svd;
- if (!lex_force_match (s->lexer, T_LPAREN))
- goto error;
- svd->expr = matrix_expr_parse (s);
- if (!svd->expr
- || !lex_force_match (s->lexer, T_COMMA)
- || !matrix_parse_dst_var (s, &svd->u)
- || !lex_force_match (s->lexer, T_COMMA)
- || !matrix_parse_dst_var (s, &svd->s)
- || !lex_force_match (s->lexer, T_COMMA)
- || !matrix_parse_dst_var (s, &svd->v)
- || !lex_force_match (s->lexer, T_RPAREN))
- goto error;
-
- return cmd;
-
-error:
- matrix_command_destroy (cmd);
- return NULL;
-}
-
-static void
-matrix_svd_execute (struct matrix_svd *svd)
-{
- gsl_matrix *m = matrix_expr_evaluate (svd->expr);
- if (!m)
- return;
-
- if (m->size1 >= m->size2)
- {
- gsl_matrix *A = m;
- gsl_matrix *V = gsl_matrix_alloc (A->size2, A->size2);
- gsl_matrix *S = gsl_matrix_calloc (A->size2, A->size2);
- gsl_vector Sv = gsl_matrix_diagonal (S).vector;
- gsl_vector *work = gsl_vector_alloc (A->size2);
- gsl_linalg_SV_decomp (A, V, &Sv, work);
- gsl_vector_free (work);
-
- matrix_var_set (svd->u, A);
- matrix_var_set (svd->s, S);
- matrix_var_set (svd->v, V);
- }
- else
- {
- gsl_matrix *At = gsl_matrix_alloc (m->size2, m->size1);
- gsl_matrix_transpose_memcpy (At, m);
- gsl_matrix_free (m);
-
- gsl_matrix *Vt = gsl_matrix_alloc (At->size2, At->size2);
- gsl_matrix *St = gsl_matrix_calloc (At->size2, At->size2);
- gsl_vector Stv = gsl_matrix_diagonal (St).vector;
- gsl_vector *work = gsl_vector_alloc (At->size2);
- gsl_linalg_SV_decomp (At, Vt, &Stv, work);
- gsl_vector_free (work);
-
- matrix_var_set (svd->v, At);
- matrix_var_set (svd->s, St);
- matrix_var_set (svd->u, Vt);
- }
-}
-\f
-/* The main MATRIX command logic. */
-
-static bool
-matrix_command_execute (struct matrix_command *cmd)
-{
- switch (cmd->type)
- {
- case MCMD_COMPUTE:
- matrix_compute_execute (cmd);
- break;
-
- case MCMD_PRINT:
- matrix_print_execute (&cmd->print);
- break;
-
- case MCMD_DO_IF:
- return matrix_do_if_execute (&cmd->do_if);
-
- case MCMD_LOOP:
- matrix_loop_execute (&cmd->loop);
- break;
-
- case MCMD_BREAK:
- return false;
-
- case MCMD_DISPLAY:
- matrix_display_execute (&cmd->display);
- break;
-
- case MCMD_RELEASE:
- matrix_release_execute (&cmd->release);
- break;
-
- case MCMD_SAVE:
- matrix_save_execute (cmd);
- break;
-
- case MCMD_READ:
- matrix_read_execute (cmd);
- break;
-
- case MCMD_WRITE:
- matrix_write_execute (&cmd->write);
- break;
-
- case MCMD_GET:
- matrix_get_execute (cmd);
- break;
-
- case MCMD_MSAVE:
- matrix_msave_execute (cmd);
- break;
-
- case MCMD_MGET:
- matrix_mget_execute (cmd);
- break;
-
- case MCMD_EIGEN:
- matrix_eigen_execute (cmd);
- break;
-
- case MCMD_SETDIAG:
- matrix_setdiag_execute (cmd);
- break;
-
- case MCMD_SVD:
- matrix_svd_execute (&cmd->svd);
- break;
- }
-
- return true;
-}
-
-static void
-matrix_command_destroy (struct matrix_command *cmd)
-{
- if (!cmd)
- return;
-
- msg_location_destroy (cmd->location);
-
- switch (cmd->type)
- {
- case MCMD_COMPUTE:
- matrix_lvalue_destroy (cmd->compute.lvalue);
- matrix_expr_destroy (cmd->compute.rvalue);
- break;
-
- case MCMD_PRINT:
- matrix_expr_destroy (cmd->print.expression);
- free (cmd->print.title);
- print_labels_destroy (cmd->print.rlabels);
- print_labels_destroy (cmd->print.clabels);
- break;
-
- case MCMD_DO_IF:
- for (size_t i = 0; i < cmd->do_if.n_clauses; i++)
- {
- matrix_expr_destroy (cmd->do_if.clauses[i].condition);
- matrix_commands_uninit (&cmd->do_if.clauses[i].commands);
- }
- free (cmd->do_if.clauses);
- break;
-
- case MCMD_LOOP:
- matrix_expr_destroy (cmd->loop.start);
- matrix_expr_destroy (cmd->loop.end);
- matrix_expr_destroy (cmd->loop.increment);
- matrix_expr_destroy (cmd->loop.top_condition);
- matrix_expr_destroy (cmd->loop.bottom_condition);
- matrix_commands_uninit (&cmd->loop.commands);
- break;
-
- case MCMD_BREAK:
- break;
-
- case MCMD_DISPLAY:
- break;
-
- case MCMD_RELEASE:
- free (cmd->release.vars);
- break;
-
- case MCMD_SAVE:
- matrix_expr_destroy (cmd->save.expression);
- break;
-
- case MCMD_READ:
- matrix_lvalue_destroy (cmd->read.dst);
- matrix_expr_destroy (cmd->read.size);
- break;
-
- case MCMD_WRITE:
- matrix_expr_destroy (cmd->write.expression);
- free (cmd->write.format);
- break;
-
- case MCMD_GET:
- matrix_lvalue_destroy (cmd->get.dst);
- fh_unref (cmd->get.file);
- free (cmd->get.encoding);
- var_syntax_destroy (cmd->get.vars, cmd->get.n_vars);
- break;
-
- case MCMD_MSAVE:
- matrix_expr_destroy (cmd->msave.expr);
- break;
-
- case MCMD_MGET:
- fh_unref (cmd->mget.file);
- stringi_set_destroy (&cmd->mget.rowtypes);
- break;
-
- case MCMD_EIGEN:
- matrix_expr_destroy (cmd->eigen.expr);
- break;
-
- case MCMD_SETDIAG:
- matrix_expr_destroy (cmd->setdiag.expr);
- break;
-
- case MCMD_SVD:
- matrix_expr_destroy (cmd->svd.expr);
- break;
- }
- free (cmd);
-}
-
-static bool
-matrix_commands_parse (struct matrix_state *s, struct matrix_commands *c,
- const char *command_name,
- const char *stop1, const char *stop2)
-{
- lex_end_of_command (s->lexer);
- lex_discard_rest_of_command (s->lexer);
-
- size_t allocated = 0;
- for (;;)
- {
- while (lex_token (s->lexer) == T_ENDCMD)
- lex_get (s->lexer);
-
- if (lex_at_phrase (s->lexer, stop1)
- || (stop2 && lex_at_phrase (s->lexer, stop2)))
- return true;
-
- if (lex_at_phrase (s->lexer, "END MATRIX"))
- {
- lex_next_error (s->lexer, 0, 1,
- _("Premature END MATRIX within %s."), command_name);
- return false;
- }
-
- struct matrix_command *cmd = matrix_command_parse (s);
- if (!cmd)
- return false;
-
- if (c->n >= allocated)
- c->commands = x2nrealloc (c->commands, &allocated, sizeof *c->commands);
- c->commands[c->n++] = cmd;
- }
-}
-
-static void
-matrix_commands_uninit (struct matrix_commands *cmds)
-{
- for (size_t i = 0; i < cmds->n; i++)
- matrix_command_destroy (cmds->commands[i]);
- free (cmds->commands);
-}
-
-struct matrix_command_name
- {
- const char *name;
- struct matrix_command *(*parse) (struct matrix_state *);
- };
-
-static const struct matrix_command_name *
-matrix_command_name_parse (struct lexer *lexer)
-{
- static const struct matrix_command_name commands[] = {
- { "COMPUTE", matrix_compute_parse },
- { "CALL EIGEN", matrix_eigen_parse },
- { "CALL SETDIAG", matrix_setdiag_parse },
- { "CALL SVD", matrix_svd_parse },
- { "PRINT", matrix_print_parse },
- { "DO IF", matrix_do_if_parse },
- { "LOOP", matrix_loop_parse },
- { "BREAK", matrix_break_parse },
- { "READ", matrix_read_parse },
- { "WRITE", matrix_write_parse },
- { "GET", matrix_get_parse },
- { "SAVE", matrix_save_parse },
- { "MGET", matrix_mget_parse },
- { "MSAVE", matrix_msave_parse },
- { "DISPLAY", matrix_display_parse },
- { "RELEASE", matrix_release_parse },
- };
- static size_t n = sizeof commands / sizeof *commands;
-
- for (const struct matrix_command_name *c = commands; c < &commands[n]; c++)
- if (lex_match_phrase (lexer, c->name))
- return c;
- return NULL;
-}
-
-static struct matrix_command *
-matrix_command_parse (struct matrix_state *s)
-{
- int start_ofs = lex_ofs (s->lexer);
- size_t nesting_level = SIZE_MAX;
-
- struct matrix_command *c = NULL;
- const struct matrix_command_name *cmd = matrix_command_name_parse (s->lexer);
- if (!cmd)
- lex_error (s->lexer, _("Unknown matrix command."));
- else if (!cmd->parse)
- lex_error (s->lexer, _("Matrix command %s is not yet implemented."),
- cmd->name);
- else
- {
- nesting_level = output_open_group (
- group_item_create_nocopy (utf8_to_title (cmd->name),
- utf8_to_title (cmd->name)));
- c = cmd->parse (s);
- }
-
- if (c)
- {
- c->location = lex_ofs_location (s->lexer, start_ofs, lex_ofs (s->lexer));
- msg_location_remove_columns (c->location);
- lex_end_of_command (s->lexer);
- }
- lex_discard_rest_of_command (s->lexer);
- if (nesting_level != SIZE_MAX)
- output_close_groups (nesting_level);
-
- return c;
-}
-
-int
-cmd_matrix (struct lexer *lexer, struct dataset *ds)
-{
- if (!lex_force_match (lexer, T_ENDCMD))
- return CMD_FAILURE;
-
- struct matrix_state state = {
- .dataset = ds,
- .session = dataset_session (ds),
- .lexer = lexer,
- .vars = HMAP_INITIALIZER (state.vars),
- };
-
- for (;;)
- {
- while (lex_match (lexer, T_ENDCMD))
- continue;
- if (lex_token (lexer) == T_STOP)
- {
- msg (SE, _("Unexpected end of input expecting matrix command."));
- break;
- }
-
- if (lex_match_phrase (lexer, "END MATRIX"))
- break;
-
- struct matrix_command *c = matrix_command_parse (&state);
- if (c)
- {
- matrix_command_execute (c);
- matrix_command_destroy (c);
- }
- }
-
- struct matrix_var *var, *next;
- HMAP_FOR_EACH_SAFE (var, next, struct matrix_var, hmap_node, &state.vars)
- {
- free (var->name);
- gsl_matrix_free (var->value);
- hmap_delete (&state.vars, &var->hmap_node);
- free (var);
- }
- hmap_destroy (&state.vars);
- msave_common_destroy (state.msave_common);
- fh_unref (state.prev_read_file);
- for (size_t i = 0; i < state.n_read_files; i++)
- read_file_destroy (state.read_files[i]);
- free (state.read_files);
- fh_unref (state.prev_write_file);
- for (size_t i = 0; i < state.n_write_files; i++)
- write_file_destroy (state.write_files[i]);
- free (state.write_files);
- fh_unref (state.prev_save_file);
- for (size_t i = 0; i < state.n_save_files; i++)
- save_file_destroy (state.save_files[i]);
- free (state.save_files);
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "mcnemar.h"
-
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_randist.h>
-
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "data/variable.h"
-#include "language/stats/npar.h"
-#include "libpspp/str.h"
-#include "output/pivot-table.h"
-#include "libpspp/message.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "data/value.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct mcnemar
-{
- union value val0;
- union value val1;
-
- double n00;
- double n01;
- double n10;
- double n11;
-};
-
-static void
-output_freq_table (variable_pair *vp,
- const struct mcnemar *param,
- const struct dictionary *dict);
-
-
-static void
-output_statistics_table (const struct two_sample_test *t2s,
- const struct mcnemar *param,
- const struct dictionary *dict);
-
-
-void
-mcnemar_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- int i;
- bool warn = true;
-
- const struct dictionary *dict = dataset_dict (ds);
-
- const struct two_sample_test *t2s = UP_CAST (test, const struct two_sample_test, parent);
- struct ccase *c;
-
- struct casereader *r = input;
-
- struct mcnemar *mc = XCALLOC (t2s->n_pairs, struct mcnemar);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- mc[i].val0.f = mc[i].val1.f = SYSMIS;
- }
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- const double weight = dict_get_case_weight (dict, c, &warn);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
- const union value *value0 = case_data (c, (*vp)[0]);
- const union value *value1 = case_data (c, (*vp)[1]);
-
- if (var_is_value_missing ((*vp)[0], value0) & exclude)
- continue;
-
- if (var_is_value_missing ((*vp)[1], value1) & exclude)
- continue;
-
-
- if (mc[i].val0.f == SYSMIS)
- {
- if (mc[i].val1.f != value0->f)
- mc[i].val0.f = value0->f;
- else if (mc[i].val1.f != value1->f)
- mc[i].val0.f = value1->f;
- }
-
- if (mc[i].val1.f == SYSMIS)
- {
- if (mc[i].val0.f != value1->f)
- mc[i].val1.f = value1->f;
- else if (mc[i].val0.f != value0->f)
- mc[i].val1.f = value0->f;
- }
-
- if (mc[i].val0.f == value0->f && mc[i].val0.f == value1->f)
- {
- mc[i].n00 += weight;
- }
- else if (mc[i].val0.f == value0->f && mc[i].val1.f == value1->f)
- {
- mc[i].n10 += weight;
- }
- else if (mc[i].val1.f == value0->f && mc[i].val0.f == value1->f)
- {
- mc[i].n01 += weight;
- }
- else if (mc[i].val1.f == value0->f && mc[i].val1.f == value1->f)
- {
- mc[i].n11 += weight;
- }
- else
- {
- msg (ME, _("The McNemar test is appropriate only for dichotomous variables"));
- }
- }
- }
-
- casereader_destroy (r);
-
- for (i = 0 ; i < t2s->n_pairs ; ++i)
- output_freq_table (&t2s->pairs[i], mc + i, dict);
-
- output_statistics_table (t2s, mc, dict);
-
- free (mc);
-}
-
-static char *
-make_pair_name (const variable_pair *pair)
-{
- return xasprintf ("%s & %s", var_to_string ((*pair)[0]),
- var_to_string ((*pair)[1]));
-}
-
-static void
-output_freq_table (variable_pair *vp,
- const struct mcnemar *param,
- const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_user_text_nocopy (make_pair_name (vp)), "Frequencies");
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- struct pivot_dimension *vars[2];
- for (int i = 0; i < 2; i++)
- {
- vars[i] = pivot_dimension_create__ (
- table, i ? PIVOT_AXIS_COLUMN : PIVOT_AXIS_ROW,
- pivot_value_new_variable ((*vp)[i]));
- vars[i]->root->show_label = true;
-
- for (int j = 0; j < 2; j++)
- {
- const union value *val = j ? ¶m->val1 : ¶m->val0;
- pivot_category_create_leaf_rc (
- vars[i]->root, pivot_value_new_var_value ((*vp)[0], val),
- PIVOT_RC_COUNT);
- }
- }
-
- struct entry
- {
- int idx0;
- int idx1;
- double x;
- }
- entries[] = {
- { 0, 0, param->n00 },
- { 1, 0, param->n01 },
- { 0, 1, param->n10 },
- { 1, 1, param->n11 },
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put2 (table, e->idx0, e->idx1,
- pivot_value_new_number (e->x));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-output_statistics_table (const struct two_sample_test *t2s,
- const struct mcnemar *mc,
- const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Point Probability"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Pairs"));
-
- for (size_t i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
- int pair_idx = pivot_category_create_leaf (
- pairs->root, pivot_value_new_user_text_nocopy (make_pair_name (vp)));
-
- double n = mc[i].n00 + mc[i].n01 + mc[i].n10 + mc[i].n11;
- double sig = gsl_cdf_binomial_P ((mc[i].n01 > mc[i].n10) ? mc[i].n10: mc[i].n01,
- 0.5, mc[i].n01 + mc[i].n10);
-
- double point = gsl_ran_binomial_pdf (mc[i].n01, 0.5,
- mc[i].n01 + mc[i].n10);
- double entries[] = { n, 2.0 * sig, sig, point };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, pair_idx,
- pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !mcnemar_h
-#define mcnemar_h 1
-
-
-#include <stdbool.h>
-#include "data/missing-values.h"
-
-struct casereader;
-struct dataset;
-struct npar_test;
-
-void mcnemar_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer);
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011, 2012, 2013, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "data/case.h"
-#include "data/format.h"
-#include "data/variable.h"
-
-#include "libpspp/bt.h"
-#include "libpspp/hmap.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-
-#include "math/moments.h"
-#include "output/pivot-table.h"
-
-#include <math.h>
-
-#include "means.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-/* A base struct for all statistics. */
-struct statistic
-{
-};
-
-/* Statistics which accumulate a single value. */
-struct statistic_simple
-{
- struct statistic parent;
- double acc;
-};
-
-/* Statistics based on moments. */
-struct statistic_moment
-{
- struct statistic parent;
- struct moments1 *mom;
-};
-
-
-static struct statistic *
-default_create (struct pool *pool)
-{
- struct statistic_moment *pvd = pool_alloc (pool, sizeof *pvd);
-
- pvd->mom = moments1_create (MOMENT_KURTOSIS);
-
- return (struct statistic *) pvd;
-}
-
-static void
-default_update (struct statistic *stat, double w, double x)
-{
- struct statistic_moment *pvd = (struct statistic_moment *) stat;
-
- moments1_add (pvd->mom, x, w);
-}
-
-static void
-default_destroy (struct statistic *stat)
-{
- struct statistic_moment *pvd = (struct statistic_moment *) stat;
- moments1_destroy (pvd->mom);
-}
-
-
-/* Simple statistics have nothing to destroy. */
-static void
-simple_destroy (struct statistic *stat UNUSED)
-{
-}
-
-\f
-
-/* HARMONIC MEAN: The reciprocal of the sum of the reciprocals:
- 1 / (1/(x_0) + 1/(x_1) + ... + 1/(x_{n-1})) */
-
-struct harmonic_mean
-{
- struct statistic parent;
- double rsum;
- double n;
-};
-
-static struct statistic *
-harmonic_create (struct pool *pool)
-{
- struct harmonic_mean *hm = pool_alloc (pool, sizeof *hm);
-
- hm->rsum = 0;
- hm->n = 0;
-
- return (struct statistic *) hm;
-}
-
-
-static void
-harmonic_update (struct statistic *stat, double w, double x)
-{
- struct harmonic_mean *hm = (struct harmonic_mean *) stat;
- hm->rsum += w / x;
- hm->n += w;
-}
-
-
-static double
-harmonic_get (const struct statistic *pvd)
-{
- const struct harmonic_mean *hm = (const struct harmonic_mean *) pvd;
-
- return hm->n / hm->rsum;
-}
-
-\f
-
-/* GEOMETRIC MEAN: The nth root of the product of all n observations
- pow ((x_0 * x_1 * ... x_{n - 1}), 1/n) */
-struct geometric_mean
-{
- struct statistic parent;
- double prod;
- double n;
-};
-
-static struct statistic *
-geometric_create (struct pool *pool)
-{
- struct geometric_mean *gm = pool_alloc (pool, sizeof *gm);
-
- gm->prod = 1.0;
- gm->n = 0;
-
- return (struct statistic *) gm;
-}
-
-static void
-geometric_update (struct statistic *pvd, double w, double x)
-{
- struct geometric_mean *gm = (struct geometric_mean *)pvd;
- gm->prod *= pow (x, w);
- gm->n += w;
-}
-
-
-static double
-geometric_get (const struct statistic *pvd)
-{
- const struct geometric_mean *gm = (const struct geometric_mean *)pvd;
- return pow (gm->prod, 1.0 / gm->n);
-}
-
-\f
-
-/* The getters for moment based statistics simply calculate the
- moment. The only exception is Std Dev. which needs to call
- sqrt as well. */
-
-static double
-sum_get (const struct statistic *pvd)
-{
- double n, mean;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, &mean, 0, 0, 0);
-
- return mean * n;
-}
-
-
-static double
-n_get (const struct statistic *pvd)
-{
- double n;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, 0, 0, 0, 0);
-
- return n;
-}
-
-static double
-arithmean_get (const struct statistic *pvd)
-{
- double n, mean;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, &mean, 0, 0, 0);
-
- return mean;
-}
-
-static double
-variance_get (const struct statistic *pvd)
-{
- double n, mean, variance;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, &mean, &variance, 0, 0);
-
- return variance;
-}
-
-
-static double
-stddev_get (const struct statistic *pvd)
-{
- return sqrt (variance_get (pvd));
-}
-
-static double
-skew_get (const struct statistic *pvd)
-{
- double skew;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, NULL, NULL, NULL, &skew, 0);
-
- return skew;
-}
-
-static double
-sekurt_get (const struct statistic *pvd)
-{
- double n;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, NULL, NULL, NULL, NULL);
-
- return calc_sekurt (n);
-}
-
-static double
-seskew_get (const struct statistic *pvd)
-{
- double n;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, NULL, NULL, NULL, NULL);
-
- return calc_seskew (n);
-}
-
-static double
-kurt_get (const struct statistic *pvd)
-{
- double kurt;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, NULL, NULL, NULL, NULL, &kurt);
-
- return kurt;
-}
-
-static double
-semean_get (const struct statistic *pvd)
-{
- double n, var;
-
- moments1_calculate (((struct statistic_moment *)pvd)->mom, &n, NULL, &var, NULL, NULL);
-
- return sqrt (var / n);
-}
-
-\f
-
-/* MIN: The smallest (closest to minus infinity) value. */
-
-static struct statistic *
-min_create (struct pool *pool)
-{
- struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
-
- pvd->acc = DBL_MAX;
-
- return (struct statistic *) pvd;
-}
-
-static void
-min_update (struct statistic *pvd, double w UNUSED, double x)
-{
- double *r = &((struct statistic_simple *)pvd)->acc;
-
- if (x < *r)
- *r = x;
-}
-
-static double
-min_get (const struct statistic *pvd)
-{
- double *r = &((struct statistic_simple *)pvd)->acc;
-
- return *r;
-}
-
-/* MAX: The largest (closest to plus infinity) value. */
-
-static struct statistic *
-max_create (struct pool *pool)
-{
- struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
-
- pvd->acc = -DBL_MAX;
-
- return (struct statistic *) pvd;
-}
-
-static void
-max_update (struct statistic *pvd, double w UNUSED, double x)
-{
- double *r = &((struct statistic_simple *)pvd)->acc;
-
- if (x > *r)
- *r = x;
-}
-
-static double
-max_get (const struct statistic *pvd)
-{
- double *r = &((struct statistic_simple *)pvd)->acc;
-
- return *r;
-}
-
-\f
-
-struct range
-{
- struct statistic parent;
- double min;
- double max;
-};
-
-/* Initially min and max are set to their most (inverted) extreme possible
- values. */
-static struct statistic *
-range_create (struct pool *pool)
-{
- struct range *r = pool_alloc (pool, sizeof *r);
-
- r->min = DBL_MAX;
- r->max = -DBL_MAX;
-
- return (struct statistic *) r;
-}
-
-/* On each update, set min and max to X or leave unchanged,
- as appropriate. */
-static void
-range_update (struct statistic *pvd, double w UNUSED, double x)
-{
- struct range *r = (struct range *) pvd;
-
- if (x > r->max)
- r->max = x;
-
- if (x < r->min)
- r->min = x;
-}
-
-/* Get the difference between min and max. */
-static double
-range_get (const struct statistic *pvd)
-{
- const struct range *r = (struct range *) pvd;
-
- return r->max - r->min;
-}
-
-\f
-
-/* LAST: The last value (the one closest to the end of the file). */
-
-static struct statistic *
-last_create (struct pool *pool)
-{
- struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
-
- return (struct statistic *) pvd;
-}
-
-static void
-last_update (struct statistic *pvd, double w UNUSED, double x)
-{
- struct statistic_simple *stat = (struct statistic_simple *) pvd;
-
- stat->acc = x;
-}
-
-static double
-last_get (const struct statistic *pvd)
-{
- const struct statistic_simple *stat = (struct statistic_simple *) pvd;
-
- return stat->acc;
-}
-
-/* FIRST: The first value (the one closest to the start of the file). */
-
-static struct statistic *
-first_create (struct pool *pool)
-{
- struct statistic_simple *pvd = pool_alloc (pool, sizeof *pvd);
-
- pvd->acc = SYSMIS;
-
- return (struct statistic *) pvd;
-}
-
-static void
-first_update (struct statistic *pvd, double w UNUSED, double x)
-{
- struct statistic_simple *stat = (struct statistic_simple *) pvd;
-
- if (stat->acc == SYSMIS)
- stat->acc = x;
-}
-
-static double
-first_get (const struct statistic *pvd)
-{
- const struct statistic_simple *stat = (struct statistic_simple *) pvd;
-
- return stat->acc;
-}
-
-/* Table of cell_specs */
-const struct cell_spec cell_spec[n_MEANS_STATISTICS] = {
- {N_("Mean"), "MEAN", NULL , default_create, default_update, arithmean_get, default_destroy},
- {N_("N"), "COUNT", PIVOT_RC_COUNT, default_create, default_update, n_get, default_destroy},
- {N_("Std. Deviation"), "STDDEV", NULL , default_create, default_update, stddev_get, default_destroy},
-#if 0
- {N_("Median"), "MEDIAN", NULL , default_create, default_update, NULL, default_destroy},
- {N_("Group Median"), "GMEDIAN", NULL , default_create, default_update, NULL, default_destroy},
-#endif
- {N_("S.E. Mean"), "SEMEAN", NULL , default_create, default_update, semean_get, default_destroy},
- {N_("Sum"), "SUM", NULL , default_create, default_update, sum_get, default_destroy},
- {N_("Minimum"), "MIN", NULL , min_create, min_update, min_get, simple_destroy},
- {N_("Maximum"), "MAX", NULL , max_create, max_update, max_get, simple_destroy},
- {N_("Range"), "RANGE", NULL , range_create, range_update, range_get, simple_destroy},
- {N_("Variance"), "VARIANCE", PIVOT_RC_OTHER, default_create, default_update, variance_get, default_destroy},
- {N_("Kurtosis"), "KURT", PIVOT_RC_OTHER, default_create, default_update, kurt_get, default_destroy},
- {N_("S.E. Kurt"), "SEKURT", PIVOT_RC_OTHER, default_create, default_update, sekurt_get, default_destroy},
- {N_("Skewness"), "SKEW", PIVOT_RC_OTHER, default_create, default_update, skew_get, default_destroy},
- {N_("S.E. Skew"), "SESKEW", PIVOT_RC_OTHER, default_create, default_update, seskew_get, default_destroy},
- {N_("First"), "FIRST", NULL , first_create, first_update, first_get, simple_destroy},
- {N_("Last"), "LAST", NULL , last_create, last_update, last_get, simple_destroy},
-#if 0
- {N_("Percent N"), "NPCT", PIVOT_RC_PERCENT, default_create, default_update, NULL, default_destroy},
- {N_("Percent Sum"), "SPCT", PIVOT_RC_PERCENT, default_create, default_update, NULL, default_destroy},
-#endif
- {N_("Harmonic Mean"), "HARMONIC", NULL , harmonic_create, harmonic_update, harmonic_get, simple_destroy},
- {N_("Geom. Mean"), "GEOMETRIC", NULL , geometric_create, geometric_update, geometric_get, simple_destroy}
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011, 2012, 2013, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-
-#include "libpspp/pool.h"
-
-#include "means.h"
-
-/* Parse the /TABLES stanza of the command. */
-static bool
-parse_means_table_syntax (struct lexer *lexer, const struct means *cmd,
- struct mtable *table)
-{
- memset (table, 0, sizeof *table);
-
- /* Dependent variable (s) */
- if (!parse_variables_const_pool (lexer, cmd->pool, cmd->dict,
- &table->dep_vars, &table->n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- return false;
-
- /* Factor variable (s) */
- while (lex_match (lexer, T_BY))
- {
- struct layer *layer = pool_zalloc (cmd->pool, sizeof *layer);
-
- table->layers =
- pool_nrealloc (cmd->pool, table->layers, table->n_layers + 1,
- sizeof *table->layers);
- table->layers[table->n_layers] = layer;
- table->n_layers++;
-
- if (!parse_variables_const_pool
- (lexer, cmd->pool, cmd->dict,
- &layer->factor_vars,
- &layer->n_factor_vars,
- PV_NO_DUPLICATE))
- return false;
- }
-
- return true;
-}
-
-/* Match a variable.
- If the match succeeds, the variable will be placed in VAR.
- Returns true if successful */
-static bool
-lex_is_variable (struct lexer *lexer, const struct dictionary *dict,
- int n)
-{
- if (lex_next_token (lexer, n) != T_ID)
- return false;
-
- const char *tstr = lex_next_tokcstr (lexer, n);
- return dict_lookup_var (dict, tstr) != NULL;
-}
-
-static const struct cell_spec *
-match_cell (struct lexer *lexer)
-{
- for (size_t i = 0; i < n_MEANS_STATISTICS; ++i)
- {
- const struct cell_spec *cs = &cell_spec[i];
- if (lex_match_id (lexer, cs->keyword))
- return cs;
- }
- return NULL;
-}
-
-static void
-add_statistic (struct means *means, int statistic)
-{
- if (means->n_statistics >= means->allocated_statistics)
- means->statistics = pool_2nrealloc (means->pool, means->statistics,
- &means->allocated_statistics,
- sizeof *means->statistics);
- means->statistics[means->n_statistics++] = statistic;
-}
-
-void
-means_set_default_statistics (struct means *means)
-{
- means->n_statistics = 0;
- add_statistic (means, MEANS_MEAN);
- add_statistic (means, MEANS_N);
- add_statistic (means, MEANS_STDDEV);
-}
-
-bool
-means_parse (struct lexer *lexer, struct means *means)
-{
- /* Optional TABLES=. */
- if (lex_match_id (lexer, "TABLES") && !lex_force_match (lexer, T_EQUALS))
- return false;
-
- /* Parse the "tables" */
- for (;;)
- {
- means->table = pool_realloc (means->pool, means->table,
- (means->n_tables + 1) * sizeof *means->table);
-
- if (!parse_means_table_syntax (lexer, means,
- &means->table[means->n_tables]))
- return false;
- means->n_tables++;
-
- /* Look ahead to see if there are more tables to be parsed */
- if (lex_next_token (lexer, 0) != T_SLASH
- || !lex_is_variable (lexer, means->dict, 1))
- break;
- lex_match (lexer, T_SLASH);
- }
-
- /* /MISSING subcommand */
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "MISSING"))
- {
- /* If no MISSING subcommand is specified, each combination of a
- dependent variable and categorical variables is handled
- separately. */
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "INCLUDE"))
- {
- /* Use the subcommand "/MISSING=INCLUDE" to include user-missing
- values in the analysis. */
-
- means->ctrl_exclude = MV_SYSTEM;
- means->dep_exclude = MV_SYSTEM;
- }
- else if (lex_match_id (lexer, "DEPENDENT"))
- /* Use the command "/MISSING=DEPENDENT" to include user-missing
- values for the categorical variables, while excluding them for
- the dependent variables.
-
- Cases are dropped only when user-missing values appear in
- dependent variables. User-missing values for categorical
- variables are treated according to their face value.
-
- Cases are ALWAYS dropped when System Missing values appear in
- the categorical variables. */
- {
- means->dep_exclude = MV_ANY;
- means->ctrl_exclude = MV_SYSTEM;
- }
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "DEPENDENT");
- return false;
- }
- }
- else if (lex_match_id (lexer, "CELLS"))
- {
- lex_match (lexer, T_EQUALS);
-
- /* The default values become overwritten */
- means->n_statistics = 0;
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match (lexer, T_ALL))
- {
- means->n_statistics = 0;
- for (int i = 0; i < n_MEANS_STATISTICS; ++i)
- add_statistic (means, i);
- }
- else if (lex_match_id (lexer, "NONE"))
- means->n_statistics = 0;
- else if (lex_match_id (lexer, "DEFAULT"))
- means_set_default_statistics (means);
- else
- {
- const struct cell_spec *cs = match_cell (lexer);
- if (cs)
- add_statistic (means, cs - cell_spec);
- else
- {
- const char *keywords[n_MEANS_STATISTICS];
- for (int i = 0; i < n_MEANS_STATISTICS; ++i)
- keywords[i] = cell_spec[i].keyword;
- lex_error_expecting_array (lexer, keywords,
- n_MEANS_STATISTICS);
- return false;
- }
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "MISSING", "CELLS");
- return false;
- }
- }
- return true;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-
-#include "libpspp/hmap.h"
-#include "libpspp/bt.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-
-#include "count-one-bits.h"
-#include "count-leading-zeros.h"
-
-#include "output/pivot-table.h"
-
-#include "means.h"
-
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-
-/* A "cell" in this procedure represents a distinct value of the
- procedure's categorical variables, and a set of summary statistics
- of all cases which whose categorical variables have that set of
- values. For example, the dataset
-
- v1 v2 cat1 cat2
- 100 202 0 1
- 100 202 0 2
- 100 202 1 0
- 100 202 0 1
-
-
- has three cells in layer 0 and two cells in layer 1 in addition
- to a "grand summary" cell to which all (non-missing) cases
- contribute.
-
- The cells form a n-ary tree structure with the "grand summary"
- cell at the root.
-*/
-struct cell
-{
- struct hmap_node hmap_node; /* Element in hash table. */
- struct bt_node bt_node; /* Element in binary tree */
-
- int n_children;
- struct cell_container *children;
-
- /* The statistics to be calculated for the cell. */
- struct statistic **stat;
-
- /* The parent of this cell, or NULL if this is the root cell. */
- const struct cell *parent_cell;
-
- /* A bit-field variable which indicates which control variables
- are allocated a fixed value (for this cell), and which are
- "wildcards".
-
- A one indicates a fixed value. A zero indicates a wildcard.
- Wildcard values are used to calculate totals and sub-totals.
- */
- unsigned int not_wild;
-
- /* The value(s). */
- union value *values;
-
- /* The variables corresponding to the above values. */
- const struct variable **vars;
-};
-
-/* A structure used to find the union of all values used
- within a layer, and to sort those values. */
-struct instance
-{
- struct hmap_node hmap_node; /* Element in hash table. */
- struct bt_node bt_node; /* Element in binary tree */
-
- /* A unique, consecutive, zero based index identifying this
- instance. */
- int index;
-
- /* The top level value of this instance. */
- union value value;
- const struct variable *var;
-};
-
-
-static void
-destroy_workspace (const struct mtable *mt, struct workspace *ws)
-{
- for (int l = 0; l < mt->n_layers; ++l)
- {
- struct cell_container *instances = ws->instances + l;
- struct instance *inst;
- struct instance *next;
- HMAP_FOR_EACH_SAFE (inst, next, struct instance, hmap_node,
- &instances->map)
- {
- int width = var_get_width (inst->var);
- value_destroy (&inst->value, width);
- free (inst);
- }
- hmap_destroy (&instances->map);
- }
- free (ws->control_idx);
- free (ws->instances);
-}
-
-/* Destroy CELL. */
-static void
-destroy_cell (const struct means *means,
- const struct mtable *mt, struct cell *cell)
-{
- int idx = 0;
- for (int i = 0; i < mt->n_layers; ++i)
- {
- if (0 == ((cell->not_wild >> i) & 0x1))
- continue;
-
- const struct layer *layer = mt->layers[i];
- for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
- {
- struct workspace *ws = mt->ws + cmb;
- const struct variable *var
- = layer->factor_vars[ws->control_idx[i]];
-
- int width = var_get_width (var);
- value_destroy (&cell->values[idx++], width);
- }
- }
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *container = cell->children + i;
- hmap_destroy (&container->map);
- }
-
- for (int v = 0; v < mt->n_dep_vars; ++v)
- {
- for (int s = 0; s < means->n_statistics; ++s)
- {
- stat_destroy *des = cell_spec[means->statistics[s]].sf;
- des (cell->stat[s + v * means->n_statistics]);
- }
- }
- free (cell->stat);
-
- free (cell->children);
- free (cell->values);
- free (cell->vars);
- free (cell);
-}
-
-
-/* Walk the tree in postorder starting from CELL and destroy all the
- cells. */
-static void
-means_destroy_cells (const struct means *means, struct cell *cell,
- const struct mtable *table)
-{
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *container = cell->children + i;
- struct cell *sub_cell;
- struct cell *next;
- HMAP_FOR_EACH_SAFE (sub_cell, next, struct cell, hmap_node,
- &container->map)
- {
- means_destroy_cells (means, sub_cell, table);
- }
- }
-
- destroy_cell (means, table, cell);
-}
-
-#if 0
-
-static void
-dump_cell (const struct cell *cell, const struct mtable *mt, int level)
-{
- for (int l = 0; l < level; ++l)
- putchar (' ');
- printf ("%p: ", cell);
- for (int i = 0; i < mt->n_layers; ++i)
- {
- putchar (((cell->not_wild >> i) & 0x1) ? 'w' : '.');
- }
- printf (" - ");
- int x = 0;
- for (int i = 0; i < mt->n_layers; ++i)
- {
- if ((cell->not_wild >> i) & 0x1)
- {
- printf ("%s: ", var_get_name (cell->vars[x]));
- printf ("%g ", cell->values[x++].f);
- }
- else
- printf ("x ");
- }
- stat_get *sg = cell_spec[MEANS_N].sd;
- printf ("--- S1: %g", sg (cell->stat[0]));
-
- printf ("--- N Children: %d", cell->n_children);
- // printf ("--- Level: %d", level);
- printf ("--- Parent: %p", cell->parent_cell);
- printf ("\n");
-}
-
-static void
-dump_indeces (const size_t *indexes, int n)
-{
- for (int i = 0; i < n; ++i)
- {
- printf ("%ld; ", indexes[i]);
- }
- printf ("\n");
-}
-
-/* Dump the tree in pre-order. */
-static void
-dump_tree (const struct cell *cell, const struct mtable *table,
- int level, const struct cell *parent)
-{
- assert (cell->parent_cell == parent);
- dump_cell (cell, table, level);
-
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *container = cell->children + i;
- struct cell *sub_cell;
- BT_FOR_EACH (sub_cell, struct cell, bt_node, &container->bt)
- {
- dump_tree (sub_cell, table, level + 1, cell);
- }
- }
-}
-
-#endif
-
-/* Generate a hash based on the values of the N variables in
- the array VARS which are taken from the case C. */
-static unsigned int
-generate_hash (const struct mtable *mt,
- const struct ccase *c,
- unsigned int not_wild,
- const struct workspace *ws)
-{
- unsigned int hash = 0;
- for (int i = 0; i < mt->n_layers; ++i)
- {
- if (0 == ((not_wild >> i) & 0x1))
- continue;
-
- const struct layer *layer = mt->layers[i];
- const struct variable *var = layer->factor_vars[ws->control_idx[i]];
- const union value *vv = case_data (c, var);
- int width = var_get_width (var);
- hash = hash_int (i, hash);
- hash = value_hash (vv, width, hash);
- }
-
- return hash;
-}
-
-/* Create a cell based on the N variables in the array VARS,
- which are indeces into the case C.
- The caller is responsible for destroying this cell when
- no longer needed. */
-static struct cell *
-generate_cell (const struct means *means,
- const struct mtable *mt,
- const struct ccase *c,
- unsigned int not_wild,
- const struct cell *pcell,
- const struct workspace *ws)
-{
- int n_vars = count_one_bits (not_wild);
- struct cell *cell = XZALLOC (struct cell);
- cell->values = xcalloc (n_vars, sizeof *cell->values);
- cell->vars = xcalloc (n_vars, sizeof *cell->vars);
- cell->not_wild = not_wild;
-
- cell->parent_cell = pcell;
- cell->n_children = mt->n_layers -
- (sizeof (cell->not_wild) * CHAR_BIT) +
- count_leading_zeros (cell->not_wild);
-
- int idx = 0;
- for (int i = 0; i < mt->n_layers; ++i)
- {
- if (0 == ((not_wild >> i) & 0x1))
- continue;
-
- const struct layer *layer = mt->layers[i];
- const struct variable *var = layer->factor_vars[ws->control_idx[i]];
- const union value *vv = case_data (c, var);
- int width = var_get_width (var);
- cell->vars[idx] = var;
- value_clone (&cell->values[idx++], vv, width);
- }
- assert (idx == n_vars);
-
- cell->children = xcalloc (cell->n_children, sizeof *cell->children);
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *container = cell->children + i;
- hmap_init (&container->map);
- }
-
- cell->stat = xcalloc (means->n_statistics * mt->n_dep_vars, sizeof *cell->stat);
- for (int v = 0; v < mt->n_dep_vars; ++v)
- {
- for (int stat = 0; stat < means->n_statistics; ++stat)
- {
- stat_create *sc = cell_spec[means->statistics[stat]].sc;
-
- cell->stat[stat + v * means->n_statistics] = sc (means->pool);
- }
- }
- return cell;
-}
-
-
-/* If a cell based on the N variables in the array VARS,
- which are indeces into the case C and whose hash is HASH,
- exists in HMAP, then return that cell.
- Otherwise, return NULL. */
-static struct cell *
-lookup_cell (const struct mtable *mt,
- struct hmap *hmap, unsigned int hash,
- const struct ccase *c,
- unsigned int not_wild,
- const struct workspace *ws)
-{
- struct cell *cell = NULL;
- HMAP_FOR_EACH_WITH_HASH (cell, struct cell, hmap_node, hash, hmap)
- {
- bool match = true;
- int idx = 0;
- if (cell->not_wild != not_wild)
- continue;
- for (int i = 0; i < mt->n_layers; ++i)
- {
- if (0 == ((cell->not_wild >> i) & 0x1))
- continue;
-
- const struct layer *layer = mt->layers[i];
- const struct variable *var = layer->factor_vars[ws->control_idx[i]];
- const union value *vv = case_data (c, var);
- int width = var_get_width (var);
- assert (var == cell->vars[idx]);
- if (!value_equal (vv, &cell->values[idx++], width))
- {
- match = false;
- break;
- }
- }
- if (match)
- return cell;
- }
- return NULL;
-}
-
-
-/* A comparison function used to sort cells in a binary tree.
- Only the innermost value needs to be compared, because no
- two cells with similar outer values will appear in the same
- tree/map. */
-static int
-cell_compare_3way (const struct bt_node *a,
- const struct bt_node *b,
- const void *aux UNUSED)
-{
- const struct cell *fa = BT_DATA (a, struct cell, bt_node);
- const struct cell *fb = BT_DATA (b, struct cell, bt_node);
-
- assert (fa->not_wild == fb->not_wild);
- int vidx = count_one_bits (fa->not_wild) - 1;
- assert (fa->vars[vidx] == fb->vars[vidx]);
-
- return value_compare_3way (&fa->values[vidx],
- &fb->values[vidx],
- var_get_width (fa->vars[vidx]));
-}
-
-/* A comparison function used to sort cells in a binary tree. */
-static int
-compare_instance_3way (const struct bt_node *a,
- const struct bt_node *b,
- const void *aux UNUSED)
-{
- const struct instance *fa = BT_DATA (a, struct instance, bt_node);
- const struct instance *fb = BT_DATA (b, struct instance, bt_node);
-
- assert (fa->var == fb->var);
-
- return value_compare_3way (&fa->value,
- &fb->value,
- var_get_width (fa->var));
-}
-
-
-static void arrange_cells (struct workspace *ws,
- struct cell *cell, const struct mtable *table);
-
-
-/* Iterate CONTAINER's map inserting a copy of its elements into
- CONTAINER's binary tree. Also, for each layer in TABLE, create
- an instance container, containing the union of all elements in
- CONTAINER. */
-static void
-arrange_cell (struct workspace *ws, struct cell_container *container,
- const struct mtable *mt)
-{
- struct bt *bt = &container->bt;
- struct hmap *map = &container->map;
- bt_init (bt, cell_compare_3way, NULL);
-
- struct cell *cell;
- HMAP_FOR_EACH (cell, struct cell, hmap_node, map)
- {
- bt_insert (bt, &cell->bt_node);
-
- int idx = 0;
- for (int i = 0; i < mt->n_layers; ++i)
- {
- if (0 == ((cell->not_wild >> i) & 0x1))
- continue;
-
- struct cell_container *instances = ws->instances + i;
- const struct variable *var = cell->vars[idx];
- int width = var_get_width (var);
- unsigned int hash
- = value_hash (&cell->values[idx], width, 0);
-
- struct instance *inst = NULL;
- struct instance *next = NULL;
- HMAP_FOR_EACH_WITH_HASH_SAFE (inst, next, struct instance,
- hmap_node,
- hash, &instances->map)
- {
- assert (cell->vars[idx] == var);
- if (value_equal (&inst->value,
- &cell->values[idx],
- width))
- {
- break;
- }
- }
-
- if (!inst)
- {
- inst = xzalloc (sizeof *inst);
- inst->index = -1;
- inst->var = var;
- value_clone (&inst->value, &cell->values[idx],
- width);
- hmap_insert (&instances->map, &inst->hmap_node, hash);
- }
-
- idx++;
- }
-
- arrange_cells (ws, cell, mt);
- }
-}
-
-/* Arrange the children and then all the subtotals. */
-static void
-arrange_cells (struct workspace *ws, struct cell *cell,
- const struct mtable *table)
-{
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *container = cell->children + i;
- arrange_cell (ws, container, table);
- }
-}
-
-
-\f
-
-/* If the top level value in CELL, has an instance in the L_IDX'th layer,
- then return that instance. Otherwise return NULL. */
-static const struct instance *
-lookup_instance (const struct mtable *mt, const struct workspace *ws,
- int l_idx, const struct cell *cell)
-{
- const struct layer *layer = mt->layers[l_idx];
- int n_vals = count_one_bits (cell->not_wild);
- const struct variable *var = layer->factor_vars[ws->control_idx[l_idx]];
- const union value *val = cell->values + n_vals - 1;
- int width = var_get_width (var);
- unsigned int hash = value_hash (val, width, 0);
- const struct cell_container *instances = ws->instances + l_idx;
- struct instance *inst = NULL;
- struct instance *next;
- HMAP_FOR_EACH_WITH_HASH_SAFE (inst, next,
- struct instance, hmap_node,
- hash, &instances->map)
- {
- if (value_equal (val, &inst->value, width))
- break;
- }
- return inst;
-}
-
-/* Enter the values into PT. */
-static void
-populate_table (const struct means *means, const struct mtable *mt,
- const struct workspace *ws,
- const struct cell *cell,
- struct pivot_table *pt)
-{
- size_t *indexes = XCALLOC (pt->n_dimensions, size_t);
- for (int v = 0; v < mt->n_dep_vars; ++v)
- {
- for (int s = 0; s < means->n_statistics; ++s)
- {
- int i = 0;
- if (mt->n_dep_vars > 1)
- indexes[i++] = v;
- indexes[i++] = s;
- int stat = means->statistics[s];
- stat_get *sg = cell_spec[stat].sd;
- {
- const struct cell *pc = cell;
- for (; i < pt->n_dimensions; ++i)
- {
- int l_idx = pt->n_dimensions - i - 1;
- const struct cell_container *instances = ws->instances + l_idx;
- if (0 == (cell->not_wild >> l_idx & 0x1U))
- {
- indexes [i] = hmap_count (&instances->map);
- }
- else
- {
- assert (pc);
- const struct instance *inst
- = lookup_instance (mt, ws, l_idx, pc);
- assert (inst);
- indexes [i] = inst->index;
- pc = pc->parent_cell;
- }
- }
- }
-
- int idx = s + v * means->n_statistics;
- struct pivot_value *pv
- = pivot_value_new_number (sg (cell->stat[idx]));
- if (NULL == cell_spec[stat].rc)
- {
- const struct variable *dv = mt->dep_vars[v];
- pv->numeric.format = * var_get_print_format (dv);
- }
- pivot_table_put (pt, indexes, pt->n_dimensions, pv);
- }
- }
- free (indexes);
-
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *container = cell->children + i;
- struct cell *child = NULL;
- BT_FOR_EACH (child, struct cell, bt_node, &container->bt)
- {
- populate_table (means, mt, ws, child, pt);
- }
- }
-}
-
-static void
-create_table_structure (const struct mtable *mt, struct pivot_table *pt,
- const struct workspace *ws)
-{
- int * lindexes = ws->control_idx;
- /* The inner layers are situated rightmost in the table.
- So this iteration is in reverse order. */
- for (int l = mt->n_layers - 1; l >= 0; --l)
- {
- const struct layer *layer = mt->layers[l];
- const struct cell_container *instances = ws->instances + l;
- const struct variable *var = layer->factor_vars[lindexes[l]];
- struct pivot_dimension *dim_layer
- = pivot_dimension_create (pt, PIVOT_AXIS_ROW,
- var_to_string (var));
- dim_layer->root->show_label = true;
-
- /* Place the values of the control variables as table headings. */
- {
- struct instance *inst = NULL;
- BT_FOR_EACH (inst, struct instance, bt_node, &instances->bt)
- {
- struct substring space = SS_LITERAL_INITIALIZER ("\t ");
- struct string str;
- ds_init_empty (&str);
- var_append_value_name (var,
- &inst->value,
- &str);
-
- ds_ltrim (&str, space);
-
- pivot_category_create_leaf (dim_layer->root,
- pivot_value_new_text (ds_cstr (&str)));
-
- ds_destroy (&str);
- }
- }
-
- pivot_category_create_leaf (dim_layer->root,
- pivot_value_new_text ("Total"));
- }
-}
-
-/* Initialise C_DES with a string describing the control variable
- relating to MT, LINDEXES. */
-static void
-layers_to_string (const struct mtable *mt, const int *lindexes,
- struct string *c_des)
-{
- for (int l = 0; l < mt->n_layers; ++l)
- {
- const struct layer *layer = mt->layers[l];
- const struct variable *ctrl_var = layer->factor_vars[lindexes[l]];
- if (l > 0)
- ds_put_cstr (c_des, " * ");
- ds_put_cstr (c_des, var_get_name (ctrl_var));
- }
-}
-
-static void
-populate_case_processing_summary (struct pivot_category *pc,
- const struct mtable *mt,
- const int *lindexes)
-{
- struct string ds;
- ds_init_empty (&ds);
- int l = 0;
- for (l = 0; l < mt->n_layers; ++l)
- {
- const struct layer *layer = mt->layers[l];
- const struct variable *ctrl_var = layer->factor_vars[lindexes[l]];
- if (l > 0)
- ds_put_cstr (&ds, " * ");
- ds_put_cstr (&ds, var_get_name (ctrl_var));
- }
- for (int dv = 0; dv < mt->n_dep_vars; ++dv)
- {
- struct string dss;
- ds_init_empty (&dss);
- ds_put_cstr (&dss, var_get_name (mt->dep_vars[dv]));
- if (mt->n_layers > 0)
- {
- ds_put_cstr (&dss, " * ");
- ds_put_substring (&dss, ds.ss);
- }
- pivot_category_create_leaf (pc,
- pivot_value_new_text (ds_cstr (&dss)));
- ds_destroy (&dss);
- }
-
- ds_destroy (&ds);
-}
-
-/* Create the "Case Processing Summary" table. */
-static void
-means_case_processing_summary (const struct mtable *mt)
-{
- struct pivot_table *pt = pivot_table_create (N_("Case Processing Summary"));
-
- struct pivot_dimension *dim_cases =
- pivot_dimension_create (pt, PIVOT_AXIS_COLUMN, N_("Cases"));
- dim_cases->root->show_label = true;
-
- struct pivot_category *cats[3];
- cats[0] = pivot_category_create_group (dim_cases->root,
- N_("Included"), NULL);
- cats[1] = pivot_category_create_group (dim_cases->root,
- N_("Excluded"), NULL);
- cats[2] = pivot_category_create_group (dim_cases->root,
- N_("Total"), NULL);
- for (int i = 0; i < 3; ++i)
- {
- pivot_category_create_leaf_rc (cats[i],
- pivot_value_new_text (N_("N")),
- PIVOT_RC_COUNT);
- pivot_category_create_leaf_rc (cats[i],
- pivot_value_new_text (N_("Percent")),
- PIVOT_RC_PERCENT);
- }
-
- struct pivot_dimension *rows =
- pivot_dimension_create (pt, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
- {
- const struct workspace *ws = mt->ws + cmb;
- populate_case_processing_summary (rows->root, mt, ws->control_idx);
- for (int dv = 0; dv < mt->n_dep_vars; ++dv)
- {
- int idx = cmb * mt->n_dep_vars + dv;
- const struct summary *summ = mt->summ + idx;
- double n_included = summ->n_total - summ->n_missing;
- pivot_table_put2 (pt, 5, idx,
- pivot_value_new_number (100.0 * summ->n_total / summ->n_total));
- pivot_table_put2 (pt, 4, idx,
- pivot_value_new_number (summ->n_total));
-
- pivot_table_put2 (pt, 3, idx,
- pivot_value_new_number (100.0 * summ->n_missing / summ->n_total));
- pivot_table_put2 (pt, 2, idx,
- pivot_value_new_number (summ->n_missing));
-
- pivot_table_put2 (pt, 1, idx,
- pivot_value_new_number (100.0 * n_included / summ->n_total));
- pivot_table_put2 (pt, 0, idx,
- pivot_value_new_number (n_included));
- }
- }
-
- pivot_table_submit (pt);
-}
-
-static void
-means_shipout_single (const struct mtable *mt, const struct means *means,
- const struct workspace *ws)
-{
- struct pivot_table *pt = pivot_table_create (N_("Report"));
-
- struct pivot_dimension *dim_cells =
- pivot_dimension_create (pt, PIVOT_AXIS_COLUMN, N_("Statistics"));
-
- /* Set the statistics headings, eg "Mean", "Std. Dev" etc. */
- for (int i = 0; i < means->n_statistics; ++i)
- {
- const struct cell_spec *cs = cell_spec + means->statistics[i];
- pivot_category_create_leaf_rc
- (dim_cells->root,
- pivot_value_new_text (gettext (cs->title)), cs->rc);
- }
-
- create_table_structure (mt, pt, ws);
- populate_table (means, mt, ws, ws->root_cell, pt);
- pivot_table_submit (pt);
-}
-
-
-static void
-means_shipout_multivar (const struct mtable *mt, const struct means *means,
- const struct workspace *ws)
-{
- struct string dss;
- ds_init_empty (&dss);
- for (int dv = 0; dv < mt->n_dep_vars; ++dv)
- {
- if (dv > 0)
- ds_put_cstr (&dss, " * ");
- ds_put_cstr (&dss, var_get_name (mt->dep_vars[dv]));
- }
-
- for (int l = 0; l < mt->n_layers; ++l)
- {
- ds_put_cstr (&dss, " * ");
- const struct layer *layer = mt->layers[l];
- const struct variable *var = layer->factor_vars[ws->control_idx[l]];
- ds_put_cstr (&dss, var_get_name (var));
- }
-
- struct pivot_table *pt = pivot_table_create (ds_cstr (&dss));
- ds_destroy (&dss);
-
- struct pivot_dimension *dim_cells =
- pivot_dimension_create (pt, PIVOT_AXIS_COLUMN, N_("Variables"));
-
- for (int i = 0; i < mt->n_dep_vars; ++i)
- {
- pivot_category_create_leaf
- (dim_cells->root,
- pivot_value_new_variable (mt->dep_vars[i]));
- }
-
- struct pivot_dimension *dim_stats
- = pivot_dimension_create (pt, PIVOT_AXIS_ROW,
- N_ ("Statistics"));
- dim_stats->root->show_label = false;
-
- for (int i = 0; i < means->n_statistics; ++i)
- {
- const struct cell_spec *cs = cell_spec + means->statistics[i];
- pivot_category_create_leaf_rc
- (dim_stats->root,
- pivot_value_new_text (gettext (cs->title)), cs->rc);
- }
-
- create_table_structure (mt, pt, ws);
- populate_table (means, mt, ws, ws->root_cell, pt);
- pivot_table_submit (pt);
-}
-
-static void
-means_shipout (const struct mtable *mt, const struct means *means)
-{
- for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
- {
- const struct workspace *ws = mt->ws + cmb;
- if (ws->root_cell == NULL)
- {
- struct string des;
- ds_init_empty (&des);
- layers_to_string (mt, ws->control_idx, &des);
- msg (MW, _("The table \"%s\" has no non-empty control variables."
- " No result for this table will be displayed."),
- ds_cstr (&des));
- ds_destroy (&des);
- continue;
- }
- if (mt->n_dep_vars > 1)
- means_shipout_multivar (mt, means, ws);
- else
- means_shipout_single (mt, means, ws);
- }
-}
-
-
-\f
-
-static bool
-control_var_missing (const struct means *means,
- const struct mtable *mt,
- unsigned int not_wild UNUSED,
- const struct ccase *c,
- const struct workspace *ws)
-{
- bool miss = false;
- for (int l = 0; l < mt->n_layers; ++l)
- {
- /* if (0 == ((not_wild >> l) & 0x1)) */
- /* { */
- /* continue; */
- /* } */
-
- const struct layer *layer = mt->layers[l];
- const struct variable *var = layer->factor_vars[ws->control_idx[l]];
- const union value *vv = case_data (c, var);
-
- miss = (var_is_value_missing (var, vv) & means->ctrl_exclude) != 0;
- if (miss)
- break;
- }
-
- return miss;
-}
-
-/* Lookup the set of control variables described by MT, C and NOT_WILD,
- in the hash table MAP. If there is no such entry, then create a
- cell with these paremeters and add is to MAP.
- If the generated cell has childen, repeat for all the children.
- Returns the root cell.
-*/
-static struct cell *
-service_cell_map (const struct means *means, const struct mtable *mt,
- const struct ccase *c,
- unsigned int not_wild,
- struct hmap *map,
- const struct cell *pcell,
- int level,
- const struct workspace *ws)
-{
- struct cell *cell = NULL;
- if (map)
- {
- if (!control_var_missing (means, mt, not_wild, c, ws))
- {
- /* Lookup this set of values in the cell's hash table. */
- unsigned int hash = generate_hash (mt, c, not_wild, ws);
- cell = lookup_cell (mt, map, hash, c, not_wild, ws);
-
- /* If it has not been seen before, then create a new
- subcell, with this set of values, and insert it
- into the table. */
- if (cell == NULL)
- {
- cell = generate_cell (means, mt, c, not_wild, pcell, ws);
- hmap_insert (map, &cell->hmap_node, hash);
- }
- }
- }
- else
- {
- /* This condition should only happen in the root node case. */
- cell = ws->root_cell;
- if (cell == NULL &&
- !control_var_missing (means, mt, not_wild, c, ws))
- cell = generate_cell (means, mt, c, not_wild, pcell, ws);
- }
-
- if (cell)
- {
- /* Here is where the business really happens! After
- testing for missing values, the cell's statistics
- are accumulated. */
- if (!control_var_missing (means, mt, not_wild, c, ws))
- {
- for (int v = 0; v < mt->n_dep_vars; ++v)
- {
- const struct variable *dep_var = mt->dep_vars[v];
- const union value *vv = case_data (c, dep_var);
- if (var_is_value_missing (dep_var, vv) & means->dep_exclude)
- continue;
-
- for (int stat = 0; stat < means->n_statistics; ++stat)
- {
- const double weight = dict_get_case_weight (means->dict, c,
- NULL);
- stat_update *su = cell_spec[means->statistics[stat]].su;
- su (cell->stat[stat + v * means->n_statistics], weight,
- case_num (c, dep_var));
- }
- }
- }
-
- /* Recurse into all the children (if there are any). */
- for (int i = 0; i < cell->n_children; ++i)
- {
- struct cell_container *cc = cell->children + i;
- service_cell_map (means, mt, c,
- not_wild | (0x1U << (i + level)),
- &cc->map, cell, level + i + 1, ws);
- }
- }
-
- return cell;
-}
-
-/* Do all the necessary preparation and pre-calculation that
- needs to be done before iterating the data. */
-static void
-prepare_means (struct means *cmd)
-{
- for (int t = 0; t < cmd->n_tables; ++t)
- {
- struct mtable *mt = cmd->table + t;
-
- for (int i = 0; i < mt->n_combinations; ++i)
- {
- struct workspace *ws = mt->ws + i;
- ws->root_cell = NULL;
- ws->control_idx = xcalloc (mt->n_layers, sizeof *ws->control_idx);
- ws->instances = xcalloc (mt->n_layers, sizeof *ws->instances);
- int cmb = i;
- for (int l = mt->n_layers - 1; l >= 0; --l)
- {
- struct cell_container *instances = ws->instances + l;
- const struct layer *layer = mt->layers[l];
- ws->control_idx[l] = cmb % layer->n_factor_vars;
- cmb /= layer->n_factor_vars;
- hmap_init (&instances->map);
- }
- }
- }
-}
-
-
-/* Do all the necessary calculations that occur AFTER iterating
- the data. */
-static void
-post_means (struct means *cmd)
-{
- for (int t = 0; t < cmd->n_tables; ++t)
- {
- struct mtable *mt = cmd->table + t;
- for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
- {
- struct workspace *ws = mt->ws + cmb;
- if (ws->root_cell == NULL)
- continue;
- arrange_cells (ws, ws->root_cell, mt);
- /* The root cell should have no parent. */
- assert (ws->root_cell->parent_cell == 0);
-
- for (int l = 0; l < mt->n_layers; ++l)
- {
- struct cell_container *instances = ws->instances + l;
- bt_init (&instances->bt, compare_instance_3way, NULL);
-
- /* Iterate the instance hash table, and insert each instance
- into the binary tree BT. */
- struct instance *inst;
- HMAP_FOR_EACH (inst, struct instance, hmap_node,
- &instances->map)
- {
- bt_insert (&instances->bt, &inst->bt_node);
- }
-
- /* Iterate the binary tree (in order) and assign the index
- member accordingly. */
- int index = 0;
- BT_FOR_EACH (inst, struct instance, bt_node, &instances->bt)
- {
- inst->index = index++;
- }
- }
- }
- }
-}
-
-
-/* Update the summary information (the missings and the totals). */
-static void
-update_summaries (const struct means *means, struct mtable *mt,
- const struct ccase *c, double weight)
-{
- for (int dv = 0; dv < mt->n_dep_vars; ++dv)
- {
- for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
- {
- struct workspace *ws = mt->ws + cmb;
- struct summary *summ = mt->summ
- + cmb * mt->n_dep_vars + dv;
-
- summ->n_total += weight;
- const struct variable *var = mt->dep_vars[dv];
- const union value *vv = case_data (c, var);
- /* First check if the dependent variable is missing. */
- if (var_is_value_missing (var, vv) & means->dep_exclude)
- summ->n_missing += weight;
- /* If the dep var is not missing, then check each
- control variable. */
- else
- for (int l = 0; l < mt->n_layers; ++l)
- {
- const struct layer *layer = mt->layers [l];
- const struct variable *var
- = layer->factor_vars[ws->control_idx[l]];
- const union value *vv = case_data (c, var);
- if (var_is_value_missing (var, vv) & means->ctrl_exclude)
- {
- summ->n_missing += weight;
- break;
- }
- }
- }
- }
-}
-
-
-void
-run_means (struct means *cmd, struct casereader *input,
- const struct dataset *ds UNUSED)
-{
- struct ccase *c = NULL;
- struct casereader *reader;
-
- prepare_means (cmd);
-
- for (reader = input;
- (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- const double weight
- = dict_get_case_weight (cmd->dict, c, NULL);
- for (int t = 0; t < cmd->n_tables; ++t)
- {
- struct mtable *mt = cmd->table + t;
- update_summaries (cmd, mt, c, weight);
-
- for (int cmb = 0; cmb < mt->n_combinations; ++cmb)
- {
- struct workspace *ws = mt->ws + cmb;
-
- ws->root_cell = service_cell_map (cmd, mt, c,
- 0U, NULL, NULL, 0, ws);
- }
- }
- }
- casereader_destroy (reader);
-
- post_means (cmd);
-}
-
-int
-cmd_means (struct lexer *lexer, struct dataset *ds)
-{
- struct means means = {
- .pool = pool_create (),
- .ctrl_exclude = MV_ANY,
- .dep_exclude = MV_ANY,
- .dict = dataset_dict (ds),
- };
- means_set_default_statistics (&means);
-
- if (!means_parse (lexer, &means))
- goto error;
-
- /* Calculate some constant data for each table. */
- for (int t = 0; t < means.n_tables; ++t)
- {
- struct mtable *mt = means.table + t;
- mt->n_combinations = 1;
- for (int l = 0; l < mt->n_layers; ++l)
- mt->n_combinations *= mt->layers[l]->n_factor_vars;
- }
-
- struct casegrouper *grouper
- = casegrouper_create_splits (proc_open (ds), means.dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- /* Allocate the workspaces. */
- for (int t = 0; t < means.n_tables; ++t)
- {
- struct mtable *mt = means.table + t;
- mt->summ = xcalloc (mt->n_combinations * mt->n_dep_vars,
- sizeof *mt->summ);
- mt->ws = xcalloc (mt->n_combinations, sizeof *mt->ws);
- }
- run_means (&means, group, ds);
- for (int t = 0; t < means.n_tables; ++t)
- {
- const struct mtable *mt = means.table + t;
-
- means_case_processing_summary (mt);
- means_shipout (mt, &means);
-
- for (int i = 0; i < mt->n_combinations; ++i)
- {
- struct workspace *ws = mt->ws + i;
- if (ws->root_cell)
- means_destroy_cells (&means, ws->root_cell, mt);
- }
- }
-
- /* Destroy the workspaces. */
- for (int t = 0; t < means.n_tables; ++t)
- {
- struct mtable *mt = means.table + t;
- free (mt->summ);
- for (int i = 0; i < mt->n_combinations; ++i)
- {
- struct workspace *ws = mt->ws + i;
- destroy_workspace (mt, ws);
- }
- free (mt->ws);
- }
- }
-
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
- if (!ok)
- goto error;
-
- pool_destroy (means.pool);
- return CMD_SUCCESS;
-
- error:
- pool_destroy (means.pool);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011, 2012, 2013, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef MEANS_H
-#define MEANS_H
-
-#include "libpspp/hmap.h"
-#include "libpspp/bt.h"
-#include "libpspp/compiler.h"
-
-struct casereader;
-struct dataset;
-struct lexer;
-
-struct cell_container
-{
- /* A hash table containing the cells. The table is indexed by a hash
- based on the cell's categorical value. */
- struct hmap map;
-
- /* A binary tree containing the cells. This is
- used to sort the elements in order of their categorical
- values. */
- struct bt bt;
-};
-
-
-
-struct layer
-{
- size_t n_factor_vars;
- const struct variable **factor_vars;
-};
-
-
-struct statistic;
-
-typedef struct statistic *stat_create (struct pool *pool);
-typedef void stat_update (struct statistic *stat, double w, double x);
-typedef double stat_get (const struct statistic *);
-typedef void stat_destroy (struct statistic *);
-
-
-struct cell_spec
-{
- /* Printable title for output */
- const char *title;
-
- /* Keyword for syntax */
- const char *keyword;
-
- /* The result class for the datum. */
- const char *rc;
-
- stat_create *sc;
- stat_update *su;
- stat_get *sd;
- stat_destroy *sf;
-};
-
-struct summary
-{
- double n_total;
- double n_missing;
-};
-
-/* Intermediate data per table. */
-struct workspace
-{
- /* An array of n_layers integers which are used
- to permute access into the factor_vars of each layer. */
- int *control_idx;
-
- /* An array of n_layers cell_containers which hold the union
- of instances used respectively by each layer. */
- struct cell_container *instances;
-
- struct cell *root_cell;
-};
-
-/* The thing parsed after TABLES= */
-struct mtable
-{
- size_t n_dep_vars;
- const struct variable **dep_vars;
-
- struct layer **layers;
- int n_layers;
-
- int n_combinations;
-
- /* An array of n_combinations workspaces. */
- struct workspace *ws;
-
- /* An array of n_combinations * n_dep_vars summaries.
- These are displayed in the Case Processing
- Summary box. */
- struct summary *summ;
-};
-
-/* A structure created by the parser. Contains the definition of the
- what the procedure should calculate. */
-struct means
-{
- const struct dictionary *dict;
-
- /* The "tables" (ie, a definition of how the data should
- be broken down). */
- struct mtable *table;
- size_t n_tables;
-
- /* Missing value class for categorical variables. */
- enum mv_class ctrl_exclude;
-
- /* Missing value class for dependent variables */
- enum mv_class dep_exclude;
-
- /* The statistics to be calculated for each cell. */
- int *statistics;
- int n_statistics;
- size_t allocated_statistics;
-
- /* Pool on which cell functions may allocate data. */
- struct pool *pool;
-};
-
-
-
-#define n_MEANS_STATISTICS 17
-extern const struct cell_spec cell_spec[n_MEANS_STATISTICS];
-
-/* This enum must be consistent with the array cell_spec (in means-calc.c).
- A bitfield instead of enums would in my opinion be
- more elegent. However we want the order of the specified
- statistics to be retained in the output. */
-enum
- {
- MEANS_MEAN = 0,
- MEANS_N,
- MEANS_STDDEV
- };
-
-void run_means (struct means *, struct casereader *, const struct dataset *);
-bool means_parse (struct lexer *, struct means *);
-void means_set_default_statistics (struct means *);
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include <config.h>
-#include "median.h"
-
-#include <gsl/gsl_cdf.h>
-
-#include "data/format.h"
-
-
-#include "data/variable.h"
-#include "data/case.h"
-#include "data/dictionary.h"
-#include "data/dataset.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/subcase.h"
-#include "data/value.h"
-
-#include "math/percentiles.h"
-#include "math/sort.h"
-
-#include "libpspp/cast.h"
-#include "libpspp/hmap.h"
-#include "libpspp/array.h"
-#include "libpspp/str.h"
-#include "libpspp/misc.h"
-
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct val_node
-{
- struct hmap_node node;
- union value val;
- casenumber le;
- casenumber gt;
-};
-
-struct results
-{
- const struct variable *var;
- struct val_node **sorted_array;
- double n;
- double median;
- double chisq;
-};
-
-
-
-static int
-val_node_cmp_3way (const void *a_, const void *b_, const void *aux)
-{
- const struct variable *indep_var = aux;
- const struct val_node *const *a = a_;
- const struct val_node *const *b = b_;
-
- return value_compare_3way (&(*a)->val, &(*b)->val, var_get_width (indep_var));
-}
-
-static void
-show_frequencies (const struct n_sample_test *nst, const struct results *results, int n_vals, const struct dictionary *);
-
-static void
-show_test_statistics (const struct n_sample_test *nst, const struct results *results, int, const struct dictionary *);
-
-
-static struct val_node *
-find_value (const struct hmap *map, const union value *val,
- const struct variable *var)
-{
- struct val_node *foo = NULL;
- size_t hash = value_hash (val, var_get_width (var), 0);
- HMAP_FOR_EACH_WITH_HASH (foo, struct val_node, node, hash, map)
- if (value_equal (val, &foo->val, var_get_width (var)))
- break;
-
- return foo;
-}
-
-void
-median_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- const struct dictionary *dict = dataset_dict (ds);
- const struct variable *wvar = dict_get_weight (dict);
- bool warn = true;
- int v;
- const struct median_test *mt = UP_CAST (test, const struct median_test,
- parent.parent);
-
- const struct n_sample_test *nst = UP_CAST (test, const struct n_sample_test,
- parent);
-
- const bool n_sample_test = (value_compare_3way (&nst->val2, &nst->val1,
- var_get_width (nst->indep_var)) > 0);
-
- struct results *results = XCALLOC (nst->n_vars, struct results);
- int n_vals = 0;
- for (v = 0; v < nst->n_vars; ++v)
- {
- double count = 0;
- double cc = 0;
- double median = mt->median;
- const struct variable *var = nst->vars[v];
- struct ccase *c;
- struct hmap map = HMAP_INITIALIZER (map);
- struct casereader *r = casereader_clone (input);
-
-
-
- if (n_sample_test == false)
- {
- struct val_node *vn = XZALLOC (struct val_node);
- value_clone (&vn->val, &nst->val1, var_get_width (nst->indep_var));
- hmap_insert (&map, &vn->node, value_hash (&nst->val1,
- var_get_width (nst->indep_var), 0));
-
- vn = xzalloc (sizeof *vn);
- value_clone (&vn->val, &nst->val2, var_get_width (nst->indep_var));
- hmap_insert (&map, &vn->node, value_hash (&nst->val2,
- var_get_width (nst->indep_var), 0));
- }
-
- if (median == SYSMIS)
- {
- struct percentile *ptl;
- struct order_stats *os;
-
- struct casereader *rr;
- struct subcase sc;
- struct casewriter *writer;
- subcase_init_var (&sc, var, SC_ASCEND);
- rr = casereader_clone (r);
- writer = sort_create_writer (&sc, casereader_get_proto (rr));
-
- for (; (c = casereader_read (rr)) != NULL;)
- {
- if (var_is_value_missing (var, case_data (c, var)) & exclude)
- {
- case_unref (c);
- continue;
- }
-
- cc += dict_get_case_weight (dict, c, &warn);
- casewriter_write (writer, c);
- }
- subcase_uninit (&sc);
- casereader_destroy (rr);
-
- rr = casewriter_make_reader (writer);
-
- ptl = percentile_create (0.5, cc);
- os = &ptl->parent;
-
- order_stats_accumulate (&os, 1,
- rr,
- wvar,
- var,
- exclude);
-
- median = percentile_calculate (ptl, PC_HAVERAGE);
- statistic_destroy (&ptl->parent.parent);
- }
-
- results[v].median = median;
-
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- struct val_node *vn ;
- const double weight = dict_get_case_weight (dict, c, &warn);
- const union value *val = case_data (c, var);
- const union value *indep_val = case_data (c, nst->indep_var);
-
- if (var_is_value_missing (var, case_data (c, var)) & exclude)
- {
- continue;
- }
-
- if (n_sample_test)
- {
- int width = var_get_width (nst->indep_var);
- /* Ignore out of range values */
- if (
- value_compare_3way (indep_val, &nst->val1, width) < 0
- ||
- value_compare_3way (indep_val, &nst->val2, width) > 0
- )
- {
- continue;
- }
- }
-
- vn = find_value (&map, indep_val, nst->indep_var);
- if (vn == NULL)
- {
- if (n_sample_test == true)
- {
- int width = var_get_width (nst->indep_var);
- vn = xzalloc (sizeof *vn);
- value_clone (&vn->val, indep_val, width);
-
- hmap_insert (&map, &vn->node, value_hash (indep_val, width, 0));
- }
- else
- {
- continue;
- }
- }
-
- if (val->f <= median)
- vn->le += weight;
- else
- vn->gt += weight;
-
- count += weight;
- }
- casereader_destroy (r);
-
- {
- int x = 0;
- struct val_node *vn = NULL;
- double r_0 = 0;
- double r_1 = 0;
- HMAP_FOR_EACH (vn, struct val_node, node, &map)
- {
- r_0 += vn->le;
- r_1 += vn->gt;
- }
-
- results[v].n = count;
- results[v].sorted_array = XCALLOC (hmap_count (&map), struct val_node *);
- results[v].var = var;
-
- HMAP_FOR_EACH (vn, struct val_node, node, &map)
- {
- double e_0j = r_0 * (vn->le + vn->gt) / count;
- double e_1j = r_1 * (vn->le + vn->gt) / count;
-
- results[v].chisq += pow2 (vn->le - e_0j) / e_0j;
- results[v].chisq += pow2 (vn->gt - e_1j) / e_1j;
-
- results[v].sorted_array[x++] = vn;
- }
-
- n_vals = x;
- hmap_destroy (&map);
-
- sort (results[v].sorted_array, x, sizeof (*results[v].sorted_array),
- val_node_cmp_3way, nst->indep_var);
-
- }
- }
-
- casereader_destroy (input);
-
- show_frequencies (nst, results, n_vals, dict);
- show_test_statistics (nst, results, n_vals, dict);
-
- for (v = 0; v < nst->n_vars; ++v)
- {
- int i;
- const struct results *rs = results + v;
-
- for (i = 0; i < n_vals; ++i)
- {
- struct val_node *vn = rs->sorted_array[i];
- value_destroy (&vn->val, var_get_width (nst->indep_var));
- free (vn);
- }
- free (rs->sorted_array);
- }
- free (results);
-}
-
-
-
-static void
-show_frequencies (const struct n_sample_test *nst, const struct results *results, int n_vals, const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (N_("Frequencies"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- struct pivot_dimension *indep = pivot_dimension_create__ (
- table, PIVOT_AXIS_COLUMN, pivot_value_new_variable (nst->indep_var));
- indep->root->show_label = true;
- for (int i = 0; i < n_vals; ++i)
- pivot_category_create_leaf_rc (
- indep->root, pivot_value_new_var_value (
- nst->indep_var, &results->sorted_array[i]->val), PIVOT_RC_COUNT);
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("> Median"), N_("≤ Median"));
-
- struct pivot_dimension *dep = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- for (int v = 0; v < nst->n_vars; ++v)
- {
- const struct results *rs = &results[v];
-
- int dep_idx = pivot_category_create_leaf (
- dep->root, pivot_value_new_variable (rs->var));
-
- for (int indep_idx = 0; indep_idx < n_vals; indep_idx++)
- {
- const struct val_node *vn = rs->sorted_array[indep_idx];
- pivot_table_put3 (table, indep_idx, 0, dep_idx,
- pivot_value_new_number (vn->gt));
- pivot_table_put3 (table, indep_idx, 1, dep_idx,
- pivot_value_new_number (vn->le));
- }
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_test_statistics (const struct n_sample_test *nst,
- const struct results *results,
- int n_vals,
- const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Median"),
- N_("Chi-Square"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_COUNT,
- N_("Asymp. Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (int v = 0; v < nst->n_vars; ++v)
- {
- double df = n_vals - 1;
- const struct results *rs = &results[v];
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (rs->var));
-
- double entries[] = {
- rs->n,
- rs->median,
- rs->chisq,
- df,
- gsl_cdf_chisq_Q (rs->chisq, df),
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- struct pivot_value *value
- = pivot_value_new_number (entries[i]);
- if (i == 1)
- value->numeric.format = *var_get_print_format (rs->var);
- pivot_table_put2 (table, i, var_idx, value);
- }
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !median_h
-#define median_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-struct median_test
-{
- struct n_sample_test parent;
- double median;
-};
-
-struct casereader;
-struct dataset;
-
-void median_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer
- );
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/npar-summary.h"
-
-#include <math.h>
-
-#include "data/case.h"
-#include "data/casereader.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "math/moments.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-void
-npar_summary_calc_descriptives (struct descriptives *desc,
- struct casereader *input,
- const struct dictionary *dict,
- const struct variable *const *vv,
- int n_vars,
- enum mv_class filter)
-{
- int i = 0;
- for (i = 0 ; i < n_vars; ++i)
- {
- double minimum = DBL_MAX;
- double maximum = -DBL_MAX;
- double var;
- struct moments1 *moments = moments1_create (MOMENT_VARIANCE);
- struct ccase *c;
- const struct variable *v = vv[i];
- struct casereader *pass;
-
- pass = casereader_clone (input);
- pass = casereader_create_filter_missing (pass,
- &v, 1,
- filter, NULL, NULL);
- pass = casereader_create_filter_weight (pass, dict, NULL, NULL);
- while ((c = casereader_read (pass)) != NULL)
- {
- double val = case_num (c, v);
- double w = dict_get_case_weight (dict, c, NULL);
- minimum = MIN (minimum, val);
- maximum = MAX (maximum, val);
- moments1_add (moments, val, w);
- case_unref (c);
- }
- casereader_destroy (pass);
-
- moments1_calculate (moments,
- &desc[i].n,
- &desc[i].mean,
- &var,
- NULL, NULL);
-
- desc[i].std_dev = sqrt (var);
-
- moments1_destroy (moments);
-
- desc[i].min = minimum;
- desc[i].max = maximum;
- }
-
- casereader_destroy (input);
-}
-
-
-
-void
-do_summary_box (const struct descriptives *desc,
- const struct variable *const *vv,
- int n_vars,
- const struct fmt_spec *wfmt)
-{
- if (!desc)
- return;
-
- struct pivot_table *table = pivot_table_create (
- N_("Descriptive Statistics"));
- pivot_table_set_weight_format (table, wfmt);
-
- pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("Minimum"), N_("Maximum"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (int v = 0; v < n_vars; ++v)
- {
- const struct variable *var = vv[v];
-
- int row = pivot_category_create_leaf (variables->root,
- pivot_value_new_variable (var));
-
- double entries[] = { desc[v].n, desc[v].mean, desc[v].std_dev };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, row, pivot_value_new_number (entries[j]));
-
- union value extrema[2] = { { .f = desc[v].min }, { .f = desc[v].max } };
- for (size_t j = 0; j < 2; j++)
- pivot_table_put2 (table, 3 + j, row,
- pivot_value_new_var_value (var, &extrema[j]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !n_par_summary_h
-#define n_par_summary_h 1
-
-#include "data/missing-values.h"
-
-struct variable ;
-struct casereader ;
-struct dictionary;
-struct fmt_spec;
-
-struct descriptives
-{
- double n;
- double mean;
- double std_dev;
- double min;
- double max;
-};
-
-void
-do_summary_box (const struct descriptives *desc,
- const struct variable *const *vv,
- int n_vars,
- const struct fmt_spec *wfmt);
-
-void npar_summary_calc_descriptives (struct descriptives *desc,
- struct casereader *input,
- const struct dictionary *dict,
- const struct variable *const *vv,
- int n_vars,
- enum mv_class filter);
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis. -*-c-*-
- Copyright (C) 2006, 2008, 2009, 2010, 2011, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/npar.h"
-
-#include <stdlib.h>
-#include <math.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/settings.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "language/stats/binomial.h"
-#include "language/stats/chisquare.h"
-#include "language/stats/ks-one-sample.h"
-#include "language/stats/cochran.h"
-#include "language/stats/friedman.h"
-#include "language/stats/jonckheere-terpstra.h"
-#include "language/stats/kruskal-wallis.h"
-#include "language/stats/mann-whitney.h"
-#include "language/stats/mcnemar.h"
-#include "language/stats/median.h"
-#include "language/stats/npar-summary.h"
-#include "language/stats/runs.h"
-#include "language/stats/sign.h"
-#include "language/stats/wilcoxon.h"
-#include "libpspp/array.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmapx.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-#include "libpspp/taint.h"
-#include "math/moments.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* NPAR TESTS structure. */
-struct npar_specs
-{
- struct pool *pool;
- struct npar_test **test;
- size_t n_tests;
-
- const struct variable **vv; /* Compendium of all variables
- (those mentioned on ANY subcommand */
- int n_vars; /* Number of variables in vv */
-
- enum mv_class filter; /* Missing values to filter. */
- bool listwise_missing;
-
- bool descriptives; /* Descriptive statistics should be calculated */
- bool quartiles; /* Quartiles should be calculated */
-
- bool exact; /* Whether exact calculations have been requested */
- double timer; /* Maximum time (in minutes) to wait for exact calculations */
-};
-
-
-/* Prototype for custom subcommands of NPAR TESTS. */
-static bool npar_chisquare (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_binomial (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_ks_one_sample (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_runs (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_friedman (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_kendall (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_cochran (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_wilcoxon (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_sign (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_kruskal_wallis (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_jonckheere_terpstra (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_mann_whitney (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_mcnemar (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_median (struct lexer *, struct dataset *, struct npar_specs *);
-static bool npar_method (struct lexer *, struct npar_specs *);
-
-/* Command parsing functions. */
-
-static int
-parse_npar_tests (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *nps)
-{
- bool seen_missing = false;
- bool seen_method = false;
- lex_match (lexer, T_SLASH);
- do
- {
- if (lex_match_id (lexer, "COCHRAN"))
- {
- if (!npar_cochran (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "FRIEDMAN"))
- {
- if (!npar_friedman (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "KENDALL"))
- {
- if (!npar_kendall (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "RUNS"))
- {
- if (!npar_runs (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "CHISQUARE"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_chisquare (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "BINOMIAL"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_binomial (lexer, ds, nps))
- return false;
- }
- else if (lex_match_phrase (lexer, "K-S") ||
- lex_match_phrase (lexer, "KOLMOGOROV-SMIRNOV"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_ks_one_sample (lexer, ds, nps))
- return false;
- }
- else if (lex_match_phrase (lexer, "J-T") ||
- lex_match_phrase (lexer, "JONCKHEERE-TERPSTRA"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_jonckheere_terpstra (lexer, ds, nps))
- return false;
- }
- else if (lex_match_phrase (lexer, "K-W") ||
- lex_match_phrase (lexer, "KRUSKAL-WALLIS"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_kruskal_wallis (lexer, ds, nps))
- return false;
- }
- else if (lex_match_phrase (lexer, "MCNEMAR"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_mcnemar (lexer, ds, nps))
- return false;
- }
- else if (lex_match_phrase (lexer, "M-W") ||
- lex_match_phrase (lexer, "MANN-WHITNEY"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_mann_whitney (lexer, ds, nps))
- return false;
- }
- else if (lex_match_phrase (lexer, "MEDIAN"))
- {
- if (!npar_median (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "WILCOXON"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_wilcoxon (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "SIGN"))
- {
- lex_match (lexer, T_EQUALS);
- if (!npar_sign (lexer, ds, nps))
- return false;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- if (seen_missing)
- {
- lex_sbc_only_once (lexer, "MISSING");
- return false;
- }
- seen_missing = true;
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "ANALYSIS"))
- nps->listwise_missing = false;
- else if (lex_match_id (lexer, "LISTWISE"))
- nps->listwise_missing = true;
- else if (lex_match_id (lexer, "INCLUDE"))
- nps->filter = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- nps->filter = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "ANALYSIS", "LISTWISE",
- "INCLUDE", "EXCLUDE");
- return false;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "METHOD"))
- {
- lex_match (lexer, T_EQUALS);
- if (seen_method)
- {
- lex_sbc_only_once (lexer, "METHOD");
- return false;
- }
- seen_method = true;
- if (!npar_method (lexer, nps))
- return false;
- }
- else if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "DESCRIPTIVES"))
- nps->descriptives = true;
- else if (lex_match_id (lexer, "QUARTILES"))
- nps->quartiles = true;
- else if (lex_match (lexer, T_ALL))
- nps->descriptives = nps->quartiles = true;
- else
- {
- lex_error_expecting (lexer, "DESCRIPTIVES", "QUARTILES",
- "ALL");
- return false;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "ALGORITHM"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "COMPATIBLE"))
- settings_set_cmd_algorithm (COMPATIBLE);
- else if (lex_match_id (lexer, "ENHANCED"))
- settings_set_cmd_algorithm (ENHANCED);
- else
- {
- lex_error_expecting (lexer, "COMPATIBLE", "ENHANCED");
- return false;
- }
- }
- else
- {
- lex_error_expecting (lexer, "COCHRAN", "FRIEDMAN", "KENDALL", "RUNS",
- "CHISQUARE", "BINOMIAL", "K-S", "J-T", "K-W",
- "MCNEMAR", "M-W", "MEDIAN", "WILCOXON",
- "SIGN", "MISSING", "METHOD", "STATISTICS",
- "ALGORITHM");
- return false;
- }
- }
- while (lex_match (lexer, T_SLASH));
-
- return true;
-}
-
-static void one_sample_insert_variables (const struct npar_test *test,
- struct hmapx *);
-
-static void two_sample_insert_variables (const struct npar_test *test,
- struct hmapx *);
-
-static void n_sample_insert_variables (const struct npar_test *test,
- struct hmapx *);
-
-static void
-npar_execute (struct casereader *input,
- const struct npar_specs *specs,
- const struct dataset *ds)
-{
- struct descriptives *summary_descriptives = NULL;
-
- for (size_t t = 0; t < specs->n_tests; ++t)
- {
- const struct npar_test *test = specs->test[t];
- test->execute (ds, casereader_clone (input), specs->filter,
- test, specs->exact, specs->timer);
- }
-
- if (specs->descriptives && specs->n_vars > 0)
- {
- summary_descriptives = xnmalloc (sizeof (*summary_descriptives),
- specs->n_vars);
-
- npar_summary_calc_descriptives (summary_descriptives,
- casereader_clone (input),
- dataset_dict (ds),
- specs->vv, specs->n_vars,
- specs->filter);
- }
-
- if ((specs->descriptives || specs->quartiles)
- && !taint_has_tainted_successor (casereader_get_taint (input)))
- do_summary_box (summary_descriptives, specs->vv, specs->n_vars,
- dict_get_weight_format (dataset_dict (ds)));
-
- free (summary_descriptives);
- casereader_destroy (input);
-}
-
-int
-cmd_npar_tests (struct lexer *lexer, struct dataset *ds)
-{
- struct npar_specs npar_specs = {
- .pool = pool_create (),
- .filter = MV_ANY,
- .listwise_missing = false,
- };
-
- if (!parse_npar_tests (lexer, ds, &npar_specs))
- {
- pool_destroy (npar_specs.pool);
- return CMD_FAILURE;
- }
-
- struct hmapx var_map = HMAPX_INITIALIZER (var_map);
- for (size_t i = 0; i < npar_specs.n_tests; ++i)
- {
- const struct npar_test *test = npar_specs.test[i];
- test->insert_variables (test, &var_map);
- }
-
- struct hmapx_node *node;
- struct variable *var;
- npar_specs.vv = pool_alloc (npar_specs.pool,
- hmapx_count (&var_map) * sizeof *npar_specs.vv);
- HMAPX_FOR_EACH (var, node, &var_map)
- npar_specs.vv[npar_specs.n_vars++] = var;
- assert (npar_specs.n_vars == hmapx_count (&var_map));
-
- sort (npar_specs.vv, npar_specs.n_vars, sizeof *npar_specs.vv,
- compare_var_ptrs_by_name, NULL);
-
- struct casereader *input = proc_open (ds);
- if (npar_specs.listwise_missing)
- input = casereader_create_filter_missing (input,
- npar_specs.vv,
- npar_specs.n_vars,
- npar_specs.filter,
- NULL, NULL);
-
- struct casegrouper *grouper = casegrouper_create_splits (input, dataset_dict (ds));
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- npar_execute (group, &npar_specs, ds);
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- pool_destroy (npar_specs.pool);
- hmapx_destroy (&var_map);
-
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-}
-
-static void
-add_test (struct npar_specs *specs, struct npar_test *nt)
-{
- specs->test = pool_realloc (specs->pool, specs->test,
- (specs->n_tests + 1) * sizeof *specs->test);
-
- specs->test[specs->n_tests++] = nt;
-}
-
-static bool
-npar_runs (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct runs_test *rt = pool_alloc (specs->pool, sizeof (*rt));
- struct one_sample_test *tp = &rt->parent;
- struct npar_test *nt = &tp->parent;
-
- nt->execute = runs_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- if (!lex_force_match (lexer, T_LPAREN))
- return false;
-
- if (lex_match_id (lexer, "MEAN"))
- rt->cp_mode = CP_MEAN;
- else if (lex_match_id (lexer, "MEDIAN"))
- rt->cp_mode = CP_MEDIAN;
- else if (lex_match_id (lexer, "MODE"))
- rt->cp_mode = CP_MODE;
- else if (lex_is_number (lexer))
- {
- rt->cutpoint = lex_number (lexer);
- rt->cp_mode = CP_CUSTOM;
- lex_get (lexer);
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting %s, %s, %s or a number."),
- "MEAN", "MEDIAN", "MODE");
- return false;
- }
-
- if (!lex_force_match_phrase (lexer, ")="))
- return false;
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &tp->vars, &tp->n_vars,
- PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_friedman (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct friedman_test *ft = pool_alloc (specs->pool, sizeof (*ft));
- struct one_sample_test *ost = &ft->parent;
- struct npar_test *nt = &ost->parent;
-
- ft->kendalls_w = false;
- nt->execute = friedman_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- lex_match (lexer, T_EQUALS);
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &ost->vars, &ost->n_vars,
- PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_kendall (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct friedman_test *kt = pool_alloc (specs->pool, sizeof (*kt));
- struct one_sample_test *ost = &kt->parent;
- struct npar_test *nt = &ost->parent;
-
- kt->kendalls_w = true;
- nt->execute = friedman_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- lex_match (lexer, T_EQUALS);
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &ost->vars, &ost->n_vars,
- PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-
-static bool
-npar_cochran (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct one_sample_test *ft = pool_alloc (specs->pool, sizeof (*ft));
- struct npar_test *nt = &ft->parent;
-
- nt->execute = cochran_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- lex_match (lexer, T_EQUALS);
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &ft->vars, &ft->n_vars,
- PV_NO_SCRATCH | PV_NO_DUPLICATE | PV_NUMERIC))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_chisquare (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct chisquare_test *cstp = pool_alloc (specs->pool, sizeof (*cstp));
- struct one_sample_test *tp = &cstp->parent;
- struct npar_test *nt = &tp->parent;
-
- nt->execute = chisquare_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &tp->vars, &tp->n_vars,
- PV_NO_SCRATCH | PV_NO_DUPLICATE))
- return false;
-
- cstp->ranged = false;
-
- if (lex_match (lexer, T_LPAREN))
- {
- cstp->ranged = true;
- if (!lex_force_num (lexer))
- return false;
- cstp->lo = lex_number (lexer);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_COMMA))
- return false;
- if (!lex_force_num_range_open (lexer, "HI", cstp->lo, DBL_MAX))
- return false;
- cstp->hi = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
-
- cstp->n_expected = 0;
- cstp->expected = NULL;
- int expected_start = 0;
- int expected_end = 0;
- if (lex_match_phrase (lexer, "/EXPECTED"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- return false;
-
- if (!lex_match_id (lexer, "EQUAL"))
- {
- expected_start = lex_ofs (lexer);
- while (lex_is_number (lexer))
- {
- int n = 1;
- double f = lex_number (lexer);
- lex_get (lexer);
- if (lex_match (lexer, T_ASTERISK))
- {
- n = f;
- if (!lex_force_num (lexer))
- return false;
- f = lex_number (lexer);
- lex_get (lexer);
- }
- lex_match (lexer, T_COMMA);
-
- cstp->n_expected += n;
- cstp->expected = pool_realloc (specs->pool,
- cstp->expected,
- sizeof (double) * cstp->n_expected);
- for (int i = cstp->n_expected - n; i < cstp->n_expected; ++i)
- cstp->expected[i] = f;
- }
- expected_end = lex_ofs (lexer) - 1;
- }
- }
-
- if (cstp->ranged && cstp->n_expected > 0 &&
- cstp->n_expected != cstp->hi - cstp->lo + 1)
- {
- lex_ofs_error (lexer, expected_start, expected_end,
- _("%d expected values were given, but the specified "
- "range (%d-%d) requires exactly %d values."),
- cstp->n_expected, cstp->lo, cstp->hi,
- cstp->hi - cstp->lo +1);
- return false;
- }
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_binomial (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct binomial_test *btp = pool_alloc (specs->pool, sizeof (*btp));
- struct one_sample_test *tp = &btp->parent;
- struct npar_test *nt = &tp->parent;
-
- nt->execute = binomial_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- btp->category1 = btp->category2 = btp->cutpoint = SYSMIS;
-
- btp->p = 0.5;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num (lexer))
- return false;
- btp->p = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- if (!lex_force_match (lexer, T_EQUALS))
- return false;
- }
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &tp->vars, &tp->n_vars,
- PV_NUMERIC | PV_NO_SCRATCH | PV_NO_DUPLICATE))
- return false;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num (lexer))
- return false;
- btp->category1 = lex_number (lexer);
- lex_get (lexer);
- if (lex_match (lexer, T_COMMA))
- {
- if (!lex_force_num (lexer))
- return false;
- btp->category2 = lex_number (lexer);
- lex_get (lexer);
- }
- else
- btp->cutpoint = btp->category1;
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
-
- add_test (specs, nt);
- return true;
-}
-
-static void
-ks_one_sample_parse_params (struct lexer *lexer, struct ks_one_sample_test *kst, int params)
-{
- assert (params == 1 || params == 2);
-
- if (lex_is_number (lexer))
- {
- kst->p[0] = lex_number (lexer);
-
- lex_get (lexer);
- if (params == 2)
- {
- lex_match (lexer, T_COMMA);
- if (lex_force_num (lexer))
- {
- kst->p[1] = lex_number (lexer);
- lex_get (lexer);
- }
- }
- }
-}
-
-static bool
-npar_ks_one_sample (struct lexer *lexer, struct dataset *ds, struct npar_specs *specs)
-{
- struct ks_one_sample_test *kst = pool_alloc (specs->pool, sizeof (*kst));
- struct one_sample_test *tp = &kst->parent;
- struct npar_test *nt = &tp->parent;
-
- nt->execute = ks_one_sample_execute;
- nt->insert_variables = one_sample_insert_variables;
-
- kst->p[0] = kst->p[1] = SYSMIS;
-
- if (!lex_force_match (lexer, T_LPAREN))
- return false;
-
- if (lex_match_id (lexer, "NORMAL"))
- {
- kst->dist = KS_NORMAL;
- ks_one_sample_parse_params (lexer, kst, 2);
- }
- else if (lex_match_id (lexer, "POISSON"))
- {
- kst->dist = KS_POISSON;
- ks_one_sample_parse_params (lexer, kst, 1);
- }
- else if (lex_match_id (lexer, "UNIFORM"))
- {
- kst->dist = KS_UNIFORM;
- ks_one_sample_parse_params (lexer, kst, 2);
- }
- else if (lex_match_id (lexer, "EXPONENTIAL"))
- {
- kst->dist = KS_EXPONENTIAL;
- ks_one_sample_parse_params (lexer, kst, 1);
- }
- else
- {
- lex_error_expecting (lexer, "NORMAL", "POISSON", "UNIFORM",
- "EXPONENTIAL");
- return false;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
-
- lex_match (lexer, T_EQUALS);
-
- if (!parse_variables_const_pool (lexer, specs->pool, dataset_dict (ds),
- &tp->vars, &tp->n_vars,
- PV_NUMERIC | PV_NO_SCRATCH | PV_NO_DUPLICATE))
- return false;
-
- add_test (specs, nt);
-
- return true;
-}
-
-static bool
-parse_two_sample_related_test (struct lexer *lexer,
- const struct dictionary *dict,
- struct two_sample_test *tp,
- struct pool *pool)
-{
- tp->parent.insert_variables = two_sample_insert_variables;
-
- const struct variable **v1;
- size_t n1;
- int vars_start = lex_ofs (lexer);
- if (!parse_variables_const_pool (lexer, pool, dict, &v1, &n1,
- PV_NUMERIC | PV_NO_SCRATCH | PV_DUPLICATE))
- return false;
-
- bool with = false;
- bool paired = false;
- const struct variable **v2 = NULL;
- size_t n2 = 0;
- if (lex_match (lexer, T_WITH))
- {
- with = true;
- if (!parse_variables_const_pool (lexer, pool, dict, &v2, &n2,
- PV_NUMERIC | PV_NO_SCRATCH | PV_DUPLICATE))
- return false;
- int vars_end = lex_ofs (lexer) - 1;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_match_phrase (lexer, "PAIRED)"))
- return false;
- paired = true;
-
- if (n1 != n2)
- {
- lex_ofs_error (lexer, vars_start, vars_end,
- _("PAIRED was specified, but the number of "
- "variables preceding WITH (%zu) does not match "
- "the number following (%zu)."),
- n1, n2);
- return false;
- }
- }
- }
-
- tp->n_pairs = (paired ? n1
- : with ? n1 * n2
- : (n1 * (n1 - 1)) / 2);
- tp->pairs = pool_alloc (pool, sizeof (variable_pair) * tp->n_pairs);
-
- size_t n = 0;
- if (!with)
- for (size_t i = 0; i < n1 - 1; ++i)
- for (size_t j = i + 1; j < n1; ++j)
- {
- assert (n < tp->n_pairs);
- tp->pairs[n][0] = v1[i];
- tp->pairs[n][1] = v1[j];
- n++;
- }
- else if (paired)
- {
- assert (n1 == n2);
- for (size_t i = 0; i < n1; ++i)
- {
- tp->pairs[n][0] = v1[i];
- tp->pairs[n][1] = v2[i];
- n++;
- }
- }
- else
- {
- for (size_t i = 0; i < n1; ++i)
- for (size_t j = 0; j < n2; ++j)
- {
- tp->pairs[n][0] = v1[i];
- tp->pairs[n][1] = v2[j];
- n++;
- }
- }
- assert (n == tp->n_pairs);
-
- return true;
-}
-
-static bool
-parse_n_sample_related_test (struct lexer *lexer, const struct dictionary *dict,
- struct n_sample_test *nst, struct pool *pool)
-{
- if (!parse_variables_const_pool (lexer, pool, dict, &nst->vars, &nst->n_vars,
- PV_NUMERIC | PV_NO_SCRATCH | PV_NO_DUPLICATE))
- return false;
-
- if (!lex_force_match (lexer, T_BY))
- return false;
-
- nst->indep_var = parse_variable_const (lexer, dict);
- if (!nst->indep_var)
- return false;
-
- if (!lex_force_match (lexer, T_LPAREN))
- return false;
-
- value_init (&nst->val1, var_get_width (nst->indep_var));
- if (!parse_value (lexer, &nst->val1, nst->indep_var))
- {
- value_destroy (&nst->val1, var_get_width (nst->indep_var));
- return false;
- }
-
- lex_match (lexer, T_COMMA);
-
- value_init (&nst->val2, var_get_width (nst->indep_var));
- if (!parse_value (lexer, &nst->val2, nst->indep_var))
- {
- value_destroy (&nst->val2, var_get_width (nst->indep_var));
- return false;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
-
- return true;
-}
-
-static bool
-npar_wilcoxon (struct lexer *lexer,
- struct dataset *ds,
- struct npar_specs *specs)
-{
- struct two_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
- struct npar_test *nt = &tp->parent;
- nt->execute = wilcoxon_execute;
-
- if (!parse_two_sample_related_test (lexer, dataset_dict (ds),
- tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_mann_whitney (struct lexer *lexer,
- struct dataset *ds,
- struct npar_specs *specs)
-{
- struct n_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
- struct npar_test *nt = &tp->parent;
-
- nt->insert_variables = n_sample_insert_variables;
- nt->execute = mann_whitney_execute;
-
- if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_median (struct lexer *lexer,
- struct dataset *ds,
- struct npar_specs *specs)
-{
- struct median_test *mt = pool_alloc (specs->pool, sizeof (*mt));
- struct n_sample_test *tp = &mt->parent;
- struct npar_test *nt = &tp->parent;
-
- mt->median = SYSMIS;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num (lexer))
- return false;
- mt->median = lex_number (lexer);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
-
- lex_match (lexer, T_EQUALS);
-
- nt->insert_variables = n_sample_insert_variables;
- nt->execute = median_execute;
-
- if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_sign (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct two_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
- struct npar_test *nt = &tp->parent;
-
- nt->execute = sign_execute;
-
- if (!parse_two_sample_related_test (lexer, dataset_dict (ds),
- tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_mcnemar (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct two_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
- struct npar_test *nt = &tp->parent;
-
- nt->execute = mcnemar_execute;
-
- if (!parse_two_sample_related_test (lexer, dataset_dict (ds),
- tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-
-static bool
-npar_jonckheere_terpstra (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct n_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
- struct npar_test *nt = &tp->parent;
-
- nt->insert_variables = n_sample_insert_variables;
- nt->execute = jonckheere_terpstra_execute;
-
- if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static bool
-npar_kruskal_wallis (struct lexer *lexer, struct dataset *ds,
- struct npar_specs *specs)
-{
- struct n_sample_test *tp = pool_alloc (specs->pool, sizeof (*tp));
- struct npar_test *nt = &tp->parent;
-
- nt->insert_variables = n_sample_insert_variables;
-
- nt->execute = kruskal_wallis_execute;
-
- if (!parse_n_sample_related_test (lexer, dataset_dict (ds), tp, specs->pool))
- return false;
-
- add_test (specs, nt);
- return true;
-}
-
-static void
-insert_variable_into_map (struct hmapx *var_map, const struct variable *var)
-{
- size_t hash = hash_pointer (var, 0);
- struct hmapx_node *node;
- const struct variable *v = NULL;
-
- HMAPX_FOR_EACH_WITH_HASH (v, node, hash, var_map)
- if (v == var)
- return;
-
- hmapx_insert (var_map, CONST_CAST (struct variable *, var), hash);
-}
-
-/* Insert the variables for TEST into VAR_MAP */
-static void
-one_sample_insert_variables (const struct npar_test *test,
- struct hmapx *var_map)
-{
- const struct one_sample_test *ost = UP_CAST (test, const struct one_sample_test, parent);
-
- for (size_t i = 0; i < ost->n_vars; ++i)
- insert_variable_into_map (var_map, ost->vars[i]);
-}
-
-
-static void
-two_sample_insert_variables (const struct npar_test *test,
- struct hmapx *var_map)
-{
- const struct two_sample_test *tst = UP_CAST (test, const struct two_sample_test, parent);
-
- for (size_t i = 0; i < tst->n_pairs; ++i)
- {
- variable_pair *pair = &tst->pairs[i];
-
- insert_variable_into_map (var_map, (*pair)[0]);
- insert_variable_into_map (var_map, (*pair)[1]);
- }
-}
-
-static void
-n_sample_insert_variables (const struct npar_test *test,
- struct hmapx *var_map)
-{
- const struct n_sample_test *tst = UP_CAST (test, const struct n_sample_test, parent);
-
- for (size_t i = 0; i < tst->n_vars; ++i)
- insert_variable_into_map (var_map, tst->vars[i]);
-
- insert_variable_into_map (var_map, tst->indep_var);
-}
-
-static bool
-npar_method (struct lexer *lexer, struct npar_specs *specs)
-{
- if (lex_match_id (lexer, "EXACT"))
- {
- specs->exact = true;
- specs->timer = 0.0;
- if (lex_match_id (lexer, "TIMER"))
- {
- specs->timer = 5.0;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num (lexer))
- return false;
- specs->timer = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- }
- }
-
- return true;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !npar_h
-#define npar_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "data/missing-values.h"
-#include "data/value.h"
-
-typedef const struct variable *variable_pair[2];
-
-struct hmapx;
-struct casefilter;
-struct casereader;
-struct dataset;
-
-
-struct npar_test
-{
- void (*execute) (const struct dataset *,
- struct casereader *,
- enum mv_class exclude,
- const struct npar_test *,
- bool,
- double);
-
- void (*insert_variables) (const struct npar_test *,
- struct hmapx *);
-};
-
-
-struct one_sample_test
-{
- struct npar_test parent;
- const struct variable **vars;
- size_t n_vars;
-};
-
-struct two_sample_test
-{
- struct npar_test parent;
- variable_pair *pairs;
- size_t n_pairs;
-};
-
-
-struct n_sample_test
-{
- struct npar_test parent;
- const struct variable **vars;
- size_t n_vars;
-
- union value val1, val2;
- const struct variable *indep_var;
-};
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2007, 2009, 2010, 2011, 2012, 2013, 2014,
- 2020 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_matrix.h>
-#include <math.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/value.h"
-#include "language/command.h"
-#include "language/dictionary/split-file.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/ll.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/taint.h"
-#include "linreg/sweep.h"
-#include "tukey/tukey.h"
-#include "math/categoricals.h"
-#include "math/interaction.h"
-#include "math/covariance.h"
-#include "math/levene.h"
-#include "math/moments.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-/* Workspace variable for each dependent variable */
-struct per_var_ws
- {
- struct interaction *iact;
- struct categoricals *cat;
- struct covariance *cov;
- struct levene *nl;
-
- double n;
-
- double sst;
- double sse;
- double ssa;
-
- int n_groups;
-
- double mse;
- };
-
-/* Per category data */
-struct descriptive_data
- {
- const struct variable *var;
- struct moments1 *mom;
-
- double minimum;
- double maximum;
- };
-
-enum missing_type
- {
- MISS_LISTWISE,
- MISS_ANALYSIS,
- };
-
-struct coeff_node
-{
- struct ll ll;
- double coeff;
-};
-
-
-struct contrasts_node
-{
- struct ll ll;
- struct ll_list coefficient_list;
-};
-
-
-struct oneway_spec;
-
-typedef double df_func (const struct per_var_ws *pvw, const struct moments1 *mom_i, const struct moments1 *mom_j);
-typedef double ts_func (int k, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err);
-typedef double p1tail_func (double ts, double df1, double df2);
-
-typedef double pinv_func (double std_err, double alpha, double df, int k, const struct moments1 *mom_i, const struct moments1 *mom_j);
-
-
-struct posthoc
-{
- const char *syntax;
- const char *label;
-
- df_func *dff;
- ts_func *tsf;
- p1tail_func *p1f;
-
- pinv_func *pinv;
-};
-
-struct oneway_spec
-{
- size_t n_vars;
- const struct variable **vars;
-
- const struct variable *indep_var;
-
- bool descriptive_stats;
- bool homogeneity_stats;
-
- enum missing_type missing_type;
- enum mv_class exclude;
-
- /* List of contrasts */
- struct ll_list contrast_list;
-
- /* The weight variable */
- const struct variable *wv;
- const struct fmt_spec *wfmt;
-
- /* The confidence level for multiple comparisons */
- double alpha;
-
- int *posthoc;
- int n_posthoc;
-};
-
-static double
-df_common (const struct per_var_ws *pvw, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
-{
- return pvw->n - pvw->n_groups;
-}
-
-static double
-df_individual (const struct per_var_ws *pvw UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j)
-{
- double n_i, var_i;
- double n_j, var_j;
- double nom,denom;
-
- moments1_calculate (mom_i, &n_i, NULL, &var_i, 0, 0);
- moments1_calculate (mom_j, &n_j, NULL, &var_j, 0, 0);
-
- if (n_i <= 1.0 || n_j <= 1.0)
- return SYSMIS;
-
- nom = pow2 (var_i/n_i + var_j/n_j);
- denom = pow2 (var_i/n_i) / (n_i - 1) + pow2 (var_j/n_j) / (n_j - 1);
-
- return nom / denom;
-}
-
-static double lsd_pinv (double std_err, double alpha, double df, int k UNUSED, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
-{
- return std_err * gsl_cdf_tdist_Pinv (1.0 - alpha / 2.0, df);
-}
-
-static double bonferroni_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
-{
- const int m = k * (k - 1) / 2;
- return std_err * gsl_cdf_tdist_Pinv (1.0 - alpha / (2.0 * m), df);
-}
-
-static double sidak_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
-{
- const double m = k * (k - 1) / 2;
- double lp = 1.0 - exp (log (1.0 - alpha) / m);
- return std_err * gsl_cdf_tdist_Pinv (1.0 - lp / 2.0, df);
-}
-
-static double tukey_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
-{
- if (k < 2 || df < 2)
- return SYSMIS;
-
- return std_err / sqrt (2.0) * qtukey (1 - alpha, 1.0, k, df, 1, 0);
-}
-
-static double scheffe_pinv (double std_err, double alpha, double df, int k, const struct moments1 *mom_i UNUSED, const struct moments1 *mom_j UNUSED)
-{
- double x = (k - 1) * gsl_cdf_fdist_Pinv (1.0 - alpha, k - 1, df);
- return std_err * sqrt (x);
-}
-
-static double gh_pinv (double std_err UNUSED, double alpha, double df, int k, const struct moments1 *mom_i, const struct moments1 *mom_j)
-{
- double n_i, mean_i, var_i;
- double n_j, mean_j, var_j;
- double m;
-
- moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
- moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
-
- m = sqrt ((var_i/n_i + var_j/n_j) / 2.0);
-
- if (k < 2 || df < 2)
- return SYSMIS;
-
- return m * qtukey (1 - alpha, 1.0, k, df, 1, 0);
-}
-
-
-static double
-multiple_comparison_sig (double std_err,
- const struct per_var_ws *pvw,
- const struct descriptive_data *dd_i, const struct descriptive_data *dd_j,
- const struct posthoc *ph)
-{
- int k = pvw->n_groups;
- double df = ph->dff (pvw, dd_i->mom, dd_j->mom);
- double ts = ph->tsf (k, dd_i->mom, dd_j->mom, std_err);
- if (df == SYSMIS)
- return SYSMIS;
- return ph->p1f (ts, k - 1, df);
-}
-
-static double
-mc_half_range (const struct oneway_spec *cmd, const struct per_var_ws *pvw, double std_err, const struct descriptive_data *dd_i, const struct descriptive_data *dd_j, const struct posthoc *ph)
-{
- int k = pvw->n_groups;
- double df = ph->dff (pvw, dd_i->mom, dd_j->mom);
- if (df == SYSMIS)
- return SYSMIS;
-
- return ph->pinv (std_err, cmd->alpha, df, k, dd_i->mom, dd_j->mom);
-}
-
-static double tukey_1tailsig (double ts, double df1, double df2)
-{
- double twotailedsig;
-
- if (df2 < 2 || df1 < 1)
- return SYSMIS;
-
- twotailedsig = 1.0 - ptukey (ts, 1.0, df1 + 1, df2, 1, 0);
-
- return twotailedsig / 2.0;
-}
-
-static double lsd_1tailsig (double ts, double df1 UNUSED, double df2)
-{
- return ts < 0 ? gsl_cdf_tdist_P (ts, df2) : gsl_cdf_tdist_Q (ts, df2);
-}
-
-static double sidak_1tailsig (double ts, double df1, double df2)
-{
- double ex = (df1 + 1.0) * df1 / 2.0;
- double lsd_sig = 2 * lsd_1tailsig (ts, df1, df2);
-
- return 0.5 * (1.0 - pow (1.0 - lsd_sig, ex));
-}
-
-static double bonferroni_1tailsig (double ts, double df1, double df2)
-{
- const int m = (df1 + 1) * df1 / 2;
-
- double p = ts < 0 ? gsl_cdf_tdist_P (ts, df2) : gsl_cdf_tdist_Q (ts, df2);
- p *= m;
-
- return p > 0.5 ? 0.5 : p;
-}
-
-static double scheffe_1tailsig (double ts, double df1, double df2)
-{
- return 0.5 * gsl_cdf_fdist_Q (ts, df1, df2);
-}
-
-
-static double tukey_test_stat (int k UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err)
-{
- double ts;
- double n_i, mean_i, var_i;
- double n_j, mean_j, var_j;
-
- moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
- moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
-
- ts = (mean_i - mean_j) / std_err;
- ts = fabs (ts) * sqrt (2.0);
-
- return ts;
-}
-
-static double lsd_test_stat (int k UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err)
-{
- double n_i, mean_i, var_i;
- double n_j, mean_j, var_j;
-
- moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
- moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
-
- return (mean_i - mean_j) / std_err;
-}
-
-static double scheffe_test_stat (int k, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err)
-{
- double t;
- double n_i, mean_i, var_i;
- double n_j, mean_j, var_j;
-
- moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
- moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
-
- t = (mean_i - mean_j) / std_err;
- t = pow2 (t);
- t /= k - 1;
-
- return t;
-}
-
-static double gh_test_stat (int k UNUSED, const struct moments1 *mom_i, const struct moments1 *mom_j, double std_err UNUSED)
-{
- double ts;
- double thing;
- double n_i, mean_i, var_i;
- double n_j, mean_j, var_j;
-
- moments1_calculate (mom_i, &n_i, &mean_i, &var_i, 0, 0);
- moments1_calculate (mom_j, &n_j, &mean_j, &var_j, 0, 0);
-
- thing = var_i / n_i + var_j / n_j;
- thing /= 2.0;
- thing = sqrt (thing);
-
- ts = (mean_i - mean_j) / thing;
-
- return fabs (ts);
-}
-
-
-
-static const struct posthoc ph_tests [] =
- {
- { "LSD", N_("LSD"), df_common, lsd_test_stat, lsd_1tailsig, lsd_pinv},
- { "TUKEY", N_("Tukey HSD"), df_common, tukey_test_stat, tukey_1tailsig, tukey_pinv},
- { "BONFERRONI", N_("Bonferroni"), df_common, lsd_test_stat, bonferroni_1tailsig, bonferroni_pinv},
- { "SCHEFFE", N_("Scheffé"), df_common, scheffe_test_stat, scheffe_1tailsig, scheffe_pinv},
- { "GH", N_("Games-Howell"), df_individual, gh_test_stat, tukey_1tailsig, gh_pinv},
- { "SIDAK", N_("Šidák"), df_common, lsd_test_stat, sidak_1tailsig, sidak_pinv}
- };
-
-
-struct oneway_workspace
-{
- /* The number of distinct values of the independent variable, when all
- missing values are disregarded */
- int actual_number_of_groups;
-
- struct per_var_ws *vws;
-
- /* An array of descriptive data. One for each dependent variable */
- struct descriptive_data **dd_total;
-};
-
-/* Routines to show the output tables */
-static void show_anova_table (const struct oneway_spec *, const struct oneway_workspace *);
-static void show_descriptives (const struct oneway_spec *, const struct oneway_workspace *);
-static void show_homogeneity (const struct oneway_spec *, const struct oneway_workspace *);
-
-static void output_oneway (const struct oneway_spec *, struct oneway_workspace *ws);
-static void run_oneway (const struct oneway_spec *cmd, struct casereader *input, const struct dataset *ds);
-
-
-static void
-destroy_coeff_list (struct contrasts_node *coeff_list)
-{
- struct coeff_node *cn = NULL;
- struct coeff_node *cnx = NULL;
- struct ll_list *cl = &coeff_list->coefficient_list;
-
- ll_for_each_safe (cn, cnx, struct coeff_node, ll, cl)
- {
- free (cn);
- }
-
- free (coeff_list);
-}
-
-static void
-oneway_cleanup (struct oneway_spec *cmd)
-{
- struct contrasts_node *coeff_list = NULL;
- struct contrasts_node *coeff_next = NULL;
- ll_for_each_safe (coeff_list, coeff_next, struct contrasts_node, ll, &cmd->contrast_list)
- {
- destroy_coeff_list (coeff_list);
- }
-
- free (cmd->posthoc);
-}
-
-
-
-int
-cmd_oneway (struct lexer *lexer, struct dataset *ds)
-{
- const struct dictionary *dict = dataset_dict (ds);
- struct oneway_spec oneway = {
- .missing_type = MISS_ANALYSIS,
- .exclude = MV_ANY,
- .wv = dict_get_weight (dict),
- .wfmt = dict_get_weight_format (dict),
- .alpha = 0.05,
- };
-
- ll_init (&oneway.contrast_list);
- if (lex_match (lexer, T_SLASH))
- {
- if (!lex_force_match_id (lexer, "VARIABLES"))
- goto error;
- lex_match (lexer, T_EQUALS);
- }
-
- if (!parse_variables_const (lexer, dict,
- &oneway.vars, &oneway.n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
-
- if (!lex_force_match (lexer, T_BY))
- goto error;
-
- oneway.indep_var = parse_variable_const (lexer, dict);
- if (oneway.indep_var == NULL)
- goto error;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "STATISTICS"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "DESCRIPTIVES"))
- oneway.descriptive_stats = true;
- else if (lex_match_id (lexer, "HOMOGENEITY"))
- oneway.homogeneity_stats = true;
- else
- {
- lex_error_expecting (lexer, "DESCRIPTIVES", "HOMOGENEITY");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "POSTHOC"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- bool method = false;
- for (size_t p = 0; p < sizeof ph_tests / sizeof *ph_tests; ++p)
- if (lex_match_id (lexer, ph_tests[p].syntax))
- {
- oneway.n_posthoc++;
- oneway.posthoc = xrealloc (oneway.posthoc, sizeof (*oneway.posthoc) * oneway.n_posthoc);
- oneway.posthoc[oneway.n_posthoc - 1] = p;
- method = true;
- break;
- }
- if (method == false)
- {
- if (lex_match_id (lexer, "ALPHA"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num (lexer))
- goto error;
- oneway.alpha = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else
- {
- lex_error (lexer, _("Unknown post hoc analysis method."));
- goto error;
- }
- }
- }
- }
- else if (lex_match_id (lexer, "CONTRAST"))
- {
- struct contrasts_node *cl = XZALLOC (struct contrasts_node);
-
- struct ll_list *coefficient_list = &cl->coefficient_list;
- lex_match (lexer, T_EQUALS);
-
- ll_init (coefficient_list);
-
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (!lex_force_num (lexer))
- {
- destroy_coeff_list (cl);
- goto error;
- }
-
- struct coeff_node *cc = xmalloc (sizeof *cc);
- cc->coeff = lex_number (lexer);
-
- ll_push_tail (coefficient_list, &cc->ll);
- lex_get (lexer);
- }
-
- if (ll_count (coefficient_list) <= 0)
- {
- destroy_coeff_list (cl);
- goto error;
- }
-
- ll_push_tail (&oneway.contrast_list, &cl->ll);
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- oneway.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- oneway.exclude = MV_ANY;
- else if (lex_match_id (lexer, "LISTWISE"))
- oneway.missing_type = MISS_LISTWISE;
- else if (lex_match_id (lexer, "ANALYSIS"))
- oneway.missing_type = MISS_ANALYSIS;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE",
- "LISTWISE", "ANALYSIS");
- goto error;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "STATISTICS", "POSTHOC", "CONTRAST",
- "MISSING");
- goto error;
- }
- }
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- run_oneway (&oneway, group, ds);
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- oneway_cleanup (&oneway);
- free (oneway.vars);
- return CMD_SUCCESS;
-
- error:
- oneway_cleanup (&oneway);
- free (oneway.vars);
- return CMD_FAILURE;
-}
-\f
-static struct descriptive_data *
-dd_create (const struct variable *var)
-{
- struct descriptive_data *dd = xmalloc (sizeof *dd);
-
- dd->mom = moments1_create (MOMENT_VARIANCE);
- dd->minimum = DBL_MAX;
- dd->maximum = -DBL_MAX;
- dd->var = var;
-
- return dd;
-}
-
-static void
-dd_destroy (struct descriptive_data *dd)
-{
- moments1_destroy (dd->mom);
- free (dd);
-}
-
-static void *
-makeit (const void *aux1, void *aux2 UNUSED)
-{
- const struct variable *var = aux1;
-
- struct descriptive_data *dd = dd_create (var);
-
- return dd;
-}
-
-static void
-killit (const void *aux1 UNUSED, void *aux2 UNUSED, void *user_data)
-{
- struct descriptive_data *dd = user_data;
-
- dd_destroy (dd);
-}
-
-
-static void
-updateit (const void *aux1, void *aux2, void *user_data,
- const struct ccase *c, double weight)
-{
- struct descriptive_data *dd = user_data;
-
- const struct variable *varp = aux1;
-
- const union value *valx = case_data (c, varp);
-
- struct descriptive_data *dd_total = aux2;
-
- moments1_add (dd->mom, valx->f, weight);
- if (valx->f < dd->minimum)
- dd->minimum = valx->f;
-
- if (valx->f > dd->maximum)
- dd->maximum = valx->f;
-
- {
- const struct variable *var = dd_total->var;
- const union value *val = case_data (c, var);
-
- moments1_add (dd_total->mom,
- val->f,
- weight);
-
- if (val->f < dd_total->minimum)
- dd_total->minimum = val->f;
-
- if (val->f > dd_total->maximum)
- dd_total->maximum = val->f;
- }
-}
-
-static void
-run_oneway (const struct oneway_spec *cmd, struct casereader *input,
- const struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct oneway_workspace ws = {
- .vws = xcalloc (cmd->n_vars, sizeof *ws.vws),
- .dd_total = XCALLOC (cmd->n_vars, struct descriptive_data *),
- };
-
- for (size_t v = 0; v < cmd->n_vars; ++v)
- ws.dd_total[v] = dd_create (cmd->vars[v]);
-
- for (size_t v = 0; v < cmd->n_vars; ++v)
- {
- static const struct payload payload =
- {
- .create = makeit,
- .update = updateit,
- .destroy = killit,
- };
-
- ws.vws[v].iact = interaction_create (cmd->indep_var);
- ws.vws[v].cat = categoricals_create (&ws.vws[v].iact, 1, cmd->wv,
- cmd->exclude);
-
- categoricals_set_payload (ws.vws[v].cat, &payload,
- CONST_CAST (struct variable *, cmd->vars[v]),
- ws.dd_total[v]);
-
-
- ws.vws[v].cov = covariance_2pass_create (1, &cmd->vars[v],
- ws.vws[v].cat,
- cmd->wv, cmd->exclude, true);
- ws.vws[v].nl = levene_create (var_get_width (cmd->indep_var), NULL);
- }
-
- output_split_file_values_peek (ds, input);
-
- struct taint *taint = taint_clone (casereader_get_taint (input));
- input = casereader_create_filter_missing (input, &cmd->indep_var, 1,
- cmd->exclude, NULL, NULL);
- if (cmd->missing_type == MISS_LISTWISE)
- input = casereader_create_filter_missing (input, cmd->vars, cmd->n_vars,
- cmd->exclude, NULL, NULL);
- input = casereader_create_filter_weight (input, dict, NULL, NULL);
-
- struct casereader *reader = casereader_clone (input);
- struct ccase *c;
- for (; (c = casereader_read (reader)) != NULL; case_unref (c))
- {
- double w = dict_get_case_weight (dict, c, NULL);
-
- for (size_t i = 0; i < cmd->n_vars; ++i)
- {
- struct per_var_ws *pvw = &ws.vws[i];
- const struct variable *v = cmd->vars[i];
- const union value *val = case_data (c, v);
-
- if (MISS_ANALYSIS == cmd->missing_type)
- {
- if (var_is_value_missing (v, val) & cmd->exclude)
- continue;
- }
-
- covariance_accumulate_pass1 (pvw->cov, c);
- levene_pass_one (pvw->nl, val->f, w, case_data (c, cmd->indep_var));
- }
- }
- casereader_destroy (reader);
-
- reader = casereader_clone (input);
- for (; (c = casereader_read (reader)); case_unref (c))
- {
- double w = dict_get_case_weight (dict, c, NULL);
- for (size_t i = 0; i < cmd->n_vars; ++i)
- {
- struct per_var_ws *pvw = &ws.vws[i];
- const struct variable *v = cmd->vars[i];
- const union value *val = case_data (c, v);
-
- if (MISS_ANALYSIS == cmd->missing_type)
- {
- if (var_is_value_missing (v, val) & cmd->exclude)
- continue;
- }
-
- covariance_accumulate_pass2 (pvw->cov, c);
- levene_pass_two (pvw->nl, val->f, w, case_data (c, cmd->indep_var));
- }
- }
- casereader_destroy (reader);
-
- reader = casereader_clone (input);
- for (; (c = casereader_read (reader)); case_unref (c))
- {
- double w = dict_get_case_weight (dict, c, NULL);
-
- for (size_t i = 0; i < cmd->n_vars; ++i)
- {
- struct per_var_ws *pvw = &ws.vws[i];
- const struct variable *v = cmd->vars[i];
- const union value *val = case_data (c, v);
-
- if (MISS_ANALYSIS == cmd->missing_type)
- {
- if (var_is_value_missing (v, val) & cmd->exclude)
- continue;
- }
-
- levene_pass_three (pvw->nl, val->f, w, case_data (c, cmd->indep_var));
- }
- }
- casereader_destroy (reader);
-
- for (size_t v = 0; v < cmd->n_vars; ++v)
- {
- struct per_var_ws *pvw = &ws.vws[v];
-
- const struct categoricals *cats = covariance_get_categoricals (pvw->cov);
- if (!categoricals_sane (cats))
- {
- msg (MW, _("Dependent variable %s has no non-missing values. "
- "No analysis for this variable will be done."),
- var_get_name (cmd->vars[v]));
- continue;
- }
-
- const gsl_matrix *ucm = covariance_calculate_unnormalized (pvw->cov);
-
- gsl_matrix *cm = gsl_matrix_alloc (ucm->size1, ucm->size2);
- gsl_matrix_memcpy (cm, ucm);
-
- moments1_calculate (ws.dd_total[v]->mom, &pvw->n, NULL, NULL, NULL, NULL);
-
- pvw->sst = gsl_matrix_get (cm, 0, 0);
-
- reg_sweep (cm, 0);
-
- pvw->sse = gsl_matrix_get (cm, 0, 0);
- gsl_matrix_free (cm);
-
- pvw->ssa = pvw->sst - pvw->sse;
-
- pvw->n_groups = categoricals_n_total (cats);
-
- pvw->mse = (pvw->sst - pvw->ssa) / (pvw->n - pvw->n_groups);
- }
-
- for (size_t v = 0; v < cmd->n_vars; ++v)
- {
- const struct categoricals *cats = covariance_get_categoricals (ws.vws[v].cov);
- if (categoricals_is_complete (cats))
- {
- if (categoricals_n_total (cats) > ws.actual_number_of_groups)
- ws.actual_number_of_groups = categoricals_n_total (cats);
- }
- }
- casereader_destroy (input);
-
- if (!taint_has_tainted_successor (taint))
- output_oneway (cmd, &ws);
-
- taint_destroy (taint);
-
- for (size_t v = 0; v < cmd->n_vars; ++v)
- {
- covariance_destroy (ws.vws[v].cov);
- levene_destroy (ws.vws[v].nl);
- dd_destroy (ws.dd_total[v]);
- interaction_destroy (ws.vws[v].iact);
- }
-
- free (ws.vws);
- free (ws.dd_total);
-}
-
-static void show_contrast_coeffs (const struct oneway_spec *, const struct oneway_workspace *);
-static void show_contrast_tests (const struct oneway_spec *, const struct oneway_workspace *);
-static void show_comparisons (const struct oneway_spec *, const struct oneway_workspace *, int depvar);
-
-static void
-output_oneway (const struct oneway_spec *cmd, struct oneway_workspace *ws)
-{
- size_t list_idx = 0;
-
- /* Check the sanity of the given contrast values */
- struct contrasts_node *coeff_list = NULL;
- struct contrasts_node *coeff_next = NULL;
- ll_for_each_safe (coeff_list, coeff_next, struct contrasts_node, ll, &cmd->contrast_list)
- {
- struct coeff_node *cn = NULL;
- double sum = 0;
- struct ll_list *cl = &coeff_list->coefficient_list;
- ++list_idx;
-
- if (ll_count (cl) != ws->actual_number_of_groups)
- {
- msg (SW,
- _("In contrast list %zu, the number of coefficients (%zu) does not equal the number of groups (%d). This contrast list will be ignored."),
- list_idx, ll_count (cl), ws->actual_number_of_groups);
-
- ll_remove (&coeff_list->ll);
- destroy_coeff_list (coeff_list);
- continue;
- }
-
- ll_for_each (cn, struct coeff_node, ll, cl)
- sum += cn->coeff;
-
- if (sum != 0.0)
- msg (SW, _("Coefficients for contrast %zu do not total zero"),
- list_idx);
- }
-
- if (cmd->descriptive_stats)
- show_descriptives (cmd, ws);
-
- if (cmd->homogeneity_stats)
- show_homogeneity (cmd, ws);
-
- show_anova_table (cmd, ws);
-
- if (ll_count (&cmd->contrast_list) > 0)
- {
- show_contrast_coeffs (cmd, ws);
- show_contrast_tests (cmd, ws);
- }
-
- if (cmd->posthoc)
- for (size_t v = 0; v < cmd->n_vars; ++v)
- {
- const struct categoricals *cats = covariance_get_categoricals (ws->vws[v].cov);
-
- if (categoricals_is_complete (cats))
- show_comparisons (cmd, ws, v);
- }
-}
-
-
-/* Show the ANOVA table */
-static void
-show_anova_table (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
-{
- struct pivot_table *table = pivot_table_create (N_("ANOVA"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Sum of Squares"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Mean Square"), PIVOT_RC_OTHER,
- N_("F"), PIVOT_RC_OTHER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Type"),
- N_("Between Groups"), N_("Within Groups"),
- N_("Total"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0; i < cmd->n_vars; ++i)
- {
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (cmd->vars[i]));
-
- const struct per_var_ws *pvw = &ws->vws[i];
-
- double n;
- moments1_calculate (ws->dd_total[i]->mom, &n, NULL, NULL, NULL, NULL);
-
- double df1 = pvw->n_groups - 1;
- double df2 = n - pvw->n_groups;
- double msa = pvw->ssa / df1;
- double F = msa / pvw->mse;
-
- struct entry
- {
- int stat_idx;
- int type_idx;
- double x;
- }
- entries[] = {
- /* Sums of Squares. */
- { 0, 0, pvw->ssa },
- { 0, 1, pvw->sse },
- { 0, 2, pvw->sst },
- /* Degrees of Freedom. */
- { 1, 0, df1 },
- { 1, 1, df2 },
- { 1, 2, n - 1 },
- /* Mean Squares. */
- { 2, 0, msa },
- { 2, 1, pvw->mse },
- /* F. */
- { 3, 0, F },
- /* Significance. */
- { 4, 0, gsl_cdf_fdist_Q (F, df1, df2) },
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- {
- const struct entry *e = &entries[j];
- pivot_table_put3 (table, e->stat_idx, e->type_idx, var_idx,
- pivot_value_new_number (e->x));
- }
- }
-
- pivot_table_submit (table);
-}
-
-/* Show the descriptives table */
-static void
-show_descriptives (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
-{
- if (!cmd->n_vars)
- return;
- const struct categoricals *cats = covariance_get_categoricals (
- ws->vws[0].cov);
-
- struct pivot_table *table = pivot_table_create (N_("Descriptives"));
- pivot_table_set_weight_format (table, cmd->wfmt);
-
- const double confidence = 0.95;
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT, N_("Mean"), N_("Std. Deviation"),
- N_("Std. Error"));
- struct pivot_category *interval = pivot_category_create_group__ (
- statistics->root,
- pivot_value_new_text_format (N_("%g%% Confidence Interval for Mean"),
- confidence * 100.0));
- pivot_category_create_leaves (interval, N_("Lower Bound"),
- N_("Upper Bound"));
- pivot_category_create_leaves (statistics->root,
- N_("Minimum"), N_("Maximum"));
-
- struct pivot_dimension *indep_var = pivot_dimension_create__ (
- table, PIVOT_AXIS_ROW, pivot_value_new_variable (cmd->indep_var));
- indep_var->root->show_label = true;
- size_t n;
- union value *values = categoricals_get_var_values (cats, cmd->indep_var, &n);
- for (size_t j = 0; j < n; j++)
- pivot_category_create_leaf (
- indep_var->root, pivot_value_new_var_value (cmd->indep_var, &values[j]));
- pivot_category_create_leaf (
- indep_var->root, pivot_value_new_text_format (N_("Total")));
-
- struct pivot_dimension *dep_var = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variable"));
-
- const double q = (1.0 - confidence) / 2.0;
- for (int v = 0; v < cmd->n_vars; ++v)
- {
- int dep_var_idx = pivot_category_create_leaf (
- dep_var->root, pivot_value_new_variable (cmd->vars[v]));
-
- struct per_var_ws *pvw = &ws->vws[v];
- const struct categoricals *cats = covariance_get_categoricals (pvw->cov);
-
- int count;
- for (count = 0; count < categoricals_n_total (cats); ++count)
- {
- const struct descriptive_data *dd
- = categoricals_get_user_data_by_category (cats, count);
-
- double n, mean, variance;
- moments1_calculate (dd->mom, &n, &mean, &variance, NULL, NULL);
-
- double std_dev = sqrt (variance);
- double std_error = std_dev / sqrt (n);
- double T = gsl_cdf_tdist_Qinv (q, n - 1);
-
- double entries[] = {
- n,
- mean,
- std_dev,
- std_error,
- mean - T * std_error,
- mean + T * std_error,
- dd->minimum,
- dd->maximum,
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put3 (table, i, count, dep_var_idx,
- pivot_value_new_number (entries[i]));
- }
-
- if (categoricals_is_complete (cats))
- {
- double n, mean, variance;
- moments1_calculate (ws->dd_total[v]->mom, &n, &mean, &variance,
- NULL, NULL);
-
- double std_dev = sqrt (variance);
- double std_error = std_dev / sqrt (n);
- double T = gsl_cdf_tdist_Qinv (q, n - 1);
-
- double entries[] = {
- n,
- mean,
- std_dev,
- std_error,
- mean - T * std_error,
- mean + T * std_error,
- ws->dd_total[v]->minimum,
- ws->dd_total[v]->maximum,
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put3 (table, i, count, dep_var_idx,
- pivot_value_new_number (entries[i]));
- }
- }
-
- pivot_table_submit (table);
-}
-
-/* Show the homogeneity table */
-static void
-show_homogeneity (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Test of Homogeneity of Variances"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Levene Statistic"), PIVOT_RC_OTHER,
- N_("df1"), PIVOT_RC_INTEGER,
- N_("df2"), PIVOT_RC_INTEGER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (int v = 0; v < cmd->n_vars; ++v)
- {
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (cmd->vars[v]));
-
- double n;
- moments1_calculate (ws->dd_total[v]->mom, &n, NULL, NULL, NULL, NULL);
-
- const struct per_var_ws *pvw = &ws->vws[v];
- double df1 = pvw->n_groups - 1;
- double df2 = n - pvw->n_groups;
- double F = levene_calculate (pvw->nl);
-
- double entries[] =
- {
- F,
- df1,
- df2,
- gsl_cdf_fdist_Q (F, df1, df2),
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, var_idx,
- pivot_value_new_number (entries[i]));
- }
-
- pivot_table_submit (table);
-}
-
-
-/* Show the contrast coefficients table */
-static void
-show_contrast_coeffs (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
-{
- struct pivot_table *table = pivot_table_create (N_("Contrast Coefficients"));
-
- struct pivot_dimension *indep_var = pivot_dimension_create__ (
- table, PIVOT_AXIS_COLUMN, pivot_value_new_variable (cmd->indep_var));
- indep_var->root->show_label = true;
-
- struct pivot_dimension *contrast = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Contrast"));
- contrast->root->show_label = true;
-
- const struct covariance *cov = ws->vws[0].cov;
-
- const struct contrasts_node *cn;
- int c_num = 1;
- ll_for_each (cn, struct contrasts_node, ll, &cmd->contrast_list)
- {
- int contrast_idx = pivot_category_create_leaf (
- contrast->root, pivot_value_new_integer (c_num++));
-
- const struct coeff_node *coeffn;
- int indep_idx = 0;
- ll_for_each (coeffn, struct coeff_node, ll, &cn->coefficient_list)
- {
- const struct categoricals *cats = covariance_get_categoricals (cov);
- const struct ccase *gcc = categoricals_get_case_by_category (
- cats, indep_idx);
-
- if (!contrast_idx)
- pivot_category_create_leaf (
- indep_var->root, pivot_value_new_var_value (
- cmd->indep_var, case_data (gcc, cmd->indep_var)));
-
- pivot_table_put2 (table, indep_idx++, contrast_idx,
- pivot_value_new_integer (coeffn->coeff));
- }
- }
-
- pivot_table_submit (table);
-}
-
-/* Show the results of the contrast tests */
-static void
-show_contrast_tests (const struct oneway_spec *cmd, const struct oneway_workspace *ws)
-{
- struct pivot_table *table = pivot_table_create (N_("Contrast Tests"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Value of Contrast"), PIVOT_RC_OTHER,
- N_("Std. Error"), PIVOT_RC_OTHER,
- N_("t"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_OTHER,
- N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *contrasts = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Contrast"));
- contrasts->root->show_label = true;
- int n_contrasts = ll_count (&cmd->contrast_list);
- for (int i = 1; i <= n_contrasts; i++)
- pivot_category_create_leaf (contrasts->root, pivot_value_new_integer (i));
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Assumption"),
- N_("Assume equal variances"),
- N_("Does not assume equal variances"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variable"));
-
- for (int v = 0; v < cmd->n_vars; ++v)
- {
- const struct per_var_ws *pvw = &ws->vws[v];
- const struct categoricals *cats = covariance_get_categoricals (pvw->cov);
- if (!categoricals_is_complete (cats))
- continue;
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (cmd->vars[v]));
-
- struct contrasts_node *cn;
- int contrast_idx = 0;
- ll_for_each (cn, struct contrasts_node, ll, &cmd->contrast_list)
- {
-
- /* Note: The calculation of the degrees of freedom in the
- "variances not equal" case is painfull!!
- The following formula may help to understand it:
- \frac{\left (\sum_{i=1}^k{c_i^2\frac{s_i^2}{n_i}}\right)^2}
- {
- \sum_{i=1}^k\left (
- \frac{\left (c_i^2\frac{s_i^2}{n_i}\right)^2} {n_i-1}
- \right)
- }
- */
-
- double grand_n;
- moments1_calculate (ws->dd_total[v]->mom, &grand_n, NULL, NULL,
- NULL, NULL);
- double df = grand_n - pvw->n_groups;
-
- double contrast_value = 0.0;
- double coef_msq = 0.0;
- double sec_vneq = 0.0;
- double df_denominator = 0.0;
- double df_numerator = 0.0;
- struct coeff_node *coeffn;
- int ci = 0;
- ll_for_each (coeffn, struct coeff_node, ll, &cn->coefficient_list)
- {
- const struct descriptive_data *dd
- = categoricals_get_user_data_by_category (cats, ci);
- const double coef = coeffn->coeff;
-
- double n, mean, variance;
- moments1_calculate (dd->mom, &n, &mean, &variance, NULL, NULL);
-
- double winv = variance / n;
- contrast_value += coef * mean;
- coef_msq += pow2 (coef) / n;
- sec_vneq += pow2 (coef) * variance / n;
- df_numerator += pow2 (coef) * winv;
- df_denominator += pow2(pow2 (coef) * winv) / (n - 1);
-
- ci++;
- }
- sec_vneq = sqrt (sec_vneq);
- df_numerator = pow2 (df_numerator);
-
- double std_error_contrast = sqrt (pvw->mse * coef_msq);
- double T = contrast_value / std_error_contrast;
- double T_ne = contrast_value / sec_vneq;
- double df_ne = df_numerator / df_denominator;
-
- struct entry
- {
- int stat_idx;
- int assumption_idx;
- double x;
- }
- entries[] =
- {
- /* Assume equal. */
- { 0, 0, contrast_value },
- { 1, 0, std_error_contrast },
- { 2, 0, T },
- { 3, 0, df },
- { 4, 0, 2 * gsl_cdf_tdist_Q (fabs(T), df) },
- /* Do not assume equal. */
- { 0, 1, contrast_value },
- { 1, 1, sec_vneq },
- { 2, 1, T_ne },
- { 3, 1, df_ne },
- { 4, 1, 2 * gsl_cdf_tdist_Q (fabs(T_ne), df_ne) },
- };
-
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put4 (
- table, e->stat_idx, contrast_idx, e->assumption_idx, var_idx,
- pivot_value_new_number (e->x));
- }
-
- contrast_idx++;
- }
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_comparisons (const struct oneway_spec *cmd, const struct oneway_workspace *ws, int v)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_user_text_nocopy (xasprintf (
- _("Multiple Comparisons (%s)"),
- var_to_string (cmd->vars[v]))),
- "Multiple Comparisons");
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Mean Difference (I - J)"), PIVOT_RC_OTHER,
- N_("Std. Error"), PIVOT_RC_OTHER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
- struct pivot_category *interval = pivot_category_create_group__ (
- statistics->root,
- pivot_value_new_text_format (N_("%g%% Confidence Interval"),
- (1 - cmd->alpha) * 100.0));
- pivot_category_create_leaves (interval,
- N_("Lower Bound"), PIVOT_RC_OTHER,
- N_("Upper Bound"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *j_family = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("(J) Family"));
- j_family->root->show_label = true;
-
- struct pivot_dimension *i_family = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("(J) Family"));
- i_family->root->show_label = true;
-
- const struct per_var_ws *pvw = &ws->vws[v];
- const struct categoricals *cat = pvw->cat;
- for (int i = 0; i < pvw->n_groups; i++)
- {
- const struct ccase *gcc = categoricals_get_case_by_category (cat, i);
- for (int j = 0; j < 2; j++)
- pivot_category_create_leaf (
- j ? j_family->root : i_family->root,
- pivot_value_new_var_value (cmd->indep_var,
- case_data (gcc, cmd->indep_var)));
- }
-
- struct pivot_dimension *test = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Test"));
-
- for (int p = 0; p < cmd->n_posthoc; ++p)
- {
- const struct posthoc *ph = &ph_tests[cmd->posthoc[p]];
-
- int test_idx = pivot_category_create_leaf (
- test->root, pivot_value_new_text (ph->label));
-
- for (int i = 0; i < pvw->n_groups; ++i)
- {
- struct descriptive_data *dd_i
- = categoricals_get_user_data_by_category (cat, i);
- double weight_i, mean_i, var_i;
- moments1_calculate (dd_i->mom, &weight_i, &mean_i, &var_i, 0, 0);
-
- for (int j = 0; j < pvw->n_groups; ++j)
- {
- if (j == i)
- continue;
-
- struct descriptive_data *dd_j
- = categoricals_get_user_data_by_category (cat, j);
- double weight_j, mean_j, var_j;
- moments1_calculate (dd_j->mom, &weight_j, &mean_j, &var_j, 0, 0);
-
- double std_err = pvw->mse;
- std_err *= weight_i + weight_j;
- std_err /= weight_i * weight_j;
- std_err = sqrt (std_err);
-
- double sig = 2 * multiple_comparison_sig (std_err, pvw,
- dd_i, dd_j, ph);
- double half_range = mc_half_range (cmd, pvw, std_err,
- dd_i, dd_j, ph);
- double entries[] = {
- mean_i - mean_j,
- std_err,
- sig,
- (mean_i - mean_j) - half_range,
- (mean_i - mean_j) + half_range,
- };
- for (size_t k = 0; k < sizeof entries / sizeof *entries; k++)
- pivot_table_put4 (table, k, j, i, test_idx,
- pivot_value_new_number (entries[k]));
- }
- }
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011, 2012, 2015, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_matrix.h>
-#include <gsl/gsl_permutation.h>
-#include <gsl/gsl_sort_vector.h>
-#include <gsl/gsl_statistics.h>
-#include <stdio.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/assertion.h"
-#include "libpspp/str.h"
-#include "math/random.h"
-#include "output/pivot-table.h"
-#include "output/output-item.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-enum missing_type
- {
- MISS_LISTWISE,
- MISS_PAIRWISE,
- };
-
-
-struct save_trans_data
- {
- /* A writer which contains the values (if any) to be appended to
- each case in the active dataset */
- struct casewriter *writer;
-
- /* A reader created from the writer above. */
- struct casereader *appending_reader;
-
- /* The indices to be used to access values in the above,
- reader/writer */
- int membership_case_idx;
- int distance_case_idx;
-
- /* The variables created to hold the values appended to the dataset */
- struct variable *membership;
- struct variable *distance;
- };
-
-
-struct qc
- {
- struct dataset *dataset;
- struct dictionary *dict;
-
- const struct variable **vars;
- size_t n_vars;
-
- double epsilon; /* The convergence criterion */
-
- int ngroups; /* Number of group. (Given by the user) */
- int maxiter; /* Maximum iterations (Given by the user) */
- bool print_cluster_membership; /* true => print membership */
- bool print_initial_clusters; /* true => print initial cluster */
- bool initial; /* false => simplified initial cluster selection */
- bool update; /* false => do not iterate */
-
- const struct variable *wv; /* Weighting variable. */
-
- enum missing_type missing_type;
- enum mv_class exclude;
-
- /* Which values are to be saved? */
- bool save_membership;
- bool save_distance;
-
- /* The name of the new variable to contain the cluster of each case. */
- char *var_membership;
-
- /* The name of the new variable to contain the distance of each case
- from its cluster centre. */
- char *var_distance;
-
- struct save_trans_data *save_trans_data;
- };
-
-/* Holds all of the information for the functions. int n, holds the number of
- observation and its default value is -1. We set it in
- kmeans_recalculate_centers in first invocation. */
-struct Kmeans
- {
- gsl_matrix *centers; /* Centers for groups. */
- gsl_matrix *updated_centers;
- casenumber n;
-
- gsl_vector_long *num_elements_groups;
-
- gsl_matrix *initial_centers; /* Initial random centers. */
- double convergence_criteria;
- gsl_permutation *group_order; /* Group order for reporting. */
- };
-
-static struct Kmeans *kmeans_create (const struct qc *);
-
-static void kmeans_get_nearest_group (const struct Kmeans *,
- struct ccase *, const struct qc *,
- int *, double *, int *, double *);
-
-static void kmeans_order_groups (struct Kmeans *, const struct qc *);
-
-static void kmeans_cluster (struct Kmeans *, struct casereader *,
- const struct qc *);
-
-static void quick_cluster_show_centers (struct Kmeans *, bool initial,
- const struct qc *);
-
-static void quick_cluster_show_membership (struct Kmeans *,
- const struct casereader *,
- struct qc *);
-
-static void quick_cluster_show_number_cases (struct Kmeans *,
- const struct qc *);
-
-static void quick_cluster_show_results (struct Kmeans *,
- const struct casereader *,
- struct qc *);
-
-int cmd_quick_cluster (struct lexer *, struct dataset *);
-
-static void kmeans_destroy (struct Kmeans *);
-
-/* Creates and returns a struct of Kmeans with given casereader 'cs', parsed
- variables 'variables', number of cases 'n', number of variables 'm', number
- of clusters and amount of maximum iterations. */
-static struct Kmeans *
-kmeans_create (const struct qc *qc)
-{
- struct Kmeans *kmeans = xmalloc (sizeof *kmeans);
- *kmeans = (struct Kmeans) {
- .centers = gsl_matrix_alloc (qc->ngroups, qc->n_vars),
- .updated_centers = gsl_matrix_alloc (qc->ngroups, qc->n_vars),
- .num_elements_groups = gsl_vector_long_alloc (qc->ngroups),
- .group_order = gsl_permutation_alloc (qc->ngroups),
- };
- return kmeans;
-}
-
-static void
-kmeans_destroy (struct Kmeans *kmeans)
-{
- gsl_matrix_free (kmeans->centers);
- gsl_matrix_free (kmeans->updated_centers);
- gsl_matrix_free (kmeans->initial_centers);
-
- gsl_vector_long_free (kmeans->num_elements_groups);
-
- gsl_permutation_free (kmeans->group_order);
-
- free (kmeans);
-}
-
-static double
-diff_matrix (const gsl_matrix *m1, const gsl_matrix *m2)
-{
- double max_diff = -INFINITY;
- for (size_t i = 0; i < m1->size1; ++i)
- {
- double diff = 0;
- for (size_t j = 0; j < m1->size2; ++j)
- diff += pow2 (gsl_matrix_get (m1,i,j) - gsl_matrix_get (m2,i,j));
- if (diff > max_diff)
- max_diff = diff;
- }
-
- return max_diff;
-}
-
-
-
-static double
-matrix_mindist (const gsl_matrix *m, int *mn, int *mm)
-{
- double mindist = INFINITY;
- for (size_t i = 0; i + 1 < m->size1; ++i)
- for (size_t j = i + 1; j < m->size1; ++j)
- {
- double diff_sq = 0;
- for (size_t k = 0; k < m->size2; ++k)
- diff_sq += pow2 (gsl_matrix_get (m, j, k) - gsl_matrix_get (m, i, k));
- if (diff_sq < mindist)
- {
- mindist = diff_sq;
- if (mn)
- *mn = i;
- if (mm)
- *mm = j;
- }
- }
- return mindist;
-}
-
-/* Return the distance of C from the group whose index is WHICH */
-static double
-dist_from_case (const struct Kmeans *kmeans, const struct ccase *c,
- const struct qc *qc, int which)
-{
- double dist = 0;
- for (size_t j = 0; j < qc->n_vars; j++)
- {
- const union value *val = case_data (c, qc->vars[j]);
- assert (!(var_is_value_missing (qc->vars[j], val) & qc->exclude));
- dist += pow2 (gsl_matrix_get (kmeans->centers, which, j) - val->f);
- }
-
- return dist;
-}
-
-/* Return the minimum distance of the group WHICH and all other groups */
-static double
-min_dist_from (const struct Kmeans *kmeans, const struct qc *qc, int which)
-{
- double mindist = INFINITY;
- for (size_t i = 0; i < qc->ngroups; i++)
- {
- if (i == which)
- continue;
-
- double dist = 0;
- for (size_t j = 0; j < qc->n_vars; j++)
- dist += pow2 (gsl_matrix_get (kmeans->centers, i, j)
- - gsl_matrix_get (kmeans->centers, which, j));
-
- if (dist < mindist)
- mindist = dist;
- }
-
- return mindist;
-}
-
-/* Calculate the initial cluster centers. */
-static void
-kmeans_initial_centers (struct Kmeans *kmeans,
- const struct casereader *reader,
- const struct qc *qc)
-{
- int nc = 0;
-
- struct casereader *cs = casereader_clone (reader);
- struct ccase *c;
- for (; (c = casereader_read (cs)) != NULL; case_unref (c))
- {
- bool missing = false;
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- const union value *val = case_data (c, qc->vars[j]);
- if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
- {
- missing = true;
- break;
- }
-
- if (nc < qc->ngroups)
- gsl_matrix_set (kmeans->centers, nc, j, val->f);
- }
- if (missing)
- continue;
-
- if (nc++ < qc->ngroups)
- continue;
-
- if (qc->initial)
- {
- int mn, mm;
- double m = matrix_mindist (kmeans->centers, &mn, &mm);
-
- int mq, mp;
- double delta;
- kmeans_get_nearest_group (kmeans, c, qc, &mq, &delta, &mp, NULL);
- if (delta > m)
- /* If the distance between C and the nearest group, is greater than the distance
- between the two groups which are clostest to each
- other, then one group must be replaced. */
- {
- /* Out of mn and mm, which is the clostest of the two groups to C ? */
- int which = (dist_from_case (kmeans, c, qc, mn)
- > dist_from_case (kmeans, c, qc, mm)) ? mm : mn;
-
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- const union value *val = case_data (c, qc->vars[j]);
- gsl_matrix_set (kmeans->centers, which, j, val->f);
- }
- }
- else if (dist_from_case (kmeans, c, qc, mp) > min_dist_from (kmeans, qc, mq))
- /* If the distance between C and the second nearest group
- (MP) is greater than the smallest distance between the
- nearest group (MQ) and any other group, then replace
- MQ with C. */
- {
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- const union value *val = case_data (c, qc->vars[j]);
- gsl_matrix_set (kmeans->centers, mq, j, val->f);
- }
- }
- }
- }
-
- casereader_destroy (cs);
-
- kmeans->convergence_criteria = qc->epsilon * matrix_mindist (kmeans->centers, NULL, NULL);
-
- /* As it is the first iteration, the variable kmeans->initial_centers is NULL
- and it is created once for reporting issues. */
- kmeans->initial_centers = gsl_matrix_alloc (qc->ngroups, qc->n_vars);
- gsl_matrix_memcpy (kmeans->initial_centers, kmeans->centers);
-}
-
-/* Return the index of the group which is nearest to the case C */
-static void
-kmeans_get_nearest_group (const struct Kmeans *kmeans, struct ccase *c,
- const struct qc *qc, int *g_q, double *delta_q,
- int *g_p, double *delta_p)
-{
- int result0 = -1;
- int result1 = -1;
- double mindist0 = INFINITY;
- double mindist1 = INFINITY;
- for (size_t i = 0; i < qc->ngroups; i++)
- {
- double dist = 0;
- for (size_t j = 0; j < qc->n_vars; j++)
- {
- const union value *val = case_data (c, qc->vars[j]);
- if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
- continue;
-
- dist += pow2 (gsl_matrix_get (kmeans->centers, i, j) - val->f);
- }
-
- if (dist < mindist0)
- {
- mindist1 = mindist0;
- result1 = result0;
-
- mindist0 = dist;
- result0 = i;
- }
- else if (dist < mindist1)
- {
- mindist1 = dist;
- result1 = i;
- }
- }
-
- if (delta_q)
- *delta_q = mindist0;
-
- if (g_q)
- *g_q = result0;
-
- if (delta_p)
- *delta_p = mindist1;
-
- if (g_p)
- *g_p = result1;
-}
-
-static void
-kmeans_order_groups (struct Kmeans *kmeans, const struct qc *qc)
-{
- gsl_vector *v = gsl_vector_alloc (qc->ngroups);
- gsl_matrix_get_col (v, kmeans->centers, 0);
- gsl_sort_vector_index (kmeans->group_order, v);
- gsl_vector_free (v);
-}
-
-/* Main algorithm.
- Does iterations, checks convergency. */
-static void
-kmeans_cluster (struct Kmeans *kmeans, struct casereader *reader,
- const struct qc *qc)
-{
- kmeans_initial_centers (kmeans, reader, qc);
-
- gsl_matrix_memcpy (kmeans->updated_centers, kmeans->centers);
- for (int xx = 0; xx < qc->maxiter; ++xx)
- {
- gsl_vector_long_set_all (kmeans->num_elements_groups, 0.0);
-
- kmeans->n = 0;
- if (qc->update)
- {
- struct casereader *r = casereader_clone (reader);
- struct ccase *c;
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- bool missing = false;
- for (size_t j = 0; j < qc->n_vars; j++)
- {
- const union value *val = case_data (c, qc->vars[j]);
- if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
- missing = true;
- }
- if (missing)
- continue;
-
- double mindist = INFINITY;
- int group = -1;
- for (size_t g = 0; g < qc->ngroups; ++g)
- {
- double d = dist_from_case (kmeans, c, qc, g);
-
- if (d < mindist)
- {
- mindist = d;
- group = g;
- }
- }
-
- long *n = gsl_vector_long_ptr (kmeans->num_elements_groups, group);
- *n += qc->wv ? case_num (c, qc->wv) : 1.0;
- kmeans->n++;
-
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- const union value *val = case_data (c, qc->vars[j]);
- if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
- continue;
- double *x = gsl_matrix_ptr (kmeans->updated_centers, group, j);
- *x += val->f * (qc->wv ? case_num (c, qc->wv) : 1.0);
- }
- }
-
- casereader_destroy (r);
- }
-
- /* Divide the cluster sums by the number of items in each cluster */
- for (size_t g = 0; g < qc->ngroups; ++g)
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- long n = gsl_vector_long_get (kmeans->num_elements_groups, g);
- double *x = gsl_matrix_ptr (kmeans->updated_centers, g, j);
- *x /= n + 1; // Plus 1 for the initial centers
- }
- gsl_matrix_memcpy (kmeans->centers, kmeans->updated_centers);
-
- kmeans->n = 0;
- /* Step 3 */
- gsl_vector_long_set_all (kmeans->num_elements_groups, 0.0);
- gsl_matrix_set_all (kmeans->updated_centers, 0.0);
- struct ccase *c;
- struct casereader *cs = casereader_clone (reader);
- for (; (c = casereader_read (cs)) != NULL; case_unref (c))
- {
- int group = -1;
- kmeans_get_nearest_group (kmeans, c, qc, &group, NULL, NULL, NULL);
-
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- const union value *val = case_data (c, qc->vars[j]);
- if (var_is_value_missing (qc->vars[j], val) & qc->exclude)
- continue;
-
- double *x = gsl_matrix_ptr (kmeans->updated_centers, group, j);
- *x += val->f;
- }
-
- long *n = gsl_vector_long_ptr (kmeans->num_elements_groups, group);
- *n += qc->wv ? case_num (c, qc->wv) : 1.0;
- kmeans->n++;
- }
- casereader_destroy (cs);
-
- /* Divide the cluster sums by the number of items in each cluster */
- for (size_t g = 0; g < qc->ngroups; ++g)
- for (size_t j = 0; j < qc->n_vars; ++j)
- {
- long n = gsl_vector_long_get (kmeans->num_elements_groups, g);
- double *x = gsl_matrix_ptr (kmeans->updated_centers, g, j);
- *x /= n;
- }
-
- double d = diff_matrix (kmeans->updated_centers, kmeans->centers);
- if (d < kmeans->convergence_criteria)
- break;
-
- if (!qc->update)
- break;
- }
-}
-
-/* Reports centers of clusters.
- Initial parameter is optional for future use.
- If initial is true, initial cluster centers are reported. Otherwise,
- resulted centers are reported. */
-static void
-quick_cluster_show_centers (struct Kmeans *kmeans, bool initial, const struct qc *qc)
-{
- struct pivot_table *table
- = pivot_table_create (initial
- ? N_("Initial Cluster Centers")
- : N_("Final Cluster Centers"));
-
- struct pivot_dimension *clusters
- = pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Cluster"));
-
- clusters->root->show_label = true;
- for (size_t i = 0; i < qc->ngroups; i++)
- pivot_category_create_leaf (clusters->root,
- pivot_value_new_integer (i + 1));
-
- struct pivot_dimension *variables
- = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Variable"));
-
- for (size_t i = 0; i < qc->n_vars; i++)
- pivot_category_create_leaf (variables->root,
- pivot_value_new_variable (qc->vars[i]));
-
- const gsl_matrix *matrix = (initial
- ? kmeans->initial_centers
- : kmeans->centers);
- for (size_t i = 0; i < qc->ngroups; i++)
- for (size_t j = 0; j < qc->n_vars; j++)
- {
- double x = gsl_matrix_get (matrix, kmeans->group_order->data[i], j);
- union value v = { .f = x };
- pivot_table_put2 (table, i, j,
- pivot_value_new_var_value (qc->vars[j], &v));
- }
-
- pivot_table_submit (table);
-}
-
-
-/* A transformation function which juxtaposes the dataset with the
- (pre-prepared) dataset containing membership and/or distance
- values. */
-static enum trns_result
-save_trans_func (void *aux, struct ccase **c, casenumber x UNUSED)
-{
- const struct save_trans_data *std = aux;
- struct ccase *ca = casereader_read (std->appending_reader);
- if (ca == NULL)
- return TRNS_CONTINUE;
-
- *c = case_unshare (*c);
-
- if (std->membership_case_idx >= 0)
- *case_num_rw (*c, std->membership) = case_num_idx (ca, std->membership_case_idx);
-
- if (std->distance_case_idx >= 0)
- *case_num_rw (*c, std->distance) = case_num_idx (ca, std->distance_case_idx);
-
- case_unref (ca);
-
- return TRNS_CONTINUE;
-}
-
-/* Free the resources of the transformation. */
-static bool
-save_trans_destroy (void *aux)
-{
- struct save_trans_data *std = aux;
- casereader_destroy (std->appending_reader);
- free (std);
- return true;
-}
-
-/* Reports cluster membership for each case, and is requested saves the
- membership and the distance of the case from the cluster centre. */
-static void
-quick_cluster_show_membership (struct Kmeans *kmeans,
- const struct casereader *reader,
- struct qc *qc)
-{
- struct pivot_table *table = NULL;
- struct pivot_dimension *cases = NULL;
- if (qc->print_cluster_membership)
- {
- table = pivot_table_create (N_("Cluster Membership"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Cluster"),
- N_("Cluster"));
-
- cases
- = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Case Number"));
-
- cases->root->show_label = true;
- }
-
- gsl_permutation *ip = gsl_permutation_alloc (qc->ngroups);
- gsl_permutation_inverse (ip, kmeans->group_order);
-
- struct caseproto *proto = caseproto_create ();
- if (qc->save_membership || qc->save_distance)
- {
- /* Prepare data which may potentially be used in a
- transformation appending new variables to the active
- dataset. */
- int idx = 0;
- int membership_case_idx = -1;
- if (qc->save_membership)
- {
- proto = caseproto_add_width (proto, 0);
- membership_case_idx = idx++;
- }
-
- int distance_case_idx = -1;
- if (qc->save_distance)
- {
- proto = caseproto_add_width (proto, 0);
- distance_case_idx = idx++;
- }
-
- qc->save_trans_data = xmalloc (sizeof *qc->save_trans_data);
- *qc->save_trans_data = (struct save_trans_data) {
- .membership_case_idx = membership_case_idx,
- .distance_case_idx = distance_case_idx,
- .writer = autopaging_writer_create (proto),
- };
- }
-
- struct casereader *cs = casereader_clone (reader);
- struct ccase *c;
- for (int i = 0; (c = casereader_read (cs)) != NULL; i++, case_unref (c))
- {
- assert (i < kmeans->n);
- int clust;
- kmeans_get_nearest_group (kmeans, c, qc, &clust, NULL, NULL, NULL);
- int cluster = ip->data[clust];
-
- if (qc->save_trans_data)
- {
- /* Calculate the membership and distance values. */
- struct ccase *outc = case_create (proto);
- if (qc->save_membership)
- *case_num_rw_idx (outc, qc->save_trans_data->membership_case_idx) = cluster + 1;
-
- if (qc->save_distance)
- *case_num_rw_idx (outc, qc->save_trans_data->distance_case_idx)
- = sqrt (dist_from_case (kmeans, c, qc, clust));
-
- casewriter_write (qc->save_trans_data->writer, outc);
- }
-
- if (qc->print_cluster_membership)
- {
- /* Print the cluster membership to the table. */
- int case_idx = pivot_category_create_leaf (cases->root,
- pivot_value_new_integer (i + 1));
- pivot_table_put2 (table, 0, case_idx,
- pivot_value_new_integer (cluster + 1));
- }
- }
-
- caseproto_unref (proto);
- gsl_permutation_free (ip);
-
- if (qc->print_cluster_membership)
- pivot_table_submit (table);
- casereader_destroy (cs);
-}
-
-
-/* Reports number of cases of each single cluster. */
-static void
-quick_cluster_show_number_cases (struct Kmeans *kmeans, const struct qc *qc)
-{
- struct pivot_table *table
- = pivot_table_create (N_("Number of Cases in each Cluster"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Count"));
-
- struct pivot_dimension *clusters
- = pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Clusters"));
-
- struct pivot_category *group
- = pivot_category_create_group (clusters->root, N_("Cluster"));
-
- long int total = 0;
- for (int i = 0; i < qc->ngroups; i++)
- {
- int cluster_idx
- = pivot_category_create_leaf (group, pivot_value_new_integer (i + 1));
- int count = kmeans->num_elements_groups->data [kmeans->group_order->data[i]];
- pivot_table_put2 (table, 0, cluster_idx, pivot_value_new_integer (count));
- total += count;
- }
-
- int cluster_idx = pivot_category_create_leaf (clusters->root,
- pivot_value_new_text (N_("Valid")));
- pivot_table_put2 (table, 0, cluster_idx, pivot_value_new_integer (total));
- pivot_table_submit (table);
-}
-
-/* Reports. */
-static void
-quick_cluster_show_results (struct Kmeans *kmeans, const struct casereader *reader,
- struct qc *qc)
-{
- kmeans_order_groups (kmeans, qc); /* what does this do? */
-
- if (qc->print_initial_clusters)
- quick_cluster_show_centers (kmeans, true, qc);
- quick_cluster_show_centers (kmeans, false, qc);
- quick_cluster_show_number_cases (kmeans, qc);
-
- quick_cluster_show_membership (kmeans, reader, qc);
-}
-
-/* Parse the QUICK CLUSTER command and populate QC accordingly.
- Returns false on error. */
-static bool
-quick_cluster_parse (struct lexer *lexer, struct qc *qc)
-{
- if (!parse_variables_const (lexer, qc->dict, &qc->vars, &qc->n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- return false;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "LISTWISE")
- || lex_match_id (lexer, "DEFAULT"))
- qc->missing_type = MISS_LISTWISE;
- else if (lex_match_id (lexer, "PAIRWISE"))
- qc->missing_type = MISS_PAIRWISE;
- else if (lex_match_id (lexer, "INCLUDE"))
- qc->exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- qc->exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "LISTWISE", "DEFAULT",
- "PAIRWISE", "INCLUDE", "EXCLUDE");
- return false;
- }
- }
- }
- else if (lex_match_id (lexer, "PRINT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "CLUSTER"))
- qc->print_cluster_membership = true;
- else if (lex_match_id (lexer, "INITIAL"))
- qc->print_initial_clusters = true;
- else
- {
- lex_error_expecting (lexer, "CLUSTER", "INITIAL");
- return false;
- }
- }
- }
- else if (lex_match_id (lexer, "SAVE"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "CLUSTER"))
- {
- qc->save_membership = true;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_id (lexer))
- return false;
-
- free (qc->var_membership);
- qc->var_membership = xstrdup (lex_tokcstr (lexer));
- if (NULL != dict_lookup_var (qc->dict, qc->var_membership))
- {
- lex_error (lexer,
- _("A variable called `%s' already exists."),
- qc->var_membership);
- free (qc->var_membership);
- qc->var_membership = NULL;
- return false;
- }
-
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- }
- else if (lex_match_id (lexer, "DISTANCE"))
- {
- qc->save_distance = true;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_id (lexer))
- return false;
-
- free (qc->var_distance);
- qc->var_distance = xstrdup (lex_tokcstr (lexer));
- if (NULL != dict_lookup_var (qc->dict, qc->var_distance))
- {
- lex_error (lexer,
- _("A variable called `%s' already exists."),
- qc->var_distance);
- free (qc->var_distance);
- qc->var_distance = NULL;
- return false;
- }
-
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- }
- else
- {
- lex_error_expecting (lexer, "CLUSTER", "DISTANCE");
- return false;
- }
- }
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "CLUSTERS"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "CLUSTERS", 1, INT_MAX))
- return false;
- qc->ngroups = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- else if (lex_match_id (lexer, "CONVERGE"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_num_range_open (lexer, "CONVERGE",
- 0, DBL_MAX))
- return false;
- qc->epsilon = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- else if (lex_match_id (lexer, "MXITER"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "MXITER", 1, INT_MAX))
- return false;
- qc->maxiter = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- else if (lex_match_id (lexer, "NOINITIAL"))
- qc->initial = false;
- else if (lex_match_id (lexer, "NOUPDATE"))
- qc->update = false;
- else
- {
- lex_error_expecting (lexer, "CLUSTERS", "CONVERGE", "MXITER",
- "NOINITIAL", "NOUPDATE");
- return false;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "MISSING", "PRINT", "SAVE", "CRITERIA");
- return false;
- }
- }
- return true;
-}
-
-int
-cmd_quick_cluster (struct lexer *lexer, struct dataset *ds)
-{
- struct qc qc = {
- .dataset = ds,
- .dict = dataset_dict (ds),
- .ngroups = 2,
- .maxiter = 10,
- .epsilon = DBL_EPSILON,
- .missing_type = MISS_LISTWISE,
- .exclude = MV_ANY,
- .initial = true,
- .update = true,
- };
-
- if (!quick_cluster_parse (lexer, &qc))
- goto error;
-
- qc.wv = dict_get_weight (qc.dict);
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), qc.dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- if (qc.missing_type == MISS_LISTWISE)
- group = casereader_create_filter_missing (group, qc.vars, qc.n_vars,
- qc.exclude, NULL, NULL);
-
- struct Kmeans *kmeans = kmeans_create (&qc);
- kmeans_cluster (kmeans, group, &qc);
- quick_cluster_show_results (kmeans, group, &qc);
- kmeans_destroy (kmeans);
- casereader_destroy (group);
- }
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- /* If requested, set a transformation to append the cluster and
- distance values to the current dataset. */
- if (qc.save_trans_data)
- {
- struct save_trans_data *std = qc.save_trans_data;
-
- std->appending_reader = casewriter_make_reader (std->writer);
-
- if (qc.save_membership)
- {
- /* Invent a variable name if necessary. */
- int idx = 0;
- struct string name;
- ds_init_empty (&name);
- while (qc.var_membership == NULL)
- {
- ds_clear (&name);
- ds_put_format (&name, "QCL_%d", idx++);
-
- if (!dict_lookup_var (qc.dict, ds_cstr (&name)))
- {
- qc.var_membership = strdup (ds_cstr (&name));
- break;
- }
- }
- ds_destroy (&name);
-
- std->membership = dict_create_var_assert (qc.dict, qc.var_membership, 0);
- }
-
- if (qc.save_distance)
- {
- /* Invent a variable name if necessary. */
- int idx = 0;
- struct string name;
- ds_init_empty (&name);
- while (qc.var_distance == NULL)
- {
- ds_clear (&name);
- ds_put_format (&name, "QCL_%d", idx++);
-
- if (!dict_lookup_var (qc.dict, ds_cstr (&name)))
- {
- qc.var_distance = strdup (ds_cstr (&name));
- break;
- }
- }
- ds_destroy (&name);
-
- std->distance = dict_create_var_assert (qc.dict, qc.var_distance, 0);
- }
-
- static const struct trns_class trns_class = {
- .name = "QUICK CLUSTER",
- .execute = save_trans_func,
- .destroy = save_trans_destroy,
- };
- add_transformation (qc.dataset, &trns_class, std);
- }
-
- free (qc.var_distance);
- free (qc.var_membership);
- free (qc.vars);
- return ok;
-
- error:
- free (qc.var_distance);
- free (qc.var_membership);
- free (qc.vars);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2005, 2006, 2007, 2009, 2010, 2011, 2012, 2013, 2014, 2016 Free Software Foundation, Inc
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <math.h>
-#include <gsl/gsl_cdf.h>
-
-#include "data/case.h"
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "data/subcase.h"
-#include "data/casewriter.h"
-#include "data/short-names.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "language/stats/sort-criteria.h"
-#include "math/sort.h"
-#include "libpspp/assertion.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/pool.h"
-#include "libpspp/stringi-set.h"
-#include "libpspp/taint.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-struct rank;
-
-typedef double (*rank_function_t) (const struct rank*, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_proportion (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_normal (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_percent (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_rfraction (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_rank (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_n (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_savage (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-static double rank_ntiles (const struct rank *, double c, double cc, double cc_1,
- int i, double w);
-
-
-enum rank_func
- {
- RANK,
- NORMAL,
- PERCENT,
- RFRACTION,
- PROPORTION,
- N,
- NTILES,
- SAVAGE,
- n_RANK_FUNCS
- };
-
-static const struct fmt_spec dest_format[n_RANK_FUNCS] = {
- [RANK] = { .type = FMT_F, .w = 9, .d = 3 },
- [NORMAL] = { .type = FMT_F, .w = 6, .d = 4 },
- [PERCENT] = { .type = FMT_F, .w = 6, .d = 2 },
- [RFRACTION] = { .type = FMT_F, .w = 6, .d = 4 },
- [PROPORTION] = { .type = FMT_F, .w = 6, .d = 4 },
- [N] = { .type = FMT_F, .w = 6, .d = 0 },
- [NTILES] = { .type = FMT_F, .w = 3, .d = 0 },
- [SAVAGE] = { .type = FMT_F, .w = 8, .d = 4 }
-};
-
-static const char * const function_name[n_RANK_FUNCS] = {
- "RANK",
- "NORMAL",
- "PERCENT",
- "RFRACTION",
- "PROPORTION",
- "N",
- "NTILES",
- "SAVAGE"
-};
-
-static const rank_function_t rank_func[n_RANK_FUNCS] = {
- rank_rank,
- rank_normal,
- rank_percent,
- rank_rfraction,
- rank_proportion,
- rank_n,
- rank_ntiles,
- rank_savage
-};
-
-static enum measure rank_measures[n_RANK_FUNCS] = {
- [RANK] = MEASURE_ORDINAL,
- [NORMAL] = MEASURE_ORDINAL,
- [PERCENT] = MEASURE_ORDINAL,
- [RFRACTION] = MEASURE_ORDINAL,
- [PROPORTION] = MEASURE_ORDINAL,
- [N] = MEASURE_SCALE,
- [NTILES] = MEASURE_ORDINAL,
- [SAVAGE] = MEASURE_ORDINAL,
-};
-
-enum ties
- {
- TIES_LOW,
- TIES_HIGH,
- TIES_MEAN,
- TIES_CONDENSE
- };
-
-enum fraction
- {
- FRAC_BLOM,
- FRAC_RANKIT,
- FRAC_TUKEY,
- FRAC_VW
- };
-
-struct rank_spec
-{
- enum rank_func rfunc;
- const char **dest_names;
- const char **dest_labels;
-};
-
-/* If NEW_NAME exists in DICT or NEW_NAMES, returns NULL without changing
- anything. Otherwise, inserts NEW_NAME in NEW_NAMES and returns the copy of
- NEW_NAME now in NEW_NAMES. In any case, frees NEW_NAME. */
-static const char *
-try_new_name (char *new_name,
- const struct dictionary *dict, struct stringi_set *new_names)
-{
- const char *retval = (!dict_lookup_var (dict, new_name)
- && stringi_set_insert (new_names, new_name)
- ? stringi_set_find_node (new_names, new_name)->string
- : NULL);
- free (new_name);
- return retval;
-}
-
-/* Returns a variable name for storing ranks of a variable named SRC_NAME
- according to the rank function F. The name chosen will not be one already in
- DICT or NEW_NAMES.
-
- If successful, adds the new name to NEW_NAMES and returns the name added.
- If no name can be generated, returns NULL. */
-static const char *
-rank_choose_dest_name (struct dictionary *dict, struct stringi_set *new_names,
- enum rank_func f, const char *src_name)
-{
- /* Try the first character of the ranking function followed by the first 7
- bytes of the srcinal variable name. */
- char *src_name_7 = utf8_encoding_trunc (src_name, dict_get_encoding (dict),
- 7);
- const char *s = try_new_name (
- xasprintf ("%c%s", function_name[f][0], src_name_7), dict, new_names);
- free (src_name_7);
- if (s)
- return s;
-
- /* Try "fun###". */
- for (int i = 1; i <= 999; i++)
- {
- s = try_new_name (xasprintf ("%.3s%03d", function_name[f], i),
- dict, new_names);
- if (s)
- return s;
- }
-
- /* Try "RNKfn##". */
- for (int i = 1; i <= 99; i++)
- {
- s = try_new_name (xasprintf ("RNK%.2s%02d", function_name[f], i),
- dict, new_names);
- if (s)
- return s;
- }
-
- msg (ME, _("Cannot generate variable name for ranking %s with %s. "
- "All candidates in use."),
- src_name, function_name[f]);
- return NULL;
-}
-
-struct rank
-{
- struct dictionary *dict;
-
- struct subcase sc;
-
- const struct variable **vars;
- size_t n_vars;
-
- const struct variable **group_vars;
- size_t n_group_vars;
-
-
- enum mv_class exclude;
-
- struct rank_spec *rs;
- size_t n_rs;
-
- enum ties ties;
-
- enum fraction fraction;
- int k_ntiles;
-
- bool print;
-
- /* Pool on which cell functions may allocate data */
- struct pool *pool;
-};
-
-
-static void
-destroy_rank (struct rank *rank)
-{
- free (rank->vars);
- free (rank->group_vars);
- subcase_uninit (&rank->sc);
- pool_destroy (rank->pool);
-}
-
-static bool
-parse_into (struct lexer *lexer, struct rank *cmd,
- struct stringi_set *new_names)
-{
- enum rank_func rfunc;
- if (lex_match_id (lexer, "RANK"))
- rfunc = RANK;
- else if (lex_match_id (lexer, "NORMAL"))
- rfunc = NORMAL;
- else if (lex_match_id (lexer, "RFRACTION"))
- rfunc = RFRACTION;
- else if (lex_match_id (lexer, "N"))
- rfunc = N;
- else if (lex_match_id (lexer, "SAVAGE"))
- rfunc = SAVAGE;
- else if (lex_match_id (lexer, "PERCENT"))
- rfunc = PERCENT;
- else if (lex_match_id (lexer, "PROPORTION"))
- rfunc = PROPORTION;
- else if (lex_match_id (lexer, "NTILES"))
- {
- if (!lex_force_match (lexer, T_LPAREN)
- || !lex_force_int_range (lexer, "NTILES", 1, INT_MAX))
- return false;
-
- cmd->k_ntiles = lex_integer (lexer);
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
-
- rfunc = NTILES;
- }
- else
- {
- lex_error_expecting (lexer, "RANK", "NORMAL", "RFRACTION", "N",
- "SAVAGE", "PERCENT", "PROPORTION", "NTILES");
- return false;
- }
-
- cmd->rs = pool_realloc (cmd->pool, cmd->rs, sizeof (*cmd->rs) * (cmd->n_rs + 1));
- struct rank_spec *rs = &cmd->rs[cmd->n_rs++];
- *rs = (struct rank_spec) {
- .rfunc = rfunc,
- .dest_names = pool_calloc (cmd->pool, cmd->n_vars,
- sizeof *rs->dest_names),
- };
-
- if (lex_match_id (lexer, "INTO"))
- {
- int vars_start = lex_ofs (lexer);
- size_t var_count = 0;
- while (lex_token (lexer) == T_ID)
- {
- const char *name = lex_tokcstr (lexer);
-
- if (var_count >= subcase_get_n_fields (&cmd->sc))
- lex_ofs_error (lexer, vars_start, lex_ofs (lexer),
- _("Too many variables in %s clause."), "INTO");
- else if (dict_lookup_var (cmd->dict, name) != NULL)
- lex_error (lexer, _("Variable %s already exists."), name);
- else if (stringi_set_contains (new_names, name))
- lex_error (lexer, _("Duplicate variable name %s."), name);
- else
- {
- stringi_set_insert (new_names, name);
- rs->dest_names[var_count++] = pool_strdup (cmd->pool, name);
- lex_get (lexer);
- continue;
- }
-
- /* Error path. */
- return false;
- }
- }
-
- return true;
-}
-
-/* Hardly a rank function. */
-static double
-rank_n (const struct rank *cmd UNUSED, double c UNUSED, double cc UNUSED, double cc_1 UNUSED,
- int i UNUSED, double w)
-{
- return w;
-}
-
-static double
-rank_rank (const struct rank *cmd, double c, double cc, double cc_1,
- int i, double w UNUSED)
-{
- double rank;
-
- if (c >= 1.0)
- {
- switch (cmd->ties)
- {
- case TIES_LOW:
- rank = cc_1 + 1;
- break;
- case TIES_HIGH:
- rank = cc;
- break;
- case TIES_MEAN:
- rank = cc_1 + (c + 1.0)/ 2.0;
- break;
- case TIES_CONDENSE:
- rank = i;
- break;
- default:
- NOT_REACHED ();
- }
- }
- else
- {
- switch (cmd->ties)
- {
- case TIES_LOW:
- rank = cc_1;
- break;
- case TIES_HIGH:
- rank = cc;
- break;
- case TIES_MEAN:
- rank = cc_1 + c / 2.0;
- break;
- case TIES_CONDENSE:
- rank = i;
- break;
- default:
- NOT_REACHED ();
- }
- }
-
- return rank;
-}
-
-
-static double
-rank_rfraction (const struct rank *cmd, double c, double cc, double cc_1,
- int i, double w)
-{
- return rank_rank (cmd, c, cc, cc_1, i, w) / w;
-}
-
-
-static double
-rank_percent (const struct rank *cmd, double c, double cc, double cc_1,
- int i, double w)
-{
- return rank_rank (cmd, c, cc, cc_1, i, w) * 100.0 / w;
-}
-
-
-static double
-rank_proportion (const struct rank *cmd, double c, double cc, double cc_1,
- int i, double w)
-{
- const double r = rank_rank (cmd, c, cc, cc_1, i, w);
-
- double f;
-
- switch (cmd->fraction)
- {
- case FRAC_BLOM:
- f = (r - 3.0/8.0) / (w + 0.25);
- break;
- case FRAC_RANKIT:
- f = (r - 0.5) / w;
- break;
- case FRAC_TUKEY:
- f = (r - 1.0/3.0) / (w + 1.0/3.0);
- break;
- case FRAC_VW:
- f = r / (w + 1.0);
- break;
- default:
- NOT_REACHED ();
- }
-
-
- return (f > 0) ? f : SYSMIS;
-}
-
-static double
-rank_normal (const struct rank *cmd, double c, double cc, double cc_1,
- int i, double w)
-{
- double f = rank_proportion (cmd, c, cc, cc_1, i, w);
-
- return gsl_cdf_ugaussian_Pinv (f);
-}
-
-static double
-rank_ntiles (const struct rank *cmd, double c, double cc, double cc_1,
- int i, double w)
-{
- double r = rank_rank (cmd, c, cc, cc_1, i, w);
-
-
- return (floor ((r * cmd->k_ntiles) / (w + 1)) + 1);
-}
-
-/* Expected value of the order statistics from an exponential distribution */
-static double
-ee (int j, double w_star)
-{
- double sum = 0.0;
-
- for (int k = 1; k <= j; k++)
- sum += 1.0 / (w_star + 1 - k);
-
- return sum;
-}
-
-
-static double
-rank_savage (const struct rank *cmd UNUSED, double c, double cc, double cc_1,
- int i UNUSED, double w)
-{
- double int_part;
- const int i_1 = floor (cc_1);
- const int i_2 = floor (cc);
-
- const double w_star = (modf (w, &int_part) == 0) ? w : floor (w) + 1;
-
- const double g_1 = cc_1 - i_1;
- const double g_2 = cc - i_2;
-
- /* The second factor is infinite, when the first is zero.
- Therefore, evaluate the second, only when the first is non-zero */
- const double expr1 = (1 - g_1) ? (1 - g_1) * ee(i_1+1, w_star) : (1 - g_1);
- const double expr2 = g_2 ? g_2 * ee (i_2+1, w_star) : g_2;
-
- if (i_1 == i_2)
- return ee (i_1 + 1, w_star) - 1;
-
- if (i_1 + 1 == i_2)
- return ((expr1 + expr2)/c) - 1;
-
- if (i_1 + 2 <= i_2)
- {
- double sigma = 0.0;
- for (int j = i_1 + 2; j <= i_2; ++j)
- sigma += ee (j, w_star);
- return ((expr1 + expr2 + sigma) / c) -1;
- }
-
- NOT_REACHED ();
-}
-
-static double
-sum_weights (const struct casereader *input, int weight_idx)
-{
- if (weight_idx == -1)
- return casereader_count_cases (input);
-
- double w = 0.0;
-
- struct casereader *pass = casereader_clone (input);
- struct ccase *c;
- for (; (c = casereader_read (pass)) != NULL; case_unref (c))
- w += case_num_idx (c, weight_idx);
- casereader_destroy (pass);
-
- return w;
-}
-
-static void
-rank_sorted_file (struct casereader *input,
- struct casewriter *output,
- int weight_idx,
- const struct rank *cmd)
-{
- int tie_group = 1;
- double cc = 0.0;
-
- /* Get total group weight. */
- double w = sum_weights (input, weight_idx);
-
- /* Do ranking. */
- struct subcase input_var = SUBCASE_EMPTY_INITIALIZER;
- subcase_add (&input_var, 0, 0, SC_ASCEND);
- struct casegrouper *tie_grouper = casegrouper_create_subcase (input, &input_var);
- subcase_uninit (&input_var);
-
- struct casereader *tied_cases;
- for (; casegrouper_get_next_group (tie_grouper, &tied_cases);
- casereader_destroy (tied_cases))
- {
- double tw = sum_weights (tied_cases, weight_idx);
- double cc_1 = cc;
- cc += tw;
-
- taint_propagate (casereader_get_taint (tied_cases),
- casewriter_get_taint (output));
-
- /* Rank tied cases. */
- struct ccase *c;
- for (; (c = casereader_read (tied_cases)) != NULL; case_unref (c))
- {
- struct ccase *out_case = case_create (casewriter_get_proto (output));
- *case_num_rw_idx (out_case, 0) = case_num_idx (c, 1);
- for (size_t i = 0; i < cmd->n_rs; ++i)
- {
- rank_function_t func = rank_func[cmd->rs[i].rfunc];
- double rank = func (cmd, tw, cc, cc_1, tie_group, w);
- *case_num_rw_idx (out_case, i + 1) = rank;
- }
-
- casewriter_write (output, out_case);
- }
- tie_group++;
- }
- casegrouper_destroy (tie_grouper);
-}
-
-
-static bool
-rank_cmd (struct dataset *ds, const struct rank *cmd);
-
-static const char *
-fraction_name (const struct rank *cmd)
-{
- switch (cmd->fraction)
- {
- case FRAC_BLOM: return "BLOM";
- case FRAC_RANKIT: return "RANKIT";
- case FRAC_TUKEY: return "TUKEY";
- case FRAC_VW: return "VW";
- default: NOT_REACHED ();
- }
-}
-
-/* Returns a label for a variable derived from SRC_VAR with function F. */
-static const char *
-create_var_label (struct rank *cmd, const struct variable *src_var,
- enum rank_func f)
-{
- if (cmd->n_group_vars > 0)
- {
- struct string group_var_str = DS_EMPTY_INITIALIZER;
- for (size_t g = 0; g < cmd->n_group_vars; ++g)
- {
- if (g > 0)
- ds_put_cstr (&group_var_str, " ");
- ds_put_cstr (&group_var_str, var_get_name (cmd->group_vars[g]));
- }
-
- const char *label = pool_asprintf (
- cmd->pool, _("%s of %s by %s"), function_name[f],
- var_get_name (src_var), ds_cstr (&group_var_str));
- ds_destroy (&group_var_str);
- return label;
- }
- else
- return pool_asprintf (cmd->pool, _("%s of %s"),
- function_name[f], var_get_name (src_var));
-}
-
-int
-cmd_rank (struct lexer *lexer, struct dataset *ds)
-{
- struct stringi_set new_names = STRINGI_SET_INITIALIZER (new_names);
- struct rank rank = {
- .sc = SUBCASE_EMPTY_INITIALIZER,
- .exclude = MV_ANY,
- .dict = dataset_dict (ds),
- .ties = TIES_MEAN,
- .fraction = FRAC_BLOM,
- .print = true,
- .pool = pool_create (),
- };
-
- if (lex_match_id (lexer, "VARIABLES") && !lex_force_match (lexer, T_EQUALS))
- goto error;
-
- if (!parse_sort_criteria (lexer, rank.dict, &rank.sc, &rank.vars, NULL))
- goto error;
- rank.n_vars = rank.sc.n_fields;
-
- if (lex_match (lexer, T_BY)
- && !parse_variables_const (lexer, rank.dict,
- &rank.group_vars, &rank.n_group_vars,
- PV_NO_DUPLICATE | PV_NO_SCRATCH))
- goto error;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- if (!lex_force_match (lexer, T_SLASH))
- goto error;
- if (lex_match_id (lexer, "TIES"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
- if (lex_match_id (lexer, "MEAN"))
- rank.ties = TIES_MEAN;
- else if (lex_match_id (lexer, "LOW"))
- rank.ties = TIES_LOW;
- else if (lex_match_id (lexer, "HIGH"))
- rank.ties = TIES_HIGH;
- else if (lex_match_id (lexer, "CONDENSE"))
- rank.ties = TIES_CONDENSE;
- else
- {
- lex_error_expecting (lexer, "MEAN", "LOW", "HIGH", "CONDENSE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "FRACTION"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
- if (lex_match_id (lexer, "BLOM"))
- rank.fraction = FRAC_BLOM;
- else if (lex_match_id (lexer, "TUKEY"))
- rank.fraction = FRAC_TUKEY;
- else if (lex_match_id (lexer, "VW"))
- rank.fraction = FRAC_VW;
- else if (lex_match_id (lexer, "RANKIT"))
- rank.fraction = FRAC_RANKIT;
- else
- {
- lex_error_expecting (lexer, "BLOM", "TUKEY", "VW", "RANKIT");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "PRINT"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
- if (lex_match_id (lexer, "YES"))
- rank.print = true;
- else if (lex_match_id (lexer, "NO"))
- rank.print = false;
- else
- {
- lex_error_expecting (lexer, "YES", "NO");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
- if (lex_match_id (lexer, "INCLUDE"))
- rank.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- rank.exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- else if (!parse_into (lexer, &rank, &new_names))
- goto error;
- }
-
-
- /* If no rank specs are given, then apply a default */
- if (rank.n_rs == 0)
- {
- struct rank_spec *rs = pool_malloc (rank.pool, sizeof *rs);
- *rs = (struct rank_spec) {
- .rfunc = RANK,
- .dest_names = pool_calloc (rank.pool, rank.n_vars,
- sizeof *rs->dest_names),
- };
-
- rank.rs = rs;
- rank.n_rs = 1;
- }
-
- /* Choose variable names for all rank destinations which haven't already been
- created with INTO. */
- for (struct rank_spec *rs = rank.rs; rs < &rank.rs[rank.n_rs]; rs++)
- {
- rs->dest_labels = pool_calloc (rank.pool, rank.n_vars,
- sizeof *rs->dest_labels);
- for (int v = 0; v < rank.n_vars; v ++)
- {
- const char **dst_name = &rs->dest_names[v];
- if (*dst_name == NULL)
- {
- *dst_name = rank_choose_dest_name (rank.dict, &new_names,
- rs->rfunc,
- var_get_name (rank.vars[v]));
- if (*dst_name == NULL)
- goto error;
- }
-
- rs->dest_labels[v] = create_var_label (&rank, rank.vars[v],
- rs->rfunc);
- }
- }
-
- if (rank.print)
- {
- struct pivot_table *table = pivot_table_create (
- N_("Variables Created by RANK"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("New Variable"),
- N_("New Variable"), N_("Function"),
- N_("Fraction"), N_("Grouping Variables"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Existing Variable"),
- N_("Existing Variable"));
- variables->root->show_label = true;
-
- for (size_t i = 0; i < rank.n_rs; ++i)
- {
- for (size_t v = 0; v < rank.n_vars; v ++)
- {
- int row_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (rank.vars[v]));
-
- struct string group_vars = DS_EMPTY_INITIALIZER;
- for (int g = 0; g < rank.n_group_vars; ++g)
- {
- if (g)
- ds_put_byte (&group_vars, ' ');
- ds_put_cstr (&group_vars, var_get_name (rank.group_vars[g]));
- }
-
- enum rank_func rfunc = rank.rs[i].rfunc;
- bool has_fraction = rfunc == NORMAL || rfunc == PROPORTION;
- const char *entries[] =
- {
- rank.rs[i].dest_names[v],
- function_name[rank.rs[i].rfunc],
- has_fraction ? fraction_name (&rank) : NULL,
- rank.n_group_vars ? ds_cstr (&group_vars) : NULL,
- };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- {
- const char *entry = entries[j];
- if (entry)
- pivot_table_put2 (table, j, row_idx,
- pivot_value_new_user_text (entry, -1));
- }
- ds_destroy (&group_vars);
- }
- }
-
- pivot_table_submit (table);
- }
-
- /* Do the ranking */
- rank_cmd (ds, &rank);
-
- destroy_rank (&rank);
- stringi_set_destroy (&new_names);
- return CMD_SUCCESS;
-
- error:
- destroy_rank (&rank);
- stringi_set_destroy (&new_names);
- return CMD_FAILURE;
-}
-
-/* RANK transformation. */
-struct rank_trns
- {
- int order_case_idx;
-
- struct rank_trns_input_var *input_vars;
- size_t n_input_vars;
-
- size_t n_funcs;
- };
-
-struct rank_trns_input_var
- {
- struct casereader *input;
- struct ccase *current;
-
- struct variable **output_vars;
- };
-
-static void
-advance_ranking (struct rank_trns_input_var *iv)
-{
- case_unref (iv->current);
- iv->current = casereader_read (iv->input);
-}
-
-static enum trns_result
-rank_trns_proc (void *trns_, struct ccase **c, casenumber case_idx UNUSED)
-{
- struct rank_trns *trns = trns_;
- double order = case_num_idx (*c, trns->order_case_idx);
- struct rank_trns_input_var *iv;
-
- *c = case_unshare (*c);
- for (iv = trns->input_vars; iv < &trns->input_vars[trns->n_input_vars]; iv++)
- while (iv->current != NULL)
- {
- double iv_order = case_num_idx (iv->current, 0);
- if (iv_order == order)
- {
- size_t i;
-
- for (i = 0; i < trns->n_funcs; i++)
- *case_num_rw (*c, iv->output_vars[i])
- = case_num_idx (iv->current, i + 1);
- advance_ranking (iv);
- break;
- }
- else if (iv_order > order)
- break;
- else
- advance_ranking (iv);
- }
- return TRNS_CONTINUE;
-}
-
-static bool
-rank_trns_free (void *trns_)
-{
- struct rank_trns *trns = trns_;
- struct rank_trns_input_var *iv;
-
- for (iv = trns->input_vars; iv < &trns->input_vars[trns->n_input_vars]; iv++)
- {
- casereader_destroy (iv->input);
- case_unref (iv->current);
-
- free (iv->output_vars);
- }
- free (trns->input_vars);
- free (trns);
-
- return true;
-}
-
-static const struct trns_class rank_trns_class = {
- .name = "RANK",
- .execute = rank_trns_proc,
- .destroy = rank_trns_free,
-};
-
-static bool
-rank_cmd (struct dataset *ds, const struct rank *cmd)
-{
- struct dictionary *d = dataset_dict (ds);
- struct variable *weight_var = dict_get_weight (d);
- bool ok = true;
-
- struct variable *order_var = add_permanent_ordering_transformation (ds);
-
- /* Create output files. */
- struct caseproto *output_proto = caseproto_create ();
- for (size_t i = 0; i < cmd->n_rs + 1; i++)
- output_proto = caseproto_add_width (output_proto, 0);
-
- struct subcase by_order;
- subcase_init (&by_order, 0, 0, SC_ASCEND);
-
- struct casewriter **outputs = xnmalloc (cmd->n_vars, sizeof *outputs);
- for (size_t i = 0; i < cmd->n_vars; i++)
- outputs[i] = sort_create_writer (&by_order, output_proto);
-
- subcase_uninit (&by_order);
- caseproto_unref (output_proto);
-
- /* Open the active file and make one pass per input variable. */
- struct casereader *input = proc_open (ds);
- input = casereader_create_filter_weight (input, d, NULL, NULL);
- for (size_t i = 0; i < cmd->n_vars; ++i)
- {
- const struct variable *input_var = cmd->vars[i];
-
- /* Discard cases that have missing values of input variable. */
- struct casereader *input_pass
- = i == cmd->n_vars - 1 ? input : casereader_clone (input);
- input_pass = casereader_create_filter_missing (input_pass, &input_var, 1,
- cmd->exclude, NULL, NULL);
-
- /* Keep only the columns we really need, to save time and space when we
- sort them just below.
-
- After this projection, the input_pass case indexes look like:
-
- - 0: input_var.
- - 1: order_var.
- - 2 and up: cmd->n_group_vars group variables
- - 2 + cmd->n_group_vars and up: split variables
- - 2 + cmd->n_group_vars + n_split_vars: weight var
- */
- struct subcase projection = SUBCASE_EMPTY_INITIALIZER;
- subcase_add_var_always (&projection, input_var, SC_ASCEND);
- subcase_add_var_always (&projection, order_var, SC_ASCEND);
- subcase_add_vars_always (&projection,
- cmd->group_vars, cmd->n_group_vars);
- subcase_add_vars_always (&projection, dict_get_split_vars (d),
- dict_get_n_splits (d));
- int weight_idx;
- if (weight_var != NULL)
- {
- subcase_add_var_always (&projection, weight_var, SC_ASCEND);
- weight_idx = 2 + cmd->n_group_vars + dict_get_n_splits (d);
- }
- else
- weight_idx = -1;
- input_pass = casereader_project (input_pass, &projection);
- subcase_uninit (&projection);
-
- /* Prepare 'group_vars' as the set of grouping variables. */
- struct subcase group_vars = SUBCASE_EMPTY_INITIALIZER;
- for (size_t j = 0; j < cmd->n_group_vars; j++)
- subcase_add_always (&group_vars,
- j + 2, var_get_width (cmd->group_vars[j]),
- SC_ASCEND);
-
- /* Prepare 'rank_ordering' for sorting with the group variables as
- primary key and the input variable as secondary key. */
- struct subcase rank_ordering;
- subcase_clone (&rank_ordering, &group_vars);
- subcase_add (&rank_ordering, 0, 0, subcase_get_direction (&cmd->sc, i));
-
- /* Group by split variables */
- struct subcase split_vars = SUBCASE_EMPTY_INITIALIZER;
- for (size_t j = 0; j < dict_get_n_splits (d); j++)
- subcase_add_always (&split_vars, 2 + j + cmd->n_group_vars,
- var_get_width (dict_get_split_vars (d)[j]),
- SC_ASCEND);
-
- struct casegrouper *split_grouper
- = casegrouper_create_subcase (input_pass, &split_vars);
- subcase_uninit (&split_vars);
-
- struct casereader *split_group;
- while (casegrouper_get_next_group (split_grouper, &split_group))
- {
- struct casereader *ordered;
- struct casegrouper *by_grouper;
- struct casereader *by_group;
-
- ordered = sort_execute (split_group, &rank_ordering);
- by_grouper = casegrouper_create_subcase (ordered, &group_vars);
- while (casegrouper_get_next_group (by_grouper, &by_group))
- rank_sorted_file (by_group, outputs[i], weight_idx, cmd);
- ok = casegrouper_destroy (by_grouper) && ok;
- }
- subcase_uninit (&group_vars);
- subcase_uninit (&rank_ordering);
-
- ok = casegrouper_destroy (split_grouper) && ok;
- }
- ok = proc_commit (ds) && ok;
-
- /* Re-fetch the dictionary and order variable, because if TEMPORARY was in
- effect then there's a new dictionary. */
- d = dataset_dict (ds);
- order_var = dict_lookup_var_assert (d, "$ORDER");
-
- /* Merge the original data set with the ranks (which we already sorted on
- $ORDER). */
- struct rank_trns *trns = xmalloc (sizeof *trns);
- trns->order_case_idx = var_get_case_index (order_var);
- trns->input_vars = xnmalloc (cmd->n_vars, sizeof *trns->input_vars);
- trns->n_input_vars = cmd->n_vars;
- trns->n_funcs = cmd->n_rs;
- for (size_t i = 0; i < trns->n_input_vars; i++)
- {
- struct rank_trns_input_var *iv = &trns->input_vars[i];
-
- iv->input = casewriter_make_reader (outputs[i]);
- iv->current = casereader_read (iv->input);
- iv->output_vars = xnmalloc (trns->n_funcs, sizeof *iv->output_vars);
- for (size_t j = 0; j < trns->n_funcs; j++)
- {
- struct rank_spec *rs = &cmd->rs[j];
- struct variable *var;
-
- var = dict_create_var_assert (d, rs->dest_names[i], 0);
- var_set_both_formats (var, &dest_format[rs->rfunc]);
- var_set_label (var, rs->dest_labels[i]);
- var_set_measure (var, rank_measures[rs->rfunc]);
-
- iv->output_vars[j] = var;
- }
- }
- free (outputs);
-
- add_transformation (ds, &rank_trns_class, trns);
-
- /* Delete our sort key, which we don't need anymore. */
- dict_delete_var (d, order_var);
-
- return ok;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2005, 2009, 2010, 2011, 2012, 2013, 2014,
- 2016, 2017, 2019 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <stdbool.h>
-
-#include <gsl/gsl_math.h>
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_matrix.h>
-
-#include <data/dataset.h>
-#include <data/casewriter.h>
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dictionary.h"
-
-#include "math/covariance.h"
-#include "math/linreg.h"
-#include "math/moments.h"
-
-#include "libpspp/message.h"
-#include "libpspp/taint.h"
-
-#include "output/pivot-table.h"
-
-#include "gl/intprops.h"
-#include "gl/minmax.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-#define STATS_R (1 << 0)
-#define STATS_COEFF (1 << 1)
-#define STATS_ANOVA (1 << 2)
-#define STATS_OUTS (1 << 3)
-#define STATS_CI (1 << 4)
-#define STATS_BCOV (1 << 5)
-#define STATS_TOL (1 << 6)
-
-#define STATS_DEFAULT (STATS_R | STATS_COEFF | STATS_ANOVA | STATS_OUTS)
-
-
-struct regression
- {
- struct dataset *ds;
-
- const struct variable **vars;
- size_t n_vars;
-
- const struct variable **dep_vars;
- size_t n_dep_vars;
-
- unsigned int stats;
- double ci;
-
- bool resid;
- bool pred;
-
- bool origin;
- };
-
-struct regression_workspace
-{
- /* The new variables which will be introduced by /SAVE */
- const struct variable **predvars;
- const struct variable **residvars;
-
- /* A reader/writer pair to temporarily hold the
- values of the new variables */
- struct casewriter *writer;
- struct casereader *reader;
-
- /* Indeces of the new values in the reader/writer (-1 if not applicable) */
- int res_idx;
- int pred_idx;
-
- /* 0, 1 or 2 depending on what new variables are to be created */
- int extras;
-};
-
-static void run_regression (const struct regression *cmd,
- struct regression_workspace *ws,
- struct casereader *input);
-
-
-/* Return a string based on PREFIX which may be used as the name
- of a new variable in DICT */
-static char *
-reg_get_name (const struct dictionary *dict, const char *prefix)
-{
- for (size_t i = 1; ; i++)
- {
- char *name = xasprintf ("%s%zu", prefix, i);
- if (!dict_lookup_var (dict, name))
- return name;
- free (name);
- }
-}
-
-static const struct variable *
-create_aux_var (struct dataset *ds, const char *prefix)
-{
- struct dictionary *dict = dataset_dict (ds);
- char *name = reg_get_name (dict, prefix);
- struct variable *var = dict_create_var_assert (dict, name, 0);
- free (name);
- return var;
-}
-
-/* Auxiliary data for transformation when /SAVE is entered */
-struct save_trans_data
- {
- int n_dep_vars;
- struct regression_workspace *ws;
- };
-
-static bool
-save_trans_free (void *aux)
-{
- struct save_trans_data *save_trans_data = aux;
- free (save_trans_data->ws->predvars);
- free (save_trans_data->ws->residvars);
-
- casereader_destroy (save_trans_data->ws->reader);
- free (save_trans_data->ws);
- free (save_trans_data);
- return true;
-}
-
-static enum trns_result
-save_trans_func (void *aux, struct ccase **c, casenumber x UNUSED)
-{
- struct save_trans_data *save_trans_data = aux;
- struct regression_workspace *ws = save_trans_data->ws;
- struct ccase *in = casereader_read (ws->reader);
-
- if (in)
- {
- *c = case_unshare (*c);
-
- for (size_t k = 0; k < save_trans_data->n_dep_vars; ++k)
- {
- if (ws->pred_idx != -1)
- {
- double pred = case_num_idx (in, ws->extras * k + ws->pred_idx);
- *case_num_rw (*c, ws->predvars[k]) = pred;
- }
-
- if (ws->res_idx != -1)
- {
- double resid = case_num_idx (in, ws->extras * k + ws->res_idx);
- *case_num_rw (*c, ws->residvars[k]) = resid;
- }
- }
- case_unref (in);
- }
-
- return TRNS_CONTINUE;
-}
-
-int
-cmd_regression (struct lexer *lexer, struct dataset *ds)
-{
- const struct dictionary *dict = dataset_dict (ds);
-
- struct regression regression = {
- .ci = 0.95,
- .stats = STATS_DEFAULT,
- .pred = false,
- .resid = false,
- .ds = ds,
- .origin = false,
- };
-
- bool variables_seen = false;
- bool method_seen = false;
- bool dependent_seen = false;
- int save_start = 0;
- int save_end = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "VARIABLES"))
- {
- if (method_seen)
- {
- lex_next_error (lexer, -1, -1,
- _("VARIABLES may not appear after %s"), "METHOD");
- goto error;
- }
- if (dependent_seen)
- {
- lex_next_error (lexer, -1, -1,
- _("VARIABLES may not appear after %s"), "DEPENDENT");
- goto error;
- }
- variables_seen = true;
- lex_match (lexer, T_EQUALS);
-
- if (!parse_variables_const (lexer, dict,
- ®ression.vars, ®ression.n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- }
- else if (lex_match_id (lexer, "DEPENDENT"))
- {
- dependent_seen = true;
- lex_match (lexer, T_EQUALS);
-
- free (regression.dep_vars);
- regression.n_dep_vars = 0;
-
- if (!parse_variables_const (lexer, dict,
- ®ression.dep_vars,
- ®ression.n_dep_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- }
- else if (lex_match_id (lexer, "ORIGIN"))
- regression.origin = true;
- else if (lex_match_id (lexer, "NOORIGIN"))
- regression.origin = false;
- else if (lex_match_id (lexer, "METHOD"))
- {
- method_seen = true;
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_match_id (lexer, "ENTER"))
- goto error;
-
- if (!variables_seen)
- {
- if (!parse_variables_const (lexer, dict,
- ®ression.vars, ®ression.n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- }
- }
- else if (lex_match_id (lexer, "STATISTICS"))
- {
- unsigned long statistics = 0;
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match (lexer, T_ALL))
- statistics = ~0;
- else if (lex_match_id (lexer, "DEFAULTS"))
- statistics |= STATS_DEFAULT;
- else if (lex_match_id (lexer, "R"))
- statistics |= STATS_R;
- else if (lex_match_id (lexer, "COEFF"))
- statistics |= STATS_COEFF;
- else if (lex_match_id (lexer, "ANOVA"))
- statistics |= STATS_ANOVA;
- else if (lex_match_id (lexer, "BCOV"))
- statistics |= STATS_BCOV;
- else if (lex_match_id (lexer, "TOL"))
- statistics |= STATS_TOL;
- else if (lex_match_id (lexer, "CI"))
- {
- statistics |= STATS_CI;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num (lexer))
- goto error;
- regression.ci = lex_number (lexer) / 100.0;
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
- else
- {
- lex_error_expecting (lexer, "ALL", "DEFAULTS", "R", "COEFF",
- "ANOVA", "BCOV", "TOL", "CI");
- goto error;
- }
- }
-
- if (statistics)
- regression.stats = statistics;
- }
- else if (lex_match_id (lexer, "SAVE"))
- {
- save_start = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
-
- while (lex_token (lexer) != T_ENDCMD
- && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "PRED"))
- regression.pred = true;
- else if (lex_match_id (lexer, "RESID"))
- regression.resid = true;
- else
- {
- lex_error_expecting (lexer, "PRED", "RESID");
- goto error;
- }
- }
- save_end = lex_ofs (lexer) - 1;
- }
- else
- {
- lex_error_expecting (lexer, "VARIABLES", "DEPENDENT", "ORIGIN",
- "NOORIGIN", "METHOD", "STATISTICS", "SAVE");
- goto error;
- }
- }
-
- if (!regression.vars)
- dict_get_vars (dict, ®ression.vars, ®ression.n_vars, 0);
-
- struct regression_workspace workspace = {
- .res_idx = -1,
- .pred_idx = -1,
- };
-
- bool save = regression.pred || regression.resid;
- if (save)
- {
- struct caseproto *proto = caseproto_create ();
-
- if (regression.resid)
- {
- workspace.res_idx = workspace.extras ++;
- workspace.residvars = xcalloc (regression.n_dep_vars, sizeof (*workspace.residvars));
-
- for (size_t i = 0; i < regression.n_dep_vars; ++i)
- {
- workspace.residvars[i] = create_aux_var (ds, "RES");
- proto = caseproto_add_width (proto, 0);
- }
- }
-
- if (regression.pred)
- {
- workspace.pred_idx = workspace.extras ++;
- workspace.predvars = xcalloc (regression.n_dep_vars, sizeof (*workspace.predvars));
-
- for (size_t i = 0; i < regression.n_dep_vars; ++i)
- {
- workspace.predvars[i] = create_aux_var (ds, "PRED");
- proto = caseproto_add_width (proto, 0);
- }
- }
-
- if (proc_make_temporary_transformations_permanent (ds))
- lex_ofs_msg (lexer, SW, save_start, save_end,
- _("REGRESSION with SAVE ignores TEMPORARY. "
- "Temporary transformations will be made permanent."));
-
- if (dict_get_filter (dict))
- lex_ofs_msg (lexer, SW, save_start, save_end,
- _("REGRESSION with SAVE ignores FILTER. "
- "All cases will be processed."));
-
- workspace.writer = autopaging_writer_create (proto);
- caseproto_unref (proto);
- }
-
- struct casegrouper *grouper = casegrouper_create_splits (
- proc_open_filtering (ds, !save), dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- run_regression (®ression,
- &workspace,
- group);
-
- }
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
- if (workspace.writer)
- {
- struct save_trans_data *save_trans_data = xmalloc (sizeof *save_trans_data);
- struct casereader *r = casewriter_make_reader (workspace.writer);
- workspace.writer = NULL;
- workspace.reader = r;
- save_trans_data->ws = xmalloc (sizeof (workspace));
- memcpy (save_trans_data->ws, &workspace, sizeof (workspace));
- save_trans_data->n_dep_vars = regression.n_dep_vars;
-
- static const struct trns_class trns_class = {
- .name = "REGRESSION",
- .execute = save_trans_func,
- .destroy = save_trans_free,
- };
- add_transformation (ds, &trns_class, save_trans_data);
- }
-
- free (regression.vars);
- free (regression.dep_vars);
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-
-error:
- free (regression.vars);
- free (regression.dep_vars);
- return CMD_FAILURE;
-}
-
-/* Return the size of the union of dependent and independent variables */
-static size_t
-get_n_all_vars (const struct regression *cmd)
-{
- size_t result = cmd->n_vars + cmd->n_dep_vars;
- for (size_t i = 0; i < cmd->n_dep_vars; i++)
- for (size_t j = 0; j < cmd->n_vars; j++)
- if (cmd->vars[j] == cmd->dep_vars[i])
- result--;
- return result;
-}
-
-/* Fill VARS with the union of dependent and independent variables */
-static void
-fill_all_vars (const struct variable **vars, const struct regression *cmd)
-{
- for (size_t i = 0; i < cmd->n_vars; i++)
- vars[i] = cmd->vars[i];
-
- size_t x = 0;
- for (size_t i = 0; i < cmd->n_dep_vars; i++)
- {
- bool absent = true;
- for (size_t j = 0; j < cmd->n_vars; j++)
- if (cmd->dep_vars[i] == cmd->vars[j])
- {
- absent = false;
- break;
- }
- if (absent)
- vars[cmd->n_vars + x++] = cmd->dep_vars[i];
- }
-}
-
-
-/* Fill the array VARS, with all the predictor variables from CMD, except
- variable X */
-static void
-fill_predictor_x (const struct variable **vars, const struct variable *x, const struct regression *cmd)
-{
- size_t n = 0;
- for (size_t i = 0; i < cmd->n_vars; i++)
- if (cmd->vars[i] != x)
- vars[n++] = cmd->vars[i];
-}
-
-/*
- Is variable k the dependent variable?
-*/
-static bool
-is_depvar (const struct regression *cmd, size_t k, const struct variable *v)
-{
- return v == cmd->vars[k];
-}
-
-/* Identify the explanatory variables in v_variables. Returns
- the number of independent variables. */
-static int
-identify_indep_vars (const struct regression *cmd,
- const struct variable **indep_vars,
- const struct variable *depvar)
-{
- int n_indep_vars = 0;
-
- for (size_t i = 0; i < cmd->n_vars; i++)
- if (!is_depvar (cmd, i, depvar))
- indep_vars[n_indep_vars++] = cmd->vars[i];
- if (n_indep_vars < 1 && is_depvar (cmd, 0, depvar))
- {
- /*
- There is only one independent variable, and it is the same
- as the dependent variable. Print a warning and continue.
- */
- msg (SW,
- _("The dependent variable is equal to the independent variable. "
- "The least squares line is therefore Y=X. "
- "Standard errors and related statistics may be meaningless."));
- n_indep_vars = 1;
- indep_vars[0] = cmd->vars[0];
- }
- return n_indep_vars;
-}
-
-static double
-fill_covariance (gsl_matrix * cov, struct covariance *all_cov,
- const struct variable **vars,
- size_t n_vars, const struct variable *dep_var,
- const struct variable **all_vars, size_t n_all_vars,
- double *means)
-{
- const gsl_matrix *cm = covariance_calculate_unnormalized (all_cov);
- if (!cm)
- return 0;
-
- size_t *rows = xnmalloc (cov->size1 - 1, sizeof (*rows));
-
- size_t dep_subscript = SIZE_MAX;
- for (size_t i = 0; i < n_all_vars; i++)
- {
- for (size_t j = 0; j < n_vars; j++)
- if (vars[j] == all_vars[i])
- rows[j] = i;
- if (all_vars[i] == dep_var)
- dep_subscript = i;
- }
- assert (dep_subscript != SIZE_MAX);
-
- const gsl_matrix *mean_matrix = covariance_moments (all_cov, MOMENT_MEAN);
- const gsl_matrix *ssize_matrix = covariance_moments (all_cov, MOMENT_NONE);
- for (size_t i = 0; i < cov->size1 - 1; i++)
- {
- means[i] = gsl_matrix_get (mean_matrix, rows[i], 0)
- / gsl_matrix_get (ssize_matrix, rows[i], 0);
- for (size_t j = 0; j < cov->size2 - 1; j++)
- {
- gsl_matrix_set (cov, i, j, gsl_matrix_get (cm, rows[i], rows[j]));
- gsl_matrix_set (cov, j, i, gsl_matrix_get (cm, rows[j], rows[i]));
- }
- }
- means[cov->size1 - 1] = gsl_matrix_get (mean_matrix, dep_subscript, 0)
- / gsl_matrix_get (ssize_matrix, dep_subscript, 0);
- const gsl_matrix *ssizes = covariance_moments (all_cov, MOMENT_NONE);
- double result = gsl_matrix_get (ssizes, dep_subscript, rows[0]);
- for (size_t i = 0; i < cov->size1 - 1; i++)
- {
- gsl_matrix_set (cov, i, cov->size1 - 1,
- gsl_matrix_get (cm, rows[i], dep_subscript));
- gsl_matrix_set (cov, cov->size1 - 1, i,
- gsl_matrix_get (cm, rows[i], dep_subscript));
- if (result > gsl_matrix_get (ssizes, rows[i], dep_subscript))
- result = gsl_matrix_get (ssizes, rows[i], dep_subscript);
- }
- gsl_matrix_set (cov, cov->size1 - 1, cov->size1 - 1,
- gsl_matrix_get (cm, dep_subscript, dep_subscript));
- free (rows);
- return result;
-}
-
-\f
-
-struct model_container
-{
- struct linreg **models;
-};
-
-/*
- STATISTICS subcommand output functions.
-*/
-static void reg_stats_r (const struct linreg *, const struct variable *);
-static void reg_stats_coeff (const struct regression *, const struct linreg *,
- const struct model_container *, const gsl_matrix *,
- const struct variable *);
-static void reg_stats_anova (const struct linreg *, const struct variable *);
-static void reg_stats_bcov (const struct linreg *, const struct variable *);
-
-
-static struct linreg **
-run_regression_get_models (const struct regression *cmd,
- struct casereader *input,
- bool output)
-{
- struct model_container *model_container = XCALLOC (cmd->n_vars, struct model_container);
-
- struct ccase *c;
- struct covariance *cov;
- struct casereader *reader;
-
- if (cmd->stats & STATS_TOL)
- for (size_t i = 0; i < cmd->n_vars; i++)
- {
- struct regression subreg = {
- .origin = cmd->origin,
- .ds = cmd->ds,
- .n_vars = cmd->n_vars - 1,
- .n_dep_vars = 1,
- .vars = xmalloc ((cmd->n_vars - 1) * sizeof *subreg.vars),
- .dep_vars = &cmd->vars[i],
- .stats = STATS_R,
- .ci = 0,
- .resid = false,
- .pred = false,
- };
- fill_predictor_x (subreg.vars, cmd->vars[i], cmd);
-
- model_container[i].models =
- run_regression_get_models (&subreg, input, false);
- free (subreg.vars);
- }
-
- size_t n_all_vars = get_n_all_vars (cmd);
- const struct variable **all_vars = xnmalloc (n_all_vars, sizeof (*all_vars));
-
- /* In the (rather pointless) case where the dependent variable is
- the independent variable, n_all_vars == 1.
- However this would result in a buffer overflow so we must
- over-allocate the space required in this malloc call.
- See bug #58599 */
- double *means = xnmalloc (MAX (2, n_all_vars), sizeof *means);
- fill_all_vars (all_vars, cmd);
- cov = covariance_1pass_create (n_all_vars, all_vars,
- dict_get_weight (dataset_dict (cmd->ds)),
- MV_ANY, cmd->origin == false);
-
- reader = casereader_clone (input);
- reader = casereader_create_filter_missing (reader, all_vars, n_all_vars,
- MV_ANY, NULL, NULL);
-
- struct casereader *r = casereader_clone (reader);
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- covariance_accumulate (cov, c);
- casereader_destroy (r);
-
- struct linreg **models = XCALLOC (cmd->n_dep_vars, struct linreg*);
- for (size_t k = 0; k < cmd->n_dep_vars; k++)
- {
- const struct variable **vars = xnmalloc (cmd->n_vars, sizeof *vars);
- const struct variable *dep_var = cmd->dep_vars[k];
- int n_indep = identify_indep_vars (cmd, vars, dep_var);
- gsl_matrix *cov_matrix = gsl_matrix_alloc (n_indep + 1, n_indep + 1);
- double n_data = fill_covariance (cov_matrix, cov, vars, n_indep,
- dep_var, all_vars, n_all_vars, means);
- models[k] = linreg_alloc (dep_var, vars, n_data, n_indep, cmd->origin);
- for (size_t i = 0; i < n_indep; i++)
- linreg_set_indep_variable_mean (models[k], i, means[i]);
- linreg_set_depvar_mean (models[k], means[n_indep]);
- if (n_data > 0)
- {
- linreg_fit (cov_matrix, models[k]);
-
- if (output
- && !taint_has_tainted_successor (casereader_get_taint (input)))
- {
- /*
- Find the least-squares estimates and other statistics.
- */
- if (cmd->stats & STATS_R)
- reg_stats_r (models[k], dep_var);
-
- if (cmd->stats & STATS_ANOVA)
- reg_stats_anova (models[k], dep_var);
-
- if (cmd->stats & STATS_COEFF)
- reg_stats_coeff (cmd, models[k],
- model_container,
- cov_matrix, dep_var);
-
- if (cmd->stats & STATS_BCOV)
- reg_stats_bcov (models[k], dep_var);
- }
- }
- else
- msg (SE, _("No valid data found. This command was skipped."));
- free (vars);
- gsl_matrix_free (cov_matrix);
- }
-
- casereader_destroy (reader);
-
- for (size_t i = 0; i < cmd->n_vars; i++)
- {
- if (model_container[i].models)
- linreg_unref (model_container[i].models[0]);
- free (model_container[i].models);
- }
- free (model_container);
-
- free (all_vars);
- free (means);
- covariance_destroy (cov);
- return models;
-}
-
-static void
-run_regression (const struct regression *cmd,
- struct regression_workspace *ws,
- struct casereader *input)
-{
- struct linreg **models = run_regression_get_models (cmd, input, true);
-
- if (ws->extras > 0)
- {
- struct ccase *c;
- struct casereader *r = casereader_clone (input);
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- struct ccase *outc = case_create (casewriter_get_proto (ws->writer));
- for (int k = 0; k < cmd->n_dep_vars; k++)
- {
- const struct variable **vars = xnmalloc (cmd->n_vars, sizeof (*vars));
- const struct variable *dep_var = cmd->dep_vars[k];
- int n_indep = identify_indep_vars (cmd, vars, dep_var);
- double *vals = xnmalloc (n_indep, sizeof (*vals));
- for (int i = 0; i < n_indep; i++)
- {
- const union value *tmp = case_data (c, vars[i]);
- vals[i] = tmp->f;
- }
-
- if (cmd->pred)
- {
- double pred = linreg_predict (models[k], vals, n_indep);
- *case_num_rw_idx (outc, k * ws->extras + ws->pred_idx) = pred;
- }
-
- if (cmd->resid)
- {
- double obs = case_num (c, linreg_dep_var (models[k]));
- double res = linreg_residual (models[k], obs, vals, n_indep);
- *case_num_rw_idx (outc, k * ws->extras + ws->res_idx) = res;
- }
- free (vals);
- free (vars);
- }
- casewriter_write (ws->writer, outc);
- }
- casereader_destroy (r);
- }
-
- for (size_t k = 0; k < cmd->n_dep_vars; k++)
- linreg_unref (models[k]);
-
- free (models);
- casereader_destroy (input);
-}
-
-\f
-
-
-static void
-reg_stats_r (const struct linreg * c, const struct variable *var)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_text_format (N_("Model Summary (%s)"),
- var_to_string (var)),
- "Model Summary");
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("R"), N_("R Square"), N_("Adjusted R Square"),
- N_("Std. Error of the Estimate"));
-
- double rsq = linreg_ssreg (c) / linreg_sst (c);
- double adjrsq = (rsq -
- (1.0 - rsq) * linreg_n_coeffs (c)
- / (linreg_n_obs (c) - linreg_n_coeffs (c) - 1));
- double std_error = sqrt (linreg_mse (c));
-
- double entries[] = {
- sqrt (rsq), rsq, adjrsq, std_error
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
-
- pivot_table_submit (table);
-}
-
-/*
- Table showing estimated regression coefficients.
-*/
-static void
-reg_stats_coeff (const struct regression *cmd, const struct linreg *c,
- const struct model_container *mc, const gsl_matrix *cov,
- const struct variable *var)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_text_format (N_("Coefficients (%s)"), var_to_string (var)),
- "Coefficients");
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- pivot_category_create_group (statistics->root,
- N_("Unstandardized Coefficients"),
- N_("B"), N_("Std. Error"));
- pivot_category_create_group (statistics->root,
- N_("Standardized Coefficients"), N_("Beta"));
- pivot_category_create_leaves (statistics->root, N_("t"),
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
- if (cmd->stats & STATS_CI)
- {
- struct pivot_category *interval = pivot_category_create_group__ (
- statistics->root, pivot_value_new_text_format (
- N_("%g%% Confidence Interval for B"),
- cmd->ci * 100.0));
- pivot_category_create_leaves (interval, N_("Lower Bound"),
- N_("Upper Bound"));
- }
-
- if (cmd->stats & STATS_TOL)
- pivot_category_create_group (statistics->root,
- N_("Collinearity Statistics"),
- N_("Tolerance"), N_("VIF"));
-
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- double df = linreg_n_obs (c) - linreg_n_coeffs (c) - 1;
- double q = (1 - cmd->ci) / 2.0; /* 2-tailed test */
- double tval = gsl_cdf_tdist_Qinv (q, df);
-
- if (!cmd->origin)
- {
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_text (N_("(Constant)")));
-
- double std_err = sqrt (gsl_matrix_get (linreg_cov (c), 0, 0));
- double t_stat = linreg_intercept (c) / std_err;
- double base_entries[] = {
- linreg_intercept (c),
- std_err,
- 0.0,
- t_stat,
- 2.0 * gsl_cdf_tdist_Q (fabs (t_stat),
- linreg_n_obs (c) - linreg_n_coeffs (c)),
- };
-
- size_t col = 0;
- for (size_t i = 0; i < sizeof base_entries / sizeof *base_entries; i++)
- pivot_table_put2 (table, col++, var_idx,
- pivot_value_new_number (base_entries[i]));
-
- if (cmd->stats & STATS_CI)
- {
- double interval_entries[] = {
- linreg_intercept (c) - tval * std_err,
- linreg_intercept (c) + tval * std_err,
- };
-
- for (size_t i = 0; i < sizeof interval_entries / sizeof *interval_entries; i++)
- pivot_table_put2 (table, col++, var_idx,
- pivot_value_new_number (interval_entries[i]));
- }
- }
-
- for (size_t j = 0; j < linreg_n_coeffs (c); j++)
- {
- const struct variable *v = linreg_indep_var (c, j);
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (v));
-
- double std_err = sqrt (gsl_matrix_get (linreg_cov (c), j + 1, j + 1));
- double t_stat = linreg_coeff (c, j) / std_err;
- double base_entries[] = {
- linreg_coeff (c, j),
- sqrt (gsl_matrix_get (linreg_cov (c), j + 1, j + 1)),
- (sqrt (gsl_matrix_get (cov, j, j)) * linreg_coeff (c, j) /
- sqrt (gsl_matrix_get (cov, cov->size1 - 1, cov->size2 - 1))),
- t_stat,
- 2 * gsl_cdf_tdist_Q (fabs (t_stat), df)
- };
-
- size_t col = 0;
- for (size_t i = 0; i < sizeof base_entries / sizeof *base_entries; i++)
- pivot_table_put2 (table, col++, var_idx,
- pivot_value_new_number (base_entries[i]));
-
- if (cmd->stats & STATS_CI)
- {
- double interval_entries[] = {
- linreg_coeff (c, j) - tval * std_err,
- linreg_coeff (c, j) + tval * std_err,
- };
-
-
- for (size_t i = 0; i < sizeof interval_entries / sizeof *interval_entries; i++)
- pivot_table_put2 (table, col++, var_idx,
- pivot_value_new_number (interval_entries[i]));
- }
-
- if (cmd->stats & STATS_TOL)
- {
- {
- struct linreg *m = mc[j].models[0];
- double rsq = linreg_ssreg (m) / linreg_sst (m);
- pivot_table_put2 (table, col++, var_idx, pivot_value_new_number (1.0 - rsq));
- pivot_table_put2 (table, col++, var_idx, pivot_value_new_number (1.0 / (1.0 - rsq)));
- }
- }
- }
-
- pivot_table_submit (table);
-}
-
-/*
- Display the ANOVA table.
-*/
-static void
-reg_stats_anova (const struct linreg * c, const struct variable *var)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_text_format (N_("ANOVA (%s)"), var_to_string (var)),
- "ANOVA");
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Sum of Squares"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_INTEGER,
- N_("Mean Square"), PIVOT_RC_OTHER,
- N_("F"), PIVOT_RC_OTHER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Source"),
- N_("Regression"), N_("Residual"), N_("Total"));
-
- double msm = linreg_ssreg (c) / linreg_dfmodel (c);
- double mse = linreg_mse (c);
- double F = msm / mse;
-
- struct entry
- {
- int stat_idx;
- int source_idx;
- double x;
- }
- entries[] = {
- /* Sums of Squares. */
- { 0, 0, linreg_ssreg (c) },
- { 0, 1, linreg_sse (c) },
- { 0, 2, linreg_sst (c) },
- /* Degrees of freedom. */
- { 1, 0, linreg_dfmodel (c) },
- { 1, 1, linreg_dferror (c) },
- { 1, 2, linreg_dftotal (c) },
- /* Mean Squares. */
- { 2, 0, msm },
- { 2, 1, mse },
- /* F */
- { 3, 0, F },
- /* Significance. */
- { 4, 0, gsl_cdf_fdist_Q (F, linreg_dfmodel (c), linreg_dferror (c)) },
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put2 (table, e->stat_idx, e->source_idx,
- pivot_value_new_number (e->x));
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-reg_stats_bcov (const struct linreg * c, const struct variable *var)
-{
- struct pivot_table *table = pivot_table_create__ (
- pivot_value_new_text_format (N_("Coefficient Correlations (%s)"),
- var_to_string (var)),
- "Coefficient Correlations");
-
- for (size_t i = 0; i < 2; i++)
- {
- struct pivot_dimension *models = pivot_dimension_create (
- table, i ? PIVOT_AXIS_ROW : PIVOT_AXIS_COLUMN, N_("Models"));
- for (size_t j = 0; j < linreg_n_coeffs (c); j++)
- pivot_category_create_leaf (
- models->root, pivot_value_new_variable (
- linreg_indep_var (c, j)));
- }
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Covariances"));
-
- for (size_t i = 0; i < linreg_n_coeffs (c); i++)
- for (size_t k = 0; k < linreg_n_coeffs (c); k++)
- {
- double cov = gsl_matrix_get (linreg_cov (c), MIN (i, k), MAX (i, k));
- pivot_table_put3 (table, k, i, 0, pivot_value_new_number (cov));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009, 2010, 2011, 2013, 2015, 2016 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <math.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/str.h"
-#include "math/moments.h"
-#include "output/pivot-table.h"
-#include "output/output-item.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-struct cronbach
- {
- const struct variable **items;
- size_t n_items;
- double alpha;
- double sum_of_variances;
- double variance_of_sums;
- int totals_idx; /* Casereader index into the totals */
-
- struct moments1 **m; /* Moments of the items */
- struct moments1 *total; /* Moments of the totals */
- };
-
-#if 0
-static void
-dump_cronbach (const struct cronbach *s)
-{
- int i;
- printf ("N items %d\n", s->n_items);
- for (i = 0; i < s->n_items; ++i)
- {
- printf ("%s\n", var_get_name (s->items[i]));
- }
-
- printf ("Totals idx %d\n", s->totals_idx);
-
- printf ("scale variance %g\n", s->variance_of_sums);
- printf ("alpha %g\n", s->alpha);
- putchar ('\n');
-}
-#endif
-
-enum model
- {
- MODEL_ALPHA,
- MODEL_SPLIT
- };
-
-
-struct reliability
-{
- const struct variable **vars;
- size_t n_vars;
- enum mv_class exclude;
-
- struct cronbach *sc;
- int n_sc;
-
- int total_start;
-
- char *scale_name;
-
- enum model model;
- int split_point;
-
- bool summary_total;
-
- const struct variable *wv;
-};
-
-
-static bool run_reliability (struct dataset *ds, const struct reliability *reliability);
-
-static void
-reliability_destroy (struct reliability *rel)
-{
- int j;
- free (rel->scale_name);
- if (rel->sc)
- for (j = 0; j < rel->n_sc; ++j)
- {
- int x;
- free (rel->sc[j].items);
- moments1_destroy (rel->sc[j].total);
- if (rel->sc[j].m)
- for (x = 0; x < rel->sc[j].n_items; ++x)
- free (rel->sc[j].m[x]);
- free (rel->sc[j].m);
- }
-
- free (rel->sc);
- free (rel->vars);
-}
-
-int
-cmd_reliability (struct lexer *lexer, struct dataset *ds)
-{
- const struct dictionary *dict = dataset_dict (ds);
-
- struct reliability r = {
- .model = MODEL_ALPHA,
- .exclude = MV_ANY,
- .wv = dict_get_weight (dict),
- .scale_name = xstrdup ("ANY"),
- };
-
- lex_match (lexer, T_SLASH);
-
- if (!lex_force_match_id (lexer, "VARIABLES"))
- goto error;
-
- lex_match (lexer, T_EQUALS);
-
- int vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, dict, &r.vars, &r.n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
- int vars_end = lex_ofs (lexer) - 1;
-
- if (r.n_vars < 2)
- lex_ofs_msg (lexer, SW, vars_start, vars_end,
- _("Reliability on a single variable is not useful."));
-
- /* Create a default scale. */
- r.n_sc = 1;
- r.sc = xcalloc (r.n_sc, sizeof (struct cronbach));
-
- struct cronbach *c = &r.sc[0];
- c->n_items = r.n_vars;
- c->items = xcalloc (c->n_items, sizeof (struct variable*));
- for (size_t i = 0; i < c->n_items; ++i)
- c->items[i] = r.vars[i];
-
- int split_ofs = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "SCALE"))
- {
- struct const_var_set *vs;
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- if (!lex_force_string (lexer))
- goto error;
- free (r.scale_name);
- r.scale_name = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
-
- lex_match (lexer, T_EQUALS);
-
- vs = const_var_set_create_from_array (r.vars, r.n_vars);
-
- free (r.sc->items);
- if (!parse_const_var_set_vars (lexer, vs, &r.sc->items, &r.sc->n_items, 0))
- {
- const_var_set_destroy (vs);
- goto error;
- }
-
- const_var_set_destroy (vs);
- }
- else if (lex_match_id (lexer, "MODEL"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "ALPHA"))
- r.model = MODEL_ALPHA;
- else if (lex_match_id (lexer, "SPLIT"))
- {
- r.model = MODEL_SPLIT;
- r.split_point = -1;
-
- if (lex_match (lexer, T_LPAREN))
- {
- if (!lex_force_num (lexer))
- goto error;
- split_ofs = lex_ofs (lexer);
- r.split_point = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
- else
- {
- lex_error_expecting (lexer, "ALPHA", "SPLIT");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "SUMMARY"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "TOTAL") || lex_match (lexer, T_ALL))
- r.summary_total = true;
- else
- {
- lex_error_expecting (lexer, "TOTAL", "ALL");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- r.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- r.exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "STATISTICS"))
- {
- int statistics_start = lex_ofs (lexer) - 1;
- lex_match (lexer, T_EQUALS);
- while (lex_match (lexer, T_ID))
- continue;
- int statistics_end = lex_ofs (lexer) - 1;
-
- lex_ofs_msg (lexer, SW, statistics_start, statistics_end,
- _("The STATISTICS subcommand is not yet implemented. "
- "No statistics will be produced."));
- }
- else
- {
- lex_error_expecting (lexer, "SCALE", "MODEL", "SUMMARY", "MISSING",
- "STATISTICS");
- goto error;
- }
- }
-
- if (r.model == MODEL_SPLIT)
- {
- if (r.split_point >= r.n_vars)
- {
- lex_ofs_error (lexer, split_ofs, split_ofs,
- _("The split point must be less than the "
- "number of variables."));
- lex_ofs_msg (lexer, SN, vars_start, vars_end,
- ngettext ("There is %zu variable.",
- "There are %zu variables.", r.n_vars),
- r.n_vars);
- goto error;
- }
-
- r.n_sc += 2;
- r.sc = xrealloc (r.sc, sizeof (struct cronbach) * r.n_sc);
-
- const struct cronbach *s = &r.sc[0];
-
- r.sc[1].n_items = r.split_point == -1 ? s->n_items / 2 : r.split_point;
-
- r.sc[2].n_items = s->n_items - r.sc[1].n_items;
- r.sc[1].items = XCALLOC (r.sc[1].n_items, const struct variable *);
- r.sc[2].items = XCALLOC (r.sc[2].n_items, const struct variable *);
-
- size_t i = 0;
- while (i < r.sc[1].n_items)
- {
- r.sc[1].items[i] = s->items[i];
- i++;
- }
- while (i < s->n_items)
- {
- r.sc[2].items[i - r.sc[1].n_items] = s->items[i];
- i++;
- }
- }
-
- if (r.summary_total)
- {
- const int base_sc = r.n_sc;
-
- r.total_start = base_sc;
-
- r.n_sc += r.sc[0].n_items;
- r.sc = xrealloc (r.sc, sizeof (struct cronbach) * r.n_sc);
-
- for (size_t i = 0; i < r.sc[0].n_items; ++i)
- {
- struct cronbach *s = &r.sc[i + base_sc];
-
- s->n_items = r.sc[0].n_items - 1;
- s->items = xcalloc (s->n_items, sizeof (struct variable *));
-
- size_t v_dest = 0;
- for (size_t v_src = 0; v_src < r.sc[0].n_items; ++v_src)
- if (v_src != i)
- s->items[v_dest++] = r.sc[0].items[v_src];
- }
- }
-
- if (!run_reliability (ds, &r))
- goto error;
-
- reliability_destroy (&r);
- return CMD_SUCCESS;
-
- error:
- reliability_destroy (&r);
- return CMD_FAILURE;
-}
-
-
-static void
-do_reliability (struct casereader *group, struct dataset *ds,
- const struct reliability *rel);
-
-
-static void reliability_summary_total (const struct reliability *rel);
-
-static void reliability_statistics (const struct reliability *rel);
-
-
-static bool
-run_reliability (struct dataset *ds, const struct reliability *reliability)
-{
- for (size_t si = 0; si < reliability->n_sc; ++si)
- {
- struct cronbach *s = &reliability->sc[si];
-
- s->m = xcalloc (s->n_items, sizeof *s->m);
- s->total = moments1_create (MOMENT_VARIANCE);
-
- for (size_t i = 0; i < s->n_items; ++i)
- s->m[i] = moments1_create (MOMENT_VARIANCE);
- }
-
- struct dictionary *dict = dataset_dict (ds);
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
- struct casereader *group;
- while (casegrouper_get_next_group (grouper, &group))
- {
- do_reliability (group, ds, reliability);
-
- reliability_statistics (reliability);
-
- if (reliability->summary_total)
- reliability_summary_total (reliability);
- }
-
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
- return ok;
-}
-\f
-/* Return the sum of all the item variables in S */
-static double
-append_sum (const struct ccase *c, casenumber n UNUSED, void *aux)
-{
- double sum = 0;
- const struct cronbach *s = aux;
-
- for (int v = 0; v < s->n_items; ++v)
- sum += case_num (c, s->items[v]);
-
- return sum;
-}
-
-static void
-case_processing_summary (casenumber n_valid, casenumber n_missing,
- const struct dictionary *);
-
-static double
-alpha (int k, double sum_of_variances, double variance_of_sums)
-{
- return k / (k - 1.0) * (1 - sum_of_variances / variance_of_sums);
-}
-
-static void
-do_reliability (struct casereader *input, struct dataset *ds,
- const struct reliability *rel)
-{
- for (size_t si = 0; si < rel->n_sc; ++si)
- {
- struct cronbach *s = &rel->sc[si];
-
- moments1_clear (s->total);
- for (size_t i = 0; i < s->n_items; ++i)
- moments1_clear (s->m[i]);
- }
-
- casenumber n_missing;
- input = casereader_create_filter_missing (input,
- rel->vars,
- rel->n_vars,
- rel->exclude,
- &n_missing,
- NULL);
-
- for (size_t si = 0; si < rel->n_sc; ++si)
- {
- struct cronbach *s = &rel->sc[si];
- s->totals_idx = caseproto_get_n_widths (casereader_get_proto (input));
- input = casereader_create_append_numeric (input, append_sum, s, NULL);
- }
-
- struct ccase *c;
- casenumber n_valid = 0;
- for (; (c = casereader_read (input)) != NULL; case_unref (c))
- {
- double weight = 1.0;
- n_valid++;
-
- for (size_t si = 0; si < rel->n_sc; ++si)
- {
- struct cronbach *s = &rel->sc[si];
-
- for (size_t i = 0; i < s->n_items; ++i)
- moments1_add (s->m[i], case_num (c, s->items[i]), weight);
- moments1_add (s->total, case_num_idx (c, s->totals_idx), weight);
- }
- }
- casereader_destroy (input);
-
- for (size_t si = 0; si < rel->n_sc; ++si)
- {
- struct cronbach *s = &rel->sc[si];
-
- s->sum_of_variances = 0;
- for (size_t i = 0; i < s->n_items; ++i)
- {
- double weight, mean, variance;
- moments1_calculate (s->m[i], &weight, &mean, &variance, NULL, NULL);
-
- s->sum_of_variances += variance;
- }
-
- moments1_calculate (s->total, NULL, NULL, &s->variance_of_sums,
- NULL, NULL);
-
- s->alpha = alpha (s->n_items, s->sum_of_variances, s->variance_of_sums);
- }
-
- output_item_submit (text_item_create_nocopy (
- TEXT_ITEM_TITLE,
- xasprintf (_("Scale: %s"), rel->scale_name),
- NULL));
-
- case_processing_summary (n_valid, n_missing, dataset_dict (ds));
-}
-
-static void
-case_processing_summary (casenumber n_valid, casenumber n_missing,
- const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Case Processing Summary"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Percent"), PIVOT_RC_PERCENT);
-
- struct pivot_dimension *cases = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Cases"), N_("Valid"), N_("Excluded"),
- N_("Total"));
- cases->root->show_label = true;
-
- casenumber total = n_missing + n_valid;
-
- struct entry
- {
- int stat_idx;
- int case_idx;
- double x;
- }
- entries[] = {
- { 0, 0, n_valid },
- { 0, 1, n_missing },
- { 0, 2, total },
- { 1, 0, 100.0 * n_valid / total },
- { 1, 1, 100.0 * n_missing / total },
- { 1, 2, 100.0 }
- };
-
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put2 (table, e->stat_idx, e->case_idx,
- pivot_value_new_number (e->x));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-reliability_summary_total (const struct reliability *rel)
-{
- struct pivot_table *table = pivot_table_create (N_("Item-Total Statistics"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Scale Mean if Item Deleted"),
- N_("Scale Variance if Item Deleted"),
- N_("Corrected Item-Total Correlation"),
- N_("Cronbach's Alpha if Item Deleted"));
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0; i < rel->sc[0].n_items; ++i)
- {
- const struct cronbach *s = &rel->sc[rel->total_start + i];
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (rel->sc[0].items[i]));
-
- double mean;
- moments1_calculate (s->total, NULL, &mean, NULL, NULL, NULL);
-
- double var;
- moments1_calculate (rel->sc[0].m[i], NULL, NULL, &var, NULL, NULL);
- double cov
- = (rel->sc[0].variance_of_sums + var - s->variance_of_sums) / 2.0;
-
- double entries[] = {
- mean,
- s->variance_of_sums,
- (cov - var) / sqrt (var * s->variance_of_sums),
- s->alpha,
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, var_idx,
- pivot_value_new_number (entries[i]));
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-reliability_statistics (const struct reliability *rel)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Reliability Statistics"));
- pivot_table_set_weight_var (table, rel->wv);
-
- if (rel->model == MODEL_ALPHA)
- {
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN,
- N_("Statistics"),
- N_("Cronbach's Alpha"), PIVOT_RC_OTHER,
- N_("N of Items"), PIVOT_RC_COUNT);
-
- const struct cronbach *s = &rel->sc[0];
- pivot_table_put1 (table, 0, pivot_value_new_number (s->alpha));
- pivot_table_put1 (table, 1, pivot_value_new_number (s->n_items));
- }
- else
- {
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"));
- struct pivot_category *alpha = pivot_category_create_group (
- statistics->root, N_("Cronbach's Alpha"));
- pivot_category_create_group (alpha, N_("Part 1"),
- N_("Value"), PIVOT_RC_OTHER,
- N_("N of Items"), PIVOT_RC_COUNT);
- pivot_category_create_group (alpha, N_("Part 2"),
- N_("Value"), PIVOT_RC_OTHER,
- N_("N of Items"), PIVOT_RC_COUNT);
- pivot_category_create_leaves (alpha,
- N_("Total N of Items"), PIVOT_RC_COUNT);
- pivot_category_create_leaves (statistics->root,
- N_("Correlation Between Forms"),
- PIVOT_RC_OTHER);
- pivot_category_create_group (statistics->root,
- N_("Spearman-Brown Coefficient"),
- N_("Equal Length"), PIVOT_RC_OTHER,
- N_("Unequal Length"), PIVOT_RC_OTHER);
- pivot_category_create_leaves (statistics->root,
- N_("Guttman Split-Half Coefficient"),
- PIVOT_RC_OTHER);
-
- /* R is the correlation between the two parts */
- double r0 = rel->sc[0].variance_of_sums -
- rel->sc[1].variance_of_sums -
- rel->sc[2].variance_of_sums;
- double r1 = (r0 / sqrt (rel->sc[1].variance_of_sums)
- / sqrt (rel->sc[2].variance_of_sums)
- / 2.0);
-
- /* Guttman Split Half Coefficient */
- double g = 2 * r0 / rel->sc[0].variance_of_sums;
-
- double tmp = (1.0 - r1*r1) * rel->sc[1].n_items * rel->sc[2].n_items /
- pow2 (rel->sc[0].n_items);
-
- double entries[] = {
- rel->sc[1].alpha,
- rel->sc[1].n_items,
- rel->sc[2].alpha,
- rel->sc[2].n_items,
- rel->sc[1].n_items + rel->sc[2].n_items,
- r1,
- 2 * r1 / (1.0 + r1),
- (sqrt (pow4 (r1) + 4 * pow2 (r1) * tmp) - pow2 (r1)) / (2 * tmp),
- g,
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put1 (table, i, pivot_value_new_number (entries[i]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/roc.h"
-
-#include <gsl/gsl_cdf.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/misc.h"
-#include "math/sort.h"
-#include "output/charts/roc-chart.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-
-struct cmd_roc
-{
- size_t n_vars;
- const struct variable **vars;
- const struct dictionary *dict;
-
- const struct variable *state_var;
- union value state_value;
- size_t state_var_width;
-
- /* Plot the roc curve */
- bool curve;
- /* Plot the reference line */
- bool reference;
-
- double ci;
-
- bool print_coords;
- bool print_se;
- bool bi_neg_exp; /* True iff the bi-negative exponential critieria
- should be used */
- enum mv_class exclude;
-
- bool invert; /* True iff a smaller test result variable indicates
- a positive result */
-
- double pos;
- double neg;
- double pos_weighted;
- double neg_weighted;
-};
-
-static int run_roc (struct dataset *, struct cmd_roc *);
-static void do_roc (struct cmd_roc *, struct casereader *, struct dictionary *);
-
-
-int
-cmd_roc (struct lexer *lexer, struct dataset *ds)
-{
- const struct dictionary *dict = dataset_dict (ds);
-
- struct cmd_roc roc = {
- .exclude = MV_ANY,
- .curve = true,
- .ci = 95,
- .dict = dict,
- .state_var_width = -1,
- };
-
- lex_match (lexer, T_SLASH);
- if (!parse_variables_const (lexer, dict, &roc.vars, &roc.n_vars,
- PV_APPEND | PV_NO_DUPLICATE | PV_NUMERIC))
- goto error;
-
- if (!lex_force_match (lexer, T_BY))
- goto error;
-
- roc.state_var = parse_variable (lexer, dict);
- if (!roc.state_var)
- goto error;
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
-
- roc.state_var_width = var_get_width (roc.state_var);
- value_init (&roc.state_value, roc.state_var_width);
- if (!parse_value (lexer, &roc.state_value, roc.state_var)
- || !lex_force_match (lexer, T_RPAREN))
- goto error;
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- roc.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- roc.exclude = MV_ANY;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "PLOT"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "CURVE"))
- {
- roc.curve = true;
- if (lex_match (lexer, T_LPAREN))
- {
- roc.reference = true;
- if (!lex_force_match_id (lexer, "REFERENCE")
- || !lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- }
- else if (lex_match_id (lexer, "NONE"))
- roc.curve = false;
- else
- {
- lex_error_expecting (lexer, "CURVE", "NONE");
- goto error;
- }
- }
- else if (lex_match_id (lexer, "PRINT"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "SE"))
- roc.print_se = true;
- else if (lex_match_id (lexer, "COORDINATES"))
- roc.print_coords = true;
- else
- {
- lex_error_expecting (lexer, "SE", "COORDINATES");
- goto error;
- }
- }
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "CUTOFF"))
- {
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
- if (lex_match_id (lexer, "INCLUDE"))
- roc.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- roc.exclude = MV_USER | MV_SYSTEM;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "TESTPOS"))
- {
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
- if (lex_match_id (lexer, "LARGE"))
- roc.invert = false;
- else if (lex_match_id (lexer, "SMALL"))
- roc.invert = true;
- else
- {
- lex_error_expecting (lexer, "LARGE", "SMALL");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "CI"))
- {
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
- if (!lex_force_num (lexer))
- goto error;
- roc.ci = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else if (lex_match_id (lexer, "DISTRIBUTION"))
- {
- if (!lex_force_match (lexer, T_LPAREN))
- goto error;
- if (lex_match_id (lexer, "FREE"))
- roc.bi_neg_exp = false;
- else if (lex_match_id (lexer, "NEGEXPO"))
- roc.bi_neg_exp = true;
- else
- {
- lex_error_expecting (lexer, "FREE", "NEGEXPO");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "CUTOFF", "TESTPOS", "CI",
- "DISTRIBUTION");
- goto error;
- }
- }
- }
- else
- {
- lex_error_expecting (lexer, "MISSING", "PLOT", "PRINT", "CRITERIA");
- goto error;
- }
- }
-
- if (!run_roc (ds, &roc))
- goto error;
-
- if (roc.state_var)
- value_destroy (&roc.state_value, roc.state_var_width);
- free (roc.vars);
- return CMD_SUCCESS;
-
- error:
- if (roc.state_var)
- value_destroy (&roc.state_value, roc.state_var_width);
- free (roc.vars);
- return CMD_FAILURE;
-}
-
-static int
-run_roc (struct dataset *ds, struct cmd_roc *roc)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct casereader *group;
-
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
- while (casegrouper_get_next_group (grouper, &group))
- do_roc (roc, group, dataset_dict (ds));
-
- bool ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
- return ok;
-}
-
-#if 0
-static void
-dump_casereader (struct casereader *reader)
-{
- struct ccase *c;
- struct casereader *r = casereader_clone (reader);
-
- for (; (c = casereader_read (r)); case_unref (c))
- {
- for (size_t i = 0; i < case_get_n_values (c); ++i)
- printf ("%g ", case_num_idx (c, i));
- printf ("\n");
- }
-
- casereader_destroy (r);
-}
-#endif
-
-
-/*
- Return true iff the state variable indicates that C has positive actual state.
-
- As a side effect, this function also accumulates the roc->{pos,neg} and
- roc->{pos,neg}_weighted counts.
- */
-static bool
-match_positives (const struct ccase *c, void *aux)
-{
- struct cmd_roc *roc = aux;
- const struct variable *wv = dict_get_weight (roc->dict);
- const double weight = wv ? case_num (c, wv) : 1.0;
-
- const bool positive =
- (0 == value_compare_3way (case_data (c, roc->state_var), &roc->state_value,
- var_get_width (roc->state_var)));
-
- if (positive)
- {
- roc->pos++;
- roc->pos_weighted += weight;
- }
- else
- {
- roc->neg++;
- roc->neg_weighted += weight;
- }
-
- return positive;
-}
-
-
-#define VALUE 0
-#define N_EQ 1
-#define N_PRED 2
-
-/* Some intermediate state for calculating the cutpoints and the
- standard error values */
-struct roc_state
-{
- double auc; /* Area under the curve */
-
- double n1; /* total weight of positives */
- double n2; /* total weight of negatives */
-
- /* intermediates for standard error */
- double q1hat;
- double q2hat;
-
- /* intermediates for cutpoints */
- struct casewriter *cutpoint_wtr;
- struct casereader *cutpoint_rdr;
- double prev_result;
- double min;
- double max;
-};
-
-/*
- Return a new casereader based upon CUTPOINT_RDR.
- The number of "positive" cases are placed into
- the position TRUE_INDEX, and the number of "negative" cases
- into FALSE_INDEX.
- POS_COND and RESULT determine the semantics of what is
- "positive".
- WEIGHT is the value of a single count.
- */
-static struct casereader *
-accumulate_counts (struct casereader *input,
- double result, double weight,
- bool (*pos_cond) (double, double),
- int true_index, int false_index)
-{
- const struct caseproto *proto = casereader_get_proto (input);
- struct casewriter *w =
- autopaging_writer_create (proto);
- struct ccase *cpc;
- double prev_cp = SYSMIS;
-
- for (; (cpc = casereader_read (input)); case_unref (cpc))
- {
- struct ccase *new_case;
- const double cp = case_num_idx (cpc, ROC_CUTPOINT);
-
- assert (cp != SYSMIS);
-
- /* We don't want duplicates here */
- if (cp == prev_cp)
- continue;
-
- new_case = case_clone (cpc);
-
- int index = pos_cond (result, cp) ? true_index : false_index;
- *case_num_rw_idx (new_case, index) += weight;
-
- prev_cp = cp;
-
- casewriter_write (w, new_case);
- }
- casereader_destroy (input);
-
- return casewriter_make_reader (w);
-}
-
-
-
-static void output_roc (struct roc_state *rs, const struct cmd_roc *roc);
-
-/*
- This function does 3 things:
-
- 1. Counts the number of cases which are equal to every other case in READER,
- and those cases for which the relationship between it and every other case
- satifies PRED (normally either > or <). VAR is variable defining a case's value
- for this purpose.
-
- 2. Counts the number of true and false cases in reader, and populates
- CUTPOINT_RDR accordingly. TRUE_INDEX and FALSE_INDEX are the indices
- which receive these values. POS_COND is the condition defining true
- and false.
-
- 3. CC is filled with the cumulative weight of all cases of READER.
-*/
-static struct casereader *
-process_group (const struct variable *var, struct casereader *reader,
- bool (*pred) (double, double),
- const struct dictionary *dict,
- double *cc,
- struct casereader **cutpoint_rdr,
- bool (*pos_cond) (double, double),
- int true_index,
- int false_index)
-{
- const struct variable *w = dict_get_weight (dict);
-
- struct casereader *r1 =
- casereader_create_distinct (sort_execute_1var (reader, var), var, w);
-
- const int weight_idx = w ? var_get_case_index (w) :
- caseproto_get_n_widths (casereader_get_proto (r1)) - 1;
-
- struct ccase *c1;
-
- struct casereader *rclone = casereader_clone (r1);
- struct casewriter *wtr;
- struct caseproto *proto = caseproto_create ();
-
- proto = caseproto_add_width (proto, 0);
- proto = caseproto_add_width (proto, 0);
- proto = caseproto_add_width (proto, 0);
-
- wtr = autopaging_writer_create (proto);
-
- *cc = 0;
-
- for (; (c1 = casereader_read (r1)); case_unref (c1))
- {
- struct ccase *new_case = case_create (proto);
- struct ccase *c2;
- struct casereader *r2 = casereader_clone (rclone);
-
- const double weight1 = case_num_idx (c1, weight_idx);
- const double d1 = case_num (c1, var);
- double n_eq = 0.0;
- double n_pred = 0.0;
-
- *cutpoint_rdr = accumulate_counts (*cutpoint_rdr, d1, weight1,
- pos_cond,
- true_index, false_index);
-
- *cc += weight1;
-
- for (; (c2 = casereader_read (r2)); case_unref (c2))
- {
- const double d2 = case_num (c2, var);
- const double weight2 = case_num_idx (c2, weight_idx);
-
- if (d1 == d2)
- {
- n_eq += weight2;
- continue;
- }
- else if (pred (d2, d1))
- {
- n_pred += weight2;
- }
- }
-
- *case_num_rw_idx (new_case, VALUE) = d1;
- *case_num_rw_idx (new_case, N_EQ) = n_eq;
- *case_num_rw_idx (new_case, N_PRED) = n_pred;
-
- casewriter_write (wtr, new_case);
-
- casereader_destroy (r2);
- }
-
- casereader_destroy (r1);
- casereader_destroy (rclone);
-
- caseproto_unref (proto);
-
- return casewriter_make_reader (wtr);
-}
-
-/* Some more indeces into case data */
-#define N_POS_EQ 1 /* number of positive cases with values equal to n */
-#define N_POS_GT 2 /* number of positive cases with values greater than n */
-#define N_NEG_EQ 3 /* number of negative cases with values equal to n */
-#define N_NEG_LT 4 /* number of negative cases with values less than n */
-
-static bool
-gt (double d1, double d2)
-{
- return d1 > d2;
-}
-
-
-static bool
-ge (double d1, double d2)
-{
- return d1 > d2;
-}
-
-static bool
-lt (double d1, double d2)
-{
- return d1 < d2;
-}
-
-
-/*
- Return a casereader with width 3,
- populated with cases based upon READER.
- The cases will have the values:
- (N, number of cases equal to N, number of cases greater than N)
- As a side effect, update RS->n1 with the number of positive cases.
-*/
-static struct casereader *
-process_positive_group (const struct variable *var, struct casereader *reader,
- const struct dictionary *dict,
- struct roc_state *rs)
-{
- return process_group (var, reader, gt, dict, &rs->n1,
- &rs->cutpoint_rdr,
- ge,
- ROC_TP, ROC_FN);
-}
-
-/*
- Return a casereader with width 3,
- populated with cases based upon READER.
- The cases will have the values:
- (N, number of cases equal to N, number of cases less than N)
- As a side effect, update RS->n2 with the number of negative cases.
-*/
-static struct casereader *
-process_negative_group (const struct variable *var, struct casereader *reader,
- const struct dictionary *dict,
- struct roc_state *rs)
-{
- return process_group (var, reader, lt, dict, &rs->n2,
- &rs->cutpoint_rdr,
- lt,
- ROC_TN, ROC_FP);
-}
-
-
-
-
-static void
-append_cutpoint (struct casewriter *writer, double cutpoint)
-{
- struct ccase *cc = case_create (casewriter_get_proto (writer));
-
- *case_num_rw_idx (cc, ROC_CUTPOINT) = cutpoint;
- *case_num_rw_idx (cc, ROC_TP) = 0;
- *case_num_rw_idx (cc, ROC_FN) = 0;
- *case_num_rw_idx (cc, ROC_TN) = 0;
- *case_num_rw_idx (cc, ROC_FP) = 0;
-
- casewriter_write (writer, cc);
-}
-
-/*
- Create and initialise the rs[x].cutpoint_rdr casereaders. That is, the
- readers will be created with width 5, ready to take the values (cutpoint,
- ROC_TP, ROC_FN, ROC_TN, ROC_FP), and the reader will be populated with its
- final number of cases. However on exit from this function, only
- ROC_CUTPOINT entries will be set to their final value. The other entries
- will be initialised to zero.
-*/
-static struct roc_state *
-prepare_cutpoints (struct cmd_roc *roc, struct casereader *input)
-{
- struct casereader *r = casereader_clone (input);
- struct ccase *c;
-
- struct subcase ordering;
- subcase_init (&ordering, ROC_CUTPOINT, 0, SC_ASCEND);
-
- struct caseproto *proto = caseproto_create ();
- proto = caseproto_add_width (proto, 0); /* cutpoint */
- proto = caseproto_add_width (proto, 0); /* ROC_TP */
- proto = caseproto_add_width (proto, 0); /* ROC_FN */
- proto = caseproto_add_width (proto, 0); /* ROC_TN */
- proto = caseproto_add_width (proto, 0); /* ROC_FP */
-
- struct roc_state *rs = xnmalloc (roc->n_vars, sizeof *rs);
- for (size_t i = 0; i < roc->n_vars; ++i)
- rs[i] = (struct roc_state) {
- .cutpoint_wtr = sort_create_writer (&ordering, proto),
- .prev_result = SYSMIS,
- .max = -DBL_MAX,
- .min = DBL_MAX,
- };
-
- caseproto_unref (proto);
- subcase_uninit (&ordering);
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- for (size_t i = 0; i < roc->n_vars; ++i)
- {
- const union value *v = case_data (c, roc->vars[i]);
- const double result = v->f;
-
- if (mv_is_value_missing (var_get_missing_values (roc->vars[i]), v)
- & roc->exclude)
- continue;
-
- minimize (&rs[i].min, result);
- maximize (&rs[i].max, result);
-
- if (rs[i].prev_result != SYSMIS && rs[i].prev_result != result)
- {
- const double mean = (result + rs[i].prev_result) / 2.0;
- append_cutpoint (rs[i].cutpoint_wtr, mean);
- }
-
- rs[i].prev_result = result;
- }
- casereader_destroy (r);
-
- /* Append the min and max cutpoints */
- for (size_t i = 0; i < roc->n_vars; ++i)
- {
- append_cutpoint (rs[i].cutpoint_wtr, rs[i].min - 1);
- append_cutpoint (rs[i].cutpoint_wtr, rs[i].max + 1);
-
- rs[i].cutpoint_rdr = casewriter_make_reader (rs[i].cutpoint_wtr);
- }
-
- return rs;
-}
-
-static void
-do_roc (struct cmd_roc *roc, struct casereader *reader, struct dictionary *dict)
-{
- struct casereader *input = casereader_create_filter_missing (
- reader, roc->vars, roc->n_vars, roc->exclude, NULL, NULL);
- input = casereader_create_filter_missing (
- input, &roc->state_var, 1, roc->exclude, NULL, NULL);
-
- struct roc_state *rs = prepare_cutpoints (roc, input);
-
- /* Separate the positive actual state cases from the negative ones */
- struct casewriter *neg_wtr
- = autopaging_writer_create (casereader_get_proto (input));
- struct casereader *positives = casereader_create_filter_func (
- input, match_positives, NULL, roc, neg_wtr);
-
- struct caseproto *n_proto = caseproto_create ();
- for (size_t i = 0; i < 5; i++)
- n_proto = caseproto_add_width (n_proto, 0);
-
- struct subcase up_ordering;
- struct subcase down_ordering;
- subcase_init (&up_ordering, VALUE, 0, SC_ASCEND);
- subcase_init (&down_ordering, VALUE, 0, SC_DESCEND);
-
- struct casereader *negatives = NULL;
- for (size_t i = 0; i < roc->n_vars; ++i)
- {
- const struct variable *var = roc->vars[i];
-
- struct casereader *pos = casereader_clone (positives);
-
- struct casereader *n_pos_reader =
- process_positive_group (var, pos, dict, &rs[i]);
-
- if (!negatives)
- negatives = casewriter_make_reader (neg_wtr);
-
- struct casereader *neg = casereader_clone (negatives);
- struct casereader *n_neg_reader
- = process_negative_group (var, neg, dict, &rs[i]);
-
- /* Merge the n_pos and n_neg casereaders */
- struct casewriter *w = sort_create_writer (&up_ordering, n_proto);
- struct ccase *cpos;
- for (; (cpos = casereader_read (n_pos_reader)); case_unref (cpos))
- {
- struct ccase *pos_case = case_create (n_proto);
- const double jpos = case_num_idx (cpos, VALUE);
-
- struct ccase *cneg;
- while ((cneg = casereader_read (n_neg_reader)))
- {
- struct ccase *nc = case_create (n_proto);
-
- const double jneg = case_num_idx (cneg, VALUE);
-
- *case_num_rw_idx (nc, VALUE) = jneg;
- *case_num_rw_idx (nc, N_POS_EQ) = 0;
-
- *case_num_rw_idx (nc, N_POS_GT) = SYSMIS;
-
- *case_data_rw_idx (nc, N_NEG_EQ) = *case_data_idx (cneg, N_EQ);
- *case_data_rw_idx (nc, N_NEG_LT) = *case_data_idx (cneg, N_PRED);
-
- casewriter_write (w, nc);
-
- case_unref (cneg);
- if (jneg > jpos)
- break;
- }
-
- *case_num_rw_idx (pos_case, VALUE) = jpos;
- *case_data_rw_idx (pos_case, N_POS_EQ) = *case_data_idx (cpos, N_EQ);
- *case_data_rw_idx (pos_case, N_POS_GT) = *case_data_idx (cpos, N_PRED);
- *case_num_rw_idx (pos_case, N_NEG_EQ) = 0;
- *case_num_rw_idx (pos_case, N_NEG_LT) = SYSMIS;
-
- casewriter_write (w, pos_case);
- }
-
- casereader_destroy (n_pos_reader);
- casereader_destroy (n_neg_reader);
-
- struct casereader *r = casewriter_make_reader (w);
-
- /* Propagate the N_POS_GT values from the positive cases
- to the negative ones */
- double prev_pos_gt = rs[i].n1;
- w = sort_create_writer (&down_ordering, n_proto);
-
- struct ccase *c;
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double n_pos_gt = case_num_idx (c, N_POS_GT);
- struct ccase *nc = case_clone (c);
-
- if (n_pos_gt == SYSMIS)
- {
- n_pos_gt = prev_pos_gt;
- *case_num_rw_idx (nc, N_POS_GT) = n_pos_gt;
- }
-
- casewriter_write (w, nc);
- prev_pos_gt = n_pos_gt;
- }
- casereader_destroy (r);
- r = casewriter_make_reader (w);
-
- /* Propagate the N_NEG_LT values from the negative cases
- to the positive ones */
- double prev_neg_lt = rs[i].n2;
- w = sort_create_writer (&up_ordering, n_proto);
-
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double n_neg_lt = case_num_idx (c, N_NEG_LT);
- struct ccase *nc = case_clone (c);
-
- if (n_neg_lt == SYSMIS)
- {
- n_neg_lt = prev_neg_lt;
- *case_num_rw_idx (nc, N_NEG_LT) = n_neg_lt;
- }
-
- casewriter_write (w, nc);
- prev_neg_lt = n_neg_lt;
- }
-
- casereader_destroy (r);
- r = casewriter_make_reader (w);
-
- struct ccase *prev_case = NULL;
- for (; (c = casereader_read (r)); case_unref (c))
- {
- struct ccase *next_case = casereader_peek (r, 0);
-
- const double j = case_num_idx (c, VALUE);
- double n_pos_eq = case_num_idx (c, N_POS_EQ);
- double n_pos_gt = case_num_idx (c, N_POS_GT);
- double n_neg_eq = case_num_idx (c, N_NEG_EQ);
- double n_neg_lt = case_num_idx (c, N_NEG_LT);
-
- if (prev_case && j == case_num_idx (prev_case, VALUE))
- {
- if (0 == case_num_idx (c, N_POS_EQ))
- {
- n_pos_eq = case_num_idx (prev_case, N_POS_EQ);
- n_pos_gt = case_num_idx (prev_case, N_POS_GT);
- }
-
- if (0 == case_num_idx (c, N_NEG_EQ))
- {
- n_neg_eq = case_num_idx (prev_case, N_NEG_EQ);
- n_neg_lt = case_num_idx (prev_case, N_NEG_LT);
- }
- }
-
- if (NULL == next_case || j != case_num_idx (next_case, VALUE))
- {
- rs[i].auc += n_pos_gt * n_neg_eq + (n_pos_eq * n_neg_eq) / 2.0;
-
- rs[i].q1hat +=
- n_neg_eq * (pow2 (n_pos_gt) + n_pos_gt * n_pos_eq + pow2 (n_pos_eq) / 3.0);
- rs[i].q2hat +=
- n_pos_eq * (pow2 (n_neg_lt) + n_neg_lt * n_neg_eq + pow2 (n_neg_eq) / 3.0);
-
- }
-
- case_unref (next_case);
- case_unref (prev_case);
- prev_case = case_clone (c);
- }
- casereader_destroy (r);
- case_unref (prev_case);
-
- rs[i].auc /= rs[i].n1 * rs[i].n2;
- if (roc->invert)
- rs[i].auc = 1 - rs[i].auc;
-
- if (roc->bi_neg_exp)
- {
- rs[i].q1hat = rs[i].auc / (2 - rs[i].auc);
- rs[i].q2hat = 2 * pow2 (rs[i].auc) / (1 + rs[i].auc);
- }
- else
- {
- rs[i].q1hat /= rs[i].n2 * pow2 (rs[i].n1);
- rs[i].q2hat /= rs[i].n1 * pow2 (rs[i].n2);
- }
- }
-
- casereader_destroy (positives);
- casereader_destroy (negatives);
-
- caseproto_unref (n_proto);
- subcase_uninit (&up_ordering);
- subcase_uninit (&down_ordering);
-
- output_roc (rs, roc);
-
- for (size_t i = 0; i < roc->n_vars; ++i)
- casereader_destroy (rs[i].cutpoint_rdr);
-
- free (rs);
-}
-
-static void
-show_auc (struct roc_state *rs, const struct cmd_roc *roc)
-{
- struct pivot_table *table = pivot_table_create (N_("Area Under the Curve"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Area"), PIVOT_RC_OTHER);
- if (roc->print_se)
- {
- pivot_category_create_leaves (
- statistics->root,
- N_("Std. Error"), PIVOT_RC_OTHER,
- N_("Asymptotic Sig."), PIVOT_RC_SIGNIFICANCE);
- struct pivot_category *interval = pivot_category_create_group__ (
- statistics->root,
- pivot_value_new_text_format (N_("Asymp. %g%% Confidence Interval"),
- roc->ci));
- pivot_category_create_leaves (interval,
- N_("Lower Bound"), PIVOT_RC_OTHER,
- N_("Upper Bound"), PIVOT_RC_OTHER);
- }
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variable under test"));
- variables->root->show_label = true;
-
- for (size_t i = 0; i < roc->n_vars; ++i)
- {
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (roc->vars[i]));
-
- pivot_table_put2 (table, 0, var_idx, pivot_value_new_number (rs[i].auc));
-
- if (roc->print_se)
- {
- double se = (rs[i].auc * (1 - rs[i].auc)
- + (rs[i].n1 - 1) * (rs[i].q1hat - pow2 (rs[i].auc))
- + (rs[i].n2 - 1) * (rs[i].q2hat - pow2 (rs[i].auc)));
- se /= rs[i].n1 * rs[i].n2;
- se = sqrt (se);
-
- double ci = 1 - roc->ci / 100.0;
- double yy = gsl_cdf_gaussian_Qinv (ci, se);
-
- double sd_0_5 = sqrt ((rs[i].n1 + rs[i].n2 + 1) /
- (12 * rs[i].n1 * rs[i].n2));
- double sig = 2.0 * gsl_cdf_ugaussian_Q (fabs ((rs[i].auc - 0.5)
- / sd_0_5));
- double entries[] = { se, sig, rs[i].auc - yy, rs[i].auc + yy };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i + 1, var_idx,
- pivot_value_new_number (entries[i]));
- }
- }
-
- pivot_table_submit (table);
-}
-
-static void
-show_summary (const struct cmd_roc *roc)
-{
- struct pivot_table *table = pivot_table_create (N_("Case Summary"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Valid N (listwise)"),
- N_("Unweighted"), PIVOT_RC_INTEGER,
- N_("Weighted"), PIVOT_RC_OTHER);
- statistics->root->show_label = true;
-
- struct pivot_dimension *cases = pivot_dimension_create__ (
- table, PIVOT_AXIS_ROW, pivot_value_new_variable (roc->state_var));
- cases->root->show_label = true;
- pivot_category_create_leaves (cases->root, N_("Positive"), N_("Negative"));
-
- struct entry
- {
- int stat_idx;
- int case_idx;
- double x;
- }
- entries[] = {
- { 0, 0, roc->pos },
- { 0, 1, roc->neg },
- { 1, 0, roc->pos_weighted },
- { 1, 1, roc->neg_weighted },
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put2 (table, e->stat_idx, e->case_idx,
- pivot_value_new_number (e->x));
- }
- pivot_table_submit (table);
-}
-
-static void
-show_coords (struct roc_state *rs, const struct cmd_roc *roc)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Coordinates of the Curve"));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("Positive if greater than or equal to"),
- N_("Sensitivity"), N_("1 - Specificity"));
-
- struct pivot_dimension *coordinates = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Coordinates"));
- coordinates->hide_all_labels = true;
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Test variable"));
- variables->root->show_label = true;
-
-
- int n_coords = 0;
- for (size_t i = 0; i < roc->n_vars; ++i)
- {
- struct casereader *r = casereader_clone (rs[i].cutpoint_rdr);
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (roc->vars[i]));
-
- struct ccase *cc;
- int coord_idx = 0;
- for (; (cc = casereader_read (r)) != NULL; case_unref (cc))
- {
- const double se = case_num_idx (cc, ROC_TP) /
- (case_num_idx (cc, ROC_TP) + case_num_idx (cc, ROC_FN));
-
- const double sp = case_num_idx (cc, ROC_TN) /
- (case_num_idx (cc, ROC_TN) + case_num_idx (cc, ROC_FP));
-
- if (coord_idx >= n_coords)
- {
- assert (coord_idx == n_coords);
- pivot_category_create_leaf (
- coordinates->root, pivot_value_new_integer (++n_coords));
- }
-
- pivot_table_put3 (
- table, 0, coord_idx, var_idx,
- pivot_value_new_var_value (roc->vars[i],
- case_data_idx (cc, ROC_CUTPOINT)));
-
- pivot_table_put3 (table, 1, coord_idx, var_idx,
- pivot_value_new_number (se));
- pivot_table_put3 (table, 2, coord_idx, var_idx,
- pivot_value_new_number (1 - sp));
- coord_idx++;
- }
-
- casereader_destroy (r);
- }
-
- pivot_table_submit (table);
-}
-
-static void
-output_roc (struct roc_state *rs, const struct cmd_roc *roc)
-{
- show_summary (roc);
-
- if (roc->curve)
- {
- struct roc_chart *rc = roc_chart_create (roc->reference);
- for (size_t i = 0; i < roc->n_vars; i++)
- roc_chart_add_var (rc, var_get_name (roc->vars[i]),
- rs[i].cutpoint_rdr);
- roc_chart_submit (rc);
- }
-
- show_auc (rs, roc);
-
- if (roc->print_coords)
- show_coords (rs, roc);
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef LANGUAGE_STATS_ROC_H
-#define LANGUAGE_STATS_ROC_H 1
-
-/* These are case indexes into the cutpoint case readers for ROC
- output, used by roc.c and roc-chart.c. */
-#define ROC_CUTPOINT 0
-#define ROC_TP 1
-#define ROC_FN 2
-#define ROC_TN 3
-#define ROC_FP 4
-
-#endif /* language/stats/roc.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis. -*-c-*-
- Copyright (C) 2010, 2011, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include <config.h>
-
-#include "language/stats/runs.h"
-
-#include <float.h>
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/percentiles.h"
-#include "math/sort.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct run_state
-{
- /* The value used to dichotimise the data */
- double cutpoint;
-
- /* The number of cases not less than cutpoint */
- double np;
-
- /* The number of cases less than cutpoint */
- double nn;
-
- /* The sum of np and nn */
- double n;
-
- /* The number of runs */
- long runs;
-
- /* The sign of the last case seen */
- short last_sign;
-};
-
-
-
-/* Return the Z statistic representing the assympototic
- distribution of the number of runs */
-static double
-runs_statistic (const struct run_state *rs)
-{
- double z;
- double sigma;
- double mu = 2 * rs->np * rs->nn;
- mu /= rs->np + rs->nn;
- mu += 1.0;
-
- z = rs->runs - mu;
-
- if (rs->n < 50)
- {
- if (z <= -0.5)
- z += 0.5;
- else if (z >= 0.5)
- z -= 0.5;
- else
- return 0;
- }
-
- sigma = 2 * rs->np * rs->nn;
- sigma *= 2 * rs->np * rs->nn - rs->nn - rs->np;
- sigma /= pow2 (rs->np + rs->nn);
- sigma /= rs->np + rs->nn - 1.0;
- sigma = sqrt (sigma);
-
- z /= sigma;
-
- return z;
-}
-
-static void show_runs_result (const struct runs_test *, const struct run_state *, const struct dictionary *);
-
-void
-runs_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- int v;
- struct ccase *c;
- const struct dictionary *dict = dataset_dict (ds);
- const struct variable *weight = dict_get_weight (dict);
-
- struct one_sample_test *otp = UP_CAST (test, struct one_sample_test, parent);
- struct runs_test *rt = UP_CAST (otp, struct runs_test, parent);
- struct run_state *rs = XCALLOC (otp->n_vars, struct run_state);
-
- switch (rt->cp_mode)
- {
- case CP_MODE:
- {
- for (v = 0; v < otp->n_vars; ++v)
- {
- bool multimodal = false;
- struct run_state *run = &rs[v];
- double last_cc;
- struct casereader *group = NULL;
- struct casegrouper *grouper;
- struct casereader *reader = casereader_clone (input);
- const struct variable *var = otp->vars[v];
-
- reader = sort_execute_1var (reader, var);
-
- grouper = casegrouper_create_vars (reader, &var, 1);
- last_cc = SYSMIS;
- while (casegrouper_get_next_group (grouper, &group))
- {
- double x = SYSMIS;
- double cc = 0.0;
- struct ccase *c;
- for (; (c = casereader_read (group)); case_unref (c))
- {
- const double w = weight ? case_num (c, weight) : 1.0;
- const union value *val = case_data (c, var);
- if (var_is_value_missing (var, val) & exclude)
- continue;
- x = val->f;
- cc += w;
- }
-
- if (cc > last_cc)
- {
- run->cutpoint = x;
- }
- else if (cc == last_cc)
- {
- multimodal = true;
- if (x > run->cutpoint)
- run->cutpoint = x;
- }
- last_cc = cc;
- casereader_destroy (group);
- }
- casegrouper_destroy (grouper);
- if (multimodal)
- msg (MW, _("Multiple modes exist for variable `%s'. "
- "Using %.*g as the threshold value."),
- var_get_name (var), DBL_DIG + 1, run->cutpoint);
- }
- }
- break;
- case CP_MEDIAN:
- {
- for (v = 0; v < otp->n_vars; ++v)
- {
- double cc = 0.0;
- struct ccase *c;
- struct run_state *run = &rs[v];
- struct casereader *reader = casereader_clone (input);
- const struct variable *var = otp->vars[v];
- struct casewriter *writer;
- struct percentile *median;
- struct order_stats *os;
- struct subcase sc;
- subcase_init_var (&sc, var, SC_ASCEND);
- writer = sort_create_writer (&sc, casereader_get_proto (reader));
-
- for (; (c = casereader_read (reader));)
- {
- const union value *val = case_data (c, var);
- const double w = weight ? case_num (c, weight) : 1.0;
- if (var_is_value_missing (var, val) & exclude)
- {
- case_unref (c);
- continue;
- }
-
- cc += w;
- casewriter_write (writer, c);
- }
- subcase_uninit (&sc);
- casereader_destroy (reader);
- reader = casewriter_make_reader (writer);
-
- median = percentile_create (0.5, cc);
- os = &median->parent;
-
- order_stats_accumulate (&os, 1,
- reader,
- weight,
- var,
- exclude);
-
- run->cutpoint = percentile_calculate (median, PC_HAVERAGE);
- statistic_destroy (&median->parent.parent);
- }
- }
- break;
- case CP_MEAN:
- {
- struct casereader *reader = casereader_clone (input);
- for (; (c = casereader_read (reader)); case_unref (c))
- {
- const double w = weight ? case_num (c, weight) : 1.0;
- for (v = 0; v < otp->n_vars; ++v)
- {
- const struct variable *var = otp->vars[v];
- const union value *val = case_data (c, var);
- const double x = val->f;
- struct run_state *run = &rs[v];
-
- if (var_is_value_missing (var, val) & exclude)
- continue;
-
- run->cutpoint += x * w;
- run->n += w;
- }
- }
- casereader_destroy (reader);
- for (v = 0; v < otp->n_vars; ++v)
- {
- struct run_state *run = &rs[v];
- run->cutpoint /= run->n;
- }
- }
- break;
- case CP_CUSTOM:
- {
- for (v = 0; v < otp->n_vars; ++v)
- {
- struct run_state *run = &rs[v];
- run->cutpoint = rt->cutpoint;
- }
- }
- break;
- }
-
- for (; (c = casereader_read (input)); case_unref (c))
- {
- const double w = weight ? case_num (c, weight) : 1.0;
-
- for (v = 0; v < otp->n_vars; ++v)
- {
- struct run_state *run = &rs[v];
- const struct variable *var = otp->vars[v];
- const union value *val = case_data (c, var);
- double x = val->f;
- double d = x - run->cutpoint;
- short sign = 0;
-
- if (var_is_value_missing (var, val) & exclude)
- continue;
-
- if (d >= 0)
- {
- sign = +1;
- run->np += w;
- }
- else
- {
- sign = -1;
- run->nn += w;
- }
-
- if (sign != run->last_sign)
- run->runs++;
-
- run->last_sign = sign;
- }
- }
- casereader_destroy (input);
-
- for (v = 0; v < otp->n_vars; ++v)
- {
- struct run_state *run = &rs[v];
- run->n = run->np + run->nn;
- }
-
- show_runs_result (rt, rs, dict);
-
- free (rs);
-}
-
-\f
-
-static void
-show_runs_result (const struct runs_test *rt, const struct run_state *rs, const struct dictionary *dict)
-{
- const struct one_sample_test *otp = &rt->parent;
-
- struct pivot_table *table = pivot_table_create (N_("Runs Test"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- (rt->cp_mode == CP_CUSTOM ? N_("Test Value")
- : rt->cp_mode == CP_MODE ? N_("Test Value (mode)")
- : rt->cp_mode == CP_MEAN ? N_("Test Value (mean)")
- : N_("Test Value (median)")), PIVOT_RC_OTHER,
- N_("Cases < Test Value"), PIVOT_RC_COUNT,
- N_("Cases ≥ Test Value"), PIVOT_RC_COUNT,
- N_("Total Cases"), PIVOT_RC_COUNT,
- N_("Number of Runs"), PIVOT_RC_INTEGER,
- N_("Z"), PIVOT_RC_OTHER,
- N_("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Variable"));
-
- for (size_t i = 0 ; i < otp->n_vars; ++i)
- {
- const struct run_state *run = &rs[i];
-
- int col = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (otp->vars[i]));
-
- double z = runs_statistic (run);
-
- double rows[] = {
- run->cutpoint,
- run->nn,
- run->np,
- run->n,
- run->runs,
- z,
- 2.0 * (1.0 - gsl_cdf_ugaussian_P (fabs (z))),
- };
-
- for (int row = 0; row < sizeof rows / sizeof *rows; row++)
- pivot_table_put2 (table, row, col, pivot_value_new_number (rows[row]));
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2006, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !runs_h
-#define runs_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-#include "language/stats/npar.h"
-
-enum cp_mode
- {
- CP_MEAN,
- CP_MEDIAN,
- CP_MODE,
- CP_CUSTOM
- };
-
-
-struct runs_test
-{
- struct one_sample_test parent;
-
- double cutpoint;
-
- enum cp_mode cp_mode;
-};
-
-
-void runs_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool,
- double);
-
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/sign.h"
-
-#include <gsl/gsl_cdf.h>
-#include <gsl/gsl_randist.h>
-
-#include "data/casereader.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/missing-values.h"
-#include "data/variable.h"
-#include "language/stats/npar.h"
-#include "libpspp/str.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-struct sign_test_params
-{
- double pos;
- double ties;
- double neg;
-
- double one_tailed_sig;
- double point_prob;
-};
-
-static int
-add_pair_leaf (struct pivot_dimension *dimension, variable_pair *pair)
-{
- char *label = xasprintf ("%s - %s", var_to_string ((*pair)[0]),
- var_to_string ((*pair)[1]));
- return pivot_category_create_leaf (
- dimension->root,
- pivot_value_new_user_text_nocopy (label));
-}
-
-static void
-output_frequency_table (const struct two_sample_test *t2s,
- const struct sign_test_params *param,
- const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (N_("Frequencies"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("N"),
- N_("N"), PIVOT_RC_COUNT);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Differences"),
- N_("Negative Differences"),
- N_("Positive Differences"),
- N_("Ties"), N_("Total"));
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Pairs"));
-
- for (size_t i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
-
- int pair_idx = add_pair_leaf (pairs, vp);
-
- const struct sign_test_params *p = ¶m[i];
- double values[] = { p->neg, p->pos, p->ties, p->ties + p->neg + p->pos };
- for (size_t j = 0; j < sizeof values / sizeof *values; j++)
- pivot_table_put3 (table, 0, j, pair_idx,
- pivot_value_new_number (values[j]));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-output_statistics_table (const struct two_sample_test *t2s,
- const struct sign_test_params *param)
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Point Probability"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Pairs"));
-
- for (size_t i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
- int pair_idx = add_pair_leaf (pairs, vp);
-
- const struct sign_test_params *p = ¶m[i];
- double values[] = { p->one_tailed_sig * 2,
- p->one_tailed_sig,
- p->point_prob };
- for (size_t j = 0; j < sizeof values / sizeof *values; j++)
- pivot_table_put2 (table, j, pair_idx,
- pivot_value_new_number (values[j]));
- }
-
- pivot_table_submit (table);
-}
-
-void
-sign_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact UNUSED,
- double timer UNUSED)
-{
- int i;
- bool warn = true;
- const struct dictionary *dict = dataset_dict (ds);
- const struct two_sample_test *t2s = UP_CAST (test, const struct two_sample_test, parent);
- struct ccase *c;
-
- struct sign_test_params *stp = XCALLOC (t2s->n_pairs, struct sign_test_params);
-
- struct casereader *r = input;
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- const double weight = dict_get_case_weight (dict, c, &warn);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
- const union value *value0 = case_data (c, (*vp)[0]);
- const union value *value1 = case_data (c, (*vp)[1]);
- const double diff = value0->f - value1->f;
-
- if (var_is_value_missing ((*vp)[0], value0) & exclude)
- continue;
-
- if (var_is_value_missing ((*vp)[1], value1) & exclude)
- continue;
-
- if (diff > 0)
- stp[i].pos += weight;
- else if (diff < 0)
- stp[i].neg += weight;
- else
- stp[i].ties += weight;
- }
- }
-
- casereader_destroy (r);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- int r = MIN (stp[i].pos, stp[i].neg);
- stp[i].one_tailed_sig = gsl_cdf_binomial_P (r,
- 0.5,
- stp[i].pos + stp[i].neg);
-
- stp[i].point_prob = gsl_ran_binomial_pdf (r, 0.5,
- stp[i].pos + stp[i].neg);
- }
-
- output_frequency_table (t2s, stp, dict);
-
- output_statistics_table (t2s, stp);
-
- free (stp);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2009, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !sign_h
-#define sign_h 1
-
-
-#include <stdbool.h>
-#include "data/missing-values.h"
-
-struct casereader;
-struct dataset;
-struct npar_test;
-
-void sign_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer);
-
-#endif
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <limits.h>
-#include <stdlib.h>
-#include <sys/types.h>
-
-#include "data/dataset.h"
-#include "data/settings.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/stats/sort-criteria.h"
-#include "libpspp/message.h"
-#include "math/sort.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-
-/* Performs the SORT CASES procedures. */
-int
-cmd_sort_cases (struct lexer *lexer, struct dataset *ds)
-{
- struct subcase ordering = SUBCASE_EMPTY_INITIALIZER;
- bool ok = false;
-
- lex_match (lexer, T_BY);
-
- proc_cancel_temporary_transformations (ds);
- if (!parse_sort_criteria (lexer, dataset_dict (ds), &ordering, NULL, NULL))
- return CMD_CASCADING_FAILURE;
-
- if (settings_get_testing_mode () && lex_match (lexer, T_SLASH))
- {
- if (!lex_force_match_id (lexer, "BUFFERS"))
- goto done;
- lex_match (lexer, T_EQUALS);
- if (!lex_force_int_range (lexer, "BUFFERS", 2, INT_MAX))
- goto done;
- min_buffers = max_buffers = lex_integer (lexer);
- lex_get (lexer);
- }
-
- proc_discard_output (ds);
- struct casereader *output = sort_execute (proc_open_filtering (ds, false),
- &ordering);
- ok = proc_commit (ds);
- ok = dataset_set_source (ds, output) && ok;
-
- done:
- min_buffers = 64;
- max_buffers = INT_MAX;
-
- subcase_uninit (&ordering);
- return ok ? CMD_SUCCESS : CMD_CASCADING_FAILURE;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/stats/sort-criteria.h"
-
-#include <stdlib.h>
-
-#include "data/dictionary.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Parses a list of sort fields and appends them to ORDERING,
- which the caller must already have initialized.
- Returns true if successful, false on error.
- If SAW_DIRECTION is nonnull, sets *SAW_DIRECTION to true if at
- least one parenthesized sort direction was specified, false
- otherwise. */
-bool
-parse_sort_criteria (struct lexer *lexer, const struct dictionary *dict,
- struct subcase *ordering,
- const struct variable ***vars, bool *saw_direction)
-{
- const struct variable **local_vars = NULL;
- size_t n_vars = 0;
-
- if (vars == NULL)
- vars = &local_vars;
- *vars = NULL;
-
- if (saw_direction != NULL)
- *saw_direction = false;
-
- int start_ofs = lex_ofs (lexer);
- do
- {
- size_t prev_n_vars = n_vars;
-
- /* Variables. */
- if (!parse_variables_const (lexer, dict, vars, &n_vars,
- PV_APPEND | PV_DUPLICATE | PV_NO_SCRATCH))
- goto error;
-
- /* Sort direction. */
- enum subcase_direction direction;
- if (lex_match (lexer, T_LPAREN))
- {
- if (lex_match_id (lexer, "D") || lex_match_id (lexer, "DOWN"))
- direction = SC_DESCEND;
- else if (lex_match_id (lexer, "A") || lex_match_id (lexer, "UP"))
- direction = SC_ASCEND;
- else
- {
- lex_error_expecting (lexer, "A", "D");
- goto error;
- }
- if (!lex_force_match (lexer, T_RPAREN))
- goto error;
- if (saw_direction != NULL)
- *saw_direction = true;
- }
- else
- direction = SC_ASCEND;
-
- for (size_t i = prev_n_vars; i < n_vars; i++)
- {
- const struct variable *var = (*vars)[i];
- if (!subcase_add_var (ordering, var, direction))
- lex_ofs_msg (lexer, SW, start_ofs, lex_ofs (lexer) - 1,
- _("Variable %s specified twice in sort criteria."),
- var_get_name (var));
- }
- }
- while (lex_token (lexer) == T_ID
- && dict_lookup_var (dict, lex_tokcstr (lexer)) != NULL);
-
- free (local_vars);
- return true;
-
-error:
- subcase_uninit (ordering);
- subcase_init_empty (ordering);
- free (local_vars);
- if (vars)
- *vars = NULL;
- return false;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef SORT_CRITERIA_H
-#define SORT_CRITERIA_H
-
-#include <stdbool.h>
-#include <stddef.h>
-
-struct dictionary;
-struct lexer;
-struct variable;
-struct subcase;
-
-bool parse_sort_criteria (struct lexer *, const struct dictionary *,
- struct subcase *, const struct variable ***vars,
- bool *saw_direction);
-
-
-#endif /* sort-criteria.h */
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011, 2020 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "t-test.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-#include "libpspp/misc.h"
-
-#include "libpspp/str.h"
-#include "data/casereader.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-#include "math/moments.h"
-#include "math/levene.h"
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct indep_samples
-{
- const struct variable *gvar;
- bool cut;
- const union value *gval0;
- const union value *gval1;
-};
-
-struct pair_stats
-{
- struct moments *mom[2];
- double lev ;
- struct levene *nl;
-};
-
-
-static void indep_summary (const struct tt *tt, struct indep_samples *is, const struct pair_stats *ps);
-static void indep_test (const struct tt *tt, const struct pair_stats *ps);
-
-static int
-which_group (const union value *v, const struct indep_samples *is)
-{
- int width = var_get_width (is->gvar);
- int cmp = value_compare_3way (v, is->gval0, width);
- if (is->cut)
- return (cmp < 0);
-
- if (cmp == 0)
- return 0;
-
- if (0 == value_compare_3way (v, is->gval1, width))
- return 1;
-
- return -1;
-}
-
-void
-indep_run (struct tt *tt, const struct variable *gvar,
- bool cut,
- const union value *gval0, const union value *gval1,
- struct casereader *reader)
-{
- struct indep_samples is;
- struct ccase *c;
- struct casereader *r;
-
- struct pair_stats *ps = XCALLOC (tt->n_vars, struct pair_stats);
-
- int v;
-
- for (v = 0; v < tt->n_vars; ++v)
- {
- ps[v].mom[0] = moments_create (MOMENT_VARIANCE);
- ps[v].mom[1] = moments_create (MOMENT_VARIANCE);
- ps[v].nl = levene_create (var_get_width (gvar), cut ? gval0: NULL);
- }
-
- is.gvar = gvar;
- is.gval0 = gval0;
- is.gval1 = gval1;
- is.cut = cut;
-
- r = casereader_clone (reader);
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
-
- const union value *gv = case_data (c, gvar);
-
- int grp = which_group (gv, &is);
- if (grp < 0)
- continue;
-
- for (v = 0; v < tt->n_vars; ++v)
- {
- const union value *val = case_data (c, tt->vars[v]);
- if (var_is_value_missing (tt->vars[v], val) & tt->exclude)
- continue;
-
- moments_pass_one (ps[v].mom[grp], val->f, w);
- levene_pass_one (ps[v].nl, val->f, w, gv);
- }
- }
- casereader_destroy (r);
-
- r = casereader_clone (reader);
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
-
- const union value *gv = case_data (c, gvar);
-
- int grp = which_group (gv, &is);
- if (grp < 0)
- continue;
-
- for (v = 0; v < tt->n_vars; ++v)
- {
- const union value *val = case_data (c, tt->vars[v]);
- if (var_is_value_missing (tt->vars[v], val) & tt->exclude)
- continue;
-
- moments_pass_two (ps[v].mom[grp], val->f, w);
- levene_pass_two (ps[v].nl, val->f, w, gv);
- }
- }
- casereader_destroy (r);
-
- r = reader;
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
-
- const union value *gv = case_data (c, gvar);
-
- int grp = which_group (gv, &is);
- if (grp < 0)
- continue;
-
- for (v = 0; v < tt->n_vars; ++v)
- {
- const union value *val = case_data (c, tt->vars[v]);
- if (var_is_value_missing (tt->vars[v], val) & tt->exclude)
- continue;
-
- levene_pass_three (ps[v].nl, val->f, w, gv);
- }
- }
- casereader_destroy (r);
-
-
- for (v = 0; v < tt->n_vars; ++v)
- ps[v].lev = levene_calculate (ps[v].nl);
-
- indep_summary (tt, &is, ps);
- indep_test (tt, ps);
-
-
- for (v = 0; v < tt->n_vars; ++v)
- {
- moments_destroy (ps[v].mom[0]);
- moments_destroy (ps[v].mom[1]);
- levene_destroy (ps[v].nl);
- }
- free (ps);
-}
-
-
-static void
-indep_summary (const struct tt *tt, struct indep_samples *is, const struct pair_stats *ps)
-{
- struct pivot_table *table = pivot_table_create (N_("Group Statistics"));
- pivot_table_set_weight_var (table, tt->wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("S.E. Mean"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *group = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Group"));
- group->root->show_label = true;
- if (is->cut)
- {
- struct string vallab0 = DS_EMPTY_INITIALIZER;
- ds_put_cstr (&vallab0, "≥");
- var_append_value_name (is->gvar, is->gval0, &vallab0);
- pivot_category_create_leaf (group->root,
- pivot_value_new_user_text_nocopy (
- ds_steal_cstr (&vallab0)));
-
- struct string vallab1 = DS_EMPTY_INITIALIZER;
- ds_put_cstr (&vallab1, "<");
- var_append_value_name (is->gvar, is->gval0, &vallab1);
- pivot_category_create_leaf (group->root,
- pivot_value_new_user_text_nocopy (
- ds_steal_cstr (&vallab1)));
- }
- else
- {
- pivot_category_create_leaf (
- group->root, pivot_value_new_var_value (is->gvar, is->gval0));
- pivot_category_create_leaf (
- group->root, pivot_value_new_var_value (is->gvar, is->gval1));
- }
-
- struct pivot_dimension *dep_vars = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- for (size_t v = 0; v < tt->n_vars; ++v)
- {
- const struct variable *var = tt->vars[v];
-
- int dep_var_idx = pivot_category_create_leaf (
- dep_vars->root, pivot_value_new_variable (var));
-
- for (int i = 0 ; i < 2; ++i)
- {
- double cc, mean, sigma;
- moments_calculate (ps[v].mom[i], &cc, &mean, &sigma, NULL, NULL);
-
- double entries[] = { cc, mean, sqrt (sigma), sqrt (sigma / cc) };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put3 (table, j, i, dep_var_idx,
- pivot_value_new_number (entries[j]));
- }
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-indep_test (const struct tt *tt, const struct pair_stats *ps)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Independent Samples Test"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- pivot_category_create_group (
- statistics->root, N_("Levene's Test for Equality of Variances"),
- N_("F"), PIVOT_RC_OTHER,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
- struct pivot_category *group = pivot_category_create_group (
- statistics->root, N_("T-Test for Equality of Means"),
- N_("t"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_OTHER,
- N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Mean Difference"), PIVOT_RC_OTHER,
- N_("Std. Error Difference"), PIVOT_RC_OTHER);
- pivot_category_create_group (
- /* xgettext:no-c-format */
- group, N_("95% Confidence Interval of the Difference"),
- N_("Lower"), PIVOT_RC_OTHER,
- N_("Upper"), PIVOT_RC_OTHER);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Assumptions"),
- N_("Equal variances assumed"),
- N_("Equal variances not assumed"));
-
- struct pivot_dimension *dep_vars = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- for (size_t v = 0; v < tt->n_vars; ++v)
- {
- int dep_var_idx = pivot_category_create_leaf (
- dep_vars->root, pivot_value_new_variable (tt->vars[v]));
-
- double cc0, mean0, sigma0;
- double cc1, mean1, sigma1;
- moments_calculate (ps[v].mom[0], &cc0, &mean0, &sigma0, NULL, NULL);
- moments_calculate (ps[v].mom[1], &cc1, &mean1, &sigma1, NULL, NULL);
-
- double mean_diff = mean0 - mean1;
-
-
- /* Equal variances assumed. */
- double e_df = cc0 + cc1 - 2.0;
- double e_pooled_variance = ((cc0 - 1)* sigma0 + (cc1 - 1) * sigma1) / e_df;
- double e_tval = ((mean0 - mean1) / sqrt (e_pooled_variance)
- / sqrt ((cc0 + cc1) / (cc0 * cc1)));
- double e_p = gsl_cdf_tdist_P (e_tval, e_df);
- double e_q = gsl_cdf_tdist_Q (e_tval, e_df);
- double e_sig = 2.0 * (e_tval > 0 ? e_q : e_p);
- double e_std_err_diff = sqrt (e_pooled_variance * (1.0/cc0 + 1.0/cc1));
- double e_tval_qinv = gsl_cdf_tdist_Qinv ((1 - tt->confidence) / 2.0, e_df);
-
- /* Equal variances not assumed */
- const double s0 = sigma0 / cc0;
- const double s1 = sigma1 / cc1;
- double d_df = (pow2 (s0 + s1) / (pow2 (s0) / (cc0 - 1)
- + pow2 (s1) / (cc1 - 1)));
- double d_tval = mean_diff / sqrt (sigma0 / cc0 + sigma1 / cc1);
- double d_p = gsl_cdf_tdist_P (d_tval, d_df);
- double d_q = gsl_cdf_tdist_Q (d_tval, d_df);
- double d_sig = 2.0 * (d_tval > 0 ? d_q : d_p);
- double d_std_err_diff = sqrt ((sigma0 / cc0) + (sigma1 / cc1));
- double d_tval_qinv = gsl_cdf_tdist_Qinv ((1 - tt->confidence) / 2.0, d_df);
-
- struct entry
- {
- int assumption_idx;
- int stat_idx;
- double x;
- }
- entries[] =
- {
- { 0, 0, ps[v].lev },
- { 0, 1, gsl_cdf_fdist_Q (ps[v].lev, 1, cc0 + cc1 - 2) },
-
- { 0, 2, e_tval },
- { 0, 3, e_df },
- { 0, 4, e_sig },
- { 0, 5, mean_diff },
- { 0, 6, e_std_err_diff },
- { 0, 7, mean_diff - e_tval_qinv * e_std_err_diff },
- { 0, 8, mean_diff + e_tval_qinv * e_std_err_diff },
-
- { 1, 2, d_tval },
- { 1, 3, d_df },
- { 1, 4, d_sig },
- { 1, 5, mean_diff },
- { 1, 6, d_std_err_diff },
- { 1, 7, mean_diff - d_tval_qinv * d_std_err_diff },
- { 1, 8, mean_diff + d_tval_qinv * d_std_err_diff },
- };
-
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- {
- const struct entry *e = &entries[i];
- pivot_table_put3 (table, e->stat_idx, e->assumption_idx,
- dep_var_idx, pivot_value_new_number (e->x));
- }
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include <config.h>
-
-
-#include "t-test.h"
-
-#include <math.h>
-#include <gsl/gsl_cdf.h>
-
-#include "data/variable.h"
-#include "data/format.h"
-#include "data/casereader.h"
-#include "data/dictionary.h"
-#include "libpspp/hash-functions.h"
-#include "libpspp/hmapx.h"
-#include "math/moments.h"
-
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct per_var_stats
-{
- const struct variable *var;
-
- /* N, Mean, Variance */
- struct moments *mom;
-
- /* Sum of the differences */
- double sum_diff;
-};
-
-
-struct one_samp
-{
- struct per_var_stats *stats;
- size_t n_stats;
- double testval;
-};
-
-
-static void
-one_sample_test (const struct tt *tt, const struct one_samp *os)
-{
- struct pivot_table *table = pivot_table_create (N_("One-Sample Test"));
- pivot_table_set_weight_var (table, tt->wv);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- struct pivot_category *group = pivot_category_create_group__ (
- statistics->root, pivot_value_new_user_text_nocopy (
- xasprintf (_("Test Value = %.*g"), DBL_DIG + 1, os->testval)));
- pivot_category_create_leaves (
- group,
- N_("t"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_COUNT,
- N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Mean Difference"), PIVOT_RC_OTHER);
- struct pivot_category *subgroup = pivot_category_create_group__ (
- group, pivot_value_new_user_text_nocopy (
- xasprintf (_("%g%% Confidence Interval of the Difference"),
- tt->confidence * 100.0)));
- pivot_category_create_leaves (subgroup,
- N_("Lower"), PIVOT_RC_OTHER,
- N_("Upper"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *dep_vars = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Dependent Variables"));
-
- for (size_t i = 0; i < os->n_stats; i++)
- {
- const struct per_var_stats *per_var_stats = &os->stats[i];
- const struct moments *m = per_var_stats->mom;
-
- int dep_var_idx = pivot_category_create_leaf (
- dep_vars->root, pivot_value_new_variable (per_var_stats->var));
-
- double cc, mean, sigma;
- moments_calculate (m, &cc, &mean, &sigma, NULL, NULL);
- double tval = (mean - os->testval) * sqrt (cc / sigma);
- double mean_diff = per_var_stats->sum_diff / cc;
- double se_mean = sqrt (sigma / cc);
- double df = cc - 1.0;
- double p = gsl_cdf_tdist_P (tval, df);
- double q = gsl_cdf_tdist_Q (tval, df);
- double sig = 2.0 * (tval > 0 ? q : p);
- double tval_qinv = gsl_cdf_tdist_Qinv ((1.0 - tt->confidence) / 2.0, df);
- double lower = mean_diff - tval_qinv * se_mean;
- double upper = mean_diff + tval_qinv * se_mean;
-
- double entries[] = { tval, df, sig, mean_diff, lower, upper };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, dep_var_idx,
- pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
-}
-
-static void
-one_sample_summary (const struct tt *tt, const struct one_samp *os)
-{
- struct pivot_table *table = pivot_table_create (N_("One-Sample Statistics"));
- pivot_table_set_weight_var (table, tt->wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("S.E. Mean"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0; i < os->n_stats; i++)
- {
- const struct per_var_stats *per_var_stats = &os->stats[i];
- const struct moments *m = per_var_stats->mom;
-
- int var_idx = pivot_category_create_leaf (
- variables->root, pivot_value_new_variable (per_var_stats->var));
-
- double cc, mean, sigma;
- moments_calculate (m, &cc, &mean, &sigma, NULL, NULL);
-
- double entries[] = { cc, mean, sqrt (sigma), sqrt (sigma / cc) };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, var_idx,
- pivot_value_new_number (entries[j]));
- }
-
- pivot_table_submit (table);
-}
-
-void
-one_sample_run (const struct tt *tt, double testval, struct casereader *reader)
-{
- struct one_samp os;
- os.testval = testval;
- os.stats = xcalloc (tt->n_vars, sizeof *os.stats);
- os.n_stats = tt->n_vars;
- for (size_t i = 0; i < tt->n_vars; ++i)
- {
- struct per_var_stats *per_var_stats = &os.stats[i];
- per_var_stats->var = tt->vars[i];
- per_var_stats->mom = moments_create (MOMENT_VARIANCE);
- }
-
- struct casereader *r = casereader_clone (reader);
- struct ccase *c;
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
- for (size_t i = 0; i < os.n_stats; i++)
- {
- const struct per_var_stats *per_var_stats = &os.stats[i];
- const struct variable *var = per_var_stats->var;
- const union value *val = case_data (c, var);
- if (var_is_value_missing (var, val) & tt->exclude)
- continue;
-
- moments_pass_one (per_var_stats->mom, val->f, w);
- }
- }
- casereader_destroy (r);
-
- r = reader;
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
- for (size_t i = 0; i < os.n_stats; i++)
- {
- struct per_var_stats *per_var_stats = &os.stats[i];
- const struct variable *var = per_var_stats->var;
- const union value *val = case_data (c, var);
- if (var_is_value_missing (var, val) & tt->exclude)
- continue;
-
- moments_pass_two (per_var_stats->mom, val->f, w);
- per_var_stats->sum_diff += w * (val->f - os.testval);
- }
- }
- casereader_destroy (r);
-
- one_sample_summary (tt, &os);
- one_sample_test (tt, &os);
-
- for (size_t i = 0; i < os.n_stats; i++)
- {
- const struct per_var_stats *per_var_stats = &os.stats[i];
- moments_destroy (per_var_stats->mom);
- }
- free (os.stats);
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <math.h>
-#include <gsl/gsl_cdf.h>
-
-#include "t-test.h"
-
-#include "math/moments.h"
-#include "math/correlation.h"
-#include "data/casereader.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/variable.h"
-
-#include "output/pivot-table.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-
-struct pair_stats
-{
- double sum_of_prod;
- struct moments *mom0;
- const struct variable *var0;
-
- struct moments *mom1;
- const struct variable *var1;
-
- struct moments *mom_diff;
-};
-
-struct paired_samp
-{
- struct pair_stats *ps;
- size_t n_ps;
-};
-
-static void paired_summary (const struct tt *tt, struct paired_samp *os);
-static void paired_correlations (const struct tt *tt, struct paired_samp *os);
-static void paired_test (const struct tt *tt, const struct paired_samp *os);
-
-void
-paired_run (const struct tt *tt, size_t n_pairs, vp *pairs, struct casereader *reader)
-{
- struct ccase *c;
- struct paired_samp ps;
- struct casereader *r;
-
- ps.ps = xcalloc (n_pairs, sizeof *ps.ps);
- ps.n_ps = n_pairs;
- for (size_t i = 0; i < n_pairs; ++i)
- {
- vp *pair = &pairs[i];
- struct pair_stats *pp = &ps.ps[i];
- pp->var0 = (*pair)[0];
- pp->var1 = (*pair)[1];
- pp->mom0 = moments_create (MOMENT_VARIANCE);
- pp->mom1 = moments_create (MOMENT_VARIANCE);
- pp->mom_diff = moments_create (MOMENT_VARIANCE);
- }
-
- r = casereader_clone (reader);
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
-
- for (int i = 0; i < ps.n_ps; i++)
- {
- struct pair_stats *pp = &ps.ps[i];
- const union value *val0 = case_data (c, pp->var0);
- const union value *val1 = case_data (c, pp->var1);
- if (var_is_value_missing (pp->var0, val0) & tt->exclude)
- continue;
-
- if (var_is_value_missing (pp->var1, val1) & tt->exclude)
- continue;
-
- moments_pass_one (pp->mom0, val0->f, w);
- moments_pass_one (pp->mom1, val1->f, w);
- moments_pass_one (pp->mom_diff, val0->f - val1->f, w);
- }
- }
- casereader_destroy (r);
-
- r = reader;
- for (; (c = casereader_read (r)); case_unref (c))
- {
- double w = dict_get_case_weight (tt->dict, c, NULL);
-
- for (int i = 0; i < ps.n_ps; i++)
- {
- struct pair_stats *pp = &ps.ps[i];
- const union value *val0 = case_data (c, pp->var0);
- const union value *val1 = case_data (c, pp->var1);
- if (var_is_value_missing (pp->var0, val0) & tt->exclude)
- continue;
-
- if (var_is_value_missing (pp->var1, val1) & tt->exclude)
- continue;
-
- moments_pass_two (pp->mom0, val0->f, w);
- moments_pass_two (pp->mom1, val1->f, w);
- moments_pass_two (pp->mom_diff, val0->f - val1->f, w);
- pp->sum_of_prod += val0->f * val1->f * w;
- }
- }
- casereader_destroy (r);
-
- paired_summary (tt, &ps);
- paired_correlations (tt, &ps);
- paired_test (tt, &ps);
-
- /* Clean up */
-
- for (int i = 0; i < ps.n_ps; i++)
- {
- struct pair_stats *pp = &ps.ps[i];
- moments_destroy (pp->mom0);
- moments_destroy (pp->mom1);
- moments_destroy (pp->mom_diff);
- }
- free (ps.ps);
-}
-
-static void
-paired_summary (const struct tt *tt, struct paired_samp *os)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Paired Sample Statistics"));
- pivot_table_set_weight_var (table, tt->wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("S.E. Mean"), PIVOT_RC_OTHER);
-
- struct pivot_dimension *variables = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Variables"));
-
- for (size_t i = 0; i < os->n_ps; i++)
- {
- struct pair_stats *pp = &os->ps[i];
- struct pivot_category *pair = pivot_category_create_group__ (
- variables->root, pivot_value_new_text_format (N_("Pair %zu"), i + 1));
-
- for (int j = 0; j < 2; j++)
- {
- const struct variable *var = j ? pp->var1 : pp->var0;
- const struct moments *mom = j ? pp->mom1 : pp->mom0;
- double cc, mean, sigma;
- moments_calculate (mom, &cc, &mean, &sigma, NULL, NULL);
-
- int var_idx = pivot_category_create_leaf (
- pair, pivot_value_new_variable (var));
-
- double entries[] = { cc, mean, sqrt (sigma), sqrt (sigma / cc) };
- for (size_t j = 0; j < sizeof entries / sizeof *entries; j++)
- pivot_table_put2 (table, j, var_idx,
- pivot_value_new_number (entries[j]));
- }
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-paired_correlations (const struct tt *tt, struct paired_samp *os)
-{
- struct pivot_table *table = pivot_table_create (
- N_("Paired Samples Correlations"));
- pivot_table_set_weight_var (table, tt->wv);
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Correlation"), PIVOT_RC_CORRELATION,
- N_("Sig."), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Pairs"));
-
- for (size_t i = 0; i < os->n_ps; i++)
- {
- struct pair_stats *pp = &os->ps[i];
- struct pivot_category *group = pivot_category_create_group__ (
- pairs->root, pivot_value_new_text_format (N_("Pair %zu"), i + 1));
-
- int row = pivot_category_create_leaf (
- group, pivot_value_new_text_format (N_("%s & %s"),
- var_to_string (pp->var0),
- var_to_string (pp->var1)));
-
- double cc0, mean0, sigma0;
- double cc1, mean1, sigma1;
- moments_calculate (pp->mom0, &cc0, &mean0, &sigma0, NULL, NULL);
- moments_calculate (pp->mom1, &cc1, &mean1, &sigma1, NULL, NULL);
- /* If this fails, then we're not dealing with missing values properly */
- assert (cc0 == cc1);
-
- double corr = ((pp->sum_of_prod / cc0 - mean0 * mean1)
- / sqrt (sigma0 * sigma1) * cc0 / (cc0 - 1));
- double sig = 2.0 * significance_of_correlation (corr, cc0);
- double entries[] = { cc0, corr, sig };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-paired_test (const struct tt *tt, const struct paired_samp *os)
-{
- struct pivot_table *table = pivot_table_create (N_("Paired Samples Test"));
- pivot_table_set_weight_var (table, tt->wv);
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Statistics"));
- struct pivot_category *group = pivot_category_create_group (
- statistics->root, N_("Paired Differences"),
- N_("Mean"), PIVOT_RC_OTHER,
- N_("Std. Deviation"), PIVOT_RC_OTHER,
- N_("S.E. Mean"), PIVOT_RC_OTHER);
- struct pivot_category *interval = pivot_category_create_group__ (
- group, pivot_value_new_text_format (
- N_("%g%% Confidence Interval of the Difference"),
- tt->confidence * 100.0));
- pivot_category_create_leaves (interval,
- N_("Lower"), PIVOT_RC_OTHER,
- N_("Upper"), PIVOT_RC_OTHER);
- pivot_category_create_leaves (statistics->root,
- N_("t"), PIVOT_RC_OTHER,
- N_("df"), PIVOT_RC_COUNT,
- N_("Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Pairs"));
-
- for (size_t i = 0; i < os->n_ps; i++)
- {
- struct pair_stats *pp = &os->ps[i];
- struct pivot_category *group = pivot_category_create_group__ (
- pairs->root, pivot_value_new_text_format (N_("Pair %zu"), i + 1));
-
- int row = pivot_category_create_leaf (
- group, pivot_value_new_text_format (N_("%s - %s"),
- var_to_string (pp->var0),
- var_to_string (pp->var1)));
-
- double cc, mean, sigma;
- moments_calculate (pp->mom_diff, &cc, &mean, &sigma, NULL, NULL);
-
- double df = cc - 1.0;
-
- double t = mean * sqrt (cc / sigma);
- double se_mean = sqrt (sigma / cc);
-
- double p = gsl_cdf_tdist_P (t, df);
- double q = gsl_cdf_tdist_Q (t, df);
- double sig = 2.0 * (t > 0 ? q : p);
-
- double t_qinv = gsl_cdf_tdist_Qinv ((1.0 - tt->confidence) / 2.0, df);
-
- double entries[] = {
- mean,
- sqrt (sigma),
- se_mean,
- mean - t_qinv * se_mean,
- mean + t_qinv * se_mean,
- t,
- df,
- sig,
- };
- for (size_t i = 0; i < sizeof entries / sizeof *entries; i++)
- pivot_table_put2 (table, i, row, pivot_value_new_number (entries[i]));
-
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011, 2015 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "libpspp/message.h"
-
-#include "data/casegrouper.h"
-#include "data/casereader.h"
-#include "data/dictionary.h"
-#include "data/dataset.h"
-#include "data/missing-values.h"
-
-#include "language/lexer/lexer.h"
-#include "language/command.h"
-#include "language/lexer/variable-parser.h"
-#include "language/lexer/value-parser.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-#include "t-test.h"
-
-int
-cmd_t_test (struct lexer *lexer, struct dataset *ds)
-{
- bool ok = false;
-
- /* Variables pertaining to the paired mode */
- const struct variable **v1 = NULL;
- size_t n_v1 = 0;
- const struct variable **v2 = NULL;
- size_t n_v2 = 0;
-
- size_t n_pairs = 0;
- vp *pairs = NULL;
-
- /* One sample mode */
- double testval = SYSMIS;
-
- /* Independent samples mode */
- const struct variable *gvar;
- union value gval0;
- union value gval1;
- int gval_width = -1;
- bool cut = false;
-
- const struct dictionary *dict = dataset_dict (ds);
- struct tt tt = {
- .wv = dict_get_weight (dict),
- .dict = dict,
- .confidence = 0.95,
- .exclude = MV_ANY,
- .missing_type = MISS_ANALYSIS,
- .n_vars = 0,
- .vars = NULL,
- };
-
- lex_match (lexer, T_EQUALS);
-
- int mode_count = 0;
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
- if (lex_match_id (lexer, "TESTVAL"))
- {
- mode_count++;
- tt.mode = MODE_SINGLE;
- lex_match (lexer, T_EQUALS);
- if (!lex_force_num (lexer))
- goto exit;
- testval = lex_number (lexer);
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "GROUPS"))
- {
- mode_count++;
- cut = false;
- tt.mode = MODE_INDEP;
- lex_match (lexer, T_EQUALS);
-
- int groups_start = lex_ofs (lexer);
- gvar = parse_variable (lexer, dict);
- if (!gvar)
- goto exit;
-
- gval_width = var_get_width (gvar);
- value_init (&gval0, gval_width);
- value_init (&gval1, gval_width);
-
- int n;
- if (lex_match (lexer, T_LPAREN))
- {
- if (!parse_value (lexer, &gval0, gvar))
- goto exit;
- if (lex_token (lexer) != T_RPAREN)
- {
- lex_match (lexer, T_COMMA);
- if (!parse_value (lexer, &gval1, gvar))
- goto exit;
- cut = false;
- n = 2;
- }
- else
- {
- cut = true;
- n = 1;
- }
-
- if (!lex_force_match (lexer, T_RPAREN))
- goto exit;
- }
- else
- {
- gval0.f = 1.0;
- gval1.f = 2.0;
- cut = false;
- n = 0;
- }
- int groups_end = lex_ofs (lexer) - 1;
-
- if (n != 2 && var_is_alpha (gvar))
- {
- lex_ofs_error (lexer, groups_start, groups_end,
- _("When applying %s to a string variable, two "
- "values must be specified."), "GROUPS");
- goto exit;
- }
- }
- else if (lex_match_id (lexer, "PAIRS"))
- {
- bool with = false;
- bool paired = false;
-
- if (tt.n_vars > 0)
- {
- lex_next_error (lexer, -1, -1,
- _("%s subcommand may not be used with %s."),
- "VARIABLES", "PAIRS");
- goto exit;
- }
-
- mode_count++;
- tt.mode = MODE_PAIRED;
- lex_match (lexer, T_EQUALS);
-
- int vars_start = lex_ofs (lexer);
- if (!parse_variables_const (lexer, dict,
- &v1, &n_v1,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto exit;
-
- if (lex_match (lexer, T_WITH))
- {
- with = true;
- if (!parse_variables_const (lexer, dict,
- &v2, &n_v2,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto exit;
- int vars_end = lex_ofs (lexer) - 1;
-
- if (lex_match_phrase (lexer, "(PAIRED)"))
- {
- paired = true;
- if (n_v1 != n_v2)
- {
- lex_ofs_error (lexer, vars_start, vars_end,
- _("PAIRED was specified, but the number "
- "of variables preceding WITH (%zu) "
- "does not match the number following "
- "(%zu)."),
- n_v1, n_v2);
- goto exit;
- }
- }
- }
-
- n_pairs = (paired ? n_v1
- : with ? n_v1 * n_v2
- : (n_v1 * (n_v1 - 1)) / 2.0);
- pairs = xcalloc (n_pairs, sizeof *pairs);
-
- if (paired)
- {
- for (size_t i = 0; i < n_v1; ++i)
- {
- vp *pair = &pairs[i];
- (*pair)[0] = v1[i];
- (*pair)[1] = v2[i];
- }
- }
- else if (with)
- {
- int x = 0;
- for (size_t i = 0; i < n_v1; ++i)
- for (size_t j = 0; j < n_v2; ++j)
- {
- vp *pair = &pairs[x++];
- (*pair)[0] = v1[i];
- (*pair)[1] = v2[j];
- }
- }
- else
- {
- int x = 0;
- for (size_t i = 0; i < n_v1; ++i)
- for (size_t j = i + 1; j < n_v1; ++j)
- {
- vp *pair = &pairs[x++];
- (*pair)[0] = v1[i];
- (*pair)[1] = v1[j];
- }
- }
- }
- else if (lex_match_id (lexer, "VARIABLES"))
- {
- if (mode_count && tt.mode == MODE_PAIRED)
- {
- lex_next_error (lexer, -1, -1,
- _("%s subcommand may not be used with %s."),
- "VARIABLES", "PAIRS");
- goto exit;
- }
-
- lex_match (lexer, T_EQUALS);
-
- if (!parse_variables_const (lexer, dict,
- &tt.vars,
- &tt.n_vars,
- PV_NO_DUPLICATE | PV_NUMERIC))
- goto exit;
- }
- else if (lex_match_id (lexer, "MISSING"))
- {
- lex_match (lexer, T_EQUALS);
- while (lex_token (lexer) != T_ENDCMD && lex_token (lexer) != T_SLASH)
- {
- if (lex_match_id (lexer, "INCLUDE"))
- tt.exclude = MV_SYSTEM;
- else if (lex_match_id (lexer, "EXCLUDE"))
- tt.exclude = MV_ANY;
- else if (lex_match_id (lexer, "LISTWISE"))
- tt.missing_type = MISS_LISTWISE;
- else if (lex_match_id (lexer, "ANALYSIS"))
- tt.missing_type = MISS_ANALYSIS;
- else
- {
- lex_error_expecting (lexer, "INCLUDE", "EXCLUDE",
- "LISTWISE", "ANALYSIS");
- goto exit;
- }
- lex_match (lexer, T_COMMA);
- }
- }
- else if (lex_match_id (lexer, "CRITERIA"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_match_id (lexer, "CIN") && !lex_match_id (lexer, "CI"))
- {
- lex_error_expecting (lexer, "CIN", "CI");
- goto exit;
- }
- if (!lex_force_match (lexer, T_LPAREN))
- goto exit;
- if (!lex_force_num (lexer))
- goto exit;
- tt.confidence = lex_number (lexer);
- lex_get (lexer);
- if (!lex_force_match (lexer, T_RPAREN))
- goto exit;
- }
- else
- {
- lex_error_expecting (lexer, "TESTVAL", "GROUPS", "PAIRS",
- "VARIABLES", "MISSING", "CRITERIA");
- goto exit;
- }
- }
-
- if (mode_count != 1)
- {
- msg (SE, _("Exactly one of TESTVAL, GROUPS and PAIRS subcommands "
- "must be specified."));
- goto exit;
- }
-
- if (tt.n_vars == 0 && tt.mode != MODE_PAIRED)
- {
- lex_sbc_missing (lexer, "VARIABLES");
- goto exit;
- }
-
- struct casereader *group;
- struct casegrouper *grouper = casegrouper_create_splits (proc_open (ds), dict);
- while (casegrouper_get_next_group (grouper, &group))
- switch (tt.mode)
- {
- case MODE_SINGLE:
- if (tt.missing_type == MISS_LISTWISE)
- group = casereader_create_filter_missing (group, tt.vars, tt.n_vars,
- tt.exclude, NULL, NULL);
- one_sample_run (&tt, testval, group);
- break;
-
- case MODE_PAIRED:
- if (tt.missing_type == MISS_LISTWISE)
- {
- group = casereader_create_filter_missing (group, v1, n_v1,
- tt.exclude, NULL, NULL);
- group = casereader_create_filter_missing (group, v2, n_v2,
- tt.exclude, NULL, NULL);
- }
- paired_run (&tt, n_pairs, pairs, group);
- break;
-
- case MODE_INDEP:
- if (tt.missing_type == MISS_LISTWISE)
- {
- group = casereader_create_filter_missing (group, tt.vars, tt.n_vars,
- tt.exclude, NULL, NULL);
- group = casereader_create_filter_missing (group, &gvar, 1,
- tt.exclude, NULL, NULL);
- }
- indep_run (&tt, gvar, cut, &gval0, &gval1, group);
- break;
- }
-
- ok = casegrouper_destroy (grouper);
- ok = proc_commit (ds) && ok;
-
-exit:
- if (gval_width != -1)
- {
- value_destroy (&gval0, gval_width);
- value_destroy (&gval1, gval_width);
- }
- free (pairs);
- free (v1);
- free (v2);
- free (tt.vars);
-
- return ok ? CMD_SUCCESS : CMD_FAILURE;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#ifndef T_TEST_H
-#define T_TEST_H 1
-
-#include "data/missing-values.h"
-
-struct variable;
-typedef const struct variable *vp[2];
-
-enum missing_type
- {
- MISS_LISTWISE,
- MISS_ANALYSIS,
- };
-
-enum mode
- {
- MODE_PAIRED,
- MODE_INDEP,
- MODE_SINGLE,
- };
-
-struct tt
-{
- size_t n_vars;
- const struct variable **vars;
- enum mode mode;
- enum missing_type missing_type;
- enum mv_class exclude;
- double confidence;
- const struct variable *wv;
- const struct dictionary *dict;
-};
-
-struct casereader;
-union value;
-
-void one_sample_run (const struct tt *tt, double testval, struct casereader *reader);
-void paired_run (const struct tt *tt, size_t n_pairs, vp *pairs, struct casereader *reader);
-void indep_run (struct tt *tt, const struct variable *gvar,
- bool cut,
- const union value *gval0, const union value *gval1,
- struct casereader *reader);
-
-
-#endif
+++ /dev/null
-/* Pspp - a program for statistical analysis.
- Copyright (C) 2008, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-
-
-#include <config.h>
-
-#include "language/stats/wilcoxon.h"
-
-#include <gsl/gsl_cdf.h>
-#include <math.h>
-
-#include "data/casereader.h"
-#include "data/casewriter.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/subcase.h"
-#include "data/variable.h"
-#include "libpspp/assertion.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "math/sort.h"
-#include "math/wilcoxon-sig.h"
-#include "output/pivot-table.h"
-
-#include "gl/minmax.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define N_(msgid) msgid
-#define _(msgid) gettext (msgid)
-
-static double
-append_difference (const struct ccase *c, casenumber n UNUSED, void *aux)
-{
- const variable_pair *vp = aux;
-
- return case_num (c, (*vp)[0]) - case_num (c, (*vp)[1]);
-}
-
-static void show_ranks_box (const struct wilcoxon_state *,
- const struct two_sample_test *,
- const struct dictionary *);
-
-static void show_tests_box (const struct wilcoxon_state *,
- const struct two_sample_test *,
- bool exact, double timer);
-
-
-
-static void
-distinct_callback (double v UNUSED, casenumber n, double w UNUSED, void *aux)
-{
- struct wilcoxon_state *ws = aux;
-
- ws->tiebreaker += pow3 (n) - n;
-}
-
-#define WEIGHT_IDX 2
-
-void
-wilcoxon_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer)
-{
- int i;
- bool warn = true;
- const struct dictionary *dict = dataset_dict (ds);
- const struct two_sample_test *t2s = UP_CAST (test, const struct two_sample_test, parent);
-
- struct wilcoxon_state *ws = XCALLOC (t2s->n_pairs, struct wilcoxon_state);
- const struct variable *weight = dict_get_weight (dict);
- struct variable *weightx = dict_create_internal_var (WEIGHT_IDX, 0);
- struct caseproto *proto;
-
- input =
- casereader_create_filter_weight (input, dict, &warn, NULL);
-
- proto = caseproto_create ();
- proto = caseproto_add_width (proto, 0);
- proto = caseproto_add_width (proto, 0);
- if (weight != NULL)
- proto = caseproto_add_width (proto, 0);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- struct casereader *r = casereader_clone (input);
- struct casewriter *writer;
- struct ccase *c;
- struct subcase ordering;
- variable_pair *vp = &t2s->pairs[i];
-
- ws[i].sign = dict_create_internal_var (0, 0);
- ws[i].absdiff = dict_create_internal_var (1, 0);
-
- r = casereader_create_filter_missing (r, *vp, 2,
- exclude,
- NULL, NULL);
-
- subcase_init_var (&ordering, ws[i].absdiff, SC_ASCEND);
- writer = sort_create_writer (&ordering, proto);
- subcase_uninit (&ordering);
-
- for (; (c = casereader_read (r)) != NULL; case_unref (c))
- {
- struct ccase *output = case_create (proto);
- double d = append_difference (c, 0, vp);
-
- if (d > 0)
- *case_num_rw (output, ws[i].sign) = 1.0;
- else if (d < 0)
- *case_num_rw (output, ws[i].sign) = -1.0;
- else
- {
- double w = 1.0;
- if (weight)
- w = case_num (c, weight);
-
- /* Central point values should be dropped */
- ws[i].n_zeros += w;
- case_unref (output);
- continue;
- }
-
- *case_num_rw (output, ws[i].absdiff) = fabs (d);
-
- if (weight)
- *case_num_rw (output, weightx) = case_num (c, weight);
-
- casewriter_write (writer, output);
- }
- casereader_destroy (r);
- ws[i].reader = casewriter_make_reader (writer);
- }
- caseproto_unref (proto);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- struct casereader *rr ;
- struct ccase *c;
- enum rank_error err = 0;
-
- rr = casereader_create_append_rank (ws[i].reader, ws[i].absdiff,
- weight ? weightx : NULL, &err,
- distinct_callback, &ws[i]
- );
-
- for (; (c = casereader_read (rr)) != NULL; case_unref (c))
- {
- double sign = case_num (c, ws[i].sign);
- double rank = case_num_idx (c, weight ? 3 : 2);
- double w = weight ? case_num (c, weightx) : 1.0;
-
- if (sign > 0)
- {
- ws[i].positives.sum += rank * w;
- ws[i].positives.n += w;
- }
- else if (sign < 0)
- {
- ws[i].negatives.sum += rank * w;
- ws[i].negatives.n += w;
- }
- else
- NOT_REACHED ();
- }
-
- casereader_destroy (rr);
- }
-
- casereader_destroy (input);
-
- dict_destroy_internal_var (weightx);
-
- show_ranks_box (ws, t2s, dict);
- show_tests_box (ws, t2s, exact, timer);
-
- for (i = 0 ; i < t2s->n_pairs; ++i)
- {
- dict_destroy_internal_var (ws[i].sign);
- dict_destroy_internal_var (ws[i].absdiff);
- }
-
- free (ws);
-}
-\f
-static void
-put_row (struct pivot_table *table, int var_idx, int sign_idx,
- double n, double sum)
-{
- pivot_table_put3 (table, 0, sign_idx, var_idx, pivot_value_new_number (n));
- if (sum != SYSMIS)
- {
- pivot_table_put3 (table, 1, sign_idx, var_idx,
- pivot_value_new_number (sum / n));
- pivot_table_put3 (table, 2, sign_idx, var_idx,
- pivot_value_new_number (sum));
- }
-}
-
-static int
-add_pair_leaf (struct pivot_dimension *dimension, variable_pair *pair)
-{
- char *label = xasprintf ("%s - %s", var_to_string ((*pair)[0]),
- var_to_string ((*pair)[1]));
- return pivot_category_create_leaf (
- dimension->root,
- pivot_value_new_user_text_nocopy (label));
-}
-
-static void
-show_ranks_box (const struct wilcoxon_state *ws,
- const struct two_sample_test *t2s,
- const struct dictionary *dict)
-{
- struct pivot_table *table = pivot_table_create (N_("Ranks"));
- pivot_table_set_weight_var (table, dict_get_weight (dict));
-
- pivot_dimension_create (table, PIVOT_AXIS_COLUMN, N_("Statistics"),
- N_("N"), PIVOT_RC_COUNT,
- N_("Mean Rank"), PIVOT_RC_OTHER,
- N_("Sum of Ranks"), PIVOT_RC_OTHER);
-
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Sign"),
- N_("Negative Ranks"), N_("Positive Ranks"),
- N_("Ties"), N_("Total"));
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Pairs"));
-
- for (size_t i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
- int pair_idx = add_pair_leaf (pairs, vp);
-
- const struct wilcoxon_state *w = &ws[i];
- put_row (table, pair_idx, 0, w->negatives.n, w->negatives.sum);
- put_row (table, pair_idx, 1, w->positives.n, w->positives.sum);
- put_row (table, pair_idx, 2, w->n_zeros, SYSMIS);
- put_row (table, pair_idx, 3,
- w->n_zeros + w->positives.n + w->negatives.n, SYSMIS);
- }
-
- pivot_table_submit (table);
-}
-
-
-static void
-show_tests_box (const struct wilcoxon_state *ws,
- const struct two_sample_test *t2s,
- bool exact,
- double timer UNUSED
- )
-{
- struct pivot_table *table = pivot_table_create (N_("Test Statistics"));
-
- struct pivot_dimension *statistics = pivot_dimension_create (
- table, PIVOT_AXIS_ROW, N_("Statistics"),
- N_("Z"), PIVOT_RC_OTHER,
- N_("Asymp. Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE);
- if (exact)
- pivot_category_create_leaves (
- statistics->root,
- N_("Exact Sig. (2-tailed)"), PIVOT_RC_SIGNIFICANCE,
- N_("Exact Sig. (1-tailed)"), PIVOT_RC_SIGNIFICANCE);
-
- struct pivot_dimension *pairs = pivot_dimension_create (
- table, PIVOT_AXIS_COLUMN, N_("Pairs"));
-
- struct pivot_footnote *too_many_pairs = pivot_table_create_footnote (
- table, pivot_value_new_text (
- N_("Too many pairs to calculate exact significance")));
-
- for (size_t i = 0 ; i < t2s->n_pairs; ++i)
- {
- variable_pair *vp = &t2s->pairs[i];
- int pair_idx = add_pair_leaf (pairs, vp);
-
- double n = ws[i].positives.n + ws[i].negatives.n;
- double z = MIN (ws[i].positives.sum, ws[i].negatives.sum);
- z -= n * (n + 1)/ 4.0;
- z /= sqrt (n * (n + 1) * (2*n + 1)/24.0 - ws[i].tiebreaker / 48.0);
-
- double entries[4];
- int n_entries = 0;
- entries[n_entries++] = z;
- entries[n_entries++] = 2.0 * gsl_cdf_ugaussian_P (z);
-
- int footnote_idx = -1;
- if (exact)
- {
- double p = LevelOfSignificanceWXMPSR (ws[i].positives.sum, n);
- if (p < 0)
- {
- footnote_idx = n_entries;
- entries[n_entries++] = SYSMIS;
- }
- else
- {
- entries[n_entries++] = p;
- entries[n_entries++] = p / 2.0;
- }
- }
-
- for (int j = 0; j < n_entries; j++)
- {
- struct pivot_value *value = pivot_value_new_number (entries[j]);
- if (j == footnote_idx)
- pivot_value_add_footnote (value, too_many_pairs);
- pivot_table_put2 (table, j, pair_idx, value);
- }
- }
-
- pivot_table_submit (table);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2008, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#if !wilcoxon_h
-#define wilcoxon_h 1
-
-#include <stddef.h>
-#include <stdbool.h>
-
-#include "data/case.h"
-#include "language/stats/npar.h"
-
-struct rank_sum
-{
- double n;
- double sum;
-};
-
-struct wilcoxon_state
-{
- struct casereader *reader;
- struct variable *sign;
- struct variable *absdiff;
-
- struct rank_sum positives;
- struct rank_sum negatives;
- double n_zeros;
-
- double tiebreaker;
-};
-
-
-struct wilcoxon_test
-{
- struct two_sample_test parent;
-};
-
-struct casereader;
-struct dataset;
-
-
-void wilcoxon_execute (const struct dataset *ds,
- struct casereader *input,
- enum mv_class exclude,
- const struct npar_test *test,
- bool exact,
- double timer
- );
-
-
-
-#endif
+++ /dev/null
-# PSPP - a program for statistical analysis.
-# Copyright (C) 2017 Free Software Foundation, Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-## Process this file with automake to produce Makefile.in -*- makefile -*-
-
-
-language_utilities_sources = \
- src/language/utilities/cache.c \
- src/language/utilities/cd.c \
- src/language/utilities/date.c \
- src/language/utilities/echo.c \
- src/language/utilities/host.c \
- src/language/utilities/set.c \
- src/language/utilities/title.c \
- src/language/utilities/include.c \
- src/language/utilities/output.c \
- src/language/utilities/permissions.c
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2010 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <errno.h>
-#include <unistd.h>
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Parses the CACHE command. */
-int
-cmd_cache (struct lexer *lexer UNUSED, struct dataset *ds UNUSED)
-{
- return CMD_SUCCESS;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2008, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/command.h"
-
-#include <errno.h>
-#include <unistd.h>
-
-#include "language/lexer/lexer.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Parses the CD command. */
-int
-cmd_cd (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- char *path = 0;
-
- if (! lex_force_string (lexer))
- goto error;
-
- path = utf8_to_filename (lex_tokcstr (lexer));
-
- if (-1 == chdir (path))
- {
- int err = errno;
- lex_error (lexer, _("Cannot change directory to %s: %s"), path,
- strerror (err));
- goto error;
- }
-
- free (path);
- lex_get (lexer);
-
- return CMD_SUCCESS;
-
- error:
-
- free(path);
-
- return CMD_FAILURE;
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2004, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Stub for USE command. */
-int
-cmd_use (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- if (lex_match (lexer, T_ALL))
- return CMD_SUCCESS;
-
- lex_msg (lexer, SW, _("Only %s is currently implemented."), "USE ALL");
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2005, 2009, 2010 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "output/output-item.h"
-#include "output/driver.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Echos a string to the output stream */
-int
-cmd_echo (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- if (!lex_force_string (lexer))
- return CMD_FAILURE;
-
- output_log ("%s", lex_tokcstr (lexer));
- lex_get (lexer);
-
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* pspp - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <errno.h>
-#include <fcntl.h>
-#include <signal.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <sys/time.h>
-#include <unistd.h>
-#if HAVE_SYS_WAIT_H
-#include <sys/wait.h>
-#endif
-
-#include "data/settings.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "libpspp/string-array.h"
-#include "libpspp/temp-file.h"
-#include "output/driver.h"
-
-#include "gl/error.h"
-#include "gl/intprops.h"
-#include "gl/localcharset.h"
-#include "gl/read-file.h"
-#include "gl/timespec.h"
-#include "gl/xalloc.h"
-#include "gl/xmalloca.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) msgid
-\f
-#if !HAVE_FORK
-#define TIME_LIMIT_SUPPORTED 0
-static bool
-run_commands (const struct string_array *commands, double time_limit)
-{
- assert (time_limit == DBL_MAX);
-
- for (size_t i = 0; i < commands->n; i++)
- {
- /* XXX No way to capture command output */
- char *s = recode_string (locale_charset (), "UTF-8",
- commands->strings[i], -1);
- int retval = system (s);
- free (s);
-
- if (retval)
- {
- msg (SE, _("%s: Command exited with status %d."),
- commands->strings[i], retval);
- return false;
- }
- }
- return true;
-}
-#else
-#define TIME_LIMIT_SUPPORTED 1
-static bool
-run_command (const char *command, struct timespec timeout)
-{
- /* Same exit codes used by 'sh'. */
- enum {
- EXIT_CANNOT_INVOKE = 126,
- EXIT_ENOENT = 127,
- };
-
- /* Create a temporary file to capture command output. */
- FILE *output_file = create_temp_file ();
- if (!output_file)
- {
- msg (SE, _("Failed to create temporary file (%s)."), strerror (errno));
- return false;
- }
-
- int dev_null_fd = open ("/dev/null", O_RDONLY);
- if (dev_null_fd < 0)
- {
- msg (SE, _("/dev/null: Failed to open (%s)."), strerror (errno));
- fclose (output_file);
- return false;
- }
-
- char *locale_command = recode_string (locale_charset (), "UTF-8",
- command, -1);
-
- pid_t pid = fork ();
- if (pid < 0)
- {
- close (dev_null_fd);
- fclose (output_file);
- free (locale_command);
-
- msg (SE, _("Couldn't fork: %s."), strerror (errno));
- return false;
- }
- else if (!pid)
- {
- /* Running in the child. */
-
-#if __GNU__
- /* Hurd doesn't support inheriting process timers in a way that works. */
- if (setpgid (0, 0) < 0)
- error (1, errno, _("Failed to set process group."));
-#else
- /* Set up timeout. */
- if (timeout.tv_sec < TYPE_MAXIMUM (time_t))
- {
- signal (SIGALRM, SIG_DFL);
-
- struct timespec left = timespec_sub (timeout, current_timespec ());
- if (timespec_sign (left) <= 0)
- raise (SIGALRM);
-
- struct itimerval it = {
- .it_value = {
- .tv_sec = left.tv_sec,
- .tv_usec = left.tv_nsec / 1000
- }
- };
- if (setitimer (ITIMER_REAL, &it, NULL) < 0)
- error (1, errno, _("Failed to set timeout."));
- }
-#endif
-
- /* Set up file descriptors:
- - /dev/null for stdin
- - Temporary file to capture stdout and stderr.
- - Close everything else.
- */
- dup2 (dev_null_fd, 0);
- dup2 (fileno (output_file), 1);
- dup2 (fileno (output_file), 2);
- close (dev_null_fd);
- for (int fd = 3; fd < 256; fd++)
- close (fd);
-
- /* Choose the shell. */
- const char *shell = getenv ("SHELL");
- if (shell == NULL)
- shell = "/bin/sh";
-
- /* Run subprocess. */
- execl (shell, shell, "-c", locale_command, NULL);
-
- /* Failed to start the shell. */
- _exit (errno == ENOENT ? EXIT_ENOENT : EXIT_CANNOT_INVOKE);
- }
-
- /* Running in the parent. */
- close (dev_null_fd);
- free (locale_command);
-
- /* Wait for child to exit. */
- int status = 0;
- int error = 0;
- for (;;)
- {
-#if __GNU__
- if (timespec_cmp (current_timespec (), timeout) >= 0)
- kill (-pid, SIGALRM);
-
- int flags = WNOHANG;
-#else
- int flags = 0;
-#endif
- pid_t retval = waitpid (pid, &status, flags);
- if (retval == pid)
- break;
- else if (retval < 0)
- {
- if (errno != EINTR)
- {
- error = errno;
- break;
- }
- }
-#if __GNU__
- else if (retval == 0)
- sleep (1);
-#endif
- else
- NOT_REACHED ();
- }
-
- bool ok = true;
- if (error)
- {
- msg (SW, _("While running \"%s\", waiting for child process "
- "failed (%s)."),
- command, strerror (errno));
- ok = false;
- }
-
- if (WIFSIGNALED (status))
- {
- int signum = WTERMSIG (status);
- if (signum == SIGALRM)
- msg (SW, _("Command \"%s\" timed out."), command);
- else
- msg (SW, _("Command \"%s\" terminated by signal %d."), command, signum);
- ok = false;
- }
- else if (WIFEXITED (status) && WEXITSTATUS (status))
- {
- int exit_code = WEXITSTATUS (status);
- const char *detail = (exit_code == EXIT_ENOENT
- ? _("Command or shell not found")
- : exit_code == EXIT_CANNOT_INVOKE
- ? _("Could not invoke command or shell")
- : NULL);
- if (detail)
- msg (SW, _("Command \"%s\" exited with status %d (%s)."),
- command, exit_code, detail);
- else
- msg (SW, _("Command \"%s\" exited with status %d."),
- command, exit_code);
- ok = false;
- }
-
- rewind (output_file);
- size_t length;
- char *locale_output = fread_file (output_file, 0, &length);
- if (!locale_output)
- {
- msg (SW, _("Command \"%s\" output could not be read (%s)."),
- command, strerror (errno));
- ok = false;
- }
- else if (length > 0)
- {
- char *output = recode_string ("UTF-8", locale_charset (),
- locale_output, -1);
-
- /* Drop final new-line, if any. */
- char *end = strchr (output, '\0');
- if (end > output && end[-1] == '\n')
- end[-1] = '\0';
-
- output_log_nocopy (output);
- }
- free (locale_output);
-
- return ok;
-}
-
-static bool
-run_commands (const struct string_array *commands, double time_limit)
-{
- struct timespec timeout = timespec_add (dtotimespec (time_limit),
- current_timespec ());
-
- for (size_t i = 0; i < commands->n; i++)
- {
- if (!run_command (commands->strings[i], timeout))
- return false;
- }
-
- return true;
-}
-#endif
-
-int
-cmd_host (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- if (settings_get_safer_mode ())
- {
- lex_next_error (lexer, -1, -1,
- _("This command not allowed when the %s option is set."),
- "SAFER");
- return CMD_FAILURE;
- }
-
- if (!lex_force_match_phrase (lexer, "COMMAND=[")
- || !lex_force_string (lexer))
- return CMD_FAILURE;
-
- struct string_array commands = STRING_ARRAY_INITIALIZER;
- while (lex_token (lexer) == T_STRING)
- {
- string_array_append (&commands, lex_tokcstr (lexer));
- lex_get (lexer);
- }
- if (!lex_force_match (lexer, T_RBRACK))
- {
- string_array_destroy (&commands);
- return CMD_FAILURE;
- }
-
- double time_limit = DBL_MAX;
- if (lex_match_id (lexer, "TIMELIMIT"))
- {
- int time_limit_start = lex_ofs (lexer) - 1;
- if (!lex_force_match (lexer, T_EQUALS)
- || !lex_force_num (lexer))
- {
- string_array_destroy (&commands);
- return CMD_FAILURE;
- }
-
- double num = lex_number (lexer);
- lex_get (lexer);
- time_limit = num < 0.0 ? 0.0 : num;
-
- int time_limit_end = lex_ofs (lexer) - 1;
- if (!TIME_LIMIT_SUPPORTED)
- {
- lex_ofs_error (lexer, time_limit_start, time_limit_end,
- _("Time limit not supported on this platform."));
- string_array_destroy (&commands);
- return false;
- }
- }
-
- enum cmd_result result = lex_end_of_command (lexer);
- if (result == CMD_SUCCESS && !run_commands (&commands, time_limit))
- result = CMD_FAILURE;
- string_array_destroy (&commands);
- return result;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2007, 2010, 2011, 2020 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <errno.h>
-#include <stdlib.h>
-#include <string.h>
-#include <unistd.h>
-
-#include "data/dataset.h"
-#include "data/session.h"
-#include "language/command.h"
-#include "language/lexer/include-path.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/dirname.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-enum variant
- {
- INSERT,
- INCLUDE
- };
-
-static int
-do_insert (struct lexer *lexer, struct dataset *ds, enum variant variant)
-{
- /* Skip optional FILE=. */
- if (lex_match_id (lexer, "FILE"))
- lex_match (lexer, T_EQUALS);
-
- if (!lex_force_string_or_id (lexer))
- return CMD_FAILURE;
-
- char *relative_name = utf8_to_filename (lex_tokcstr (lexer));
- char *filename = include_path_search (relative_name);
- free (relative_name);
-
- if (!filename)
- {
- msg (SE, _("Can't find `%s' in include file search path."),
- lex_tokcstr (lexer));
- return CMD_FAILURE;
- }
- lex_get (lexer);
-
- enum segmenter_mode syntax_mode = SEG_MODE_INTERACTIVE;
- enum lex_error_mode error_mode = LEX_ERROR_CONTINUE;
- bool cd = false;
- int status = CMD_FAILURE;
- char *encoding = xstrdup (session_get_default_syntax_encoding (
- dataset_session (ds)));
- while (T_ENDCMD != lex_token (lexer))
- {
- if (lex_match_id (lexer, "ENCODING"))
- {
- lex_match (lexer, T_EQUALS);
- if (!lex_force_string (lexer))
- goto exit;
-
- free (encoding);
- encoding = xstrdup (lex_tokcstr (lexer));
- lex_get (lexer);
- }
- else if (variant == INSERT && lex_match_id (lexer, "SYNTAX"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "INTERACTIVE"))
- syntax_mode = SEG_MODE_INTERACTIVE;
- else if (lex_match_id (lexer, "BATCH"))
- syntax_mode = SEG_MODE_BATCH;
- else if (lex_match_id (lexer, "AUTO"))
- syntax_mode = SEG_MODE_AUTO;
- else
- {
- lex_error_expecting (lexer, "BATCH", "INTERACTIVE", "AUTO");
- goto exit;
- }
- }
- else if (variant == INSERT && lex_match_id (lexer, "CD"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "YES"))
- cd = true;
- else if (lex_match_id (lexer, "NO"))
- cd = false;
- else
- {
- lex_error_expecting (lexer, "YES", "NO");
- goto exit;
- }
- }
- else if (variant == INSERT && lex_match_id (lexer, "ERROR"))
- {
- lex_match (lexer, T_EQUALS);
- if (lex_match_id (lexer, "CONTINUE"))
- error_mode = LEX_ERROR_CONTINUE;
- else if (lex_match_id (lexer, "STOP"))
- error_mode = LEX_ERROR_STOP;
- else if (settings_get_testing_mode ()
- && lex_match_id (lexer, "IGNORE"))
- error_mode = LEX_ERROR_IGNORE;
- else
- {
- lex_error_expecting (lexer, "CONTINUE", "STOP");
- goto exit;
- }
- }
- else
- {
- if (variant == INSERT)
- lex_error_expecting (lexer, "ENCODING", "SYNTAX", "CD", "ERROR");
- else
- lex_error_expecting (lexer, "ENCODING");
- goto exit;
- }
- }
- status = lex_end_of_command (lexer);
-
- if (status == CMD_SUCCESS)
- {
- struct lex_reader *reader = lex_reader_for_file (filename, encoding,
- syntax_mode, error_mode);
- if (reader != NULL)
- {
- lex_discard_rest_of_command (lexer);
- lex_include (lexer, reader);
-
- if (cd)
- {
- char *directory = dir_name (filename);
- if (chdir (directory))
- {
- int err = errno;
- msg (SE, _("Cannot change directory to %s: %s"), directory,
- strerror (err));
- status = CMD_FAILURE;
- }
-
- free (directory);
- }
- }
- }
-
-exit:
- free (encoding);
- free (filename);
- return status;
-}
-
-int
-cmd_include (struct lexer *lexer, struct dataset *ds)
-{
- return do_insert (lexer, ds, INCLUDE);
-}
-
-int
-cmd_insert (struct lexer *lexer, struct dataset *ds)
-{
- return do_insert (lexer, ds, INSERT);
-}
-
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/settings.h"
-#include "data/format.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/format-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/string-set.h"
-#include "libpspp/version.h"
-#include "output/pivot-table.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-int
-cmd_output_modify (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- struct string_set rc_names = STRING_SET_INITIALIZER (rc_names);
-
- while (lex_token (lexer) != T_ENDCMD)
- {
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "SELECT"))
- {
- if (!lex_force_match_id (lexer, "TABLES"))
- goto error;
- }
- else if (lex_match_id (lexer, "TABLECELLS"))
- {
- string_set_clear (&rc_names);
- struct fmt_spec fmt = { .type = 0 };
-
- while (lex_token (lexer) != T_SLASH &&
- lex_token (lexer) != T_ENDCMD)
- {
- if (lex_match_id (lexer, "SELECT"))
- {
- if (!lex_force_match (lexer, T_EQUALS))
- goto error;
-
- if (!lex_force_match (lexer, T_LBRACK))
- goto error;
-
- while (lex_token (lexer) == T_ID)
- {
- string_set_insert (&rc_names, lex_tokcstr (lexer));
- lex_get (lexer);
- }
-
- if (!lex_force_match (lexer, T_RBRACK))
- goto error;
- }
- else if (lex_match_id (lexer, "FORMAT"))
- {
- char type[FMT_TYPE_LEN_MAX + 1];
- uint16_t width;
- uint8_t decimals;
-
- if (!lex_force_match (lexer, T_EQUALS)
- || !parse_abstract_format_specifier (lexer, type,
- &width, &decimals))
- goto error;
-
- if (width <= 0)
- {
- const struct fmt_spec *dflt = settings_get_format ();
- width = dflt->w;
- }
-
- if (!fmt_from_name (type, &fmt.type))
- {
- lex_error (lexer, _("Unknown format type `%s'."), type);
- goto error;
- }
-
- fmt.w = width;
- fmt.d = decimals;
- }
- else
- {
- lex_error_expecting (lexer, "SELECT", "FORMAT");
- goto error;
- }
- }
-
- if (fmt.w)
- {
- const struct string_set_node *node;
- const char *s;
- STRING_SET_FOR_EACH (s, node, &rc_names)
- if (!pivot_result_class_change (s, &fmt))
- lex_error (lexer, _("Unknown cell class %s."), s);
- }
- }
- else
- {
- lex_error_expecting (lexer, "SELECT", "TABLECELLS");
- goto error;
- }
- }
-
- string_set_destroy (&rc_names);
- return CMD_SUCCESS;
-
- error:
- string_set_destroy (&rc_names);
- return CMD_FAILURE;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2004, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <errno.h>
-#include <stdlib.h>
-#include <sys/stat.h>
-#include <sys/types.h>
-#include <unistd.h>
-
-#include "data/settings.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/cast.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/str.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-enum PER {PER_RO, PER_RW};
-
-int change_permissions(const char *file_name, enum PER per);
-
-
-/* Parses the PERMISSIONS command. */
-int
-cmd_permissions (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- if (settings_get_safer_mode ())
- {
- lex_next_error (lexer, -1, -1,
- _("This command not allowed when the %s option is set."),
- "SAFER");
- return 0;
- }
-
- char *fn = NULL;
- const char *str = NULL;
- lex_match (lexer, T_SLASH);
-
- if (lex_match_id (lexer, "FILE"))
- lex_match (lexer, T_EQUALS);
-
- str = lex_tokcstr (lexer);
- if (str)
- fn = strdup (str);
-
- if (!lex_force_match (lexer, T_STRING) || str == NULL)
- goto error;
-
- lex_match (lexer, T_SLASH);
-
- if (! lex_match_id (lexer, "PERMISSIONS"))
- goto error;
-
- lex_match (lexer, T_EQUALS);
-
- if (lex_match_id (lexer, "READONLY"))
- {
- if (! change_permissions (fn, PER_RO))
- goto error;
- }
- else if (lex_match_id (lexer, "WRITEABLE"))
- {
- if (! change_permissions (fn, PER_RW))
- goto error;
- }
- else
- {
- lex_error_expecting (lexer, "WRITEABLE", "READONLY");
- goto error;
- }
-
- free (fn);
-
- return CMD_SUCCESS;
-
- error:
-
- free(fn);
-
- return CMD_FAILURE;
-}
-
-
-
-int
-change_permissions (const char *file_name, enum PER per)
-{
- char *locale_file_name;
- struct stat buf;
- mode_t mode;
-
- locale_file_name = utf8_to_filename (file_name);
- if (-1 == stat(locale_file_name, &buf))
- {
- const int errnum = errno;
- msg (SE, _("Cannot stat %s: %s"), file_name, strerror(errnum));
- free (locale_file_name);
- return 0;
- }
-
- if (per == PER_RW)
- mode = buf.st_mode | 0200;
- else
- mode = buf.st_mode & ~0222;
-
- if (-1 == chmod(locale_file_name, mode))
-
- {
- const int errnum = errno;
- msg (SE, _("Cannot change mode of %s: %s"), file_name, strerror(errnum));
- free (locale_file_name);
- return 0;
- }
-
- free (locale_file_name);
-
- return 1;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2006, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <stdio.h>
-#include <errno.h>
-#include <stdlib.h>
-#include <time.h>
-#include <unistd.h>
-
-#include "data/casereader.h"
-#include "data/data-in.h"
-#include "data/data-out.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/settings.h"
-#include "data/value.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/format-parser.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/token.h"
-#include "libpspp/assertion.h"
-#include "libpspp/compiler.h"
-#include "libpspp/copyleft.h"
-#include "libpspp/temp-file.h"
-#include "libpspp/version.h"
-#include "libpspp/float-format.h"
-#include "libpspp/i18n.h"
-#include "libpspp/integer-format.h"
-#include "libpspp/message.h"
-#include "math/random.h"
-#include "output/driver.h"
-#include "output/journal.h"
-#include "output/pivot-table.h"
-
-#include "gl/ftoastr.h"
-#include "gl/minmax.h"
-#include "gl/relocatable.h"
-#include "gl/vasnprintf.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-#define N_(msgid) (msgid)
-
-struct setting
- {
- const char *name;
- bool (*set) (struct lexer *);
- char *(*show) (const struct dataset *);
- };
-
-static bool
-match_subcommand (struct lexer *lexer, const char *name)
-{
- if (lex_match_id (lexer, name))
- {
- lex_match (lexer, T_EQUALS);
- return true;
- }
- else
- return false;
-}
-
-static int
-subcommand_start_ofs (struct lexer *lexer)
-{
- int ofs = lex_ofs (lexer) - 1;
- return lex_ofs_token (lexer, ofs)->type == T_EQUALS ? ofs - 1 : ofs;
-}
-
-static int
-parse_enum_valist (struct lexer *lexer, va_list args)
-{
- for (;;)
- {
- const char *name = va_arg (args, char *);
- if (!name)
- return -1;
- int value = va_arg (args, int);
-
- if (lex_match_id (lexer, name))
- return value;
- }
-}
-
-#define parse_enum(...) parse_enum (__VA_ARGS__, NULL_SENTINEL)
-static int SENTINEL(0)
-(parse_enum) (struct lexer *lexer, ...)
-{
- va_list args;
-
- va_start (args, lexer);
- int retval = parse_enum_valist (lexer, args);
- va_end (args);
-
- return retval;
-}
-
-#define force_parse_enum(...) force_parse_enum (__VA_ARGS__, NULL_SENTINEL)
-static int SENTINEL(0)
-(force_parse_enum) (struct lexer *lexer, ...)
-{
- va_list args;
-
- va_start (args, lexer);
- int retval = parse_enum_valist (lexer, args);
- va_end (args);
-
- if (retval == -1)
- {
- enum { MAX_OPTIONS = 9 };
- const char *options[MAX_OPTIONS];
- int n = 0;
-
- va_start (args, lexer);
- while (n < MAX_OPTIONS)
- {
- const char *name = va_arg (args, char *);
- if (!name)
- break;
- va_arg (args, int);
-
- options[n++] = name;
- }
- va_end (args);
-
- lex_error_expecting_array (lexer, options, n);
- }
-
- return retval;
-}
-
-static int
-parse_bool (struct lexer *lexer)
-{
- return parse_enum (lexer,
- "ON", true, "YES", true,
- "OFF", false, "NO", false);
-}
-
-static int
-force_parse_bool (struct lexer *lexer)
-{
- return force_parse_enum (lexer,
- "ON", true, "YES", true,
- "OFF", false, "NO", false);
-}
-
-static bool
-parse_output_routing (struct lexer *lexer, enum settings_output_type type)
-{
- enum settings_output_devices devices;
- if (lex_match_id (lexer, "ON") || lex_match_id (lexer, "BOTH"))
- devices = SETTINGS_DEVICE_LISTING | SETTINGS_DEVICE_TERMINAL;
- else if (lex_match_id (lexer, "TERMINAL"))
- devices = SETTINGS_DEVICE_TERMINAL;
- else if (lex_match_id (lexer, "LISTING"))
- devices = SETTINGS_DEVICE_LISTING;
- else if (lex_match_id (lexer, "OFF") || lex_match_id (lexer, "NONE"))
- devices = 0;
- else
- {
- lex_error_expecting (lexer, "ON", "BOTH", "TERMINAL", "LISTING",
- "OFF", "NONE");
- return false;
- }
-
- settings_set_output_routing (type, devices);
-
- return true;
-}
-
-static char *
-show_output_routing (enum settings_output_type type)
-{
- enum settings_output_devices devices;
- const char *s;
-
- devices = settings_get_output_routing (type);
- if (devices & SETTINGS_DEVICE_LISTING)
- s = devices & SETTINGS_DEVICE_TERMINAL ? "BOTH" : "LISTING";
- else if (devices & SETTINGS_DEVICE_TERMINAL)
- s = "TERMINAL";
- else
- s = "NONE";
-
- return xstrdup (s);
-}
-
-static bool
-parse_integer_format (struct lexer *lexer,
- void (*set_format) (enum integer_format))
-{
- int value = force_parse_enum (lexer,
- "MSBFIRST", INTEGER_MSB_FIRST,
- "LSBFIRST", INTEGER_LSB_FIRST,
- "VAX", INTEGER_VAX,
- "NATIVE", INTEGER_NATIVE);
- if (value >= 0)
- set_format (value);
- return value >= 0;
-}
-
-/* Returns a name for the given INTEGER_FORMAT value. */
-static char *
-show_integer_format (enum integer_format integer_format)
-{
- return xasprintf ("%s (%s)",
- (integer_format == INTEGER_MSB_FIRST ? "MSBFIRST"
- : integer_format == INTEGER_LSB_FIRST ? "LSBFIRST"
- : "VAX"),
- integer_format == INTEGER_NATIVE ? "NATIVE" : "nonnative");
-}
-
-static bool
-parse_real_format (struct lexer *lexer,
- void (*set_format) (enum float_format))
-{
- int value = force_parse_enum (lexer,
- "NATIVE", FLOAT_NATIVE_DOUBLE,
- "ISL", FLOAT_IEEE_SINGLE_LE,
- "ISB", FLOAT_IEEE_SINGLE_BE,
- "IDL", FLOAT_IEEE_DOUBLE_LE,
- "IDB", FLOAT_IEEE_DOUBLE_BE,
- "VF", FLOAT_VAX_F,
- "VD", FLOAT_VAX_D,
- "VG", FLOAT_VAX_G,
- "ZS", FLOAT_Z_SHORT,
- "ZL", FLOAT_Z_LONG);
- if (value >= 0)
- set_format (value);
- return value >= 0;
-}
-
-/* Returns a name for the given FLOAT_FORMAT value. */
-static char *
-show_real_format (enum float_format float_format)
-{
- const char *format_name = "";
-
- switch (float_format)
- {
- case FLOAT_IEEE_SINGLE_LE:
- format_name = _("ISL (32-bit IEEE 754 single, little-endian)");
- break;
- case FLOAT_IEEE_SINGLE_BE:
- format_name = _("ISB (32-bit IEEE 754 single, big-endian)");
- break;
- case FLOAT_IEEE_DOUBLE_LE:
- format_name = _("IDL (64-bit IEEE 754 double, little-endian)");
- break;
- case FLOAT_IEEE_DOUBLE_BE:
- format_name = _("IDB (64-bit IEEE 754 double, big-endian)");
- break;
-
- case FLOAT_VAX_F:
- format_name = _("VF (32-bit VAX F, VAX-endian)");
- break;
- case FLOAT_VAX_D:
- format_name = _("VD (64-bit VAX D, VAX-endian)");
- break;
- case FLOAT_VAX_G:
- format_name = _("VG (64-bit VAX G, VAX-endian)");
- break;
-
- case FLOAT_Z_SHORT:
- format_name = _("ZS (32-bit IBM Z hexadecimal short, big-endian)");
- break;
- case FLOAT_Z_LONG:
- format_name = _("ZL (64-bit IBM Z hexadecimal long, big-endian)");
- break;
-
- case FLOAT_FP:
- case FLOAT_HEX:
- NOT_REACHED ();
- }
-
- return xasprintf ("%s (%s)", format_name,
- (float_format == FLOAT_NATIVE_DOUBLE
- ? "NATIVE" : "nonnative"));
-}
-
-static bool
-parse_unimplemented (struct lexer *lexer, const char *name)
-{
- int start = subcommand_start_ofs (lexer);
- if (lex_token (lexer) != T_SLASH && lex_token (lexer) != T_ENDCMD)
- lex_get (lexer);
- int end = lex_ofs (lexer) - 1;
-
- lex_ofs_msg (lexer, SW, start, end, _("%s is not yet implemented."), name);
- return true;
-}
-
-static bool
-parse_ccx (struct lexer *lexer, enum fmt_type ccx)
-{
- if (!lex_force_string (lexer))
- return false;
-
- char *error = settings_set_cc (lex_tokcstr (lexer), ccx);
- if (error)
- {
- lex_error (lexer, "%s", error);
- free (error);
- return false;
- }
-
- lex_get (lexer);
- return true;
-}
-\f
-static bool
-parse_BASETEXTDIRECTION (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "BASETEXTDIRECTION");
-}
-
-static bool
-parse_BLANKS (struct lexer *lexer)
-{
- if (lex_match_id (lexer, "SYSMIS"))
- settings_set_blanks (SYSMIS);
- else
- {
- if (!lex_force_num (lexer))
- return false;
- settings_set_blanks (lex_number (lexer));
- lex_get (lexer);
- }
- return true;
-}
-
-static char *
-show_BLANKS (const struct dataset *ds UNUSED)
-{
- return (settings_get_blanks () == SYSMIS
- ? xstrdup ("SYSMIS")
- : xasprintf ("%.*g", DBL_DIG + 1, settings_get_blanks ()));
-}
-
-static bool
-parse_BLOCK (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "BLOCK");
-}
-
-static bool
-parse_BOX (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "BOX");
-}
-
-static bool
-parse_CACHE (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "CACHE");
-}
-
-static bool
-parse_CCA (struct lexer *lexer)
-{
- return parse_ccx (lexer, FMT_CCA);
-}
-
-static bool
-parse_CCB (struct lexer *lexer)
-{
- return parse_ccx (lexer, FMT_CCB);
-}
-
-static bool
-parse_CCC (struct lexer *lexer)
-{
- return parse_ccx (lexer, FMT_CCC);
-}
-
-static bool
-parse_CCD (struct lexer *lexer)
-{
- return parse_ccx (lexer, FMT_CCD);
-}
-
-static bool
-parse_CCE (struct lexer *lexer)
-{
- return parse_ccx (lexer, FMT_CCE);
-}
-
-static char *
-show_cc (enum fmt_type type)
-{
- return fmt_number_style_to_string (fmt_settings_get_style (
- settings_get_fmt_settings (), type));
-}
-
-static char *
-show_CCA (const struct dataset *ds UNUSED)
-{
- return show_cc (FMT_CCA);
-}
-
-static char *
-show_CCB (const struct dataset *ds UNUSED)
-{
- return show_cc (FMT_CCB);
-}
-
-static char *
-show_CCC (const struct dataset *ds UNUSED)
-{
- return show_cc (FMT_CCC);
-}
-
-static char *
-show_CCD (const struct dataset *ds UNUSED)
-{
- return show_cc (FMT_CCD);
-}
-
-static char *
-show_CCE (const struct dataset *ds UNUSED)
-{
- return show_cc (FMT_CCE);
-}
-
-static bool
-parse_CELLSBREAK (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "CELLSBREAK");
-}
-
-static bool
-parse_CMPTRANS (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "CMPTRANS");
-}
-
-static bool
-parse_COMPRESSION (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "COMPRESSION");
-}
-
-static bool
-parse_CTEMPLATE (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "CTEMPLATE");
-}
-
-static bool
-parse_DECIMAL (struct lexer *lexer)
-{
- int decimal_char = force_parse_enum (lexer,
- "DOT", '.',
- "COMMA", ',');
- if (decimal_char != -1)
- settings_set_decimal_char (decimal_char);
- return decimal_char != -1;
-}
-
-static char *
-show_DECIMAL (const struct dataset *ds UNUSED)
-{
- return xasprintf ("`%c'", settings_get_fmt_settings ()->decimal);
-}
-
-static bool
-parse_EPOCH (struct lexer *lexer)
-{
- if (lex_match_id (lexer, "AUTOMATIC"))
- settings_set_epoch (-1);
- else if (lex_is_integer (lexer))
- {
- if (!lex_force_int_range (lexer, "EPOCH", 1500, INT_MAX))
- return false;
- settings_set_epoch (lex_integer (lexer));
- lex_get (lexer);
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting %s or year."), "AUTOMATIC");
- return false;
- }
-
- return true;
-}
-
-static char *
-show_EPOCH (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_epoch ());
-}
-
-static bool
-parse_ERRORS (struct lexer *lexer)
-{
- return parse_output_routing (lexer, SETTINGS_OUTPUT_ERROR);
-}
-
-static char *
-show_ERRORS (const struct dataset *ds UNUSED)
-{
- return show_output_routing (SETTINGS_OUTPUT_ERROR);
-}
-
-static bool
-parse_FORMAT (struct lexer *lexer)
-{
- int start = subcommand_start_ofs (lexer);
- struct fmt_spec fmt;
-
- if (!parse_format_specifier (lexer, &fmt))
- return false;
-
- char *error = fmt_check_output__ (&fmt);
- if (error)
- {
- lex_next_error (lexer, -1, -1, "%s", error);
- free (error);
- return false;
- }
-
- int end = lex_ofs (lexer) - 1;
- if (fmt_is_string (fmt.type))
- {
- char str[FMT_STRING_LEN_MAX + 1];
- lex_ofs_error (lexer, start, end,
- _("%s requires numeric output format as an argument. "
- "Specified format %s is of type string."),
- "FORMAT", fmt_to_string (&fmt, str));
- return false;
- }
-
- settings_set_format (&fmt);
- return true;
-}
-
-static char *
-show_FORMAT (const struct dataset *ds UNUSED)
-{
- char str[FMT_STRING_LEN_MAX + 1];
- return xstrdup (fmt_to_string (settings_get_format (), str));
-}
-
-static bool
-parse_FUZZBITS (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "FUZZBITS", 0, 20))
- return false;
- settings_set_fuzzbits (lex_integer (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_FUZZBITS (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_fuzzbits ());
-}
-
-static bool
-parse_HEADER (struct lexer *lexer)
-{
- return parse_unimplemented (lexer, "HEADER");
-}
-
-static bool
-parse_INCLUDE (struct lexer *lexer)
-{
- int include = force_parse_bool (lexer);
- if (include != -1)
- settings_set_include (include);
- return include != -1;
-}
-
-static char *
-show_INCLUDE (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_include () ? "ON" : "OFF");
-}
-
-static bool
-parse_JOURNAL (struct lexer *lexer)
-{
- int b = parse_bool (lexer);
- if (b == true)
- journal_enable ();
- else if (b == false)
- journal_disable ();
- else if (lex_is_string (lexer) || lex_token (lexer) == T_ID)
- {
- char *filename = utf8_to_filename (lex_tokcstr (lexer));
- journal_set_file_name (filename);
- free (filename);
-
- lex_get (lexer);
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting ON or OFF or a file name."));
- return false;
- }
- return true;
-}
-
-static char *
-show_JOURNAL (const struct dataset *ds UNUSED)
-{
- const char *enabled = journal_is_enabled () ? "ON" : "OFF";
- const char *file_name = journal_get_file_name ();
- return (file_name
- ? xasprintf ("%s (%s)", enabled, file_name)
- : xstrdup (enabled));
-}
-
-static bool
-parse_LEADZERO (struct lexer *lexer)
-{
- int leadzero = force_parse_bool (lexer);
- if (leadzero != -1)
- settings_set_include_leading_zero (leadzero);
- return leadzero != -1;
-}
-
-static char *
-show_LEADZERO (const struct dataset *ds UNUSED)
-{
- bool leadzero = settings_get_fmt_settings ()->include_leading_zero;
- return xstrdup (leadzero ? "ON" : "OFF");
-}
-
-static bool
-parse_LENGTH (struct lexer *lexer)
-{
- int page_length;
-
- if (lex_match_id (lexer, "NONE"))
- page_length = -1;
- else
- {
- if (!lex_force_int_range (lexer, "LENGTH", 1, INT_MAX))
- return false;
- page_length = lex_integer (lexer);
- lex_get (lexer);
- }
-
- if (page_length != -1)
- settings_set_viewlength (page_length);
-
- return true;
-}
-
-static char *
-show_LENGTH (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_viewlength ());
-}
-
-static bool
-parse_LOCALE (struct lexer *lexer)
-{
- if (!lex_force_string (lexer))
- return false;
-
- /* Try the argument as an encoding name, then as a locale name or alias. */
- const char *s = lex_tokcstr (lexer);
- if (valid_encoding (s))
- set_default_encoding (s);
- else if (!set_encoding_from_locale (s))
- {
- lex_error (lexer, _("%s is not a recognized encoding or locale name"), s);
- return false;
- }
-
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_LOCALE (const struct dataset *ds UNUSED)
-{
- return xstrdup (get_default_encoding ());
-}
-
-static bool
-parse_MDISPLAY (struct lexer *lexer)
-{
- int mdisplay = force_parse_enum (lexer,
- "TEXT", SETTINGS_MDISPLAY_TEXT,
- "TABLES", SETTINGS_MDISPLAY_TABLES);
- if (mdisplay >= 0)
- settings_set_mdisplay (mdisplay);
- return mdisplay >= 0;
-}
-
-static char *
-show_MDISPLAY (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_mdisplay () == SETTINGS_MDISPLAY_TEXT
- ? "TEXT" : "TABLES");
-}
-
-static bool
-parse_MESSAGES (struct lexer *lexer)
-{
- return parse_output_routing (lexer, SETTINGS_OUTPUT_NOTE);
-}
-
-static char *
-show_MESSAGES (const struct dataset *ds UNUSED)
-{
- return show_output_routing (SETTINGS_OUTPUT_NOTE);
-}
-
-static bool
-parse_MEXPAND (struct lexer *lexer)
-{
- int mexpand = force_parse_bool (lexer);
- if (mexpand != -1)
- settings_set_mexpand (mexpand);
- return mexpand != -1;
-}
-
-static char *
-show_MEXPAND (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_mexpand () ? "ON" : "OFF");
-}
-
-static bool
-parse_MITERATE (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "MITERATE", 1, INT_MAX))
- return false;
- settings_set_miterate (lex_integer (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_MITERATE (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_miterate ());
-}
-
-static bool
-parse_MNEST (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "MNEST", 1, INT_MAX))
- return false;
- settings_set_mnest (lex_integer (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_MNEST (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_mnest ());
-}
-
-static bool
-parse_MPRINT (struct lexer *lexer)
-{
- int mprint = force_parse_bool (lexer);
- if (mprint != -1)
- settings_set_mprint (mprint);
- return mprint != -1;
-}
-
-static char *
-show_MPRINT (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_mprint () ? "ON" : "OFF");
-}
-
-static bool
-parse_MXERRS (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "MXERRS", 1, INT_MAX))
- return false;
- settings_set_max_messages (MSG_S_ERROR, lex_integer (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_MXERRS (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_max_messages (MSG_S_ERROR));
-}
-
-static bool
-parse_MXLOOPS (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "MXLOOPS", 1, INT_MAX))
- return false;
- settings_set_mxloops (lex_integer (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_MXLOOPS (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_mxloops ());
-}
-
-static bool
-parse_MXWARNS (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "MXWARNS", 0, INT_MAX))
- return false;
- settings_set_max_messages (MSG_S_WARNING, lex_integer (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_MXWARNS (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_max_messages (MSG_S_WARNING));
-}
-
-static bool
-parse_PRINTBACK (struct lexer *lexer)
-{
- return parse_output_routing (lexer, SETTINGS_OUTPUT_SYNTAX);
-}
-
-static char *
-show_PRINTBACK (const struct dataset *ds UNUSED)
-{
- return show_output_routing (SETTINGS_OUTPUT_SYNTAX);
-}
-
-static bool
-parse_RESULTS (struct lexer *lexer)
-{
- return parse_output_routing (lexer, SETTINGS_OUTPUT_RESULT);
-}
-
-static char *
-show_RESULTS (const struct dataset *ds UNUSED)
-{
- return show_output_routing (SETTINGS_OUTPUT_RESULT);
-}
-
-static bool
-parse_RIB (struct lexer *lexer)
-{
- return parse_integer_format (lexer, settings_set_input_integer_format);
-}
-
-static char *
-show_RIB (const struct dataset *ds UNUSED)
-{
- return show_integer_format (settings_get_input_integer_format ());
-}
-
-static bool
-parse_RRB (struct lexer *lexer)
-{
- return parse_real_format (lexer, settings_set_input_float_format);
-}
-
-static char *
-show_RRB (const struct dataset *ds UNUSED)
-{
- return show_real_format (settings_get_input_float_format ());
-}
-
-static bool
-parse_SAFER (struct lexer *lexer)
-{
- bool ok = force_parse_enum (lexer, "ON", true, "YES", true) != -1;
- if (ok)
- settings_set_safer_mode ();
- return ok;
-}
-
-static char *
-show_SAFER (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_safer_mode () ? "ON" : "OFF");
-}
-
-static bool
-parse_SCOMPRESSION (struct lexer *lexer)
-{
- int value = force_parse_bool (lexer);
- if (value >= 0)
- settings_set_scompression (value);
- return value >= 0;
-}
-
-static char *
-show_SCOMPRESSION (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_scompression () ? "ON" : "OFF");
-}
-
-static bool
-parse_SEED (struct lexer *lexer)
-{
- if (lex_match_id (lexer, "RANDOM"))
- set_rng (time (0));
- else
- {
- if (!lex_force_num (lexer))
- return false;
- set_rng (lex_number (lexer));
- lex_get (lexer);
- }
-
- return true;
-}
-
-static bool
-parse_SMALL (struct lexer *lexer)
-{
- if (!lex_force_num (lexer))
- return false;
- settings_set_small (lex_number (lexer));
- lex_get (lexer);
- return true;
-}
-
-static char *
-show_SMALL (const struct dataset *ds UNUSED)
-{
- char buf[DBL_BUFSIZE_BOUND];
- if (dtoastr (buf, sizeof buf, 0, 0, settings_get_small ()) < 0)
- abort ();
- return xstrdup (buf);
-}
-
-static char *
-show_SUBTITLE (const struct dataset *ds UNUSED)
-{
- return xstrdup (output_get_subtitle ());
-}
-
-static char *
-show_TEMPDIR (const struct dataset *ds UNUSED)
-{
- return xstrdup (temp_dir_name ());
-}
-
-static char *
-show_TITLE (const struct dataset *ds UNUSED)
-{
- return xstrdup (output_get_title ());
-}
-
-static bool
-parse_TNUMBERS (struct lexer *lexer)
-{
- int value = force_parse_enum (lexer,
- "LABELS", SETTINGS_VALUE_SHOW_LABEL,
- "VALUES", SETTINGS_VALUE_SHOW_VALUE,
- "BOTH", SETTINGS_VALUE_SHOW_BOTH);
- if (value >= 0)
- settings_set_show_values (value);
- return value >= 0;
-}
-
-static char *
-show_TNUMBERS (const struct dataset *ds UNUSED)
-{
- enum settings_value_show tnumbers = settings_get_show_values ();
- return xstrdup (tnumbers == SETTINGS_VALUE_SHOW_LABEL ? "LABELS"
- : tnumbers == SETTINGS_VALUE_SHOW_VALUE ? "VALUES"
- : "BOTH");
-}
-
-static bool
-parse_TVARS (struct lexer *lexer)
-{
- int value = force_parse_enum (lexer,
- "LABELS", SETTINGS_VALUE_SHOW_LABEL,
- "NAMES", SETTINGS_VALUE_SHOW_VALUE,
- "BOTH", SETTINGS_VALUE_SHOW_BOTH);
- if (value >= 0)
- settings_set_show_variables (value);
- return value >= 0;
-}
-
-static char *
-show_TVARS (const struct dataset *ds UNUSED)
-{
- enum settings_value_show tvars = settings_get_show_variables ();
- return xstrdup (tvars == SETTINGS_VALUE_SHOW_LABEL ? "LABELS"
- : tvars == SETTINGS_VALUE_SHOW_VALUE ? "NAMES"
- : "BOTH");
-}
-
-static bool
-parse_TLOOK (struct lexer *lexer)
-{
- if (lex_match_id (lexer, "NONE"))
- pivot_table_look_set_default (pivot_table_look_builtin_default ());
- else if (lex_is_string (lexer))
- {
- struct pivot_table_look *look;
- char *error = pivot_table_look_read (lex_tokcstr (lexer), &look);
- lex_get (lexer);
-
- if (error)
- {
- msg (SE, "%s", error);
- free (error);
- return false;
- }
-
- pivot_table_look_set_default (look);
- pivot_table_look_unref (look);
- }
-
- return true;
-}
-
-static bool
-parse_UNDEFINED (struct lexer *lexer)
-{
- int value = force_parse_enum (lexer,
- "WARN", true,
- "NOWARN", false);
- if (value >= 0)
- settings_set_undefined (value);
- return value >= 0;
-}
-
-static char *
-show_UNDEFINED (const struct dataset *ds UNUSED)
-{
- return xstrdup (settings_get_undefined () ? "WARN" : "NOWARN");
-}
-
-static char *
-show_VERSION (const struct dataset *ds UNUSED)
-{
- return strdup (announced_version);
-}
-
-static char *
-show_WEIGHT (const struct dataset *ds)
-{
- const struct variable *var = dict_get_weight (dataset_dict (ds));
- return xstrdup (var != NULL ? var_get_name (var) : "OFF");
-}
-
-static bool
-parse_WIB (struct lexer *lexer)
-{
- return parse_integer_format (lexer, settings_set_output_integer_format);
-}
-
-static char *
-show_WIB (const struct dataset *ds UNUSED)
-{
- return show_integer_format (settings_get_output_integer_format ());
-}
-
-static bool
-parse_WRB (struct lexer *lexer)
-{
- return parse_real_format (lexer, settings_set_output_float_format);
-}
-
-static char *
-show_WRB (const struct dataset *ds UNUSED)
-{
- return show_real_format (settings_get_output_float_format ());
-}
-
-static bool
-parse_WIDTH (struct lexer *lexer)
-{
- if (lex_match_id (lexer, "NARROW"))
- settings_set_viewwidth (79);
- else if (lex_match_id (lexer, "WIDE"))
- settings_set_viewwidth (131);
- else
- {
- if (!lex_force_int_range (lexer, "WIDTH", 40, INT_MAX))
- return false;
- settings_set_viewwidth (lex_integer (lexer));
- lex_get (lexer);
- }
-
- return true;
-}
-
-static char *
-show_WIDTH (const struct dataset *ds UNUSED)
-{
- return xasprintf ("%d", settings_get_viewwidth ());
-}
-
-static bool
-parse_WORKSPACE (struct lexer *lexer)
-{
- if (!lex_force_int_range (lexer, "WORKSPACE",
- settings_get_testing_mode () ? 1 : 1024,
- INT_MAX))
- return false;
- int workspace = lex_integer (lexer);
- lex_get (lexer);
- settings_set_workspace (MIN (workspace, INT_MAX / 1024) * 1024);
- return true;
-}
-
-static char *
-show_WORKSPACE (const struct dataset *ds UNUSED)
-{
- size_t ws = settings_get_workspace () / 1024L;
- return xasprintf ("%zu", ws);
-}
-\f
-static char *
-show_DIRECTORY (const struct dataset *ds UNUSED)
-{
- char *buf = NULL;
- char *wd = NULL;
- size_t len = 256;
-
- do
- {
- len <<= 1;
- buf = xrealloc (buf, len);
- }
- while (NULL == (wd = getcwd (buf, len)));
-
- return wd;
-}
-
-static char *
-show_N (const struct dataset *ds)
-{
- const struct casereader *reader = dataset_source (ds);
- return (reader
- ? xasprintf ("%lld", (long long int) casereader_count_cases (reader))
- : xstrdup (_("Unknown")));
-}
-
-static void
-do_show (const struct dataset *ds, const struct setting *s,
- struct pivot_table **ptp)
-{
- struct pivot_table *pt = *ptp;
- if (!pt)
- {
- pt = *ptp = pivot_table_create (N_("Settings"));
- pivot_dimension_create (pt, PIVOT_AXIS_ROW, N_("Setting"));
- }
-
- struct pivot_value *name = pivot_value_new_user_text (s->name, SIZE_MAX);
- char *text = s->show (ds);
- if (!text)
- text = xstrdup("empty");
- struct pivot_value *value = pivot_value_new_user_text_nocopy (text);
-
- int row = pivot_category_create_leaf (pt->dimensions[0]->root, name);
- pivot_table_put1 (pt, row, value);
-}
-
-static void
-show_warranty (const struct dataset *ds UNUSED)
-{
- fputs (lack_of_warranty, stdout);
-}
-
-static void
-show_copying (const struct dataset *ds UNUSED)
-{
- fputs (copyleft, stdout);
-}
-
-static void
-add_row (struct pivot_table *table, const char *attribute,
- const char *value)
-{
- int row = pivot_category_create_leaf (table->dimensions[0]->root,
- pivot_value_new_text (attribute));
- if (value)
- pivot_table_put1 (table, row, pivot_value_new_user_text (value, -1));
-}
-
-static void
-show_system (const struct dataset *ds UNUSED)
-{
- struct pivot_table *table = pivot_table_create (N_("System Information"));
- pivot_dimension_create (table, PIVOT_AXIS_ROW, N_("Attribute"));
-
- add_row (table, N_("Version"), version);
- add_row (table, N_("Host System"), host_system);
- add_row (table, N_("Build System"), build_system);
- add_row (table, N_("Locale Directory"), relocate (locale_dir));
- add_row (table, N_("Compiler Version"),
-#ifdef __VERSION__
- __VERSION__
-#else
- "Unknown"
-#endif
- );
- pivot_table_submit (table);
-}
-\f
-static const struct setting settings[] = {
- { "BASETEXTDIRECTION", parse_BASETEXTDIRECTION, NULL },
- { "BLANKS", parse_BLANKS, show_BLANKS },
- { "BLOCK", parse_BLOCK, NULL },
- { "BOX", parse_BOX, NULL },
- { "CACHE", parse_CACHE, NULL },
- { "CCA", parse_CCA, show_CCA },
- { "CCB", parse_CCB, show_CCB },
- { "CCC", parse_CCC, show_CCC },
- { "CCD", parse_CCD, show_CCD },
- { "CCE", parse_CCE, show_CCE },
- { "CELLSBREAK", parse_CELLSBREAK, NULL },
- { "CMPTRANS", parse_CMPTRANS, NULL },
- { "COMPRESSION", parse_COMPRESSION, NULL },
- { "CTEMPLATE", parse_CTEMPLATE, NULL },
- { "DECIMAL", parse_DECIMAL, show_DECIMAL },
- { "DIRECTORY", NULL, show_DIRECTORY },
- { "EPOCH", parse_EPOCH, show_EPOCH },
- { "ERRORS", parse_ERRORS, show_ERRORS },
- { "FORMAT", parse_FORMAT, show_FORMAT },
- { "FUZZBITS", parse_FUZZBITS, show_FUZZBITS },
- { "HEADER", parse_HEADER, NULL },
- { "INCLUDE", parse_INCLUDE, show_INCLUDE },
- { "JOURNAL", parse_JOURNAL, show_JOURNAL },
- { "LEADZERO", parse_LEADZERO, show_LEADZERO },
- { "LENGTH", parse_LENGTH, show_LENGTH },
- { "LOCALE", parse_LOCALE, show_LOCALE },
- { "MDISPLAY", parse_MDISPLAY, show_MDISPLAY },
- { "MESSAGES", parse_MESSAGES, show_MESSAGES },
- { "MEXPAND", parse_MEXPAND, show_MEXPAND },
- { "MITERATE", parse_MITERATE, show_MITERATE },
- { "MNEST", parse_MNEST, show_MNEST },
- { "MPRINT", parse_MPRINT, show_MPRINT },
- { "MXERRS", parse_MXERRS, show_MXERRS },
- { "MXLOOPS", parse_MXLOOPS, show_MXLOOPS },
- { "MXWARNS", parse_MXWARNS, show_MXWARNS },
- { "N", NULL, show_N },
- { "PRINTBACK", parse_PRINTBACK, show_PRINTBACK },
- { "RESULTS", parse_RESULTS, show_RESULTS },
- { "RIB", parse_RIB, show_RIB },
- { "RRB", parse_RRB, show_RRB },
- { "SAFER", parse_SAFER, show_SAFER },
- { "SCOMPRESSION", parse_SCOMPRESSION, show_SCOMPRESSION },
- { "SEED", parse_SEED, NULL },
- { "SMALL", parse_SMALL, show_SMALL },
- { "TEMPDIR", NULL, show_TEMPDIR },
- { "TNUMBERS", parse_TNUMBERS, show_TNUMBERS },
- { "TVARS", parse_TVARS, show_TVARS },
- { "TLOOK", parse_TLOOK, NULL },
- { "UNDEFINED", parse_UNDEFINED, show_UNDEFINED },
- { "VERSION", NULL, show_VERSION },
- { "WEIGHT", NULL, show_WEIGHT },
- { "WIB", parse_WIB, show_WIB },
- { "WRB", parse_WRB, show_WRB },
- { "WIDTH", parse_WIDTH, show_WIDTH },
- { "WORKSPACE", parse_WORKSPACE, show_WORKSPACE },
-};
-enum { N_SETTINGS = sizeof settings / sizeof *settings };
-
-static bool
-parse_setting (struct lexer *lexer)
-{
- for (size_t i = 0; i < N_SETTINGS; i++)
- if (settings[i].set && match_subcommand (lexer, settings[i].name))
- return settings[i].set (lexer);
-
- lex_error (lexer, _("Syntax error expecting the name of a setting."));
- return false;
-}
-
-int
-cmd_set (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- for (;;)
- {
- lex_match (lexer, T_SLASH);
- if (lex_token (lexer) == T_ENDCMD)
- break;
-
- if (!parse_setting (lexer))
- return CMD_FAILURE;
- }
-
- return CMD_SUCCESS;
-}
-
-static void
-show_all (const struct dataset *ds, struct pivot_table **ptp)
-{
- for (size_t i = 0; i < sizeof settings / sizeof *settings; i++)
- if (settings[i].show)
- do_show (ds, &settings[i], ptp);
-}
-
-static void
-show_all_cc (const struct dataset *ds, struct pivot_table **ptp)
-{
- for (size_t i = 0; i < sizeof settings / sizeof *settings; i++)
- {
- const struct setting *s = &settings[i];
- if (s->show && !strncmp (s->name, "CC", 2))
- do_show (ds, s, ptp);
- }
-}
-
-int
-cmd_show (struct lexer *lexer, struct dataset *ds)
-{
- struct pivot_table *pt = NULL;
- if (lex_token (lexer) == T_ENDCMD)
- {
- show_all (ds, &pt);
- pivot_table_submit (pt);
- return CMD_SUCCESS;
- }
-
- do
- {
- if (lex_match (lexer, T_ALL))
- show_all (ds, &pt);
- else if (lex_match_id (lexer, "CC"))
- show_all_cc (ds, &pt);
- else if (lex_match_id (lexer, "WARRANTY"))
- show_warranty (ds);
- else if (lex_match_id (lexer, "COPYING") || lex_match_id (lexer, "LICENSE"))
- show_copying (ds);
- else if (lex_match_id (lexer, "SYSTEM"))
- show_system (ds);
- else if (lex_match_id (lexer, "TITLE"))
- {
- struct setting s = { .name = "TITLE", .show = show_TITLE };
- do_show (ds, &s, &pt);
- }
- else if (lex_match_id (lexer, "SUBTITLE"))
- {
- struct setting s = { .name = "SUBTITLE", .show = show_SUBTITLE };
- do_show (ds, &s, &pt);
- }
- else if (lex_token (lexer) == T_ID)
- {
- for (size_t i = 0; i < sizeof settings / sizeof *settings; i++)
- {
- const struct setting *s = &settings[i];
- if (s->show && lex_match_id (lexer, s->name))
- {
- do_show (ds, s, &pt);
- goto found;
- }
- }
- lex_error (lexer, _("Syntax error expecting the name of a setting."));
- return CMD_FAILURE;
-
- found: ;
- }
- else
- {
- lex_error (lexer, _("Syntax error expecting the name of a setting."));
- return CMD_FAILURE;
- }
-
- lex_match (lexer, T_SLASH);
- }
- while (lex_token (lexer) != T_ENDCMD);
-
- if (pt)
- pivot_table_submit (pt);
-
- return CMD_SUCCESS;
-}
-\f
-#define MAX_SAVED_SETTINGS 5
-
-static struct settings *saved_settings[MAX_SAVED_SETTINGS];
-static int n_saved_settings;
-
-int
-cmd_preserve (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- if (n_saved_settings < MAX_SAVED_SETTINGS)
- {
- saved_settings[n_saved_settings++] = settings_get ();
- return CMD_SUCCESS;
- }
- else
- {
- lex_next_error (lexer, -1, -1,
- _("Too many %s commands without a %s: at most "
- "%d levels of saved settings are allowed."),
- "PRESERVE", "RESTORE",
- MAX_SAVED_SETTINGS);
- return CMD_CASCADING_FAILURE;
- }
-}
-
-int
-cmd_restore (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- if (n_saved_settings > 0)
- {
- struct settings *s = saved_settings[--n_saved_settings];
- settings_set (s);
- settings_destroy (s);
- return CMD_SUCCESS;
- }
- else
- {
- lex_next_error (lexer, -1, -1,
- _("%s without matching %s."), "RESTORE", "PRESERVE");
- return CMD_FAILURE;
- }
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2007, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/token.h"
-#include "libpspp/message.h"
-#include "libpspp/start-date.h"
-#include "libpspp/version.h"
-#include "output/driver.h"
-
-#include "gl/c-ctype.h"
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-static int
-parse_title (struct lexer *lexer, void (*set_title) (const char *))
-{
- if (lex_token (lexer) == T_STRING)
- {
- set_title (lex_tokcstr (lexer));
- lex_get (lexer);
- }
- else
- {
- int start_ofs = lex_ofs (lexer);
- while (lex_token (lexer) != T_ENDCMD)
- lex_get (lexer);
-
- /* Get the raw representation of all the tokens, including any space
- between them, and use it as the title. */
- char *title = lex_ofs_representation (lexer, start_ofs,
- lex_ofs (lexer) - 1);
- set_title (title);
- free (title);
- }
- return CMD_SUCCESS;
-}
-
-int
-cmd_title (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- return parse_title (lexer, output_set_title);
-}
-
-int
-cmd_subtitle (struct lexer *lexer, struct dataset *ds UNUSED)
-{
- return parse_title (lexer, output_set_subtitle);
-}
-
-/* Performs the FILE LABEL command. */
-int
-cmd_file_label (struct lexer *lexer, struct dataset *ds)
-{
- if (!lex_force_string (lexer))
- return CMD_FAILURE;
-
- dict_set_label (dataset_dict (ds), lex_tokcstr (lexer));
- lex_get (lexer);
-
- return CMD_SUCCESS;
-}
-
-/* Performs the DOCUMENT command. */
-int
-cmd_document (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- char *trailer;
-
- if (!lex_force_string (lexer))
- return CMD_FAILURE;
-
- while (lex_is_string (lexer))
- {
- dict_add_document_line (dict, lex_tokcstr (lexer), true);
- lex_get (lexer);
- }
-
- trailer = xasprintf (_(" (Entered %s)"), get_start_date ());
- dict_add_document_line (dict, trailer, true);
- free (trailer);
-
- return CMD_SUCCESS;
-}
-
-/* Performs the ADD DOCUMENTS command. */
-int
-cmd_add_documents (struct lexer *lexer, struct dataset *ds)
-{
- return cmd_document (lexer, ds);
-}
-
-/* Performs the DROP DOCUMENTS command. */
-int
-cmd_drop_documents (struct lexer *lexer UNUSED, struct dataset *ds)
-{
- dict_clear_documents (dataset_dict (ds));
- return CMD_SUCCESS;
-}
+++ /dev/null
-# PSPP - a program for statistical analysis.
-# Copyright (C) 2017 Free Software Foundation, Inc.
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-#
-## Process this file with automake to produce Makefile.in -*- makefile -*-
-
-
-language_xforms_sources = \
- src/language/xforms/compute.c \
- src/language/xforms/count.c \
- src/language/xforms/fail.c \
- src/language/xforms/sample.c \
- src/language/xforms/recode.c \
- src/language/xforms/select-if.c
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2014 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <float.h>
-#include <stdint.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "data/vector.h"
-#include "language/command.h"
-#include "language/expressions/public.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-#include "libpspp/misc.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-struct compute_trns;
-struct lvalue;
-
-/* COMPUTE or IF target variable or vector element.
- For a variable, the `variable' member is non-null.
- For a vector element, the `vector' member is non-null. */
-struct lvalue
- {
- struct msg_location *location; /* Syntax for variable or vector. */
-
- struct variable *variable; /* Destination variable. */
- bool is_new_variable; /* Did we create the variable? */
-
- const struct vector *vector; /* Destination vector, if any, or NULL. */
- struct expression *element; /* Destination vector element, or NULL. */
- };
-
-/* Target of a COMPUTE or IF assignment, either a variable or a
- vector element. */
-static struct lvalue *lvalue_parse (struct lexer *lexer, struct dataset *);
-static int lvalue_get_type (const struct lvalue *);
-static bool lvalue_is_vector (const struct lvalue *);
-static void lvalue_finalize (struct lvalue *,
- struct compute_trns *, struct dictionary *);
-static void lvalue_destroy (struct lvalue *, struct dictionary *);
-
-/* COMPUTE and IF transformation. */
-struct compute_trns
- {
- /* Test expression (IF only). */
- struct expression *test; /* Test expression. */
-
- /* Variable lvalue, if variable != NULL. */
- struct variable *variable; /* Destination variable, if any. */
- int width; /* Lvalue string width; 0=numeric. */
-
- /* Vector lvalue, if vector != NULL. */
- const struct vector *vector; /* Destination vector, if any. */
- struct expression *element; /* Destination vector element expr. */
-
- struct msg_location *lvalue_location;
-
- /* Rvalue. */
- struct expression *rvalue; /* Rvalue expression. */
- };
-
-static struct expression *parse_rvalue (struct lexer *lexer,
- const struct lvalue *,
- struct dataset *);
-
-static struct compute_trns *compute_trns_create (void);
-static bool compute_trns_free (void *compute_);
-static const struct trns_class *get_trns_class (const struct lvalue *);
-\f
-/* COMPUTE. */
-
-int
-cmd_compute (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct lvalue *lvalue = NULL;
- struct compute_trns *compute = NULL;
-
- compute = compute_trns_create ();
-
- lvalue = lvalue_parse (lexer, ds);
- if (lvalue == NULL)
- goto fail;
-
- if (!lex_force_match (lexer, T_EQUALS))
- goto fail;
- compute->rvalue = parse_rvalue (lexer, lvalue, ds);
- if (compute->rvalue == NULL)
- goto fail;
-
- add_transformation (ds, get_trns_class (lvalue), compute);
-
- lvalue_finalize (lvalue, compute, dict);
-
- return CMD_SUCCESS;
-
- fail:
- lvalue_destroy (lvalue, dict);
- compute_trns_free (compute);
- return CMD_CASCADING_FAILURE;
-}
-\f
-/* Transformation functions. */
-
-/* Handle COMPUTE or IF with numeric target variable. */
-static enum trns_result
-compute_num (void *compute_, struct ccase **c, casenumber case_num)
-{
- struct compute_trns *compute = compute_;
-
- if (compute->test == NULL
- || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
- {
- *c = case_unshare (*c);
- *case_num_rw (*c, compute->variable)
- = expr_evaluate_num (compute->rvalue, *c, case_num);
- }
-
- return TRNS_CONTINUE;
-}
-
-/* Handle COMPUTE or IF with numeric vector element target
- variable. */
-static enum trns_result
-compute_num_vec (void *compute_, struct ccase **c, casenumber case_num)
-{
- struct compute_trns *compute = compute_;
-
- if (compute->test == NULL
- || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
- {
- double index; /* Index into the vector. */
- int rindx; /* Rounded index value. */
-
- index = expr_evaluate_num (compute->element, *c, case_num);
- rindx = floor (index + EPSILON);
- if (index == SYSMIS
- || rindx < 1 || rindx > vector_get_n_vars (compute->vector))
- {
- if (index == SYSMIS)
- msg_at (SW, compute->lvalue_location,
- _("When executing COMPUTE: SYSMIS is not a valid value "
- "as an index into vector %s."),
- vector_get_name (compute->vector));
- else
- msg_at (SW, compute->lvalue_location,
- _("When executing COMPUTE: %.*g is not a valid value as "
- "an index into vector %s."),
- DBL_DIG + 1, index, vector_get_name (compute->vector));
- return TRNS_CONTINUE;
- }
-
- *c = case_unshare (*c);
- *case_num_rw (*c, vector_get_var (compute->vector, rindx - 1))
- = expr_evaluate_num (compute->rvalue, *c, case_num);
- }
-
- return TRNS_CONTINUE;
-}
-
-/* Handle COMPUTE or IF with string target variable. */
-static enum trns_result
-compute_str (void *compute_, struct ccase **c, casenumber case_num)
-{
- struct compute_trns *compute = compute_;
-
- if (compute->test == NULL
- || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
- {
- char *s;
-
- *c = case_unshare (*c);
- s = CHAR_CAST_BUG (char *, case_str_rw (*c, compute->variable));
- expr_evaluate_str (compute->rvalue, *c, case_num, s, compute->width);
- }
-
- return TRNS_CONTINUE;
-}
-
-/* Handle COMPUTE or IF with string vector element target
- variable. */
-static enum trns_result
-compute_str_vec (void *compute_, struct ccase **c, casenumber case_num)
-{
- struct compute_trns *compute = compute_;
-
- if (compute->test == NULL
- || expr_evaluate_num (compute->test, *c, case_num) == 1.0)
- {
- double index; /* Index into the vector. */
- int rindx; /* Rounded index value. */
- struct variable *vr; /* Variable reference by indexed vector. */
-
- index = expr_evaluate_num (compute->element, *c, case_num);
- rindx = floor (index + EPSILON);
- if (index == SYSMIS)
- {
- msg_at (SW, compute->lvalue_location,
- _("When executing COMPUTE: SYSMIS is not a valid "
- "value as an index into vector %s."),
- vector_get_name (compute->vector));
- return TRNS_CONTINUE;
- }
- else if (rindx < 1 || rindx > vector_get_n_vars (compute->vector))
- {
- msg_at (SW, compute->lvalue_location,
- _("When executing COMPUTE: %.*g is not a valid value as "
- "an index into vector %s."),
- DBL_DIG + 1, index, vector_get_name (compute->vector));
- return TRNS_CONTINUE;
- }
-
- vr = vector_get_var (compute->vector, rindx - 1);
- *c = case_unshare (*c);
- expr_evaluate_str (compute->rvalue, *c, case_num,
- CHAR_CAST_BUG (char *, case_str_rw (*c, vr)),
- var_get_width (vr));
- }
-
- return TRNS_CONTINUE;
-}
-\f
-/* IF. */
-
-int
-cmd_if (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- struct compute_trns *compute = NULL;
- struct lvalue *lvalue = NULL;
-
- compute = compute_trns_create ();
-
- /* Test expression. */
- compute->test = expr_parse_bool (lexer, ds);
- if (compute->test == NULL)
- goto fail;
-
- /* Lvalue variable. */
- lvalue = lvalue_parse (lexer, ds);
- if (lvalue == NULL)
- goto fail;
-
- /* Rvalue expression. */
- if (!lex_force_match (lexer, T_EQUALS))
- goto fail;
- compute->rvalue = parse_rvalue (lexer, lvalue, ds);
- if (compute->rvalue == NULL)
- goto fail;
-
- add_transformation (ds, get_trns_class (lvalue), compute);
-
- lvalue_finalize (lvalue, compute, dict);
-
- return CMD_SUCCESS;
-
- fail:
- lvalue_destroy (lvalue, dict);
- compute_trns_free (compute);
- return CMD_CASCADING_FAILURE;
-}
-\f
-/* Code common to COMPUTE and IF. */
-
-static const struct trns_class *
-get_trns_class (const struct lvalue *lvalue)
-{
- static const struct trns_class classes[2][2] = {
- [false][false] = {
- .name = "COMPUTE",
- .execute = compute_str,
- .destroy = compute_trns_free
- },
- [false][true] = {
- .name = "COMPUTE",
- .execute = compute_str_vec,
- .destroy = compute_trns_free
- },
- [true][false] = {
- .name = "COMPUTE",
- .execute = compute_num,
- .destroy = compute_trns_free
- },
- [true][true] = {
- .name = "COMPUTE",
- .execute = compute_num_vec,
- .destroy = compute_trns_free
- },
- };
-
- bool is_numeric = lvalue_get_type (lvalue) == VAL_NUMERIC;
- bool is_vector = lvalue_is_vector (lvalue);
- return &classes[is_numeric][is_vector];
-}
-
-/* Parses and returns an rvalue expression of the same type as
- LVALUE, or a null pointer on failure. */
-static struct expression *
-parse_rvalue (struct lexer *lexer,
- const struct lvalue *lvalue, struct dataset *ds)
-{
- if (lvalue->is_new_variable)
- return expr_parse_new_variable (lexer, ds, var_get_name (lvalue->variable),
- lvalue->location);
- else
- return expr_parse (lexer, ds, lvalue_get_type (lvalue));
-}
-
-/* Returns a new struct compute_trns after initializing its fields. */
-static struct compute_trns *
-compute_trns_create (void)
-{
- struct compute_trns *compute = xmalloc (sizeof *compute);
- *compute = (struct compute_trns) { .test = NULL };
- return compute;
-}
-
-/* Deletes all the fields in COMPUTE. */
-static bool
-compute_trns_free (void *compute_)
-{
- struct compute_trns *compute = compute_;
-
- if (compute != NULL)
- {
- msg_location_destroy (compute->lvalue_location);
- expr_free (compute->test);
- expr_free (compute->element);
- expr_free (compute->rvalue);
- free (compute);
- }
- return true;
-}
-\f
-/* Parses the target variable or vector element into a new
- `struct lvalue', which is returned. */
-static struct lvalue *
-lvalue_parse (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
-
- struct lvalue *lvalue = xmalloc (sizeof *lvalue);
- *lvalue = (struct lvalue) { .variable = NULL };
-
- if (!lex_force_id (lexer))
- goto lossage;
-
- int start_ofs = lex_ofs (lexer);
- if (lex_next_token (lexer, 1) == T_LPAREN)
- {
- /* Vector. */
- lvalue->vector = dict_lookup_vector (dict, lex_tokcstr (lexer));
- if (lvalue->vector == NULL)
- {
- lex_error (lexer, _("There is no vector named %s."),
- lex_tokcstr (lexer));
- goto lossage;
- }
-
- /* Vector element. */
- lex_get (lexer);
- if (!lex_force_match (lexer, T_LPAREN))
- goto lossage;
- lvalue->element = expr_parse (lexer, ds, VAL_NUMERIC);
- if (lvalue->element == NULL)
- goto lossage;
- if (!lex_force_match (lexer, T_RPAREN))
- goto lossage;
- }
- else
- {
- /* Variable name. */
- const char *var_name = lex_tokcstr (lexer);
- lvalue->variable = dict_lookup_var (dict, var_name);
- if (lvalue->variable == NULL)
- {
- lvalue->variable = dict_create_var_assert (dict, var_name, 0);
- lvalue->is_new_variable = true;
- }
- lex_get (lexer);
- }
- int end_ofs = lex_ofs (lexer) - 1;
- lvalue->location = lex_ofs_location (lexer, start_ofs, end_ofs);
- return lvalue;
-
- lossage:
- lvalue_destroy (lvalue, dict);
- return NULL;
-}
-
-/* Returns the type (NUMERIC or ALPHA) of the target variable or
- vector in LVALUE. */
-static int
-lvalue_get_type (const struct lvalue *lvalue)
-{
- return (lvalue->variable != NULL
- ? var_get_type (lvalue->variable)
- : vector_get_type (lvalue->vector));
-}
-
-/* Returns true if LVALUE has a vector as its target. */
-static bool
-lvalue_is_vector (const struct lvalue *lvalue)
-{
- return lvalue->vector != NULL;
-}
-
-/* Finalizes making LVALUE the target of COMPUTE, by creating the
- target variable if necessary and setting fields in COMPUTE. */
-static void
-lvalue_finalize (struct lvalue *lvalue,
- struct compute_trns *compute,
- struct dictionary *dict)
-{
- compute->lvalue_location = lvalue->location;
- lvalue->location = NULL;
-
- if (lvalue->vector == NULL)
- {
- compute->variable = lvalue->variable;
- compute->width = var_get_width (compute->variable);
-
- /* Goofy behavior, but compatible: Turn off LEAVE. */
- if (!var_must_leave (compute->variable))
- var_set_leave (compute->variable, false);
-
- /* Prevent lvalue_destroy from deleting variable. */
- lvalue->is_new_variable = false;
- }
- else
- {
- compute->vector = lvalue->vector;
- compute->element = lvalue->element;
- lvalue->element = NULL;
- }
-
- lvalue_destroy (lvalue, dict);
-}
-
-/* Destroys LVALUE. */
-static void
-lvalue_destroy (struct lvalue *lvalue, struct dictionary *dict)
-{
- if (lvalue == NULL)
- return;
-
- if (lvalue->is_new_variable)
- dict_delete_var (dict, lvalue->variable);
- expr_free (lvalue->element);
- msg_location_destroy (lvalue->location);
- free (lvalue);
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011, 2015 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/compiler.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* Value or range? */
-enum value_type
- {
- CNT_SINGLE, /* Single value. */
- CNT_RANGE /* a <= x <= b. */
- };
-
-/* Numeric count criteria. */
-struct num_value
- {
- enum value_type type; /* How to interpret a, b. */
- double a, b; /* Values to count. */
- };
-
-struct criteria
- {
- struct criteria *next;
-
- /* Variables to count. */
- const struct variable **vars;
- size_t n_vars;
-
- /* Count special values? */
- bool count_system_missing; /* Count system missing? */
- bool count_user_missing; /* Count user missing? */
-
- /* Criterion values. */
- size_t n_values;
- union
- {
- struct num_value *num;
- char **str;
- }
- values;
- };
-
-struct dst_var
- {
- struct dst_var *next;
- struct variable *var; /* Destination variable. */
- char *name; /* Name of dest var. */
- struct criteria *crit; /* The criteria specifications. */
- };
-
-struct count_trns
- {
- struct dst_var *dst_vars;
- struct pool *pool;
- };
-
-static const struct trns_class count_trns_class;
-
-static bool parse_numeric_criteria (struct lexer *, struct pool *, struct criteria *);
-static bool parse_string_criteria (struct lexer *, struct pool *,
- struct criteria *,
- const char *dict_encoding);
-static bool count_trns_free (void *trns_);
-\f
-int
-cmd_count (struct lexer *lexer, struct dataset *ds)
-{
- struct dst_var *dv; /* Destination var being parsed. */
- struct count_trns *trns; /* Transformation. */
-
- /* Parses each slash-delimited specification. */
- trns = pool_create_container (struct count_trns, pool);
- trns->dst_vars = dv = pool_alloc (trns->pool, sizeof *dv);
- for (;;)
- {
- struct criteria *crit;
-
- /* Initialize this struct dst_var to ensure proper cleanup. */
- dv->next = NULL;
- dv->var = NULL;
- dv->crit = NULL;
-
- /* Get destination variable, or at least its name. */
- if (!lex_force_id (lexer))
- goto fail;
- dv->var = dict_lookup_var (dataset_dict (ds), lex_tokcstr (lexer));
- if (dv->var != NULL)
- {
- if (var_is_alpha (dv->var))
- {
- lex_error (lexer, _("Destination cannot be a string variable."));
- goto fail;
- }
- }
- else
- dv->name = pool_strdup (trns->pool, lex_tokcstr (lexer));
-
- lex_get (lexer);
- if (!lex_force_match (lexer, T_EQUALS))
- goto fail;
-
- crit = dv->crit = pool_alloc (trns->pool, sizeof *crit);
- for (;;)
- {
- struct dictionary *dict = dataset_dict (ds);
- bool ok;
-
- crit->next = NULL;
- crit->vars = NULL;
- if (!parse_variables_const (lexer, dict, &crit->vars,
- &crit->n_vars,
- PV_DUPLICATE | PV_SAME_TYPE))
- goto fail;
- pool_register (trns->pool, free, crit->vars);
-
- if (!lex_force_match (lexer, T_LPAREN))
- goto fail;
-
- crit->n_values = 0;
- if (var_is_numeric (crit->vars[0]))
- ok = parse_numeric_criteria (lexer, trns->pool, crit);
- else
- ok = parse_string_criteria (lexer, trns->pool, crit,
- dict_get_encoding (dict));
- if (!ok)
- goto fail;
-
- if (lex_token (lexer) == T_SLASH || lex_token (lexer) == T_ENDCMD)
- break;
-
- crit = crit->next = pool_alloc (trns->pool, sizeof *crit);
- }
-
- if (lex_token (lexer) == T_ENDCMD)
- break;
-
- if (!lex_force_match (lexer, T_SLASH))
- goto fail;
- dv = dv->next = pool_alloc (trns->pool, sizeof *dv);
- }
-
- /* Create all the nonexistent destination variables. */
- for (dv = trns->dst_vars; dv; dv = dv->next)
- if (dv->var == NULL)
- {
- /* It's valid, though motivationally questionable, to count to
- the same dest var more than once. */
- dv->var = dict_lookup_var (dataset_dict (ds), dv->name);
-
- if (dv->var == NULL)
- dv->var = dict_create_var_assert (dataset_dict (ds), dv->name, 0);
- }
-
- add_transformation (ds, &count_trns_class, trns);
- return CMD_SUCCESS;
-
-fail:
- count_trns_free (trns);
- return CMD_FAILURE;
-}
-
-/* Parses a set of numeric criterion values. Returns success. */
-static bool
-parse_numeric_criteria (struct lexer *lexer, struct pool *pool, struct criteria *crit)
-{
- size_t allocated = 0;
-
- crit->values.num = NULL;
- crit->count_system_missing = false;
- crit->count_user_missing = false;
- for (;;)
- {
- double low, high;
-
- if (lex_match_id (lexer, "SYSMIS"))
- crit->count_system_missing = true;
- else if (lex_match_id (lexer, "MISSING"))
- crit->count_system_missing = crit->count_user_missing = true;
- else if (parse_num_range (lexer, &low, &high, NULL))
- {
- struct num_value *cur;
-
- if (crit->n_values >= allocated)
- crit->values.num = pool_2nrealloc (pool, crit->values.num,
- &allocated,
- sizeof *crit->values.num);
- cur = &crit->values.num[crit->n_values++];
- cur->type = low == high ? CNT_SINGLE : CNT_RANGE;
- cur->a = low;
- cur->b = high;
- }
- else
- return false;
-
- lex_match (lexer, T_COMMA);
- if (lex_match (lexer, T_RPAREN))
- break;
- }
- return true;
-}
-
-/* Parses a set of string criteria values. Returns success. */
-static bool
-parse_string_criteria (struct lexer *lexer, struct pool *pool,
- struct criteria *crit, const char *dict_encoding)
-{
- int len = 0;
- size_t allocated = 0;
- size_t i;
-
- for (i = 0; i < crit->n_vars; i++)
- if (var_get_width (crit->vars[i]) > len)
- len = var_get_width (crit->vars[i]);
-
- crit->values.str = NULL;
- for (;;)
- {
- char **cur;
- char *s;
-
- if (crit->n_values >= allocated)
- crit->values.str = pool_2nrealloc (pool, crit->values.str,
- &allocated,
- sizeof *crit->values.str);
-
- if (!lex_force_string (lexer))
- return false;
-
- s = recode_string (dict_encoding, "UTF-8", lex_tokcstr (lexer),
- ss_length (lex_tokss (lexer)));
-
- cur = &crit->values.str[crit->n_values++];
- *cur = pool_alloc (pool, len + 1);
- str_copy_rpad (*cur, len + 1, s);
- lex_get (lexer);
-
- free (s);
-
- lex_match (lexer, T_COMMA);
- if (lex_match (lexer, T_RPAREN))
- break;
- }
-
- return true;
-}
-\f
-/* Transformation. */
-
-/* Counts the number of values in case C matching CRIT. */
-static int
-count_numeric (struct criteria *crit, const struct ccase *c)
-{
- int counter = 0;
- size_t i;
-
- for (i = 0; i < crit->n_vars; i++)
- {
- double x = case_num (c, crit->vars[i]);
- struct num_value *v;
-
- for (v = crit->values.num; v < crit->values.num + crit->n_values;
- v++)
- if (v->type == CNT_SINGLE ? x == v->a : x >= v->a && x <= v->b)
- {
- counter++;
- break;
- }
-
- if (var_is_num_missing (crit->vars[i], x)
- && (x == SYSMIS
- ? crit->count_system_missing
- : crit->count_user_missing))
- {
- counter++;
- continue;
- }
-
- }
-
- return counter;
-}
-
-/* Counts the number of values in case C matching CRIT. */
-static int
-count_string (struct criteria *crit, const struct ccase *c)
-{
- int counter = 0;
- size_t i;
-
- for (i = 0; i < crit->n_vars; i++)
- {
- char **v;
- for (v = crit->values.str; v < crit->values.str + crit->n_values; v++)
- if (!memcmp (case_str (c, crit->vars[i]), *v,
- var_get_width (crit->vars[i])))
- {
- counter++;
- break;
- }
- }
-
- return counter;
-}
-
-/* Performs the COUNT transformation T on case C. */
-static enum trns_result
-count_trns_proc (void *trns_, struct ccase **c,
- casenumber case_num UNUSED)
-{
- struct count_trns *trns = trns_;
- struct dst_var *dv;
-
- *c = case_unshare (*c);
- for (dv = trns->dst_vars; dv; dv = dv->next)
- {
- struct criteria *crit;
- int counter;
-
- counter = 0;
- for (crit = dv->crit; crit; crit = crit->next)
- if (var_is_numeric (crit->vars[0]))
- counter += count_numeric (crit, *c);
- else
- counter += count_string (crit, *c);
- *case_num_rw (*c, dv->var) = counter;
- }
- return TRNS_CONTINUE;
-}
-
-/* Destroys all dynamic data structures associated with TRNS. */
-static bool
-count_trns_free (void *trns_)
-{
- struct count_trns *trns = trns_;
- pool_destroy (trns->pool);
- return true;
-}
-
-static const struct trns_class count_trns_class = {
- .name = "COUNT",
- .execute = count_trns_proc,
- .destroy = count_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 2007, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/dataset.h"
-#include "data/transformations.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/message.h"
-\f
-/* A transformation which is guaranteed to fail. */
-
-static enum trns_result
-trns_fail (void *x UNUSED, struct ccase **c UNUSED,
- casenumber n UNUSED)
-{
- msg (SE, "DEBUG XFORM FAIL transformation executed");
- return TRNS_ERROR;
-}
-
-int
-cmd_debug_xform_fail (struct lexer *lexer UNUSED, struct dataset *ds)
-{
- static const struct trns_class fail_trns_class = {
- .name = "DEBUG XFORM FAIL",
- .execute = trns_fail
- };
- add_transformation (ds, &fail_trns_class, NULL);
- return CMD_SUCCESS;
-}
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <ctype.h>
-#include <math.h>
-#include <stdlib.h>
-
-#include "data/case.h"
-#include "data/data-in.h"
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/format.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/value-parser.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/assertion.h"
-#include "libpspp/cast.h"
-#include "libpspp/compiler.h"
-#include "libpspp/i18n.h"
-#include "libpspp/message.h"
-#include "libpspp/pool.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-\f
-/* Definitions. */
-
-/* Type of source value for RECODE. */
-enum map_in_type
- {
- MAP_SINGLE, /* Specific value. */
- MAP_RANGE, /* Range of values. */
- MAP_SYSMIS, /* System missing value. */
- MAP_MISSING, /* Any missing value. */
- MAP_ELSE, /* Any value. */
- MAP_CONVERT /* "123" => 123. */
- };
-
-/* Describes input values to be mapped. */
-struct map_in
- {
- enum map_in_type type; /* One of MAP_*. */
- union value x, y; /* Source values. */
- };
-
-/* Describes the value used as output from a mapping. */
-struct map_out
- {
- bool copy_input; /* If true, copy input to output. */
- union value value; /* If copy_input false, recoded value. */
- int width; /* If copy_input false, output value width. */
- int ofs; /* Lexical location. */
- };
-
-/* Describes how to recode a single value or range of values into a
- single value. */
-struct mapping
- {
- struct map_in in; /* Input values. */
- struct map_out out; /* Output value. */
- };
-
-/* RECODE transformation. */
-struct recode_trns
- {
- struct pool *pool;
-
- /* Variable types, for convenience. */
- enum val_type src_type; /* src_vars[*] type. */
- enum val_type dst_type; /* dst_vars[*] type. */
-
- /* Variables. */
- const struct variable **src_vars; /* Source variables. */
- const struct variable **dst_vars; /* Destination variables. */
- const struct dictionary *dst_dict; /* Dictionary of dst_vars */
- char **dst_names; /* Name of dest variables, if they're new. */
- size_t n_vars; /* Number of variables. */
-
- /* Mappings. */
- struct mapping *mappings; /* Value mappings. */
- size_t n_maps; /* Number of mappings. */
- int max_src_width; /* Maximum width of src_vars[*]. */
- int max_dst_width; /* Maximum width of any map_out in mappings. */
- };
-
-static bool parse_src_vars (struct lexer *, struct recode_trns *, const struct dictionary *dict);
-static bool parse_mappings (struct lexer *, struct recode_trns *,
- const char *dict_encoding);
-static bool parse_dst_vars (struct lexer *, struct recode_trns *,
- const struct dictionary *,
- int src_start, int src_end,
- int mappings_start, int mappings_end);
-
-static void add_mapping (struct recode_trns *,
- size_t *map_allocated, const struct map_in *);
-
-static bool parse_map_in (struct lexer *lexer, struct map_in *, struct pool *,
- enum val_type src_type, size_t max_src_width,
- const char *dict_encoding);
-static void set_map_in_str (struct map_in *, struct pool *,
- struct substring, size_t width,
- const char *dict_encoding);
-
-static bool parse_map_out (struct lexer *lexer, struct pool *, struct map_out *);
-static void set_map_out_str (struct map_out *, struct pool *,
- struct substring);
-
-static bool enlarge_dst_widths (struct lexer *, struct recode_trns *,
- int dst_start, int dst_end);
-static void create_dst_vars (struct recode_trns *, struct dictionary *);
-
-static bool recode_trns_free (void *trns_);
-
-static const struct trns_class recode_trns_class;
-\f
-/* Parser. */
-
-static bool
-parse_one_recoding (struct lexer *lexer, struct dataset *ds,
- struct recode_trns *trns)
-{
- struct dictionary *dict = dataset_dict (ds);
-
- /* Parse source variable names,
- then input to output mappings,
- then destination variable names. */
- int src_start = lex_ofs (lexer);
- if (!parse_src_vars (lexer, trns, dict))
- return false;
- int src_end = lex_ofs (lexer) - 1;
-
- int mappings_start = lex_ofs (lexer);
- if (!parse_mappings (lexer, trns, dict_get_encoding (dict)))
- return false;
- int mappings_end = lex_ofs (lexer) - 1;
-
- int dst_start = lex_ofs (lexer);
- if (!parse_dst_vars (lexer, trns, dict,
- src_start, src_end, mappings_start, mappings_end))
- return false;
- int dst_end = lex_ofs (lexer) - 1;
- if (dst_end < dst_start)
- {
- /* There was no target variable syntax, so the target variables are the
- same as the source variables. */
- dst_start = src_start;
- dst_end = src_end;
- }
-
- /* Ensure that all the output strings are at least as wide
- as the widest destination variable. */
- if (trns->dst_type == VAL_STRING
- && !enlarge_dst_widths (lexer, trns, dst_start, dst_end))
- return false;
-
- /* Create destination variables, if needed.
- This must be the final step; otherwise we'd have to
- delete destination variables on failure. */
- trns->dst_dict = dict;
- if (trns->src_vars != trns->dst_vars)
- create_dst_vars (trns, dict);
-
- /* Done. */
- add_transformation (ds, &recode_trns_class, trns);
- return true;
-}
-
-/* Parses the RECODE transformation. */
-int
-cmd_recode (struct lexer *lexer, struct dataset *ds)
-{
- do
- {
- struct pool *pool = pool_create ();
- struct recode_trns *trns = pool_alloc (pool, sizeof *trns);
- *trns = (struct recode_trns) { .pool = pool };
-
- if (!parse_one_recoding (lexer, ds, trns))
- {
- recode_trns_free (trns);
- return CMD_FAILURE;
- }
- }
- while (lex_match (lexer, T_SLASH));
-
- return CMD_SUCCESS;
-}
-
-/* Parses a set of variables to recode into TRNS->src_vars and
- TRNS->n_vars. Sets TRNS->src_type. Returns true if
- successful, false on parse error. */
-static bool
-parse_src_vars (struct lexer *lexer,
- struct recode_trns *trns, const struct dictionary *dict)
-{
- if (!parse_variables_const (lexer, dict, &trns->src_vars, &trns->n_vars,
- PV_SAME_TYPE))
- return false;
- pool_register (trns->pool, free, trns->src_vars);
- trns->src_type = var_get_type (trns->src_vars[0]);
- return true;
-}
-
-/* Parses a set of mappings, which take the form (input=output),
- into TRNS->mappings and TRNS->n_maps. Sets TRNS->dst_type.
- Returns true if successful, false on parse error. */
-static bool
-parse_mappings (struct lexer *lexer, struct recode_trns *trns,
- const char *dict_encoding)
-{
- /* Find length of longest source variable. */
- trns->max_src_width = var_get_width (trns->src_vars[0]);
- for (size_t i = 1; i < trns->n_vars; i++)
- {
- size_t var_width = var_get_width (trns->src_vars[i]);
- if (var_width > trns->max_src_width)
- trns->max_src_width = var_width;
- }
-
- /* Parse the mappings in parentheses. */
- size_t map_allocated = 0;
- bool have_dst_type = false;
- if (!lex_force_match (lexer, T_LPAREN))
- return false;
- do
- {
- enum val_type dst_type;
-
- if (!lex_match_id (lexer, "CONVERT"))
- {
- size_t first_map_idx = trns->n_maps;
-
- /* Parse source specifications. */
- do
- {
- struct map_in in;
-
- if (!parse_map_in (lexer, &in, trns->pool,
- trns->src_type, trns->max_src_width,
- dict_encoding))
- return false;
- add_mapping (trns, &map_allocated, &in);
- lex_match (lexer, T_COMMA);
- }
- while (!lex_match (lexer, T_EQUALS));
-
- struct map_out out;
- if (!parse_map_out (lexer, trns->pool, &out))
- return false;
-
- dst_type = (out.copy_input
- ? trns->src_type
- : val_type_from_width (out.width));
- for (size_t i = first_map_idx; i < trns->n_maps; i++)
- trns->mappings[i].out = out;
- }
- else
- {
- /* Parse CONVERT as a special case. */
- struct map_in in = { .type = MAP_CONVERT };
- add_mapping (trns, &map_allocated, &in);
-
- int ofs = lex_ofs (lexer) - 1;
- trns->mappings[trns->n_maps - 1].out = (struct map_out) {
- .ofs = ofs,
- };
-
- dst_type = VAL_NUMERIC;
- if (trns->src_type != VAL_STRING)
- {
- lex_ofs_error (lexer, ofs, ofs,
- _("CONVERT requires string input values."));
- return false;
- }
- }
- if (have_dst_type && dst_type != trns->dst_type)
- {
- msg (SE, _("Output values must be all numeric or all string."));
-
- assert (trns->n_maps > 1);
- const struct map_out *numeric = &trns->mappings[trns->n_maps - 2].out;
- const struct map_out *string = &trns->mappings[trns->n_maps - 1].out;
-
- if (trns->dst_type == VAL_STRING)
- {
- const struct map_out *tmp = numeric;
- numeric = string;
- string = tmp;
- }
-
- lex_ofs_msg (lexer, SN, numeric->ofs, numeric->ofs,
- _("This output value is numeric."));
- lex_ofs_msg (lexer, SN, string->ofs, string->ofs,
- _("This output value is string."));
- return false;
- }
- trns->dst_type = dst_type;
- have_dst_type = true;
-
- if (!lex_force_match (lexer, T_RPAREN))
- return false;
- }
- while (lex_match (lexer, T_LPAREN));
-
- return true;
-}
-
-/* Parses a mapping input value into IN, allocating memory from
- POOL. The source value type must be provided as SRC_TYPE and,
- if string, the maximum width of a string source variable must
- be provided in MAX_SRC_WIDTH. Returns true if successful,
- false on parse error. */
-static bool
-parse_map_in (struct lexer *lexer, struct map_in *in, struct pool *pool,
- enum val_type src_type, size_t max_src_width,
- const char *dict_encoding)
-{
-
- if (lex_match_id (lexer, "ELSE"))
- *in = (struct map_in) { .type = MAP_ELSE };
- else if (src_type == VAL_NUMERIC)
- {
- if (lex_match_id (lexer, "MISSING"))
- *in = (struct map_in) { .type = MAP_MISSING };
- else if (lex_match_id (lexer, "SYSMIS"))
- *in = (struct map_in) { .type = MAP_SYSMIS };
- else
- {
- double x, y;
- if (!parse_num_range (lexer, &x, &y, NULL))
- return false;
- *in = (struct map_in) {
- .type = x == y ? MAP_SINGLE : MAP_RANGE,
- .x = { .f = x },
- .y = { .f = y },
- };
- }
- }
- else
- {
- if (lex_match_id (lexer, "MISSING"))
- *in = (struct map_in) { .type = MAP_MISSING };
- else if (!lex_force_string (lexer))
- return false;
- else
- {
- set_map_in_str (in, pool, lex_tokss (lexer), max_src_width,
- dict_encoding);
- lex_get (lexer);
- if (lex_match_id (lexer, "THRU"))
- {
- lex_next_error (lexer, -1, -1,
- _("%s is not allowed with string variables."),
- "THRU");
- return false;
- }
- }
- }
-
- return true;
-}
-
-/* Adds IN to the list of mappings in TRNS.
- MAP_ALLOCATED is the current number of allocated mappings,
- which is updated as needed. */
-static void
-add_mapping (struct recode_trns *trns,
- size_t *map_allocated, const struct map_in *in)
-{
- struct mapping *m;
- if (trns->n_maps >= *map_allocated)
- trns->mappings = pool_2nrealloc (trns->pool, trns->mappings,
- map_allocated,
- sizeof *trns->mappings);
- m = &trns->mappings[trns->n_maps++];
- m->in = *in;
-}
-
-/* Sets IN as a string mapping, with STRING as the string,
- allocated from POOL. The string is padded with spaces on the
- right to WIDTH characters long. */
-static void
-set_map_in_str (struct map_in *in, struct pool *pool,
- struct substring string, size_t width,
- const char *dict_encoding)
-{
- *in = (struct map_in) { .type = MAP_SINGLE };
-
- char *s = recode_string (dict_encoding, "UTF-8",
- ss_data (string), ss_length (string));
- value_init_pool (pool, &in->x, width);
- value_copy_buf_rpad (&in->x, width,
- CHAR_CAST (uint8_t *, s), strlen (s), ' ');
- free (s);
-}
-
-/* Parses a mapping output value into OUT, allocating memory from
- POOL. Returns true if successful, false on parse error. */
-static bool
-parse_map_out (struct lexer *lexer, struct pool *pool, struct map_out *out)
-{
- if (lex_is_number (lexer))
- {
- *out = (struct map_out) { .value = { .f = lex_number (lexer) } };
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "SYSMIS"))
- *out = (struct map_out) { .value = { .f = SYSMIS } };
- else if (lex_is_string (lexer))
- {
- set_map_out_str (out, pool, lex_tokss (lexer));
- lex_get (lexer);
- }
- else if (lex_match_id (lexer, "COPY"))
- *out = (struct map_out) { .copy_input = true };
- else
- {
- lex_error (lexer, _("Syntax error expecting output value."));
- return false;
- }
- out->ofs = lex_ofs (lexer) - 1;
- return true;
-}
-
-/* Sets OUT as a string mapping output with the given VALUE. */
-static void
-set_map_out_str (struct map_out *out, struct pool *pool,
- const struct substring value)
-{
- const char *string = ss_data (value);
- size_t length = ss_length (value);
-
- if (length == 0)
- {
- /* A length of 0 will yield a numeric value, which is not
- what we want. */
- string = " ";
- length = 1;
- }
-
- *out = (struct map_out) { .width = length };
- value_init_pool (pool, &out->value, length);
- memcpy (out->value.s, string, length);
-}
-
-/* Parses a set of target variables into TRNS->dst_vars and
- TRNS->dst_names. */
-static bool
-parse_dst_vars (struct lexer *lexer, struct recode_trns *trns,
- const struct dictionary *dict, int src_start, int src_end,
- int mappings_start, int mappings_end)
-{
- int dst_start, dst_end;
- if (lex_match_id (lexer, "INTO"))
- {
- dst_start = lex_ofs (lexer);
- size_t n_names;
- if (!parse_mixed_vars_pool (lexer, dict, trns->pool,
- &trns->dst_names, &n_names,
- PV_NONE))
- return false;
- dst_end = lex_ofs (lexer) - 1;
-
- if (n_names != trns->n_vars)
- {
- msg (SE, _("Source and target variable counts must match."));
- lex_ofs_msg (lexer, SN, src_start, src_end,
- ngettext ("There is %zu source variable.",
- "There are %zu source variables.",
- trns->n_vars),
- trns->n_vars);
- lex_ofs_msg (lexer, SN, dst_start, dst_end,
- ngettext ("There is %zu target variable.",
- "There are %zu target variables.",
- n_names),
- n_names);
- return false;
- }
-
- trns->dst_vars = pool_nalloc (trns->pool,
- trns->n_vars, sizeof *trns->dst_vars);
- for (size_t i = 0; i < trns->n_vars; i++)
- {
- const struct variable *v;
- v = trns->dst_vars[i] = dict_lookup_var (dict, trns->dst_names[i]);
- if (v == NULL && trns->dst_type == VAL_STRING)
- {
- msg (SE, _("All string variables specified on INTO must already "
- "exist. (Use the STRING command to create a string "
- "variable.)"));
- lex_ofs_msg (lexer, SN, dst_start, dst_end,
- _("There is no variable named %s."),
- trns->dst_names[i]);
- return false;
- }
- }
- }
- else
- {
- dst_start = src_start;
- dst_end = src_end;
-
- trns->dst_vars = trns->src_vars;
- if (trns->src_type != trns->dst_type)
- {
- if (trns->src_type == VAL_NUMERIC)
- lex_ofs_error (lexer, mappings_start, mappings_end,
- _("INTO is required with numeric input values "
- "and string output values."));
- else
- lex_ofs_error (lexer, mappings_start, mappings_end,
- _("INTO is required with string input values "
- "and numeric output values."));
- return false;
- }
- }
-
- for (size_t i = 0; i < trns->n_vars; i++)
- {
- const struct variable *v = trns->dst_vars[i];
- if (v && var_get_type (v) != trns->dst_type)
- {
- if (trns->dst_type == VAL_STRING)
- lex_ofs_error (lexer, dst_start, dst_end,
- _("Type mismatch: cannot store string data in "
- "numeric variable %s."), var_get_name (v));
- else
- lex_ofs_error (lexer, dst_start, dst_end,
- _("Type mismatch: cannot store numeric data in "
- "string variable %s."), var_get_name (v));
- return false;
- }
- }
-
- return true;
-}
-
-/* Ensures that all the output values in TRNS are as wide as the
- widest destination variable. */
-static bool
-enlarge_dst_widths (struct lexer *lexer, struct recode_trns *trns,
- int dst_start, int dst_end)
-{
- const struct variable *narrow_var = NULL;
- int min_dst_width = INT_MAX;
- trns->max_dst_width = 0;
-
- for (size_t i = 0; i < trns->n_vars; i++)
- {
- const struct variable *v = trns->dst_vars[i];
- if (var_get_width (v) > trns->max_dst_width)
- trns->max_dst_width = var_get_width (v);
-
- if (var_get_width (v) < min_dst_width)
- {
- min_dst_width = var_get_width (v);
- narrow_var = v;
- }
- }
-
- for (size_t i = 0; i < trns->n_maps; i++)
- {
- struct map_out *out = &trns->mappings[i].out;
- if (!out->copy_input)
- {
- if (out->width > min_dst_width)
- {
- msg (SE, _("At least one target variable is too narrow for "
- "the output values."));
- lex_ofs_msg (lexer, SN, out->ofs, out->ofs,
- _("This recoding output value has width %d."),
- out->width);
- lex_ofs_msg (lexer, SN, dst_start, dst_end,
- _("Target variable %s only has width %d."),
- var_get_name (narrow_var),
- var_get_width (narrow_var));
- return false;
- }
-
- value_resize_pool (trns->pool, &out->value,
- out->width, trns->max_dst_width);
- }
- }
-
- return true;
-}
-
-/* Creates destination variables that don't already exist. */
-static void
-create_dst_vars (struct recode_trns *trns, struct dictionary *dict)
-{
- for (size_t i = 0; i < trns->n_vars; i++)
- {
- const struct variable **var = &trns->dst_vars[i];
- const char *name = trns->dst_names[i];
-
- *var = dict_lookup_var (dict, name);
- if (*var == NULL)
- *var = dict_create_var_assert (dict, name, 0);
- assert (var_get_type (*var) == trns->dst_type);
- }
-}
-\f
-/* Data transformation. */
-
-/* Returns the output mapping in TRNS for an input of VALUE on
- variable V, or a null pointer if there is no mapping. */
-static const struct map_out *
-find_src_numeric (struct recode_trns *trns, double value, const struct variable *v)
-{
- for (struct mapping *m = trns->mappings; m < trns->mappings + trns->n_maps;
- m++)
- {
- const struct map_in *in = &m->in;
- const struct map_out *out = &m->out;
- bool match;
-
- switch (in->type)
- {
- case MAP_SINGLE:
- match = value == in->x.f;
- break;
- case MAP_MISSING:
- match = var_is_num_missing (v, value) != 0;
- break;
- case MAP_RANGE:
- match = value >= in->x.f && value <= in->y.f;
- break;
- case MAP_SYSMIS:
- match = value == SYSMIS;
- break;
- case MAP_ELSE:
- match = true;
- break;
- default:
- NOT_REACHED ();
- }
-
- if (match)
- return out;
- }
-
- return NULL;
-}
-
-/* Returns the output mapping in TRNS for an input of VALUE with
- the given WIDTH, or a null pointer if there is no mapping. */
-static const struct map_out *
-find_src_string (struct recode_trns *trns, const uint8_t *value,
- const struct variable *src_var)
-{
- const char *encoding = dict_get_encoding (trns->dst_dict);
- int width = var_get_width (src_var);
- for (struct mapping *m = trns->mappings; m < trns->mappings + trns->n_maps;
- m++)
- {
- const struct map_in *in = &m->in;
- struct map_out *out = &m->out;
- bool match;
-
- switch (in->type)
- {
- case MAP_SINGLE:
- match = !memcmp (value, in->x.s, width);
- break;
- case MAP_ELSE:
- match = true;
- break;
- case MAP_CONVERT:
- {
- union value uv;
- char *error;
-
- error = data_in (ss_buffer (CHAR_CAST_BUG (char *, value), width),
- C_ENCODING, FMT_F, settings_get_fmt_settings (),
- &uv, 0, encoding);
- match = error == NULL;
- free (error);
-
- out->value.f = uv.f;
- break;
- }
- case MAP_MISSING:
- match = var_is_str_missing (src_var, value) != 0;
- break;
- default:
- NOT_REACHED ();
- }
-
- if (match)
- return out;
- }
-
- return NULL;
-}
-
-/* Performs RECODE transformation. */
-static enum trns_result
-recode_trns_proc (void *trns_, struct ccase **c, casenumber case_idx UNUSED)
-{
- struct recode_trns *trns = trns_;
-
- *c = case_unshare (*c);
- for (size_t i = 0; i < trns->n_vars; i++)
- {
- const struct variable *src_var = trns->src_vars[i];
- const struct variable *dst_var = trns->dst_vars[i];
- const struct map_out *out;
-
- if (trns->src_type == VAL_NUMERIC)
- out = find_src_numeric (trns, case_num (*c, src_var), src_var);
- else
- out = find_src_string (trns, case_str (*c, src_var), src_var);
-
- if (trns->dst_type == VAL_NUMERIC)
- {
- double *dst = case_num_rw (*c, dst_var);
- if (out != NULL)
- *dst = !out->copy_input ? out->value.f : case_num (*c, src_var);
- else if (trns->src_vars != trns->dst_vars)
- *dst = SYSMIS;
- }
- else
- {
- char *dst = CHAR_CAST_BUG (char *, case_str_rw (*c, dst_var));
- if (out != NULL)
- {
- if (!out->copy_input)
- memcpy (dst, out->value.s, var_get_width (dst_var));
- else if (trns->src_vars != trns->dst_vars)
- {
- union value *dst_data = case_data_rw (*c, dst_var);
- const union value *src_data = case_data (*c, src_var);
- value_copy_rpad (dst_data, var_get_width (dst_var),
- src_data, var_get_width (src_var), ' ');
- }
- }
- else if (trns->src_vars != trns->dst_vars)
- memset (dst, ' ', var_get_width (dst_var));
- }
- }
-
- return TRNS_CONTINUE;
-}
-
-/* Frees a RECODE transformation. */
-static bool
-recode_trns_free (void *trns_)
-{
- struct recode_trns *trns = trns_;
- pool_destroy (trns->pool);
- return true;
-}
-
-static const struct trns_class recode_trns_class = {
- .name = "RECODE",
- .execute = recode_trns_proc,
- .destroy = recode_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009-2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <gsl/gsl_rng.h>
-#include <limits.h>
-#include <stdio.h>
-#include <math.h>
-
-#include "data/dataset.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/lexer/lexer.h"
-#include "libpspp/compiler.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-#include "math/random.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* The two different types of samples. */
-enum
- {
- TYPE_A_FROM_B, /* 5 FROM 10 */
- TYPE_FRACTION /* 0.5 */
- };
-
-/* SAMPLE transformation. */
-struct sample_trns
- {
- int type; /* One of TYPE_*. */
- int n, N; /* TYPE_A_FROM_B: n from N. */
- int m, t; /* TYPE_A_FROM_B: # picked so far; # so far. */
- unsigned frac; /* TYPE_FRACTION: a fraction of UINT_MAX. */
- };
-
-static const struct trns_class sample_trns_class;
-
-int
-cmd_sample (struct lexer *lexer, struct dataset *ds)
-{
- struct sample_trns *trns;
-
- int type;
- int a, b;
- unsigned frac;
-
- if (!lex_force_num (lexer))
- return CMD_FAILURE;
- if (!lex_is_integer (lexer))
- {
- unsigned long min = gsl_rng_min (get_rng ());
- unsigned long max = gsl_rng_max (get_rng ());
-
- type = TYPE_FRACTION;
- if (!lex_force_num_range_open (lexer, "SAMPLE", 0, 1))
- return CMD_FAILURE;
-
- frac = lex_tokval (lexer) * (max - min) + min;
- a = b = 0;
- }
- else
- {
- type = TYPE_A_FROM_B;
- a = lex_integer (lexer);
- lex_get (lexer);
- if (!lex_force_match_id (lexer, "FROM"))
- return CMD_FAILURE;
- if (!lex_force_int_range (lexer, "FROM", a + 1, INT_MAX))
- return CMD_FAILURE;
- b = lex_integer (lexer);
- frac = 0;
- }
- lex_get (lexer);
-
- trns = xmalloc (sizeof *trns);
- trns->type = type;
- trns->n = a;
- trns->N = b;
- trns->m = trns->t = 0;
- trns->frac = frac;
- add_transformation (ds, &sample_trns_class, trns);
-
- return CMD_SUCCESS;
-}
-
-/* Executes a SAMPLE transformation. */
-static enum trns_result
-sample_trns_proc (void *t_, struct ccase **c UNUSED,
- casenumber case_num UNUSED)
-{
- struct sample_trns *t = t_;
- double U;
-
- if (t->type == TYPE_FRACTION)
- {
- if (gsl_rng_get (get_rng ()) <= t->frac)
- return TRNS_CONTINUE;
- else
- return TRNS_DROP_CASE;
- }
-
- if (t->m >= t->n)
- return TRNS_DROP_CASE;
-
- U = gsl_rng_uniform (get_rng ());
- if ((t->N - t->t) * U >= t->n - t->m)
- {
- t->t++;
- return TRNS_DROP_CASE;
- }
- else
- {
- t->m++;
- t->t++;
- return TRNS_CONTINUE;
- }
-}
-
-static bool
-sample_trns_free (void *t_)
-{
- struct sample_trns *t = t_;
- free (t);
- return true;
-}
-
-static const struct trns_class sample_trns_class = {
- .name = "SAMPLE",
- .execute = sample_trns_proc,
- .destroy = sample_trns_free,
-};
+++ /dev/null
-/* PSPP - a program for statistical analysis.
- Copyright (C) 1997-9, 2000, 2009, 2010, 2011 Free Software Foundation, Inc.
-
- This program is free software: you can redistribute it and/or modify
- it under the terms of the GNU General Public License as published by
- the Free Software Foundation, either version 3 of the License, or
- (at your option) any later version.
-
- This program is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- GNU General Public License for more details.
-
- You should have received a copy of the GNU General Public License
- along with this program. If not, see <http://www.gnu.org/licenses/>. */
-
-#include <config.h>
-
-#include <stdlib.h>
-
-#include "data/dataset.h"
-#include "data/dictionary.h"
-#include "data/transformations.h"
-#include "data/variable.h"
-#include "language/command.h"
-#include "language/expressions/public.h"
-#include "language/lexer/lexer.h"
-#include "language/lexer/variable-parser.h"
-#include "libpspp/message.h"
-#include "libpspp/str.h"
-
-#include "gl/xalloc.h"
-
-#include "gettext.h"
-#define _(msgid) gettext (msgid)
-
-/* SELECT IF transformation. */
-struct select_if_trns
- {
- struct expression *e; /* Test expression. */
- };
-
-static const struct trns_class select_if_trns_class;
-
-/* Parses the SELECT IF transformation. */
-int
-cmd_select_if (struct lexer *lexer, struct dataset *ds)
-{
- struct expression *e;
- struct select_if_trns *t;
-
- e = expr_parse_bool (lexer, ds);
- if (!e)
- return CMD_CASCADING_FAILURE;
-
- if (lex_token (lexer) != T_ENDCMD)
- {
- expr_free (e);
- lex_error (lexer, _("Syntax error expecting end of command."));
- return CMD_CASCADING_FAILURE;
- }
-
- t = xmalloc (sizeof *t);
- t->e = e;
- add_transformation (ds, &select_if_trns_class, t);
-
- return CMD_SUCCESS;
-}
-
-/* Performs the SELECT IF transformation T on case C. */
-static enum trns_result
-select_if_proc (void *t_, struct ccase **c,
- casenumber case_num)
-{
- struct select_if_trns *t = t_;
- return (expr_evaluate_num (t->e, *c, case_num) == 1.0
- ? TRNS_CONTINUE : TRNS_DROP_CASE);
-}
-
-/* Frees SELECT IF transformation T. */
-static bool
-select_if_free (void *t_)
-{
- struct select_if_trns *t = t_;
- expr_free (t->e);
- free (t);
- return true;
-}
-
-static const struct trns_class select_if_trns_class = {
- .name = "SELECT IF",
- .execute = select_if_proc,
- .destroy = select_if_free,
-};
-
-/* Parses the FILTER command. */
-int
-cmd_filter (struct lexer *lexer, struct dataset *ds)
-{
- struct dictionary *dict = dataset_dict (ds);
- if (lex_match_id (lexer, "OFF"))
- dict_set_filter (dict, NULL);
- else if (lex_match (lexer, T_BY))
- {
- struct variable *v = parse_variable (lexer, dict);
- if (!v)
- return CMD_FAILURE;
-
- if (var_is_alpha (v))
- {
- lex_next_error (lexer, -1, -1,
- _("The filter variable must be numeric."));
- return CMD_FAILURE;
- }
-
- if (dict_class_from_id (var_get_name (v)) == DC_SCRATCH)
- {
- lex_next_error (lexer, -1, -1,
- _("The filter variable may not be scratch."));
- return CMD_FAILURE;
- }
-
- dict_set_filter (dict, v);
- }
- else
- {
- lex_error_expecting (lexer, "OFF", "BY");
- return CMD_FAILURE;
- }
-
- return CMD_SUCCESS;
-}
#include "gl/xalloc.h"
#include "data/variable.h"
#include "data/settings.h"
-#include "language/stats/freq.h"
+#include "language/commands/freq.h"
static int
#include "libpspp/str.h"
#include "output/chart.h"
-#include "language/stats/freq.h"
+#include "language/commands/freq.h"
struct piechart
{
#include "data/case.h"
#include "data/casereader.h"
-#include "language/stats/roc.h"
+#include "language/commands/roc.h"
#include "output/cairo-chart.h"
#include "gettext.h"
#include "output/charts/roc-chart.h"
#include "data/casereader.h"
-#include "language/stats/roc.h"
+#include "language/commands/roc.h"
#include "output/chart-provider.h"
#include "gl/xalloc.h"
#include "psppire-data-store.h"
#include "t-test-options.h"
-#include "src/language/stats/chart-category.h"
-#include "src/language/stats/aggregate.h"
+#include "src/language/commands/chart-category.h"
+#include "src/language/commands/aggregate.h"
const GEnumValue align[1];
const GEnumValue measure[1];
#include "psppire-delimited-text.h"
#include "psppire-text-file.h"
-#include "language/data-io/data-parser.h"
+#include "language/commands/data-parser.h"
#include "libpspp/str.h"
#include "libpspp/string-array.h"
#include "libpspp/i18n.h"
#include "dialog-common.h"
-#include <language/stats/aggregate.h>
+#include <language/commands/aggregate.h>
#include "psppire-var-view.h"
#include "psppire-selector.h"
#include "psppire-dict.h"
#include "libpspp/str.h"
-#include "language/stats/chart-category.h"
+#include "language/commands/chart-category.h"
static void
psppire_dialog_action_barchart_class_init (PsppireDialogActionBarchartClass *class);
tests/data/test-decrypted.spv \
tests/data/test-encrypted.spv \
tests/language/mann-whitney.txt \
- tests/language/data-io/Book1.gnm.unzipped \
- tests/language/data-io/test.ods \
- tests/language/data-io/newone.ods \
- tests/language/data-io/readnames.ods \
- tests/language/stats/nhtsa.sav \
- tests/language/stats/llz.zsav \
+ tests/language/commands/Book1.gnm.unzipped \
+ tests/language/commands/test.ods \
+ tests/language/commands/newone.ods \
+ tests/language/commands/readnames.ods \
+ tests/language/commands/nhtsa.sav \
+ tests/language/commands/llz.zsav \
tests/utilities/regress.spv
CLEANFILES += *.save pspp.* foo*
tests/data/sys-file.at \
tests/data/encrypted-file.at \
tests/language/command.at \
- tests/language/control/define.at \
- tests/language/control/do-if.at \
- tests/language/control/do-repeat.at \
- tests/language/control/loop.at \
- tests/language/control/temporary.at \
- tests/language/data-io/add-files.at \
- tests/language/data-io/data-list.at \
- tests/language/data-io/data-reader.at \
- tests/language/data-io/dataset.at \
- tests/language/data-io/file-handle.at \
- tests/language/data-io/get-data.at \
- tests/language/data-io/get-data-spreadsheet.at \
- tests/language/data-io/get-data-psql.at \
- tests/language/data-io/get-data-txt.at \
- tests/language/data-io/get.at \
- tests/language/data-io/inpt-pgm.at \
- tests/language/data-io/list.at \
- tests/language/data-io/match-files.at \
- tests/language/data-io/matrix-data.at \
- tests/language/data-io/matrix-reader.at \
- tests/language/data-io/mconvert.at \
- tests/language/data-io/print-space.at \
- tests/language/data-io/print.at \
- tests/language/data-io/save.at \
- tests/language/data-io/save-translate.at \
- tests/language/data-io/update.at \
- tests/language/dictionary/attributes.at \
- tests/language/dictionary/apply.at \
- tests/language/dictionary/delete-variables.at \
- tests/language/dictionary/formats.at \
- tests/language/dictionary/leave.at \
- tests/language/dictionary/missing-values.at \
- tests/language/dictionary/mrsets.at \
- tests/language/dictionary/numeric.at \
- tests/language/dictionary/rename-variables.at \
- tests/language/dictionary/sort-variables.at \
- tests/language/dictionary/split-file.at \
- tests/language/dictionary/string.at \
- tests/language/dictionary/sys-file-info.at \
- tests/language/dictionary/value-labels.at \
- tests/language/dictionary/variable-display.at \
- tests/language/dictionary/vector.at \
- tests/language/dictionary/weight.at \
- tests/language/expressions/evaluate.at \
- tests/language/expressions/parse.at \
tests/language/lexer/command-name.at \
tests/language/lexer/lexer.at \
tests/language/lexer/scan.at \
tests/language/lexer/segment.at \
tests/language/lexer/variable-parser.at \
- tests/language/stats/aggregate.at \
- tests/language/stats/autorecode.at \
- tests/language/stats/correlations.at \
- tests/language/stats/crosstabs.at \
- tests/language/stats/ctables.at \
- tests/language/stats/descriptives.at \
- tests/language/stats/examine.at \
- tests/language/stats/graph.at \
- tests/language/stats/factor.at \
- tests/language/stats/flip.at \
- tests/language/stats/frequencies.at \
- tests/language/stats/glm.at \
- tests/language/stats/logistic.at \
- tests/language/stats/matrix.at \
- tests/language/stats/means.at \
- tests/language/stats/npar.at \
- tests/language/stats/oneway.at \
- tests/language/stats/quick-cluster.at \
- tests/language/stats/rank.at \
- tests/language/stats/regression.at \
- tests/language/stats/reliability.at \
- tests/language/stats/roc.at \
- tests/language/stats/sort-cases.at \
- tests/language/stats/t-test.at \
- tests/language/utilities/cache.at \
- tests/language/utilities/cd.at \
- tests/language/utilities/date.at \
- tests/language/utilities/host.at \
- tests/language/utilities/insert.at \
- tests/language/utilities/output.at \
- tests/language/utilities/permissions.at \
- tests/language/utilities/set.at \
- tests/language/utilities/show.at \
- tests/language/utilities/title.at \
- tests/language/xforms/compute.at \
- tests/language/xforms/count.at \
- tests/language/xforms/recode.at \
- tests/language/xforms/sample.at \
- tests/language/xforms/select-if.at \
+ tests/language/expressions/evaluate.at \
+ tests/language/expressions/parse.at \
+ tests/language/commands/add-files.at \
+ tests/language/commands/aggregate.at \
+ tests/language/commands/apply.at \
+ tests/language/commands/attributes.at \
+ tests/language/commands/autorecode.at \
+ tests/language/commands/cache.at \
+ tests/language/commands/cd.at \
+ tests/language/commands/compute.at \
+ tests/language/commands/correlations.at \
+ tests/language/commands/count.at \
+ tests/language/commands/crosstabs.at \
+ tests/language/commands/ctables.at \
+ tests/language/commands/data-list.at \
+ tests/language/commands/data-reader.at \
+ tests/language/commands/dataset.at \
+ tests/language/commands/date.at \
+ tests/language/commands/define.at \
+ tests/language/commands/delete-variables.at \
+ tests/language/commands/descriptives.at \
+ tests/language/commands/do-if.at \
+ tests/language/commands/do-repeat.at \
+ tests/language/commands/examine.at \
+ tests/language/commands/factor.at \
+ tests/language/commands/file-handle.at \
+ tests/language/commands/flip.at \
+ tests/language/commands/formats.at \
+ tests/language/commands/frequencies.at \
+ tests/language/commands/get-data-psql.at \
+ tests/language/commands/get-data-spreadsheet.at \
+ tests/language/commands/get-data-txt.at \
+ tests/language/commands/get-data.at \
+ tests/language/commands/get.at \
+ tests/language/commands/glm.at \
+ tests/language/commands/graph.at \
+ tests/language/commands/host.at \
+ tests/language/commands/inpt-pgm.at \
+ tests/language/commands/insert.at \
+ tests/language/commands/leave.at \
+ tests/language/commands/list.at \
+ tests/language/commands/logistic.at \
+ tests/language/commands/loop.at \
+ tests/language/commands/match-files.at \
+ tests/language/commands/matrix-data.at \
+ tests/language/commands/matrix-reader.at \
+ tests/language/commands/matrix.at \
+ tests/language/commands/mconvert.at \
+ tests/language/commands/means.at \
+ tests/language/commands/missing-values.at \
+ tests/language/commands/mrsets.at \
+ tests/language/commands/npar.at \
+ tests/language/commands/numeric.at \
+ tests/language/commands/oneway.at \
+ tests/language/commands/output.at \
+ tests/language/commands/permissions.at \
+ tests/language/commands/print-space.at \
+ tests/language/commands/print.at \
+ tests/language/commands/quick-cluster.at \
+ tests/language/commands/rank.at \
+ tests/language/commands/recode.at \
+ tests/language/commands/regression.at \
+ tests/language/commands/reliability.at \
+ tests/language/commands/rename-variables.at \
+ tests/language/commands/roc.at \
+ tests/language/commands/sample.at \
+ tests/language/commands/save-translate.at \
+ tests/language/commands/save.at \
+ tests/language/commands/select-if.at \
+ tests/language/commands/set.at \
+ tests/language/commands/show.at \
+ tests/language/commands/sort-cases.at \
+ tests/language/commands/sort-variables.at \
+ tests/language/commands/split-file.at \
+ tests/language/commands/string.at \
+ tests/language/commands/sys-file-info.at \
+ tests/language/commands/t-test.at \
+ tests/language/commands/temporary.at \
+ tests/language/commands/title.at \
+ tests/language/commands/update.at \
+ tests/language/commands/value-labels.at \
+ tests/language/commands/variable-display.at \
+ tests/language/commands/vector.at \
+ tests/language/commands/weight.at \
tests/libpspp/abt.at \
tests/libpspp/bt.at \
tests/libpspp/encoding-guesser.at \
--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>
+<gnm:Workbook xmlns:gnm="http://www.gnumeric.org/v10.dtd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gnumeric.org/v8.xsd">
+ <gnm:Version Epoch="1" Major="6" Minor="3" Full="1.6.3"/>
+ <gnm:Attributes>
+ <gnm:Attribute>
+ <gnm:type>4</gnm:type>
+ <gnm:name>WorkbookView::show_horizontal_scrollbar</gnm:name>
+ <gnm:value>TRUE</gnm:value>
+ </gnm:Attribute>
+ <gnm:Attribute>
+ <gnm:type>4</gnm:type>
+ <gnm:name>WorkbookView::show_vertical_scrollbar</gnm:name>
+ <gnm:value>TRUE</gnm:value>
+ </gnm:Attribute>
+ <gnm:Attribute>
+ <gnm:type>4</gnm:type>
+ <gnm:name>WorkbookView::show_notebook_tabs</gnm:name>
+ <gnm:value>TRUE</gnm:value>
+ </gnm:Attribute>
+ <gnm:Attribute>
+ <gnm:type>4</gnm:type>
+ <gnm:name>WorkbookView::do_auto_completion</gnm:name>
+ <gnm:value>TRUE</gnm:value>
+ </gnm:Attribute>
+ <gnm:Attribute>
+ <gnm:type>4</gnm:type>
+ <gnm:name>WorkbookView::is_protected</gnm:name>
+ <gnm:value>FALSE</gnm:value>
+ </gnm:Attribute>
+ </gnm:Attributes>
+ <gnm:Summary>
+ <gnm:Item>
+ <gnm:name>application</gnm:name>
+ <gnm:val-string>gnumeric</gnm:val-string>
+ </gnm:Item>
+ <gnm:Item>
+ <gnm:name>author</gnm:name>
+ <gnm:val-string>John Darrington</gnm:val-string>
+ </gnm:Item>
+ </gnm:Summary>
+ <gnm:SheetNameIndex>
+ <gnm:SheetName>This</gnm:SheetName>
+ <gnm:SheetName>vars</gnm:SheetName>
+ <gnm:SheetName>That</gnm:SheetName>
+ <gnm:SheetName>Empty</gnm:SheetName>
+ <gnm:SheetName>Blank</gnm:SheetName>
+ </gnm:SheetNameIndex>
+ <gnm:Geometry Width="1278" Height="633"/>
+ <gnm:Sheets>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
+ <gnm:Name>This</gnm:Name>
+ <gnm:MaxCol>9</gnm:MaxCol>
+ <gnm:MaxRow>17</gnm:MaxRow>
+ <gnm:Zoom>1</gnm:Zoom>
+ <gnm:PrintInformation>
+ <gnm:Margins>
+ <gnm:top Points="120" PrefUnit="cm"/>
+ <gnm:bottom Points="120" PrefUnit="cm"/>
+ </gnm:Margins>
+ <gnm:Scale type="percentage" percentage="100"/>
+ <gnm:vcenter value="0"/>
+ <gnm:hcenter value="0"/>
+ <gnm:grid value="0"/>
+ <gnm:even_if_only_styles value="0"/>
+ <gnm:monochrome value="0"/>
+ <gnm:draft value="0"/>
+ <gnm:titles value="0"/>
+ <gnm:order>d_then_r</gnm:order>
+ <gnm:orientation>portrait</gnm:orientation>
+ <gnm:Header Left="" Middle="&[TAB]" Right=""/>
+ <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
+ </gnm:PrintInformation>
+ <gnm:Styles>
+ <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ </gnm:Styles>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="10"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="15"/>
+ <gnm:RowInfo No="15" Unit="9.75" MarginA="0" MarginB="0" HardSize="1"/>
+ <gnm:RowInfo No="17" Unit="12.75" MarginA="0" MarginB="0"/>
+ </gnm:Rows>
+ <gnm:Selections CursorCol="7" CursorRow="7">
+ <gnm:Selection startCol="7" startRow="7" endCol="7" endRow="7"/>
+ </gnm:Selections>
+ <gnm:Cells>
+ <gnm:Cell Col="0" Row="0" ValueType="60">numeral</gnm:Cell>
+ <gnm:Cell Col="1" Row="0" ValueType="60">eng_name</gnm:Cell>
+ <gnm:Cell Col="2" Row="0" ValueType="60">xxx</gnm:Cell>
+ <gnm:Cell Col="0" Row="1" ValueType="30">1</gnm:Cell>
+ <gnm:Cell Col="1" Row="1" ValueType="60">One</gnm:Cell>
+ <gnm:Cell Col="2" Row="1" ValueType="60">Eins</gnm:Cell>
+ <gnm:Cell Col="0" Row="2" ValueType="30">2</gnm:Cell>
+ <gnm:Cell Col="1" Row="2" ValueType="60">Two</gnm:Cell>
+ <gnm:Cell Col="2" Row="2" ValueType="60">Zwei</gnm:Cell>
+ <gnm:Cell Col="0" Row="3" ValueType="30">3</gnm:Cell>
+ <gnm:Cell Col="1" Row="3" ValueType="60">Three</gnm:Cell>
+ <gnm:Cell Col="2" Row="3" ValueType="60">Drei</gnm:Cell>
+ <gnm:Cell Col="2" Row="4" ValueType="60">Vier</gnm:Cell>
+ <gnm:Cell Col="5" Row="6" ValueType="60">XY</gnm:Cell>
+ <gnm:Cell Col="6" Row="6" ValueType="60">xxx</gnm:Cell>
+ <gnm:Cell Col="7" Row="6" ValueType="60">xxxx</gnm:Cell>
+ <gnm:Cell Col="8" Row="6" ValueType="60">xxxx</gnm:Cell>
+ <gnm:Cell Col="5" Row="7" ValueType="60">yyy</gnm:Cell>
+ <gnm:Cell Col="6" Row="7" ValueType="60">V1</gnm:Cell>
+ <gnm:Cell Col="7" Row="7" ValueType="60">V2</gnm:Cell>
+ <gnm:Cell Col="5" Row="8" ValueType="60">yyy</gnm:Cell>
+ <gnm:Cell Col="6" Row="8" ValueType="30">0</gnm:Cell>
+ <gnm:Cell Col="7" Row="8" ValueType="60">fred</gnm:Cell>
+ <gnm:Cell Col="8" Row="8" ValueType="30">20</gnm:Cell>
+ <gnm:Cell Col="9" Row="8" ValueType="60">$$$$</gnm:Cell>
+ <gnm:Cell Col="5" Row="9" ValueType="60">yyy</gnm:Cell>
+ <gnm:Cell Col="6" Row="9" ValueType="30">1</gnm:Cell>
+ <gnm:Cell Col="7" Row="9" ValueType="30">11</gnm:Cell>
+ <gnm:Cell Col="8" Row="9" ValueType="30">21</gnm:Cell>
+ <gnm:Cell Col="9" Row="9" ValueType="60">$$$$</gnm:Cell>
+ <gnm:Cell Col="5" Row="10" ValueType="60">yyyy</gnm:Cell>
+ <gnm:Cell Col="6" Row="10" ValueType="30">2</gnm:Cell>
+ <gnm:Cell Col="7" Row="10" ValueType="60">twelve</gnm:Cell>
+ <gnm:Cell Col="8" Row="10" ValueType="30">22</gnm:Cell>
+ <gnm:Cell Col="9" Row="10" ValueType="60">$$$$</gnm:Cell>
+ <gnm:Cell Col="5" Row="11" ValueType="60">yyyy</gnm:Cell>
+ <gnm:Cell Col="6" Row="11" ValueType="30">3</gnm:Cell>
+ <gnm:Cell Col="7" Row="11" ValueType="30">13</gnm:Cell>
+ <gnm:Cell Col="8" Row="11" ValueType="30">23</gnm:Cell>
+ <gnm:Cell Col="9" Row="11" ValueType="60">$$$$</gnm:Cell>
+ <gnm:Cell Col="1" Row="12" ValueType="60">Eleven</gnm:Cell>
+ <gnm:Cell Col="5" Row="12" ValueType="60">yyyy</gnm:Cell>
+ <gnm:Cell Col="6" Row="12" ValueType="30">4</gnm:Cell>
+ <gnm:Cell Col="7" Row="12" ValueType="30">14</gnm:Cell>
+ <gnm:Cell Col="8" Row="12" ValueType="30">24</gnm:Cell>
+ <gnm:Cell Col="9" Row="12" ValueType="60">$$$$</gnm:Cell>
+ <gnm:Cell Col="5" Row="13" ValueType="60">zzz</gnm:Cell>
+ <gnm:Cell Col="6" Row="13" ValueType="60">zzz</gnm:Cell>
+ <gnm:Cell Col="7" Row="13" ValueType="60">zzz</gnm:Cell>
+ <gnm:Cell Col="8" Row="13" ValueType="60">zzz</gnm:Cell>
+ <gnm:Cell Col="9" Row="13" ValueType="60">zzz</gnm:Cell>
+ <gnm:Cell Col="1" Row="17" ValueType="60">Seventeen</gnm:Cell>
+ </gnm:Cells>
+ <gnm:SheetLayout TopLeft="A1"/>
+ <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
+ </gnm:Sheet>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
+ <gnm:Name>vars</gnm:Name>
+ <gnm:MaxCol>4</gnm:MaxCol>
+ <gnm:MaxRow>8</gnm:MaxRow>
+ <gnm:Zoom>1</gnm:Zoom>
+ <gnm:PrintInformation>
+ <gnm:Margins>
+ <gnm:top Points="120" PrefUnit="cm"/>
+ <gnm:bottom Points="120" PrefUnit="cm"/>
+ </gnm:Margins>
+ <gnm:Scale type="percentage" percentage="100"/>
+ <gnm:vcenter value="0"/>
+ <gnm:hcenter value="0"/>
+ <gnm:grid value="0"/>
+ <gnm:even_if_only_styles value="0"/>
+ <gnm:monochrome value="0"/>
+ <gnm:draft value="0"/>
+ <gnm:titles value="0"/>
+ <gnm:order>d_then_r</gnm:order>
+ <gnm:orientation>portrait</gnm:orientation>
+ <gnm:Header Left="" Middle="&[TAB]" Right=""/>
+ <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
+ </gnm:PrintInformation>
+ <gnm:Styles>
+ <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ </gnm:Styles>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="5"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="5"/>
+ </gnm:Rows>
+ <gnm:Selections CursorCol="0" CursorRow="0">
+ <gnm:Selection startCol="0" startRow="0" endCol="0" endRow="0"/>
+ </gnm:Selections>
+ <gnm:Cells>
+ <gnm:Cell Col="0" Row="0" ValueType="60">1v12</gnm:Cell>
+ <gnm:Cell Col="1" Row="0" ValueType="60">var&x@</gnm:Cell>
+ <gnm:Cell Col="2" Row="0" ValueType="60">a(43)</gnm:Cell>
+ <gnm:Cell Col="3" Row="0" ValueType="60">varx</gnm:Cell>
+ <gnm:Cell Col="4" Row="0" ValueType="60">varx</gnm:Cell>
+ <gnm:Cell Col="0" Row="1" ValueType="30">1</gnm:Cell>
+ <gnm:Cell Col="1" Row="1" ValueType="30">2</gnm:Cell>
+ <gnm:Cell Col="2" Row="1" ValueType="30">23</gnm:Cell>
+ <gnm:Cell Col="3" Row="1" ValueType="30">2</gnm:Cell>
+ <gnm:Cell Col="4" Row="1" ValueType="30">4</gnm:Cell>
+ <gnm:Cell Col="0" Row="2" ValueType="30">3</gnm:Cell>
+ <gnm:Cell Col="1" Row="2" ValueType="30">4</gnm:Cell>
+ <gnm:Cell Col="2" Row="2" ValueType="30">23</gnm:Cell>
+ <gnm:Cell Col="3" Row="2" ValueType="30">3</gnm:Cell>
+ <gnm:Cell Col="4" Row="2" ValueType="30">4</gnm:Cell>
+ <gnm:Cell Col="1" Row="8" ValueType="10"></gnm:Cell>
+ </gnm:Cells>
+ <gnm:SheetLayout TopLeft="A1"/>
+ <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
+ </gnm:Sheet>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
+ <gnm:Name>That</gnm:Name>
+ <gnm:MaxCol>3</gnm:MaxCol>
+ <gnm:MaxRow>4</gnm:MaxRow>
+ <gnm:Zoom>1</gnm:Zoom>
+ <gnm:PrintInformation>
+ <gnm:Margins>
+ <gnm:top Points="120" PrefUnit="cm"/>
+ <gnm:bottom Points="120" PrefUnit="cm"/>
+ </gnm:Margins>
+ <gnm:Scale type="percentage" percentage="100"/>
+ <gnm:vcenter value="0"/>
+ <gnm:hcenter value="0"/>
+ <gnm:grid value="0"/>
+ <gnm:even_if_only_styles value="0"/>
+ <gnm:monochrome value="0"/>
+ <gnm:draft value="0"/>
+ <gnm:titles value="0"/>
+ <gnm:order>d_then_r</gnm:order>
+ <gnm:orientation>portrait</gnm:orientation>
+ <gnm:Header Left="" Middle="&[TAB]" Right=""/>
+ <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
+ </gnm:PrintInformation>
+ <gnm:Styles>
+ <gnm:StyleRegion startCol="0" startRow="4096" endCol="63" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="4" startRow="0" endCol="15" endRow="255">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="0" startRow="16" endCol="3" endRow="255">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="16" startRow="0" endCol="63" endRow="4095">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="64" startRow="0" endCol="255" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="3" startRow="0" endCol="3" endRow="15">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="2" startRow="5" endCol="2" endRow="15">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="2" startRow="1" endCol="2" endRow="4">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="0.00">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="0" startRow="256" endCol="15" endRow="4095">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="0" startRow="0" endCol="1" endRow="15">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ <gnm:StyleRegion startCol="2" startRow="0" endCol="2" endRow="0">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ </gnm:Styles>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="2"/>
+ <gnm:ColInfo No="2" Unit="90.75" MarginA="2" MarginB="2" HardSize="1"/>
+ <gnm:ColInfo No="3" Unit="81" MarginA="2" MarginB="2" HardSize="1"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="5"/>
+ </gnm:Rows>
+ <gnm:Selections CursorCol="0" CursorRow="5">
+ <gnm:Selection startCol="0" startRow="5" endCol="0" endRow="5"/>
+ </gnm:Selections>
+ <gnm:Cells>
+ <gnm:Cell Col="0" Row="0" ValueType="60">name</gnm:Cell>
+ <gnm:Cell Col="1" Row="0" ValueType="60">id</gnm:Cell>
+ <gnm:Cell Col="2" Row="0" ValueType="60">height</gnm:Cell>
+ <gnm:Cell Col="0" Row="1" ValueType="60">fred</gnm:Cell>
+ <gnm:Cell Col="1" Row="1" ValueType="30">0</gnm:Cell>
+ <gnm:Cell Col="2" Row="1" ValueType="40">23.4</gnm:Cell>
+ <gnm:Cell Col="0" Row="2" ValueType="60">bert </gnm:Cell>
+ <gnm:Cell Col="1" Row="2" ValueType="30">1</gnm:Cell>
+ <gnm:Cell Col="2" Row="2" ValueType="40">0.56</gnm:Cell>
+ <gnm:Cell Col="0" Row="3" ValueType="60">charlie</gnm:Cell>
+ <gnm:Cell Col="1" Row="3" ValueType="30">2</gnm:Cell>
+ <gnm:Cell Col="2" Row="3" ValueType="60">n/a</gnm:Cell>
+ <gnm:Cell Col="0" Row="4" ValueType="60">dick</gnm:Cell>
+ <gnm:Cell Col="1" Row="4" ValueType="30">3</gnm:Cell>
+ <gnm:Cell Col="2" Row="4" ValueType="40" ValueFormat="0.00">-34.09</gnm:Cell>
+ </gnm:Cells>
+ <gnm:SheetLayout TopLeft="A1"/>
+ <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
+ </gnm:Sheet>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
+ <gnm:Name>Empty</gnm:Name>
+ <gnm:MaxCol>-1</gnm:MaxCol>
+ <gnm:MaxRow>-1</gnm:MaxRow>
+ <gnm:Zoom>1</gnm:Zoom>
+ <gnm:PrintInformation>
+ <gnm:Margins>
+ <gnm:top Points="120" PrefUnit="cm"/>
+ <gnm:bottom Points="120" PrefUnit="cm"/>
+ </gnm:Margins>
+ <gnm:Scale type="percentage" percentage="100"/>
+ <gnm:vcenter value="0"/>
+ <gnm:hcenter value="0"/>
+ <gnm:grid value="0"/>
+ <gnm:even_if_only_styles value="0"/>
+ <gnm:monochrome value="0"/>
+ <gnm:draft value="0"/>
+ <gnm:titles value="0"/>
+ <gnm:order>d_then_r</gnm:order>
+ <gnm:orientation>portrait</gnm:orientation>
+ <gnm:Header Left="" Middle="&[TAB]" Right=""/>
+ <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
+ </gnm:PrintInformation>
+ <gnm:Styles>
+ <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ </gnm:Styles>
+ <gnm:Cols DefaultSizePts="48"/>
+ <gnm:Rows DefaultSizePts="12.75"/>
+ <gnm:Selections CursorCol="0" CursorRow="0">
+ <gnm:Selection startCol="0" startRow="0" endCol="0" endRow="0"/>
+ </gnm:Selections>
+ <gnm:Cells/>
+ <gnm:SheetLayout TopLeft="A1"/>
+ <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
+ </gnm:Sheet>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
+ <gnm:Name>Blank</gnm:Name>
+ <gnm:MaxCol>3</gnm:MaxCol>
+ <gnm:MaxRow>2</gnm:MaxRow>
+ <gnm:Zoom>1</gnm:Zoom>
+ <gnm:PrintInformation>
+ <gnm:Margins>
+ <gnm:top Points="120" PrefUnit="cm"/>
+ <gnm:bottom Points="120" PrefUnit="cm"/>
+ </gnm:Margins>
+ <gnm:Scale type="percentage" percentage="100"/>
+ <gnm:vcenter value="0"/>
+ <gnm:hcenter value="0"/>
+ <gnm:grid value="0"/>
+ <gnm:even_if_only_styles value="0"/>
+ <gnm:monochrome value="0"/>
+ <gnm:draft value="0"/>
+ <gnm:titles value="0"/>
+ <gnm:order>d_then_r</gnm:order>
+ <gnm:orientation>portrait</gnm:orientation>
+ <gnm:Header Left="" Middle="&[TAB]" Right=""/>
+ <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
+ </gnm:PrintInformation>
+ <gnm:Styles>
+ <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
+ <gnm:StyleBorder>
+ <gnm:Top Style="0"/>
+ <gnm:Bottom Style="0"/>
+ <gnm:Left Style="0"/>
+ <gnm:Right Style="0"/>
+ <gnm:Diagonal Style="0"/>
+ <gnm:Rev-Diagonal Style="0"/>
+ </gnm:StyleBorder>
+ </gnm:Style>
+ </gnm:StyleRegion>
+ </gnm:Styles>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="4"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="3"/>
+ </gnm:Rows>
+ <gnm:Selections CursorCol="3" CursorRow="1">
+ <gnm:Selection startCol="3" startRow="1" endCol="3" endRow="1"/>
+ </gnm:Selections>
+ <gnm:Cells>
+ <gnm:Cell Col="0" Row="0" ValueType="60">vone</gnm:Cell>
+ <gnm:Cell Col="1" Row="0" ValueType="60">vtwo</gnm:Cell>
+ <gnm:Cell Col="2" Row="0" ValueType="60">vthree</gnm:Cell>
+ <gnm:Cell Col="3" Row="0" ValueType="60">v4</gnm:Cell>
+ <gnm:Cell Col="0" Row="1" ValueType="30">1</gnm:Cell>
+ <gnm:Cell Col="1" Row="1" ValueType="30">3</gnm:Cell>
+ <gnm:Cell Col="3" Row="1" ValueType="30">5</gnm:Cell>
+ <gnm:Cell Col="0" Row="2" ValueType="30">2</gnm:Cell>
+ <gnm:Cell Col="1" Row="2" ValueType="30">4</gnm:Cell>
+ <gnm:Cell Col="3" Row="2" ValueType="30">6</gnm:Cell>
+ </gnm:Cells>
+ <gnm:SheetLayout TopLeft="A1"/>
+ <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
+ </gnm:Sheet>
+ </gnm:Sheets>
+ <gnm:UIData SelectedTab="4"/>
+ <gnm:Calculation ManualRecalc="0" EnableIteration="1" MaxIterations="100" IterationTolerance="0.001"/>
+</gnm:Workbook>
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+m4_define([CHECK_ADD_FILES],
+ [AT_SETUP([ADD FILES -- $1 $2 with $3])
+ AT_DATA([a.data], [dnl
+1aB
+8aM
+3aE
+5aG
+0aA
+5aH
+6aI
+7aJ
+2aD
+7aK
+1aC
+7aL
+4aF
+])
+ AT_DATA([b.data], [dnl
+1bN
+3bO
+4bP
+6bQ
+7bR
+9bS
+])
+ m4_if([$2], [sav],
+ [AT_DATA([save-a.sps], [dnl
+DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).
+SAVE OUTFILE='a.sav'.
+])
+ AT_CHECK([pspp -O format=csv save-a.sps])])
+ m4_if([$3], [sav],
+ [AT_DATA([save-b.sps], [dnl
+DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).
+SAVE OUTFILE='b.sav'.
+])
+ AT_CHECK([pspp -O format=csv save-b.sps])])
+ m4_if([$1], [interleave],
+ [m4_pushdef([BY], [[/BY a /FIRST=first /LAST=last]])
+ m4_pushdef([SORT], [[/SORT]])],
+ [m4_pushdef([BY], [])
+ m4_pushdef([SORT], [])])
+ AT_DATA([add-files.sps], [dnl
+m4_if([$2], [sav], [], [DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).])
+m4_if([$3], [sav], [], [DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).])
+ADD FILES
+ m4_if([$2], [sav], [FILE='a.sav'], [FILE=*]) /IN=InA SORT
+ m4_if([$3], [sav], [FILE='b.sav'], [FILE=*]) /IN=InB /RENAME c=d
+ BY[].
+LIST.
+])
+ m4_popdef([BY])
+ m4_popdef([SORT])
+ AT_CHECK([pspp -O format=csv add-files.sps], [0],
+[m4_if([$1], [interleave], [dnl
+Table: Data List
+a,b,c,d,InA,InB,first,last
+0,a,A,,1,0,1,1
+1,a,B,,1,0,1,0
+1,a,C,,1,0,0,0
+1,b,,N,0,1,0,1
+2,a,D,,1,0,1,1
+3,a,E,,1,0,1,0
+3,b,,O,0,1,0,1
+4,a,F,,1,0,1,0
+4,b,,P,0,1,0,1
+5,a,G,,1,0,1,0
+5,a,H,,1,0,0,1
+6,a,I,,1,0,1,0
+6,b,,Q,0,1,0,1
+7,a,J,,1,0,1,0
+7,a,K,,1,0,0,0
+7,a,L,,1,0,0,0
+7,b,,R,0,1,0,1
+8,a,M,,1,0,1,1
+9,b,,S,0,1,1,1
+], [dnl
+Table: Data List
+a,b,c,d,InA,InB
+1,a,B,,1,0
+8,a,M,,1,0
+3,a,E,,1,0
+5,a,G,,1,0
+0,a,A,,1,0
+5,a,H,,1,0
+6,a,I,,1,0
+7,a,J,,1,0
+2,a,D,,1,0
+7,a,K,,1,0
+1,a,C,,1,0
+7,a,L,,1,0
+4,a,F,,1,0
+1,b,,N,0,1
+3,b,,O,0,1
+4,b,,P,0,1
+6,b,,Q,0,1
+7,b,,R,0,1
+9,b,,S,0,1
+])])
+AT_CLEANUP
+])
+
+AT_BANNER([ADD FILES])
+
+CHECK_ADD_FILES([interleave], [sav], [sav])
+CHECK_ADD_FILES([interleave], [sav], [inline])
+CHECK_ADD_FILES([interleave], [inline], [sav])
+CHECK_ADD_FILES([concatenate], [sav], [sav])
+CHECK_ADD_FILES([concatenate], [sav], [inline])
+CHECK_ADD_FILES([concatenate], [inline], [sav])
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([AGGREGATE procedure])
+
+dnl CHECK_AGGREGATE(OUTFILE, SORT, MISSING)
+dnl
+dnl Checks the AGGREGATE procedure with the specified combination of:
+dnl
+dnl - OUTFILE: One of "dataset", "active", or "external" according to
+dnl where AGGREGATE's output should be directed.
+dnl
+dnl - SORT: Either "presorted" or "unsorted" according to whether
+dnl AGGREGATE should received presorted input.
+dnl
+dnl - MISSING: Either "itemwise" or "columnwise" according to the basis
+dnl on which missing values should be eliminated.
+dnl
+m4_define([CHECK_AGGREGATE], [
+ AT_SETUP([AGGREGATE $2 data to $1 file, $3 missing])
+ AT_DATA([aggregate.data],
+ [2 42
+1001
+4 41
+3112
+1112
+2661
+1221
+2771
+1331
+1441
+2881
+1551
+])
+ AT_DATA([aggregate.sps],
+ [DATA LIST NOTABLE FILE='aggregate.data' /G N 1-2 S 3(a) W 4.
+WEIGHT BY w.
+MISSING VALUES n(4) s('4').
+m4_if([$1], [dataset], [DATASET DECLARE aggregate.])
+m4_if([$2], [presorted], [SORT CASES BY g.])
+AGGREGATE dnl
+m4_if([$1], [active], [OUTFILE=*],
+ [$1], [external], [OUTFILE='aggregate.sys'],
+ [outfile=aggregate]) dnl
+m4_if([$3], [columnwise], [/MISSING=COLUMNWISE])
+m4_if([$2], [presorted], [/PRESORTED]) dnl
+ /DOCUMENT
+ /BREAK=g
+ /N = n
+ /NI = n./
+ NU = nu
+ /NUI = nu./
+ NFGT2 = fgt(n, 2)
+ /NFGT2I = fgt.(n, 2)
+ /SFGT2 = fgt(s, '2')
+ /SFGT2I = fgt.(s, '2')
+ /NFIN23 = fin(n, 2, 3)
+ /NFIN23I = fin.(n, 2, 3)
+ /SFIN23 = fin(s, '2', '3')
+ /SFIN23I = fin.(s, '2', '3')
+ /NFLT2 = flt(n, 2)
+ /NFLT2I = flt.(n, 2)
+ /SFLT2 = flt(s, '2')
+ /SFLT2I = flt.(s, '2')
+ /NFIRST = first(n)
+ /NFIRSTI = first.(n)
+ /SFIRST = first(s)
+ /SFIRSTI = first.(s)
+ /NFOUT23 = fout(n, 3, 2)
+ /NFOUT23I = fout.(n, 3, 2)
+ /SFOUT23 = fout(s, '3', '2')
+ /SFOUT23I = fout.(s, '3', '2')
+ /NLAST = last(n)
+ /NLASTI = last.(n)
+ /SLAST = last(s)
+ /SLASTI = last.(s)
+ /NMAX = max(n)
+ /NMAXI = max.(n)
+ /SMAX = max(s)
+ /SMAXI = max.(s)
+ /NMEAN = mean(n)
+ /NMEANI = mean.(n)
+ /NMIN = min(n)
+ /NMINI = min.(n)
+ /SMIN = min(s)
+ /SMINI = min.(s)
+ /NN = n(n)
+ /NNI = n.(n)
+ /SN = n(s)
+ /SNI = n.(s)
+ /NNMISS = nmiss(n)
+ /NNMISSI = nmiss.(n)
+ /SNMISS = nmiss(s)
+ /SNMISSI = nmiss.(s)
+ /NNU = nu(n)
+ /NNUI = nu.(n)
+ /SNU = nu(s)
+ /SNUI = nu.(s)
+ /NNUMISS = numiss(n)
+ /NNUMISSI = numiss.(n)
+ /SNUMISS = numiss(s)
+ /SNUMISSI = numiss.(s)
+ /NPGT2 = pgt(n, 2)
+ /NPGT2I = pgt.(n, 2)
+ /SPGT2 = pgt(s, '2')
+ /SPGT2I = pgt.(s, '2')
+ /NPIN23 = pin(n, 2, 3)
+ /NPIN23I = pin.(n, 2, 3)
+ /SPIN23 = pin(s, '2', '3')
+ /SPIN23I = pin.(s, '2', '3')
+ /NPLT2 = plt(n, 2)
+ /NPLT2I = plt.(n, 2)
+ /SPLT2 = plt(s, '2')
+ /SPLT2I = plt.(s, '2')
+ /NPOUT23 = pout(n, 2, 3)
+ /NPOUT23I = pout.(n, 2, 3)
+ /SPOUT23 = pout(s, '2', '3')
+ /SPOUT23I = pout.(s, '2', '3')
+ /NMEDIAN = median(n)
+ /NMEDIANI = median.(n)
+ /NSD = sd(n)
+ /NSDI = sd.(n)
+ /NSUM = sum(n)
+ /NSUMI = sum.(n).
+m4_if([$1], [external], [GET FILE='aggregate.sys'.],
+ [$1], [dataset], [DATASET ACTIVATE aggregate.])
+LIST.
+])
+ AT_CHECK([pspp -O format=csv aggregate.sps], [0],
+ [m4_if([$3], [itemwise], [dnl
+"aggregate.sps:29.28-29.31: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 29 | /NFOUT23 = fout(n, 3, 2)
+ | ^~~~"
+
+"aggregate.sps:30.30-30.33: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 30 | /NFOUT23I = fout.(n, 3, 2)
+ | ^~~~"
+
+"aggregate.sps:31.28-31.35: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 31 | /SFOUT23 = fout(s, '3', '2')
+ | ^~~~~~~~"
+
+"aggregate.sps:32.30-32.37: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 32 | /SFOUT23I = fout.(s, '3', '2')
+ | ^~~~~~~~"
+
+Table: Data List
+G,N,NI,NU,NUI,NFGT2,NFGT2I,SFGT2,SFGT2I,NFIN23,NFIN23I,SFIN23,SFIN23I,NFLT2,NFLT2I,SFLT2,SFLT2I,NFIRST,NFIRSTI,SFIRST,SFIRSTI,NFOUT23,NFOUT23I,SFOUT23,SFOUT23I,NLAST,NLASTI,SLAST,SLASTI,NMAX,NMAXI,SMAX,SMAXI,NMEAN,NMEANI,NMIN,NMINI,SMIN,SMINI,NN,NNI,SN,SNI,NNMISS,NNMISSI,SNMISS,SNMISSI,NNU,NNUI,SNU,SNUI,NNUMISS,NNUMISSI,SNUMISS,SNUMISSI,NPGT2,NPGT2I,SPGT2,SPGT2I,NPIN23,NPIN23I,SPIN23,SPIN23I,NPLT2,NPLT2I,SPLT2,SPLT2I,NPOUT23,NPOUT23I,SPOUT23,SPOUT23I,NMEDIAN,NMEDIANI,NSD,NSDI,NSUM,NSUMI
+1,7.00,7.00,6,6,.333,.429,.333,.429,.333,.286,.333,.286,.500,.429,.500,.429,0,0,0,0,.667,.714,.667,.714,5,5,5,5,5,5,5,5,2.00,2.29,0,0,0,0,6.00,7.00,6.00,7.00,1.00,.00,1.00,.00,5,6,5,6,1,0,1,0,33.3,42.9,33.3,42.9,33.3,28.6,33.3,28.6,50.0,42.9,50.0,42.9,66.7,71.4,66.7,71.4,1.50,2.00,1.79,1.80,12.00,16.00
+2,5.00,5.00,4,4,1.000,1.000,1.000,1.000,.000,.000,.000,.000,.000,.000,.000,.000,6,6,6,4,1.000,1.000,1.000,1.000,8,8,8,8,8,8,8,8,7.00,7.00,6,6,6,4,3.00,3.00,3.00,5.00,2.00,2.00,2.00,.00,3,3,3,4,1,1,1,0,100.0,100.0,100.0,100.0,.0,.0,.0,.0,.0,.0,.0,.0,100.0,100.0,100.0,100.0,7.00,7.00,1.00,1.00,21.00,21.00
+3,2.00,2.00,1,1,.000,.000,.000,.000,.000,.000,.000,.000,1.000,1.000,1.000,1.000,1,1,1,1,1.000,1.000,1.000,1.000,1,1,1,1,1,1,1,1,1.00,1.00,1,1,1,1,2.00,2.00,2.00,2.00,.00,.00,.00,.00,1,1,1,1,0,0,0,0,.0,.0,.0,.0,.0,.0,.0,.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,1.00,1.00,.00,.00,2.00,2.00
+4,1.00,1.00,1,1,. ,. ,. ,1.000,. ,. ,. ,.000,. ,. ,. ,.000,.,.,,4,. ,. ,. ,1.000,.,.,,4,.,.,,4,. ,. ,.,.,,4,.00,.00,.00,1.00,1.00,1.00,1.00,.00,0,0,0,1,1,1,1,0,. ,. ,. ,100.0,. ,. ,. ,.0,. ,. ,. ,.0,. ,. ,. ,100.0,NaN,NaN,. ,. ,. ,. @&t@
+],
+ [dnl
+"aggregate.sps:29.28-29.31: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 29 | /NFOUT23 = fout(n, 3, 2)
+ | ^~~~"
+
+"aggregate.sps:30.30-30.33: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 30 | /NFOUT23I = fout.(n, 3, 2)
+ | ^~~~"
+
+"aggregate.sps:31.28-31.35: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 31 | /SFOUT23 = fout(s, '3', '2')
+ | ^~~~~~~~"
+
+"aggregate.sps:32.30-32.37: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
+ 32 | /SFOUT23I = fout.(s, '3', '2')
+ | ^~~~~~~~"
+
+Table: Data List
+G,N,NI,NU,NUI,NFGT2,NFGT2I,SFGT2,SFGT2I,NFIN23,NFIN23I,SFIN23,SFIN23I,NFLT2,NFLT2I,SFLT2,SFLT2I,NFIRST,NFIRSTI,SFIRST,SFIRSTI,NFOUT23,NFOUT23I,SFOUT23,SFOUT23I,NLAST,NLASTI,SLAST,SLASTI,NMAX,NMAXI,SMAX,SMAXI,NMEAN,NMEANI,NMIN,NMINI,SMIN,SMINI,NN,NNI,SN,SNI,NNMISS,NNMISSI,SNMISS,SNMISSI,NNU,NNUI,SNU,SNUI,NNUMISS,NNUMISSI,SNUMISS,SNUMISSI,NPGT2,NPGT2I,SPGT2,SPGT2I,NPIN23,NPIN23I,SPIN23,SPIN23I,NPLT2,NPLT2I,SPLT2,SPLT2I,NPOUT23,NPOUT23I,SPOUT23,SPOUT23I,NMEDIAN,NMEDIANI,NSD,NSDI,NSUM,NSUMI
+1,7.00,7.00,6,6,. ,.429,. ,.429,. ,.286,. ,.286,. ,.429,. ,.429,.,0,,0,. ,.714,. ,.714,.,5,,5,.,5,,5,. ,2.29,.,0,,0,6.00,7.00,6.00,7.00,1.00,.00,1.00,.00,5,6,5,6,1,0,1,0,. ,42.9,. ,42.9,. ,28.6,. ,28.6,. ,42.9,. ,42.9,. ,71.4,. ,71.4,. ,2.00,. ,1.80,. ,16.00
+2,5.00,5.00,4,4,. ,. ,. ,1.000,. ,. ,. ,.000,. ,. ,. ,.000,.,.,,4,. ,. ,. ,1.000,.,.,,8,.,.,,8,. ,. ,.,.,,4,3.00,3.00,3.00,5.00,2.00,2.00,2.00,.00,3,3,3,4,1,1,1,0,. ,. ,. ,100.0,. ,. ,. ,.0,. ,. ,. ,.0,. ,. ,. ,100.0,. ,. ,. ,. ,. ,. @&t@
+3,2.00,2.00,1,1,.000,.000,.000,.000,.000,.000,.000,.000,1.000,1.000,1.000,1.000,1,1,1,1,1.000,1.000,1.000,1.000,1,1,1,1,1,1,1,1,1.00,1.00,1,1,1,1,2.00,2.00,2.00,2.00,.00,.00,.00,.00,1,1,1,1,0,0,0,0,.0,.0,.0,.0,.0,.0,.0,.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,1.00,1.00,.00,.00,2.00,2.00
+4,1.00,1.00,1,1,. ,. ,. ,1.000,. ,. ,. ,.000,. ,. ,. ,.000,.,.,,4,. ,. ,. ,1.000,.,.,,4,.,.,,4,. ,. ,.,.,,4,.00,.00,.00,1.00,1.00,1.00,1.00,.00,0,0,0,1,1,1,1,0,. ,. ,. ,100.0,. ,. ,. ,.0,. ,. ,. ,.0,. ,. ,. ,100.0,. ,. ,. ,. ,. ,. @&t@
+])])
+ AT_CLEANUP])
+
+CHECK_AGGREGATE([dataset], [presorted], [itemwise])
+CHECK_AGGREGATE([dataset], [presorted], [columnwise])
+CHECK_AGGREGATE([dataset], [unsorted], [itemwise])
+CHECK_AGGREGATE([dataset], [unsorted], [columnwise])
+CHECK_AGGREGATE([active], [presorted], [itemwise])
+CHECK_AGGREGATE([active], [presorted], [columnwise])
+CHECK_AGGREGATE([active], [unsorted], [itemwise])
+CHECK_AGGREGATE([active], [unsorted], [columnwise])
+CHECK_AGGREGATE([external], [presorted], [itemwise])
+CHECK_AGGREGATE([external], [presorted], [columnwise])
+CHECK_AGGREGATE([external], [unsorted], [itemwise])
+CHECK_AGGREGATE([external], [unsorted], [columnwise])
+
+AT_SETUP([AGGREGATE crash with MAX function])
+AT_DATA([aggregate.sps],
+ [DATA LIST LIST /X (F8.2) Y (a25).
+
+BEGIN DATA.
+87.50 foo
+87.34 bar
+1 bar
+END DATA.
+
+AGGREGATE OUTFILE=* /BREAK=y /X=MAX(x).
+LIST /x y.
+])
+AT_CHECK([pspp -O format=csv aggregate.sps], [0],
+ [Table: Reading free-form data from INLINE.
+Variable,Format
+X,F8.2
+Y,A25
+
+Table: Data List
+X,Y
+87.34,bar
+87.50,foo
+])
+AT_CLEANUP
+
+AT_SETUP([AGGREGATE crash with invalid syntax])
+AT_DATA([aggregate.sps],
+ [INPUT PROGRAM.
+LOOP c=1 TO 20.
+ COMPUTE x=UNIFORM(10)
+ END CASE.
+END LOOP.
+END FILE.
+END INPUT PROGRAM.
+
+AGGREGATE /BREAK=x .
+])
+AT_CHECK([pspp -O format=csv aggregate.sps], [1], [ignore], [])
+AT_CLEANUP
+
+
+AT_SETUP([AGGREGATE mode=addvariables])
+AT_DATA([addvariables.sps],
+ [data list notable list /x * cn * y *.
+begin data.
+1 1 2
+3 2 3
+3 3 4
+5 4 6
+7 5 8
+7 6 9
+7 7 20
+9 8 11
+end data.
+
+aggregate outfile=* mode=addvariables
+ /break = x
+ /sum = sum(y)
+ /mean = mean (y)
+ /median = median (y).
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv addvariables.sps], [0],
+ [Table: Data List
+x,cn,y,sum,mean,median
+1.00,1.00,2.00,2.00,2.00,2.00
+3.00,2.00,3.00,7.00,3.50,3.50
+3.00,3.00,4.00,7.00,3.50,3.50
+5.00,4.00,6.00,6.00,6.00,6.00
+7.00,5.00,8.00,37.00,12.33,9.00
+7.00,6.00,9.00,37.00,12.33,9.00
+7.00,7.00,20.00,37.00,12.33,9.00
+9.00,8.00,11.00,11.00,11.00,11.00
+])
+
+AT_CLEANUP
+
+AT_SETUP([AGGREGATE duplicate variable errors])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='aggregate.sps' ERROR=IGNORE.
+])
+AT_DATA([aggregate.sps], [dnl
+DATA LIST NOTABLE LIST /x.
+AGGREGATE OUTFILE=* /BREAK=x /x=N.
+AGGREGATE OUTFILE=* MODE=ADDVARIABLES /x=N.
+AGGREGATE OUTFILE=* MODE=ADDVARIABLES /y=N /y=N.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"aggregate.sps:2.31: error: AGGREGATE: Variable name x duplicates the name of a break variable.
+ 2 | AGGREGATE OUTFILE=* /BREAK=x /x=N.
+ | ^"
+
+"aggregate.sps:3.40: error: AGGREGATE: Variable name x duplicates the name of a variable in the active file dictionary.
+ 3 | AGGREGATE OUTFILE=* MODE=ADDVARIABLES /x=N.
+ | ^"
+
+"aggregate.sps:4.45: error: AGGREGATE: Duplicate target variable name y.
+ 4 | AGGREGATE OUTFILE=* MODE=ADDVARIABLES /y=N /y=N.
+ | ^"
+])
+AT_CLEANUP
+
+AT_SETUP([AGGREGATE presorted warnings])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='aggregate.sps' ERROR=IGNORE.
+])
+AT_DATA([aggregate.sps], [dnl
+DATA LIST NOTABLE LIST /x.
+AGGREGATE/PRESORTED/BREAK=x(A).
+AGGREGATE/BREAK=x(A).
+AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"aggregate.sps:2.27-2.30: warning: AGGREGATE: When the input data is presorted, specifying sorting directions with (A) or (D) has no effect. Output data will be sorted the same way as the input data.
+ 2 | AGGREGATE/PRESORTED/BREAK=x(A).
+ | ^~~~"
+
+"aggregate.sps:2.11-2.19: note: AGGREGATE: The PRESORTED subcommand state that the input data is presorted.
+ 2 | AGGREGATE/PRESORTED/BREAK=x(A).
+ | ^~~~~~~~~"
+
+"aggregate.sps:2.31: error: AGGREGATE: Syntax error expecting `/'.
+ 2 | AGGREGATE/PRESORTED/BREAK=x(A).
+ | ^"
+
+"aggregate.sps:3.17-3.20: warning: AGGREGATE: When the input data is presorted, specifying sorting directions with (A) or (D) has no effect. Output data will be sorted the same way as the input data.
+ 3 | AGGREGATE/BREAK=x(A).
+ | ^~~~"
+
+aggregate.sps:3: note: AGGREGATE: The input data must be presorted because the OUTFILE subcommand is not specified.
+
+"aggregate.sps:3.21: error: AGGREGATE: Syntax error expecting `/'.
+ 3 | AGGREGATE/BREAK=x(A).
+ | ^"
+
+"aggregate.sps:4.45-4.48: warning: AGGREGATE: When the input data is presorted, specifying sorting directions with (A) or (D) has no effect. Output data will be sorted the same way as the input data.
+ 4 | AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
+ | ^~~~"
+
+"aggregate.sps:4.26-4.37: note: AGGREGATE: ADDVARIABLES implies that the input data is presorted.
+ 4 | AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
+ | ^~~~~~~~~~~~"
+
+"aggregate.sps:4.49: error: AGGREGATE: Syntax error expecting `/'.
+ 4 | AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
+ | ^"
+])
+AT_CLEANUP
+
+AT_SETUP([AGGREGATE - subcommand syntax errors])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='aggregate.sps' ERROR=IGNORE.
+])
+AT_DATA([aggregate.sps], [dnl
+DATA LIST NOTABLE LIST /x.
+AGGREGATE OUTFILE=**.
+AGGREGATE OUTFILE=* MODE=**.
+AGGREGATE /MISSING=**.
+AGGREGATE /BREAK=**.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"aggregate.sps:2.19-2.20: error: AGGREGATE: Syntax error expecting a file name or handle name.
+ 2 | AGGREGATE OUTFILE=**.
+ | ^~"
+
+"aggregate.sps:3.26-3.27: error: AGGREGATE: Syntax error expecting ADDVARIABLES or REPLACE.
+ 3 | AGGREGATE OUTFILE=* MODE=**.
+ | ^~"
+
+"aggregate.sps:4.20-4.21: error: AGGREGATE: Syntax error expecting COLUMNWISE.
+ 4 | AGGREGATE /MISSING=**.
+ | ^~"
+
+"aggregate.sps:5.18-5.19: error: AGGREGATE: Syntax error expecting variable name.
+ 5 | AGGREGATE /BREAK=**.
+ | ^~"
+])
+AT_CLEANUP
+
+AT_SETUP([AGGREGATE - aggregation function syntax errors])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='aggregate.sps' ERROR=IGNORE.
+])
+AT_DATA([aggregate.sps], [dnl
+DATA LIST NOTABLE LIST /x (f8.2) s (a8).
+AGGREGATE **.
+AGGREGATE / **.
+AGGREGATE /y.
+AGGREGATE /y=**.
+AGGREGATE /y=xyzzy.
+AGGREGATE /y=mean.
+AGGREGATE /y=mean(**).
+AGGREGATE /y=fgt(x **).
+AGGREGATE /y=fgt(x 'xyzzy').
+AGGREGATE /y=fgt(s 1).
+AGGREGATE /y=fgt(s x).
+AGGREGATE /y=sum(s).
+AGGREGATE /y=sum(x. /* )
+AGGREGATE /y=min(x, s).
+AGGREGATE /y t=min(x).
+AGGREGATE /y=pin(x, 2, 1).
+AGGREGATE /y=mean(x)**.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"aggregate.sps:2.11-2.12: error: AGGREGATE: Syntax error expecting `/'.
+ 2 | AGGREGATE **.
+ | ^~"
+
+"aggregate.sps:3.13-3.14: error: AGGREGATE: Syntax error expecting variable name.
+ 3 | AGGREGATE / **.
+ | ^~"
+
+"aggregate.sps:4.13: error: AGGREGATE: Syntax error expecting variable name.
+ 4 | AGGREGATE /y.
+ | ^"
+
+"aggregate.sps:5.14-5.15: error: AGGREGATE: Syntax error expecting aggregation function.
+ 5 | AGGREGATE /y=**.
+ | ^~"
+
+"aggregate.sps:6.14-6.18: error: AGGREGATE: Unknown aggregation function xyzzy.
+ 6 | AGGREGATE /y=xyzzy.
+ | ^~~~~"
+
+"aggregate.sps:7.18: error: AGGREGATE: Syntax error expecting `('.
+ 7 | AGGREGATE /y=mean.
+ | ^"
+
+"aggregate.sps:8.19-8.20: error: AGGREGATE: Syntax error expecting variable name.
+ 8 | AGGREGATE /y=mean(**).
+ | ^~"
+
+"aggregate.sps:9.20-9.21: error: AGGREGATE: Missing argument 1 to FGT.
+ 9 | AGGREGATE /y=fgt(x **).
+ | ^~"
+
+aggregate.sps:10: error: AGGREGATE: Arguments to FGT must be of same type as source variables.
+
+"aggregate.sps:10.20-10.26: note: AGGREGATE: The argument is a string.
+ 10 | AGGREGATE /y=fgt(x 'xyzzy').
+ | ^~~~~~~"
+
+"aggregate.sps:10.18: note: AGGREGATE: The variables are numeric.
+ 10 | AGGREGATE /y=fgt(x 'xyzzy').
+ | ^"
+
+aggregate.sps:11: error: AGGREGATE: Arguments to FGT must be of same type as source variables.
+
+"aggregate.sps:11.20: note: AGGREGATE: The argument is numeric.
+ 11 | AGGREGATE /y=fgt(s 1).
+ | ^"
+
+"aggregate.sps:11.18: note: AGGREGATE: The variables have string type.
+ 11 | AGGREGATE /y=fgt(s 1).
+ | ^"
+
+"aggregate.sps:12.20: error: AGGREGATE: s and x are not the same type. All variables in this variable list must be of the same type. x will be omitted from the list.
+ 12 | AGGREGATE /y=fgt(s x).
+ | ^"
+
+"aggregate.sps:12.21: error: AGGREGATE: Missing argument 1 to FGT.
+ 12 | AGGREGATE /y=fgt(s x).
+ | ^"
+
+"aggregate.sps:13.18: warning: AGGREGATE: s is not a numeric variable. It will not be included in the variable list.
+ 13 | AGGREGATE /y=sum(s).
+ | ^"
+
+"aggregate.sps:14.19: error: AGGREGATE: Syntax error expecting `)'.
+ 14 | AGGREGATE /y=sum(x. /* )
+ | ^"
+
+aggregate.sps:15: error: AGGREGATE: Number of source variables (2) does not match number of target variables (1).
+
+"aggregate.sps:15.18-15.21: note: AGGREGATE: These are the source variables.
+ 15 | AGGREGATE /y=min(x, s).
+ | ^~~~"
+
+"aggregate.sps:15.12: note: AGGREGATE: These are the target variables.
+ 15 | AGGREGATE /y=min(x, s).
+ | ^"
+
+aggregate.sps:16: error: AGGREGATE: Number of source variables (1) does not match number of target variables (2).
+
+"aggregate.sps:16.20: note: AGGREGATE: These are the source variables.
+ 16 | AGGREGATE /y t=min(x).
+ | ^"
+
+"aggregate.sps:16.12-16.14: note: AGGREGATE: These are the target variables.
+ 16 | AGGREGATE /y t=min(x).
+ | ^~~"
+
+"aggregate.sps:17.21-17.24: warning: AGGREGATE: The value arguments passed to the PIN function are out of order. They will be treated as if they had been specified in the correct order.
+ 17 | AGGREGATE /y=pin(x, 2, 1).
+ | ^~~~"
+
+"aggregate.sps:18.1-18.9: error: AGGREGATE: Syntax error expecting `BEGIN DATA'.
+ 18 | AGGREGATE /y=mean(x)**.
+ | ^~~~~~~~~"
+
+"aggregate.sps:18.1-18.9: error: AGGREGATE: Syntax error expecting end of command.
+ 18 | AGGREGATE /y=mean(x)**.
+ | ^~~~~~~~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2019 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([APPLY DICTIONARY])
+
+AT_SETUP([APPLY DICTIONARY])
+AT_DATA([apply-dict.sps], [dnl
+data list notable list /foo (TIME22.0) bar (a22).
+begin data
+end data.
+Variable label foo "This is a label".
+save outfile='ugg.sav'.
+
+new file.
+data list notable list /foo bar *.
+begin data
+end data.
+display dictionary.
+apply dictionary from = 'ugg.sav'.
+display dictionary.
+])
+
+AT_CHECK([pspp -O format=csv apply-dict.sps], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+foo,1,Nominal,Input,8,Right,F8.2,F8.2
+bar,2,Nominal,Input,8,Right,F8.2,F8.2
+
+"apply-dict.sps:12: warning: APPLY DICTIONARY: Variable bar is numeric in target file, but string in source file."
+
+Table: Variables
+Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+foo,1,This is a label,Nominal,Input,8,Right,TIME22.0,TIME22.0
+bar,2,,Nominal,Input,8,Right,F8.2,F8.2
+])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([VARIABLE ATTRIBUTES and DATAFILE ATTRIBUTES])
+
+AT_SETUP([VARIABLE ATTRIBUTES and DATAFILE ATTRIBUTES])
+AT_DATA([save-attrs.pspp],
+ [[DATA LIST FREE/a b c.
+BEGIN DATA.
+1 2 3
+END DATA.
+
+DATAFILE ATTRIBUTE
+ ATTRIBUTE=key('value')
+ array('array element 1')
+ Array[2]('array element 2').
+VARIABLE ATTRIBUTE
+ VARIABLES=a b
+ ATTRIBUTE=ValidationRule[2]("a + b > 2")
+ ValidationRule[1]('a * b > 3')
+ /VARIABLES=c
+ ATTRIBUTE=QuestionWording('X or Y?').
+DISPLAY ATTRIBUTES.
+
+SAVE OUTFILE='attributes.sav'.
+]])
+AT_DATA([get-attrs.pspp],
+ [[GET FILE='attributes.sav'.
+
+DATAFILE ATTRIBUTE
+ DELETE=Array[1] Array[2].
+VARIABLE ATTRIBUTE
+ VARIABLES=a
+ DELETE=ValidationRule
+ /VARIABLE=b
+ DELETE=validationrule[2].
+
+DISPLAY ATTRIBUTES.
+]])
+AT_CHECK([pspp -O format=csv save-attrs.pspp], [0],
+ [[Table: Variable and Dataset Attributes
+Variable and Name,,Value
+(dataset),array[1],array element 1
+,array[2],array element 2
+,key,value
+a,ValidationRule[1],a * b > 3
+,ValidationRule[2],a + b > 2
+b,ValidationRule[1],a * b > 3
+,ValidationRule[2],a + b > 2
+c,QuestionWording,X or Y?
+]])
+AT_CHECK([pspp -O format=csv get-attrs.pspp], [0], [dnl
+Table: Variable and Dataset Attributes
+Variable and Name,,Value
+(dataset),array,array element 2
+,key,value
+b,ValidationRule,a * b > 3
+c,QuestionWording,X or Y?
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([AUTORECODE procedure])
+
+AT_SETUP([AUTORECODE multiple missing values])
+AT_DATA([autorecode.sps],
+ [DATA LIST LIST NOTABLE /u v w x y z (F2.0).
+BEGIN DATA.
+11 11 11 11 11 11
+12 12 12 12 12 12
+13 13 13 13 13 13
+14 14 14 14 14 14
+15 15 15 15 15 15
+16 16 16 16 16 16
+END DATA.
+
+MISSING VALUES u (11)
+ v (11, 12)
+ w (11, 12, 13)
+ x (11 THRU 14)
+ y (11 THRU 15)
+ z (11 THRU 16).
+
+AUTORECODE u v w x y z INTO a b c d e f/print.
+LIST.
+DISPLAY VARIABLES/VARIABLES=a TO f.
+])
+AT_CHECK([pspp -O format=csv autorecode.sps], [0], [dnl
+Table: Recoding u into a.
+Old Value,New Value,Value Label
+12,1,12
+13,2,13
+14,3,14
+15,4,15
+16,5,16
+11,6,11
+
+Table: Recoding v into b.
+Old Value,New Value,Value Label
+13,1,13
+14,2,14
+15,3,15
+16,4,16
+11,5,11
+12,6,12
+
+Table: Recoding w into c.
+Old Value,New Value,Value Label
+14,1,14
+15,2,15
+16,3,16
+11,4,11
+12,5,12
+13,6,13
+
+Table: Recoding x into d.
+Old Value,New Value,Value Label
+15,1,15
+16,2,16
+11,3,11
+12,4,12
+13,5,13
+14,6,14
+
+Table: Recoding y into e.
+Old Value,New Value,Value Label
+16,1,16
+11,2,11
+12,3,12
+13,4,13
+14,5,14
+15,6,15
+
+Table: Recoding z into f.
+Old Value,New Value,Value Label
+11,1,11
+12,2,12
+13,3,13
+14,4,14
+15,5,15
+16,6,16
+
+Table: Data List
+u,v,w,x,y,z,a,b,c,d,e,f
+11,11,11,11,11,11,6,5,4,3,2,1
+12,12,12,12,12,12,1,6,5,4,3,2
+13,13,13,13,13,13,2,1,6,5,4,3
+14,14,14,14,14,14,3,2,1,6,5,4
+15,15,15,15,15,15,4,3,2,1,6,5
+16,16,16,16,16,16,5,4,3,2,1,6
+
+Table: Variables
+Name,Position,Print Format,Write Format,Missing Values
+a,7,F1.0,F1.0,6
+b,8,F1.0,F1.0,5; 6
+c,9,F1.0,F1.0,4; 5; 6
+d,10,F1.0,F1.0,3 THRU 6
+e,11,F1.0,F1.0,2 THRU 6
+f,12,F1.0,F1.0,1 THRU 6
+])
+AT_CLEANUP
+
+AT_SETUP([AUTORECODE numbers and short strings])
+AT_DATA([autorecode.sps],
+ [data list /X 1-5(a) Y 7.
+begin data.
+lasdj 1
+asdfk 0
+asdfj 2
+asdfj 1
+asdfk 2
+asdfj 9
+lajks 9
+asdfk 0
+asdfk 1
+end data.
+
+missing values x('asdfk') y(9).
+
+autorecode x y into A B/descend/print.
+
+list.
+compute Z=trunc(y/2).
+formats z(F1.0).
+autorecode z into W.
+list.
+])
+AT_CHECK([pspp -O format=csv autorecode.sps], [0],
+ [Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+X,1,1-5,A5
+Y,1,7-7,F1.0
+
+Table: Recoding X into A.
+Old Value,New Value,Value Label
+lasdj,1,lasdj
+lajks,2,lajks
+asdfj,3,asdfj
+asdfk,4,asdfk
+
+Table: Recoding Y into B.
+Old Value,New Value,Value Label
+2,1,2
+1,2,1
+0,3,0
+9,4,9
+
+Table: Data List
+X,Y,A,B
+lasdj,1,1,2
+asdfk,0,4,3
+asdfj,2,3,1
+asdfj,1,3,2
+asdfk,2,4,1
+asdfj,9,3,4
+lajks,9,2,4
+asdfk,0,4,3
+asdfk,1,4,2
+
+Table: Data List
+X,Y,A,B,Z,W
+lasdj,1,1,2,0,1
+asdfk,0,4,3,0,1
+asdfj,2,3,1,1,2
+asdfj,1,3,2,0,1
+asdfk,2,4,1,1,2
+asdfj,9,3,4,.,.
+lajks,9,2,4,.,.
+asdfk,0,4,3,0,1
+asdfk,1,4,2,0,1
+])
+AT_CLEANUP
+
+AT_SETUP([AUTORECODE long strings and check the value labels])
+AT_DATA([ar.sps],
+ [data list notable list /s (a16) x (f1.0).
+begin data.
+widgets 1
+thingummies 2
+oojars 3
+widgets 4
+oojars 5
+thingummies 6
+oojimiflips 7
+end data.
+
+variable labels s 'tracking my stuff'.
+value labels /s 'thingummies' 'Funny sticky things'.
+
+autorecode s into new/print.
+
+list.
+
+display dictionary/variables=new.
+])
+
+AT_CHECK([pspp -O format=csv ar.sps], [0],
+ [Table: Recoding s into new (tracking my stuff).
+Old Value,New Value,Value Label
+oojars,1,oojars
+oojimiflips,2,oojimiflips
+thingummies,3,Funny sticky things
+widgets,4,widgets
+
+Table: Data List
+s,x,new
+widgets,1,4
+thingummies,2,3
+oojars,3,1
+widgets,4,4
+oojars,5,1
+thingummies,6,3
+oojimiflips,7,2
+
+Table: Variables
+Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+new,3,tracking my stuff,Nominal,Input,8,Right,F1.0,F1.0
+
+Table: Value Labels
+Variable Value,,Label
+tracking my stuff,1,oojars
+,2,oojimiflips
+,3,Funny sticky things
+,4,widgets
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([AUTORECODE group subcommand])
+AT_DATA([ar-group.sps],
+[data list notable list /x y (f8.0).
+begin data.
+11 10
+12 12
+13 15
+14 11
+15 12
+16 18
+end data.
+
+missing values y (12).
+
+autorecode
+ x y into a b
+ /group
+ /print.
+
+list.
+display variables /variables=a b.
+])
+
+AT_CHECK([pspp -O format=csv ar-group.sps], [0],
+[Table: Recoding grouped variables.
+Old Value,New Value,Value Label
+10,1,10
+11,2,11
+13,3,13
+14,4,14
+15,5,15
+16,6,16
+18,7,18
+12,8,12
+
+Table: Data List
+x,y,a,b
+11,10,2,1
+12,12,8,8
+13,15,3,5
+14,11,4,2
+15,12,5,8
+16,18,6,7
+
+Table: Variables
+Name,Position,Print Format,Write Format,Missing Values
+a,3,F1.0,F1.0,8
+b,4,F1.0,F1.0,8
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([AUTORECODE group - string variables])
+AT_DATA([strings.sps],
+[data list notable list /x (a8) y (a16).
+begin data.
+fred bert
+charlie " "
+delta echo
+" " windows
+" " nothing
+end data.
+
+
+autorecode x y into a b
+ /group
+ /print.
+
+delete variables x y.
+
+list.
+
+])
+
+AT_CHECK([pspp -O format=csv strings.sps], [0],
+[Table: Recoding grouped variables.
+Old Value,New Value,Value Label
+,1,
+bert,2,bert
+charlie,3,charlie
+delta,4,delta
+echo,5,echo
+fred,6,fred
+nothing,7,nothing
+windows,8,windows
+
+Table: Data List
+a,b
+6,2
+3,1
+4,5
+1,8
+1,7
+])
+
+AT_CLEANUP
+
+
+dnl Tests for a crash which happened when the /GROUP subcommand
+dnl appeared with string variables of different widths.
+AT_SETUP([AUTORECODE group vs. strings])
+AT_DATA([ar-strings.sps],
+ [data list notable list /a (a12) b (a6).
+begin data.
+one nine
+two ten
+three eleven
+four nought
+end data.
+
+autorecode a b into x y
+ /group
+ /print.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv ar-strings.sps], [0], [dnl
+Table: Recoding grouped variables.
+Old Value,New Value,Value Label
+eleven,1,eleven
+four,2,four
+nine,3,nine
+nought,4,nought
+one,5,one
+ten,6,ten
+three,7,three
+two,8,two
+
+Table: Data List
+a,b,x,y
+one,nine,5,3
+two,ten,8,6
+three,eleven,7,1
+four,nought,2,4
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([AUTORECODE /blank])
+
+AT_DATA([auto-blank.sps], [dnl
+data list notable list /x (a8) y (f8.0) z (a16).
+begin data.
+one 2 fred
+two 4 ""
+"" 4 fred
+"" 2 charliebrown
+three 2 charliebrown
+end data.
+
+autorecode variables x y z into a b c /blank=missing /print.
+
+list a b c y.
+])
+
+AT_CHECK([pspp -O format=csv auto-blank.sps], [0], [dnl
+Table: Recoding x into a.
+Old Value,New Value,Value Label
+one,1,one
+three,2,three
+two,3,two
+
+Table: Recoding y into b.
+Old Value,New Value,Value Label
+2,1,2
+4,2,4
+
+Table: Recoding z into c.
+Old Value,New Value,Value Label
+charliebrown,1,charliebrown
+fred,2,fred
+
+Table: Data List
+a,b,c,y
+1,1,2,2
+3,2,.,4
+.,2,2,4
+.,1,1,2
+2,1,1,2
+])
+
+AT_CLEANUP
+
+dnl AUTORECODE had a use-after-free error when TEMPORARY was in use.
+dnl Bug #32757.
+AT_SETUP([AUTORECODE with TEMPORARY])
+AT_DATA([autorecode.sps],
+ [data list /X 1-5(a) Y 7.
+begin data.
+lasdj 1
+asdfk 0
+asdfj 2
+asdfj 1
+asdfk 2
+asdfj 9
+lajks 9
+asdfk 0
+asdfk 1
+end data.
+
+temporary.
+select if y > 1.
+autorecode x y into A B/descend/print.
+list.
+])
+AT_CHECK([pspp -O format=csv autorecode.sps], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+X,1,1-5,A5
+Y,1,7-7,F1.0
+
+Table: Recoding X into A.
+Old Value,New Value,Value Label
+lajks,1,lajks
+asdfk,2,asdfk
+asdfj,3,asdfj
+
+Table: Recoding Y into B.
+Old Value,New Value,Value Label
+9,1,9
+2,2,2
+
+Table: Data List
+X,Y,A,B
+lasdj,1,.,.
+asdfk,0,2,.
+asdfj,2,3,2
+asdfj,1,3,.
+asdfk,2,2,2
+asdfj,9,3,1
+lajks,9,1,1
+asdfk,0,2,.
+asdfk,1,2,.
+])
+AT_CLEANUP
+
+
+dnl For compatibility, make sure that /INTO (with leading slash) is accepted
+dnl (bug #48762)
+AT_SETUP([AUTORECODE with /INTO])
+AT_DATA([autorecode.sps],
+ [data list list notable /x (f8.0).
+begin data.
+1
+8
+-901
+4
+1
+99
+8
+end data.
+
+autorecode x /into y /print.
+
+list.
+])
+AT_CHECK([pspp -O format=csv autorecode.sps], [0],
+[Table: Recoding x into y.
+Old Value,New Value,Value Label
+-901,1,-901
+1,2,1
+4,3,4
+8,4,8
+99,5,99
+
+Table: Data List
+x,y
+1,2
+8,4
+-901,1
+4,3
+1,2
+99,5
+8,4
+])
+AT_CLEANUP
+
+
+AT_SETUP([AUTORECODE with /BLANK without specifier])
+
+AT_DATA([autorecode.sps], [data list notable list /x (a18).
+begin data
+one
+two
+three
+end data.
+
+* /BLANK should be either =MISSING or =VALID
+autorecode x /into y
+ /blank
+
+execute.
+])
+
+AT_CHECK([pspp -O format=csv autorecode.sps], [1], [ignore])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([CACHE])
+
+AT_SETUP([CACHE])
+AT_DATA([cache.sps], [dnl
+CACHE.
+])
+AT_CHECK([pspp -O format=csv cache.sps])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([CD])
+
+AT_SETUP([CD])
+mkdir subdir
+AT_DATA([cd.sps], [dnl
+cd 'subdir'.
+host command=[['pwd > mydir']].
+])
+AT_CHECK([pspp -O format=csv cd.sps])
+AT_CAPTURE_FILE([subdir/mydir])
+AT_CHECK([sed 's,.*/,,' subdir/mydir], [0], [subdir
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([COMPUTE transformation])
+
+AT_SETUP([COMPUTE crash with SAVE])
+AT_DATA([compute.sps],
+ [INPUT PROGRAM.
+COMPUTE num = 3.
+END FILE.
+END INPUT PROGRAM.
+EXECUTE.
+
+SAVE outfile='temp.sav'.
+])
+AT_CHECK([pspp -O format=csv compute.sps])
+AT_DATA([list.sps],
+ [GET FILE='temp.sav'.
+LIST.
+])
+AT_CHECK([pspp -O format=csv list.sps], [0],
+ [])
+AT_CLEANUP
+
+AT_SETUP([COMPUTE bug in long string UPCASE])
+AT_DATA([compute.sps],
+ [DATA LIST LIST
+ /A (A161)
+ B (A3).
+
+BEGIN DATA
+abc def
+ghi jkl
+END DATA.
+
+COMPUTE A=upcase(A).
+EXECUTE.
+LIST.
+])
+AT_CHECK([pspp -O format=csv compute.sps], [0],
+ [Table: Reading free-form data from INLINE.
+Variable,Format
+A,A161
+B,A3
+
+Table: Data List
+A,B
+ABC,def
+GHI,jkl
+])
+AT_CLEANUP
+
+AT_SETUP([COMPUTE bug with long variable names])
+AT_DATA([compute.sps],
+ [DATA LIST LIST /longVariablename * x *.
+BEGIN DATA.
+1 2
+3 4
+END DATA.
+
+
+COMPUTE longvariableName=100-longvariablename.
+
+LIST.
+])
+AT_CHECK([pspp -O format=csv compute.sps], [0],
+ [Table: Reading free-form data from INLINE.
+Variable,Format
+longVariablename,F8.0
+x,F8.0
+
+Table: Data List
+longVariablename,x
+99.00,2.00
+97.00,4.00
+])
+AT_CLEANUP
+
+AT_SETUP([COMPUTE self-reference to new variable])
+AT_DATA([compute.sps],
+ [DATA LIST /ITEM 1-3.
+COMPUTE SUM=SUM+ITEM.
+PRINT OUTFILE='compute-sum.out' /ITEM SUM.
+LEAVE SUM
+BEGIN DATA.
+123
+404
+555
+999
+END DATA.
+])
+AT_CHECK([pspp -O format=csv compute.sps], [0],
+ [Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+ITEM,1,1-3,F3.0
+])
+AT_CHECK([cat compute-sum.out], [0],
+ [ 123 123.00 @&t@
+ 404 527.00 @&t@
+ 555 1082.00 @&t@
+ 999 2081.00 @&t@
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([CORRELATIONS])
+
+AT_SETUP([CORRELATIONS -- unweighted])
+AT_DATA([correlations.sps], [dnl
+set format = F11.3.
+data list notable list /foo * bar * wiz * bang *.
+begin data.
+1 0 3 1
+3 9 -50 5
+3 4 3 203
+4 -9 0 -4
+98 78 104 2
+3 50 -49 200
+. 4 4 4
+5 3 0 .
+end data.
+
+correlations
+ variables = foo bar wiz bang
+ /print nosig
+ /missing = listwise
+ .
+
+correlations
+ variables = bar wiz
+ /print nosig
+ /missing = listwise
+ .
+
+correlations
+ variables = foo bar wiz bang
+ /print nosig
+ /missing = pairwise
+ .
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt correlations.sps])
+AT_CHECK([cat pspp.csv], [0],
+[[Table: Correlations
+,,foo,bar,wiz,bang
+foo,Pearson Correlation,1.000,.802,.890[a],-.308
+,Sig. (2-tailed),,.055,.017,.553
+bar,Pearson Correlation,.802,1.000,.519,.118
+,Sig. (2-tailed),.055,,.291,.824
+wiz,Pearson Correlation,.890[a],.519,1.000,-.344
+,Sig. (2-tailed),.017,.291,,.505
+bang,Pearson Correlation,-.308,.118,-.344,1.000
+,Sig. (2-tailed),.553,.824,.505,
+Footnote: a. Significant at .05 level
+
+Table: Correlations
+,,bar,wiz
+bar,Pearson Correlation,1.000,.497
+,Sig. (2-tailed),,.210
+wiz,Pearson Correlation,.497,1.000
+,Sig. (2-tailed),.210,
+
+Table: Correlations
+,,foo,bar,wiz,bang
+foo,Pearson Correlation,1.000,.805[a],.883[a],-.308
+,Sig. (2-tailed),,.029,.008,.553
+,N,7,7,7,6
+bar,Pearson Correlation,.805[a],1.000,.497,.164
+,Sig. (2-tailed),.029,,.210,.725
+,N,7,8,8,7
+wiz,Pearson Correlation,.883[a],.497,1.000,-.337
+,Sig. (2-tailed),.008,.210,,.460
+,N,7,8,8,7
+bang,Pearson Correlation,-.308,.164,-.337,1.000
+,Sig. (2-tailed),.553,.725,.460,
+,N,6,7,7,7
+Footnote: a. Significant at .05 level
+]])
+AT_CLEANUP
+
+AT_SETUP([CORRELATIONS -- weighted])
+AT_DATA([correlations1.sps], [dnl
+set format = F11.3.
+data list notable list /foo * bar * wiz * bang * w *.
+begin data.
+1 0 3 1 1
+3 9 -50 5 2
+3 4 3 203 1
+4 -9 0 -4 1
+98 78 104 2 3
+3 50 -49 200 1
+end data.
+
+weight by w.
+
+correlations
+ variables = foo bar wiz bang
+ /statistics=descriptives xprod
+ .
+])
+AT_DATA([correlations2.sps], [dnl
+set format = F11.3.
+data list notable list /foo * bar * wiz * bang * w *.
+begin data.
+1 0 3 1 1
+3 9 -50 5 1
+3 9 -50 5 1
+3 4 3 203 1
+4 -9 0 -4 1
+98 78 104 2 1
+98 78 104 2 1
+98 78 104 2 1
+3 50 -49 200 1
+end data.
+
+weight by w.
+
+correlations
+ variables = foo bar wiz bang
+ /statistics=descriptives xprod
+ .
+])
+AT_CHECK([pspp -O format=csv correlations1.sps], [0], [stdout])
+mv stdout expout
+AT_CHECK([pspp -O format=csv correlations2.sps], [0], [expout])
+AT_CLEANUP
+
+
+AT_SETUP([CORRELATIONS -- non-square])
+AT_DATA([corr-ns.sps], [dnl
+set format = F11.3.
+data list notable list /foo * bar * wiz *.
+begin data.
+1 1 6
+2 2 5
+3 3 4
+4 4 3
+5 5 2
+6 6 1
+end data.
+
+correlations
+ variables = foo with bar wiz
+ .
+])
+
+AT_CHECK([pspp -O format=csv corr-ns.sps], [0], [dnl
+Table: Correlations
+,,bar,wiz
+foo,Pearson Correlation,1.000,-1.000
+,Sig. (2-tailed),.000,.000
+,N,6,6
+])
+
+AT_CLEANUP
+
+dnl Checks for bug #38661.
+AT_SETUP([CORRELATIONS -- crash with WITH keyword])
+AT_DATA([correlations.sps], [dnl
+DATA LIST LIST NOTABLE /a b c d e f g h i.
+.
+BEGIN DATA.
+20 21 17 28 23 4.35 24 19 25
+28 18 29 30 23 4.55 17 23 28
+47 18 30 30 29 4.35 26 31 31
+20 7 19 22 22 4.80 24 16 27
+19 12 17 27 22 . 22 14 25
+22 9 19 30 33 5 29 30 27
+41 16 22 32 23 3.90 26 27 23
+18 18 20 26 22 5.80 17 20 39
+18 24 25 25 31 5.15 27 27 34
+19 22 26 23 37 6 41 32 27
+23 12 15 29 25 4.10 21 27 20
+21 4 28 37 31 5.65 27 18 42
+19 5 17 17 29 3.10 19 16 19
+21 17 20 35 31 . 28 30 22
+END DATA.
+
+CORRELATIONS VARIABLE=a f b WITH c g h i e d/STATISTICS=DESCRIPTIVES.
+])
+AT_CHECK([pspp -o pspp.csv correlations.sps])
+# Check the output, ignoring the actual correlations values since
+# they look pretty nonsensical to me for this input (they include NaNs).
+AT_CHECK([sed '/a,Pearson/,$s/,\([[^,]]*\),.*/,\1,.../' pspp.csv], [0], [dnl
+Table: Descriptive Statistics
+,Mean,Std. Deviation,N
+a,24.00,8.93,14
+f,4.73,.85,12
+b,14.50,6.41,14
+c,21.71,4.98,14
+g,24.86,6.09,14
+h,23.57,6.30,14
+i,27.79,6.73,14
+e,27.21,4.95,14
+d,27.93,5.23,14
+
+Table: Correlations
+,,c,g,h,i,e,d
+a,Pearson Correlation,...
+,Sig. (2-tailed),...
+,N,...
+f,Pearson Correlation,...
+,Sig. (2-tailed),...
+,N,...
+b,Pearson Correlation,...
+,Sig. (2-tailed),...
+,N,...
+])
+AT_CLEANUP
+
+
+
+dnl Checks for bug #40661
+AT_SETUP([CORRELATIONS -- incorrect subtable selection])
+AT_DATA([correlations.sps], [dnl
+set format = F12.4.
+OUTPUT MODIFY /SELECT TABLES /TABLECELLS SELECT = [[CORRELATIONS]] FORMAT=F12.4.
+set decimal = dot.
+data list notable list /var1 var2 var3 var4 var5 *.
+begin data.
+7,6,9,2,3
+9,12,8,5,8
+8,9,7,8,6
+8,8,9,10,8
+7,6,4,5,3
+7,9,8,2,1
+9,8,11,,10
+8,7,6,,5
+6,7,6,,8
+6,,3,,4
+6,,7,3,3
+5,4,2,7,8
+9,8,6,11,10
+5,6,2,2,4
+8,7,6,8,7
+10,13,8,12,10
+7,8,7,11,2
+8,7,7,9,6
+10,11,11,8,1
+5,8,6,9,9
+8,7,5,5,6
+5,7,2,1,8
+9,8,8,13,6
+5,8,5,6,4
+,7,5,4,5
+,8,4,4,3
+,6,4,9,5
+8,11,9,12,3
+9,11,8,10,6
+10,10,7,8,1
+6,6,3,8,9
+10,9,7,12,2
+6,8,,7,4
+6,8,3,2,9
+7,8,8,2,9
+5,6,5,5,5
+9,9,7,7,5
+9,10,11,7,8
+8,11,9,3,3
+5,4,4,0,5
+9,9,11,14,2
+5,6,2,4,4
+8,8,7,4,1
+9,9,8,14,
+6,8,7,2,
+10,9,9,6,
+8,8,10,9,
+7,8,4,12,
+6,6,6,7,1
+5,7,7,4,10
+9,10,10,13,4
+9,11,9,8,7
+10,13,12,6,8
+8,11,6,8,5
+7,8,7,12,2
+6,7,4,1,10
+5,4,5,6,10
+7,8,6,12,10
+6,5,3,9,2
+7,8,8,7,2
+5,4,4,9,8
+5,7,6,3,9
+10,10,9,13,1
+8,10,9,5,4
+8,9,8,8,7
+7,9,9,6,7
+10,9,7,12,6
+10,13,12,12,4
+7,10,9,7,2
+6,8,7,11,6
+8,11,5,13,2
+7,10,6,12,8
+10,10,9,7,9
+9,12,6,7,10
+6,6,8,2,9
+10,9,12,13,10
+8,9,8,3,6
+8,7,6,4,10
+8,7,10,12,2
+7,6,8,2,7
+8,11,6,9,4
+6,6,7,8,2
+6,7,3,11,4
+5,6,3,0,5
+10,10,11,15,6
+5,4,7,6,8
+5,4,4,1,3
+6,9,8,1,6
+10,11,10,15,8
+7,10,4,11,7
+9,12,8,6,3
+10,10,11,15,2
+10,9,9,15,3
+6,6,8,5,1
+5,7,7,0,3
+9,8,10,6,8
+9,8,11,11,4
+8,10,7,3,4
+7,8,7,3,3
+8,9,10,13,8
+end data.
+
+CORRELATION
+ /VARIABLES = var1 var2 var3 WITH var4 var5
+ /PRINT = TWOTAIL NOSIG.
+
+CORRELATION
+ /VARIABLES = var3 var4 var5 WITH var1 var2
+ /PRINT = TWOTAIL NOSIG.
+
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt correlations.sps])
+AT_CHECK([cat pspp.csv], [0],
+[[Table: Correlations
+,,var4,var5
+var1,Pearson Correlation,.5693[a],-.0519
+,Sig. (2-tailed),.000,.623
+,N,93,92
+var2,Pearson Correlation,.3792[a],-.0407
+,Sig. (2-tailed),.000,.698
+,N,95,93
+var3,Pearson Correlation,.3699[a],-.0543
+,Sig. (2-tailed),.000,.603
+,N,95,94
+Footnote: a. Significant at .05 level
+
+Table: Correlations
+,,var1,var2
+var3,Pearson Correlation,.6964[a],.5615[a]
+,Sig. (2-tailed),.000,.000
+,N,96,97
+var4,Pearson Correlation,.5693[a],.3792[a]
+,Sig. (2-tailed),.000,.000
+,N,93,95
+var5,Pearson Correlation,-.0519,-.0407
+,Sig. (2-tailed),.623,.698
+,N,92,93
+Footnote: a. Significant at .05 level
+]])
+
+AT_CLEANUP
+
+
+dnl Crash found by zzuf
+AT_SETUP([CORRELATIONS -- empty dataset])
+
+AT_DATA([correlations.sps], [dnl
+data list list /a b c q g *.
+CORRELATIONS 'VARIABLES = a b.]
+)
+
+AT_CHECK([pspp -o pspp.csv correlations.sps], [1], [ignore])
+
+AT_CLEANUP
+
+dnl Another Crash found by zzuf
+AT_SETUP([CORRELATIONS -- empty dataset 2])
+
+AT_DATA([correlations.sps], [dnl
+data list notable list /foo * bar * wiz bang *.
+begin data.
+ 1 00 3 .
+ 3 9 -50 .
+98 78 104 .
+ . 4 4 .
+ 5 3 0 .
+end data.
+
+correlations
+ variables = foo bar wiz bang
+ /missing = listwise
+ .
+])
+
+AT_CHECK([pspp -O format=csv correlations.sps], [1], [dnl
+correlations.sps:13: error: CORRELATIONS: The data for the chosen variables are all missing or empty.
+])
+
+AT_CLEANUP
+
+AT_SETUP([CORRELATIONS -- syntax errors])
+AT_DATA([correlations.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+CORRELATIONS MISSING=**.
+CORRELATIONS PRINT=**.
+CORRELATIONS STATISTICS=**.
+CORRELATIONS **.
+CORRELATIONS x y z WITH **.
+CORRELATIONS.
+])
+AT_CHECK([pspp -O format=csv correlations.sps], [1], [dnl
+"correlations.sps:2.22-2.23: error: CORRELATIONS: Syntax error expecting PAIRWISE, LISTWISE, INCLUDE, or EXCLUDE.
+ 2 | CORRELATIONS MISSING=**.
+ | ^~"
+
+"correlations.sps:3.20-3.21: error: CORRELATIONS: Syntax error expecting TWOTAIL, ONETAIL, SIG, or NOSIG.
+ 3 | CORRELATIONS PRINT=**.
+ | ^~"
+
+"correlations.sps:4.25-4.26: error: CORRELATIONS: Syntax error expecting DESCRIPTIVES, XPROD, or ALL.
+ 4 | CORRELATIONS STATISTICS=**.
+ | ^~"
+
+"correlations.sps:5.14-5.15: error: CORRELATIONS: Syntax error expecting variable name.
+ 5 | CORRELATIONS **.
+ | ^~"
+
+"correlations.sps:6.25-6.26: error: CORRELATIONS: Syntax error expecting variable name.
+ 6 | CORRELATIONS x y z WITH **.
+ | ^~"
+
+"correlations.sps:7.1-7.12: error: CORRELATIONS: No variables specified.
+ 7 | CORRELATIONS.
+ | ^~~~~~~~~~~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([COUNT])
+
+AT_SETUP([COUNT -- numeric data])
+AT_DATA([count.sps], [dnl
+DATA LIST LIST /x y.
+BEGIN DATA.
+1 2
+2 3
+4 5
+2 2
+5 6
+7 2
+. 2
+END DATA.
+
+MISSING VALUES x(7)/y(3).
+
+COUNT c=x y (2)/d=x y(7)/e=x y(missing)/f=x y(sysmis).
+
+FORMATS ALL(F1).
+
+LIST.
+])
+AT_CHECK([pspp -O format=csv count.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+y,F8.0
+
+Table: Data List
+x,y,c,d,e,f
+1,2,1,0,0,0
+2,3,1,0,1,0
+4,5,0,0,0,0
+2,2,2,0,0,0
+5,6,0,0,0,0
+7,2,1,1,1,0
+.,2,1,0,1,1
+])
+AT_CLEANUP
+
+AT_SETUP([COUNT -- string data])
+AT_DATA([count.sps], [dnl
+TITLE 'Test COUNT transformation'.
+
+DATA LIST /v1 to v2 1-4(a).
+BEGIN DATA.
+1234
+321
+2 13
+4121
+1104
+03 4
+0193
+END DATA.
+COUNT c=v1 to v2('2',' 4','1').
+LIST.
+])
+AT_CHECK([pspp -O format=csv count.sps], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+v1,1,1-2,A2
+v2,1,3-4,A2
+
+Table: Data List
+v1,v2,c
+12,34,.00
+32,1,1.00
+2,13,1.00
+41,21,.00
+11,04,.00
+03,4,1.00
+01,93,.00
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([CROSSTABS procedure])
+
+dnl Based on bug #60982.
+AT_SETUP([CROSSTABS residuals])
+AT_DATA([crosstabs.sps],
+ [DATASET CLOSE ALL.
+DATA LIST LIST NOTABLE/ r c n.
+BEGIN DATA
+1 1 26
+1 2 31
+2 1 12
+2 2 32
+3 1 27
+3 2 18
+4 1 8
+4 2 7
+END DATA.
+WEIGHT by n.
+CROSSTABS r by c /STATISTICS=CHISQ
+/CELLS=COUNT EXPECTED RESID SRESID ASRESID.
+])
+AT_CHECK([pspp -O format=csv crosstabs.sps], [0],
+ [Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+r × c,161.00,100.0%,.00,.0%,161.00,100.0%
+
+Table: r × c
+,,,c,,Total
+,,,1.00,2.00,
+r,1.00,Count,26.00,31.00,57.00
+,,Expected,25.84,31.16,.35
+,,Residual,.16,-.16,
+,,Std. Residual,.03,-.03,
+,,Adjusted Residual,.05,-.05,
+,2.00,Count,12.00,32.00,44.00
+,,Expected,19.95,24.05,.27
+,,Residual,-7.95,7.95,
+,,Std. Residual,-1.78,1.62,
+,,Adjusted Residual,-2.82,2.82,
+,3.00,Count,27.00,18.00,45.00
+,,Expected,20.40,24.60,.28
+,,Residual,6.60,-6.60,
+,,Std. Residual,1.46,-1.33,
+,,Adjusted Residual,2.33,-2.33,
+,4.00,Count,8.00,7.00,15.00
+,,Expected,6.80,8.20,.09
+,,Residual,1.20,-1.20,
+,,Std. Residual,.46,-.42,
+,,Adjusted Residual,.65,-.65,
+Total,,Count,73.00,88.00,161.00
+,,Expected,.45,.55,1.00
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed)
+Pearson Chi-Square,10.09,3.00,.018
+Likelihood Ratio,10.35,3.00,.016
+Linear-by-Linear Association,1.96,1.00,.162
+N of Valid Cases,161.00,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS integer mode crash])
+AT_DATA([crosstabs.sps],
+ [DATA LIST LIST /A * B * X * Y * .
+BEGIN DATA.
+2 3 4 5
+END DATA.
+
+CROSSTABS VARIABLES X (1,7) Y (1,7) /TABLES X BY Y.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
+AT_CHECK([cat pspp.csv], [0],
+ [[Table: Reading free-form data from INLINE.
+Variable,Format
+A,F8.0
+B,F8.0
+X,F8.0
+Y,F8.0
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X × Y,1,100.0%,0,.0%,1,100.0%
+
+Table: X × Y
+,,,Y,,,,,,,Total
+,,,1.00,2.00,3.00,4.00,5.00,6.00,7.00,
+X,1.00,Count,0,0,0,0,0,0,0,0
+,2.00,Count,0,0,0,0,0,0,0,0
+,3.00,Count,0,0,0,0,0,0,0,0
+,4.00,Count,0,0,0,0,1,0,0,1
+,5.00,Count,0,0,0,0,0,0,0,0
+,6.00,Count,0,0,0,0,0,0,0,0
+,7.00,Count,0,0,0,0,0,0,0,0
+Total,,Count,0,0,0,0,1,0,0,1
+]])
+AT_CLEANUP
+
+# Bug #47600.
+AT_SETUP([CROSSTABS integer mode crash 2])
+AT_DATA([crosstabs.sps], [dnl
+DATA LIST lIST /x y.
+BEGIN DATA.
+4 5
+END DATA.
+
+CROSSTABS
+ VARIABLES x (1,3) y (1,7)
+ /TABLES x BY y.
+])
+AT_CHECK([pspp -O format=csv crosstabs.sps], [0],
+ [[Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+y,F8.0
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,0,.0%,1,100.0%,1,100.0%
+
+Table: x × y
+,,,y,,,,,,,Total
+,,,1.00,2.00,3.00,4.00,5.00,6.00,7.00,
+x,1.00,Count,,,,,,,,
+,2.00,Count,,,,,,,,
+,3.00,Count,,,,,,,,
+Total,,Count,,,,,,,,
+]])
+AT_CLEANUP
+
+# Bug #22037.
+AT_SETUP([CROSSTABS long string crash])
+AT_DATA([crosstabs.sps],
+ [data list list /x * y (a18).
+
+begin data.
+
+ 1. 'zero none'
+
+1 'one unity'
+2 'two duality'
+3 'three lots'
+end data.
+
+CROSSTABS /TABLES = x BY y.
+])
+AT_CHECK([pspp -o - -O format=csv -o pspp.txt crosstabs.sps], [0],
+ [[Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+y,A18
+
+"crosstabs.sps:4: warning: Missing value(s) for all variables from x onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"crosstabs.sps:6: warning: Missing value(s) for all variables from x onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,4,66.7%,2,33.3%,6,100.0%
+
+Table: x × y
+,,,y,,,,Total
+,,,one unity,three lots,two duality,zero none,
+x,1.00,Count,1,0,0,1,2
+,2.00,Count,0,0,1,0,1
+,3.00,Count,0,1,0,0,1
+Total,,Count,1,1,1,1,4
+]])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS crash])
+AT_DATA([crosstabs.sps],
+ [[DATA LIST FIXED
+ / x 1-2
+ y 3
+ z 4.
+
+BEGIN DATA.
+0111
+0222
+0311
+0412
+0521
+0612
+0711
+0811
+0912
+END DATA.
+
+LIST.
+
+
+CROSSTABS TABLES y by z.
+]])
+AT_CHECK([pspp -o - -O format=csv -o pspp.txt crosstabs.sps], [0],
+ [Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+x,1,1-2,F2.0
+y,1,3-3,F1.0
+z,1,4-4,F1.0
+
+Table: Data List
+x,y,z
+1,1,1
+2,2,2
+3,1,1
+4,1,2
+5,2,1
+6,1,2
+7,1,1
+8,1,1
+9,1,2
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+y × z,9,100.0%,0,.0%,9,100.0%
+
+Table: y × z
+,,,z,,Total
+,,,1,2,
+y,1,Count,4,3,7
+,2,Count,1,1,2
+Total,,Count,5,4,9
+])
+AT_CLEANUP
+
+# Bug #26739, which caused CROSSTABS to crash or to fail to output
+# chi-square results.
+AT_SETUP([CROSSTABS chi-square crash])
+AT_DATA([crosstabs.sps],
+ [[DATA LIST LIST /x * y *.
+BEGIN DATA.
+2 2
+3 1
+4 2
+4 1
+END DATA.
+
+CROSSTABS
+ /TABLES= x BY y
+ /STATISTICS=CHISQ.
+]])
+AT_CHECK([pspp -O format=csv crosstabs.sps], [0],
+ [[Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+y,F8.0
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,4,100.0%,0,.0%,4,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,2.00,Count,0,1,1
+,3.00,Count,1,0,1
+,4.00,Count,1,1,2
+Total,,Count,2,2,4
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed)
+Pearson Chi-Square,2.00,2,.368
+Likelihood Ratio,2.77,2,.250
+Linear-by-Linear Association,.27,1,.602
+N of Valid Cases,4,,
+]])
+AT_CLEANUP
+
+# Bug #27883.
+AT_SETUP([CROSSTABS crash with SPLIT FILE])
+AT_DATA([crosstabs.sps],
+ [data list notable / v0 to v2 1-6 (A)
+begin data.
+a c e
+a c e
+a c e
+a d e
+a d f
+b d f
+b d f
+b c f
+b d e
+a c f
+end data.
+SORT CASES BY v0.
+SPLIT FILE SEPARATE BY v0.
+
+CROSSTABS
+ /TABLES= v1 BY v2
+ /FORMAT=AVALUE TABLES
+ /STATISTICS=CHISQ
+ /CELLS=COUNT ROW COLUMN TOTAL.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Split Values
+Variable,Value
+v0,a
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+v1 × v2,6,100.0%,0,.0%,6,100.0%
+
+Table: v1 × v2
+,,,v2,,Total
+,,,e,f,
+v1,c,Count,3,1,4
+,,Row %,75.0%,25.0%,100.0%
+,,Column %,75.0%,50.0%,66.7%
+,,Total %,50.0%,16.7%,66.7%
+,d,Count,1,1,2
+,,Row %,50.0%,50.0%,100.0%
+,,Column %,25.0%,50.0%,33.3%
+,,Total %,16.7%,16.7%,33.3%
+Total,,Count,4,2,6
+,,Row %,66.7%,33.3%,100.0%
+,,Column %,100.0%,100.0%,100.0%
+,,Total %,66.7%,33.3%,100.0%
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
+Pearson Chi-Square,.38,1,.540,,
+Likelihood Ratio,.37,1,.545,,
+Fisher's Exact Test,,,,1.000,.600
+Continuity Correction,.00,1,1.000,,
+N of Valid Cases,6,,,,
+
+Table: Split Values
+Variable,Value
+v0,b
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+v1 × v2,4,100.0%,0,.0%,4,100.0%
+
+Table: v1 × v2
+,,,v2,,Total
+,,,e,f,
+v1,c,Count,0,1,1
+,,Row %,.0%,100.0%,100.0%
+,,Column %,.0%,33.3%,25.0%
+,,Total %,.0%,25.0%,25.0%
+,d,Count,1,2,3
+,,Row %,33.3%,66.7%,100.0%
+,,Column %,100.0%,66.7%,75.0%
+,,Total %,25.0%,50.0%,75.0%
+Total,,Count,1,3,4
+,,Row %,25.0%,75.0%,100.0%
+,,Column %,100.0%,100.0%,100.0%
+,,Total %,25.0%,75.0%,100.0%
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
+Pearson Chi-Square,.44,1,.505,,
+Likelihood Ratio,.68,1,.410,,
+Fisher's Exact Test,,,,1.000,.750
+Continuity Correction,.00,1,1.000,,
+N of Valid Cases,4,,,,
+])
+AT_CLEANUP
+
+# Bug #24752.
+AT_SETUP([3-way CROSSTABS])
+AT_DATA([crosstabs.sps],
+ [[DATA LIST FIXED
+ / x 1-2
+ y 3
+ z 4.
+
+BEGIN DATA.
+0111
+0222
+0311
+0412
+0521
+0612
+0711
+0811
+0912
+END DATA.
+
+LIST.
+
+
+CROSSTABS TABLES x BY y BY z/STATISTICS=ALL.
+]])
+AT_CHECK([pspp -o - -O format=csv -o pspp.csv -o pspp.txt crosstabs.sps], [0],
+ [Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+x,1,1-2,F2.0
+y,1,3-3,F1.0
+z,1,4-4,F1.0
+
+Table: Data List
+x,y,z
+1,1,1
+2,2,2
+3,1,1
+4,1,2
+5,2,1
+6,1,2
+7,1,1
+8,1,1
+9,1,2
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y × z,9,100.0%,0,.0%,9,100.0%
+
+Table: x × y × z
+,,,,,y,,Total
+,,,,,1,2,
+z,1,x,1,Count,1,0,1
+,,,3,Count,0,0,1
+,,,5,Count,1,0,1
+,,,7,Count,0,0,1
+,,,8,Count,0,1,1
+,,Total,,Count,4,1,5
+,2,x,2,Count,0,0,1
+,,,4,Count,0,1,1
+,,,6,Count,0,0,1
+,,,9,Count,1,0,1
+,,Total,,Count,3,1,4
+
+Table: Chi-Square Tests
+,,,Value,df,Asymptotic Sig. (2-tailed)
+z,1,Pearson Chi-Square,5.00,4,.287
+,,Likelihood Ratio,5.00,4,.287
+,,Linear-by-Linear Association,.01,1,.938
+,,N of Valid Cases,5,,
+,2,Pearson Chi-Square,4.00,3,.261
+,,Likelihood Ratio,4.50,3,.212
+,,Linear-by-Linear Association,1.58,1,.209
+,,N of Valid Cases,4,,
+
+Table: Symmetric Measures
+,,,,Value,Asymp. Std. Error,Approx. T
+z,1,Nominal by Nominal,Phi,1.00,,
+,,,Cramer's V,1.00,,
+,,,Contingency Coefficient,.71,,
+,,Ordinal by Ordinal,Kendall's tau-b,.00,.32,.00
+,,,Kendall's tau-c,.00,.32,.00
+,,,Gamma,.00,.50,.00
+,,,Spearman Correlation,.00,.22,.00
+,,Interval by Interval,Pearson's R,.04,.22,.07
+,,N of Valid Cases,,5,,
+,2,Nominal by Nominal,Phi,1.00,,
+,,,Cramer's V,1.00,,
+,,,Contingency Coefficient,.71,,
+,,Ordinal by Ordinal,Kendall's tau-b,-.71,.20,-1.73
+,,,Kendall's tau-c,-.75,.43,-1.73
+,,,Gamma,-1.00,.00,-1.73
+,,,Spearman Correlation,-.77,.17,-1.73
+,,Interval by Interval,Pearson's R,-.73,.18,-1.49
+,,N of Valid Cases,,4,,
+
+Table: Directional Measures
+,,,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+z,1,Nominal by Nominal,Lambda,Symmetric,.40,.28,1.12,.264
+,,,,x Dependent,.25,.22,1.12,.264
+,,,,y Dependent,1.00,.00,1.12,.264
+,,,Goodman and Kruskal tau,x Dependent,.25,,,
+,,,,y Dependent,1.00,,,
+,,,Uncertainty Coefficient,Symmetric,.47,.18,,
+,,,,x Dependent,.31,.15,2.02,
+,,,,y Dependent,1.00,.00,2.02,
+,,Ordinal by Ordinal,Somers' d,Symmetric,.00,,.00,1.000
+,,,,x Dependent,.00,.50,.00,1.000
+,,,,y Dependent,.00,.20,.00,1.000
+,,Nominal by Interval,Eta,x Dependent,.04,,,
+,,,,y Dependent,1.00,,,
+,2,Nominal by Nominal,Lambda,Symmetric,.50,.25,2.00,.046
+,,,,x Dependent,.33,.27,1.15,.248
+,,,,y Dependent,1.00,.00,1.15,.248
+,,,Goodman and Kruskal tau,x Dependent,.33,,,
+,,,,y Dependent,1.00,,,
+,,,Uncertainty Coefficient,Symmetric,.58,.17,,
+,,,,x Dependent,.41,.17,2.36,
+,,,,y Dependent,1.00,.00,2.36,
+,,Ordinal by Ordinal,Somers' d,Symmetric,-.67,,-1.73,.083
+,,,,x Dependent,-1.00,.00,-1.73,.083
+,,,,y Dependent,-.50,.29,-1.73,.083
+,,Nominal by Interval,Eta,x Dependent,.73,,,
+,,,,y Dependent,1.00,,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS rounding weights with COUNT])
+AT_DATA([crosstabs.sps],
+ [[DATA LIST NOTABLE LIST /x y w.
+BEGIN DATA.
+1 1 1.4
+1 1 1.4
+1 2 1.6
+1 2 1.6
+2 1 1
+2 2 2
+END DATA.
+WEIGHT BY w.
+
+* These should have the same effect (no rounding).
+CROSSTABS /TABLES x BY y.
+CROSSTABS /TABLES x BY y /COUNT ASIS.
+
+* Round input weights.
+CROSSTABS /TABLES x BY y /COUNT CASE ROUND.
+CROSSTABS /TABLES x BY y /COUNT CASE TRUNCATE.
+
+* Round cell weights.
+CROSSTABS /TABLES x BY y /COUNT.
+CROSSTABS /TABLES x BY y /COUNT TRUNCATE.
+]])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
+AT_CHECK([cat pspp.csv], [0],
+ [Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,9.00,100.0%,.00,.0%,9.00,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,1.00,Count,2.80,3.20,6.00
+,2.00,Count,1.00,2.00,3.00
+Total,,Count,3.80,5.20,9.00
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,9.00,100.0%,.00,.0%,9.00,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,1.00,Count,2.80,3.20,6.00
+,2.00,Count,1.00,2.00,3.00
+Total,,Count,3.80,5.20,9.00
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,9.00,100.0%,.00,.0%,9.00,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,1.00,Count,2.00,4.00,6.00
+,2.00,Count,1.00,2.00,3.00
+Total,,Count,3.00,6.00,9.00
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,7.00,100.0%,.00,.0%,7.00,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,1.00,Count,2.00,2.00,4.00
+,2.00,Count,1.00,2.00,3.00
+Total,,Count,3.00,4.00,7.00
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,9.00,100.0%,.00,.0%,9.00,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,1.00,Count,3.00,3.00,6.00
+,2.00,Count,1.00,2.00,3.00
+Total,,Count,4.00,5.00,9.00
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,8.00,100.0%,.00,.0%,8.00,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,1.00,2.00,
+x,1.00,Count,2.00,3.00,5.00
+,2.00,Count,1.00,2.00,3.00
+Total,,Count,3.00,5.00,8.00
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS descending sort order])
+AT_DATA([crosstabs-descending.sps],
+ [[DATA LIST NOTABLE LIST /x * y *.
+BEGIN DATA.
+2 2
+2 2
+3 1
+4 1
+3 2
+3 2
+END DATA.
+
+CROSSTABS
+ /TABLES= x BY y
+ /FORMAT = DVALUE.
+]])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs-descending.sps])
+AT_CHECK([cat pspp.csv], [0],
+ [Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,6,100.0%,0,.0%,6,100.0%
+
+Table: x × y
+,,,y,,Total
+,,,2.00,1.00,
+x,4.00,Count,0,1,1
+,3.00,Count,2,1,3
+,2.00,Count,2,0,2
+Total,,Count,4,2,6
+])
+AT_CLEANUP
+
+# Bug #31260.
+AT_SETUP([CROSSTABS crash when all cases missing])
+AT_DATA([crosstabs.sps], [dnl
+DATA LIST LIST NOTABLE /X1 X2.
+BEGIN DATA.
+1 1
+END DATA.
+
+MISSING VALUES x2 (1).
+
+CROSSTABS /TABLES= X1 by X2.
+])
+AT_CHECK([pspp -O format=csv crosstabs.sps], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X1 × X2,0,.0%,1,100.0%,1,100.0%
+
+"crosstabs.sps:8.20-8.27: warning: CROSSTABS: Crosstabulation X1 × X2 contained no non-missing cases.
+ 8 | CROSSTABS /TABLES= X1 by X2.
+ | ^~~~~~~~"
+])
+AT_CLEANUP
+
+
+
+dnl This example comes from http://www.ats.ucla.edu/stat/spss/whatstat/whatstat.htm#chisq
+AT_SETUP([CROSSTABS Fisher Exact Test])
+
+AT_DATA([fisher-exact.sps], [dnl
+SET FORMAT F12.3.
+SET DECIMAL DOT.
+
+DATA LIST notable LIST /schtyp (F9.2) female (F9.2) ses (F9.2) .
+begin data.
+ 1.00 .00 1.00
+ 1.00 1.00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 3.00
+ 1.00 .00 1.00
+ 1.00 .00 1.00
+ 1.00 .00 3.00
+ 2.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 2.00 .00 2.00
+ 2.00 .00 3.00
+ 1.00 .00 1.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 2.00 .00 3.00
+ 1.00 .00 2.00
+ 2.00 .00 3.00
+ 1.00 .00 3.00
+ 2.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 1.00
+ 1.00 .00 2.00
+ 2.00 .00 2.00
+ 2.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 1.00
+ 1.00 .00 3.00
+ 1.00 .00 1.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 2.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 2.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 1.00
+ 1.00 .00 2.00
+ 2.00 .00 2.00
+ 1.00 .00 2.00
+ 2.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 1.00
+ 1.00 .00 2.00
+ 2.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 1.00
+ 1.00 .00 1.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 2.00
+ 1.00 .00 1.00
+ 1.00 .00 3.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 3.00
+ 1.00 .00 1.00
+ 2.00 .00 2.00
+ 1.00 .00 1.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 3.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 3.00
+ 1.00 .00 2.00
+ 1.00 .00 1.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 2.00 1.00 3.00
+ 1.00 1.00 3.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 2.00 1.00 1.00
+ 2.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 2.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 2.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 3.00
+ 2.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 2.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 2.00 1.00 3.00
+ 1.00 1.00 2.00
+ 2.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 1.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 2.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 1.00
+ 2.00 1.00 2.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 2.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 2.00
+ 1.00 1.00 2.00
+ 2.00 1.00 3.00
+ 1.00 1.00 1.00
+ 1.00 1.00 1.00
+ 2.00 1.00 3.00
+ 2.00 1.00 2.00
+ 1.00 1.00 3.00
+ 2.00 1.00 2.00
+ 2.00 1.00 2.00
+ 1.00 1.00 2.00
+ 2.00 1.00 2.00
+ 1.00 1.00 2.00
+ 1.00 1.00 3.00
+end data.
+
+VARIABLE LABEL schtyp 'type of school'.
+ADD VALUE LABELS female 0 male 1 female.
+ADD VALUE LABELS ses 1 low 2 middle 3 high.
+ADD VALUE LABELS schtyp 1 public 2 private.
+
+crosstabs /tables = schtyp by female /statistic = chisq.
+crosstabs /tables = female by ses /statistic = chisq.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt fisher-exact.sps])
+AT_CHECK([cat pspp.csv], [0], [Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+type of school × female,200,100.0%,0,.0%,200,100.0%
+
+Table: type of school × female
+,,,female,,Total
+,,,male,female,
+type of school,public,Count,77,91,168
+,private,Count,14,18,32
+Total,,Count,91,109,200
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
+Pearson Chi-Square,.047,1,.828,,
+Likelihood Ratio,.047,1,.828,,
+Fisher's Exact Test,,,,.849,.492
+Continuity Correction,.001,1,.981,,
+Linear-by-Linear Association,.047,1,.829,,
+N of Valid Cases,200,,,,
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+female × ses,200,100.0%,0,.0%,200,100.0%
+
+Table: female × ses
+,,,ses,,,Total
+,,,low,middle,high,
+female,male,Count,15,47,29,91
+,female,Count,32,48,29,109
+Total,,Count,47,95,58,200
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed)
+Pearson Chi-Square,4.577,2,.101
+Likelihood Ratio,4.679,2,.096
+Linear-by-Linear Association,3.110,1,.078
+N of Valid Cases,200,,
+])
+
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Pearson's R - 1])
+AT_DATA([pearson.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://www.statisticslectures.com/topics/pearsonr/.
+DATA LIST FREE/x y.
+BEGIN DATA.
+1 4
+3 6
+5 10
+5 12
+6 13
+END DATA.
+CROSSTABS x BY y/STATISTICS=CORR.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,5,100.0%,0,.0%,5,100.0%
+
+Table: x × y
+,,,y,,,,,Total
+,,,4.000,6.000,10.000,12.000,13.000,
+x,1.000,Count,1,0,0,0,0,1
+,3.000,Count,0,1,0,0,0,1
+,5.000,Count,0,0,1,1,0,2
+,6.000,Count,0,0,0,0,1,1
+Total,,Count,1,1,1,1,1,5
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Spearman Correlation,.975,.022,7.550
+Interval by Interval,Pearson's R,.968,.017,6.708
+N of Valid Cases,,5,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Pearson's R - 2])
+AT_DATA([pearson2.sps], [dnl
+SET FORMAT F8.3.
+
+* Checked with http://www.socscistatistics.com/tests/pearson/Default2.aspx.
+DATA LIST FREE/x y.
+BEGIN DATA.
+1 1.5
+2 1.5
+3 4
+4 6
+5 5
+6 7
+7 6.5
+8 9
+9 10.5
+10 11
+END DATA.
+CROSSTABS x BY y/STATISTICS=CORR.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson2.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,10,100.0%,0,.0%,10,100.0%
+
+Table: x × y
+,,,y,,,,,,,,,Total
+,,,1.500,4.000,5.000,6.000,6.500,7.000,9.000,10.500,11.000,
+x,1.000,Count,1,0,0,0,0,0,0,0,0,1
+,2.000,Count,1,0,0,0,0,0,0,0,0,1
+,3.000,Count,0,1,0,0,0,0,0,0,0,1
+,4.000,Count,0,0,0,1,0,0,0,0,0,1
+,5.000,Count,0,0,1,0,0,0,0,0,0,1
+,6.000,Count,0,0,0,0,0,1,0,0,0,1
+,7.000,Count,0,0,0,0,1,0,0,0,0,1
+,8.000,Count,0,0,0,0,0,0,1,0,0,1
+,9.000,Count,0,0,0,0,0,0,0,1,0,1
+,10.000,Count,0,0,0,0,0,0,0,0,1,1
+Total,,Count,2,1,1,1,1,1,1,1,1,10
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Spearman Correlation,.973,.015,11.844
+Interval by Interval,Pearson's R,.971,.017,11.580
+N of Valid Cases,,10,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Pearson's R - 3])
+AT_DATA([pearson3.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://learntech.uwe.ac.uk/da/Default.aspx?pageid=1442.
+DATA LIST FREE/x y.
+BEGIN DATA.
+56 87
+56 91
+65 85
+65 91
+50 75
+25 28
+87 122
+44 66
+35 58
+END DATA.
+CROSSTABS x BY y/STATISTICS=CORR.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson3.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,9,100.0%,0,.0%,9,100.0%
+
+Table: x × y
+,,,y,,,,,,,,Total
+,,,28.000,58.000,66.000,75.000,85.000,87.000,91.000,122.000,
+x,25.000,Count,1,0,0,0,0,0,0,0,1
+,35.000,Count,0,1,0,0,0,0,0,0,1
+,44.000,Count,0,0,1,0,0,0,0,0,1
+,50.000,Count,0,0,0,1,0,0,0,0,1
+,56.000,Count,0,0,0,0,0,1,1,0,2
+,65.000,Count,0,0,0,0,1,0,1,0,2
+,87.000,Count,0,0,0,0,0,0,0,1,1
+Total,,Count,1,1,1,1,1,1,2,1,9
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Spearman Correlation,.911,.068,5.860
+Interval by Interval,Pearson's R,.966,.017,9.915
+N of Valid Cases,,9,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Pearson's R - 4])
+AT_DATA([pearson4.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://psychology.ucdavis.edu/faculty_sites/sommerb/sommerdemo/correlation/hand/pearson_hand.htm.
+DATA LIST FREE/x y.
+BEGIN DATA.
+5 5
+10 20
+6 4
+8 15
+4 11
+4 9
+3 12
+10 18
+2 7
+6 2
+7 14
+9 17
+END DATA.
+CROSSTABS x BY y/STATISTICS=CORR.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson4.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,12,100.0%,0,.0%,12,100.0%
+
+Table: x × y
+,,,y,,,,,,,,,,,,Total
+,,,2.000,4.000,5.000,7.000,9.000,11.000,12.000,14.000,15.000,17.000,18.000,20.000,
+x,2.000,Count,0,0,0,1,0,0,0,0,0,0,0,0,1
+,3.000,Count,0,0,0,0,0,0,1,0,0,0,0,0,1
+,4.000,Count,0,0,0,0,1,1,0,0,0,0,0,0,2
+,5.000,Count,0,0,1,0,0,0,0,0,0,0,0,0,1
+,6.000,Count,1,1,0,0,0,0,0,0,0,0,0,0,2
+,7.000,Count,0,0,0,0,0,0,0,1,0,0,0,0,1
+,8.000,Count,0,0,0,0,0,0,0,0,1,0,0,0,1
+,9.000,Count,0,0,0,0,0,0,0,0,0,1,0,0,1
+,10.000,Count,0,0,0,0,0,0,0,0,0,0,1,1,2
+Total,,Count,1,1,1,1,1,1,1,1,1,1,1,1,12
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Spearman Correlation,.657,.140,2.758
+Interval by Interval,Pearson's R,.667,.132,2.830
+N of Valid Cases,,12,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Pearson's R - 5])
+AT_DATA([pearson5.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://www.statisticslectures.com/topics/pearsonr/.
+DATA LIST FREE/x y.
+BEGIN DATA.
+18 15000
+25 29000
+57 68000
+45 52000
+26 32000
+64 80000
+37 41000
+40 45000
+24 26000
+33 33000
+END DATA.
+CROSSTABS x BY y/STATISTICS=CORR.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson5.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,10,100.0%,0,.0%,10,100.0%
+
+Table: x × y
+,,,y,,,,,,,,,,Total
+,,,15000.00,26000.00,29000.00,32000.00,33000.00,41000.00,45000.00,52000.00,68000.00,80000.00,
+x,18.000,Count,1,0,0,0,0,0,0,0,0,0,1
+,24.000,Count,0,1,0,0,0,0,0,0,0,0,1
+,25.000,Count,0,0,1,0,0,0,0,0,0,0,1
+,26.000,Count,0,0,0,1,0,0,0,0,0,0,1
+,33.000,Count,0,0,0,0,1,0,0,0,0,0,1
+,37.000,Count,0,0,0,0,0,1,0,0,0,0,1
+,40.000,Count,0,0,0,0,0,0,1,0,0,0,1
+,45.000,Count,0,0,0,0,0,0,0,1,0,0,1
+,57.000,Count,0,0,0,0,0,0,0,0,1,0,1
+,64.000,Count,0,0,0,0,0,0,0,0,0,1,1
+Total,,Count,1,1,1,1,1,1,1,1,1,1,10
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Spearman Correlation,1.000,.000,+Infinit
+Interval by Interval,Pearson's R,.992,.004,22.638
+N of Valid Cases,,10,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - 1])
+AT_DATA([lambda.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://www.csupomona.edu/~jlkorey/POWERMUTT/Topics/contingency_tables.html.
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 424
+1 2 213
+1 3 59
+3 1 55
+3 2 188
+3 3 357
+END DATA.
+
+CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,1296.000,100.0%,.000,.0%,1296.000,100.0%
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.423,.021,16.875,.000
+,,x Dependent,.497,.024,15.986,.000
+,,y Dependent,.370,.020,16.339,.000
+,Goodman and Kruskal tau,x Dependent,.382,,,
+,,y Dependent,.198,,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - 2])
+AT_DATA([lambda.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://vassarstats.net.
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 19
+1 2 26
+1 3 8
+2 1 21
+2 2 13
+2 3 5
+3 1 6
+3 2 12
+3 3 27
+END DATA.
+
+CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,137.000,100.0%,.000,.0%,137.000,100.0%
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.259,.081,2.902,.004
+,,x Dependent,.250,.089,2.479,.013
+,,y Dependent,.267,.085,2.766,.006
+,Goodman and Kruskal tau,x Dependent,.129,,,
+,,y Dependent,.123,,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - 3])
+AT_DATA([lambda.sps], [dnl
+SET FORMAT F8.3.
+
+* From Goodman, L.A., Kruskal, W.H. (1954) "Measures of association for
+ cross classifications". Part I. Journal of the American Statistical
+ Association, 49, 732-764.
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 1768
+1 2 807
+1 3 189
+1 4 47
+2 1 946
+2 2 1387
+2 3 746
+2 4 53
+3 1 115
+3 2 438
+3 3 288
+3 4 16
+END DATA.
+CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,6800.000,100.0%,.000,.0%,6800.000,100.0%
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.208,.010,18.793,.000
+,,x Dependent,.224,.013,16.076,.000
+,,y Dependent,.192,.012,14.438,.000
+,Goodman and Kruskal tau,x Dependent,.089,,,
+,,y Dependent,.081,,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - treatment of ties])
+AT_DATA([lambda.sps], [dnl
+SET FORMAT F8.3.
+
+* From Douglas Bonett.
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 225
+1 2 43
+1 3 216
+2 1 3
+2 2 1
+2 3 12
+END DATA.
+
+CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,500.000,100.0%,.000,.0%,500.000,100.0%
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.031,.013,2.336,.019
+,,x Dependent,.000,.000,NaN,NaN
+,,y Dependent,.033,.014,2.336,.019
+,Goodman and Kruskal tau,x Dependent,.012,,,
+,,y Dependent,.009,,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Somers' D, Tau-B, Tau-C, Gamma - 1])
+AT_DATA([somersd.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://stats.stackexchange.com/questions/72203/problem-with-calculating-asymptotic-standard-error-for-somers-d.
+DATA LIST LIST NOTABLE/x y * w (F10.6).
+WEIGHT BY w.
+BEGIN DATA.
+1 1 0.000025
+1 2 0.0001
+1 3 0.001
+1 4 0.0025
+1 5 0.004
+1 6 0.0075
+1 7 0.0125
+2 1 0.049975
+2 2 0.0999
+2 3 0.199
+2 4 0.2475
+2 5 0.196
+2 6 0.1425
+2 7 0.0375
+END DATA.
+CROSSTABS x BY y/STATISTICS=D/CELLS=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt somersd.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,1.000000,100.0%,.000000,.0%,1.000000,100.0%
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,-.084,,-.149,.882
+,,x Dependent,-.045,.300,-.149,.882
+,,y Dependent,-.684,2.378,-.149,.882
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Somers' D, Tau-B, Tau-C, Gamma - 2])
+AT_DATA([somersd.sps], [dnl
+SET FORMAT F8.3.
+
+* From http://uregina.ca/~gingrich/gamma.pdf.
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 34
+1 2 24
+1 3 15
+2 1 42
+2 2 74
+2 3 67
+3 1 28
+3 2 111
+3 3 292
+END DATA.
+CROSSTABS x BY y/STATISTICS=BTAU CTAU GAMMA D/CELLS=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt somersd.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,687.000,100.0%,.000,.0%,687.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,.372,.033,10.669
+,Kendall's tau-c,.310,.029,10.669
+,Gamma,.591,.043,10.669
+N of Valid Cases,,687.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,.371,,10.669,.000
+,,x Dependent,.351,.032,10.669,.000
+,,y Dependent,.394,.035,10.669,.000
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Somers' D, Tau-B, Tau-C, Gamma - 3])
+AT_DATA([ordinal.sps], [dnl
+SET FORMAT F8.3.
+
+* From https://www.iup.edu/WorkArea/DownloadAsset.aspx?id=9829, "Case 1".
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 2 40
+2 3 80
+3 4 30
+END DATA.
+CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
+
+* Same site, case 2.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 40
+2 3 80
+3 4 30
+END DATA.
+CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
+
+* Same site, case 3.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 4 40
+2 3 80
+3 2 30
+END DATA.
+CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
+
+* Same site, case 4.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 20
+1 2 20
+2 3 80
+3 4 30
+END DATA.
+CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
+
+* Same site, case 5.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 2 40
+2 2 80
+3 2 29
+3 3 1
+END DATA.
+CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
+
+* Same site, case 6.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 3
+1 2 6
+1 3 28
+1 4 61
+2 1 4
+2 2 5
+2 3 21
+2 4 20
+END DATA.
+CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
+
+* Same site, case 7.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 38
+1 2 6
+1 3 3
+1 4 51
+2 1 4
+2 2 20
+2 3 21
+2 4 5
+END DATA.
+CROSSTABS x BY y/STATISTICS=LAMBDA D PHI GAMMA/CELLS=NONE.
+
+* Same site, case 8.
+DATA LIST LIST NOTABLE /x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 2
+1 2 3
+1 3 5
+1 4 1
+2 1 2
+2 2 16
+2 3 3
+2 4 6
+3 1 3
+3 2 10
+3 3 35
+3 4 27
+4 1 6
+4 2 15
+4 3 33
+4 4 45
+END DATA.
+CROSSTABS x BY y/STATISTICS=LAMBDA D PHI BTAU/CELLS=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt ordinal.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,150.000,100.0%,.000,.0%,150.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,1.000,.000,24.841
+,Gamma,1.000,.000,24.841
+N of Valid Cases,,150.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,1.000,,24.841,.000
+,,x Dependent,1.000,.000,24.841,.000
+,,y Dependent,1.000,.000,24.841,.000
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,150.000,100.0%,.000,.0%,150.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,1.000,.000,24.841
+,Gamma,1.000,.000,24.841
+N of Valid Cases,,150.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,1.000,,24.841,.000
+,,x Dependent,1.000,.000,24.841,.000
+,,y Dependent,1.000,.000,24.841,.000
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,150.000,100.0%,.000,.0%,150.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,-1.000,.000,-24.841
+,Gamma,-1.000,.000,-24.841
+N of Valid Cases,,150.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,-1.000,,-24.841,.000
+,,x Dependent,-1.000,.000,-24.841,.000
+,,y Dependent,-1.000,.000,-24.841,.000
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,150.000,100.0%,.000,.0%,150.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,.972,.007,24.841
+,Gamma,1.000,.000,24.841
+N of Valid Cases,,150.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,.971,,24.841,.000
+,,x Dependent,.944,.013,24.841,.000
+,,y Dependent,1.000,.000,24.841,.000
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,150.000,100.0%,.000,.0%,150.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,.119,.059,1.009
+,Gamma,1.000,.000,1.009
+N of Valid Cases,,150.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,.035,,1.009,.313
+,,x Dependent,.805,.032,1.009,.313
+,,y Dependent,.018,.017,1.009,.313
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,148.000,100.0%,.000,.0%,148.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Ordinal by Ordinal,Kendall's tau-b,-.208,.078,-2.641
+,Gamma,-.381,.130,-2.641
+N of Valid Cases,,148.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Ordinal by Ordinal,Somers' d,Symmetric,-.206,,-2.641,.008
+,,x Dependent,-.182,.069,-2.641,.008
+,,y Dependent,-.237,.089,-2.641,.008
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,148.000,100.0%,.000,.0%,148.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Nominal by Nominal,Phi,.731,,
+,Cramer's V,.731,,
+Ordinal by Ordinal,Gamma,-.110,.107,-1.022
+N of Valid Cases,,148.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.338,.059,4.743,.000
+,,x Dependent,.640,.085,4.875,.000
+,,y Dependent,.174,.050,3.248,.001
+,Goodman and Kruskal tau,x Dependent,.534,,,
+,,y Dependent,.167,,,
+Ordinal by Ordinal,Somers' d,Symmetric,-.074,,-1.022,.307
+,,x Dependent,-.060,.059,-1.022,.307
+,,y Dependent,-.096,.094,-1.022,.307
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,212.000,100.0%,.000,.0%,212.000,100.0%
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Nominal by Nominal,Phi,.432,,
+,Cramer's V,.249,,
+Ordinal by Ordinal,Kendall's tau-b,.209,.062,3.338
+N of Valid Cases,,212.000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.102,.067,1.473,.141
+,,x Dependent,.027,.087,.302,.763
+,,y Dependent,.165,.065,2.349,.019
+,Goodman and Kruskal tau,x Dependent,.051,,,
+,,y Dependent,.068,,,
+Ordinal by Ordinal,Somers' d,Symmetric,.209,,3.338,.001
+,,x Dependent,.202,.060,3.338,.001
+,,y Dependent,.217,.064,3.338,.001
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS Cohens Kappa])
+
+dnl Example from Wood J. M.
+dnl "Understanding and Computing Cohen's Kappa: A Tutorial"
+dnl WebPsychEmpiricist. Oct 3 2007
+AT_DATA([kappa.sps], [dnl
+SET FORMAT=F8.3.
+
+data list notable list /p1 * p2 * w *.
+begin data.
+0 0 18
+1 0 1
+0 1 1
+end data.
+
+weight by w.
+
+crosstabs /table = p1 by p2
+ /statistics = kappa
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt kappa.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+p1 × p2,20.000,100.0%,.000,.0%,20.000,100.0%
+
+Table: p1 × p2
+,,,p2,,Total
+,,,.000,1.000,
+p1,.000,Count,18.000,1.000,19.000
+,1.000,Count,1.000,.000,1.000
+Total,,Count,19.000,1.000,20.000
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Measure of Agreement,Kappa,-.053,.037,-.235
+N of Valid Cases,,20.000,,
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([CROSSTABS many statistics])
+AT_DATA([crosstabs.sps], [dnl
+SET FORMAT=F8.4.
+
+* From http://www4.stat.ncsu.edu/~dzhang2/st744/table3.9.lst.txt.
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 25
+1 2 25
+1 3 12
+2 2 1
+2 3 3
+END DATA.
+CROSSTABS x BY y/STATISTICS=CHISQ PHI CC LAMBDA UC BTAU CTAU GAMMA D CORR/CELLS=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,66.0000,100.0%,.0000,.0%,66.0000,100.0%
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed)
+Pearson Chi-Square,6.9562,2.0000,.031
+Likelihood Ratio,6.6901,2.0000,.035
+Linear-by-Linear Association,5.8450,1.0000,.016
+N of Valid Cases,66.0000,,
+
+Table: Symmetric Measures
+,,Value,Asymp. Std. Error,Approx. T
+Nominal by Nominal,Phi,.3246,,
+,Cramer's V,.3246,,
+,Contingency Coefficient,.3088,,
+Ordinal by Ordinal,Kendall's tau-b,.2752,.0856,1.9920
+,Kendall's tau-c,.1497,.0751,1.9920
+,Gamma,.8717,.1250,1.9920
+,Spearman Correlation,.2908,.0906,2.4311
+Interval by Interval,Pearson's R,.2999,.0973,2.5147
+N of Valid Cases,,66.0000,,
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.0455,.1629,.2723,.785
+,,x Dependent,.0000,.0000,NaN,NaN
+,,y Dependent,.0500,.1791,.2723,.785
+,Goodman and Kruskal tau,x Dependent,.1054,,,
+,,y Dependent,.0434,,,
+,Uncertainty Coefficient,Symmetric,.0780,.0474,,
+,,x Dependent,.2217,.1062,1.5373,
+,,y Dependent,.0473,.0306,1.5373,
+Ordinal by Ordinal,Somers' d,Symmetric,.1960,,1.9920,.046
+,,x Dependent,.1152,.0572,1.9920,.046
+,,y Dependent,.6573,.1417,1.9920,.046
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS uncertainy coefficient])
+AT_DATA([uc.sps], [dnl
+* From http://groups.chass.utoronto.ca/pol242/5bMeasuringAssociation.htm.
+SET FORMAT=F8.3.
+
+DATA LIST LIST NOTABLE/x y w.
+WEIGHT BY w.
+BEGIN DATA.
+1 1 416
+1 2 121
+2 1 335
+2 2 2
+3 1 112
+3 2 1
+END DATA.
+CROSSTABS x BY y/STATISTICS=LAMBDA UC/CELLS=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt uc.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x × y,987.000,100.0%,.000,.0%,987.000,100.0%
+
+Table: Directional Measures
+,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
+Nominal by Nominal,Lambda,Symmetric,.000,.000,NaN,NaN
+,,x Dependent,.000,.000,NaN,NaN
+,,y Dependent,.000,.000,NaN,NaN
+,Goodman and Kruskal tau,x Dependent,.076,,,
+,,y Dependent,.108,,,
+,Uncertainty Coefficient,Symmetric,.105,.012,,
+,,x Dependent,.073,.009,7.890,
+,,y Dependent,.184,.019,7.890,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS estimated risk])
+dnl Example data and expected output from
+dnl http://www.psychology.nottingham.ac.uk/staff/ddc/c8cxpa/further/Project_resources/SPSSCrosstabW.pdf
+AT_DATA([risk.sps], [dnl
+DATA LIST LIST /factor disease count (F8.0).
+WEIGHT BY count.
+VALUE LABELS /factor 0 'Placebo' 1 'Aspirin'
+ /disease 1 'No' 0 'Yes'.
+BEGIN DATA.
+0 1 80
+0 0 20
+1 1 135
+1 0 15
+END DATA.
+CROSSTABS factor BY disease/STATISTICS=RISK CHISQ.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt risk.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+factor,F8.0
+disease,F8.0
+count,F8.0
+
+Table: Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+factor × disease,250,100.0%,0,.0%,250,100.0%
+
+Table: factor × disease
+,,,disease,,Total
+,,,Yes,No,
+factor,Placebo,Count,20,80,100
+,Aspirin,Count,15,135,150
+Total,,Count,35,215,250
+
+Table: Chi-Square Tests
+,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
+Pearson Chi-Square,4.98,1,.026,,
+Likelihood Ratio,4.88,1,.027,,
+Fisher's Exact Test,,,,.039,.021
+Continuity Correction,4.19,1,.041,,
+Linear-by-Linear Association,4.96,1,.026,,
+N of Valid Cases,250,,,,
+
+Table: Risk Estimate
+,Value,95% Confidence Interval,
+,,Lower,Upper
+Odds Ratio for factor (Placebo / Aspirin),2.25,2.25,2.25
+For cohort disease = Yes,1.08,1.08,1.08
+For cohort disease = No,.99,.99,.99
+N of Valid Cases,250.00,,
+])
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS barchart])
+AT_DATA([bc.sps], [dnl
+SET FORMAT=F8.3.
+
+DATA LIST LIST NOTABLE /x (a20) y (f8) z (f8) w (f8) .
+BEGIN DATA.
+This 1 0 416
+That 2 0 121
+Other 2 0 335
+This 2 0 231
+That 3 0 112
+Other 4 0 130
+This 1 1 160
+That 2 1 211
+Other 2 1 352
+This 2 1 212
+That 3 1 121
+Other 4 1 101
+END DATA.
+
+WEIGHT BY w.
+
+CROSSTABS
+ /table x BY y BY z
+ /table x BY y
+ /barchart.
+])
+
+AT_CHECK([pspp -O format=txt -o xxx bc.sps], [0], [ignore])
+
+AT_CHECK([test -e xxx-1.png], [0], [ignore])
+AT_CHECK([test -e xxx-2.png], [0], [ignore])
+
+AT_CHECK([diff xxx-1.png xxx-2.png], [0], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([CROSSTABS syntax errors])
+AT_DATA([crosstabs.sps], [dnl
+DATA LIST LIST NOTABLE/x y v1 to v100.
+CROSSTABS TABLES=x BY y/VARIABLES **.
+CROSSTABS VARIABLES=**.
+CROSSTABS VARIABLES=x **.
+CROSSTABS VARIABLES=x (**).
+CROSSTABS VARIABLES=x (1,**).
+CROSSTABS VARIABLES=x (1,5**).
+CROSSTABS MISSING=**.
+CROSSTABS COUNT=**.
+CROSSTABS FORMAT=**.
+CROSSTABS CELLS=**.
+CROSSTABS STATISTICS=**.
+CROSSTABS **.
+CROSSTABS
+ v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100.
+CROSSTABS BARCHART.
+CROSSTABS x BY y/MISSING=REPORT.
+])
+AT_CHECK([pspp -O format=csv crosstabs.sps], [1], [dnl
+"crosstabs.sps:2.25-2.33: error: CROSSTABS: VARIABLES must be specified before TABLES.
+ 2 | CROSSTABS TABLES=x BY y/VARIABLES **.
+ | ^~~~~~~~~"
+
+"crosstabs.sps:3.21-3.22: error: CROSSTABS: Syntax error expecting variable name.
+ 3 | CROSSTABS VARIABLES=**.
+ | ^~"
+
+"crosstabs.sps:4.23-4.24: error: CROSSTABS: Syntax error expecting `('.
+ 4 | CROSSTABS VARIABLES=x **.
+ | ^~"
+
+"crosstabs.sps:5.24-5.25: error: CROSSTABS: Syntax error expecting integer.
+ 5 | CROSSTABS VARIABLES=x (**).
+ | ^~"
+
+"crosstabs.sps:6.26-6.27: error: CROSSTABS: Syntax error expecting positive integer.
+ 6 | CROSSTABS VARIABLES=x (1,**).
+ | ^~"
+
+"crosstabs.sps:7.27-7.28: error: CROSSTABS: Syntax error expecting `)'.
+ 7 | CROSSTABS VARIABLES=x (1,5**).
+ | ^~"
+
+"crosstabs.sps:8.19-8.20: error: CROSSTABS: Syntax error expecting TABLE, INCLUDE, or REPORT.
+ 8 | CROSSTABS MISSING=**.
+ | ^~"
+
+"crosstabs.sps:9.17-9.18: error: CROSSTABS: Syntax error expecting ASIS, CASE, CELL, ROUND, or TRUNCATE.
+ 9 | CROSSTABS COUNT=**.
+ | ^~"
+
+"crosstabs.sps:10.18-10.19: error: CROSSTABS: Syntax error expecting AVALUE, DVALUE, TABLES, or NOTABLES.
+ 10 | CROSSTABS FORMAT=**.
+ | ^~"
+
+"crosstabs.sps:11.17-11.18: error: CROSSTABS: Syntax error expecting COUNT, EXPECTED, ROW, COLUMN, TOTAL, RESIDUAL, SRESIDUAL, or ASRESIDUAL.
+ 11 | CROSSTABS CELLS=**.
+ | ^~"
+
+"crosstabs.sps:12.22-12.23: error: CROSSTABS: Syntax error expecting one of the following: CHISQ, PHI, CC, LAMBDA, UC, BTAU, CTAU, RISK, GAMMA, D, KAPPA, ETA, CORR.
+ 12 | CROSSTABS STATISTICS=**.
+ | ^~"
+
+"crosstabs.sps:13.11-13.12: error: CROSSTABS: Syntax error expecting subcommand name or variable name.
+ 13 | CROSSTABS **.
+ | ^~"
+
+"crosstabs.sps:15.8-16.59: error: CROSSTABS: Too many cross-tabulation variables or dimensions.
+ 15 | v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ 16 | BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
+ | -----------------------------------------------------------"
+
+crosstabs.sps:21: error: CROSSTABS: At least one crosstabulation must be requested (using the TABLES subcommand).
+
+"crosstabs.sps:22.26-22.31: warning: CROSSTABS: Missing mode REPORT not allowed in general mode. Assuming MISSING=TABLE.
+ 22 | CROSSTABS x BY y/MISSING=REPORT.
+ | ^~~~~~"
+
+error: CROSSTABS: At end of input: Syntax error expecting `BEGIN DATA'.
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+AT_BANNER([CTABLES])
+
+dnl Known bugs:
+dnl TOTAL interaction with PCOMPUTE, e.g. the following
+dnl CTABLES
+dnl /PCOMPUTE &all_drivers=EXPR([1 THRU 2] + [3 THRU 4])
+dnl /PPROPERTIES &all_drivers LABEL='All Drivers'
+dnl /PCOMPUTE &pct_not_drivers=EXPR([5] / ([1 THRU 2] + [3 THRU 4] + [5]) * 100)
+dnl /PPROPERTIES &pct_not_drivers LABEL='% Not Drivers' FORMAT=COUNT PCT40.1
+dnl /TABLE=qn1 BY qns3a
+dnl /CATEGORIES VARIABLES=qns3a TOTAL=YES
+dnl /CATEGORIES VARIABLES=qn1 [1 THRU 2, SUBTOTAL='Frequent Drivers',
+dnl 3 THRU 4, SUBTOTAL='Infrequent Drivers',
+dnl &all_drivers, 5, &pct_not_drivers,
+dnl MISSING, SUBTOTAL='Not Drivers or Missing'].
+dnl yields gaps in the Total column:
+dnl ╭─────────────────────────────────────────────────────────────────────────┬──────────────────╮
+dnl │ │ S3a. GENDER: │
+dnl │ ├─────┬──────┬─────┤
+dnl │ │ Male│Female│Total│
+dnl │ ├─────┼──────┼─────┤
+dnl │ │Count│ Count│Count│
+dnl ├─────────────────────────────────────────────────────────────────────────┼─────┼──────┼─────┤
+dnl │ 1. How often do you usually drive a car or other Every day │ 2305│ 2362│ 4667│
+dnl │motor vehicle? Several days a week │ 440│ 834│ 1274│
+dnl │ Frequent Drivers │ 2745│ 3196│ │
+dnl │ Once a week or less │ 125│ 236│ 361│
+dnl │ Only certain times a │ 58│ 72│ 130│
+dnl │ year │ │ │ │
+dnl │ Infrequent Drivers │ 183│ 308│ │
+dnl │ All Drivers │ 2928│ 3504│ │
+dnl │ Never │ 192│ 348│ 540│
+dnl │ % Not Drivers │ 6.2%│ 9.0%│ │
+dnl │ Don't know │ 3│ 5│ 8│
+dnl │ Refused │ 9│ 10│ 19│
+dnl │ Not Drivers or │ 204│ 363│ │
+dnl │ Missing │ │ │ │
+dnl ╰─────────────────────────────────────────────────────────────────────────┴─────┴──────┴─────╯
+dnl Features not yet implemented:
+dnl - Multiple response sets
+dnl - MRSETS subcommand.
+dnl - CATEGORIES: Special case for explicit category specifications and multiple dichotomy sets.
+dnl - SIGTEST
+dnl - COMPARETEST
+dnl - Summary functions:
+dnl * .LCL and .UCL suffixes.
+dnl * .SE suffixes.
+dnl - CATEGORIES:
+dnl * Data-dependent sorting.
+
+AT_SETUP([CTABLES parsing])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /FORMAT MINCOLWIDTH=10 MAXCOLWIDTH=20 UNITS=POINTS EMPTY=ZERO MISSING="x"
+ /FORMAT MINCOLWIDTH=DEFAULT MAXCOLWIDTH=DEFAULT UNITS=INCHES EMPTY=BLANK MISSING="."
+ /FORMAT UNITS=CM EMPTY="(-)"
+ /VLABELS VARIABLES=qn1 DISPLAY=DEFAULT
+ /VLABELS VARIABLES=qn17 DISPLAY=NAME
+ /VLABELS VARIABLES=qns3a DISPLAY=LABEL
+ /VLABELS VARIABLES=qnd1 DISPLAY=BOTH
+ /VLABELS VARIABLES=qn20 DISPLAY=NONE
+ /MRSETS COUNTDUPLICATES=NO
+ /MRSETS COUNTDUPLICATES=YES
+ /SMISSING VARIABLE
+ /SMISSING LISTWISE
+ /WEIGHT VARIABLE=qns3a
+ /HIDESMALLCOUNTS
+ /HIDESMALLCOUNTS COUNT=10
+ /TABLE qnsa1
+ /SLABELS POSITION=COLUMN VISIBLE=YES
+ /SLABELS VISIBLE=NO POSITION=ROW
+ /SLABELS POSITION=LAYER
+ /CLABELS AUTO
+ /CLABELS ROWLABELS=OPPOSITE
+ /CRITERIA CILEVEL=50
+ /CATEGORIES VARIABLES=qn1 qn17
+ ORDER=A KEY=VALUE MISSING=INCLUDE TOTAL=YES LABEL="xyzzy"
+ POSITION=BEFORE EMPTY=INCLUDE.
+CTABLES /TABLE qnsa1 /CLABELS ROWLABELS=LAYER.
+CTABLES /TABLE qnsa1 /CLABELS COLLABELS=OPPOSITE.
+CTABLES /TABLE qnsa1 /CLABELS COLLABELS=LAYER.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+Count
+╭───────────────────┬────┬────╮
+│ │ RDD│CELL│
+├───────────────────┼────┼────┤
+│Sa1. SAMPLE SOURCE:│5392│1607│
+╰───────────────────┴────┴────╯
+
+ Custom Tables
+RDD
+╭───────────────────┬─────╮
+│ │Count│
+├───────────────────┼─────┤
+│Sa1. SAMPLE SOURCE:│ 5392│
+╰───────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────┬─────╮
+│ │Count│
+├────────────────────────┼─────┤
+│Sa1. SAMPLE SOURCE: RDD │ 5392│
+│ CELL│ 1607│
+╰────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────┬─────╮
+│ │Count│
+├────────────────────────┼─────┤
+│Sa1. SAMPLE SOURCE: RDD │ 5392│
+│ CELL│ 1607│
+╰────────────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES parsing - negative])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES.
+CTABLES /FORMAT MINCOLWIDTH='foo'.
+CTABLES /TABLE qn1 [**].
+CTABLES /TABLE qn1 [NOTAFUNCTION].
+CTABLES /TABLE (qn1.
+CTABLES /TABLE **.
+CTABLES /TABLE NOTAVAR.
+STRING string(A8).
+CTABLES /TABLE string[S].
+CTABLES /TABLE qn1 [PTILE 101].
+CTABLES /TABLE qn1 [MEAN F0.1].
+CTABLES /TABLE qn1 [MEAN NEGPAREN1.2].
+CTABLES /TABLE qn1 [MEAN NEGPAREN3.4].
+CTABLES /TABLE qn1 [MEAN TOTALS].
+CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [SUBTOTAL=x].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO **].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO THRU x].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1 THRU **].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['x' THRU **].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&**].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&x].
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 101).
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 MISSING=**.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 TOTAL=**.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 LABEL=**.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 POSITION=**.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 EMPTY=**.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 **.
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1,2,3] **.
+CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[0]).
+CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[1**]).
+CTABLES /PCOMPUTE &k=EXPR([LO **]).
+CTABLES /PCOMPUTE &k=EXPR([LO THRU **]).
+CTABLES /PCOMPUTE &k=EXPR([1 THRU **]).
+CTABLES /PCOMPUTE &k=EXPR([1**]).
+CTABLES /PCOMPUTE &k=EXPR((1x)).
+CTABLES /PCOMPUTE **k.
+CTABLES /PCOMPUTE &1.
+CTABLES /PCOMPUTE &k**.
+CTABLES /PCOMPUTE &k=**.
+CTABLES /PCOMPUTE &k=EXPR**.
+CTABLES /PCOMPUTE &k=EXPR(1x).
+CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
+CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=NOTAFUNCTION.
+CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=PTILE **.
+CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k LABEL=**.
+CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k HIDESOURCECATS=**.
+CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k **.
+CTABLES /FORMAT EMPTY=**.
+CTABLES /FORMAT MISSING=**.
+CTABLES /FORMAT **.
+CTABLES /FORMAT MINCOLWIDTH=20 MAXCOLWIDTH=10/.
+CTABLES /VLABELS **.
+CTABLES /VLABELS VARIABLES=NOTAVAR.
+CTABLES /VLABELS VARIABLES=qn1 **.
+CTABLES /VLABELS VARIABLES=qn1 DISPLAY=**.
+CTABLES /MRSETS **.
+CTABLES /MRSETS COUNTDUPLICATES=**.
+CTABLES /SMISSING **.
+CTABLES /WEIGHT **.
+CTABLES /WEIGHT VARIABLE=NOTAVAR.
+CTABLES /HIDESMALLCOUNTS COUNT=1.
+CTABLES /QUUX.
+CTABLES /HIDESMALLCOUNTS COUNT=2.
+CTABLES /TABLE qn1**.
+CTABLES /TABLE qn1 /SLABELS POSITION=**.
+CTABLES /TABLE qn1 /SLABELS VISIBLE=**.
+CTABLES /TABLE qn1 /SLABELS **.
+CTABLES /TABLE qn1 /CLABELS ROWLABELS=**.
+CTABLES /TABLE qn1 /CLABELS COLLABELS=**.
+CTABLES /TABLE qn1 /CLABELS **.
+CTABLES /TABLE qn1 /CRITERIA **.
+CTABLES /TABLE qn1 /CRITERIA CILEVEL=101.
+CTABLES /TABLE qn1 /TITLES **.
+CTABLES /TABLE qn1 /SIGTEST TYPE=**.
+CTABLES /TABLE qn1 /SIGTEST ALPHA=**.
+CTABLES /TABLE qn1 /SIGTEST INCLUDEMRSETS=**.
+CTABLES /TABLE qn1 /SIGTEST CATEGORIES=**.
+CTABLES /TABLE qn1 /SIGTEST **.
+CTABLES /TABLE qn1 /COMPARETEST TYPE=**.
+CTABLES /TABLE qn1 /COMPARETEST ALPHA=**.
+CTABLES /TABLE qn1 /COMPARETEST ALPHA=0,5.
+CTABLES /TABLE qn1 /COMPARETEST ADJUST=**.
+CTABLES /TABLE qn1 /COMPARETEST INCLUDEMRSETS=**.
+CTABLES /TABLE qn1 /COMPARETEST MEANSVARIANCE=**.
+CTABLES /TABLE qn1 /COMPARETEST CATEGORIES=**.
+CTABLES /TABLE qn1 /COMPARETEST MERGE=**.
+CTABLES /TABLE qn1 /COMPARETEST STYLE=**.
+CTABLES /TABLE qn1 /COMPARETEST SHOWSIG=**.
+CTABLES /TABLE qn1 /COMPARETEST **.
+CTABLES /TABLE qn1 /FORMAT.
+CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CLABELS COLLABELS=OPPOSITE.
+CTABLES /TABLE qn20 > qnd1.
+CTABLES /TABLE qn1 [ROWPCT] > qnsa1.
+NUMERIC datetime (DATETIME17.0).
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=datetime ['123'].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [1],
+[[ctables.sps:2.8: error: CTABLES: Syntax error expecting `/'.
+ 2 | CTABLES.
+ | ^
+
+ctables.sps:3.29-3.33: error: CTABLES: Syntax error expecting non-negative
+number for MINCOLWIDTH.
+ 3 | CTABLES /FORMAT MINCOLWIDTH='foo'.
+ | ^~~~~
+
+ctables.sps:4.21-4.22: error: CTABLES: Syntax error expecting identifier.
+ 4 | CTABLES /TABLE qn1 [**].
+ | ^~
+
+ctables.sps:5.21-5.32: error: CTABLES: Syntax error expecting summary function
+name.
+ 5 | CTABLES /TABLE qn1 [NOTAFUNCTION].
+ | ^~~~~~~~~~~~
+
+ctables.sps:6.20: error: CTABLES: Syntax error expecting `@:}@'.
+ 6 | CTABLES /TABLE @{:@qn1.
+ | ^
+
+ctables.sps:7.16-7.17: error: CTABLES: Syntax error expecting identifier.
+ 7 | CTABLES /TABLE **.
+ | ^~
+
+ctables.sps:8.16-8.22: error: CTABLES: NOTAVAR is not a variable name.
+ 8 | CTABLES /TABLE NOTAVAR.
+ | ^~~~~~~
+
+ctables.sps:10.16-10.24: error: CTABLES: Cannot use string variable string as a
+scale variable.
+ 10 | CTABLES /TABLE string[S].
+ | ^~~~~~~~~
+
+ctables.sps:11.27-11.29: error: CTABLES: Syntax error expecting number between 0
+and 100 for PTILE.
+ 11 | CTABLES /TABLE qn1 [PTILE 101].
+ | ^~~
+
+ctables.sps:12.26-12.29: error: CTABLES: Output format F0.1 specifies width 0,
+but F requires a width between 1 and 40.
+ 12 | CTABLES /TABLE qn1 [MEAN F0.1].
+ | ^~~~
+
+ctables.sps:13.26-13.36: error: CTABLES: Output format NEGPAREN requires width 2
+or greater.
+ 13 | CTABLES /TABLE qn1 [MEAN NEGPAREN1.2].
+ | ^~~~~~~~~~~
+
+ctables.sps:14.26-14.36: error: CTABLES: Output format NEGPAREN requires width
+greater than decimals.
+ 14 | CTABLES /TABLE qn1 [MEAN NEGPAREN3.4].
+ | ^~~~~~~~~~~
+
+ctables.sps:15.21-15.24: error: CTABLES: Summary function MEAN applies only to
+scale variables.
+ 15 | CTABLES /TABLE qn1 [MEAN TOTALS].
+ | ^~~~
+
+ctables.sps:15.16-15.18: note: CTABLES: 'QN1' is not a scale variable.
+ 15 | CTABLES /TABLE qn1 [MEAN TOTALS].
+ | ^~~
+
+ctables.sps:15.32: error: CTABLES: Syntax error expecting `@<:@'.
+ 15 | CTABLES /TABLE qn1 [MEAN TOTALS].
+ | ^
+
+ctables.sps:16.21-16.24: error: CTABLES: Summary function MEAN applies only to
+scale variables.
+ 16 | CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
+ | ^~~~
+
+ctables.sps:16.16-16.18: note: CTABLES: 'QN1' is not a scale variable.
+ 16 | CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
+ | ^~~
+
+ctables.sps:16.40: error: CTABLES: Syntax error expecting `@:>@'.
+ 16 | CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
+ | ^
+
+ctables.sps:17.56: error: CTABLES: Syntax error expecting string.
+ 17 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [SUBTOTAL=x].
+ | ^
+
+ctables.sps:18.50-18.51: error: CTABLES: Syntax error expecting THRU.
+ 18 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO **].
+ | ^~
+
+ctables.sps:19.55: error: CTABLES: Syntax error expecting number.
+ 19 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO THRU x].
+ | ^
+
+ctables.sps:20.54-20.55: error: CTABLES: Syntax error expecting number.
+ 20 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1 THRU **].
+ | ^~
+
+ctables.sps:21.56-21.57: error: CTABLES: Syntax error expecting string.
+ 21 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['x' THRU **].
+ | ^~
+
+ctables.sps:22.48-22.49: error: CTABLES: Syntax error expecting identifier.
+ 22 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&**].
+ | ^~
+
+ctables.sps:23.47-23.48: error: CTABLES: Unknown postcompute &x.
+ 23 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&x].
+ | ^~
+
+ctables.sps:24.61-24.63: error: CTABLES: Syntax error expecting number between 0
+and 100 for PTILE.
+ 24 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 101).
+ | ^~~
+
+ctables.sps:25.58: error: CTABLES: Syntax error expecting `@:}@'.
+ 25 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1.
+ | ^
+
+ctables.sps:26.54: error: CTABLES: Syntax error expecting `@{:@'.
+ 26 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN.
+ | ^
+
+ctables.sps:27.54-27.55: error: CTABLES: Syntax error expecting INCLUDE or
+EXCLUDE.
+ 27 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 MISSING=**.
+ | ^~
+
+ctables.sps:28.52-28.53: error: CTABLES: Syntax error expecting YES or NO.
+ 28 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 TOTAL=**.
+ | ^~
+
+ctables.sps:29.52-29.53: error: CTABLES: Syntax error expecting string.
+ 29 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 LABEL=**.
+ | ^~
+
+ctables.sps:30.55-30.56: error: CTABLES: Syntax error expecting BEFORE or AFTER.
+ 30 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 POSITION=**.
+ | ^~
+
+ctables.sps:31.52-31.53: error: CTABLES: Syntax error expecting INCLUDE or
+EXCLUDE.
+ 31 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 EMPTY=**.
+ | ^~
+
+ctables.sps:32.46-32.47: error: CTABLES: Syntax error expecting ORDER, KEY,
+MISSING, TOTAL, LABEL, POSITION, or EMPTY.
+ 32 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 **.
+ | ^~
+
+ctables.sps:33.54-33.55: error: CTABLES: Syntax error expecting TOTAL, LABEL,
+POSITION, or EMPTY.
+ 33 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1,2,3] **.
+ | ^~
+
+ctables.sps:34.36: error: CTABLES: Syntax error expecting positive integer for
+SUBTOTAL.
+ 34 | CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[0]).
+ | ^
+
+ctables.sps:35.37-35.38: error: CTABLES: Syntax error expecting `@:>@'.
+ 35 | CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[1**]).
+ | ^~
+
+ctables.sps:36.31-36.32: error: CTABLES: Syntax error expecting THRU.
+ 36 | CTABLES /PCOMPUTE &k=EXPR([LO **]).
+ | ^~
+
+ctables.sps:37.36-37.37: error: CTABLES: Syntax error expecting number.
+ 37 | CTABLES /PCOMPUTE &k=EXPR([LO THRU **]).
+ | ^~
+
+ctables.sps:38.35-38.36: error: CTABLES: Syntax error expecting number.
+ 38 | CTABLES /PCOMPUTE &k=EXPR([1 THRU **]).
+ | ^~
+
+ctables.sps:39.29-39.30: error: CTABLES: Syntax error expecting `@:>@'.
+ 39 | CTABLES /PCOMPUTE &k=EXPR([1**]).
+ | ^~
+
+ctables.sps:40.29: error: CTABLES: Syntax error expecting `@:}@'.
+ 40 | CTABLES /PCOMPUTE &k=EXPR((1x)).
+ | ^
+
+ctables.sps:41.19-41.20: error: CTABLES: Syntax error expecting &.
+ 41 | CTABLES /PCOMPUTE **k.
+ | ^~
+
+ctables.sps:42.20: error: CTABLES: Syntax error expecting identifier.
+ 42 | CTABLES /PCOMPUTE &1.
+ | ^
+
+ctables.sps:43.21-43.22: error: CTABLES: Syntax error expecting `=EXPR@{:@'.
+ 43 | CTABLES /PCOMPUTE &k**.
+ | ^~
+
+ctables.sps:44.21-44.23: error: CTABLES: Syntax error expecting `=EXPR@{:@'.
+ 44 | CTABLES /PCOMPUTE &k=**.
+ | ^~~
+
+ctables.sps:45.21-45.27: error: CTABLES: Syntax error expecting `=EXPR@{:@'.
+ 45 | CTABLES /PCOMPUTE &k=EXPR**.
+ | ^~~~~~~
+
+ctables.sps:46.28: error: CTABLES: Syntax error expecting `@:}@'.
+ 46 | CTABLES /PCOMPUTE &k=EXPR(1x).
+ | ^
+
+ctables.sps:47.31-47.49: warning: CTABLES: New definition of &k will override
+the previous definition.
+ 47 | CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
+ | ^~~~~~~~~~~~~~~~~~~
+
+ctables.sps:47.10-47.28: note: CTABLES: This is the previous definition.
+ 47 | CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
+ | ^~~~~~~~~~~~~~~~~~~
+
+ctables.sps:47.50: error: CTABLES: Syntax error expecting `/'.
+ 47 | CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
+ | ^
+
+ctables.sps:48.53-48.64: error: CTABLES: Syntax error expecting summary function
+name.
+ 48 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=NOTAFUNCTION.
+ | ^~~~~~~~~~~~
+
+ctables.sps:49.59-49.60: error: CTABLES: Syntax error expecting number between 0
+and 100 for PTILE.
+ 49 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=PTILE **.
+ | ^~
+
+ctables.sps:50.52-50.53: error: CTABLES: Syntax error expecting string.
+ 50 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k LABEL=**.
+ | ^~
+
+ctables.sps:51.61-51.62: error: CTABLES: Syntax error expecting YES or NO.
+ 51 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k HIDESOURCECATS=**.
+ | ^~
+
+ctables.sps:52.46-52.47: error: CTABLES: Syntax error expecting LABEL, FORMAT,
+or HIDESOURCECATS.
+ 52 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k **.
+ | ^~
+
+ctables.sps:53.23-53.24: error: CTABLES: Syntax error expecting string.
+ 53 | CTABLES /FORMAT EMPTY=**.
+ | ^~
+
+ctables.sps:54.25-54.26: error: CTABLES: Syntax error expecting string.
+ 54 | CTABLES /FORMAT MISSING=**.
+ | ^~
+
+ctables.sps:55.17-55.18: error: CTABLES: Syntax error expecting MINCOLWIDTH,
+MAXCOLWIDTH, UNITS, EMPTY, or MISSING.
+ 55 | CTABLES /FORMAT **.
+ | ^~
+
+ctables.sps:56.17-56.45: error: CTABLES: MINCOLWIDTH must not be greater than
+MAXCOLWIDTH.
+ 56 | CTABLES /FORMAT MINCOLWIDTH=20 MAXCOLWIDTH=10/.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:57.18-57.19: error: CTABLES: Syntax error expecting VARIABLES.
+ 57 | CTABLES /VLABELS **.
+ | ^~
+
+ctables.sps:58.28-58.34: error: CTABLES: NOTAVAR is not a variable name.
+ 58 | CTABLES /VLABELS VARIABLES=NOTAVAR.
+ | ^~~~~~~
+
+ctables.sps:59.32-59.33: error: CTABLES: Syntax error expecting DISPLAY.
+ 59 | CTABLES /VLABELS VARIABLES=qn1 **.
+ | ^~
+
+ctables.sps:60.40-60.41: error: CTABLES: Syntax error expecting DEFAULT, NAME,
+LABEL, BOTH, or NONE.
+ 60 | CTABLES /VLABELS VARIABLES=qn1 DISPLAY=**.
+ | ^~
+
+ctables.sps:61.17-61.18: error: CTABLES: Syntax error expecting COUNTDUPLICATES.
+ 61 | CTABLES /MRSETS **.
+ | ^~
+
+ctables.sps:62.33-62.34: error: CTABLES: Syntax error expecting YES or NO.
+ 62 | CTABLES /MRSETS COUNTDUPLICATES=**.
+ | ^~
+
+ctables.sps:63.19-63.20: error: CTABLES: Syntax error expecting VARIABLE or
+LISTWISE.
+ 63 | CTABLES /SMISSING **.
+ | ^~
+
+ctables.sps:64.17-64.18: error: CTABLES: Syntax error expecting VARIABLE.
+ 64 | CTABLES /WEIGHT **.
+ | ^~
+
+ctables.sps:65.26-65.32: error: CTABLES: NOTAVAR is not a variable name.
+ 65 | CTABLES /WEIGHT VARIABLE=NOTAVAR.
+ | ^~~~~~~
+
+ctables.sps:66.32: error: CTABLES: Syntax error expecting integer 2 or greater
+for HIDESMALLCOUNTS COUNT.
+ 66 | CTABLES /HIDESMALLCOUNTS COUNT=1.
+ | ^
+
+ctables.sps:67.10-67.13: error: CTABLES: Syntax error expecting one of the
+following: FORMAT, VLABELS, MRSETS, SMISSING, PCOMPUTE, PPROPERTIES, WEIGHT,
+HIDESMALLCOUNTS, TABLE.
+ 67 | CTABLES /QUUX.
+ | ^~~~
+
+ctables.sps:68.33: error: CTABLES: Syntax error expecting `/'.
+ 68 | CTABLES /HIDESMALLCOUNTS COUNT=2.
+ | ^
+
+ctables.sps:69.19-69.20: error: CTABLES: Syntax error expecting `/'.
+ 69 | CTABLES /TABLE qn1**.
+ | ^~
+
+ctables.sps:70.38-70.39: error: CTABLES: Syntax error expecting COLUMN, ROW, or
+LAYER.
+ 70 | CTABLES /TABLE qn1 /SLABELS POSITION=**.
+ | ^~
+
+ctables.sps:71.37-71.38: error: CTABLES: Syntax error expecting YES or NO.
+ 71 | CTABLES /TABLE qn1 /SLABELS VISIBLE=**.
+ | ^~
+
+ctables.sps:72.29-72.30: error: CTABLES: Syntax error expecting POSITION or
+VISIBLE.
+ 72 | CTABLES /TABLE qn1 /SLABELS **.
+ | ^~
+
+ctables.sps:73.39-73.40: error: CTABLES: Syntax error expecting OPPOSITE or
+LAYER.
+ 73 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=**.
+ | ^~
+
+ctables.sps:74.39-74.40: error: CTABLES: Syntax error expecting OPPOSITE or
+LAYER.
+ 74 | CTABLES /TABLE qn1 /CLABELS COLLABELS=**.
+ | ^~
+
+ctables.sps:75.29-75.30: error: CTABLES: Syntax error expecting AUTO, ROWLABELS,
+or COLLABELS.
+ 75 | CTABLES /TABLE qn1 /CLABELS **.
+ | ^~
+
+ctables.sps:76.30-76.31: error: CTABLES: Syntax error expecting CILEVEL.
+ 76 | CTABLES /TABLE qn1 /CRITERIA **.
+ | ^~
+
+ctables.sps:77.38-77.40: error: CTABLES: Syntax error expecting number in
+@<:@0,100@:}@ for CILEVEL.
+ 77 | CTABLES /TABLE qn1 /CRITERIA CILEVEL=101.
+ | ^~~
+
+ctables.sps:78.28-78.29: error: CTABLES: Syntax error expecting CAPTION, CORNER,
+or TITLE.
+ 78 | CTABLES /TABLE qn1 /TITLES **.
+ | ^~
+
+ctables.sps:79.34-79.35: error: CTABLES: Syntax error expecting CHISQUARE.
+ 79 | CTABLES /TABLE qn1 /SIGTEST TYPE=**.
+ | ^~
+
+ctables.sps:80.35-80.36: error: CTABLES: Syntax error expecting number in @<:@0,1@:}@
+for ALPHA.
+ 80 | CTABLES /TABLE qn1 /SIGTEST ALPHA=**.
+ | ^~
+
+ctables.sps:81.43-81.44: error: CTABLES: Syntax error expecting YES or NO.
+ 81 | CTABLES /TABLE qn1 /SIGTEST INCLUDEMRSETS=**.
+ | ^~
+
+ctables.sps:82.40-82.41: error: CTABLES: Syntax error expecting ALLVISIBLE or
+SUBTOTALS.
+ 82 | CTABLES /TABLE qn1 /SIGTEST CATEGORIES=**.
+ | ^~
+
+ctables.sps:83.29-83.30: error: CTABLES: Syntax error expecting TYPE, ALPHA,
+INCLUDEMRSETS, or CATEGORIES.
+ 83 | CTABLES /TABLE qn1 /SIGTEST **.
+ | ^~
+
+ctables.sps:84.38-84.39: error: CTABLES: Syntax error expecting PROP or MEAN.
+ 84 | CTABLES /TABLE qn1 /COMPARETEST TYPE=**.
+ | ^~
+
+ctables.sps:85.39-85.40: error: CTABLES: Syntax error expecting number in (0,1)
+for ALPHA.
+ 85 | CTABLES /TABLE qn1 /COMPARETEST ALPHA=**.
+ | ^~
+
+ctables.sps:86.39: error: CTABLES: Syntax error expecting number in (0,1) for
+ALPHA.
+ 86 | CTABLES /TABLE qn1 /COMPARETEST ALPHA=0,5.
+ | ^
+
+ctables.sps:87.40-87.41: error: CTABLES: Syntax error expecting BONFERRONI, BH,
+or NONE.
+ 87 | CTABLES /TABLE qn1 /COMPARETEST ADJUST=**.
+ | ^~
+
+ctables.sps:88.47-88.48: error: CTABLES: Syntax error expecting YES or NO.
+ 88 | CTABLES /TABLE qn1 /COMPARETEST INCLUDEMRSETS=**.
+ | ^~
+
+ctables.sps:89.47-89.48: error: CTABLES: Syntax error expecting ALLCATS or
+TESTEDCATS.
+ 89 | CTABLES /TABLE qn1 /COMPARETEST MEANSVARIANCE=**.
+ | ^~
+
+ctables.sps:90.44-90.45: error: CTABLES: Syntax error expecting ALLVISIBLE or
+SUBTOTALS.
+ 90 | CTABLES /TABLE qn1 /COMPARETEST CATEGORIES=**.
+ | ^~
+
+ctables.sps:91.39-91.40: error: CTABLES: Syntax error expecting YES or NO.
+ 91 | CTABLES /TABLE qn1 /COMPARETEST MERGE=**.
+ | ^~
+
+ctables.sps:92.39-92.40: error: CTABLES: Syntax error expecting APA or SIMPLE.
+ 92 | CTABLES /TABLE qn1 /COMPARETEST STYLE=**.
+ | ^~
+
+ctables.sps:93.41-93.42: error: CTABLES: Syntax error expecting YES or NO.
+ 93 | CTABLES /TABLE qn1 /COMPARETEST SHOWSIG=**.
+ | ^~
+
+ctables.sps:94.33-94.34: error: CTABLES: Syntax error expecting one of the
+following: TYPE, ALPHA, ADJUST, INCLUDEMRSETS, MEANSVARIANCE, CATEGORIES, MERGE,
+STYLE, SHOWSIG.
+ 94 | CTABLES /TABLE qn1 /COMPARETEST **.
+ | ^~
+
+ctables.sps:95.21-95.26: error: CTABLES: Syntax error expecting TABLE, SLABELS,
+CLABELS, CRITERIA, CATEGORIES, TITLES, SIGTEST, or COMPARETEST.
+ 95 | CTABLES /TABLE qn1 /FORMAT.
+ | ^~~~~~
+
+ctables.sps:95.21-95.26: note: CTABLES: This subcommand must appear before
+TABLE.
+ 95 | CTABLES /TABLE qn1 /FORMAT.
+ | ^~~~~~
+
+ctables.sps:96: error: CTABLES: ROWLABELS and COLLABELS may not both be
+specified.
+
+ctables.sps:96.21-96.46: note: CTABLES: This is the first specification.
+ 96 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CLABELS
+COLLABELS=OPPOSITE.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:96.49-96.74: note: CTABLES: This is the second specification.
+ 96 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CLABELS
+COLLABELS=OPPOSITE.
+ |
+^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:97.16-97.26: error: CTABLES: Cannot nest scale variables.
+ 97 | CTABLES /TABLE qn20 > qnd1.
+ | ^~~~~~~~~~~
+
+ctables.sps:97.16-97.19: note: CTABLES: This is an outer scale variable.
+ 97 | CTABLES /TABLE qn20 > qnd1.
+ | ^~~~
+
+ctables.sps:97.23-97.26: note: CTABLES: This is an inner scale variable.
+ 97 | CTABLES /TABLE qn20 > qnd1.
+ | ^~~~
+
+ctables.sps:98.16-98.35: error: CTABLES: Summaries may only be requested for
+categorical variables at the innermost nesting level.
+ 98 | CTABLES /TABLE qn1 [ROWPCT] > qnsa1.
+ | ^~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:98.16-98.18: note: CTABLES: This outer categorical variable has a
+summary.
+ 98 | CTABLES /TABLE qn1 [ROWPCT] > qnsa1.
+ | ^~~
+
+ctables.sps:100.52-100.56: error: CTABLES: Failed to parse category
+specification as format DATETIME: Day (123) must be between 1 and 31..
+ 100 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=datetime ['123'].
+ | ^~~~~
+]])
+AT_CLEANUP
+
+AT_SETUP([CTABLES parsing - more negative])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+CTABLES /PCOMPUTE &pc=EXPR(TOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc, SUBTOTAL, SUBTOTAL].
+
+STRING string(A8).
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['string'].
+CTABLES /TABLE string /CATEGORIES VARIABLES=string [1].
+
+CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1).
+
+CTABLES /TABLE qnd1 /CLABELS ROWLABELS=OPPOSITE.
+CTABLES /TABLE qn1 + string /CLABELS ROWLABELS=OPPOSITE.
+CTABLES /TABLE qn1 + qnsa1 /CLABELS ROWLABELS=OPPOSITE.
+CTABLES /TABLE qn105ba + qn105bb /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn105ba [1,2,3].
+
+CTABLES /PCOMPUTE &x=EXPR(1**2**3).
+CTABLES /PCOMPUTE &x=EXPR([**]).
+CTABLES /PCOMPUTE &x=EXPR(**).
+
+CTABLES /TABLE.
+
+CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
+
+CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 50).
+
+CTABLES /TABLE $mrset.
+
+CTABLES /TABLE qn113 /SIGTEST TYPE=CHISQUARE.
+CTABLES /TABLE qn113 /COMPARETEST TYPE=PROP.
+
+CTABLES /TABLE qn113 [COUNT.UCL].
+
+CTABLES /TABLE qn1 /CATEGORIES **.
+
+CTABLES /TITLES.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [1],
+[[ctables.sps:2.76-2.78: error: CTABLES: Computed category &pc references a category not included in the category list.
+ 2 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+ | ^~~
+
+ctables.sps:2.28-2.35: note: CTABLES: This is the missing category.
+ 2 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+ | ^~~~~~~~
+
+ctables.sps:2.76-2.79: note: CTABLES: To fix the problem, add subtotals to the list of categories here.
+ 2 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+ | ^~~~
+
+ctables.sps:3.73-3.75: error: CTABLES: Computed category &pc references a category not included in the category list.
+ 3 | CTABLES /PCOMPUTE &pc=EXPR(TOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+ | ^~~
+
+ctables.sps:3.28-3.32: note: CTABLES: This is the missing category.
+ 3 | CTABLES /PCOMPUTE &pc=EXPR(TOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
+ | ^~~~~
+
+ctables.sps:3: note: CTABLES: To fix the problem, add TOTAL=YES to the variable's CATEGORIES specification.
+
+ctables.sps:4.76-4.99: error: CTABLES: These categories include 2 instances of SUBTOTAL or HSUBTOTAL, so references from
+computed categories must refer to subtotals by position, e.g. SUBTOTAL[1].
+ 4 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc, SUBTOTAL, SUBTOTAL].
+ | ^~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:4.28-4.35: note: CTABLES: This is the reference that lacks a position.
+ 4 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc, SUBTOTAL, SUBTOTAL].
+ | ^~~~~~~~
+
+ctables.sps:7.47-7.54: error: CTABLES: This category specification may be applied only to string variables, but this
+subcommand tries to apply it to numeric variable QN1.
+ 7 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['string'].
+ | ^~~~~~~~
+
+ctables.sps:8.53: error: CTABLES: This category specification may be applied only to numeric variables, but this
+subcommand tries to apply it to string variable string.
+ 8 | CTABLES /TABLE string /CATEGORIES VARIABLES=string [1].
+ | ^
+
+ctables.sps:10.74-10.86: error: CTABLES: Data-dependent sorting is not implemented.
+ 10 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1).
+ | ^~~~~~~~~~~~~
+
+ctables.sps:12: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
+moved must be categorical, but qnd1 is scale.
+
+ctables.sps:12.22-12.47: note: CTABLES: This syntax moves category labels to another axis.
+ 12 | CTABLES /TABLE qnd1 /CLABELS ROWLABELS=OPPOSITE.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:13: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
+moved must all have the same width, but QN1 has width 0 and string has width 8.
+
+ctables.sps:13.30-13.55: note: CTABLES: This syntax moves category labels to another axis.
+ 13 | CTABLES /TABLE qn1 + string /CLABELS ROWLABELS=OPPOSITE.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:14: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
+moved must all have the same value labels, but QN1 and QNSA1 have different value labels.
+
+ctables.sps:14.29-14.54: note: CTABLES: This syntax moves category labels to another axis.
+ 14 | CTABLES /TABLE qn1 + qnsa1 /CLABELS ROWLABELS=OPPOSITE.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:15: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
+moved must all have the same category specifications, but QN105BA and QN105BB have different category specifications.
+
+ctables.sps:15.35-15.60: note: CTABLES: This syntax moves category labels to another axis.
+ 15 | CTABLES /TABLE qn105ba + qn105bb /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn105ba [1,2,3].
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:17.27-17.33: warning: CTABLES: The exponentiation operator (`**') is left-associative: `a**b**c' equals
+`(a**b)**c', not `a**(b**c)'. To disable this warning, insert parentheses.
+ 17 | CTABLES /PCOMPUTE &x=EXPR(1**2**3).
+ | ^~~~~~~
+
+ctables.sps:17.35: error: CTABLES: Syntax error expecting `/'.
+ 17 | CTABLES /PCOMPUTE &x=EXPR(1**2**3).
+ | ^
+
+ctables.sps:18.28-18.29: error: CTABLES: Syntax error expecting number or string or range.
+ 18 | CTABLES /PCOMPUTE &x=EXPR([**]).
+ | ^~
+
+ctables.sps:19.27-19.28: error: CTABLES: Syntax error in postcompute expression.
+ 19 | CTABLES /PCOMPUTE &x=EXPR(**).
+ | ^~
+
+ctables.sps:21.15: error: CTABLES: At least one variable must be specified.
+ 21 | CTABLES /TABLE.
+ | ^
+
+ctables.sps:23: error: CTABLES: Summaries may appear only on one axis.
+
+ctables.sps:23.50-23.54: note: CTABLES: This variable on the layers axis has a summary.
+ 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
+ | ^~~~~
+
+ctables.sps:23.16-23.20: note: CTABLES: This variable on the rows axis has a summary.
+ 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
+ | ^~~~~
+
+ctables.sps:23.33-23.37: note: CTABLES: This variable on the columns axis has a summary.
+ 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
+ | ^~~~~
+
+ctables.sps:23.33-23.37: note: CTABLES: This is a scale variable, so it always has a summary even if the syntax does not
+explicitly specify one.
+ 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
+ | ^~~~~
+
+ctables.sps:25.46-25.63: error: CTABLES: Data-dependent sorting is not implemented.
+ 25 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 50).
+ | ^~~~~~~~~~~~~~~~~~
+
+ctables.sps:27.16-27.21: error: CTABLES: Multiple response set support not implemented.
+ 27 | CTABLES /TABLE $mrset.
+ | ^~~~~~
+
+ctables.sps:29.23-29.44: error: CTABLES: Support for SIGTEST not yet implemented.
+ 29 | CTABLES /TABLE qn113 /SIGTEST TYPE=CHISQUARE.
+ | ^~~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:30.23-30.43: error: CTABLES: Support for COMPARETEST not yet implemented.
+ 30 | CTABLES /TABLE qn113 /COMPARETEST TYPE=PROP.
+ | ^~~~~~~~~~~~~~~~~~~~~
+
+ctables.sps:32.23-32.31: error: CTABLES: Support for LCL, UCL, and SE summary functions is not yet implemented.
+ 32 | CTABLES /TABLE qn113 [COUNT.UCL].
+ | ^~~~~~~~~
+
+ctables.sps:34.32-34.33: error: CTABLES: Syntax error expecting VARIABLES.
+ 34 | CTABLES /TABLE qn1 /CATEGORIES **.
+ | ^~
+
+ctables.sps:36.10-36.15: error: CTABLES: Syntax error expecting one of the following: FORMAT, VLABELS, MRSETS, SMISSING,
+PCOMPUTE, PPROPERTIES, WEIGHT, HIDESMALLCOUNTS, TABLE.
+ 36 | CTABLES /TITLES.
+ | ^~~~~~
+
+ctables.sps:36.10-36.15: note: CTABLES: TABLE must appear before this subcommand.
+ 36 | CTABLES /TITLES.
+ | ^~~~~~
+]])
+AT_CLEANUP
+
+AT_SETUP([CTABLES one categorical variable])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE qn1.
+CTABLES /TABLE BY qn1.
+CTABLES /TABLE BY BY qn1.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│ 1. How often do you usually drive a car or other Every day │ 4667│
+│motor vehicle? Several days a week │ 1274│
+│ Once a week or less │ 361│
+│ Only certain times a │ 130│
+│ year │ │
+│ Never │ 540│
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────────────────────────────╮
+│ 1. How often do you usually drive a car or other motor vehicle? │
+├─────────┬──────────────────┬──────────────────┬────────────────────────┬─────┤
+│ │ Several days a │ Once a week or │ Only certain times a │ │
+│Every day│ week │ less │ year │Never│
+├─────────┼──────────────────┼──────────────────┼────────────────────────┼─────┤
+│ Count │ Count │ Count │ Count │Count│
+├─────────┼──────────────────┼──────────────────┼────────────────────────┼─────┤
+│ 4667│ 1274│ 361│ 130│ 540│
+╰─────────┴──────────────────┴──────────────────┴────────────────────────┴─────╯
+
+Custom Tables
+Every day
+╭─────╮
+│Count│
+├─────┤
+│ 4667│
+╰─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES one string variable])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+STRING licensed(A8).
+MISSING VALUES licensed('DontKnow', 'Refused').
+RECODE qnd7a(1='Yes')(2='No')(3='DontKnow')(4='Refused') INTO licensed.
+CTABLES /TABLE licensed.
+CTABLES /TABLE licensed [COUNT, TOTALS[COUNT, VALIDN]] /CATEGORIES VARIABLES=ALL TOTAL=YES MISSING=INCLUDE.
+CTABLES /TABLE licensed /CATEGORIES VARIABLES=licensed ['Yes', 'No'] TOTAL=YES.
+* Notice that the string matching is case-sensitive.
+CTABLES /TABLE licensed /CATEGORIES VARIABLES=licensed ['Yes', 'no'] TOTAL=YES.
+CTABLES /TABLE licensed /CATEGORIES VARIABLES=licensed ['No' THRU 'yes'] TOTAL=YES.
+CTABLES
+ /PCOMPUTE ¬yes=EXPR(['No']+['DontKnow']+['Refused'])
+ /PPROPERTIES ¬yes LABEL='Not Yes' HIDESOURCECATS=YES
+ /TABLE licensed
+ /CATEGORIES VARIABLES=licensed ['Yes', ¬yes, 'No', 'DontKnow', 'Refused'].
+CTABLES
+ /PCOMPUTE ¬yes=EXPR(['DontKnow' THRU 'No'] + ['Refused'])
+ /PPROPERTIES ¬yes LABEL='Not Yes' HIDESOURCECATS=YES
+ /TABLE licensed
+ /CATEGORIES VARIABLES=licensed ['Yes', ¬yes, 'DontKnow' THRU 'No', 'Refused'].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭────────────┬─────╮
+│ │Count│
+├────────────┼─────┤
+│licensed No │ 572│
+│ Yes│ 6379│
+╰────────────┴─────╯
+
+ Custom Tables
+╭─────────────────┬─────┬───────╮
+│ │Count│Valid N│
+├─────────────────┼─────┼───────┤
+│licensed DontKnow│ 4│ │
+│ No │ 572│ │
+│ Refused │ 44│ │
+│ Yes │ 6379│ │
+│ Total │ 6999│ 6951│
+╰─────────────────┴─────┴───────╯
+
+ Custom Tables
+╭──────────────┬─────╮
+│ │Count│
+├──────────────┼─────┤
+│licensed Yes │ 6379│
+│ No │ 572│
+│ Total│ 6951│
+╰──────────────┴─────╯
+
+ Custom Tables
+╭──────────────┬─────╮
+│ │Count│
+├──────────────┼─────┤
+│licensed Yes │ 6379│
+│ no │ 0│
+│ Total│ 6379│
+╰──────────────┴─────╯
+
+ Custom Tables
+╭────────────────┬─────╮
+│ │Count│
+├────────────────┼─────┤
+│licensed No │ 572│
+│ Refused│ 44│
+│ Yes │ 6379│
+│ Total │ 6995│
+╰────────────────┴─────╯
+
+ Custom Tables
+╭────────────────┬─────╮
+│ │Count│
+├────────────────┼─────┤
+│licensed Yes │ 6379│
+│ Not Yes│ 620│
+╰────────────────┴─────╯
+
+ Custom Tables
+╭────────────────┬─────╮
+│ │Count│
+├────────────────┼─────┤
+│licensed Yes │ 6379│
+│ Not Yes│ 620│
+╰────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES one scale variable])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+DESCRIPTIVES qnd1.
+CTABLES /TABLE qnd1[COUNT, VALIDN, TOTALN, MEAN, STDDEV, MINIMUM, MAXIMUM].
+CTABLES /TABLE BY qnd1.
+CTABLES /TABLE BY BY qnd1.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Descriptive Statistics
+╭──────────────────────────┬────┬─────┬───────┬───────┬───────────╮
+│ │ N │ Mean│Std Dev│Minimum│ Maximum │
+├──────────────────────────┼────┼─────┼───────┼───────┼───────────┤
+│D1. AGE: What is your age?│6930│48.26│ 19.01│ 16│86 or older│
+│Valid N (listwise) │6999│ │ │ │ │
+│Missing N (listwise) │ 69│ │ │ │ │
+╰──────────────────────────┴────┴─────┴───────┴───────┴───────────╯
+
+ Custom Tables
+╭──────────────────────┬─────┬───────┬───────┬────┬────────────┬───────┬───────╮
+│ │ │ │ │ │ Std │ │ │
+│ │Count│Valid N│Total N│Mean│ Deviation │Minimum│Maximum│
+├──────────────────────┼─────┼───────┼───────┼────┼────────────┼───────┼───────┤
+│D1. AGE: What is your │ 6999│ 6930│ 6999│ 48│ 19│ 16│ 86│
+│age? │ │ │ │ │ │ │ │
+╰──────────────────────┴─────┴───────┴───────┴────┴────────────┴───────┴───────╯
+
+ Custom Tables
+╭──────────────────────────╮
+│D1. AGE: What is your age?│
+├──────────────────────────┤
+│ Mean │
+├──────────────────────────┤
+│ 48│
+╰──────────────────────────╯
+
+Custom Tables
+D1. AGE: What is your age?
+╭────╮
+│Mean│
+├────┤
+│ 48│
+╰────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES simple stacking])
+AT_KEYWORDS([stack stacked])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE qn105ba + qn105bb + qn105bc + qn105bd BY qns3a [COLPCT PCT8.0].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭───────────────────────────────────────────────────────────────┬──────────────╮
+│ │ S3a. GENDER: │
+│ ├──────┬───────┤
+│ │ Male │ Female│
+│ ├──────┼───────┤
+│ │Column│ Column│
+│ │ % │ % │
+├───────────────────────────────────────────────────────────────┼──────┼───────┤
+│105b. How likely is it that drivers who have had Almost │ 10%│ 11%│
+│too much to drink to drive safely will A. Get certain │ │ │
+│stopped by the police? Very likely │ 21%│ 22%│
+│ Somewhat │ 38%│ 42%│
+│ likely │ │ │
+│ Somewhat │ 21%│ 18%│
+│ unlikely │ │ │
+│ Very │ 10%│ 8%│
+│ unlikely │ │ │
+├───────────────────────────────────────────────────────────────┼──────┼───────┤
+│105b. How likely is it that drivers who have had Almost │ 14%│ 18%│
+│too much to drink to drive safely will B. Have an certain │ │ │
+│accident? Very likely │ 36%│ 45%│
+│ Somewhat │ 39%│ 32%│
+│ likely │ │ │
+│ Somewhat │ 9%│ 4%│
+│ unlikely │ │ │
+│ Very │ 3%│ 2%│
+│ unlikely │ │ │
+├───────────────────────────────────────────────────────────────┼──────┼───────┤
+│105b. How likely is it that drivers who have had Almost │ 18%│ 16%│
+│too much to drink to drive safely will C. Be certain │ │ │
+│convicted for drunk driving? Very likely │ 32%│ 28%│
+│ Somewhat │ 27%│ 32%│
+│ likely │ │ │
+│ Somewhat │ 15%│ 15%│
+│ unlikely │ │ │
+│ Very │ 9%│ 9%│
+│ unlikely │ │ │
+├───────────────────────────────────────────────────────────────┼──────┼───────┤
+│105b. How likely is it that drivers who have had Almost │ 16%│ 16%│
+│too much to drink to drive safely will D. Be certain │ │ │
+│arrested for drunk driving? Very likely │ 26%│ 27%│
+│ Somewhat │ 32%│ 35%│
+│ likely │ │ │
+│ Somewhat │ 17%│ 15%│
+│ unlikely │ │ │
+│ Very │ 9%│ 7%│
+│ unlikely │ │ │
+╰───────────────────────────────────────────────────────────────┴──────┴───────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES show or hide empty categories])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+IF (qn105ba = 2) qn105ba = 1.
+IF (qns3a = 1) qns3a = 2.
+CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0].
+CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0]
+ /CATEGORIES VAR=qn105ba EMPTY=EXCLUDE.
+CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0]
+ /CATEGORIES VAR=qns3a EMPTY=EXCLUDE.
+CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0]
+ /CATEGORIES VAR=ALL EMPTY=EXCLUDE.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭──────────────────────────────────────────────────────────────┬───────────────╮
+│ │ S3a. GENDER: │
+│ ├───────┬───────┤
+│ │ Male │ Female│
+│ ├───────┼───────┤
+│ │ Column│ Column│
+│ │ % │ % │
+├──────────────────────────────────────────────────────────────┼───────┼───────┤
+│105b. How likely is it that drivers who have had Almost │ .│ 32%│
+│too much to drink to drive safely will A. Get certain │ │ │
+│stopped by the police? Very likely│ .│ 0%│
+│ Somewhat │ .│ 40%│
+│ likely │ │ │
+│ Somewhat │ .│ 19%│
+│ unlikely │ │ │
+│ Very │ .│ 9%│
+│ unlikely │ │ │
+╰──────────────────────────────────────────────────────────────┴───────┴───────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────────────┬───────────────╮
+│ │ S3a. GENDER: │
+│ ├───────┬───────┤
+│ │ Male │ Female│
+│ ├───────┼───────┤
+│ │ Column│ Column│
+│ │ % │ % │
+├──────────────────────────────────────────────────────────────┼───────┼───────┤
+│105b. How likely is it that drivers who have had Almost │ .│ 32%│
+│too much to drink to drive safely will A. Get certain │ │ │
+│stopped by the police? Somewhat │ .│ 40%│
+│ likely │ │ │
+│ Somewhat │ .│ 19%│
+│ unlikely │ │ │
+│ Very │ .│ 9%│
+│ unlikely │ │ │
+╰──────────────────────────────────────────────────────────────┴───────┴───────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────┬─────────╮
+│ │ S3a. │
+│ │ GENDER: │
+│ ├─────────┤
+│ │ Female │
+│ ├─────────┤
+│ │ Column %│
+├────────────────────────────────────────────────────────────────────┼─────────┤
+│105b. How likely is it that drivers who have had too Almost │ 32%│
+│much to drink to drive safely will A. Get stopped by certain │ │
+│the police? Very likely │ 0%│
+│ Somewhat │ 40%│
+│ likely │ │
+│ Somewhat │ 19%│
+│ unlikely │ │
+│ Very │ 9%│
+│ unlikely │ │
+╰────────────────────────────────────────────────────────────────────┴─────────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────┬─────────╮
+│ │ S3a. │
+│ │ GENDER: │
+│ ├─────────┤
+│ │ Female │
+│ ├─────────┤
+│ │ Column %│
+├────────────────────────────────────────────────────────────────────┼─────────┤
+│105b. How likely is it that drivers who have had too Almost │ 32%│
+│much to drink to drive safely will A. Get stopped by certain │ │
+│the police? Somewhat │ 40%│
+│ likely │ │
+│ Somewhat │ 19%│
+│ unlikely │ │
+│ Very │ 9%│
+│ unlikely │ │
+╰────────────────────────────────────────────────────────────────────┴─────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES categories and EMPTY])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+DATA LIST LIST NOTABLE /class datum size.
+BEGIN DATA
+1 1 1
+2 2 1
+1 3 1
+2 4 2
+1 5 2
+2 6 2
+END DATA.
+VARIABLE LEVEL class datum size (NOMINAL).
+FORMATS class datum size (F1.0).
+
+* The following are the same except for the order of the CATEGORIES commands.
+* The test checks that they produce the same resuls.
+CTABLES /TABLE=class > datum BY size
+ /CATEGORIES VARIABLES=ALL EMPTY=EXCLUDE
+ /CATEGORIES VARIABLES=size TOTAL=YES.
+CTABLES /TABLE=class > datum BY size
+ /CATEGORIES VARIABLES=size TOTAL=YES
+ /CATEGORIES VARIABLES=ALL EMPTY=EXCLUDE.
+])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭───────────────┬─────────────────╮
+│ │ size │
+│ ├─────┬─────┬─────┤
+│ │ 1 │ 2 │Total│
+│ ├─────┼─────┼─────┤
+│ │Count│Count│Count│
+├───────────────┼─────┼─────┼─────┤
+│class 1 datum 1│ 1│ │ 1│
+│ 3│ 1│ │ 1│
+│ 5│ │ 1│ 1│
+│ ╶─────────┼─────┼─────┼─────┤
+│ 2 datum 2│ 1│ │ 1│
+│ 4│ │ 1│ 1│
+│ 6│ │ 1│ 1│
+╰───────────────┴─────┴─────┴─────╯
+
+ Custom Tables
+╭───────────────┬─────────────────╮
+│ │ size │
+│ ├─────┬─────┬─────┤
+│ │ 1 │ 2 │Total│
+│ ├─────┼─────┼─────┤
+│ │Count│Count│Count│
+├───────────────┼─────┼─────┼─────┤
+│class 1 datum 1│ 1│ │ 1│
+│ 3│ 1│ │ 1│
+│ 5│ │ 1│ 1│
+│ ╶─────────┼─────┼─────┼─────┤
+│ 2 datum 2│ 1│ │ 1│
+│ 4│ │ 1│ 1│
+│ 6│ │ 1│ 1│
+╰───────────────┴─────┴─────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES sorting categories])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+IF (QND5A=6) QND5A=-1.
+IF (QND5A=5) QND5A=-2.
+CTABLES /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=VALUE ORDER=A
+ /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=VALUE ORDER=D
+ /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=LABEL ORDER=A
+ /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=LABEL ORDER=D.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary ethnic -2.00 │ 52│
+│background? -1.00 │ 78│
+│ Cuban │ 20│
+│ Mexican │ 311│
+│ Spanish │ 48│
+│ South American │ 34│
+│ Central American │ 0│
+│ Puerto Rican, OR │ 0│
+│ Something else │ 68│
+│ Multiple - cannot choose│ 7│
+│ one │ │
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary ethnic Multiple - cannot choose│ 7│
+│background? one │ │
+│ Something else │ 68│
+│ Puerto Rican, OR │ 0│
+│ Central American │ 0│
+│ South American │ 34│
+│ Spanish │ 48│
+│ Mexican │ 311│
+│ Cuban │ 20│
+│ -1.00 │ 78│
+│ -2.00 │ 52│
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary ethnic Central American │ 0│
+│background? Cuban │ 20│
+│ Mexican │ 311│
+│ Multiple - cannot choose│ 7│
+│ one │ │
+│ Puerto Rican, OR │ 0│
+│ Something else │ 68│
+│ South American │ 34│
+│ Spanish │ 48│
+│ -2.00 │ 52│
+│ -1.00 │ 78│
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary ethnic Spanish │ 48│
+│background? South American │ 34│
+│ Something else │ 68│
+│ Puerto Rican, OR │ 0│
+│ Multiple - cannot choose│ 7│
+│ one │ │
+│ Mexican │ 311│
+│ Cuban │ 20│
+│ Central American │ 0│
+│ -1.00 │ 78│
+│ -2.00 │ 52│
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES simple nesting])
+AT_KEYWORDS([nest nested])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE (qn105ba + qn105bb + qn105bc + qn105bd) > qns3a [COUNT, TABLEPCT PCT8.0]
+ /CATEGORIES VARIABLES=qns3a TOTAL=YES.
+CTABLES /TABLE qns3a > (qn105ba + qn105bb + qn105bc + qn105bd) [TABLEPCT PCT8.0]
+ /CATEGORIES VARIABLES=qns3a TOTAL=YES
+ /CLABELS ROW=OPPOSITE.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭─────────────────────────────────────────────────────────────────┬─────┬──────╮
+│ │ │ Table│
+│ │Count│ % │
+├─────────────────────────────────────────────────────────────────┼─────┼──────┤
+│105b. How likely is it that drivers Almost S3a. Male │ 297│ 4%│
+│who have had too much to drink to certain GENDER: Female│ 403│ 6%│
+│drive safely will A. Get stopped by Total │ 700│ 10%│
+│the police? ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 660│ 10%│
+│ likely GENDER: Female│ 842│ 12%│
+│ Total │ 1502│ 22%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 1174│ 17%│
+│ likely GENDER: Female│ 1589│ 23%│
+│ Total │ 2763│ 40%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 640│ 9%│
+│ unlikely GENDER: Female│ 667│ 10%│
+│ Total │ 1307│ 19%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 311│ 5%│
+│ unlikely GENDER: Female│ 298│ 4%│
+│ Total │ 609│ 9%│
+├─────────────────────────────────────────────────────────────────┼─────┼──────┤
+│105b. How likely is it that drivers Almost S3a. Male │ 429│ 6%│
+│who have had too much to drink to certain GENDER: Female│ 671│ 10%│
+│drive safely will B. Have an accident? Total │ 1100│ 16%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 1104│ 16%│
+│ likely GENDER: Female│ 1715│ 25%│
+│ Total │ 2819│ 41%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 1203│ 17%│
+│ likely GENDER: Female│ 1214│ 18%│
+│ Total │ 2417│ 35%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 262│ 4%│
+│ unlikely GENDER: Female│ 168│ 2%│
+│ Total │ 430│ 6%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 81│ 1%│
+│ unlikely GENDER: Female│ 59│ 1%│
+│ Total │ 140│ 2%│
+├─────────────────────────────────────────────────────────────────┼─────┼──────┤
+│105b. How likely is it that drivers Almost S3a. Male │ 539│ 8%│
+│who have had too much to drink to certain GENDER: Female│ 610│ 9%│
+│drive safely will C. Be convicted for Total │ 1149│ 17%│
+│drunk driving? ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 988│ 14%│
+│ likely GENDER: Female│ 1049│ 15%│
+│ Total │ 2037│ 30%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 822│ 12%│
+│ likely GENDER: Female│ 1210│ 18%│
+│ Total │ 2032│ 30%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 446│ 7%│
+│ unlikely GENDER: Female│ 548│ 8%│
+│ Total │ 994│ 15%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 268│ 4%│
+│ unlikely GENDER: Female│ 354│ 5%│
+│ Total │ 622│ 9%│
+├─────────────────────────────────────────────────────────────────┼─────┼──────┤
+│105b. How likely is it that drivers Almost S3a. Male │ 498│ 7%│
+│who have had too much to drink to certain GENDER: Female│ 603│ 9%│
+│drive safely will D. Be arrested for Total │ 1101│ 16%│
+│drunk driving? ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 805│ 12%│
+│ likely GENDER: Female│ 1029│ 15%│
+│ Total │ 1834│ 27%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 975│ 14%│
+│ likely GENDER: Female│ 1332│ 19%│
+│ Total │ 2307│ 34%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Somewhat S3a. Male │ 535│ 8%│
+│ unlikely GENDER: Female│ 560│ 8%│
+│ Total │ 1095│ 16%│
+│ ╶──────────────────────────┼─────┼──────┤
+│ Very S3a. Male │ 270│ 4%│
+│ unlikely GENDER: Female│ 279│ 4%│
+│ Total │ 549│ 8%│
+╰─────────────────────────────────────────────────────────────────┴─────┴──────╯
+
+ Custom Tables
+╭─────────────────────────────────┬────────┬──────┬─────────┬─────────┬────────╮
+│ │ Almost │ Very │ Somewhat│ Somewhat│ Very │
+│ │ certain│likely│ likely │ unlikely│unlikely│
+│ ├────────┼──────┼─────────┼─────────┼────────┤
+│ │ │ Table│ │ │ │
+│ │ Table %│ % │ Table % │ Table % │ Table %│
+├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│S3a. Male 105b. How likely │ 4%│ 10%│ 17%│ 9%│ 5%│
+│GENDER: is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ A. Get stopped by │ │ │ │ │ │
+│ the police? │ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Female 105b. How likely │ 6%│ 12%│ 23%│ 10%│ 4%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ A. Get stopped by │ │ │ │ │ │
+│ the police? │ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Total 105b. How likely │ 10%│ 22%│ 40%│ 19%│ 9%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ A. Get stopped by │ │ │ │ │ │
+│ the police? │ │ │ │ │ │
+├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│S3a. Male 105b. How likely │ 6%│ 16%│ 17%│ 4%│ 1%│
+│GENDER: is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ B. Have an │ │ │ │ │ │
+│ accident? │ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Female 105b. How likely │ 10%│ 25%│ 18%│ 2%│ 1%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ B. Have an │ │ │ │ │ │
+│ accident? │ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Total 105b. How likely │ 16%│ 41%│ 35%│ 6%│ 2%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ B. Have an │ │ │ │ │ │
+│ accident? │ │ │ │ │ │
+├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│S3a. Male 105b. How likely │ 8%│ 14%│ 12%│ 7%│ 4%│
+│GENDER: is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ C. Be convicted │ │ │ │ │ │
+│ for drunk driving?│ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Female 105b. How likely │ 9%│ 15%│ 18%│ 8%│ 5%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ C. Be convicted │ │ │ │ │ │
+│ for drunk driving?│ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Total 105b. How likely │ 17%│ 30%│ 30%│ 15%│ 9%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ C. Be convicted │ │ │ │ │ │
+│ for drunk driving?│ │ │ │ │ │
+├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│S3a. Male 105b. How likely │ 7%│ 12%│ 14%│ 8%│ 4%│
+│GENDER: is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ D. Be arrested for│ │ │ │ │ │
+│ drunk driving? │ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Female 105b. How likely │ 9%│ 15%│ 19%│ 8%│ 4%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ D. Be arrested for│ │ │ │ │ │
+│ drunk driving? │ │ │ │ │ │
+│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
+│ Total 105b. How likely │ 16%│ 27%│ 34%│ 16%│ 8%│
+│ is it that drivers│ │ │ │ │ │
+│ who have had too │ │ │ │ │ │
+│ much to drink to │ │ │ │ │ │
+│ drive safely will │ │ │ │ │ │
+│ D. Be arrested for│ │ │ │ │ │
+│ drunk driving? │ │ │ │ │ │
+╰─────────────────────────────────┴────────┴──────┴─────────┴─────────┴────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES nesting and scale variables])
+AT_KEYWORDS([nest nested])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE=qnd1 > qn1 BY qns3a.
+CTABLES /TABLE=qnd1 [MINIMUM, MAXIMUM, MEAN] > qns3a > (qn26 + qn27).
+CTABLES /TABLE=qnsa1 > qn105ba [COLPCT] BY qns1
+ /CATEGORIES VAR=qnsa1 EMPTY=EXCLUDE.
+CTABLES /TABLE=AgeGroup > qn20 [MEAN F8.1, STDDEV F8.1].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭─────────────────────────────────────────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │ Mean│ Mean │
+├─────────────────────────────────────────────────────────────────┼─────┼──────┤
+│D1. AGE: What 1. How often do you usually drive Every day │ 46│ 46│
+│is your age? a car or other motor vehicle? Several days a │ 51│ 59│
+│ week │ │ │
+│ Once a week or │ 44│ 54│
+│ less │ │ │
+│ Only certain │ 34│ 41│
+│ times a year │ │ │
+│ Never │ 39│ 55│
+╰─────────────────────────────────────────────────────────────────┴─────┴──────╯
+
+ Custom Tables
+╭─────────────────────────────────────────────────────────┬───────┬───────┬────╮
+│ │Minimum│Maximum│Mean│
+├─────────────────────────────────────────────────────────┼───────┼───────┼────┤
+│D1. AGE: S3a. Male 26. During the last 12 Yes│ 16│ 86│ 42│
+│What is GENDER: months, has there been a │ │ │ │
+│your time when you felt you │ │ │ │
+│age? should cut down on your No │ 16│ 86│ 46│
+│ drinking? │ │ │ │
+│ ╶───────────────────────────────────────┼───────┼───────┼────┤
+│ Female 26. During the last 12 Yes│ 16│ 86│ 43│
+│ months, has there been a │ │ │ │
+│ time when you felt you │ │ │ │
+│ should cut down on your No │ 16│ 86│ 48│
+│ drinking? │ │ │ │
+├─────────────────────────────────────────────────────────┼───────┼───────┼────┤
+│D1. AGE: S3a. Male 27. During the last 12 Yes│ 16│ 86│ 38│
+│What is GENDER: months, has there been a │ │ │ │
+│your time when people criticized No │ 16│ 86│ 46│
+│age? your drinking? │ │ │ │
+│ ╶───────────────────────────────────────┼───────┼───────┼────┤
+│ Female 27. During the last 12 Yes│ 17│ 69│ 37│
+│ months, has there been a │ │ │ │
+│ time when people criticized No │ 16│ 86│ 48│
+│ your drinking? │ │ │ │
+╰─────────────────────────────────────────────────────────┴───────┴───────┴────╯
+
+ Custom Tables
+╭─────────────────────────────┬────────────────────────────────────────────────╮
+│ │S1. Including yourself, how many members of this│
+│ │ household are age 16 or older? │
+│ ├──────┬──────┬──────┬──────┬──────┬──────┬──────┤
+│ │ │ │ │ │ │ │ 6 or │
+│ │ None │ 1 │ 2 │ 3 │ 4 │ 5 │ more │
+│ ├──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ │Column│Column│Column│Column│Column│Column│Column│
+│ │ % │ % │ % │ % │ % │ % │ % │
+├─────────────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Sa1. RDD 105b. Almost │ .│ 9.5%│ 8.2%│ 12.4%│ 9.9%│ 20.0%│ 23.8%│
+│SAMPLE How certain │ │ │ │ │ │ │ │
+│SOURCE: likely │ │ │ │ │ │ │ │
+│ is it Very │ .│ 24.9%│ 18.5%│ 24.0%│ 26.6%│ 25.5%│ 33.3%│
+│ that likely │ │ │ │ │ │ │ │
+│ drivers │ │ │ │ │ │ │ │
+│ who have │ │ │ │ │ │ │ │
+│ had too Somewhat│ .│ 38.3%│ 41.9%│ 38.6%│ 37.5%│ 36.4%│ 23.8%│
+│ much to likely │ │ │ │ │ │ │ │
+│ drink to │ │ │ │ │ │ │ │
+│ drive │ │ │ │ │ │ │ │
+│ safely Somewhat│ .│ 18.1%│ 21.7%│ 16.8%│ 16.7%│ 10.9%│ 9.5%│
+│ will A. unlikely│ │ │ │ │ │ │ │
+│ Get │ │ │ │ │ │ │ │
+│ stopped Very │ .│ 9.2%│ 9.7%│ 8.2%│ 9.4%│ 7.3%│ 9.5%│
+│ by the unlikely│ │ │ │ │ │ │ │
+│ police? │ │ │ │ │ │ │ │
+╰─────────────────────────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────────────┬────┬──────────╮
+│ │ │ Std │
+│ │Mean│ Deviation│
+├──────────────────────────────────────────────────────────────┼────┼──────────┤
+│Age 16 to 25 20. On how many of the thirty days in this │ 5.2│ 6.0│
+│group typical month did you have one or more │ │ │
+│ alcoholic beverages to drink? │ │ │
+│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
+│ 26 to 35 20. On how many of the thirty days in this │ 4.7│ 5.9│
+│ typical month did you have one or more │ │ │
+│ alcoholic beverages to drink? │ │ │
+│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
+│ 36 to 45 20. On how many of the thirty days in this │ 5.5│ 6.8│
+│ typical month did you have one or more │ │ │
+│ alcoholic beverages to drink? │ │ │
+│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
+│ 46 to 55 20. On how many of the thirty days in this │ 5.8│ 7.7│
+│ typical month did you have one or more │ │ │
+│ alcoholic beverages to drink? │ │ │
+│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
+│ 56 to 65 20. On how many of the thirty days in this │ 6.3│ 8.2│
+│ typical month did you have one or more │ │ │
+│ alcoholic beverages to drink? │ │ │
+│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
+│ 66 or 20. On how many of the thirty days in this │ 7.1│ 9.2│
+│ older typical month did you have one or more │ │ │
+│ alcoholic beverages to drink? │ │ │
+╰──────────────────────────────────────────────────────────────┴────┴──────────╯
+])
+AT_CLEANUP
+
+
+AT_SETUP([CTABLES SLABELS])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE qn1 [COUNT COLPCT].
+CTABLES /TABLE qn1 [COUNT COLPCT]
+ /SLABELS POSITION=ROW.
+CTABLES /TABLE qn1 [COUNT COLPCT]
+ /SLABELS POSITION=ROW VISIBLE=NO.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭────────────────────────────────────────────────────────────────┬─────┬───────╮
+│ │ │ Column│
+│ │Count│ % │
+├────────────────────────────────────────────────────────────────┼─────┼───────┤
+│ 1. How often do you usually drive a car or Every day │ 4667│ 66.9%│
+│other motor vehicle? Several days a week│ 1274│ 18.3%│
+│ Once a week or less│ 361│ 5.2%│
+│ Only certain times │ 130│ 1.9%│
+│ a year │ │ │
+│ Never │ 540│ 7.7%│
+╰────────────────────────────────────────────────────────────────┴─────┴───────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ 1. How often do you usually drive a car or Every day Count │ 4667│
+│other motor vehicle? Column │66.9%│
+│ % │ │
+│ ╶───────────────────────────┼─────┤
+│ Several days a week Count │ 1274│
+│ Column │18.3%│
+│ % │ │
+│ ╶───────────────────────────┼─────┤
+│ Once a week or less Count │ 361│
+│ Column │ 5.2%│
+│ % │ │
+│ ╶───────────────────────────┼─────┤
+│ Only certain times Count │ 130│
+│ a year Column │ 1.9%│
+│ % │ │
+│ ╶───────────────────────────┼─────┤
+│ Never Count │ 540│
+│ Column │ 7.7%│
+│ % │ │
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ 1. How often do you usually drive a car or other Every day │ 4667│
+│motor vehicle? │66.9%│
+│ Several days a week │ 1274│
+│ │18.3%│
+│ Once a week or less │ 361│
+│ │ 5.2%│
+│ Only certain times a │ 130│
+│ year │ 1.9%│
+│ Never │ 540│
+│ │ 7.7%│
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES SLABELS with stacking different summaries])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=ALL DISPLAY=NAME
+ /TABLE qn1 [COUNT] + qnd1 [MEAN] + qn17 [UCOUNT] BY qns3a
+ /SLABELS POSITION=ROW.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭─────────────────────────────────────────────────┬───────────╮
+│ │ QNS3A │
+│ ├────┬──────┤
+│ │Male│Female│
+├─────────────────────────────────────────────────┼────┼──────┤
+│QN1 Every day Count │2305│ 2362│
+│ Unweighted Count│ │ │
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Several days a week Count │ 440│ 834│
+│ Unweighted Count│ │ │
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Once a week or less Count │ 125│ 236│
+│ Unweighted Count│ │ │
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Only certain times a year Count │ 58│ 72│
+│ Unweighted Count│ │ │
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Never Count │ 192│ 348│
+│ Unweighted Count│ │ │
+│ Mean │ │ │
+├─────────────────────────────────────────────────┼────┼──────┤
+│qnd1 Count │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Unweighted Count │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Mean │ 46│ 50│
+├─────────────────────────────────────────────────┼────┼──────┤
+│QN17 OR, something else Count │ │ │
+│ Unweighted Count│ 1│ 1│
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Beer Count │ │ │
+│ Unweighted Count│ 817│ 256│
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Light beer Count │ │ │
+│ Unweighted Count│ 406│ 214│
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Wine Count │ │ │
+│ Unweighted Count│ 390│ 1028│
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Wine coolers Count │ │ │
+│ Unweighted Count│ 20│ 117│
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Hard liquor or mixed drinks Count │ │ │
+│ Unweighted Count│ 392│ 496│
+│ Mean │ │ │
+│ ╶────────────────────────────────────────────┼────┼──────┤
+│ Flavored malt drinks Count │ │ │
+│ Unweighted Count│ 20│ 63│
+│ Mean │ │ │
+╰─────────────────────────────────────────────────┴────┴──────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES simple totals])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE=qn17
+ /CATEGORIES VARIABLES=qn17 TOTAL=YES LABEL='Number responding'.
+DESCRIPTIVES qn18/STATISTICS=MEAN.
+CTABLES /TABLE=region > qn18 [MEAN, COUNT, VALIDN, TOTALN]
+ /CATEGORIES VARIABLES=region TOTAL=YES LABEL='All regions'.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│17. When you drink alcoholic beverages, which ONE of OR, something else│ 2│
+│the following beverages do you drink MOST OFTEN? Beer │ 1073│
+│ Light beer │ 620│
+│ Wine │ 1418│
+│ Wine coolers │ 137│
+│ Hard liquor or │ 888│
+│ mixed drinks │ │
+│ Flavored malt │ 83│
+│ drinks │ │
+│ Number responding │ 4221│
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Descriptive Statistics
+╭────────────────────────────────────────────────────────────────────┬────┬────╮
+│ │ N │Mean│
+├────────────────────────────────────────────────────────────────────┼────┼────┤
+│18. When you drink ANSWERFROM(QN17R1), about how many │4218│4.62│
+│ANSWERFROM(QN17R2) do you usually drink per sitting? │ │ │
+│Valid N (listwise) │6999│ │
+│Missing N (listwise) │2781│ │
+╰────────────────────────────────────────────────────────────────────┴────┴────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────┬────┬─────┬──────┬─────╮
+│ │ │ │ Valid│Total│
+│ │Mean│Count│ N │ N │
+├──────────────────────────────────────────────────────┼────┼─────┼──────┼─────┤
+│Region NE 18. When you drink ANSWERFROM(QN17R1),│4.36│ 1409│ 949│ 1409│
+│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
+│ you usually drink per sitting? │ │ │ │ │
+│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
+│ MW 18. When you drink ANSWERFROM(QN17R1),│4.67│ 1654│ 1027│ 1654│
+│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
+│ you usually drink per sitting? │ │ │ │ │
+│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
+│ S 18. When you drink ANSWERFROM(QN17R1),│4.71│ 2390│ 1287│ 2390│
+│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
+│ you usually drink per sitting? │ │ │ │ │
+│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
+│ W 18. When you drink ANSWERFROM(QN17R1),│4.69│ 1546│ 955│ 1546│
+│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
+│ you usually drink per sitting? │ │ │ │ │
+│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
+│ All 18. When you drink ANSWERFROM(QN17R1),│4.62│ 6999│ 4218│ 6999│
+│ regions about how many ANSWERFROM(QN17R2) do │ │ │ │ │
+│ you usually drink per sitting? │ │ │ │ │
+╰──────────────────────────────────────────────────────┴────┴─────┴──────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES subtotals])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE=qn105ba BY qns1
+ /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, SUBTOTAL].
+CTABLES /TABLE=qn105ba [COLPCT] BY qns1
+ /CATEGORIES VARIABLES=qn105ba [1, 2, 3, SUBTOTAL, 4, 5, SUBTOTAL].
+CTABLES /TABLE=qn105ba BY qns1
+ /CATEGORIES VARIABLES=qn105ba [1, 2, 3, SUBTOTAL, 4, 5, SUBTOTAL]
+ /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, SUBTOTAL].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+╭─────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────╮
+│ │ S1. Including yourself, how many members of this household │
+│ │ are age 16 or older? │
+│ ├───────┬───────┬─────────┬───────┬────────┬──────┬──────────┤
+│ │ 1 │ 2 │ Subtotal│ 3 │ 4 │ 5 │ Subtotal │
+│ ├───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
+│ │ Count │ Count │ Count │ Count │ Count │ Count│ Count │
+├─────────────────────────────────────────────────────────┼───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
+│105b. How likely is it that drivers who have Almost │ 147│ 246│ 393│ 62│ 19│ 11│ 92│
+│had too much to drink to drive safely will A. certain │ │ │ │ │ │ │ │
+│Get stopped by the police? Very likely│ 384│ 552│ 936│ 120│ 51│ 14│ 185│
+│ Somewhat │ 590│ 1249│ 1839│ 193│ 72│ 20│ 285│
+│ likely │ │ │ │ │ │ │ │
+│ Somewhat │ 278│ 647│ 925│ 84│ 32│ 6│ 122│
+│ unlikely │ │ │ │ │ │ │ │
+│ Very │ 141│ 290│ 431│ 41│ 18│ 4│ 63│
+│ unlikely │ │ │ │ │ │ │ │
+╰─────────────────────────────────────────────────────────┴───────┴───────┴─────────┴───────┴────────┴──────┴──────────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────╮
+│ │ S1. Including yourself, how many members of this household │
+│ │ are age 16 or older? │
+│ ├────────┬────────┬────────┬────────┬───────┬────────┬────────┤
+│ │ │ │ │ │ │ │ 6 or │
+│ │ None │ 1 │ 2 │ 3 │ 4 │ 5 │ more │
+│ ├────────┼────────┼────────┼────────┼───────┼────────┼────────┤
+│ │ │ │ │ │ Column│ │ │
+│ │Column %│Column %│Column %│Column %│ % │Column %│Column %│
+├────────────────────────────────────────────────────────┼────────┼────────┼────────┼────────┼───────┼────────┼────────┤
+│105b. How likely is it that drivers who have Almost │ .│ 9.5%│ 8.2%│ 12.4%│ 9.9%│ 20.0%│ 23.8%│
+│had too much to drink to drive safely will certain │ │ │ │ │ │ │ │
+│A. Get stopped by the police? Very likely│ .│ 24.9%│ 18.5%│ 24.0%│ 26.6%│ 25.5%│ 33.3%│
+│ Somewhat │ .│ 38.3%│ 41.9%│ 38.6%│ 37.5%│ 36.4%│ 23.8%│
+│ likely │ │ │ │ │ │ │ │
+│ Subtotal │ │ 72.8%│ 68.6%│ 75.0%│ 74.0%│ 81.8%│ 81.0%│
+│ Somewhat │ .│ 18.1%│ 21.7%│ 16.8%│ 16.7%│ 10.9%│ 9.5%│
+│ unlikely │ │ │ │ │ │ │ │
+│ Very │ .│ 9.2%│ 9.7%│ 8.2%│ 9.4%│ 7.3%│ 9.5%│
+│ unlikely │ │ │ │ │ │ │ │
+│ Subtotal │ │ 27.2%│ 31.4%│ 25.0%│ 26.0%│ 18.2%│ 19.0%│
+╰────────────────────────────────────────────────────────┴────────┴────────┴────────┴────────┴───────┴────────┴────────╯
+
+ Custom Tables
+╭─────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────╮
+│ │ S1. Including yourself, how many members of this household │
+│ │ are age 16 or older? │
+│ ├───────┬───────┬─────────┬───────┬────────┬──────┬──────────┤
+│ │ 1 │ 2 │ Subtotal│ 3 │ 4 │ 5 │ Subtotal │
+│ ├───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
+│ │ Count │ Count │ Count │ Count │ Count │ Count│ Count │
+├─────────────────────────────────────────────────────────┼───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
+│105b. How likely is it that drivers who have Almost │ 147│ 246│ 393│ 62│ 19│ 11│ 92│
+│had too much to drink to drive safely will A. certain │ │ │ │ │ │ │ │
+│Get stopped by the police? Very likely│ 384│ 552│ 936│ 120│ 51│ 14│ 185│
+│ Somewhat │ 590│ 1249│ 1839│ 193│ 72│ 20│ 285│
+│ likely │ │ │ │ │ │ │ │
+│ Subtotal │ 1121│ 2047│ 3168│ 375│ 142│ 45│ 562│
+│ Somewhat │ 278│ 647│ 925│ 84│ 32│ 6│ 122│
+│ unlikely │ │ │ │ │ │ │ │
+│ Very │ 141│ 290│ 431│ 41│ 18│ 4│ 63│
+│ unlikely │ │ │ │ │ │ │ │
+│ Subtotal │ 419│ 937│ 1356│ 125│ 50│ 10│ 185│
+╰─────────────────────────────────────────────────────────┴───────┴───────┴─────────┴───────┴────────┴──────┴──────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES PCOMPUTE])
+AT_KEYWORDS([postcompute])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /PCOMPUTE &x=EXPR([3] + [4])
+ /PCOMPUTE &y=EXPR([4] + [5])
+ /PPROPERTIES &x LABEL='3+4' FORMAT=COUNT F8.2
+ /PPROPERTIES &y LABEL=')LABEL[5]+)LABEL[6]'
+ /TABLE=qn105ba [COUNT, ROWPCT] BY qns1
+ /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, &x, &y, SUBTOTAL] TOTAL=YES
+
+* Adding HIDESOURCECATS=YES for one PPROPERTIES.
+CTABLES
+ /PCOMPUTE &x=EXPR([3] + [4])
+ /PCOMPUTE &y=EXPR([4] + [5])
+ /PPROPERTIES &x LABEL='3+4' FORMAT=COUNT F8.2
+ /PPROPERTIES &y LABEL=')LABEL[5]+)LABEL[6]' HIDESOURCECATS=YES
+ /TABLE=qn105ba [COUNT, ROWPCT] BY qns1
+ /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, &x, &y, SUBTOTAL] TOTAL=YES
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=140], [0], [dnl
+ Custom Tables
+╭───────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ │ S1. Including yourself, how many members of this household are age 16 or older? │
+│ ├───────────┬───────────┬───────────┬───────────┬──────────┬──────────┬────────────┬──────────┬───────────┬────────────┤
+│ │ 1 │ 2 │ Subtotal │ 3 │ 4 │ 5 │ 3+4 │ 4+5 │ Subtotal │ Total │
+│ ├─────┬─────┼─────┬─────┼─────┬─────┼─────┬─────┼─────┬────┼─────┬────┼──────┬─────┼─────┬────┼─────┬─────┼─────┬──────┤
+│ │ │ │ │ │ │ │ │ │ │ Row│ │ Row│ │ │ │ Row│ │ │ │ │
+│ │Count│Row %│Count│Row %│Count│Row %│Count│Row %│Count│ % │Count│ % │ Count│Row %│Count│ % │Count│Row %│Count│ Row %│
+├───────────────────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼────┼─────┼────┼──────┼─────┼─────┼────┼─────┼─────┼─────┼──────┤
+│105b. How Almost │ 147│30.3%│ 246│50.7%│ 393│81.0%│ 62│12.8%│ 19│3.9%│ 11│2.3%│ 81.00│16.7%│ 30│6.2%│ 92│19.0%│ 485│100.0%│
+│likely is certain │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│it that │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│drivers Very │ 384│34.3%│ 552│49.2%│ 936│83.5%│ 120│10.7%│ 51│4.5%│ 14│1.2%│171.00│15.3%│ 65│5.8%│ 185│16.5%│ 1121│100.0%│
+│who have likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│had too │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│much to Somewhat│ 590│27.8%│ 1249│58.8%│ 1839│86.6%│ 193│ 9.1%│ 72│3.4%│ 20│ .9%│265.00│12.5%│ 92│4.3%│ 285│13.4%│ 2124│100.0%│
+│drink to likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│drive │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│safely Somewhat│ 278│26.6%│ 647│61.8%│ 925│88.3%│ 84│ 8.0%│ 32│3.1%│ 6│ .6%│116.00│11.1%│ 38│3.6%│ 122│11.7%│ 1047│100.0%│
+│will A. unlikely│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│Get │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│stopped by Very │ 141│28.5%│ 290│58.7%│ 431│87.2%│ 41│ 8.3%│ 18│3.6%│ 4│ .8%│ 59.00│11.9%│ 22│4.5%│ 63│12.8%│ 494│100.0%│
+│the unlikely│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│police? │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+╰───────────────────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴────┴─────┴────┴──────┴─────┴─────┴────┴─────┴─────┴─────┴──────╯
+
+ Custom Tables
+╭─────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ │ S1. Including yourself, how many members of this household are age 16 or older? │
+│ ├───────────┬───────────┬───────────┬───────────┬────────────┬──────────┬───────────┬────────────┤
+│ │ 1 │ 2 │ Subtotal │ 3 │ 3+4 │ 4+5 │ Subtotal │ Total │
+│ ├─────┬─────┼─────┬─────┼─────┬─────┼─────┬─────┼──────┬─────┼─────┬────┼─────┬─────┼─────┬──────┤
+│ │ │ │ │ │ │ │ │ │ │ │ │ Row│ │ │ │ │
+│ │Count│Row %│Count│Row %│Count│Row %│Count│Row %│ Count│Row %│Count│ % │Count│Row %│Count│ Row %│
+├─────────────────────────────────────────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼──────┼─────┼─────┼────┼─────┼─────┼─────┼──────┤
+│105b. How likely is it that Almost │ 147│30.3%│ 246│50.7%│ 393│81.0%│ 62│12.8%│ 81.00│16.7%│ 30│6.2%│ 92│19.0%│ 485│100.0%│
+│drivers who have had too much certain │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│to drink to drive safely will Very │ 384│34.3%│ 552│49.2%│ 936│83.5%│ 120│10.7%│171.00│15.3%│ 65│5.8%│ 185│16.5%│ 1121│100.0%│
+│A. Get stopped by the police? likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│ Somewhat │ 590│27.8%│ 1249│58.8%│ 1839│86.6%│ 193│ 9.1%│265.00│12.5%│ 92│4.3%│ 285│13.4%│ 2124│100.0%│
+│ likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│ Somewhat │ 278│26.6%│ 647│61.8%│ 925│88.3%│ 84│ 8.0%│116.00│11.1%│ 38│3.6%│ 122│11.7%│ 1047│100.0%│
+│ unlikely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+│ Very │ 141│28.5%│ 290│58.7%│ 431│87.2%│ 41│ 8.3%│ 59.00│11.9%│ 22│4.5%│ 63│12.8%│ 494│100.0%│
+│ unlikely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+╰─────────────────────────────────────────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴──────┴─────┴─────┴────┴─────┴─────┴─────┴──────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES PCOMPUTE - OTHERNM and MISSING])
+AT_KEYWORDS([postcompute])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /PCOMPUTE &x=EXPR(OTHERNM)
+ /PCOMPUTE &y=EXPR(MISSING)
+ /PPROPERTIES &x LABEL='Drivers'
+ /PPROPERTIES &y LABEL='Missing Values 2'
+ /TABLE=qn1 BY qns3a
+ /CATEGORIES VARIABLES=qn1 [OTHERNM, 5, &x, SUBTOTAL='Valid Values', MISSING, SUBTOTAL='Missing Values', &y]
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=140], [0], [dnl
+ Custom Tables
+╭──────────────────────────────────────────────────────────────────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├──────────────────────────────────────────────────────────────────────────────────────────┼─────┼──────┤
+│ 1. How often do you usually drive a car or other motor vehicle? Every day │ 2305│ 2362│
+│ Several days a week │ 440│ 834│
+│ Once a week or less │ 125│ 236│
+│ Only certain times a year│ 58│ 72│
+│ Never │ 192│ 348│
+│ Drivers │ 2928│ 3504│
+│ Valid Values │ 3120│ 3852│
+│ Don't know │ 3│ 5│
+│ Refused │ 9│ 10│
+│ Missing Values │ 12│ 15│
+│ Missing Values 2 │ 12│ 15│
+╰──────────────────────────────────────────────────────────────────────────────────────────┴─────┴──────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES PCOMPUTE - THRU])
+AT_KEYWORDS([postcompute])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /PCOMPUTE &x=EXPR([1 THRU 2])
+ /PCOMPUTE &y=EXPR([3 THRU 4])
+ /PCOMPUTE &z=EXPR([5] + MISSING)
+ /PPROPERTIES &x LABEL='Frequent Drivers'
+ /PPROPERTIES &y LABEL='Infrequent Drivers'
+ /PPROPERTIES &z LABEL='Not Drivers or Missing'
+ /TABLE=qn1 BY qns3a
+ /CATEGORIES VARIABLES=qn1 [1 THRU 2, &x, 3 THRU 4, &y, SUBTOTAL='Drivers', 5, MISSING, &z]
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=140], [0], [dnl
+ Custom Tables
+╭──────────────────────────────────────────────────────────────────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├──────────────────────────────────────────────────────────────────────────────────────────┼─────┼──────┤
+│ 1. How often do you usually drive a car or other motor vehicle? Every day │ 2305│ 2362│
+│ Several days a week │ 440│ 834│
+│ Frequent Drivers │ 2745│ 3196│
+│ Once a week or less │ 125│ 236│
+│ Only certain times a year│ 58│ 72│
+│ Infrequent Drivers │ 183│ 308│
+│ Drivers │ 2928│ 3504│
+│ Never │ 192│ 348│
+│ Don't know │ 3│ 5│
+│ Refused │ 9│ 10│
+│ Not Drivers or Missing │ 204│ 363│
+╰──────────────────────────────────────────────────────────────────────────────────────────┴─────┴──────╯
+])
+AT_CLEANUP
+
+dnl I'm not sure that this is the correct behavior (see
+dnl https://mail.gnu.org/archive/html/pspp-users/2022-07/msg00002.html)
+dnl but at least this test will notify us if the behavior changes.
+AT_SETUP([CTABLES intersecting PCOMPUTEs])
+AT_KEYWORDS([PCOMPUTE postcompute])
+AT_DATA([ctables.sps],
+[[DATA LIST LIST NOTABLE/x y z.
+WEIGHT by z.
+FORMATS ALL (F1.0).
+VARIABLE LEVEL x y (NOMINAL).
+BEGIN DATA.
+1 4 5
+1 5 2
+1 6 9
+2 4 2
+2 5 3
+2 6 4
+3 4 1
+3 5 6
+3 6 1
+END DATA.
+
+CTABLES
+ /PCOMPUTE &a = EXPR([1] + [2])
+ /PCOMPUTE &b = EXPR([2] + [3])
+ /PCOMPUTE &c = EXPR([4] * [5])
+ /PCOMPUTE &d = EXPR([5] * [6])
+ /TABLE x BY y
+ /CATEGORIES VARIABLES=x [1, &a, 2, &b, 3]
+ /CATEGORIES VARIABLES=y [4, &c, 5, &d, 6].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0],
+[[ Custom Tables
+╭───────────┬─────────────────────────────────────╮
+│ │ y │
+│ ├─────┬─────────┬─────┬─────────┬─────┤
+│ │ 4 │[4] * [5]│ 5 │[5] * [6]│ 6 │
+│ ├─────┼─────────┼─────┼─────────┼─────┤
+│ │Count│ Count │Count│ Count │Count│
+├───────────┼─────┼─────────┼─────┼─────────┼─────┤
+│x 1 │ 5│ 10│ 2│ 18│ 9│
+│ [1] + [2]│ 7│ .│ 5│ .│ 13│
+│ 2 │ 2│ 6│ 3│ 12│ 4│
+│ [2] + [3]│ 3│ .│ 9│ .│ 5│
+│ 3 │ 1│ 6│ 6│ 6│ 1│
+╰───────────┴─────┴─────────┴─────┴─────────┴─────╯
+]])
+AT_CLEANUP
+
+AT_SETUP([CTABLES string and date and time])
+
+weight=1
+for gender in F M; do
+ for month in Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec; do
+ for wkday in Sun Mon Tue Wed Thu Fri Sat Sun; do
+ printf "$weight $gender $month $wkday\n"
+ weight=$(expr \( $weight + 3 \) % 7 + 2)
+ done
+ done
+done > ctables.txt
+
+AT_DATA([ctables.sps],
+[[DATA LIST LIST NOTABLE FILE='ctables.txt'
+ /w (F5.0) gender (A1) fmon (MONTH3) fday (WKDAY3).
+WEIGHT by w.
+VARIABLE LEVEL w (SCALE).
+VARIABLE LEVEL gender fmon fday (NOMINAL).
+VARIABLE LABEL
+ gender 'Gender'
+ fmon 'Favorite month'
+ fday 'Favorite day of the week'.
+VALUE LABELS /gender 'M' 'Male' 'F' 'Female'.
+CTABLES
+ /PCOMPUTE &q2 = EXPR(['APR' THRU 'June'])
+ /PPROPERTIES &q2 LABEL='Q2'
+ /PCOMPUTE &weekend = EXPR(['sun'] + ['Sat'])
+ /PPROPERTIES &weekend LABEL='Weekend'
+ /TABLE fmon BY gender > fday
+ /CATEGORIES VARIABLES=fmon ['JAN', 'FEB', 'Mar', SUBTOTAL="Q1",
+ 4 THRU 6, &q2,
+ 'JUL' THRU 'sep', SUBTOTAL="Q3",
+ OTHERNM, SUBTOTAL='Q4']
+ /CATEGORIES VARIABLES=gender ['M', 'F']
+ /CATEGORIES VARIABLE=fday ['Sun', 2 THRU 6, 'Sat', &weekend] TOTAL=YES
+ /SLABELS VISIBLE=NO.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+╭──────────────────┬───────────────────────────────────────────────────────────────────────────────────╮
+│ │ Gender │
+│ ├─────────────────────────────────────────┬─────────────────────────────────────────┤
+│ │ Male │ Female │
+│ ├─────────────────────────────────────────┼─────────────────────────────────────────┤
+│ │ Favorite day of the week │ Favorite day of the week │
+│ ├───┬───┬───┬───┬───┬───┬───┬───────┬─────┼───┬───┬───┬───┬───┬───┬───┬───────┬─────┤
+│ │SUN│MON│TUE│WED│THU│FRI│SAT│Weekend│Total│SUN│MON│TUE│WED│THU│FRI│SAT│Weekend│Total│
+├──────────────────┼───┼───┼───┼───┼───┼───┼───┼───────┼─────┼───┼───┼───┼───┼───┼───┼───┼───────┼─────┤
+│Favorite month JAN│ 10│ 3│ 8│ 6│ 4│ 2│ 7│ 17│ 40│ 9│ 6│ 4│ 2│ 7│ 5│ 3│ 12│ 36│
+│ FEB│ 6│ 8│ 6│ 4│ 2│ 7│ 5│ 11│ 38│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│
+│ MAR│ 16│ 6│ 4│ 2│ 7│ 5│ 3│ 19│ 43│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│
+│ Q1 │ 32│ 17│ 18│ 12│ 13│ 14│ 15│ │ │ 29│ 12│ 13│ 14│ 15│ 16│ 17│ │ │
+│ APR│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│ 4│ 7│ 5│ 3│ 8│ 6│ 4│ 8│ 37│
+│ MAY│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│ 14│ 5│ 3│ 8│ 6│ 4│ 2│ 16│ 42│
+│ JUN│ 4│ 7│ 5│ 3│ 8│ 6│ 4│ 8│ 37│ 10│ 3│ 8│ 6│ 4│ 2│ 7│ 17│ 40│
+│ Q2 │ 24│ 13│ 14│ 15│ 16│ 17│ 18│ .│ │ 28│ 15│ 16│ 17│ 18│ 12│ 13│ .│ │
+│ JUL│ 14│ 5│ 3│ 8│ 6│ 4│ 2│ 16│ 42│ 6│ 8│ 6│ 4│ 2│ 7│ 5│ 11│ 38│
+│ AUG│ 10│ 3│ 8│ 6│ 4│ 2│ 7│ 17│ 40│ 16│ 6│ 4│ 2│ 7│ 5│ 3│ 19│ 43│
+│ SEP│ 6│ 8│ 6│ 4│ 2│ 7│ 5│ 11│ 38│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│
+│ Q3 │ 54│ 29│ 31│ 33│ 28│ 30│ 32│ │ │ 62│ 33│ 28│ 30│ 32│ 27│ 29│ │ │
+│ OCT│ 16│ 6│ 4│ 2│ 7│ 5│ 3│ 19│ 43│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│
+│ NOV│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│ 4│ 7│ 5│ 3│ 8│ 6│ 4│ 8│ 37│
+│ DEC│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│ 14│ 5│ 3│ 8│ 6│ 4│ 2│ 16│ 42│
+│ Q4 │ 36│ 12│ 13│ 14│ 15│ 16│ 17│ │ │ 26│ 14│ 15│ 16│ 17│ 18│ 12│ │ │
+╰──────────────────┴───┴───┴───┴───┴───┴───┴───┴───────┴─────┴───┴───┴───┴───┴───┴───┴───┴───────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES CLABELS])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE AgeGroup BY qns3a.
+CTABLES /TABLE AgeGroup BY qns3a /CLABELS ROWLABELS=OPPOSITE.
+CTABLES /TABLE AgeGroup BY qns3a /CLABELS COLLABELS=OPPOSITE.
+CTABLES /TABLE AgeGroup BY qns3a /CLABELS ROWLABELS=LAYER.
+CTABLES /TABLE AgeGroup BY qns3a /CLABELS COLLABELS=LAYER.
+]])
+AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+╭───────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├───────────────────────┼─────┼──────┤
+│Age group 15 or younger│ 0│ 0│
+│ 16 to 25 │ 594│ 505│
+│ 26 to 35 │ 476│ 491│
+│ 36 to 45 │ 489│ 548│
+│ 46 to 55 │ 526│ 649│
+│ 56 to 65 │ 516│ 731│
+│ 66 or older │ 531│ 943│
+╰───────────────────────┴─────┴──────╯
+
+ Custom Tables
+╭───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ │ S3a. GENDER: │
+│ ├──────────────────────────────────────────────────────┬───────────────────────────────────────────────────────┤
+│ │ Male │ Female │
+│ ├─────────┬───────┬──────┬──────┬──────┬───────┬───────┼──────────┬──────┬───────┬──────┬──────┬──────┬────────┤
+│ │ 15 or │ 16 to │ 26 to│ 36 to│ 46 to│ 56 to │ 66 or │ 15 or │ 16 to│ 26 to │ 36 to│ 46 to│ 56 to│ 66 or │
+│ │ younger │ 25 │ 35 │ 45 │ 55 │ 65 │ older │ younger │ 25 │ 35 │ 45 │ 55 │ 65 │ older │
+│ ├─────────┼───────┼──────┼──────┼──────┼───────┼───────┼──────────┼──────┼───────┼──────┼──────┼──────┼────────┤
+│ │ Count │ Count │ Count│ Count│ Count│ Count │ Count │ Count │ Count│ Count │ Count│ Count│ Count│ Count │
+├───────┼─────────┼───────┼──────┼──────┼──────┼───────┼───────┼──────────┼──────┼───────┼──────┼──────┼──────┼────────┤
+│Age │ 0│ 594│ 476│ 489│ 526│ 516│ 531│ 0│ 505│ 491│ 548│ 649│ 731│ 943│
+│group │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
+╰───────┴─────────┴───────┴──────┴──────┴──────┴───────┴───────┴──────────┴──────┴───────┴──────┴──────┴──────┴────────╯
+
+ Custom Tables
+╭──────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├────────────┤
+│ │ Count │
+├──────────────────────────────┼────────────┤
+│Age group 15 or younger Male │ 0│
+│ Female│ 0│
+│ ╶────────────────────┼────────────┤
+│ 16 to 25 Male │ 594│
+│ Female│ 505│
+│ ╶────────────────────┼────────────┤
+│ 26 to 35 Male │ 476│
+│ Female│ 491│
+│ ╶────────────────────┼────────────┤
+│ 36 to 45 Male │ 489│
+│ Female│ 548│
+│ ╶────────────────────┼────────────┤
+│ 46 to 55 Male │ 526│
+│ Female│ 649│
+│ ╶────────────────────┼────────────┤
+│ 56 to 65 Male │ 516│
+│ Female│ 731│
+│ ╶────────────────────┼────────────┤
+│ 66 or older Male │ 531│
+│ Female│ 943│
+╰──────────────────────────────┴────────────╯
+
+ Custom Tables
+15 or younger
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 0│ 0│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+16 to 25
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 594│ 505│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+26 to 35
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 476│ 491│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+36 to 45
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 489│ 548│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+46 to 55
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 526│ 649│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+56 to 65
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 516│ 731│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+66 or older
+╭─────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────┼─────┼──────┤
+│Age group│ 531│ 943│
+╰─────────┴─────┴──────╯
+
+ Custom Tables
+Male
+╭───────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├────────────┤
+│ │ Count │
+├───────────────────────┼────────────┤
+│Age group 15 or younger│ 0│
+│ 16 to 25 │ 594│
+│ 26 to 35 │ 476│
+│ 36 to 45 │ 489│
+│ 46 to 55 │ 526│
+│ 56 to 65 │ 516│
+│ 66 or older │ 531│
+╰───────────────────────┴────────────╯
+
+ Custom Tables
+Female
+╭───────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├────────────┤
+│ │ Count │
+├───────────────────────┼────────────┤
+│Age group 15 or younger│ 0│
+│ 16 to 25 │ 505│
+│ 26 to 35 │ 491│
+│ 36 to 45 │ 548│
+│ 46 to 55 │ 649│
+│ 56 to 65 │ 731│
+│ 66 or older │ 943│
+╰───────────────────────┴────────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES missing values])
+AT_DATA([ctables.sps],
+[[DATA LIST LIST NOTABLE/x y.
+BEGIN DATA.
+1 1
+1 2
+1 3
+1 4
+1 5
+1 .
+2 1
+2 2
+2 3
+2 4
+2 5
+2 .
+3 1
+3 2
+3 3
+3 4
+3 5
+3 .
+4 1
+4 2
+4 3
+4 4
+4 5
+4 .
+5 1
+5 2
+5 3
+5 4
+5 5
+5 .
+. 1
+. 2
+. 3
+. 4
+. 5
+. .
+END DATA.
+MISSING VALUES x (1, 2) y (2, 3).
+VARIABLE LEVEL ALL (NOMINAL).
+
+CTABLES /TABLE x[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN,
+ TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, VALIDN, TOTALN]]
+ /CATEGORIES VARIABLES=ALL TOTAL=YES.
+CTABLES /TABLE x[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN,
+ TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, VALIDN, TOTALN]]
+ /CATEGORIES VARIABLES=ALL TOTAL=YES MISSING=INCLUDE.
+CTABLES /TABLE x BY y[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN,
+ TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, VALIDN, TOTALN]]
+ /CATEGORIES VARIABLES=ALL TOTAL=YES
+ /SLABELS POSITION=ROW.
+CTABLES /TABLE x BY y[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN,
+ TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, VALIDN, TOTALN]]
+ /CATEGORIES VARIABLES=ALL TOTAL=YES MISSING=INCLUDE
+ /SLABELS POSITION=ROW.
+CTABLES /TABLE x BY y[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN,
+ TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, VALIDN, TOTALN]]
+ /CATEGORIES VARIABLES=x [1, 2, 3, 4] TOTAL=YES
+ /CATEGORIES VARIABLES=y [1, 3, 4, 5] TOTAL=YES
+ /SLABELS POSITION=ROW.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+╭───────┬─────┬────────┬────────────────┬────────────────┬───────┬───────╮
+│ │Count│Column %│Column Valid N %│Column Total N %│Valid N│Total N│
+├───────┼─────┼────────┼────────────────┼────────────────┼───────┼───────┤
+│x 3.00 │ 6│ 33.3%│ 33.3%│ 16.7%│ │ │
+│ 4.00 │ 6│ 33.3%│ 33.3%│ 16.7%│ │ │
+│ 5.00 │ 6│ 33.3%│ 33.3%│ 16.7%│ │ │
+│ Total│ 18│ 100.0%│ 100.0%│ 100.0%│ 18│ 36│
+╰───────┴─────┴────────┴────────────────┴────────────────┴───────┴───────╯
+dnl Note that Column Total N % doesn't add up to 100 because missing
+dnl values are included in the total but not shown as a category and this
+dnl is expected behavior.
+
+ Custom Tables
+╭───────┬─────┬────────┬────────────────┬────────────────┬───────┬───────╮
+│ │Count│Column %│Column Valid N %│Column Total N %│Valid N│Total N│
+├───────┼─────┼────────┼────────────────┼────────────────┼───────┼───────┤
+│x 1.00 │ 6│ 20.0%│ .0%│ 16.7%│ │ │
+│ 2.00 │ 6│ 20.0%│ .0%│ 16.7%│ │ │
+│ 3.00 │ 6│ 20.0%│ 33.3%│ 16.7%│ │ │
+│ 4.00 │ 6│ 20.0%│ 33.3%│ 16.7%│ │ │
+│ 5.00 │ 6│ 20.0%│ 33.3%│ 16.7%│ │ │
+│ Total│ 30│ 100.0%│ 100.0%│ 100.0%│ 18│ 36│
+╰───────┴─────┴────────┴────────────────┴────────────────┴───────┴───────╯
+dnl Note that Column Total N % doesn't add up to 100 because system-missing
+dnl values are included in the total but not shown as a category and this
+dnl is expected behavior.
+
+ Custom Tables
+╭────────────────────────┬───────────────────────────╮
+│ │ y │
+│ ├──────┬──────┬──────┬──────┤
+│ │ 1.00 │ 4.00 │ 5.00 │ Total│
+├────────────────────────┼──────┼──────┼──────┼──────┤
+│x 3.00 Count │ 1│ 1│ 1│ 3│
+│ Column % │ 33.3%│ 33.3%│ 33.3%│ .│
+│ Column Valid N %│ 33.3%│ 33.3%│ 33.3%│ .│
+│ Column Total N %│ 33.3%│ 33.3%│ 33.3%│ .│
+│ Row % │ 33.3%│ 33.3%│ 33.3%│100.0%│
+│ Row Valid N % │ 33.3%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ 3│
+│ Total N │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┤
+│ 4.00 Count │ 1│ 1│ 1│ 3│
+│ Column % │ 33.3%│ 33.3%│ 33.3%│ .│
+│ Column Valid N %│ 33.3%│ 33.3%│ 33.3%│ .│
+│ Column Total N %│ 33.3%│ 33.3%│ 33.3%│ .│
+│ Row % │ 33.3%│ 33.3%│ 33.3%│100.0%│
+│ Row Valid N % │ 33.3%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ 3│
+│ Total N │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┤
+│ 5.00 Count │ 1│ 1│ 1│ 3│
+│ Column % │ 33.3%│ 33.3%│ 33.3%│ .│
+│ Column Valid N %│ 33.3%│ 33.3%│ 33.3%│ .│
+│ Column Total N %│ 33.3%│ 33.3%│ 33.3%│ .│
+│ Row % │ 33.3%│ 33.3%│ 33.3%│100.0%│
+│ Row Valid N % │ 33.3%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ 3│
+│ Total N │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┤
+│ Total Count │ 3│ 3│ 3│ 9│
+│ Column % │100.0%│100.0%│100.0%│ .│
+│ Column Valid N %│100.0%│100.0%│100.0%│ .│
+│ Column Total N %│100.0%│100.0%│100.0%│ .│
+│ Row % │ .│ .│ .│ .│
+│ Row Valid N % │ .│ .│ .│ .│
+│ Row Total N % │ .│ .│ .│ .│
+│ Valid N │ 3│ 3│ 3│ 9│
+│ Total N │ 3│ 3│ 3│ 18│
+╰────────────────────────┴──────┴──────┴──────┴──────╯
+
+ Custom Tables
+╭────────────────────────┬─────────────────────────────────────────╮
+│ │ y │
+│ ├──────┬──────┬──────┬──────┬──────┬──────┤
+│ │ 1.00 │ 2.00 │ 3.00 │ 4.00 │ 5.00 │ Total│
+├────────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│x 1.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
+│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
+│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ 2.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
+│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
+│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ 3.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
+│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
+│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ 4.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
+│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
+│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ 5.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
+│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
+│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
+│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ Total Count │ 5│ 5│ 5│ 5│ 5│ 25│
+│ Column % │100.0%│100.0%│100.0%│100.0%│100.0%│ .│
+│ Column Valid N %│100.0%│ .│ .│100.0%│100.0%│ .│
+│ Column Total N %│100.0%│100.0%│100.0%│100.0%│100.0%│ .│
+│ Row % │ .│ .│ .│ .│ .│ .│
+│ Row Valid N % │ .│ .│ .│ .│ .│ .│
+│ Row Total N % │ .│ .│ .│ .│ .│ .│
+│ Valid N │ 5│ 0│ 0│ 5│ 5│ 15│
+│ Total N │ 5│ 5│ 5│ 5│ 5│ 30│
+╰────────────────────────┴──────┴──────┴──────┴──────┴──────┴──────╯
+
+ Custom Tables
+╭────────────────────────┬──────────────────────────────────╮
+│ │ y │
+│ ├──────┬──────┬──────┬──────┬──────┤
+│ │ 1.00 │ 3.00 │ 4.00 │ 5.00 │ Total│
+├────────────────────────┼──────┼──────┼──────┼──────┼──────┤
+│x 1.00 Count │ 1│ 1│ 1│ 1│ 4│
+│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
+│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
+│ 2.00 Count │ 1│ 1│ 1│ 1│ 4│
+│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
+│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
+│ 3.00 Count │ 1│ 1│ 1│ 1│ 4│
+│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
+│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
+│ 4.00 Count │ 1│ 1│ 1│ 1│ 4│
+│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
+│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
+│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
+│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
+│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
+│ Valid N │ │ │ │ │ 3│
+│ Total N │ │ │ │ │ 6│
+│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
+│ Total Count │ 4│ 4│ 4│ 4│ 16│
+│ Column % │100.0%│100.0%│100.0%│100.0%│ .│
+│ Column Valid N %│100.0%│ .│100.0%│100.0%│ .│
+│ Column Total N %│100.0%│100.0%│100.0%│100.0%│ .│
+│ Row % │ .│ .│ .│ .│ .│
+│ Row Valid N % │ .│ .│ .│ .│ .│
+│ Row Total N % │ .│ .│ .│ .│ .│
+│ Valid N │ 4│ 0│ 4│ 4│ 12│
+│ Total N │ 4│ 4│ 4│ 4│ 24│
+╰────────────────────────┴──────┴──────┴──────┴──────┴──────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES SMISSING=LISTWISE])
+AT_KEYWORDS([SMISSING LISTWISE])
+AT_DATA([ctables.sps],
+[[DATA LIST LIST NOTABLE/x y z.
+BEGIN DATA.
+1 . 40
+1 10 50
+1 20 60
+1 . .
+1 30 .
+END DATA.
+VARIABLE LEVEL x (NOMINAL).
+
+CTABLES /TABLE (y + z) > x.
+CTABLES /SMISSING LISTWISE /TABLE (y + z) > x.
+
+* The following doesn't come out as listwise because the tables are
+separate, not linked by an > operator.
+CTABLES /SMISSING LISTWISE /TABLE (y > x) + (z > x).
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+╭────────┬─────╮
+│ │ Mean│
+├────────┼─────┤
+│y x 1.00│20.00│
+├────────┼─────┤
+│z x 1.00│50.00│
+╰────────┴─────╯
+
+ Custom Tables
+╭────────┬─────╮
+│ │ Mean│
+├────────┼─────┤
+│y x 1.00│15.00│
+├────────┼─────┤
+│z x 1.00│55.00│
+╰────────┴─────╯
+
+ Custom Tables
+╭────────┬─────╮
+│ │ Mean│
+├────────┼─────┤
+│y x 1.00│20.00│
+├────────┼─────┤
+│z x 1.00│50.00│
+╰────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES VLABELS - variables on different axes])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=DEFAULT /TABLE qnd5a BY qns3a.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NAME /TABLE qnd5a BY qns3a.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=LABEL /TABLE qnd5a BY qns3a.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=BOTH /TABLE qnd5a BY qns3a.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NONE /TABLE qnd5a BY qns3a.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭────────────────────────────────────────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├────────────────────────────────────────────────────────────────┼─────┼──────┤
+│D5a. What would you say is your primary Cuban │ 13│ 7│
+│ethnic background? Mexican │ 175│ 136│
+│ Spanish │ 20│ 28│
+│ South American │ 21│ 13│
+│ Central American │ 27│ 25│
+│ Puerto Rican, OR │ 37│ 41│
+│ Something else │ 35│ 33│
+│ Multiple - cannot │ 2│ 5│
+│ choose one │ │ │
+╰────────────────────────────────────────────────────────────────┴─────┴──────╯
+
+ Custom Tables
+╭──────────────────────────────────┬────────────╮
+│ │ QNS3A │
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├──────────────────────────────────┼─────┼──────┤
+│QND5A Cuban │ 13│ 7│
+│ Mexican │ 175│ 136│
+│ Spanish │ 20│ 28│
+│ South American │ 21│ 13│
+│ Central American │ 27│ 25│
+│ Puerto Rican, OR │ 37│ 41│
+│ Something else │ 35│ 33│
+│ Multiple - cannot choose one│ 2│ 5│
+╰──────────────────────────────────┴─────┴──────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├────────────────────────────────────────────────────────────────┼─────┼──────┤
+│D5a. What would you say is your primary Cuban │ 13│ 7│
+│ethnic background? Mexican │ 175│ 136│
+│ Spanish │ 20│ 28│
+│ South American │ 21│ 13│
+│ Central American │ 27│ 25│
+│ Puerto Rican, OR │ 37│ 41│
+│ Something else │ 35│ 33│
+│ Multiple - cannot │ 2│ 5│
+│ choose one │ │ │
+╰────────────────────────────────────────────────────────────────┴─────┴──────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────┬────────────────╮
+│ │ QNS3A S3a. │
+│ │ GENDER: │
+│ ├───────┬────────┤
+│ │ Male │ Female │
+│ ├───────┼────────┤
+│ │ Count │ Count │
+├────────────────────────────────────────────────────────────┼───────┼────────┤
+│QND5A D5a. What would you say is your Cuban │ 13│ 7│
+│primary ethnic background? Mexican │ 175│ 136│
+│ Spanish │ 20│ 28│
+│ South American │ 21│ 13│
+│ Central American │ 27│ 25│
+│ Puerto Rican, OR │ 37│ 41│
+│ Something else │ 35│ 33│
+│ Multiple - cannot │ 2│ 5│
+│ choose one │ │ │
+╰────────────────────────────────────────────────────────────┴───────┴────────╯
+
+ Custom Tables
+╭────────────────────────────┬─────┬──────╮
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├────────────────────────────┼─────┼──────┤
+│Cuban │ 13│ 7│
+│Mexican │ 175│ 136│
+│Spanish │ 20│ 28│
+│South American │ 21│ 13│
+│Central American │ 27│ 25│
+│Puerto Rican, OR │ 37│ 41│
+│Something else │ 35│ 33│
+│Multiple - cannot choose one│ 2│ 5│
+╰────────────────────────────┴─────┴──────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES VLABELS - stacked variables])
+AT_KEYWORDS([stack stacking])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NAME /TABLE qnd5a + qns3a.
+CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NAME /TABLE qnd5a + qns3a.
+CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NONE /TABLE qnd5a + qns3a.
+CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NONE /TABLE qnd5a + qns3a.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NONE /TABLE qnd5a + qns3a.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭───────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary ethnic Cuban │ 20│
+│background? Mexican │ 311│
+│ Spanish │ 48│
+│ South American │ 34│
+│ Central American │ 52│
+│ Puerto Rican, OR │ 78│
+│ Something else │ 68│
+│ Multiple - cannot │ 7│
+│ choose one │ │
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│QNS3A Male │ 3132│
+│ Female │ 3867│
+╰───────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭─────────────────────────────────────────┬─────╮
+│ │Count│
+├─────────────────────────────────────────┼─────┤
+│QND5A Cuban │ 20│
+│ Mexican │ 311│
+│ Spanish │ 48│
+│ South American │ 34│
+│ Central American │ 52│
+│ Puerto Rican, OR │ 78│
+│ Something else │ 68│
+│ Multiple - cannot choose one│ 7│
+├─────────────────────────────────────────┼─────┤
+│S3a. GENDER: Male │ 3132│
+│ Female │ 3867│
+╰─────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭───────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary ethnic Cuban │ 20│
+│background? Mexican │ 311│
+│ Spanish │ 48│
+│ South American │ 34│
+│ Central American │ 52│
+│ Puerto Rican, OR │ 78│
+│ Something else │ 68│
+│ Multiple - cannot │ 7│
+│ choose one │ │
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│Male │ 3132│
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│Female │ 3867│
+╰───────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭─────────────────────────────┬─────╮
+│ │Count│
+├─────────────────────────────┼─────┤
+│Cuban │ 20│
+├─────────────────────────────┼─────┤
+│Mexican │ 311│
+├─────────────────────────────┼─────┤
+│Spanish │ 48│
+├─────────────────────────────┼─────┤
+│South American │ 34│
+├─────────────────────────────┼─────┤
+│Central American │ 52│
+├─────────────────────────────┼─────┤
+│Puerto Rican, OR │ 78│
+├─────────────────────────────┼─────┤
+│Something else │ 68│
+├─────────────────────────────┼─────┤
+│Multiple - cannot choose one │ 7│
+├─────────────────────────────┼─────┤
+│S3a. GENDER: Male │ 3132│
+│ Female │ 3867│
+╰─────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────┼─────┤
+│Cuban │ 20│
+│Mexican │ 311│
+│Spanish │ 48│
+│South American │ 34│
+│Central American │ 52│
+│Puerto Rican, OR │ 78│
+│Something else │ 68│
+│Multiple - cannot choose one│ 7│
+│Male │ 3132│
+│Female │ 3867│
+╰────────────────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES VLABELS - nested variables])
+AT_KEYWORDS([nest nesting])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NAME /TABLE qnd5a > qns3a.
+CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NAME /TABLE qnd5a > qns3a.
+CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NONE /TABLE qnd5a > qns3a.
+CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NONE /TABLE qnd5a > qns3a.
+CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NONE /TABLE qnd5a > qns3a.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭───────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your Cuban QNS3A Male │ 13│
+│primary ethnic background? Female│ 7│
+│ ╶─────────────────────────────────┼─────┤
+│ Mexican QNS3A Male │ 175│
+│ Female│ 136│
+│ ╶─────────────────────────────────┼─────┤
+│ Spanish QNS3A Male │ 20│
+│ Female│ 28│
+│ ╶─────────────────────────────────┼─────┤
+│ South American QNS3A Male │ 21│
+│ Female│ 13│
+│ ╶─────────────────────────────────┼─────┤
+│ Central American QNS3A Male │ 27│
+│ Female│ 25│
+│ ╶─────────────────────────────────┼─────┤
+│ Puerto Rican, OR QNS3A Male │ 37│
+│ Female│ 41│
+│ ╶─────────────────────────────────┼─────┤
+│ Something else QNS3A Male │ 35│
+│ Female│ 33│
+│ ╶─────────────────────────────────┼─────┤
+│ Multiple - cannot QNS3A Male │ 2│
+│ choose one Female│ 5│
+╰───────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├──────────────────────────────────────────────────────┼─────┤
+│QND5A Cuban S3a. GENDER: Male │ 13│
+│ Female│ 7│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ Mexican S3a. GENDER: Male │ 175│
+│ Female│ 136│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ Spanish S3a. GENDER: Male │ 20│
+│ Female│ 28│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ South American S3a. GENDER: Male │ 21│
+│ Female│ 13│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ Central American S3a. GENDER: Male │ 27│
+│ Female│ 25│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ Puerto Rican, OR S3a. GENDER: Male │ 37│
+│ Female│ 41│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ Something else S3a. GENDER: Male │ 35│
+│ Female│ 33│
+│ ╶────────────────────────────────────────────────┼─────┤
+│ Multiple - cannot choose one S3a. GENDER: Male │ 2│
+│ Female│ 5│
+╰──────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭───────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├───────────────────────────────────────────────────────────────────────┼─────┤
+│D5a. What would you say is your primary Cuban Male │ 13│
+│ethnic background? Female│ 7│
+│ ╶────────────────────────────┼─────┤
+│ Mexican Male │ 175│
+│ Female│ 136│
+│ ╶────────────────────────────┼─────┤
+│ Spanish Male │ 20│
+│ Female│ 28│
+│ ╶────────────────────────────┼─────┤
+│ South American Male │ 21│
+│ Female│ 13│
+│ ╶────────────────────────────┼─────┤
+│ Central American Male │ 27│
+│ Female│ 25│
+│ ╶────────────────────────────┼─────┤
+│ Puerto Rican, OR Male │ 37│
+│ Female│ 41│
+│ ╶────────────────────────────┼─────┤
+│ Something else Male │ 35│
+│ Female│ 33│
+│ ╶────────────────────────────┼─────┤
+│ Multiple - cannot Male │ 2│
+│ choose one Female│ 5│
+╰───────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────┼─────┤
+│Cuban S3a. GENDER: Male │ 13│
+│ Female│ 7│
+├────────────────────────────────────────────────┼─────┤
+│Mexican S3a. GENDER: Male │ 175│
+│ Female│ 136│
+├────────────────────────────────────────────────┼─────┤
+│Spanish S3a. GENDER: Male │ 20│
+│ Female│ 28│
+├────────────────────────────────────────────────┼─────┤
+│South American S3a. GENDER: Male │ 21│
+│ Female│ 13│
+├────────────────────────────────────────────────┼─────┤
+│Central American S3a. GENDER: Male │ 27│
+│ Female│ 25│
+├────────────────────────────────────────────────┼─────┤
+│Puerto Rican, OR S3a. GENDER: Male │ 37│
+│ Female│ 41│
+├────────────────────────────────────────────────┼─────┤
+│Something else S3a. GENDER: Male │ 35│
+│ Female│ 33│
+├────────────────────────────────────────────────┼─────┤
+│Multiple - cannot choose one S3a. GENDER: Male │ 2│
+│ Female│ 5│
+╰────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭───────────────────────────────────┬─────╮
+│ │Count│
+├───────────────────────────────────┼─────┤
+│Cuban Male │ 13│
+│ Female│ 7│
+├───────────────────────────────────┼─────┤
+│Mexican Male │ 175│
+│ Female│ 136│
+├───────────────────────────────────┼─────┤
+│Spanish Male │ 20│
+│ Female│ 28│
+├───────────────────────────────────┼─────┤
+│South American Male │ 21│
+│ Female│ 13│
+├───────────────────────────────────┼─────┤
+│Central American Male │ 27│
+│ Female│ 25│
+├───────────────────────────────────┼─────┤
+│Puerto Rican, OR Male │ 37│
+│ Female│ 41│
+├───────────────────────────────────┼─────┤
+│Something else Male │ 35│
+│ Female│ 33│
+├───────────────────────────────────┼─────┤
+│Multiple - cannot choose one Male │ 2│
+│ Female│ 5│
+╰───────────────────────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES FORMAT EMPTY])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /FORMAT EMPTY=ZERO /TABLE qnd5a BY qnd5.
+CTABLES /FORMAT EMPTY=BLANK /TABLE qnd5a BY qnd5.
+CTABLES /FORMAT EMPTY='n/a' /TABLE qnd5a BY qnd5.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭─────────────────────────────────────────────┬───────────────────────────────╮
+│ │ D5. ETHNICITY: Are you of │
+│ │ Hispanic or Latino origin or │
+│ │ descent? │
+│ ├───────────────┬───────────────┤
+│ │ Yes │ No │
+│ ├───────────────┼───────────────┤
+│ │ Count │ Count │
+├─────────────────────────────────────────────┼───────────────┼───────────────┤
+│D5a. What would you say is Cuban │ 20│ 0│
+│your primary ethnic Mexican │ 311│ 0│
+│background? Spanish │ 48│ 0│
+│ South American │ 34│ 0│
+│ Central American│ 52│ 0│
+│ Puerto Rican, OR│ 78│ 0│
+│ Something else │ 68│ 0│
+│ Multiple - │ 7│ 0│
+│ cannot choose │ │ │
+│ one │ │ │
+╰─────────────────────────────────────────────┴───────────────┴───────────────╯
+
+ Custom Tables
+╭─────────────────────────────────────────────┬───────────────────────────────╮
+│ │ D5. ETHNICITY: Are you of │
+│ │ Hispanic or Latino origin or │
+│ │ descent? │
+│ ├───────────────┬───────────────┤
+│ │ Yes │ No │
+│ ├───────────────┼───────────────┤
+│ │ Count │ Count │
+├─────────────────────────────────────────────┼───────────────┼───────────────┤
+│D5a. What would you say is Cuban │ 20│ │
+│your primary ethnic Mexican │ 311│ │
+│background? Spanish │ 48│ │
+│ South American │ 34│ │
+│ Central American│ 52│ │
+│ Puerto Rican, OR│ 78│ │
+│ Something else │ 68│ │
+│ Multiple - │ 7│ │
+│ cannot choose │ │ │
+│ one │ │ │
+╰─────────────────────────────────────────────┴───────────────┴───────────────╯
+
+ Custom Tables
+╭─────────────────────────────────────────────┬───────────────────────────────╮
+│ │ D5. ETHNICITY: Are you of │
+│ │ Hispanic or Latino origin or │
+│ │ descent? │
+│ ├───────────────┬───────────────┤
+│ │ Yes │ No │
+│ ├───────────────┼───────────────┤
+│ │ Count │ Count │
+├─────────────────────────────────────────────┼───────────────┼───────────────┤
+│D5a. What would you say is Cuban │ 20│n/a │
+│your primary ethnic Mexican │ 311│n/a │
+│background? Spanish │ 48│n/a │
+│ South American │ 34│n/a │
+│ Central American│ 52│n/a │
+│ Puerto Rican, OR│ 78│n/a │
+│ Something else │ 68│n/a │
+│ Multiple - │ 7│n/a │
+│ cannot choose │ │ │
+│ one │ │ │
+╰─────────────────────────────────────────────┴───────────────┴───────────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES FORMAT MISSING])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /FORMAT MISSING='(no data)' /TABLE qnd5a[COLPCT] BY qnd5.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭─────────────────────────────────────────────┬───────────────────────────────╮
+│ │ D5. ETHNICITY: Are you of │
+│ │ Hispanic or Latino origin or │
+│ │ descent? │
+│ ├───────────────┬───────────────┤
+│ │ Yes │ No │
+│ ├───────────────┼───────────────┤
+│ │ Column % │ Column % │
+├─────────────────────────────────────────────┼───────────────┼───────────────┤
+│D5a. What would you say is Cuban │ 3.2%│(no data) │
+│your primary ethnic Mexican │ 50.3%│(no data) │
+│background? Spanish │ 7.8%│(no data) │
+│ South American │ 5.5%│(no data) │
+│ Central American│ 8.4%│(no data) │
+│ Puerto Rican, OR│ 12.6%│(no data) │
+│ Something else │ 11.0%│(no data) │
+│ Multiple - │ 1.1%│(no data) │
+│ cannot choose │ │ │
+│ one │ │ │
+╰─────────────────────────────────────────────┴───────────────┴───────────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES HIDESMALLCOUNTS])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /TABLE qn38[c][COUNT, COLPCT].
+CTABLES /HIDESMALLCOUNTS /TABLE qn38[c][COUNT, COLPCT].
+CTABLES /HIDESMALLCOUNTS COUNT=10 /TABLE qn38[c][COUNT, COLPCT].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭──────────────────────────────────────────────────────────────┬─────┬────────╮
+│ │Count│Column %│
+├──────────────────────────────────────────────────────────────┼─────┼────────┤
+│38. How many drinks did you have on that Less than one│ 7│ .5%│
+│occasion? 1 │ 491│ 34.9%│
+│ 2 │ 462│ 32.9%│
+│ 3 │ 229│ 16.3%│
+│ 4 │ 82│ 5.8%│
+│ 5 │ 56│ 4.0%│
+│ 6 │ 32│ 2.3%│
+│ 7 │ 9│ .6%│
+│ 8 │ 8│ .6%│
+│ 9 │ 4│ .3%│
+│ 10 │ 6│ .4%│
+│ 11 │ 2│ .1%│
+│ 12 │ 5│ .4%│
+│ 14 │ 1│ .1%│
+│ 15 │ 1│ .1%│
+│ 18 │ 1│ .1%│
+│ 20 │ 4│ .3%│
+│ 25 │ 1│ .1%│
+│ 30 │ 3│ .2%│
+│ 60 │ 1│ .1%│
+│ 99+ │ 0│ .0%│
+╰──────────────────────────────────────────────────────────────┴─────┴────────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────────────┬─────┬────────╮
+│ │Count│Column %│
+├──────────────────────────────────────────────────────────────┼─────┼────────┤
+│38. How many drinks did you have on that Less than one│ 7│ .5%│
+│occasion? 1 │ 491│ 34.9%│
+│ 2 │ 462│ 32.9%│
+│ 3 │ 229│ 16.3%│
+│ 4 │ 82│ 5.8%│
+│ 5 │ 56│ 4.0%│
+│ 6 │ 32│ 2.3%│
+│ 7 │ 9│ .6%│
+│ 8 │ 8│ .6%│
+│ 9 │<5 │ .3%│
+│ 10 │ 6│ .4%│
+│ 11 │<5 │ .1%│
+│ 12 │ 5│ .4%│
+│ 14 │<5 │ .1%│
+│ 15 │<5 │ .1%│
+│ 18 │<5 │ .1%│
+│ 20 │<5 │ .3%│
+│ 25 │<5 │ .1%│
+│ 30 │<5 │ .2%│
+│ 60 │<5 │ .1%│
+│ 99+ │<5 │ .0%│
+╰──────────────────────────────────────────────────────────────┴─────┴────────╯
+
+ Custom Tables
+╭──────────────────────────────────────────────────────────────┬─────┬────────╮
+│ │Count│Column %│
+├──────────────────────────────────────────────────────────────┼─────┼────────┤
+│38. How many drinks did you have on that Less than one│<10 │ .5%│
+│occasion? 1 │ 491│ 34.9%│
+│ 2 │ 462│ 32.9%│
+│ 3 │ 229│ 16.3%│
+│ 4 │ 82│ 5.8%│
+│ 5 │ 56│ 4.0%│
+│ 6 │ 32│ 2.3%│
+│ 7 │<10 │ .6%│
+│ 8 │<10 │ .6%│
+│ 9 │<10 │ .3%│
+│ 10 │<10 │ .4%│
+│ 11 │<10 │ .1%│
+│ 12 │<10 │ .4%│
+│ 14 │<10 │ .1%│
+│ 15 │<10 │ .1%│
+│ 18 │<10 │ .1%│
+│ 20 │<10 │ .3%│
+│ 25 │<10 │ .1%│
+│ 30 │<10 │ .2%│
+│ 60 │<10 │ .1%│
+│ 99+ │<10 │ .0%│
+╰──────────────────────────────────────────────────────────────┴─────┴────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES FORMAT MINCOLWIDTH MAXCOLWIDTH])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES /FORMAT MINCOLWIDTH=1 MAXCOLWIDTH=2 UNITS=INCHES /TABLE BY qns3a.
+]])
+AT_CHECK([pspp ctables.sps -o - -O box=unicode -o pspp.spv], [0], [dnl
+ Custom Tables
+╭────────────╮
+│S3a. GENDER:│
+├─────┬──────┤
+│ Male│Female│
+├─────┼──────┤
+│Count│ Count│
+├─────┼──────┤
+│ 3132│ 3867│
+╰─────┴──────╯
+])
+AT_CHECK([pspp-output get-table-look pspp.spv pspp.stt])
+AT_CHECK([sed 's/ /\n/g' pspp.stt | grep ColumnWidth | sort], [0], [dnl
+maximumColumnWidth="192"
+minimumColumnWidth="96"
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES special formats])
+AT_KEYWORDS([NEGPAREN NEQUAL PAREN PCTPAREN])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+COMPUTE x = qnd3 - 4.
+CTABLES /TABLE x[MINIMUM NEGPAREN8.1, MINIMUM NEQUAL8.1, MINIMUM PAREN8.1, MINIMUM PCTPAREN8.1, MAXIMUM NEGPAREN8.1, MAXIMUM NEQUAL8.1, MAXIMUM PAREN8.1, MAXIMUM PCTPAREN8.1].
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭─┬───────┬───────┬───────┬───────┬───────┬───────┬───────┬───────╮
+│ │Minimum│Minimum│Minimum│Minimum│Maximum│Maximum│Maximum│Maximum│
+├─┼───────┼───────┼───────┼───────┼───────┼───────┼───────┼───────┤
+│x│(3.0) │N=-3.0 │(-3.0) │(-3.0%)│8.0 │N=8.0 │(8.0) │(8.0%) │
+╰─┴───────┴───────┴───────┴───────┴───────┴───────┴───────┴───────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES TITLES])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=qn1 DISPLAY=NONE
+ /TABLE ((qn1[c][COUNT])) BY qns3a[c] > qnd5
+ /TITLES TITLE='How often do you drive?' 'second line of title'
+ CAPTION='Generated )TIME on )DATE'
+ CORNER=')TABLE'.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode | sed 's/..:..:../HH:MM:SS/
+s&../../..&MM/DD/YY&'], [0], [dnl
+ How often do you drive?
+ second line of title
+╭───────────────────────────────────┬─────────────────────────────────────────╮
+│ │ S3a. GENDER: │
+│ ├────────────────────┬────────────────────┤
+│ │ Male │ Female │
+│ ├────────────────────┼────────────────────┤
+│ │ D5. ETHNICITY: Are │ D5. ETHNICITY: Are │
+│ │ you of Hispanic or │ you of Hispanic or │
+│ │ Latino origin or │ Latino origin or │
+│( ( 1. How often do you usually │ descent? │ descent? │
+│drive a car or other motor ├─────────┬──────────┼─────────┬──────────┤
+│vehicle?) ) BY S3a. GENDER: > D5. │ Yes │ No │ Yes │ No │
+│ETHNICITY: Are you of Hispanic or ├─────────┼──────────┼─────────┼──────────┤
+│Latino origin or descent? │ Count │ Count │ Count │ Count │
+├───────────────────────────────────┼─────────┼──────────┼─────────┼──────────┤
+│Every day │ 218│ 2066│ 166│ 2175│
+│Several days a week │ 44│ 391│ 45│ 782│
+│Once a week or less │ 16│ 109│ 12│ 223│
+│Only certain times a year │ 15│ 41│ 11│ 61│
+│Never │ 39│ 150│ 56│ 278│
+╰───────────────────────────────────┴─────────┴──────────┴─────────┴──────────╯
+Generated HH:MM:SS on MM/DD/YY
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES area definitions])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=ALL DISPLAY=NAME
+ /TABLE qn61 > qn57 BY qnd7a > qn86 + qn64b BY qns3a[TABLEID, LAYERID, SUBTABLEID]
+ /SLABELS POSITION=ROW
+ /TABLE qn61 > qn57 BY qnd7a > qn86 + qn64b BY qns3a[ROWID, LAYERROWID]
+ /SLABELS POSITION=ROW
+ /TABLE qn61 > qn57 BY qnd7a > qn86 + qn64b BY qns3a[COLID, LAYERCOLID]
+ /SLABELS POSITION=ROW.
+]])
+AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+Male
+╭─────────────────────────────┬─────────────┬──────╮
+│ │ QND7A │ QN64B│
+│ ├──────┬──────┼───┬──┤
+│ │ Yes │ No │ │ │
+│ ├──────┼──────┤ │ │
+│ │ QN86 │ QN86 │ │ │
+│ ├───┬──┼───┬──┤ │ │
+│ │Yes│No│Yes│No│Yes│No│
+├─────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│
+│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
+│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Subtable ID│ 4│ 4│ 5│ 5│ 6│ 6│
+│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
+│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Subtable ID│ 4│ 4│ 5│ 5│ 6│ 6│
+╰─────────────────────────────┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+╭─────────────────────────────┬─────────────┬──────╮
+│ │ QND7A │ QN64B│
+│ ├──────┬──────┼───┬──┤
+│ │ Yes │ No │ │ │
+│ ├──────┼──────┤ │ │
+│ │ QN86 │ QN86 │ │ │
+│ ├───┬──┼───┬──┤ │ │
+│ │Yes│No│Yes│No│Yes│No│
+├─────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
+│ Subtable ID│ 7│ 7│ 8│ 8│ 9│ 9│
+│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
+│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
+│ Subtable ID│ 7│ 7│ 8│ 8│ 9│ 9│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
+│ Subtable ID│ 10│10│ 11│11│ 12│12│
+│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
+│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
+│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
+│ Subtable ID│ 10│10│ 11│11│ 12│12│
+╰─────────────────────────────┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+╭──────────────────────────────┬─────────────┬──────╮
+│ │ QND7A │ QN64B│
+│ ├──────┬──────┼───┬──┤
+│ │ Yes │ No │ │ │
+│ ├──────┼──────┤ │ │
+│ │ QN86 │ QN86 │ │ │
+│ ├───┬──┼───┬──┤ │ │
+│ │Yes│No│Yes│No│Yes│No│
+├──────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│QN61 Yes QN57 Yes Row ID │ 1│ 1│ 2│ 2│ 3│ 3│
+│ Layer Row ID│ 1│ 1│ 1│ 1│ 2│ 2│
+│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Row ID │ 4│ 4│ 5│ 5│ 6│ 6│
+│ Layer Row ID│ 3│ 3│ 3│ 3│ 4│ 4│
+│ ╶─────────────────────────┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Yes Row ID │ 7│ 7│ 8│ 8│ 9│ 9│
+│ Layer Row ID│ 5│ 5│ 5│ 5│ 6│ 6│
+│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Row ID │ 10│10│ 11│11│ 12│12│
+│ Layer Row ID│ 7│ 7│ 7│ 7│ 8│ 8│
+╰──────────────────────────────┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+╭──────────────────────────────┬─────────────┬──────╮
+│ │ QND7A │ QN64B│
+│ ├──────┬──────┼───┬──┤
+│ │ Yes │ No │ │ │
+│ ├──────┼──────┤ │ │
+│ │ QN86 │ QN86 │ │ │
+│ ├───┬──┼───┬──┤ │ │
+│ │Yes│No│Yes│No│Yes│No│
+├──────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│QN61 Yes QN57 Yes Row ID │ 13│13│ 14│14│ 15│15│
+│ Layer Row ID│ 9│ 9│ 9│ 9│ 10│10│
+│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Row ID │ 16│16│ 17│17│ 18│18│
+│ Layer Row ID│ 11│11│ 11│11│ 12│12│
+│ ╶─────────────────────────┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Yes Row ID │ 19│19│ 20│20│ 21│21│
+│ Layer Row ID│ 13│13│ 13│13│ 14│14│
+│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Row ID │ 22│22│ 23│23│ 24│24│
+│ Layer Row ID│ 15│15│ 15│15│ 16│16│
+╰──────────────────────────────┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+╭─────────────────────────────────┬─────────────┬──────╮
+│ │ QND7A │ QN64B│
+│ ├──────┬──────┼───┬──┤
+│ │ Yes │ No │ │ │
+│ ├──────┼──────┤ │ │
+│ │ QN86 │ QN86 │ │ │
+│ ├───┬──┼───┬──┤ │ │
+│ │Yes│No│Yes│No│Yes│No│
+├─────────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│QN61 Yes QN57 Yes Column ID │ 1│ 2│ 3│ 4│ 5│ 6│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
+│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Column ID │ 1│ 2│ 3│ 4│ 5│ 6│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
+│ ╶────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Yes Column ID │ 7│ 8│ 9│10│ 11│12│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
+│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Column ID │ 7│ 8│ 9│10│ 11│12│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
+╰─────────────────────────────────┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+╭─────────────────────────────────┬─────────────┬──────╮
+│ │ QND7A │ QN64B│
+│ ├──────┬──────┼───┬──┤
+│ │ Yes │ No │ │ │
+│ ├──────┼──────┤ │ │
+│ │ QN86 │ QN86 │ │ │
+│ ├───┬──┼───┬──┤ │ │
+│ │Yes│No│Yes│No│Yes│No│
+├─────────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│QN61 Yes QN57 Yes Column ID │ 13│14│ 15│16│ 17│18│
+│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
+│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Column ID │ 13│14│ 15│16│ 17│18│
+│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
+│ ╶────────────────────────────┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Yes Column ID │ 19│20│ 21│22│ 23│24│
+│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
+│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
+│ No Column ID │ 19│20│ 21│22│ 23│24│
+│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
+╰─────────────────────────────────┴───┴──┴───┴──┴───┴──╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES area definitions with CLABELS ROWLABELS=OPPOSITE])
+AT_KEYWORDS([ROWLABELS OPPOSITE])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=ALL DISPLAY=NAME
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=OPPOSITE
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=OPPOSITE
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=OPPOSITE.
+]])
+AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+Male
+╭──────────────────────────────────┬───────────────────────────────────────────────────────╮
+│ │ QN27 │
+│ ├───────────────────────────┬───────────────────────────┤
+│ │ Yes │ No │
+│ ├───────────────────────────┼───────────────────────────┤
+│ │ QND7A │ QND7A │
+│ ├─────────────┬─────────────┼─────────────┬─────────────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├─────────────┼─────────────┼─────────────┼─────────────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│ 4│ 4│ 5│ 5│ 6│ 6│ 7│ 7│ 8│ 8│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│ 4│ 4│ 5│ 5│ 6│ 6│ 7│ 7│ 8│ 8│
+│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 10│10│ 11│11│ 12│12│ 13│13│ 14│14│ 15│15│ 16│16│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 10│10│ 11│11│ 12│12│ 13│13│ 14│14│ 15│15│ 16│16│
+╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+╭──────────────────────────────────┬───────────────────────────────────────────────────────╮
+│ │ QN27 │
+│ ├───────────────────────────┬───────────────────────────┤
+│ │ Yes │ No │
+│ ├───────────────────────────┼───────────────────────────┤
+│ │ QND7A │ QND7A │
+│ ├─────────────┬─────────────┼─────────────┬─────────────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├─────────────┼─────────────┼─────────────┼─────────────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 17│17│ 18│18│ 19│19│ 20│20│ 21│21│ 22│22│ 23│23│ 24│24│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 17│17│ 18│18│ 19│19│ 20│20│ 21│21│ 22│22│ 23│23│ 24│24│
+│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 25│25│ 26│26│ 27│27│ 28│28│ 29│29│ 30│30│ 31│31│ 32│32│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 25│25│ 26│26│ 27│27│ 28│28│ 29│29│ 30│30│ 31│31│ 32│32│
+╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+╭───────────────────────────────────┬───────────────────────────────────────────────────────╮
+│ │ QN27 │
+│ ├───────────────────────────┬───────────────────────────┤
+│ │ Yes │ No │
+│ ├───────────────────────────┼───────────────────────────┤
+│ │ QND7A │ QND7A │
+│ ├─────────────┬─────────────┼─────────────┬─────────────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├─────────────┼─────────────┼─────────────┼─────────────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
+├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Row ID │ 1│ 1│ 2│ 2│ 3│ 3│ 4│ 4│ 5│ 5│ 6│ 6│ 7│ 7│ 8│ 8│
+│ Layer Row ID│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 9│ 9│ 10│10│ 11│11│ 12│12│ 13│13│ 14│14│ 15│15│ 16│16│
+│ Layer Row ID│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Row ID │ 17│17│ 18│18│ 19│19│ 20│20│ 21│21│ 22│22│ 23│23│ 24│24│
+│ Layer Row ID│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 25│25│ 26│26│ 27│27│ 28│28│ 29│29│ 30│30│ 31│31│ 32│32│
+│ Layer Row ID│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
+╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+╭───────────────────────────────────┬───────────────────────────────────────────────────────╮
+│ │ QN27 │
+│ ├───────────────────────────┬───────────────────────────┤
+│ │ Yes │ No │
+│ ├───────────────────────────┼───────────────────────────┤
+│ │ QND7A │ QND7A │
+│ ├─────────────┬─────────────┼─────────────┬─────────────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├─────────────┼─────────────┼─────────────┼─────────────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
+├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Row ID │ 33│33│ 34│34│ 35│35│ 36│36│ 37│37│ 38│38│ 39│39│ 40│40│
+│ Layer Row ID│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 41│41│ 42│42│ 43│43│ 44│44│ 45│45│ 46│46│ 47│47│ 48│48│
+│ Layer Row ID│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│
+│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Row ID │ 49│49│ 50│50│ 51│51│ 52│52│ 53│53│ 54│54│ 55│55│ 56│56│
+│ Layer Row ID│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 57│57│ 58│58│ 59│59│ 60│60│ 61│61│ 62│62│ 63│63│ 64│64│
+│ Layer Row ID│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│
+╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+╭──────────────────────────────────────┬───────────────────────────────────────────────────────╮
+│ │ QN27 │
+│ ├───────────────────────────┬───────────────────────────┤
+│ │ Yes │ No │
+│ ├───────────────────────────┼───────────────────────────┤
+│ │ QND7A │ QND7A │
+│ ├─────────────┬─────────────┼─────────────┬─────────────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├─────────────┼─────────────┼─────────────┼─────────────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Column ID │ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
+│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Column ID │ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
+│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
+╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+╭──────────────────────────────────────┬───────────────────────────────────────────────────────╮
+│ │ QN27 │
+│ ├───────────────────────────┬───────────────────────────┤
+│ │ Yes │ No │
+│ ├───────────────────────────┼───────────────────────────┤
+│ │ QND7A │ QND7A │
+│ ├─────────────┬─────────────┼─────────────┬─────────────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├─────────────┼─────────────┼─────────────┼─────────────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Column ID │ 33│34│ 35│36│ 37│38│ 39│40│ 41│42│ 43│44│ 45│46│ 47│48│
+│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 33│34│ 35│36│ 37│38│ 39│40│ 41│42│ 43│44│ 45│46│ 47│48│
+│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
+│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Column ID │ 49│50│ 51│52│ 53│54│ 55│56│ 57│58│ 59│60│ 61│62│ 63│64│
+│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 49│50│ 51│52│ 53│54│ 55│56│ 57│58│ 59│60│ 61│62│ 63│64│
+│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
+╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES area definitions with CLABELS COLLABELS=OPPOSITE])
+AT_KEYWORDS([COLLABELS OPPOSITE])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=ALL DISPLAY=NAME
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
+ /SLABELS POSITION=ROW
+ /CLABELS COLLABELS=OPPOSITE
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
+ /SLABELS POSITION=ROW
+ /CLABELS COLLABELS=OPPOSITE
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
+ /SLABELS POSITION=ROW
+ /CLABELS COLLABELS=OPPOSITE.
+]])
+AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+Male
+╭──────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 2│ 2│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 2│ 2│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 3│ 3│ 4│ 4│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 3│ 3│ 4│ 4│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 5│ 5│ 6│ 6│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 5│ 5│ 6│ 6│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 7│ 7│ 8│ 8│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 7│ 7│ 8│ 8│
+│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 10│ 10│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 10│ 10│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 11│ 11│ 12│ 12│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 11│ 11│ 12│ 12│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 13│ 13│ 14│ 14│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 13│ 13│ 14│ 14│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 15│ 15│ 16│ 16│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 15│ 15│ 16│ 16│
+╰──────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+╭──────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 17│ 17│ 18│ 18│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 17│ 17│ 18│ 18│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 19│ 19│ 20│ 20│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 19│ 19│ 20│ 20│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 21│ 21│ 22│ 22│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 21│ 21│ 22│ 22│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 23│ 23│ 24│ 24│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 23│ 23│ 24│ 24│
+│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 25│ 25│ 26│ 26│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 25│ 25│ 26│ 26│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 27│ 27│ 28│ 28│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 27│ 27│ 28│ 28│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 29│ 29│ 30│ 30│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 29│ 29│ 30│ 30│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 31│ 31│ 32│ 32│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 31│ 31│ 32│ 32│
+╰──────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+╭───────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├───────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Yes Row ID │ 1│ 1│ 3│ 3│
+│ Layer Row ID│ 1│ 1│ 1│ 1│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 2│ 2│ 4│ 4│
+│ Layer Row ID│ 2│ 2│ 2│ 2│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 5│ 5│ 7│ 7│
+│ Layer Row ID│ 3│ 3│ 3│ 3│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 6│ 6│ 8│ 8│
+│ Layer Row ID│ 4│ 4│ 4│ 4│
+│ ╶─────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Row ID │ 9│ 9│ 11│ 11│
+│ Layer Row ID│ 5│ 5│ 5│ 5│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 10│ 10│ 12│ 12│
+│ Layer Row ID│ 6│ 6│ 6│ 6│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 13│ 13│ 15│ 15│
+│ Layer Row ID│ 7│ 7│ 7│ 7│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 14│ 14│ 16│ 16│
+│ Layer Row ID│ 8│ 8│ 8│ 8│
+│ ╶──────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Yes Row ID │ 17│ 17│ 19│ 19│
+│ Layer Row ID│ 9│ 9│ 9│ 9│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 18│ 18│ 20│ 20│
+│ Layer Row ID│ 10│ 10│ 10│ 10│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 21│ 21│ 23│ 23│
+│ Layer Row ID│ 11│ 11│ 11│ 11│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 22│ 22│ 24│ 24│
+│ Layer Row ID│ 12│ 12│ 12│ 12│
+│ ╶─────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Row ID │ 25│ 25│ 27│ 27│
+│ Layer Row ID│ 13│ 13│ 13│ 13│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 26│ 26│ 28│ 28│
+│ Layer Row ID│ 14│ 14│ 14│ 14│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 29│ 29│ 31│ 31│
+│ Layer Row ID│ 15│ 15│ 15│ 15│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 30│ 30│ 32│ 32│
+│ Layer Row ID│ 16│ 16│ 16│ 16│
+╰───────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+╭───────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├───────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Yes Row ID │ 33│ 33│ 35│ 35│
+│ Layer Row ID│ 17│ 17│ 17│ 17│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 34│ 34│ 36│ 36│
+│ Layer Row ID│ 18│ 18│ 18│ 18│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 37│ 37│ 39│ 39│
+│ Layer Row ID│ 19│ 19│ 19│ 19│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 38│ 38│ 40│ 40│
+│ Layer Row ID│ 20│ 20│ 20│ 20│
+│ ╶─────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Row ID │ 41│ 41│ 43│ 43│
+│ Layer Row ID│ 21│ 21│ 21│ 21│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 42│ 42│ 44│ 44│
+│ Layer Row ID│ 22│ 22│ 22│ 22│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 45│ 45│ 47│ 47│
+│ Layer Row ID│ 23│ 23│ 23│ 23│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 46│ 46│ 48│ 48│
+│ Layer Row ID│ 24│ 24│ 24│ 24│
+│ ╶──────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Yes Row ID │ 49│ 49│ 51│ 51│
+│ Layer Row ID│ 25│ 25│ 25│ 25│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 50│ 50│ 52│ 52│
+│ Layer Row ID│ 26│ 26│ 26│ 26│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 53│ 53│ 55│ 55│
+│ Layer Row ID│ 27│ 27│ 27│ 27│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 54│ 54│ 56│ 56│
+│ Layer Row ID│ 28│ 28│ 28│ 28│
+│ ╶─────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Row ID │ 57│ 57│ 59│ 59│
+│ Layer Row ID│ 29│ 29│ 29│ 29│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 58│ 58│ 60│ 60│
+│ Layer Row ID│ 30│ 30│ 30│ 30│
+│ ╶────────────────────┼────┼────┼────┼────┤
+│ No Yes Row ID │ 61│ 61│ 63│ 63│
+│ Layer Row ID│ 31│ 31│ 31│ 31│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 62│ 62│ 64│ 64│
+│ Layer Row ID│ 32│ 32│ 32│ 32│
+╰───────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+╭──────────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Yes Column ID │ 1│ 2│ 3│ 4│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 1│ 2│ 3│ 4│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 5│ 6│ 7│ 8│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 5│ 6│ 7│ 8│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶────────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Column ID │ 9│ 10│ 11│ 12│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 9│ 10│ 11│ 12│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 13│ 14│ 15│ 16│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 13│ 14│ 15│ 16│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶─────────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Yes Column ID │ 17│ 18│ 19│ 20│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 17│ 18│ 19│ 20│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 21│ 22│ 23│ 24│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 21│ 22│ 23│ 24│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶────────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Column ID │ 25│ 26│ 27│ 28│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 25│ 26│ 27│ 28│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 29│ 30│ 31│ 32│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 29│ 30│ 31│ 32│
+│ Layer Column ID│ 1│ 2│ 3│ 4│
+╰──────────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+╭──────────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Yes Column ID │ 33│ 34│ 35│ 36│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 33│ 34│ 35│ 36│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 37│ 38│ 39│ 40│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 37│ 38│ 39│ 40│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶────────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Column ID │ 41│ 42│ 43│ 44│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 41│ 42│ 43│ 44│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 45│ 46│ 47│ 48│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 45│ 46│ 47│ 48│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶─────────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Yes Column ID │ 49│ 50│ 51│ 52│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 49│ 50│ 51│ 52│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 53│ 54│ 55│ 56│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 53│ 54│ 55│ 56│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶────────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Yes Column ID │ 57│ 58│ 59│ 60│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 57│ 58│ 59│ 60│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────────┼────┼────┼────┼────┤
+│ No Yes Column ID │ 61│ 62│ 63│ 64│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 61│ 62│ 63│ 64│
+│ Layer Column ID│ 5│ 6│ 7│ 8│
+╰──────────────────────────────────────────────┴────┴────┴────┴────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES area definitions with CLABELS ROWLABELS=LAYER])
+AT_KEYWORDS([ROWLABELS LAYER])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=ALL DISPLAY=NAME
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=LAYER
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=LAYER
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=LAYER.
+]])
+AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+Male
+Yes
+╭──────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 3│ 3│ 5│ 5│ 7│ 7│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 3│ 3│ 5│ 5│ 7│ 7│
+│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 11│11│ 13│13│ 15│15│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 11│11│ 13│13│ 15│15│
+╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+No
+╭──────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 2│ 2│ 4│ 4│ 6│ 6│ 8│ 8│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 2│ 2│ 4│ 4│ 6│ 6│ 8│ 8│
+│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 10│10│ 12│12│ 14│14│ 16│16│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ Subtable ID│ 10│10│ 12│12│ 14│14│ 16│16│
+╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+Yes
+╭──────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
+│ Subtable ID│ 17│17│ 19│19│ 21│21│ 23│23│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
+│ Subtable ID│ 17│17│ 19│19│ 21│21│ 23│23│
+│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
+│ Subtable ID│ 25│25│ 27│27│ 29│29│ 31│31│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
+│ Subtable ID│ 25│25│ 27│27│ 29│29│ 31│31│
+╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+No
+╭──────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
+│ Subtable ID│ 18│18│ 20│20│ 22│22│ 24│24│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
+│ Subtable ID│ 18│18│ 20│20│ 22│22│ 24│24│
+│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
+│ Subtable ID│ 26│26│ 28│28│ 30│30│ 32│32│
+│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
+│ Subtable ID│ 26│26│ 28│28│ 30│30│ 32│32│
+╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+Yes
+╭───────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Row ID │ 1│ 1│ 3│ 3│ 5│ 5│ 7│ 7│
+│ Layer Row ID│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 9│ 9│ 11│11│ 13│13│ 15│15│
+│ Layer Row ID│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
+│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Row ID │ 17│17│ 19│19│ 21│21│ 23│23│
+│ Layer Row ID│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 25│25│ 27│27│ 29│29│ 31│31│
+│ Layer Row ID│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│
+╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+No
+╭───────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Row ID │ 2│ 2│ 4│ 4│ 6│ 6│ 8│ 8│
+│ Layer Row ID│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 10│10│ 12│12│ 14│14│ 16│16│
+│ Layer Row ID│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
+│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Row ID │ 18│18│ 20│20│ 22│22│ 24│24│
+│ Layer Row ID│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 26│26│ 28│28│ 30│30│ 32│32│
+│ Layer Row ID│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│
+╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+Yes
+╭───────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Row ID │ 33│33│ 35│35│ 37│37│ 39│39│
+│ Layer Row ID│ 9│ 9│ 9│ 9│ 9│ 9│ 9│ 9│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 41│41│ 43│43│ 45│45│ 47│47│
+│ Layer Row ID│ 11│11│ 11│11│ 11│11│ 11│11│
+│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Row ID │ 49│49│ 51│51│ 53│53│ 55│55│
+│ Layer Row ID│ 13│13│ 13│13│ 13│13│ 13│13│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 57│57│ 59│59│ 61│61│ 63│63│
+│ Layer Row ID│ 15│15│ 15│15│ 15│15│ 15│15│
+╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+No
+╭───────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Row ID │ 34│34│ 36│36│ 38│38│ 40│40│
+│ Layer Row ID│ 10│10│ 10│10│ 10│10│ 10│10│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 42│42│ 44│44│ 46│46│ 48│48│
+│ Layer Row ID│ 12│12│ 12│12│ 12│12│ 12│12│
+│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Row ID │ 50│50│ 52│52│ 54│54│ 56│56│
+│ Layer Row ID│ 14│14│ 14│14│ 14│14│ 14│14│
+│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Row ID │ 58│58│ 60│60│ 62│62│ 64│64│
+│ Layer Row ID│ 16│16│ 16│16│ 16│16│ 16│16│
+╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+Yes
+╭──────────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Column ID │ 1│ 3│ 5│ 7│ 9│11│ 13│15│
+│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 1│ 3│ 5│ 7│ 9│11│ 13│15│
+│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
+│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Column ID │ 17│19│ 21│23│ 25│27│ 29│31│
+│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 17│19│ 21│23│ 25│27│ 29│31│
+│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
+╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Male
+No
+╭──────────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Column ID │ 2│ 4│ 6│ 8│ 10│12│ 14│16│
+│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 2│ 4│ 6│ 8│ 10│12│ 14│16│
+│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
+│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Column ID │ 18│20│ 22│24│ 26│28│ 30│32│
+│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 18│20│ 22│24│ 26│28│ 30│32│
+│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
+╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+Yes
+╭──────────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Column ID │ 33│35│ 37│39│ 41│43│ 45│47│
+│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 33│35│ 37│39│ 41│43│ 45│47│
+│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
+│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Column ID │ 49│51│ 53│55│ 57│59│ 61│63│
+│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 49│51│ 53│55│ 57│59│ 61│63│
+│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
+╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+
+ Custom Tables
+Female
+No
+╭──────────────────────────────────────┬───────────────────────────╮
+│ │ QN27 │
+│ ├─────────────┬─────────────┤
+│ │ Yes │ No │
+│ ├─────────────┼─────────────┤
+│ │ QND7A │ QND7A │
+│ ├──────┬──────┼──────┬──────┤
+│ │ Yes │ No │ Yes │ No │
+│ ├──────┼──────┼──────┼──────┤
+│ │ QN86 │ QN86 │ QN86 │ QN86 │
+│ ├───┬──┼───┬──┼───┬──┼───┬──┤
+│ │Yes│No│Yes│No│Yes│No│Yes│No│
+├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│QN26 Yes QN61 Yes QN57 Column ID │ 34│36│ 38│40│ 42│44│ 46│48│
+│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 34│36│ 38│40│ 42│44│ 46│48│
+│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
+│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN61 Yes QN57 Column ID │ 50│52│ 54│56│ 58│60│ 62│64│
+│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
+│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
+│ No QN57 Column ID │ 50│52│ 54│56│ 58│60│ 62│64│
+│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
+╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES area definitions with CLABELS COLLABELS=LAYER])
+AT_KEYWORDS([COLLABELS LAYER])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /VLABELS VARIABLES=ALL DISPLAY=NAME
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
+ /SLABELS POSITION=ROW
+ /CLABELS COLLABELS=LAYER
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
+ /SLABELS POSITION=ROW
+ /CLABELS COLLABELS=LAYER
+ /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
+ /SLABELS POSITION=ROW
+ /CLABELS COLLABELS=LAYER.
+]])
+AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+Male
+Yes
+╭──────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 3│ 3│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 1│ 1│ 3│ 3│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 5│ 5│ 7│ 7│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 5│ 5│ 7│ 7│
+│ ╶─────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 11│ 11│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 9│ 9│ 11│ 11│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 13│ 13│ 15│ 15│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 1│ 1│ 1│ 1│
+│ Subtable ID│ 13│ 13│ 15│ 15│
+╰──────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+No
+╭──────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 2│ 2│ 4│ 4│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 2│ 2│ 4│ 4│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 6│ 6│ 8│ 8│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 6│ 6│ 8│ 8│
+│ ╶─────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 10│ 10│ 12│ 12│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 10│ 10│ 12│ 12│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 14│ 14│ 16│ 16│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 2│ 2│ 2│ 2│
+│ Subtable ID│ 14│ 14│ 16│ 16│
+╰──────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+Yes
+╭──────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 17│ 17│ 19│ 19│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 17│ 17│ 19│ 19│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 21│ 21│ 23│ 23│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 21│ 21│ 23│ 23│
+│ ╶─────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 25│ 25│ 27│ 27│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 25│ 25│ 27│ 27│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 29│ 29│ 31│ 31│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 3│ 3│ 3│ 3│
+│ Subtable ID│ 29│ 29│ 31│ 31│
+╰──────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+No
+╭──────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 18│ 18│ 20│ 20│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 18│ 18│ 20│ 20│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 22│ 22│ 24│ 24│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 22│ 22│ 24│ 24│
+│ ╶─────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 26│ 26│ 28│ 28│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 26│ 26│ 28│ 28│
+│ ╶────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 30│ 30│ 32│ 32│
+│ ╶───────────────┼────┼────┼────┼────┤
+│ No Table ID │ 1│ 1│ 1│ 1│
+│ Layer ID │ 4│ 4│ 4│ 4│
+│ Subtable ID│ 30│ 30│ 32│ 32│
+╰──────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+Yes
+╭───────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├───────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Row ID │ 1│ 1│ 3│ 3│
+│ Layer Row ID│ 1│ 1│ 1│ 1│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 5│ 5│ 7│ 7│
+│ Layer Row ID│ 3│ 3│ 3│ 3│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 9│ 9│ 11│ 11│
+│ Layer Row ID│ 5│ 5│ 5│ 5│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 13│ 13│ 15│ 15│
+│ Layer Row ID│ 7│ 7│ 7│ 7│
+│ ╶──────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Row ID │ 17│ 17│ 19│ 19│
+│ Layer Row ID│ 9│ 9│ 9│ 9│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 21│ 21│ 23│ 23│
+│ Layer Row ID│ 11│ 11│ 11│ 11│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 25│ 25│ 27│ 27│
+│ Layer Row ID│ 13│ 13│ 13│ 13│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 29│ 29│ 31│ 31│
+│ Layer Row ID│ 15│ 15│ 15│ 15│
+╰───────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+No
+╭───────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├───────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Row ID │ 2│ 2│ 4│ 4│
+│ Layer Row ID│ 2│ 2│ 2│ 2│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 6│ 6│ 8│ 8│
+│ Layer Row ID│ 4│ 4│ 4│ 4│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 10│ 10│ 12│ 12│
+│ Layer Row ID│ 6│ 6│ 6│ 6│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 14│ 14│ 16│ 16│
+│ Layer Row ID│ 8│ 8│ 8│ 8│
+│ ╶──────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Row ID │ 18│ 18│ 20│ 20│
+│ Layer Row ID│ 10│ 10│ 10│ 10│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 22│ 22│ 24│ 24│
+│ Layer Row ID│ 12│ 12│ 12│ 12│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 26│ 26│ 28│ 28│
+│ Layer Row ID│ 14│ 14│ 14│ 14│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 30│ 30│ 32│ 32│
+│ Layer Row ID│ 16│ 16│ 16│ 16│
+╰───────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+Yes
+╭───────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├───────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Row ID │ 33│ 33│ 35│ 35│
+│ Layer Row ID│ 17│ 17│ 17│ 17│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 37│ 37│ 39│ 39│
+│ Layer Row ID│ 19│ 19│ 19│ 19│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 41│ 41│ 43│ 43│
+│ Layer Row ID│ 21│ 21│ 21│ 21│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 45│ 45│ 47│ 47│
+│ Layer Row ID│ 23│ 23│ 23│ 23│
+│ ╶──────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Row ID │ 49│ 49│ 51│ 51│
+│ Layer Row ID│ 25│ 25│ 25│ 25│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 53│ 53│ 55│ 55│
+│ Layer Row ID│ 27│ 27│ 27│ 27│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 57│ 57│ 59│ 59│
+│ Layer Row ID│ 29│ 29│ 29│ 29│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 61│ 61│ 63│ 63│
+│ Layer Row ID│ 31│ 31│ 31│ 31│
+╰───────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+No
+╭───────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├───────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Row ID │ 34│ 34│ 36│ 36│
+│ Layer Row ID│ 18│ 18│ 18│ 18│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 38│ 38│ 40│ 40│
+│ Layer Row ID│ 20│ 20│ 20│ 20│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 42│ 42│ 44│ 44│
+│ Layer Row ID│ 22│ 22│ 22│ 22│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 46│ 46│ 48│ 48│
+│ Layer Row ID│ 24│ 24│ 24│ 24│
+│ ╶──────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Row ID │ 50│ 50│ 52│ 52│
+│ Layer Row ID│ 26│ 26│ 26│ 26│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 54│ 54│ 56│ 56│
+│ Layer Row ID│ 28│ 28│ 28│ 28│
+│ ╶─────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Row ID │ 58│ 58│ 60│ 60│
+│ Layer Row ID│ 30│ 30│ 30│ 30│
+│ ╶────────────────┼────┼────┼────┼────┤
+│ No Row ID │ 62│ 62│ 64│ 64│
+│ Layer Row ID│ 32│ 32│ 32│ 32│
+╰───────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+Yes
+╭──────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Column ID │ 1│ 3│ 5│ 7│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 1│ 3│ 5│ 7│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 9│ 11│ 13│ 15│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 9│ 11│ 13│ 15│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Column ID │ 17│ 19│ 21│ 23│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 17│ 19│ 21│ 23│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 25│ 27│ 29│ 31│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 25│ 27│ 29│ 31│
+│ Layer Column ID│ 1│ 3│ 5│ 7│
+╰──────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Male
+No
+╭──────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Column ID │ 2│ 4│ 6│ 8│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 2│ 4│ 6│ 8│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 10│ 12│ 14│ 16│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 10│ 12│ 14│ 16│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Column ID │ 18│ 20│ 22│ 24│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 18│ 20│ 22│ 24│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 26│ 28│ 30│ 32│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 26│ 28│ 30│ 32│
+│ Layer Column ID│ 2│ 4│ 6│ 8│
+╰──────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+Yes
+╭──────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Column ID │ 33│ 35│ 37│ 39│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 33│ 35│ 37│ 39│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 41│ 43│ 45│ 47│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 41│ 43│ 45│ 47│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Column ID │ 49│ 51│ 53│ 55│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 49│ 51│ 53│ 55│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 57│ 59│ 61│ 63│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 57│ 59│ 61│ 63│
+│ Layer Column ID│ 9│ 11│ 13│ 15│
+╰──────────────────────────────────────────┴────┴────┴────┴────╯
+
+ Custom Tables
+Female
+No
+╭──────────────────────────────────────────┬───────────────────╮
+│ │ QN27 │
+│ ├─────────┬─────────┤
+│ │ Yes │ No │
+│ ├─────────┼─────────┤
+│ │ QND7A │ QND7A │
+│ ├────┬────┼────┬────┤
+│ │ Yes│ No │ Yes│ No │
+│ ├────┼────┼────┼────┤
+│ │QN86│QN86│QN86│QN86│
+├──────────────────────────────────────────┼────┼────┼────┼────┤
+│QN26 Yes QN61 Yes QN57 Yes Column ID │ 34│ 36│ 38│ 40│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 34│ 36│ 38│ 40│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 42│ 44│ 46│ 48│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 42│ 44│ 46│ 48│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
+│ No QN61 Yes QN57 Yes Column ID │ 50│ 52│ 54│ 56│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 50│ 52│ 54│ 56│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶────────────────────────────┼────┼────┼────┼────┤
+│ No QN57 Yes Column ID │ 58│ 60│ 62│ 64│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+│ ╶───────────────────┼────┼────┼────┼────┤
+│ No Column ID │ 58│ 60│ 62│ 64│
+│ Layer Column ID│ 10│ 12│ 14│ 16│
+╰──────────────────────────────────────────┴────┴────┴────┴────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES categorical summary functions])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /TABLE region BY qnd5a[COUNT, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, TOTAL[COUNT, VALIDN, TOTALN]]
+ /CATEGORIES VARIABLES=qnd5a TOTAL=YES MISSING=INCLUDE
+ /SLABELS POSITION=ROW.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Custom Tables
+╭─────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────╮
+│ │ D5a. What would you say is your primary ethnic background? │
+│ ├─────┬───────┬───────┬──────────┬──────────┬─────────┬──────────┬──────────────┬──────┬───────┬─────┤
+│ │ │ │ │ │ │ │ │ Multiple - │ │ │ │
+│ │ │ │ │ South │ Central │ Puerto │ Something│ cannot choose│ Don't│ │ │
+│ │Cuban│Mexican│Spanish│ American │ American │Rican, OR│ else │ one │ know │Refused│Total│
+├─────────────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
+│Region NE Count │ 6│ 8│ 8│ 11│ 7│ 39│ 23│ 2│ 0│ 1│ 105│
+│ Row % │ 5.7%│ 7.6%│ 7.6%│ 10.5%│ 6.7%│ 37.1%│ 21.9%│ 1.9%│ .0%│ 1.0%│ │
+│ Row │ 5.8%│ 7.7%│ 7.7%│ 10.6%│ 6.7%│ 37.5%│ 22.1%│ 1.9%│ .0%│ .0%│ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Row │ .4%│ .6%│ .6%│ .8%│ .5%│ 2.8%│ 1.6%│ .1%│ .0%│ .1%│ │
+│ Total N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ 104│
+│ Total N│ │ │ │ │ │ │ │ │ │ │ 1409│
+│ ╶──────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
+│ MW Count │ 3│ 24│ 1│ 4│ 5│ 9│ 6│ 0│ 0│ 1│ 53│
+│ Row % │ 5.7%│ 45.3%│ 1.9%│ 7.5%│ 9.4%│ 17.0%│ 11.3%│ .0%│ .0%│ 1.9%│ │
+│ Row │ 5.8%│ 46.2%│ 1.9%│ 7.7%│ 9.6%│ 17.3%│ 11.5%│ .0%│ .0%│ .0%│ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Row │ .2%│ 1.5%│ .1%│ .2%│ .3%│ .5%│ .4%│ .0%│ .0%│ .1%│ │
+│ Total N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ 52│
+│ Total N│ │ │ │ │ │ │ │ │ │ │ 1654│
+│ ╶──────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
+│ S Count │ 10│ 113│ 11│ 14│ 25│ 23│ 20│ 2│ 3│ 2│ 223│
+│ Row % │ 4.5%│ 50.7%│ 4.9%│ 6.3%│ 11.2%│ 10.3%│ 9.0%│ .9%│ 1.3%│ .9%│ │
+│ Row │ 4.6%│ 51.8%│ 5.0%│ 6.4%│ 11.5%│ 10.6%│ 9.2%│ .9%│ .0%│ .0%│ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Row │ .4%│ 4.7%│ .5%│ .6%│ 1.0%│ 1.0%│ .8%│ .1%│ .1%│ .1%│ │
+│ Total N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ 218│
+│ Total N│ │ │ │ │ │ │ │ │ │ │ 2390│
+│ ╶──────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
+│ W Count │ 1│ 166│ 28│ 5│ 15│ 7│ 19│ 3│ 0│ 1│ 245│
+│ Row % │ .4%│ 67.8%│ 11.4%│ 2.0%│ 6.1%│ 2.9%│ 7.8%│ 1.2%│ .0%│ .4%│ │
+│ Row │ .4%│ 68.0%│ 11.5%│ 2.0%│ 6.1%│ 2.9%│ 7.8%│ 1.2%│ .0%│ .0%│ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Row │ .1%│ 10.7%│ 1.8%│ .3%│ 1.0%│ .5%│ 1.2%│ .2%│ .0%│ .1%│ │
+│ Total N│ │ │ │ │ │ │ │ │ │ │ │
+│ % │ │ │ │ │ │ │ │ │ │ │ │
+│ Valid N│ │ │ │ │ │ │ │ │ │ │ 244│
+│ Total N│ │ │ │ │ │ │ │ │ │ │ 1546│
+╰─────────────────┴─────┴───────┴───────┴──────────┴──────────┴─────────┴──────────┴──────────────┴──────┴───────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES scale summary functions])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+SET TVAR=NAME.
+
+* Use SPLIT FILE with FREQUENCIES to generate output equivalent to
+ CTABLES later, to make the results easier to verify.
+SPLIT FILE BY REGION.
+FREQUENCIES
+ qn19a
+ /STATISTICS=MEAN SEMEAN MEDIAN MODE STDDEV VARIANCE RANGE MINIMUM MAXIMUM SUM
+ /FORMAT NOTABLE /MISSING=INCLUDE.
+SPLIT FILE OFF.
+
+CTABLES
+ /VLABELS VARIABLE=qn19a DISPLAY=NONE
+ /TABLE region BY qn19a[VALIDN, MISSING, MEAN, SEMEAN, MEDIAN, MODE, STDDEV, VARIANCE, RANGE, MINIMUM, MAXIMUM, SUM, COUNT, TOTALN, ROWPCT.SUM]
+ /CATEGORIES VARIABLES=qn19a TOTAL=YES MISSING=INCLUDE
+ /SLABELS POSITION=ROW
+ /CLABELS ROWLABELS=OPPOSITE.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Statistics
+╭─────────┬───────────────────────────────────╮
+│ │ REGION │
+│ ├────────┬────────┬────────┬────────┤
+│ │ NE │ MW │ S │ W │
+│ ├────────┼────────┼────────┼────────┤
+│ │ QN19A │ QN19A │ QN19A │ QN19A │
+├─────────┼────────┼────────┼────────┼────────┤
+│N Valid │ 936│ 1019│ 1276│ 950│
+│ Missing│ 473│ 635│ 1114│ 596│
+├─────────┼────────┼────────┼────────┼────────┤
+│Mean │ 19.33│ 19.83│ 20.29│ 19.87│
+├─────────┼────────┼────────┼────────┼────────┤
+│S.E. Mean│ .14│ .16│ .18│ .17│
+├─────────┼────────┼────────┼────────┼────────┤
+│Median │ 18.00│ 19.00│ 19.00│ 19.00│
+├─────────┼────────┼────────┼────────┼────────┤
+│Mode │ 18.00│ 18.00│ 18.00│ 18.00│
+├─────────┼────────┼────────┼────────┼────────┤
+│Std Dev │ 4.41│ 5.15│ 6.44│ 5.25│
+├─────────┼────────┼────────┼────────┼────────┤
+│Variance │ 19.41│ 26.47│ 41.43│ 27.59│
+├─────────┼────────┼────────┼────────┼────────┤
+│Range │ 59.00│ 71.00│ 75.00│ 61.00│
+├─────────┼────────┼────────┼────────┼────────┤
+│Minimum │ .00│ 4.00│ 4.00│ 4.00│
+├─────────┼────────┼────────┼────────┼────────┤
+│Maximum │ 59.00│ 75.00│ 79.00│ 65.00│
+├─────────┼────────┼────────┼────────┼────────┤
+│Sum │18092.00│20206.00│25886.00│18877.00│
+╰─────────┴────────┴────────┴────────┴────────╯
+
+ Custom Tables
+╭────────────────────────┬────────┬────────┬────────┬────────╮
+│ │ NE │ MW │ S │ W │
+├────────────────────────┼────────┼────────┼────────┼────────┤
+│REGION Valid N │ 936│ 1019│ 1276│ 950│
+│ Missing │ 473.00│ 635.00│ 1114.00│ 596.00│
+│ Mean │ 19.33│ 19.83│ 20.29│ 19.87│
+│ Std Error of Mean│ .14│ .16│ .18│ .17│
+│ Median │ 18.00│ 19.00│ 19.00│ 19.00│
+│ Mode │ 18.00│ 18.00│ 18.00│ 18.00│
+│ Std Deviation │ 4.41│ 5.15│ 6.44│ 5.25│
+│ Variance │ 19.41│ 26.47│ 41.43│ 27.59│
+│ Range │ 59.00│ 71.00│ 75.00│ 61.00│
+│ Minimum │ .00│ 4.00│ 4.00│ 4.00│
+│ Maximum │ 59.00│ 75.00│ 79.00│ 65.00│
+│ Sum │18092.00│20206.00│25886.00│18877.00│
+│ Count │ 1409│ 1654│ 2390│ 1546│
+│ Total N │ 1409│ 1654│ 2390│ 1546│
+│ Row Sum % │ 21.8%│ 24.3%│ 31.2%│ 22.7%│
+╰────────────────────────┴────────┴────────┴────────┴────────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES scale summary functions - weighting])
+weight=1
+c=10
+for a in 1 2 9; do
+ for b in 3 4 9; do
+ for n in 1 2 3 4 5 6 7 8 9 10; do
+ if test $c -lt 15; then
+ cval=.
+ else
+ cval=$c
+ fi
+ printf "$weight $a $b $cval\n"
+ weight=$(expr \( $weight + 3 \) % 7 + 2)
+ c=$(expr \( $c + 13 \) % 29 + 7)
+ done
+ done
+done > ctables.txt
+
+AT_DATA([analysis.sps],
+[[* Use SPLIT FILE with FREQUENCIES to generate output equivalent to
+ CTABLES later, to make the results easier to verify.
+SPLIT FILE BY a b.
+FREQUENCIES
+ c
+ /STATISTICS=MEAN SEMEAN MEDIAN MODE STDDEV VARIANCE RANGE MINIMUM MAXIMUM SUM
+ /FORMAT NOTABLE /MISSING=INCLUDE.
+SPLIT FILE OFF.
+
+CTABLES
+ /TABLE c[VALIDN, MISSING, MEAN F8.2, SEMEAN F8.2, MEDIAN F8.2, MODE, STDDEV F8.2, VARIANCE F8.2, RANGE F8.2, MINIMUM, MAXIMUM, SUM F8.2, COUNT, TOTALN, LAYERROWPCT.SUM] BY a>b
+ /SLABELS POSITION=ROW
+ /CATEGORIES VARIABLES=a b MISSING=INCLUDE.
+]])
+
+AT_DATA([ctables.sps],
+[[DATA LIST LIST NOTABLE FILE='ctables.txt'
+ /w (F5.0) a b c (f2.0).
+VAR LEVEL w c (SCALE) a b (NOMINAL).
+MISSING VALUES a b (9).
+
+INCLUDE 'analysis.sps'.
+
+WEIGHT BY w.
+INCLUDE 'analysis.sps'.
+
+* Same as original analysis using unweighted versions of summaries.
+CTABLES
+ /TABLE c[UVALIDN, UMISSING, UMEAN F8.2, USEMEAN F8.2, UMEDIAN F8.2, UMODE, USTDDEV F8.2, UVARIANCE F8.2, USUM F8.2, UCOUNT, UTOTALN, ULAYERROWPCT.SUM] BY a>b
+ /SLABELS POSITION=ROW
+ /CATEGORIES VARIABLES=a b MISSING=INCLUDE.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
+ Statistics
+╭─────────┬──────────────────────────────────────────────────────────────╮
+│ │ a │
+│ ├────────────────────┬────────────────────┬────────────────────┤
+│ │ 1 │ 2 │ 9 │
+│ ├────────────────────┼────────────────────┼────────────────────┤
+│ │ b │ b │ b │
+│ ├──────┬──────┬──────┼──────┬──────┬──────┼──────┬──────┬──────┤
+│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
+│ ├──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│ │ c │ c │ c │ c │ c │ c │ c │ c │ c │
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│N Valid │ 7│ 6│ 8│ 7│ 7│ 8│ 7│ 7│ 8│
+│ Missing│ 3│ 4│ 2│ 3│ 3│ 2│ 3│ 3│ 2│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Mean │ 25.86│ 24.50│ 24.63│ 25.86│ 25.71│ 24.25│ 25.43│ 25.29│ 23.88│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│S.E. Mean│ 2.44│ 2.14│ 2.58│ 2.44│ 2.18│ 2.43│ 2.36│ 2.18│ 2.47│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Median │ 25.00│ 24.50│ 25.00│ 25.00│ 27.00│ 25.00│ 25.00│ 24.00│ 23.50│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Mode │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Std Dev │ 6.47│ 5.24│ 7.31│ 6.47│ 5.77│ 6.88│ 6.24│ 5.77│ 6.98│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Variance │ 41.81│ 27.50│ 53.41│ 41.81│ 33.24│ 47.36│ 38.95│ 33.24│ 48.70│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
+├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│Sum │181.00│147.00│197.00│181.00│180.00│194.00│178.00│177.00│191.00│
+╰─────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
+
+ Custom Tables
+╭───────────────────┬──────────────────────────────────────────────────────────────╮
+│ │ a │
+│ ├────────────────────┬────────────────────┬────────────────────┤
+│ │ 1 │ 2 │ 9 │
+│ ├────────────────────┼────────────────────┼────────────────────┤
+│ │ b │ b │ b │
+│ ├──────┬──────┬──────┼──────┬──────┬──────┼──────┬──────┬──────┤
+│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
+├───────────────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│c Valid N │ 7│ 6│ 8│ 7│ 7│ 8│ 7│ 7│ 8│
+│ Missing │ 3│ 4│ 2│ 3│ 3│ 2│ 3│ 3│ 2│
+│ Mean │ 25.86│ 24.50│ 24.63│ 25.86│ 25.71│ 24.25│ 25.43│ 25.29│ 23.88│
+│ Std Error of Mean│ 2.44│ 2.14│ 2.58│ 2.44│ 2.18│ 2.43│ 2.36│ 2.18│ 2.47│
+│ Median │ 25.00│ 24.50│ 25.00│ 25.00│ 27.00│ 25.00│ 25.00│ 24.00│ 23.50│
+│ Mode │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+│ Std Deviation │ 6.47│ 5.24│ 7.31│ 6.47│ 5.77│ 6.88│ 6.24│ 5.77│ 6.98│
+│ Variance │ 41.81│ 27.50│ 53.41│ 41.81│ 33.24│ 47.36│ 38.95│ 33.24│ 48.70│
+│ Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
+│ Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+│ Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
+│ Sum │181.00│147.00│197.00│181.00│180.00│194.00│178.00│177.00│191.00│
+│ Count │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
+│ Total N │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
+│ Layer Row Sum % │ 11.1%│ 9.0%│ 12.1%│ 11.1%│ 11.1%│ 11.9%│ 10.9%│ 10.9%│ 11.7%│
+╰───────────────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
+
+ Statistics
+╭─────────┬─────────────────────────────────────────────────────────────────╮
+│ │ a │
+│ ├─────────────────────┬─────────────────────┬─────────────────────┤
+│ │ 1 │ 2 │ 9 │
+│ ├─────────────────────┼─────────────────────┼─────────────────────┤
+│ │ b │ b │ b │
+│ ├───────┬──────┬──────┼──────┬───────┬──────┼──────┬──────┬───────┤
+│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
+│ ├───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│ │ c │ c │ c │ c │ c │ c │ c │ c │ c │
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│N Valid │ 40│ 35│ 41│ 26│ 38│ 40│ 34│ 32│ 39│
+│ Missing│ 6│ 14│ 11│ 22│ 13│ 7│ 16│ 21│ 10│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Mean │ 27.23│ 24.20│ 22.63│ 27.96│ 27.21│ 23.48│ 23.71│ 25.47│ 26.03│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│S.E. Mean│ .93│ .75│ 1.03│ 1.12│ .84│ .87│ 1.01│ 1.05│ 1.01│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Median │ 30.00│ 22.00│ 19.00│ 30.00│ 29.00│ 24.00│ 23.00│ 24.00│ 28.00│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Mode │ 34│ 29│ 19│ 34│ 33│ 28│ 23│ 18│ 30│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Std Dev │ 5.89│ 4.42│ 6.59│ 5.69│ 5.16│ 5.50│ 5.87│ 5.94│ 6.30│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Variance │ 34.64│ 19.52│ 43.39│ 32.36│ 26.66│ 30.20│ 34.46│ 35.29│ 39.71│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
+├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│Sum │1089.00│847.00│928.00│727.00│1034.00│939.00│806.00│815.00│1015.00│
+╰─────────┴───────┴──────┴──────┴──────┴───────┴──────┴──────┴──────┴───────╯
+
+ Custom Tables
+╭───────────────────┬─────────────────────────────────────────────────────────────────╮
+│ │ a │
+│ ├─────────────────────┬─────────────────────┬─────────────────────┤
+│ │ 1 │ 2 │ 9 │
+│ ├─────────────────────┼─────────────────────┼─────────────────────┤
+│ │ b │ b │ b │
+│ ├───────┬──────┬──────┼──────┬───────┬──────┼──────┬──────┬───────┤
+│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
+├───────────────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
+│c Valid N │ 40│ 35│ 41│ 26│ 38│ 40│ 34│ 32│ 39│
+│ Missing │ 6│ 14│ 11│ 22│ 13│ 7│ 16│ 21│ 10│
+│ Mean │ 27.22│ 24.20│ 22.63│ 27.96│ 27.21│ 23.48│ 23.71│ 25.47│ 26.03│
+│ Std Error of Mean│ .93│ .75│ 1.03│ 1.12│ .84│ .87│ 1.01│ 1.05│ 1.01│
+│ Median │ 30.00│ 22.00│ 19.00│ 30.00│ 29.00│ 24.00│ 23.00│ 24.00│ 28.00│
+│ Mode │ 34│ 29│ 19│ 34│ 33│ 28│ 23│ 18│ 30│
+│ Std Deviation │ 5.89│ 4.42│ 6.59│ 5.69│ 5.16│ 5.50│ 5.87│ 5.94│ 6.30│
+│ Variance │ 34.64│ 19.52│ 43.39│ 32.36│ 26.66│ 30.20│ 34.46│ 35.29│ 39.71│
+│ Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
+│ Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+│ Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
+│ Sum │1089.00│847.00│928.00│727.00│1034.00│939.00│806.00│815.00│1015.00│
+│ Count │ 46│ 49│ 52│ 48│ 51│ 47│ 50│ 53│ 49│
+│ Total N │ 46│ 49│ 52│ 48│ 51│ 47│ 50│ 53│ 49│
+│ Layer Row Sum % │ 13.3%│ 10.3%│ 11.3%│ 8.9%│ 12.6%│ 11.5%│ 9.8%│ 9.9%│ 12.4%│
+╰───────────────────┴───────┴──────┴──────┴──────┴───────┴──────┴──────┴──────┴───────╯
+
+ Custom Tables
+╭──────────────────────────────┬──────────────────────────────────────────────────────────────╮
+│ │ a │
+│ ├────────────────────┬────────────────────┬────────────────────┤
+│ │ 1 │ 2 │ 9 │
+│ ├────────────────────┼────────────────────┼────────────────────┤
+│ │ b │ b │ b │
+│ ├──────┬──────┬──────┼──────┬──────┬──────┼──────┬──────┬──────┤
+│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
+├──────────────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
+│c Unweighted Valid N │ 7│ 6│ 8│ 7│ 7│ 8│ 7│ 7│ 8│
+│ Unweighted Missing │ 3│ 4│ 2│ 3│ 3│ 2│ 3│ 3│ 2│
+│ Unweighted Mean │ 25.86│ 24.50│ 24.63│ 25.86│ 25.71│ 24.25│ 25.43│ 25.29│ 23.88│
+│ Unweighted Std Error of Mean│ 2.44│ 2.14│ 2.58│ 2.44│ 2.18│ 2.43│ 2.36│ 2.18│ 2.47│
+│ Unweighted Median │ 25.00│ 24.50│ 25.00│ 25.00│ 27.00│ 25.00│ 25.00│ 24.00│ 23.50│
+│ Unweighted Mode │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
+│ Unweighted Std Deviation │ 6.47│ 5.24│ 7.31│ 6.47│ 5.77│ 6.88│ 6.24│ 5.77│ 6.98│
+│ Unweighted Variance │ 41.81│ 27.50│ 53.41│ 41.81│ 33.24│ 47.36│ 38.95│ 33.24│ 48.70│
+│ Unweighted Sum │181.00│147.00│197.00│181.00│180.00│194.00│178.00│177.00│191.00│
+│ Unweighted Count │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
+│ Unweighted Total N │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
+│ Unweighted Layer Row Sum % │ 11.1%│ 9.0%│ 12.1%│ 11.1%│ 11.1%│ 11.9%│ 10.9%│ 10.9%│ 11.7%│
+╰──────────────────────────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES hidden scale VLABELS])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+CTABLES
+ /TABLE region BY qn19a + qn35
+ /SLABELS POSITION=ROW.
+CTABLES
+ /VLABELS VARIABLE=qn19a DISPLAY=NONE
+ /TABLE region BY qn19a + qn35
+ /SLABELS POSITION=ROW.
+CTABLES
+ /VLABELS VARIABLE=qn35 DISPLAY=NONE
+ /TABLE region BY qn19a + qn35
+ /SLABELS POSITION=ROW.
+
+* This one in particular caused a crash because no categories were
+ created on the column axis, so passing in 0 for the index was still
+ too big for that number of categories. It was fixed by creating a
+ name-only category for each variable despite the "NONE" request,
+ then hiding the entire dimension's labels if all its labels were
+ set to "NONE".
+CTABLES
+ /VLABELS VARIABLE=qn19a qn35 DISPLAY=NONE
+ /TABLE region BY qn19a + qn35
+ /SLABELS POSITION=ROW.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
+ Custom Tables
+╭──────────────┬────────────────────────────┬─────────────────────────────────╮
+│ │ 19a. About how old were you│ 35. In the past thirty days, how│
+│ │ when you first starting │ many times have you driven a │
+│ │ drinking alcohol, not │ motor vehicle WITHIN TWO HOURS │
+│ │ counting small tastes or │ AFTER drinking alcoholic │
+│ │ sips of alcohol. │ beverages? │
+├──────────────┼────────────────────────────┼─────────────────────────────────┤
+│Region NE Mean│ 19.33│ 2│
+│ ╶───────┼────────────────────────────┼─────────────────────────────────┤
+│ MW Mean│ 19.83│ 2│
+│ ╶───────┼────────────────────────────┼─────────────────────────────────┤
+│ S Mean│ 20.29│ 2│
+│ ╶───────┼────────────────────────────┼─────────────────────────────────┤
+│ W Mean│ 19.87│ 2│
+╰──────────────┴────────────────────────────┴─────────────────────────────────╯
+
+ Custom Tables
+╭──────────────┬─────┬────────────────────────────────────────────────────────╮
+│ │ │ 35. In the past thirty days, how many times have you │
+│ │ │ driven a motor vehicle WITHIN TWO HOURS AFTER drinking │
+│ │QN19A│ alcoholic beverages? │
+├──────────────┼─────┼────────────────────────────────────────────────────────┤
+│Region NE Mean│19.33│ 2│
+│ ╶───────┼─────┼────────────────────────────────────────────────────────┤
+│ MW Mean│19.83│ 2│
+│ ╶───────┼─────┼────────────────────────────────────────────────────────┤
+│ S Mean│20.29│ 2│
+│ ╶───────┼─────┼────────────────────────────────────────────────────────┤
+│ W Mean│19.87│ 2│
+╰──────────────┴─────┴────────────────────────────────────────────────────────╯
+
+ Custom Tables
+╭──────────────┬─────────────────────────────────────────────────────────┬────╮
+│ │ 19a. About how old were you when you first starting │ │
+│ │ drinking alcohol, not counting small tastes or sips of │ │
+│ │ alcohol. │qn35│
+├──────────────┼─────────────────────────────────────────────────────────┼────┤
+│Region NE Mean│ 19.33│ 2│
+│ ╶───────┼─────────────────────────────────────────────────────────┼────┤
+│ MW Mean│ 19.83│ 2│
+│ ╶───────┼─────────────────────────────────────────────────────────┼────┤
+│ S Mean│ 20.29│ 2│
+│ ╶───────┼─────────────────────────────────────────────────────────┼────┤
+│ W Mean│ 19.87│ 2│
+╰──────────────┴─────────────────────────────────────────────────────────┴────╯
+
+ Custom Tables
+╭──────────────┬───────╮
+│Region NE Mean│19.33 2│
+│ ╶───────┼───────┤
+│ MW Mean│19.83 2│
+│ ╶───────┼───────┤
+│ S Mean│20.29 2│
+│ ╶───────┼───────┤
+│ W Mean│19.87 2│
+╰──────────────┴───────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES with SPLIT FILE])
+AT_CHECK([ln $top_srcdir/tests/language/commands/nhtsa.sav . || cp $top_srcdir/tests/language/commands/nhtsa.sav .])
+AT_DATA([ctables.sps],
+[[GET 'nhtsa.sav'.
+
+SORT CASES BY qns3a.
+
+CTABLES /TABLE qn105ba.
+
+* Layered split has no effect on output.
+SPLIT FILE BY qns3a.
+CTABLES /TABLE qn105ba.
+
+* Add column variable qns3a to compare against separate splits.
+CTABLES /TABLE qn105ba BY qns3a.
+
+* Separate splits are truly output separately.
+SPLIT FILE SEPARATE BY qns3a.
+CTABLES /TABLE qn105ba.
+]])
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│105b. How likely is it that drivers who have had too much Almost │ 700│
+│to drink to drive safely will A. Get stopped by the police? certain │ │
+│ Very likely │ 1502│
+│ Somewhat │ 2763│
+│ likely │ │
+│ Somewhat │ 1307│
+│ unlikely │ │
+│ Very │ 609│
+│ unlikely │ │
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│105b. How likely is it that drivers who have had too much Almost │ 700│
+│to drink to drive safely will A. Get stopped by the police? certain │ │
+│ Very likely │ 1502│
+│ Somewhat │ 2763│
+│ likely │ │
+│ Somewhat │ 1307│
+│ unlikely │ │
+│ Very │ 609│
+│ unlikely │ │
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Custom Tables
+╭─────────────────────────────────────────────────────────────────┬────────────╮
+│ │S3a. GENDER:│
+│ ├─────┬──────┤
+│ │ Male│Female│
+│ ├─────┼──────┤
+│ │Count│ Count│
+├─────────────────────────────────────────────────────────────────┼─────┼──────┤
+│105b. How likely is it that drivers who have had too Almost │ 297│ 403│
+│much to drink to drive safely will A. Get stopped by certain │ │ │
+│the police? Very likely │ 660│ 842│
+│ Somewhat │ 1174│ 1589│
+│ likely │ │ │
+│ Somewhat │ 640│ 667│
+│ unlikely │ │ │
+│ Very │ 311│ 298│
+│ unlikely │ │ │
+╰─────────────────────────────────────────────────────────────────┴─────┴──────╯
+
+ Split Values
+╭────────────┬─────╮
+│Variable │Value│
+├────────────┼─────┤
+│S3a. GENDER:│Male │
+╰────────────┴─────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│105b. How likely is it that drivers who have had too much Almost │ 297│
+│to drink to drive safely will A. Get stopped by the police? certain │ │
+│ Very likely │ 660│
+│ Somewhat │ 1174│
+│ likely │ │
+│ Somewhat │ 640│
+│ unlikely │ │
+│ Very │ 311│
+│ unlikely │ │
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+
+ Split Values
+╭────────────┬──────╮
+│Variable │ Value│
+├────────────┼──────┤
+│S3a. GENDER:│Female│
+╰────────────┴──────╯
+
+ Custom Tables
+╭────────────────────────────────────────────────────────────────────────┬─────╮
+│ │Count│
+├────────────────────────────────────────────────────────────────────────┼─────┤
+│105b. How likely is it that drivers who have had too much Almost │ 403│
+│to drink to drive safely will A. Get stopped by the police? certain │ │
+│ Very likely │ 842│
+│ Somewhat │ 1589│
+│ likely │ │
+│ Somewhat │ 667│
+│ unlikely │ │
+│ Very │ 298│
+│ unlikely │ │
+╰────────────────────────────────────────────────────────────────────────┴─────╯
+])
+AT_CLEANUP
+
+AT_SETUP([CTABLES variable level inference])
+AT_DATA([data.txt], [dnl
+dnl n1 has 10 unique small values -> nominal.
+dnl n2 has 23 unique small values -> nominal.
+dnl n3 is all missing -> nominal.
+dnl s1 has 24 unique small values -> scale.
+dnl s2 has one negative value -> scale.
+dnl s3 has one non-integer value -> scale.
+dnl s4 has no valid values less than 10 -> scale.
+dnl s5 has no valid values less than 10,000 -> scale.
+1 1 . 1 1 1 10 10001
+2 2 . 2 2 2 11 10002
+3 3 . 3 3 3 12 10003
+4 4 . 4 4 4 13 10004
+5 5 . 5 5 5 14 10005
+6 6 . 6 6 6 15 10006
+7 7 . 7 7 7 16 10007
+8 8 . 8 8 8 17 10008
+9 9 . 9 9 9 18 10009
+10 10 . 10 10 10.5 19 110000
+1 11 . 11 -1 1 11 10001
+2 12 . 12 2 2 12 10002
+3 13 . 13 3 3 13 10003
+4 14 . 14 4 4 14 10004
+5 15 . 15 5 5 15 10005
+6 16 . 16 6 6 16 10006
+7 17 . 17 7 7 17 10007
+8 18 . 18 8 8 18 10008
+9 19 . 19 9 9 19 10009
+1 20 . 20 1 1 20 10001
+2 21 . 21 2 2 21 10002
+3 22 . 22 3 3 22 10003
+4 23 . 23 4 4 23 10004
+5 23 . 24 5 5 24 10005
+6 23 . 24 6 6 25 10006
+])
+
+AT_DATA([ctables.sps], [dnl
+DATA LIST LIST file='data.txt' NOTABLE /n1 to n3 s1 to s5.
+
+* Nominal formats (copied from data that will default to scale).
+COMPUTE n4=s1.
+COMPUTE n5=s1.
+FORMATS n4(WKDAY5) n5(MONTH5).
+
+* Scale formats (copied from data that will default to nominal).
+COMPUTE s6=n1.
+COMPUTE s7=n1.
+COMPUTE s8=n1.
+FORMATS s6(DOLLAR6.2) s7(CCA8.2) s8(DATETIME17).
+
+STRING string(A8).
+
+DISPLAY DICTIONARY.
+CTABLES /TABLE n1 + n2 + n3 + string + s1 + s2 + s3 + s4 + s5.
+DISPLAY DICTIONARY.
+])
+
+AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
+ Variables
+╭──────┬────────┬──────────────┬─────┬─────┬─────────┬────────────┬────────────╮
+│ │ │ Measurement │ │ │ │ │ │
+│Name │Position│ Level │ Role│Width│Alignment│Print Format│Write Format│
+├──────┼────────┼──────────────┼─────┼─────┼─────────┼────────────┼────────────┤
+│n1 │ 1│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│n2 │ 2│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│n3 │ 3│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│s1 │ 4│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│s2 │ 5│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│s3 │ 6│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│s4 │ 7│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│s5 │ 8│Unknown │Input│ 8│Right │F8.2 │F8.2 │
+│n4 │ 9│Unknown │Input│ 8│Right │WKDAY5 │WKDAY5 │
+│n5 │ 10│Unknown │Input│ 8│Right │MONTH5 │MONTH5 │
+│s6 │ 11│Unknown │Input│ 8│Right │DOLLAR6.2 │DOLLAR6.2 │
+│s7 │ 12│Unknown │Input│ 8│Right │CCA8.2 │CCA8.2 │
+│s8 │ 13│Unknown │Input│ 8│Right │DATETIME17.0│DATETIME17.0│
+│string│ 14│Nominal │Input│ 8│Left │A8 │A8 │
+╰──────┴────────┴──────────────┴─────┴─────┴─────────┴────────────┴────────────╯
+
+ Custom Tables
+╭────────────┬─────┬────────╮
+│ │Count│ Mean │
+├────────────┼─────┼────────┤
+│n1 1.00 │ 3│ │
+│ 2.00 │ 3│ │
+│ 3.00 │ 3│ │
+│ 4.00 │ 3│ │
+│ 5.00 │ 3│ │
+│ 6.00 │ 3│ │
+│ 7.00 │ 2│ │
+│ 8.00 │ 2│ │
+│ 9.00 │ 2│ │
+│ 10.00│ 1│ │
+├────────────┼─────┼────────┤
+│n2 1.00 │ 1│ │
+│ 2.00 │ 1│ │
+│ 3.00 │ 1│ │
+│ 4.00 │ 1│ │
+│ 5.00 │ 1│ │
+│ 6.00 │ 1│ │
+│ 7.00 │ 1│ │
+│ 8.00 │ 1│ │
+│ 9.00 │ 1│ │
+│ 10.00│ 1│ │
+│ 11.00│ 1│ │
+│ 12.00│ 1│ │
+│ 13.00│ 1│ │
+│ 14.00│ 1│ │
+│ 15.00│ 1│ │
+│ 16.00│ 1│ │
+│ 17.00│ 1│ │
+│ 18.00│ 1│ │
+│ 19.00│ 1│ │
+│ 20.00│ 1│ │
+│ 21.00│ 1│ │
+│ 22.00│ 1│ │
+│ 23.00│ 3│ │
+├────────────┼─────┼────────┤
+│string │ 25│ │
+├────────────┼─────┼────────┤
+│s1 │ │ 12.96│
+├────────────┼─────┼────────┤
+│s2 │ │ 4.76│
+├────────────┼─────┼────────┤
+│s3 │ │ 4.86│
+├────────────┼─────┼────────┤
+│s4 │ │ 16.60│
+├────────────┼─────┼────────┤
+│s5 │ │14004.44│
+╰────────────┴─────┴────────╯
+
+ Variables
+╭──────┬────────┬──────────────┬─────┬─────┬─────────┬────────────┬────────────╮
+│ │ │ Measurement │ │ │ │ │ │
+│Name │Position│ Level │ Role│Width│Alignment│Print Format│Write Format│
+├──────┼────────┼──────────────┼─────┼─────┼─────────┼────────────┼────────────┤
+│n1 │ 1│Nominal │Input│ 8│Right │F8.2 │F8.2 │
+│n2 │ 2│Nominal │Input│ 8│Right │F8.2 │F8.2 │
+│n3 │ 3│Nominal │Input│ 8│Right │F8.2 │F8.2 │
+│s1 │ 4│Scale │Input│ 8│Right │F8.2 │F8.2 │
+│s2 │ 5│Scale │Input│ 8│Right │F8.2 │F8.2 │
+│s3 │ 6│Scale │Input│ 8│Right │F8.2 │F8.2 │
+│s4 │ 7│Scale │Input│ 8│Right │F8.2 │F8.2 │
+│s5 │ 8│Scale │Input│ 8│Right │F8.2 │F8.2 │
+│n4 │ 9│Nominal │Input│ 8│Right │WKDAY5 │WKDAY5 │
+│n5 │ 10│Nominal │Input│ 8│Right │MONTH5 │MONTH5 │
+│s6 │ 11│Scale │Input│ 8│Right │DOLLAR6.2 │DOLLAR6.2 │
+│s7 │ 12│Scale │Input│ 8│Right │CCA8.2 │CCA8.2 │
+│s8 │ 13│Scale │Input│ 8│Right │DATETIME17.0│DATETIME17.0│
+│string│ 14│Nominal │Input│ 8│Left │A8 │A8 │
+╰──────┴────────┴──────────────┴─────┴─────┴─────────┴────────────┴────────────╯
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DATA LIST])
+
+AT_SETUP([DATA LIST LIST with empty fields])
+AT_DATA([data-list.pspp], [dnl
+DATA LIST LIST NOTABLE /A B C (F1.0).
+BEGIN DATA.
+,,
+,,3
+,2,
+,2,3
+1,,
+1,,3
+1,2,
+1,2,3
+END DATA.
+
+LIST.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+Table: Data List
+A,B,C
+.,.,.
+.,.,3
+.,2,.
+.,2,3
+1,.,.
+1,.,3
+1,2,.
+1,2,3
+])
+AT_CLEANUP
+
+
+AT_SETUP([DATA LIST LIST with explicit delimiters])
+AT_DATA([data-list.pspp], [dnl
+data list list ('|','X') /A B C D.
+begin data.
+1|23X45|2.03x
+2X22|34|23|
+3|34|34X34
+end data.
+
+list.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+A,F8.0
+B,F8.0
+C,F8.0
+D,F8.0
+
+data-list.pspp:3.9-3.13: warning: Data for variable D is not valid as format F: Number followed by garbage.
+
+Table: Data List
+A,B,C,D
+1.00,23.00,45.00,. @&t@
+2.00,22.00,34.00,23.00
+3.00,34.00,34.00,34.00
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST FREE with SKIP])
+AT_DATA([data-list.pspp], [dnl
+data list free skip=1/A B C D.
+begin data.
+# This record is ignored.
+,1,2,x
+,4,,5
+6
+7,
+8 9
+0,1 ,,,
+,,,,
+2
+
+3
+4
+5
+end data.
+list.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+data-list.pspp:4.6: warning: Data for variable D is not valid as format F: Field contents are not numeric.
+
+Table: Data List
+A,B,C,D
+. ,1.00,2.00,. @&t@
+. ,4.00,. ,5.00
+6.00,7.00,8.00,9.00
+.00,1.00,. ,. @&t@
+. ,. ,. ,. @&t@
+2.00,3.00,4.00,5.00
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST LIST with SKIP and tab delimiter])
+AT_DATA([data-list.pspp], [dnl
+data list list (tab) notable skip=2/A B C D.
+begin data.
+# These records
+# are skipped.
+1 2 3 4
+1 2 3 @&t@
+1 2 4
+1 2 @&t@
+1 3 4
+1 3 @&t@
+1 4
+1 @&t@
+ 2 3 4
+ 2 3 @&t@
+ 2 4
+ 2 @&t@
+ 3 4
+ 3 @&t@
+ 4
+ @&t@
+end data.
+list.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+Table: Data List
+A,B,C,D
+1.00,2.00,3.00,4.00
+1.00,2.00,3.00,. @&t@
+1.00,2.00,. ,4.00
+1.00,2.00,. ,. @&t@
+1.00,. ,3.00,4.00
+1.00,. ,3.00,. @&t@
+1.00,. ,. ,4.00
+1.00,. ,. ,. @&t@
+. ,2.00,3.00,4.00
+. ,2.00,3.00,. @&t@
+. ,2.00,. ,4.00
+. ,2.00,. ,. @&t@
+. ,. ,3.00,4.00
+. ,. ,3.00,. @&t@
+. ,. ,. ,4.00
+. ,. ,. ,. @&t@
+])
+AT_CLEANUP
+
+dnl Results of this test were confirmed with SPSS 21:
+dnl http://lists.gnu.org/archive/html/pspp-dev/2013-09/msg00003.html
+AT_SETUP([DATA LIST FREE with explicit delimiter at end of line])
+AT_DATA([data-list.pspp], [dnl
+DATA LIST FREE(',')/x y z.
+BEGIN DATA.
+1,2,3
+4,5,6
+7,8,9
+END DATA.
+LIST.
+
+DATA LIST FREE(',')/x y z.
+BEGIN DATA.
+11,12,13,
+14,15,16,
+17,18,19,
+END DATA.
+LIST.
+
+DATA LIST FREE(TAB)/x y z.
+BEGIN DATA.
+21 22 23
+24 25 26
+27 28 29
+END DATA.
+LIST.
+
+DATA LIST FREE(TAB)/x y z.
+BEGIN DATA.
+31 32 33 @&t@
+34 35 36 @&t@
+37 38 39 @&t@
+END DATA.
+LIST.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+Table: Data List
+x,y,z
+1.00,2.00,3.00
+4.00,5.00,6.00
+7.00,8.00,9.00
+
+Table: Data List
+x,y,z
+11.00,12.00,13.00
+14.00,15.00,16.00
+17.00,18.00,19.00
+
+Table: Data List
+x,y,z
+21.00,22.00,23.00
+24.00,25.00,26.00
+27.00,28.00,29.00
+
+Table: Data List
+x,y,z
+31.00,32.00,33.00
+34.00,35.00,36.00
+37.00,38.00,39.00
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST FIXED with multiple records per case])
+AT_DATA([data-list.pspp], [dnl
+data list fixed notable
+ /1 start 1-20 (adate)
+ /2 end 1-20 (adate)
+ /3 count 1-3.
+begin data.
+07-22-2007
+10-06-2007
+x
+07-14-1789
+08-26-1789
+xy
+01-01-1972
+12-31-1999
+682
+end data.
+list.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+data-list.pspp:8.1-8.3: warning: Data for variable count is not valid as format F: Field contents are not numeric.
+
+data-list.pspp:11.1-11.3: warning: Data for variable count is not valid as format F: Field contents are not numeric.
+
+Table: Data List
+start,end,count
+07/22/2007,10/06/2007,.
+07/14/1789,08/26/1789,.
+01/01/1972,12/31/1999,682
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST FIXED with empty trailing record])
+AT_DATA([data-list.pspp], [dnl
+data list fixed notable records=2/x 1 y 2.
+begin data.
+12
+
+34
+
+56
+
+78
+
+90
+
+end data.
+list.
+])
+AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
+Table: Data List
+x,y
+1,2
+3,4
+5,6
+7,8
+9,0
+])
+AT_CLEANUP
+
+dnl Test that PSPP accepts LF and CR LF as line ends, but
+dnl treats isolated CR as linear whitespace.
+AT_SETUP([DATA LIST with various line-ends])
+AT_DATA([data-list.sps], [dnl
+data list list notable file='input.txt'/a b c.
+list.
+])
+printf '1 2 3\n4 5 6\r\n7\r8\r9\r\n10 11 12\n13 14 15 \r\n16\r\r17\r18\n' > input.txt
+dnl Make sure that input.txt actually received the data that we expect.
+dnl It might not have, if we're running on a system that translates \n
+dnl into some other sequence.
+AT_CHECK([cksum input.txt], [0], [1732021750 50 input.txt
+])
+AT_CHECK([pspp -o pspp.csv data-list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+a,b,c
+1.00,2.00,3.00
+4.00,5.00,6.00
+7.00,8.00,9.00
+10.00,11.00,12.00
+13.00,14.00,15.00
+16.00,17.00,18.00
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST properly expands tabs in input])
+AT_DATA([data-list.sps], [dnl
+data list notable /X 1-50 (a).
+begin data.
+ 1 12 123 1234 12345 .
+end data.
+print /x.
+print outfile='print.txt' /x.
+write outfile='write.txt' /x.
+execute.
+])
+AT_CHECK([sed -n '/12345/l' data-list.sps], [0], [dnl
+\t1\t12\t123\t1234\t12345 .$
+])
+AT_CHECK([pspp -o pspp.csv data-list.sps])
+dnl The CSV driver drops leading spaces so they don't appear here:
+AT_CHECK([cat pspp.csv], [0], [dnl
+1 12 123 1234 12345 . @&t@
+])
+dnl But they do appear in print.txt. The PRINT command also puts a space
+dnl at the beginning of the line and after the variable:
+AT_CHECK([cat print.txt], [0], [dnl
+ 1 12 123 1234 12345 . @&t@
+])
+dnl WRITE doesn't add spaces at the beginning or end of lines:
+AT_CHECK([cat write.txt], [0], [dnl
+ 1 12 123 1234 12345 .
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST FREE and LIST report missing delimiters])
+AT_DATA([data-list.sps], [dnl
+DATA LIST FREE NOTABLE/s (a10).
+LIST.
+BEGIN DATA.
+'y'z
+END DATA.
+])
+AT_CHECK([pspp -O format=csv data-list.sps], [0], [dnl
+data-list.sps:4: warning: Missing delimiter following quoted string.
+
+Table: Data List
+s
+y
+z
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST FREE and LIST assume a width if omitted])
+AT_DATA([data-list.sps], [dnl
+DATA LIST FREE TABLE/s (a) d (datetime) f (f).
+])
+AT_CHECK([pspp -O format=csv data-list.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+s,A1
+d,DATETIME17.0
+f,F1.0
+])
+AT_CLEANUP
+
+AT_SETUP([DATA LIST Decimal comma])
+AT_DATA([data-list.sps], [dnl
+SET DECIMAL=COMMA.
+
+DATA LIST NOTABLE LIST /A *.
+BEGIN DATA
+1
+2
+3
+3,5
+4
+4,5
+5
+6
+END DATA
+
+LIST /FORMAT=NUMBERED.
+])
+
+AT_CHECK([pspp -O format=csv data-list.sps], [0], [dnl
+Table: Data List
+Case Number,A
+1,"1,00"
+2,"2,00"
+3,"3,00"
+4,"3,50"
+5,"4,00"
+6,"4,50"
+7,"5,00"
+8,"6,00"
+])
+
+AT_CLEANUP
+
+AT_SETUP([DATA LIST syntax errors])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='data-list.sps' ERROR=IGNORE.
+])
+AT_DATA([data-list.sps], [dnl
+DATA LIST FILE=**.
+DATA LIST ENCODING=**.
+DATA LIST RECORDS=1 RECORDS=2.
+DATA LIST RECORDS=0.
+DATA LIST SKIP=-1.
+DATA LIST END=**.
+INPUT PROGRAM.
+DATA LIST END=xyzzy END=xyzzy.
+END INPUT PROGRAM.
+INPUT PROGRAM.
+DATA LIST END=**.
+END INPUT PROGRAM.
+DATA LIST XYZZY.
+DATA LIST FREE LIST.
+DATA LIST LIST (**).
+DATA LIST **.
+DATA LIST ENCODING='xyzzy'/x.
+INPUT PROGRAM.
+DATA LIST LIST END=xyzzy/x.
+END INPUT PROGRAM.
+DATA LIST FIXED/0.
+DATA LIST FIXED/ **.
+DATA LIST FIXED/x 1.5.
+DATA LIST FIXED/x -1.
+DATA LIST FIXED/x 5-3.
+DATA LIST FIXED/x y 1-3.
+DATA LIST FIXED/x 1-5 (xyzzy).
+DATA LIST FIXED/x 1-5 (**).
+DATA LIST FIXED/x 1 (F,5).
+DATA LIST FIXED/x (2F8.0).
+DATA LIST FIXED/x **.
+DATA LIST FIXED/x 1 x 2.
+INPUT PROGRAM.
+DATA LIST FIXED/x 1.
+DATA LIST FIXED/x 1 (a).
+END INPUT PROGRAM.
+INPUT PROGRAM.
+DATA LIST FIXED/y 2 (a).
+DATA LIST FIXED/y 3-4 (a).
+END INPUT PROGRAM.
+DATA LIST FIXED RECORDS=1/2 x 1-2.
+])
+
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"data-list.sps:1.16-1.17: error: DATA LIST: Syntax error expecting a file name or handle name.
+ 1 | DATA LIST FILE=**.
+ | ^~"
+
+"data-list.sps:2.20-2.21: error: DATA LIST: Syntax error expecting string.
+ 2 | DATA LIST ENCODING=**.
+ | ^~"
+
+"data-list.sps:3.21-3.27: error: DATA LIST: Subcommand RECORDS may only be specified once.
+ 3 | DATA LIST RECORDS=1 RECORDS=2.
+ | ^~~~~~~"
+
+"data-list.sps:4.20: error: DATA LIST: Syntax error expecting one of the following: FILE, ENCODING, RECORDS, SKIP, END, NOTABLE, TABLE, FIXED, FREE, LIST.
+ 4 | DATA LIST RECORDS=0.
+ | ^"
+
+"data-list.sps:5.16-5.17: error: DATA LIST: Syntax error expecting non-negative integer for SKIP.
+ 5 | DATA LIST SKIP=-1.
+ | ^~"
+
+"data-list.sps:6.11-6.13: error: DATA LIST: The END subcommand may only be used within INPUT PROGRAM.
+ 6 | DATA LIST END=**.
+ | ^~~"
+
+"data-list.sps:8.21-8.23: error: DATA LIST: Subcommand END may only be specified once.
+ 8 | DATA LIST END=xyzzy END=xyzzy.
+ | ^~~"
+
+"data-list.sps:11.15-11.16: error: DATA LIST: Syntax error expecting identifier.
+ 11 | DATA LIST END=**.
+ | ^~"
+
+"data-list.sps:13.11-13.15: error: DATA LIST: Syntax error expecting one of the following: FILE, ENCODING, RECORDS, SKIP, END, NOTABLE, TABLE, FIXED, FREE, LIST.
+ 13 | DATA LIST XYZZY.
+ | ^~~~~"
+
+"data-list.sps:14.16-14.19: error: DATA LIST: Only one of FIXED, FREE, or LIST may be specified.
+ 14 | DATA LIST FREE LIST.
+ | ^~~~"
+
+"data-list.sps:15.17-15.18: error: DATA LIST: Syntax error expecting TAB or delimiter string.
+ 15 | DATA LIST LIST (**).
+ | ^~"
+
+"data-list.sps:16.11-16.12: error: DATA LIST: Syntax error expecting one of the following: FILE, ENCODING, RECORDS, SKIP, END, NOTABLE, TABLE, FIXED, FREE, LIST.
+ 16 | DATA LIST **.
+ | ^~"
+
+"data-list.sps:17.11-17.26: warning: DATA LIST: Encoding should not be specified for inline data. It will be ignored.
+ 17 | DATA LIST ENCODING='xyzzy'/x.
+ | ^~~~~~~~~~~~~~~~"
+
+"data-list.sps:17.29: error: DATA LIST: SPSS-like or Fortran-like format specification expected after variable names.
+ 17 | DATA LIST ENCODING='xyzzy'/x.
+ | ^"
+
+"data-list.sps:19.16-19.24: error: DATA LIST: The END subcommand may be used only with DATA LIST FIXED.
+ 19 | DATA LIST LIST END=xyzzy/x.
+ | ^~~~~~~~~"
+
+"data-list.sps:21.17: error: DATA LIST: Syntax error expecting positive integer.
+ 21 | DATA LIST FIXED/0.
+ | ^"
+
+"data-list.sps:22.18-22.19: error: DATA LIST: Syntax error expecting variable name.
+ 22 | DATA LIST FIXED/ **.
+ | ^~"
+
+"data-list.sps:23.19-23.21: error: DATA LIST: Syntax error expecting integer.
+ 23 | DATA LIST FIXED/x 1.5.
+ | ^~~"
+
+"data-list.sps:24.19-24.20: error: DATA LIST: Column positions for fields must be positive.
+ 24 | DATA LIST FIXED/x -1.
+ | ^~"
+
+"data-list.sps:25.19-25.21: error: DATA LIST: The ending column for a field must be greater than the starting column.
+ 25 | DATA LIST FIXED/x 5-3.
+ | ^~~"
+
+"data-list.sps:26.21-26.23: error: DATA LIST: The 3 columns 1-3 can't be evenly divided into 2 fields.
+ 26 | DATA LIST FIXED/x y 1-3.
+ | ^~~"
+
+"data-list.sps:27.24-27.28: error: DATA LIST: Unknown format type `xyzzy'.
+ 27 | DATA LIST FIXED/x 1-5 (xyzzy).
+ | ^~~~~"
+
+"data-list.sps:28.24-28.25: error: DATA LIST: Syntax error expecting `)'.
+ 28 | DATA LIST FIXED/x 1-5 (**).
+ | ^~"
+
+"data-list.sps:29.19-29.25: error: DATA LIST: Input format F1.5 specifies 5 decimal places, but width 1 allows at most 1 decimals.
+ 29 | DATA LIST FIXED/x 1 (F,5).
+ | ^~~~~~~"
+
+"data-list.sps:30.20-30.25: error: DATA LIST: Number of variables specified (1) differs from number of variable formats (2).
+ 30 | DATA LIST FIXED/x (2F8.0).
+ | ^~~~~~"
+
+"data-list.sps:31.19-31.20: error: DATA LIST: SPSS-like or Fortran-like format specification expected after variable names.
+ 31 | DATA LIST FIXED/x **.
+ | ^~"
+
+"data-list.sps:32.21: error: DATA LIST: x is a duplicate variable name.
+ 32 | DATA LIST FIXED/x 1 x 2.
+ | ^"
+
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+x,1,1-1,F1.0
+
+"data-list.sps:35.17-35.23: error: DATA LIST: There is already a variable x of a different type.
+ 35 | DATA LIST FIXED/x 1 (a).
+ | ^~~~~~~"
+
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+y,1,2-2,A1
+
+"data-list.sps:39.17-39.25: error: DATA LIST: There is already a string variable y of a different width.
+ 39 | DATA LIST FIXED/y 3-4 (a).
+ | ^~~~~~~~~"
+
+"data-list.sps:41.26-41.29: error: DATA LIST: Cannot place variable x on record 2 when RECORDS=1 is specified.
+ 41 | DATA LIST FIXED RECORDS=1/2 x 1-2.
+ | ^~~~"
+])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([BEGIN DATA])
+
+# BEGIN DATA can run as a command in itself, or it can appear as part
+# of the first procedure. First, test it after a procedure.
+AT_SETUP([BEGIN DATA as part of a procedure])
+AT_DATA([begin-data.sps], [dnl
+TITLE 'Test BEGIN DATA ... END DATA'.
+
+DATA LIST /a b 1-2.
+LIST.
+BEGIN DATA.
+12
+34
+56
+78
+90
+END DATA.
+])
+AT_CHECK([pspp -O format=csv begin-data.sps], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+a,1,1-1,F1.0
+b,1,2-2,F1.0
+
+Table: Data List
+a,b
+1,2
+3,4
+5,6
+7,8
+9,0
+])
+AT_CLEANUP
+
+# Also test BEGIN DATA as an independent command.
+AT_SETUP([BEGIN DATA as an independent command])
+AT_DATA([begin-data.sps], [dnl
+data list /A B 1-2.
+begin data.
+09
+87
+65
+43
+21
+end data.
+list.
+])
+AT_CHECK([pspp -O format=csv begin-data.sps], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+A,1,1-1,F1.0
+B,1,2-2,F1.0
+
+Table: Data List
+A,B
+0,9
+8,7
+6,5
+4,3
+2,1
+])
+AT_CLEANUP
+
+m4_define([DATA_READER_BINARY],
+ [AT_SETUP([read and write files with $1])
+$3
+ AT_DATA([input.txt], [dnl
+07-22-2007
+10-06-2007
+321
+07-14-1789
+08-26-1789
+4
+01-01-1972
+12-31-1999
+682
+])
+ AT_DATA([make-binary.py], [[
+#! /usr/bin/python3
+
+import struct
+import sys
+
+# This random number generator and the test for it below are drawn
+# from Park and Miller, "Random Number Generators: Good Ones are Hard
+# to Come By", Communications of the ACM 31:10 (October 1988). It is
+# documented to function properly on systems with a 46-bit or longer
+# real significand, which includes systems that have 64-bit IEEE reals
+# (with 53-bit significand). The test should catch any systems for
+# which this is not true, in any case.
+def my_rand(modulus):
+ global seed
+ a = 16807
+ m = 2147483647
+ tmp = a * seed
+ seed = tmp - m * (tmp // m)
+ return seed % modulus
+
+# Test the random number generator for reproducibility,
+# then reset the seed
+seed = 1
+for i in range(10000):
+ my_rand(1)
+assert seed == 1043618065
+seed = 1
+
+# ASCII to EBCDIC translation table
+ascii2ebcdic = (
+ 0x00, 0x01, 0x02, 0x03, 0x37, 0x2d, 0x2e, 0x2f,
+ 0x16, 0x05, 0x25, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ 0x10, 0x11, 0x12, 0x13, 0x3c, 0x3d, 0x32, 0x26,
+ 0x18, 0x19, 0x3f, 0x27, 0x1c, 0x1d, 0x1e, 0x1f,
+ 0x40, 0x5a, 0x7f, 0x7b, 0x5b, 0x6c, 0x50, 0x7d,
+ 0x4d, 0x5d, 0x5c, 0x4e, 0x6b, 0x60, 0x4b, 0x61,
+ 0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7,
+ 0xf8, 0xf9, 0x7a, 0x5e, 0x4c, 0x7e, 0x6e, 0x6f,
+ 0x7c, 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7,
+ 0xc8, 0xc9, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6,
+ 0xd7, 0xd8, 0xd9, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6,
+ 0xe7, 0xe8, 0xe9, 0xad, 0xe0, 0xbd, 0x9a, 0x6d,
+ 0x79, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,
+ 0x88, 0x89, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96,
+ 0x97, 0x98, 0x99, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6,
+ 0xa7, 0xa8, 0xa9, 0xc0, 0x4f, 0xd0, 0x5f, 0x07,
+ 0x20, 0x21, 0x22, 0x23, 0x24, 0x15, 0x06, 0x17,
+ 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x09, 0x0a, 0x1b,
+ 0x30, 0x31, 0x1a, 0x33, 0x34, 0x35, 0x36, 0x08,
+ 0x38, 0x39, 0x3a, 0x3b, 0x04, 0x14, 0x3e, 0xe1,
+ 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48,
+ 0x49, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57,
+ 0x58, 0x59, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67,
+ 0x68, 0x69, 0x70, 0x71, 0x72, 0x73, 0x74, 0x75,
+ 0x76, 0x77, 0x78, 0x80, 0x8a, 0x8b, 0x8c, 0x8d,
+ 0x8e, 0x8f, 0x90, 0x6a, 0x9b, 0x9c, 0x9d, 0x9e,
+ 0x9f, 0xa0, 0xaa, 0xab, 0xac, 0x4a, 0xae, 0xaf,
+ 0xb0, 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7,
+ 0xb8, 0xb9, 0xba, 0xbb, 0xbc, 0xa1, 0xbe, 0xbf,
+ 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf, 0xda, 0xdb,
+ 0xdc, 0xdd, 0xde, 0xdf, 0xea, 0xeb, 0xec, 0xed,
+ 0xee, 0xef, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff)
+assert len(ascii2ebcdic) == 256
+
+def a2e(s):
+ return bytearray((ascii2ebcdic[ord(c)] for c in s))
+
+def dump_records(out, records):
+ while records:
+ n = min(my_rand(5) + 1, len(records))
+ r = records[:n]
+ records[:n] = []
+
+ count = sum((len(rec) for rec in r))
+ out.buffer.write(struct.pack(">H xx", count + 4))
+ for rec in r:
+ out.buffer.write(rec)
+
+data = []
+for line in open('input.txt', 'r'):
+ data += [line.rstrip('\r\n')]
+
+# MODE=BINARY
+out = open('binary.bin', 'w')
+for item in data:
+ reclen = struct.pack("<I", len(item))
+ out.buffer.write(reclen)
+ out.buffer.write(bytearray([ord(c) for c in item]))
+ out.buffer.write(reclen)
+out.close()
+
+# MODE=360 /RECFORM=FIXED /LRECL=32
+out = open('fixed.bin', 'w')
+lrecl = 32
+for item in data:
+ s = item[:lrecl]
+ s += ' ' * (lrecl - len(s))
+ assert len(s) == 32
+ out.buffer.write(a2e(s))
+out.close()
+
+# MODE=360 /RECFORM=VARIABLE
+out = open('variable.bin', 'w')
+records = []
+for item in data:
+ records += [struct.pack('>H xx', len(item) + 4) + a2e(item)]
+dump_records(out, records)
+out.close()
+
+# MODE=360 /RECFORM=SPANNED
+out = open('spanned.bin', 'w')
+records = []
+for line in data:
+ r = []
+ while line:
+ n = min(my_rand(5), len(line))
+ r += [line[:n]]
+ line = line[n:]
+ for i, s in enumerate(r):
+ scc = (0 if len(r) == 1
+ else 1 if i == 0
+ else 2 if i == len(r) - 1
+ else 3)
+ records += [struct.pack('>H B x', len(s) + 4, scc) + a2e(s)]
+dump_records(out, records)
+out.close()
+]])
+ AT_CHECK([$PYTHON3 make-binary.py])
+ AT_DATA([data-reader.sps], [dnl
+FILE HANDLE input/NAME='$2'/$1.
+DATA LIST FIXED FILE=input NOTABLE
+ /1 start 1-10 (ADATE)
+ /2 end 1-10 (ADATE)
+ /3 count 1-3.
+LIST.
+
+* Output the data to a new file in the same format.
+FILE HANDLE OUTPUT/NAME='output.bin'/$1.
+COMPUTE count=count + 1.
+PRINT OUTFILE=output/start end count.
+EXECUTE.
+])
+ AT_CHECK([pspp -O format=csv data-reader.sps], [0], [dnl
+Table: Data List
+start,end,count
+07/22/2007,10/06/2007,321
+07/14/1789,08/26/1789,4
+01/01/1972,12/31/1999,682
+])
+ AT_CHECK([test -s output.bin])
+ AT_DATA([data-reader-2.sps], [dnl
+* Re-read the new data and list it, to verify that it was written correctly.
+FILE HANDLE OUTPUT/NAME='output.bin'/$1.
+DATA LIST FIXED FILE=output NOTABLE/
+ start 2-11 (ADATE)
+ end 13-22 (ADATE)
+ count 24-26.
+LIST.
+])
+ AT_CHECK([pspp -O format=csv data-reader-2.sps], [0], [dnl
+Table: Data List
+start,end,count
+07/22/2007,10/06/2007,322
+07/14/1789,08/26/1789,5
+01/01/1972,12/31/1999,683
+])
+ AT_CLEANUP])
+
+DATA_READER_BINARY([MODE=BINARY], [binary.bin])
+DATA_READER_BINARY([MODE=360 /RECFORM=FIXED /LRECL=32], [fixed.bin],
+ [AT_CHECK([i18n-test supports_encodings EBCDIC-US])])
+DATA_READER_BINARY([MODE=360 /RECFORM=VARIABLE], [variable.bin],
+ [AT_CHECK([i18n-test supports_encodings EBCDIC-US])])
+DATA_READER_BINARY([MODE=360 /RECFORM=SPANNED], [spanned.bin],
+ [AT_CHECK([i18n-test supports_encodings EBCDIC-US])])
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DATASET commands])
+
+AT_SETUP([DATASET COPY])
+AT_DATA([dataset.pspp], [dnl
+DATASET NAME initial.
+DATA LIST NOTABLE /x 1.
+COMPUTE x = x + 1.
+DATASET COPY clone.
+BEGIN DATA.
+1
+2
+3
+4
+5
+END DATA.
+
+NEW FILE.
+DATA LIST NOTABLE /y 1.
+BEGIN DATA.
+6
+7
+8
+END DATA.
+LIST.
+DATASET DISPLAY.
+
+DATASET ACTIVATE clone.
+DATASET DISPLAY.
+LIST.
+
+DATASET ACTIVATE initial.
+DATASET DISPLAY.
+LIST.
+
+COMPUTE z=y.
+DATASET COPY clone.
+
+DATASET ACTIVATE clone.
+LIST.
+DATASET COPY clone.
+DATASET DISPLAY.
+
+DATASET CLOSE initial.
+DATASET DISPLAY.
+])
+AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
+Table: Data List
+y
+6
+7
+8
+
+Table: Datasets
+clone
+initial (active dataset)
+
+Table: Datasets
+clone (active dataset)
+initial
+
+Table: Data List
+x
+2
+3
+4
+5
+6
+
+Table: Datasets
+clone
+initial (active dataset)
+
+Table: Data List
+y
+6
+7
+8
+
+Table: Data List
+y,z
+6,6.00
+7,7.00
+8,8.00
+
+Table: Datasets
+unnamed dataset (active dataset)
+initial
+
+Table: Datasets
+unnamed dataset (active dataset)
+])
+AT_CLEANUP
+
+AT_SETUP([DATASET DECLARE])
+AT_DATA([dataset.pspp], [dnl
+DATASET DECLARE second.
+DATASET DISPLAY.
+DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+1
+END DATA.
+LIST.
+DATASET ACTIVATE second.
+DATASET DISPLAY.
+LIST.
+])
+AT_CHECK([pspp -O format=csv dataset.pspp], [1], [dnl
+Table: Datasets
+unnamed dataset (active dataset)
+second
+
+Table: Data List
+x
+1
+
+Table: Datasets
+second (active dataset)
+
+"dataset.pspp:10.1-10.4: error: LIST: LIST is allowed only after the active dataset has been defined.
+ 10 | LIST.
+ | ^~~~"
+])
+AT_CLEANUP
+
+AT_SETUP([DATASET NAME deletes duplicate name])
+AT_DATA([dataset.pspp], [dnl
+DATASET NAME a.
+DATASET DECLARE b.
+DATASET DECLARE c.
+DATASET DISPLAY.
+
+DATASET NAME b.
+DATASET NAME c.
+DATASET DISPLAY.
+])
+AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
+Table: Datasets
+a (active dataset)
+b
+c
+
+Table: Datasets
+c (active dataset)
+])
+AT_CLEANUP
+
+AT_SETUP([DATASET ACTIVATE deletes unnamed dataset])
+AT_DATA([dataset.pspp], [dnl
+DATASET DECLARE x.
+DATASET DISPLAY.
+
+DATASET ACTIVATE x.
+DATASET DISPLAY.
+])
+AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
+Table: Datasets
+unnamed dataset (active dataset)
+x
+
+Table: Datasets
+x (active dataset)
+])
+AT_CLEANUP
+
+AT_SETUP([DATASET ACTIVATE executes pending transformations])
+AT_DATA([dataset.pspp], [dnl
+DATASET NAME one.
+DATASET DECLARE another.
+DATASET DISPLAY.
+
+DATA LIST NOTABLE /x 1.
+PRINT/x.
+DATASET ACTIVATE another.
+BEGIN DATA.
+1
+2
+3
+4
+5
+END DATA.
+
+LIST.
+
+DATASET ACTIVATE one.
+LIST.
+])
+AT_CHECK([pspp -O format=csv dataset.pspp], [1], [dnl
+Table: Datasets
+another
+one (active dataset)
+
+1 @&t@
+
+2 @&t@
+
+3 @&t@
+
+4 @&t@
+
+5 @&t@
+
+"dataset.pspp:16.1-16.4: error: LIST: LIST is allowed only after the active dataset has been defined.
+ 16 | LIST.
+ | ^~~~"
+
+Table: Data List
+x
+1
+2
+3
+4
+5
+])
+AT_CLEANUP
+
+AT_SETUP([DATASET CLOSE])
+AT_DATA([dataset.pspp], [dnl
+DATASET DISPLAY
+DATASET CLOSE *.
+DATASET DISPLAY.
+
+DATASET NAME this.
+DATASET DISPLAY.
+DATASET CLOSE this.
+DATASET DISPLAY.
+
+DATASET NAME this.
+DATASET DISPLAY.
+DATASET CLOSE *.
+DATASET DISPLAY.
+
+DATASET DECLARE that.
+DATASET DECLARE theother.
+DATASET DECLARE yetanother.
+DATASET DISPLAY.
+DATASET CLOSE ALL.
+DATASET DISPLAY.
+
+DATASET NAME this.
+DATASET DECLARE that.
+DATASET DECLARE theother.
+DATASET DECLARE yetanother.
+DATASET DISPLAY.
+DATASET CLOSE ALL.
+DATASET DISPLAY.
+])
+AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
+Table: Datasets
+unnamed dataset (active dataset)
+
+Table: Datasets
+unnamed dataset (active dataset)
+
+Table: Datasets
+this (active dataset)
+
+Table: Datasets
+unnamed dataset (active dataset)
+
+Table: Datasets
+this (active dataset)
+
+Table: Datasets
+unnamed dataset (active dataset)
+
+Table: Datasets
+unnamed dataset (active dataset)
+that
+theother
+yetanother
+
+Table: Datasets
+unnamed dataset (active dataset)
+
+Table: Datasets
+that
+theother
+this (active dataset)
+yetanother
+
+Table: Datasets
+unnamed dataset (active dataset)
+])
+AT_CLEANUP
+
+
+
+dnl The bug for which the following test checks, is apparent only
+dnl when compiled with -fsanitize=address or run under valgrind
+AT_SETUP([DATASET heap overflow])
+AT_DATA([dataset.pspp], [dnl
+DATASET DECLARE initial.
+DATA LIST /x 1.
+
+DATASET COPY subsq.
+
+DATA LIST /y 2-4.
+BEGIN DATA.
+7
+END DATA.
+
+DATASET ACTIVATE subsq.
+
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+x,1,1-1,F1.0
+
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+y,1,2-4,F3.0
+])
+
+AT_CLEANUP
+
+AT_SETUP([DATASET syntax errors])
+AT_DATA([dataset.sps], [dnl
+DATASET NAME **.
+DATASET NAME xyzzy WINDOW **.
+
+DATASET NAME xyzzy.
+DATASET ACTIVATE **.
+DATASET ACTIVATE xyzzy WINDOW **.
+
+DATASET COPY **.
+DATASET COPY quux WINDOW **.
+
+DATASET DECLARE **.
+DATASET DECLARE foo WINDOW **.
+
+DATASET CLOSE **.
+])
+AT_CHECK([pspp dataset.sps], [1], [dnl
+dataset.sps:1.14-1.15: error: DATASET NAME: Syntax error expecting identifier.
+ 1 | DATASET NAME **.
+ | ^~
+
+dataset.sps:2.27-2.28: error: DATASET NAME: Syntax error expecting ASIS or
+FRONT.
+ 2 | DATASET NAME xyzzy WINDOW **.
+ | ^~
+
+dataset.sps:5.18-5.19: error: DATASET ACTIVATE: Syntax error expecting
+identifier.
+ 5 | DATASET ACTIVATE **.
+ | ^~
+
+dataset.sps:6.31-6.32: error: DATASET ACTIVATE: Syntax error expecting ASIS or
+FRONT.
+ 6 | DATASET ACTIVATE xyzzy WINDOW **.
+ | ^~
+
+dataset.sps:8.14-8.15: error: DATASET COPY: Syntax error expecting identifier.
+ 8 | DATASET COPY **.
+ | ^~
+
+dataset.sps:9.26-9.27: error: DATASET COPY: Syntax error expecting MINIMIZED,
+FRONT, or HIDDEN.
+ 9 | DATASET COPY quux WINDOW **.
+ | ^~
+
+dataset.sps:11.17-11.18: error: DATASET DECLARE: Syntax error expecting
+identifier.
+ 11 | DATASET DECLARE **.
+ | ^~
+
+dataset.sps:12.28-12.29: error: DATASET DECLARE: Syntax error expecting
+MINIMIZED, FRONT, or HIDDEN.
+ 12 | DATASET DECLARE foo WINDOW **.
+ | ^~
+
+dataset.sps:14.15-14.16: error: DATASET CLOSE: Syntax error expecting
+identifier.
+ 14 | DATASET CLOSE **.
+ | ^~
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([USE])
+
+AT_SETUP([USE ALL])
+AT_DATA([use.sps], [dnl
+data list notable /X 1-2.
+begin data.
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+end data.
+use all.
+list.
+])
+AT_CHECK([pspp -o pspp.csv use.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+X
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU nGeneral Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DEFINE])
+
+AT_SETUP([simple macro expansion])
+AT_DATA([define.sps], [dnl
+DEFINE !macro()
+a b c d
+e f g h.
+i j k l
+1,2,3,4.
+5+6+7.
+m(n,o).
+"a" "b" "c" 'a' 'b' 'c'.
+"x "" y".
+!ENDDEFINE.
+DEBUG EXPAND.
+!macro
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+a b c d e f g h.
+i j k l 1, 2, 3, 4.
+5 + 6 + 7.
+m(n, o).
+"a" "b" "c" 'a' 'b' 'c'.
+"x "" y".
+])
+AT_CLEANUP
+
+AT_SETUP([redefining a macro])
+AT_DATA([define.sps], [dnl
+DEFINE !macro() 0 !ENDDEFINE.
+DEFINE !macro() 1 !ENDDEFINE.
+DEBUG EXPAND.
+!macro.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+1
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - one !TOKENS(1) positional argument])
+AT_KEYWORDS([TOKENS])
+AT_DATA([define.sps], [dnl
+DEFINE !t1(!positional=!tokens(1)) t1 (!1) !ENDDEFINE.
+DEBUG EXPAND.
+!t1 a.
+!t1 b.
+!t1 a b.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+t1(a)
+
+t1(b)
+
+t1(a)
+
+note: unexpanded token "b"
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion with positional arguments])
+AT_DATA([define.sps], [dnl
+DEFINE !title(!positional !tokens(1)) !1 !ENDDEFINE.
+DEFINE !t1(!positional !tokens(1)) t1 (!1) !ENDDEFINE.
+DEFINE !t2(!positional !tokens(2)) t2 (!1) !ENDDEFINE.
+
+DEFINE !ce(!positional=!charend('/')) ce (!1) !ENDDEFINE.
+DEFINE !ce2(!positional=!charend('(')
+ /!positional !charend(')'))
+ce2 (!1, !2)
+!ENDDEFINE.
+
+DEFINE !e(!positional !enclose('{','}')) e (!1) !ENDDEFINE.
+
+DEFINE !cmd(!positional !cmdend) cmd(!1) !ENDDEFINE.
+DEFINE !cmd2(!positional !cmdend
+ /!positional !tokens(1))
+cmd2(!1, !2)
+!ENDDEFINE.
+
+DEFINE !p(!positional !tokens(1)
+ /!positional !tokens(1)
+ /!positional !tokens(1))
+p(!1, !2, !3)(!*)
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!title "!TOKENS(1) argument."
+!t1 a.
+!t1 b.
+!t1 a b.
+
+!title "!TOKENS(2) argument."
+!t2 a b.
+!t2 b c d.
+
+!title "!CHAREND argument."
+!ce/.
+!ce x/.
+!ce x y/.
+!ce x y z/.
+
+!title "Two !CHAREND arguments."
+!ce2 x(y).
+!ce2 1 2 3 4().
+
+!title "!ENCLOSE argument."
+!e {}.
+!e {a}.
+!e {a b}.
+
+!title "!CMDEND argument."
+!cmd 1 2 3 4.
+!cmd2 5 6.
+7.
+
+!title "Three !TOKENS(1) arguments."
+!p a b c.
+!p 1 -2 -3.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+"!TOKENS(1) argument."
+
+t1(a)
+
+t1(b)
+
+t1(a)
+
+note: unexpanded token "b"
+
+"!TOKENS(2) argument."
+
+t2(a b)
+
+t2(b c)
+
+note: unexpanded token "d"
+
+"!CHAREND argument."
+
+ce( )
+
+ce(x)
+
+ce(x y)
+
+ce(x y z)
+
+"Two !CHAREND arguments."
+
+ce2(x, y)
+
+ce2(1 2 3 4, )
+
+"!ENCLOSE argument."
+
+e( )
+
+e(a)
+
+e(a b)
+
+"!CMDEND argument."
+
+cmd(1 2 3 4)
+
+cmd2(5 6, )
+
+note: unexpanded token "7"
+
+"Three !TOKENS(1) arguments."
+
+p(a, b, c) (a b c)
+
+p(1, -2, -3) (1 -2 -3)
+])
+AT_CLEANUP
+
+AT_SETUP([macro call missing positional !TOKENS arguments])
+AT_KEYWORDS([TOKENS])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !tokens(1) !default(x)
+ /!positional !tokens(1) !default(y)
+ /!positional !tokens(1) !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p a b c.
+!p a b.
+!p a.
+!p.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+(a, b, c)
+
+(a, b, z)
+
+(a, y, z)
+
+(x, y, z)
+])
+AT_CLEANUP
+
+AT_SETUP([macro call incomplete positional !TOKENS arguments])
+AT_KEYWORDS([TOKENS])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !tokens(2) !default(x)
+ /!positional !tokens(2) !default(y)
+ /!positional !tokens(2) !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p a1 a2 b1 b2 c1 c2.
+!p a1 a2 b1 b2 c1.
+!p a1 a2 b1 b2.
+!p a1 a2 b1.
+!p a1 a2.
+!p a1.
+!p.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+(a1 a2, b1 b2, c1 c2)
+
+define.sps:8.18: error: DEBUG EXPAND: Reached end of command expecting 1 more
+token in argument !3 to macro !p.
+ 8 | !p a1 a2 b1 b2 c1.
+ | ^
+
+(a1 a2, b1 b2, c1)
+
+(a1 a2, b1 b2, z)
+
+define.sps:10.12: error: DEBUG EXPAND: Reached end of command expecting 1 more
+token in argument !2 to macro !p.
+ 10 | !p a1 a2 b1.
+ | ^
+
+(a1 a2, b1, z)
+
+(a1 a2, y, z)
+
+define.sps:12.6: error: DEBUG EXPAND: Reached end of command expecting 1 more
+token in argument !1 to macro !p.
+ 12 | !p a1.
+ | ^
+
+(a1, y, z)
+
+(x, y, z)
+])
+AT_CLEANUP
+
+AT_SETUP([macro call empty positional !CHAREND arguments])
+AT_KEYWORDS([CHAREND])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !charend(',') !default(x)
+ /!positional !charend(';') !default(y)
+ /!positional !charend(':') !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p a,b;c:.
+!p a,b;:.
+!p a,;c:.
+!p a,;:.
+!p,b;c:.
+!p,b;:.
+!p,;c:.
+!p,;:.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+(a, b, c)
+
+(a, b, )
+
+(a, , c)
+
+(a, , )
+
+(, b, c)
+
+(, b, )
+
+(, , c)
+
+(, , )
+])
+AT_CLEANUP
+
+AT_SETUP([macro call missing positional !CHAREND arguments])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !charend(',') !default(x)
+ /!positional !charend(';') !default(y)
+ /!positional !charend(':') !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p a,b;c:.
+!p a,b;.
+!p a,;.
+!p ,b;.
+!p ,;.
+
+!p a,.
+!p ,.
+
+!p.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+(a, b, c)
+
+(a, b, z)
+
+(a, , z)
+
+(, b, z)
+
+(, , z)
+
+(a, y, z)
+
+(, y, z)
+
+(x, y, z)
+])
+AT_CLEANUP
+
+AT_SETUP([macro call incomplete positional !CHAREND arguments])
+AT_KEYWORDS([CHAREND])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !charend(',') !default(x)
+ /!positional !charend(';') !default(y)
+ /!positional !charend(':') !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p a,b;c:.
+!p a,b;c.
+!p a,b;.
+!p a,b.
+!p a,.
+!p a.
+!p.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+(a, b, c)
+
+define.sps:8.9: error: DEBUG EXPAND: Reached end of command expecting ":" in
+argument !3 to macro !p.
+ 8 | !p a,b;c.
+ | ^
+
+(a, b, c)
+
+(a, b, z)
+
+define.sps:10.7: error: DEBUG EXPAND: Reached end of command expecting ";" in
+argument !2 to macro !p.
+ 10 | !p a,b.
+ | ^
+
+(a, b, z)
+
+(a, y, z)
+
+define.sps:12.5: error: DEBUG EXPAND: Reached end of command expecting "," in
+argument !1 to macro !p.
+ 12 | !p a.
+ | ^
+
+(a, y, z)
+
+(x, y, z)
+])
+AT_CLEANUP
+
+AT_SETUP([macro call missing positional !ENCLOSE arguments])
+AT_KEYWORDS([ENCLOSE])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !enclose('(',')') !default(x)
+ /!positional !enclose('<','>') !default(y)
+ /!positional !enclose('{','}') !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p (a)<b>{c}.
+!p (a)<b>.
+!p (a).
+!p.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+(a, b, c)
+
+(a, b, z)
+
+(a, y, z)
+
+(x, y, z)
+])
+AT_CLEANUP
+
+AT_SETUP([macro call incomplete positional !ENCLOSE arguments])
+AT_KEYWORDS([ENCLOSE])
+AT_DATA([define.sps], [dnl
+DEFINE !p(!positional !enclose('(',')') !default(x)
+ /!positional !enclose('<','>') !default(y)
+ /!positional !enclose('{','}') !default(z))
+(!1, !2, !3)
+!ENDDEFINE.
+DEBUG EXPAND.
+!p (a)<b>{c}.
+!p (a)<b>{c.
+!p (a)<b>{.
+!p (a)<b>.
+!p (a)<b.
+!p (a)<.
+!p (a).
+!p (a.
+!p (.
+!p.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+(a, b, c)
+
+define.sps:8.12: error: DEBUG EXPAND: Reached end of command expecting "}" in
+argument !3 to macro !p.
+ 8 | !p (a)<b>{c.
+ | ^
+
+(a, b, c)
+
+define.sps:9.11: error: DEBUG EXPAND: Reached end of command expecting "}" in
+argument !3 to macro !p.
+ 9 | !p (a)<b>{.
+ | ^
+
+(a, b, )
+
+(a, b, z)
+
+define.sps:11.9: error: DEBUG EXPAND: Reached end of command expecting ">" in
+argument !2 to macro !p.
+ 11 | !p (a)<b.
+ | ^
+
+(a, b, z)
+
+define.sps:12.8: error: DEBUG EXPAND: Reached end of command expecting ">" in
+argument !2 to macro !p.
+ 12 | !p (a)<.
+ | ^
+
+(a, , z)
+
+(a, y, z)
+
+define.sps:14.6: error: DEBUG EXPAND: Reached end of command expecting ")" in
+argument !1 to macro !p.
+ 14 | !p (a.
+ | ^
+
+(a, y, z)
+
+define.sps:15.5: error: DEBUG EXPAND: Reached end of command expecting ")" in
+argument !1 to macro !p.
+ 15 | !p (.
+ | ^
+
+(, y, z)
+
+(x, y, z)
+])
+AT_CLEANUP
+
+AT_SETUP([keyword macro argument name with ! prefix])
+AT_DATA([define.sps], [dnl
+DEFINE !macro(!x !TOKENS(1).
+])
+AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
+"define.sps:1.15-1.16: error: DEFINE: Keyword macro parameter must be named in definition without ""!"" prefix.
+ 1 | DEFINE !macro@{:@!x !TOKENS(1).
+ | ^~"
+])
+AT_CLEANUP
+
+AT_SETUP([reserved macro keyword argument name])
+AT_DATA([define.sps], [dnl
+DEFINE !macro(if=!TOKENS(1).
+])
+AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
+"define.sps:1.15-1.16: error: DEFINE: Cannot use macro keyword ""if"" as an argument name.
+ 1 | DEFINE !macro@{:@if=!TOKENS(1).
+ | ^~"
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - one !TOKENS(1) keyword argument])
+AT_KEYWORDS([TOKENS])
+AT_DATA([define.sps], [dnl
+DEFINE !k(arg1 = !TOKENS(1)) k(!arg1) !ENDDEFINE.
+DEBUG EXPAND.
+!k arg1=x.
+!k arg1=x y.
+!k.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+k(x)
+
+k(x)
+
+note: unexpanded token "y"
+
+k( )
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - one !TOKENS(1) keyword argument - negative])
+AT_KEYWORDS([TOKENS])
+AT_DATA([define.sps], [dnl
+DEFINE !k(arg1 !TOKENS(1)) k(!arg1) !ENDDEFINE.
+DEBUG EXPAND.
+!k arg1.
+!k arg1=.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:3.8: error: DEBUG EXPAND: Found `.' while expecting `=' reading
+argument !arg1 to macro !k.
+ 3 | !k arg1.
+ | ^
+
+k( )
+
+define.sps:4.9: error: DEBUG EXPAND: Reached end of command expecting 1 more
+token in argument !arg1 to macro !k.
+ 4 | !k arg1=.
+ | ^
+
+k( )
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - !CHAREND keyword arguments])
+AT_KEYWORDS([CHAREND])
+AT_DATA([define.sps], [dnl
+DEFINE !k(arg1 = !CHAREND('/')
+ /arg2 = !CHAREND('/'))
+k(!arg1, !arg2)
+!ENDDEFINE.
+DEBUG EXPAND.
+!k arg1=x/ arg2=y/.
+!k arg1=x/.
+!k arg2=y/.
+!k.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+k(x, y)
+
+k(x, )
+
+k(, y)
+
+k(, )
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - !CHAREND keyword arguments - negative])
+AT_KEYWORDS([CHAREND])
+AT_DATA([define.sps], [dnl
+DEFINE !k(arg1 = !CHAREND('/')
+ /arg2 = !CHAREND('/'))
+k(!arg1, !arg2)
+!ENDDEFINE.
+DEBUG EXPAND.
+!k arg1.
+!k arg1=.
+!k arg1=x.
+!k arg1=x/ arg2=y.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:6.8: error: DEBUG EXPAND: Found `.' while expecting `=' reading
+argument !arg1 to macro !k.
+ 6 | !k arg1.
+ | ^
+
+k(, )
+
+define.sps:7.9: error: DEBUG EXPAND: Reached end of command expecting "/" in
+argument !arg1 to macro !k.
+ 7 | !k arg1=.
+ | ^
+
+k(, )
+
+define.sps:8.10: error: DEBUG EXPAND: Reached end of command expecting "/" in
+argument !arg1 to macro !k.
+ 8 | !k arg1=x.
+ | ^
+
+k(x, )
+
+define.sps:9.18: error: DEBUG EXPAND: Reached end of command expecting "/" in
+argument !arg2 to macro !k.
+ 9 | !k arg1=x/ arg2=y.
+ | ^
+
+k(x, y)
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - !ENCLOSE keyword arguments])
+AT_KEYWORDS([ENCLOSE])
+AT_DATA([define.sps], [dnl
+DEFINE !k(arg1 = !ENCLOSE('(',')')
+ /arg2 = !ENCLOSE('{','}'))
+k(!arg1, !arg2)
+!ENDDEFINE.
+DEBUG EXPAND.
+!k arg1=(x) arg2={y}.
+!k arg1=(x).
+!k arg2={y}.
+!k.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+k(x, y)
+
+k(x, )
+
+k(, y)
+
+k(, )
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion - !ENCLOSE keyword arguments - negative])
+AT_KEYWORDS([ENCLOSE])
+AT_DATA([define.sps], [dnl
+DEFINE !k(arg1 = !ENCLOSE('(',')')
+ /arg2 = !ENCLOSE('{','}'))
+k(!arg1, !arg2)
+!ENDDEFINE.
+DEBUG EXPAND.
+!k arg1.
+!k arg1=.
+!k arg1=x.
+!k arg1=(x.
+!k arg1=(x) arg2.
+!k arg1=(x) arg2=.
+!k arg1=(x) arg2=y.
+!k arg1=(x) arg2=(y.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:6.8: error: DEBUG EXPAND: Found `.' while expecting `=' reading
+argument !arg1 to macro !k.
+ 6 | !k arg1.
+ | ^
+
+k(, )
+
+define.sps:7.9: error: DEBUG EXPAND: Found `.' while expecting `@{:@' reading
+argument !arg1 to macro !k.
+ 7 | !k arg1=.
+ | ^
+
+k(, )
+
+define.sps:8.9: error: DEBUG EXPAND: Found `x' while expecting `@{:@' reading
+argument !arg1 to macro !k.
+ 8 | !k arg1=x.
+ | ^
+
+k(, )
+
+note: unexpanded token "x"
+
+define.sps:9.11: error: DEBUG EXPAND: Reached end of command expecting "@:}@" in
+argument !arg1 to macro !k.
+ 9 | !k arg1=@{:@x.
+ | ^
+
+k(x, )
+
+define.sps:10.17: error: DEBUG EXPAND: Found `.' while expecting `=' reading
+argument !arg2 to macro !k.
+ 10 | !k arg1=(x) arg2.
+ | ^
+
+k(x, )
+
+define.sps:11.18: error: DEBUG EXPAND: Found `.' while expecting `{' reading
+argument !arg2 to macro !k.
+ 11 | !k arg1=(x) arg2=.
+ | ^
+
+k(x, )
+
+define.sps:12.18: error: DEBUG EXPAND: Found `y' while expecting `{' reading
+argument !arg2 to macro !k.
+ 12 | !k arg1=(x) arg2=y.
+ | ^
+
+k(x, )
+
+note: unexpanded token "y"
+
+define.sps:13.18: error: DEBUG EXPAND: Found `@{:@' while expecting `{' reading
+argument !arg2 to macro !k.
+ 13 | !k arg1=(x) arg2=@{:@y.
+ | ^
+
+k(x, )
+
+note: unexpanded token "@{:@"
+
+note: unexpanded token "y"
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !BLANKS in the manual.
+AT_SETUP([macro expansion - !BLANKS])
+AT_KEYWORDS([BLANKS])
+AT_DATA([define.sps], [dnl
+DEFINE !b()
+!BLANKS(0).
+!QUOTE(!BLANKS(0)).
+!BLANKS(1).
+!QUOTE(!BLANKS(1)).
+!BLANKS(2).
+!QUOTE(!BLANKS(2)).
+!BLANKS(5).
+!QUOTE(!BLANKS(5)).
+!ENDDEFINE.
+DEBUG EXPAND.
+!b.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+.
+''.
+.
+' '.
+.
+' '.
+.
+' '.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !CONCAT in the manual.
+AT_SETUP([macro expansion - !CONCAT])
+AT_KEYWORDS([CONCAT])
+AT_DATA([define.sps], [dnl
+DEFINE !c()
+!CONCAT(x, y).
+!CONCAT('x', 'y').
+!CONCAT(12, 34).
+!CONCAT(!NULL, 123).
+!CONCAT(x, 0).
+!CONCAT(x, 0, y).
+!CONCAT(0, x).
+!CONCAT(0, x, y).
+!ENDDEFINE.
+DEBUG EXPAND.
+!c
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+xy.
+xy.
+1234.
+123.
+x0.
+x0y.
+0 x.
+0 xy.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !EVAL in the manual.
+AT_SETUP([macro expansion - !EVAL])
+AT_KEYWORDS([EVAL])
+AT_DATA([define.sps], [dnl
+DEFINE !vars() a b c !ENDDEFINE.
+
+DEFINE !e()
+!vars.
+!QUOTE(!vars).
+!EVAL(!vars).
+!QUOTE(!EVAL(!vars)).
+!ENDDEFINE
+
+DEFINE !e2(!positional !enclose('(',')'))
+!1.
+!QUOTE(!1).
+!EVAL(!1).
+!QUOTE(!EVAL(!1)).
+!ENDDEFINE.
+DEBUG EXPAND.
+!e.
+!e2(!vars).
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+a b c.
+'!vars'.
+a b c.
+'a b c'.
+
+a b c.
+'!vars'.
+a b c.
+'a b c'.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !HEAD in the manual.
+AT_SETUP([macro expansion - !HEAD])
+AT_KEYWORDS([HEAD])
+AT_DATA([define.sps], [dnl
+DEFINE !h()
+!HEAD('a b c').
+!HEAD('a').
+!HEAD(!NULL).
+!HEAD('').
+!ENDDEFINE.
+DEBUG EXPAND.
+!h.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+a.
+a.
+.
+.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !TAIL in the manual.
+AT_SETUP([macro expansion - !TAIL])
+AT_KEYWORDS([TAIL])
+AT_DATA([define.sps], [dnl
+DEFINE !t()
+!TAIL('a b c').
+!TAIL('a').
+!TAIL(!NULL).
+!TAIL('').
+!ENDDEFINE.
+DEBUG EXPAND.
+!t.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+b c.
+.
+.
+.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !INDEX in the manual.
+AT_SETUP([macro expansion - !INDEX])
+AT_KEYWORDS([INDEX])
+AT_DATA([define.sps], [dnl
+DEFINE !i()
+!INDEX(banana, an).
+!INDEX(banana, nan).
+!INDEX(banana, apple).
+!INDEX("banana", nan).
+!INDEX("banana", "nan").
+!INDEX(!UNQUOTE("banana"), !UNQUOTE("nan")).
+!ENDDEFINE.
+DEBUG EXPAND.
+!i.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+2.
+3.
+0.
+4.
+0.
+3.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !LENGTH in the manual.
+AT_SETUP([macro expansion - !LENGTH])
+AT_KEYWORDS([LENGTH])
+AT_DATA([define.sps], [dnl
+DEFINE !l()
+!LENGTH(123).
+!LENGTH(123.00).
+!LENGTH( 123 ).
+!LENGTH("123").
+!LENGTH(xyzzy).
+!LENGTH("xyzzy").
+!LENGTH("xy""zzy").
+!LENGTH(!UNQUOTE("xyzzy")).
+!LENGTH(!UNQUOTE("xy""zzy")).
+!LENGTH(!NULL).
+!ENDDEFINE.
+DEFINE !la(!positional !enclose('(',')'))
+!LENGTH(!1).
+!ENDDEFINE.
+DEBUG EXPAND.
+!l.
+!la(a b c).
+!la().
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+3.
+6.
+3.
+5.
+5.
+7.
+9.
+5.
+6.
+0.
+
+5.
+
+0.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !NULL in the manual.
+AT_SETUP([macro expansion - !NULL])
+AT_KEYWORDS([NULL])
+AT_DATA([define.sps], [dnl
+DEFINE !n()
+!NULL.
+!QUOTE(!NULL).
+!ENDDEFINE.
+DEBUG EXPAND.
+!n.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+.
+''.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !QUOTE and !UNQUOTE in the manual.
+AT_SETUP([macro expansion - !QUOTE and !UNQUOTE])
+AT_KEYWORDS([QUOTE UNQUOTE])
+AT_DATA([define.sps], [dnl
+DEFINE !q(!POS !CMDEND)
+!QUOTE(123.0).
+!QUOTE( 123 ).
+!QUOTE('a b c').
+!QUOTE("a b c").
+!QUOTE(!1).
+
+!UNQUOTE(123.0).
+!UNQUOTE( 123 ).
+!UNQUOTE('a b c').
+!UNQUOTE("a b c").
+!UNQUOTE(!1).
+
+!QUOTE(!UNQUOTE(123.0)).
+!QUOTE(!UNQUOTE( 123 )).
+!QUOTE(!UNQUOTE('a b c')).
+!QUOTE(!UNQUOTE("a b c")).
+!QUOTE(!UNQUOTE(!1)).
+!ENDDEFINE.
+DEBUG EXPAND.
+!q a 'b' c.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+'123.0'.
+'123'.
+'a b c'.
+"a b c".
+'a ''b'' c'.
+
+123.0.
+123.
+a b c.
+a b c.
+a 'b' c.
+
+'123.0'.
+'123'.
+'a b c'.
+'a b c'.
+'a ''b'' c'.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !SUBSTR in the manual.
+AT_SETUP([macro expansion - !SUBSTR])
+AT_KEYWORDS([SUBSTR])
+AT_DATA([define.sps], [dnl
+DEFINE !s()
+!SUBSTR(banana, 3).
+!SUBSTR(banana, 3, 3).
+!SUBSTR("banana", 1, 3).
+!SUBSTR(!UNQUOTE("banana"), 3).
+!SUBSTR("banana", 3, 3).
+!SUBSTR(banana, 3, 0).
+!SUBSTR(banana, 3, 10).
+!SUBSTR(banana, 10, 3).
+!ENDDEFINE.
+DEBUG EXPAND.
+!s.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1-10: At `"ba' in the expansion of `!s',dnl "
+
+define.sps:12.1-12.2: error: DEBUG EXPAND: Unterminated string constant.
+ 12 | !s.
+ | ^~
+
+nana.
+nan.
+.
+nana.
+ana.
+.
+nana.
+.
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with the examples for !UPCASE in the manual.
+AT_SETUP([macro expansion - !UPCASE])
+AT_KEYWORDS([UPCASE])
+AT_DATA([define.sps], [dnl
+DEFINE !u()
+!UPCASE(freckle).
+!UPCASE('freckle').
+!UPCASE('a b c').
+!UPCASE('A B C').
+!ENDDEFINE.
+DEBUG EXPAND.
+!u.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+FRECKLE.
+FRECKLE.
+A B C.
+A B C.
+])
+AT_CLEANUP
+
+dnl !* is implemented separately inside and outside function arguments
+dnl so this test makes sure to include both.
+AT_SETUP([macro expansion - !*])
+AT_DATA([define.sps], [dnl
+DEFINE !m(!POSITIONAL !TOKENS(1)
+ /!POSITIONAL !TOKENS(1))
+!*/
+!LENGTH(!*)/
+!SUBSTR(!*, 3)/
+!QUOTE(!*).
+!ENDDEFINE.
+DEBUG EXPAND.
+!m 123 b
+!m 2 3
+!m '' 'b'.
+])
+AT_CAPTURE_FILE([define.sps])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+123 b / 5 / 3 b / '123 b'.
+
+2 3 / 3 / 3 / '2 3'.
+
+'' 'b' / 6 / 'b' / ''''' ''b'''.
+])
+AT_CLEANUP
+
+AT_SETUP([macro maximum nesting level (MNEST)])
+AT_KEYWORDS([MNEST])
+AT_DATA([define.sps], [dnl
+DEFINE !macro()
+!macro
+!ENDDEFINE.
+!macro.
+])
+AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
+"define.sps:1-3: In the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:1-3: inside the expansion of `!macro',
+define.sps:4.1-4.6: error: DEFINE: Maximum nesting level 50 exceeded. (Use SET MNEST to change the limit.)
+ 4 | !macro.
+ | ^~~~~~"
+
+"define.sps:4.1-4.6: error: In syntax expanded from `!macro': Syntax error expecting command name.
+ 4 | !macro.
+ | ^~~~~~"
+])
+AT_CLEANUP
+
+AT_SETUP([macro !IF condition])
+AT_KEYWORDS([if])
+for operators in \
+ '!eq !ne !lt !gt !le !ge' \
+ ' = <> < > <= >='
+do
+ set $operators
+ AS_BOX([$operators])
+ cat > define.sps <<EOF
+DEFINE !test(!positional !tokens(1))
+!if (!1 $1 1) !then true !else false !ifend
+!if (!1 $2 1) !then true !else false !ifend
+!if (!1 $3 1) !then true !else false !ifend
+!if (!1 $4 1) !then true !else false !ifend
+!if (!1 $5 1) !then true !else false !ifend
+!if (!1 $6 1) !then true !else false !ifend.
+!ENDDEFINE.
+DEBUG EXPAND.
+!test 0
+!test 1
+!test 2
+!test '1'
+!test 1.0
+EOF
+ AT_CAPTURE_FILE([define.sps])
+ AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+false true true false true false.
+
+true false false false true true.
+
+false true false true false true.
+
+true false false false true true.
+
+false true false true false true.
+])
+done
+AT_CLEANUP
+
+AT_SETUP([macro !IF condition -- case sensitivity])
+AT_KEYWORDS([if])
+for operators in \
+ '!eq !ne !lt !gt !le !ge' \
+ ' = <> < > <= >='
+do
+ set $operators
+ AS_BOX([$operators])
+ cat > define.sps <<EOF
+DEFINE !test(!positional !tokens(1))
+!if (!1 $1 a) !then true !else false !ifend
+!if (!1 $1 A) !then true !else false !ifend
+!if (!1 $2 a) !then true !else false !ifend
+!if (!1 $2 A) !then true !else false !ifend
+!if (!1 $3 a) !then true !else false !ifend
+!if (!1 $3 A) !then true !else false !ifend
+!if (!1 $4 a) !then true !else false !ifend
+!if (!1 $4 A) !then true !else false !ifend
+!if (!1 $5 a) !then true !else false !ifend
+!if (!1 $5 A) !then true !else false !ifend
+!if (!1 $6 a) !then true !else false !ifend
+!if (!1 $6 A) !then true !else false !ifend
+!if (!1 $1 !null) !then true !else false !ifend
+!if (!1 $2 !null) !then true !else false !ifend.
+!ENDDEFINE.
+DEBUG EXPAND.
+!test a
+!test A
+!test b
+!test B
+EOF
+ AT_CAPTURE_FILE([define.sps])
+ AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+true false false true false false false true true false true true false true.
+
+false true true false true false false false true true false true false true.
+
+false false true true false false true true false false true true false true.
+
+false false true true true false false true true false false true false true.
+])
+done
+AT_CLEANUP
+
+AT_SETUP([macro !IF condition -- logical operators])
+AT_KEYWORDS([if])
+for operators in \
+ '!and !or !not' \
+ ' & | ~'
+do
+ set $operators
+ AS_BOX([$operators])
+ cat > define.sps <<EOF
+DEFINE !test_binary(!positional !tokens(1)/!positional !tokens(1))
+!if !1 $1 !2 !then true !else false !ifend
+!if !1 $2 !2 !then true !else false !ifend.
+!ENDDEFINE.
+
+DEFINE !test_unary(!positional !tokens(1))
+!if $3 !1 !then true !else false !ifend.
+!ENDDEFINE.
+
+* These are:
+ ((not A) and B) or C
+ not (A and B) or C
+ not A and (B or C)
+DEFINE !test_prec(!pos !tokens(1)/!pos !tokens(1)/!pos !tokens(1))
+!if $3 !1 $1 !2 $2 !3 !then true !else false !ifend
+!if $3 (!1 $1 !2) $2 !3 !then true !else false !ifend
+!if $3 !1 $1 (!2 $2 !3) !then true !else false !ifend
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!test_binary 0 0
+!test_binary 0 1
+!test_binary 1 0
+!test_binary 1 1
+!test_unary 0
+!test_unary 1
+!test_prec 0 0 0 !test_prec 0 0 1 !test_prec 0 1 0 !test_prec 0 1 1.
+!test_prec 1 0 0 !test_prec 1 0 1 !test_prec 1 1 0 !test_prec 1 1 1.
+EOF
+ AT_CAPTURE_FILE([define.sps])
+ AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+false false.
+
+false true.
+
+false true.
+
+true true.
+
+true.
+
+false.
+
+false true false
+true true true
+true true true
+true true true
+
+false true false
+true true false
+false false false
+true true false
+])
+done
+AT_CLEANUP
+
+AT_SETUP([macro !LET])
+AT_KEYWORDS([let])
+AT_DATA([define.sps], [dnl
+DEFINE !macro(!POS !CMDEND)
+!LET !v1 = !CONCAT('x',!1,'y')
+!LET !v2 = !QUOTE(!v1)
+!LET !v3 = (!LENGTH(!1) = 1)
+!LET !v4 = (!SUBSTR(!1, 3) = !NULL)
+v1=!v1.
+v2=!v2.
+v3=!v3.
+v4=!v4.
+!ENDDEFINE.
+DEBUG EXPAND.
+!macro 0.
+!macro.
+!macro xyzzy.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+v1 = x0y.
+v2 = x0y.
+v3 = 1.
+v4 = 1.
+
+v1 = xy.
+v2 = xy.
+v3 = 0.
+v4 = 1.
+
+v1 = xxyzzyy.
+v2 = xxyzzyy.
+v3 = 0.
+v4 = 0.
+])
+AT_CLEANUP
+
+AT_SETUP([macro indexed !DO])
+AT_KEYWORDS([index do])
+AT_DATA([define.sps], [dnl
+DEFINE !title(!POS !TOKENS(1)) !1. !ENDDEFINE.
+
+DEFINE !for(!POS !TOKENS(1) / !POS !TOKENS(1))
+!DO !var = !1 !TO !2 !var !DOEND.
+!ENDDEFINE.
+
+DEFINE !forby(!POS !TOKENS(1) / !POS !TOKENS(1) / !POS !TOKENS(1))
+!DO !var = !1 !TO !2 !BY !3 !var !DOEND.
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!title "increasing".
+!for 1 5.
+!forby 1 5 1.
+!forby 1 5 2.
+!forby 1 5 2.5.
+!forby 1 5 -1.
+
+!title "decreasing".
+!for 5 1.
+!forby 5 1 1.
+!forby 5 1 -1.
+!forby 5 1 -2.
+!forby 5 1 -3.
+
+!title "non-integer".
+!for 1.5 3.5.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+"increasing".
+
+1 2 3 4 5.
+
+1 2 3 4 5.
+
+1 3 5.
+
+1 3.5.
+
+.
+
+"decreasing".
+
+.
+
+.
+
+5 4 3 2 1.
+
+5 3 1.
+
+5 2.
+
+"non-integer".
+
+1.5 2.5 3.5.
+])
+AT_CLEANUP
+
+AT_SETUP([macro !DO invalid variable names])
+AT_KEYWORDS([index do])
+AT_DATA([define.sps], [dnl
+DEFINE !for(x=!TOKENS(1) / y=!TOKENS(1))
+!DO !x = !x !TO !y !var !DOEND.
+!ENDDEFINE.
+
+DEFINE !for2(x=!TOKENS(1) / y=!TOKENS(1))
+!DO !noexpand = !x !TO !y !var !DOEND.
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!for x=1 y=5.
+!for2 x=1 y=5.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1-3: At `!x' in the expansion of `!for',
+define.sps:10.1-10.12: error: DEBUG EXPAND: Cannot use argument name or macro
+keyword as !DO variable.
+ 10 | !for x=1 y=5.
+ | ^~~~~~~~~~~~
+
+!DO 1 = 1 !TO 5 !var !DOEND.
+
+define.sps:5-7: At `!noexpand' in the expansion of `!for2',
+define.sps:11.1-11.13: error: DEBUG EXPAND: Cannot use argument name or macro
+keyword as !DO variable.
+ 11 | !for2 x=1 y=5.
+ | ^~~~~~~~~~~~~
+
+!DO !noexpand = 1 !TO 5 !var !DOEND.
+])
+AT_CLEANUP
+
+AT_SETUP([macro indexed !DO reaches MITERATE])
+AT_KEYWORDS([index do])
+AT_DATA([define.sps], [dnl
+DEFINE !title(!POS !TOKENS(1)) !1. !ENDDEFINE.
+
+DEFINE !for(!POS !TOKENS(1) / !POS !TOKENS(1))
+!DO !var = !1 !TO !2 !var !DOEND.
+!ENDDEFINE.
+
+DEFINE !forby(!POS !TOKENS(1) / !POS !TOKENS(1) / !POS !TOKENS(1))
+!DO !var = !1 !TO !2 !BY !3 !var !DOEND.
+!ENDDEFINE.
+
+SET MITERATE=3.
+DEBUG EXPAND.
+!title "increasing".
+!for 1 5.
+!forby 1 5 1.
+!forby 1 5 2.
+!forby 1 5 2.5.
+!forby 1 5 -1.
+
+!title "decreasing".
+!for 5 1.
+!forby 5 1 1.
+!forby 5 1 -1.
+!forby 5 1 -2.
+!forby 5 1 -3.
+
+!title "non-integer".
+!for 1.5 3.5.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+"increasing".
+
+In the expansion of `!DO',
+define.sps:3-5: inside the expansion of `!for',
+define.sps:14.1-14.8: error: DEBUG EXPAND: Numerical !DO loop exceeded maximum
+number of iterations 3. (Use SET MITERATE to change the limit.)
+ 14 | !for 1 5.
+ | ^~~~~~~~
+
+1 2 3 4.
+
+In the expansion of `!DO',
+define.sps:7-9: inside the expansion of `!forby',
+define.sps:15.1-15.12: error: DEBUG EXPAND: Numerical !DO loop exceeded maximum
+number of iterations 3. (Use SET MITERATE to change the limit.)
+ 15 | !forby 1 5 1.
+ | ^~~~~~~~~~~~
+
+1 2 3 4.
+
+1 3 5.
+
+1 3.5.
+
+.
+
+"decreasing".
+
+.
+
+.
+
+In the expansion of `!DO',
+define.sps:7-9: inside the expansion of `!forby',
+define.sps:23.1-23.13: error: DEBUG EXPAND: Numerical !DO loop exceeded maximum
+number of iterations 3. (Use SET MITERATE to change the limit.)
+ 23 | !forby 5 1 -1.
+ | ^~~~~~~~~~~~~
+
+5 4 3 2.
+
+5 3 1.
+
+5 2.
+
+"non-integer".
+
+1.5 2.5 3.5.
+])
+AT_CLEANUP
+
+AT_SETUP([!BREAK with macro indexed !DO])
+AT_KEYWORDS([index do break])
+AT_DATA([define.sps], [dnl
+DEFINE !title(!POS !TOKENS(1)) !1. !ENDDEFINE.
+
+DEFINE !for(!POS !TOKENS(1) / !POS !TOKENS(1) / !POS !TOKENS(1))
+!DO !var = !1 !TO !2
+ !var
+ !IF 1 !THEN
+ !IF !var = !3 !THEN
+ x
+ !BREAK
+ y
+ !IFEND
+ ,
+ !IFEND
+!DOEND.
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!for 1 5 4.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+1, 2, 3, 4 x.
+])
+AT_CLEANUP
+
+AT_SETUP([macro list !DO])
+AT_KEYWORDS([index do])
+AT_DATA([define.sps], [dnl
+DEFINE !for(!POS !CMDEND)
+(!DO !i !IN (!1) (!i) !DOEND).
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!for a b c.
+!for 'foo bar baz quux'.
+!for.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+( (a) (b) (c) ).
+
+( (foo) (bar) (baz) (quux) ).
+
+( ).
+])
+AT_CLEANUP
+
+AT_SETUP([macro list !DO reaches MITERATE])
+AT_KEYWORDS([index do])
+AT_DATA([define.sps], [dnl
+DEFINE !for(!POS !CMDEND)
+(!DO !i !IN (!1) (!i) !DOEND).
+!ENDDEFINE.
+
+SET MITERATE=2.
+DEBUG EXPAND.
+!for a b c.
+!for 'foo bar baz quux'.
+!for.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+In the expansion of `!DO',
+define.sps:1-3: inside the expansion of `!for',
+define.sps:7.1-7.10: error: DEBUG EXPAND: !DO loop over list exceeded maximum
+number of iterations 2. (Use SET MITERATE to change the limit.)
+ 7 | !for a b c.
+ | ^~~~~~~~~~
+
+( (a) (b) ).
+
+In the expansion of `!DO',
+define.sps:1-3: inside the expansion of `!for',
+define.sps:8.1-8.23: error: DEBUG EXPAND: !DO loop over list exceeded maximum
+number of iterations 2. (Use SET MITERATE to change the limit.)
+ 8 | !for 'foo bar baz quux'.
+ | ^~~~~~~~~~~~~~~~~~~~~~~
+
+( (foo) (bar) ).
+
+( ).
+])
+AT_CLEANUP
+
+AT_SETUP([!BREAK with macro list !DO])
+AT_KEYWORDS([index break do])
+AT_DATA([define.sps], [dnl
+DEFINE !for(!POS !TOKENS(1) / !POS !CMDEND)
+(!DO !i !IN (!2)
+ (!i)
+ !IF 1 !THEN
+ !IF !i = !1 !THEN
+ x
+ !BREAK
+ y
+ !IFEND
+ ,
+ !IFEND
+!DOEND).
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!for d a b c.
+!for baz 'foo bar baz quux'.
+!for e.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+( (a), (b), (c), ).
+
+( (foo), (bar), (baz)x).
+
+( ).
+])
+AT_CLEANUP
+
+AT_SETUP([macro !LET])
+AT_DATA([define.sps], [dnl
+DEFINE !macro(!pos !enclose('(',')'))
+!LET !x=!1
+!LET !y=!QUOTE(!1)
+!LET !z=(!y="abc")
+!y !z
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!macro(1+2).
+!macro(abc).
+])
+AT_CHECK([pspp --testing-mode define.sps -O format=csv], [0], [dnl
+1 + 2 0
+
+abc 1
+])
+AT_CLEANUP
+
+AT_SETUP([macro !LET invalid variable names])
+AT_DATA([define.sps], [dnl
+DEFINE !macro(x=!tokens(1))
+!LET !x=!x
+!ENDDEFINE.
+
+DEFINE !macro2()
+!LET !do=x
+!ENDDEFINE.
+
+DEBUG EXPAND.
+!macro x=1.
+!macro2.
+])
+AT_CHECK([pspp --testing-mode define.sps -O format=csv], [1], [dnl
+"define.sps:1-3: At `!x' in the expansion of `!macro',
+define.sps:10.1-10.10: error: DEBUG EXPAND: Cannot use argument name or macro keyword ""!x"" as !LET variable.
+ 10 | !macro x=1.
+ | ^~~~~~~~~~"
+
+!LET 1 = 1
+
+"define.sps:5-7: At `!do' in the expansion of `!macro2',
+define.sps:11.1-11.7: error: DEBUG EXPAND: Cannot use argument name or macro keyword ""!do"" as !LET variable.
+ 11 | !macro2.
+ | ^~~~~~~"
+
+"define.sps:5-7: At `=' in the expansion of `!macro2',
+define.sps:11.1-11.7: error: DEBUG EXPAND: Expected macro variable name following !DO.
+ 11 | !macro2.
+ | ^~~~~~~"
+
+!LET !do = x
+])
+AT_CLEANUP
+
+AT_SETUP([BEGIN DATA inside a macro])
+AT_DATA([define.sps], [dnl
+DEFINE !macro()
+DATA LIST NOTABLE /x 1.
+BEGIN DATA
+1
+2
+3
+END DATA.
+LIST.
+!ENDDEFINE.
+
+!macro
+])
+AT_CHECK([pspp define.sps -O format=csv], [0], [dnl
+Table: Data List
+x
+1
+2
+3
+])
+AT_CLEANUP
+
+AT_SETUP([TITLE and SUBTITLE with macros])
+AT_KEYWORDS([macro])
+for command in TITLE SUBTITLE; do
+ cat >title.sps <<EOF
+DEFINE !paste(!POS !TOKENS(1) / !POS !TOKENS(1))
+!CONCAT(!1,!2)
+!ENDDEFINE.
+$command prefix !paste foo bar suffix.
+SHOW $command.
+EOF
+ cat >expout <<EOF
+Table: Settings
+$command,prefix foobar suffix
+EOF
+ AT_CHECK([pspp -O format=csv title.sps], [0], [expout])
+done
+AT_CLEANUP
+
+AT_SETUP([error message within macro expansion])
+AT_DATA([define.sps], [dnl
+DEFINE !vars(!POS !TOKENS(1)) a b C !ENDDEFINE.
+DATA LIST NOTABLE /a b 1-2.
+COMPUTE x = !vars x.
+])
+AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
+"define.sps:3.13-3.19: error: COMPUTE: In syntax expanded from `!vars x': Syntax error expecting end of command.
+ 3 | COMPUTE x = !vars x.
+ | ^~~~~~~"
+])
+AT_CLEANUP
+
+dnl A macro with keyword arguments needs a token of lookahead
+dnl to find out whether another keyword is present. Test that
+dnl this special case works OK.
+AT_SETUP([macro calls in each others' lookahead])
+AT_DATA([define.sps], [dnl
+DEFINE !k(x=!DEFAULT(0) !TOKENS(1)/y=!DEFAULT(0) !TOKENS(1))
+(x=!x)(y=!y)
+!ENDDEFINE.
+DEBUG EXPAND.
+!k
+!k x=1
+!k y=2
+!k y=2 x=1
+!k x=1 y=2.
+])
+AT_CHECK([pspp -O format=csv define.sps --testing-mode], [0], [dnl
+(x = 0) (y = 0)
+
+(x = 1) (y = 0)
+
+(x = 0) (y = 2)
+(x = 1) (y = 2)
+
+(x = 1) (y = 2)
+])
+AT_CLEANUP
+
+AT_SETUP([bad token in macro body])
+AT_DATA([define.sps], [dnl
+DEFINE !x()
+x'123'
+!ENDDEFINE.
+])
+AT_CHECK([pspp define.sps], [1], [dnl
+define.sps:2.1-2.6: error: DEFINE: String of hex digits has 3 characters, which
+is not a multiple of 2.
+ 2 | x'123'
+ | ^~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([macro name overlaps with macro function name])
+dnl !len is short for macro function !LENGTH. PSPP used to
+dnl reject the following with "`(' expected following !LENGTH".
+dnl Now PSPP only consider macro functions when the name is
+dnl followed by '('.
+AT_DATA([define.sps], [dnl
+DEFINE !len() 5 !ENDDEFINE.
+DEFINE !x() !eval(!len) !ENDDEFINE.
+DEBUG EXPAND.
+!x
+])
+AT_CHECK([pspp -O format=csv define.sps --testing-mode], [0], [dnl
+5
+])
+AT_CLEANUP
+
+AT_SETUP([generic macro function syntax errors])
+AT_DATA([define.sps], [dnl
+
+
+DEFINE !c() !SUBSTR(1x) !ENDDEFINE.
+DEFINE !d() !SUBSTR(1 !ENDDEFINE.
+DEFINE !narg_blanks() !BLANKS() !ENDDEFINE.
+DEFINE !narg_concat() !CONCAT() !ENDDEFINE.
+DEFINE !narg_eval() !EVAL() !ENDDEFINE.
+DEFINE !narg_head() !HEAD() !ENDDEFINE.
+DEFINE !narg_index() !INDEX() !ENDDEFINE.
+DEFINE !narg_length() !LENGTH() !ENDDEFINE.
+DEFINE !narg_null() !NULL() !ENDDEFINE.
+DEFINE !narg_quote() !QUOTE() !ENDDEFINE.
+DEFINE !narg_substr() !SUBSTR() !ENDDEFINE.
+DEFINE !narg_tail() !TAIL() !ENDDEFINE.
+DEFINE !narg_unquote() !UNQUOTE() !ENDDEFINE.
+DEFINE !narg_upcase() !UPCASE() !ENDDEFINE.
+dnl )
+DEBUG EXPAND.
+
+
+!c.
+!d.
+!narg_blanks.
+!narg_concat.
+!narg_eval.
+!narg_head.
+!narg_index.
+!narg_length.
+!narg_null.
+!narg_quote.
+!narg_substr.
+!narg_tail.
+!narg_unquote.
+!narg_upcase.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:3: At `x' in the expansion of `!c',
+define.sps:20.1-20.2: error: DEBUG EXPAND: `,' or `@:}@' expected in call to macro
+function !SUBSTR.
+ 20 | !c.
+ | ^~
+
+!SUBSTR(1 x)
+
+define.sps:4: In the expansion of `!d',
+define.sps:21.1-21.2: error: DEBUG EXPAND: Missing `@:}@' in call to macro
+function !SUBSTR.
+ 21 | !d.
+ | ^~
+
+!SUBSTR@{:@1
+
+define.sps:5: In the expansion of `!narg_blanks',
+define.sps:22.1-22.12: error: DEBUG EXPAND: Macro function !BLANKS takes one
+argument (not 0).
+ 22 | !narg_blanks.
+ | ^~~~~~~~~~~~
+
+!BLANKS( )
+
+define.sps:6: In the expansion of `!narg_concat',
+define.sps:23.1-23.12: error: DEBUG EXPAND: Macro function !CONCAT needs at
+least one argument.
+ 23 | !narg_concat.
+ | ^~~~~~~~~~~~
+
+!CONCAT( )
+
+define.sps:7: In the expansion of `!narg_eval',
+define.sps:24.1-24.10: error: DEBUG EXPAND: Macro function !EVAL takes one
+argument (not 0).
+ 24 | !narg_eval.
+ | ^~~~~~~~~~
+
+!EVAL( )
+
+define.sps:8: In the expansion of `!narg_head',
+define.sps:25.1-25.10: error: DEBUG EXPAND: Macro function !HEAD takes one
+argument (not 0).
+ 25 | !narg_head.
+ | ^~~~~~~~~~
+
+!HEAD( )
+
+define.sps:9: In the expansion of `!narg_index',
+define.sps:26.1-26.11: error: DEBUG EXPAND: Macro function !INDEX takes two
+arguments (not 0).
+ 26 | !narg_index.
+ | ^~~~~~~~~~~
+
+!INDEX( )
+
+define.sps:10: In the expansion of `!narg_length',
+define.sps:27.1-27.12: error: DEBUG EXPAND: Macro function !LENGTH takes one
+argument (not 0).
+ 27 | !narg_length.
+ | ^~~~~~~~~~~~
+
+!LENGTH( )
+
+( )
+
+define.sps:12: In the expansion of `!narg_quote',
+define.sps:29.1-29.11: error: DEBUG EXPAND: Macro function !QUOTE takes one
+argument (not 0).
+ 29 | !narg_quote.
+ | ^~~~~~~~~~~
+
+!QUOTE( )
+
+define.sps:13: In the expansion of `!narg_substr',
+define.sps:30.1-30.12: error: DEBUG EXPAND: Macro function !SUBSTR takes two or
+three arguments (not 0).
+ 30 | !narg_substr.
+ | ^~~~~~~~~~~~
+
+!SUBSTR( )
+
+define.sps:14: In the expansion of `!narg_tail',
+define.sps:31.1-31.10: error: DEBUG EXPAND: Macro function !TAIL takes one
+argument (not 0).
+ 31 | !narg_tail.
+ | ^~~~~~~~~~
+
+!TAIL( )
+
+define.sps:15: In the expansion of `!narg_unquote',
+define.sps:32.1-32.13: error: DEBUG EXPAND: Macro function !UNQUOTE takes one
+argument (not 0).
+ 32 | !narg_unquote.
+ | ^~~~~~~~~~~~~
+
+!UNQUOTE( )
+
+define.sps:16: In the expansion of `!narg_upcase',
+define.sps:33.1-33.12: error: DEBUG EXPAND: Macro function !UPCASE takes one
+argument (not 0).
+ 33 | !narg_upcase.
+ | ^~~~~~~~~~~~
+
+!UPCASE( )
+])
+AT_CLEANUP
+
+AT_SETUP([specific macro function syntax errors])
+AT_DATA([define.sps], [dnl
+DEFINE !a() !BLANKS(x). !ENDDEFINE.
+DEFINE !b() !SUBSTR(x, y). !ENDDEFINE.
+DEFINE !c() !SUBSTR(x, 1, z). !ENDDEFINE.
+DEBUG EXPAND.
+!a.
+!b.
+!c.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1: In the expansion of `!a',
+define.sps:5.1-5.2: error: DEBUG EXPAND: Argument to !BLANKS must be non-
+negative integer (not "x").
+ 5 | !a.
+ | ^~
+
+!BLANKS(x).
+
+define.sps:2: In the expansion of `!b',
+define.sps:6.1-6.2: error: DEBUG EXPAND: Second argument of !SUBSTR must be
+positive integer (not "y").
+ 6 | !b.
+ | ^~
+
+!SUBSTR(x, y).
+
+define.sps:3: In the expansion of `!c',
+define.sps:7.1-7.2: error: DEBUG EXPAND: Third argument of !SUBSTR must be non-
+negative integer (not "z").
+ 7 | !c.
+ | ^~
+
+!SUBSTR(x, 1, z).
+])
+AT_CLEANUP
+
+AT_SETUP([macro expression errors])
+AT_DATA([define.sps], [dnl
+DEFINE !a() !LET !x = (1. !ENDDEFINE dnl )
+
+DEFINE !b() !DO !x = x. !ENDDEFINE.
+DEFINE !c() !LET !x = (). !ENDDEFINE.
+DEBUG EXPAND.
+!a.
+!b.
+!c.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1-2: At `.' in the expansion of `!a',
+define.sps:5.1-5.2: error: DEBUG EXPAND: Expecting ')' in macro expression.
+ 5 | !a.
+ | ^~
+
+!LET !x = (1.
+
+At `x' in the expansion of `!DO',
+define.sps:2: inside the expansion of `!b',
+define.sps:6.1-6.2: error: DEBUG EXPAND: Macro expression must evaluate to a
+number (not "x").
+ 6 | !b.
+ | ^~
+
+!DO !x = x.
+
+define.sps:3: At `)' in the expansion of `!c',
+define.sps:7.1-7.2: error: DEBUG EXPAND: Expecting literal or function
+invocation in macro expression.
+ 7 | !c.
+ | ^~
+
+!LET !x = ( ).
+])
+AT_CLEANUP
+
+AT_SETUP([macro !IF errors])
+AT_KEYWORDS([IF])
+AT_DATA([define.sps], [dnl
+DEFINE !a() !IF 1 !ENDDEFINE.
+DEFINE !b() !IF 1 !THEN !ENDDEFINE.
+DEFINE !c() !IF 1 !THEN !ELSE !ENDDEFINE.
+DEBUG EXPAND.
+!a.
+!b.
+!c.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1: In the expansion of `!a',
+define.sps:5.1-5.2: error: DEBUG EXPAND: !THEN expected in macro !IF construct.
+ 5 | !a.
+ | ^~
+
+!IF 1
+
+define.sps:2: In the expansion of `!b',
+define.sps:6.1-6.2: error: DEBUG EXPAND: !ELSE or !IFEND expected in macro !IF
+construct.
+ 6 | !b.
+ | ^~
+
+!IF 1 !THEN
+
+define.sps:3: In the expansion of `!c',
+define.sps:7.1-7.2: error: DEBUG EXPAND: !IFEND expected in macro !IF
+construct.
+ 7 | !c.
+ | ^~
+
+!IF 1 !THEN !ELSE
+])
+AT_CLEANUP
+
+AT_SETUP([macro !LET errors])
+AT_KEYWORDS([LET])
+AT_DATA([define.sps], [dnl
+DEFINE !a() !LET !ENDDEFINE.
+DEFINE !b() !LET 0 !ENDDEFINE.
+DEFINE !c() !LET !x !ENDDEFINE.
+DEFINE !d() !LET !x y !ENDDEFINE.
+DEBUG EXPAND.
+!a.
+!b.
+!c.
+!d.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1: In the expansion of `!a',
+define.sps:6.1-6.2: error: DEBUG EXPAND: Expected macro variable name following
+!LET.
+ 6 | !a.
+ | ^~
+
+!LET
+
+define.sps:2: At `0' in the expansion of `!b',
+define.sps:7.1-7.2: error: DEBUG EXPAND: Expected macro variable name following
+!LET.
+ 7 | !b.
+ | ^~
+
+!LET 0
+
+define.sps:3: In the expansion of `!c',
+define.sps:8.1-8.2: error: DEBUG EXPAND: Expected `=' following !LET.
+ 8 | !c.
+ | ^~
+
+!LET !x
+
+define.sps:4: At `y' in the expansion of `!d',
+define.sps:9.1-9.2: error: DEBUG EXPAND: Expected `=' following !LET.
+ 9 | !d.
+ | ^~
+
+!LET !x y
+])
+AT_CLEANUP
+
+AT_SETUP([macro !DO errors])
+AT_KEYWORDS([DO])
+AT_DATA([define.sps], [dnl
+DEFINE !a() !DO !ENDDEFINE.
+DEFINE !b() !DO 0 !ENDDEFINE.
+DEFINE !c() !DO !x !ENDDEFINE.
+DEFINE !d() !DO !x !in (x) !ENDDEFINE.
+DEFINE !e() !DO !x = x. !ENDDEFINE.
+DEFINE !f() !DO !x = 5 x !ENDDEFINE.
+DEFINE !g() !DO !x = 5 !TO 6 !BY 0 !ENDDEFINE.
+DEFINE !h() !DO !x !ENDDEFINE.
+DEFINE !i() !DO !x 0 !ENDDEFINE.
+DEFINE !j() !BREAK !ENDDEFINE.
+DEBUG EXPAND.
+!a.
+!b.
+!c.
+!d.
+!e.
+!f.
+!g.
+!h.
+!i.
+!j.
+])
+AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
+define.sps:1: In the expansion of `!a',
+define.sps:12.1-12.2: error: DEBUG EXPAND: Expected macro variable name
+following !DO.
+ 12 | !a.
+ | ^~
+
+!DO
+
+define.sps:2: At `0' in the expansion of `!b',
+define.sps:13.1-13.2: error: DEBUG EXPAND: Expected macro variable name
+following !DO.
+ 13 | !b.
+ | ^~
+
+!DO 0
+
+define.sps:3: In the expansion of `!c',
+define.sps:14.1-14.2: error: DEBUG EXPAND: Expected `=' or !IN in !DO loop.
+ 14 | !c.
+ | ^~
+
+!DO !x
+
+In the expansion of `!DO',
+define.sps:4: inside the expansion of `!d',
+define.sps:15.1-15.2: error: DEBUG EXPAND: Missing !DOEND.
+ 15 | !d.
+ | ^~
+
+!DO !x !in(x)
+
+At `x' in the expansion of `!DO',
+define.sps:5: inside the expansion of `!e',
+define.sps:16.1-16.2: error: DEBUG EXPAND: Macro expression must evaluate to a
+number (not "x").
+ 16 | !e.
+ | ^~
+
+!DO !x = x.
+
+At `x' in the expansion of `!DO',
+define.sps:6: inside the expansion of `!f',
+define.sps:17.1-17.2: error: DEBUG EXPAND: Expected !TO in numerical !DO loop.
+ 17 | !f.
+ | ^~
+
+!DO !x = 5 x
+
+In the expansion of `!DO',
+define.sps:7: inside the expansion of `!g',
+define.sps:18.1-18.2: error: DEBUG EXPAND: !BY value cannot be zero.
+ 18 | !g.
+ | ^~
+
+!DO !x = 5 !TO 6 !BY 0
+
+define.sps:8: In the expansion of `!h',
+define.sps:19.1-19.2: error: DEBUG EXPAND: Expected `=' or !IN in !DO loop.
+ 19 | !h.
+ | ^~
+
+!DO !x
+
+define.sps:9: At `0' in the expansion of `!i',
+define.sps:20.1-20.2: error: DEBUG EXPAND: Expected `=' or !IN in !DO loop.
+ 20 | !i.
+ | ^~
+
+!DO !x 0
+
+define.sps:10: At `!BREAK' in the expansion of `!j',
+define.sps:21.1-21.2: error: DEBUG EXPAND: !BREAK outside !DO.
+ 21 | !j.
+ | ^~
+
+])
+AT_CLEANUP
+
+AT_SETUP([macros in comments])
+AT_KEYWORDS([macro])
+AT_DATA([define.sps], [dnl
+DEFINE !macro() x y z !ENDDEFINE.
+/* !macro.
+*!macro.
+DEBUG EXPAND.
+!macro.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+x y z
+])
+AT_CLEANUP
+
+AT_SETUP([DEFINE syntax errors])
+AT_KEYWORDS([macro])
+AT_DATA([define.sps], [dnl
+DEFINE !macro(!POSITIONAL !CHAREND('x y')) !ENDDEFINE.
+DEFINE !macro(a=!TOKENS(1)/!POSITIONAL !TOKENS(1)) !ENDDEFINE.
+DEFINE !macro(!a=!TOKENS(1)) !ENDDEFINE.
+DEFINE !macro(do=!TOKENS(1)) !ENDDEFINE.
+DEFINE 0() !ENDDEFINE.
+DEFINE x y () !ENDDEFINE.
+DEFINE !macro(1) !ENDDEFINE.
+DEFINE !macro(x 2) !ENDDEFINE.
+DEFINE !macro(x=!DEFAULT 3) !ENDDEFINE.
+DEFINE !macro(x=!TOKENS 4) !ENDDEFINE.
+DEFINE !macro(x=!TOKENS(x)) !ENDDEFINE.
+DEFINE !macro(x=!TOKENS(1 5)) !ENDDEFINE.
+DEFINE !macro(x=!ENCLOSE 6) !ENDDEFINE.
+DEFINE !macro(x=!ENCLOSE('x' y)) !ENDDEFINE.
+DEFINE !macro(x=!ENCLOSE('x',y)) !ENDDEFINE.
+DEFINE !macro(x=!ENCLOSE('x','y' z)) !ENDDEFINE.
+DEFINE !macro(x=!CHAREND 7) !ENDDEFINE.
+DEFINE !macro(x=!CHAREND(8)) !ENDDEFINE.
+DEFINE !macro(x=!CHAREND('x' 9)) !ENDDEFINE.
+DEFINE !macro(x=!WTF) !ENDDEFINE.
+DEFINE !macro(x=!TOKENS(1) x) !ENDDEFINE.
+DEFINE !macro(x=!DEFAULT() !DEFAULT()) !ENDDEFINE.
+DEFINE !macro(x=!TOKENS(1) !CMDEND) !ENDDEFINE.
+DEFINE !macro()
+])
+AT_CHECK([pspp define.sps], [1], [dnl
+define.sps:1.36-1.40: error: DEFINE: String must contain exactly one token.
+ 1 | DEFINE !macro(!POSITIONAL !CHAREND('x y')) !ENDDEFINE.
+ | ^~~~~
+
+define.sps:2.28-2.38: error: DEFINE: Positional parameters must precede keyword
+parameters.
+ 2 | DEFINE !macro(a=!TOKENS(1)/!POSITIONAL !TOKENS(1)) !ENDDEFINE.
+ | ^~~~~~~~~~~
+
+define.sps:2.15: note: DEFINE: Here is a previous keyword parameter.
+ 2 | DEFINE !macro(a=!TOKENS(1)/!POSITIONAL !TOKENS(1)) !ENDDEFINE.
+ | ^
+
+define.sps:3.15-3.16: error: DEFINE: Keyword macro parameter must be named in
+definition without "!" prefix.
+ 3 | DEFINE !macro(!a=!TOKENS(1)) !ENDDEFINE.
+ | ^~
+
+define.sps:4.15-4.16: error: DEFINE: Cannot use macro keyword "do" as an
+argument name.
+ 4 | DEFINE !macro(do=!TOKENS(1)) !ENDDEFINE.
+ | ^~
+
+define.sps:5.8: error: DEFINE: Syntax error expecting identifier.
+ 5 | DEFINE 0() !ENDDEFINE.
+ | ^
+
+define.sps:6.10: error: DEFINE: Syntax error expecting `@{:@'.
+ 6 | DEFINE x y () !ENDDEFINE.
+ | ^
+
+define.sps:7.15: error: DEFINE: Syntax error expecting identifier.
+ 7 | DEFINE !macro(1) !ENDDEFINE.
+ | ^
+
+define.sps:8.17: error: DEFINE: Syntax error expecting !TOKENS, !CHAREND, !
+ENCLOSE, or !CMDEND.
+ 8 | DEFINE !macro(x 2) !ENDDEFINE.
+ | ^
+
+define.sps:9.26: error: DEFINE: Syntax error expecting `@{:@'.
+ 9 | DEFINE !macro(x=!DEFAULT 3) !ENDDEFINE.
+ | ^
+
+define.sps:10.25: error: DEFINE: Syntax error expecting `@{:@'.
+ 10 | DEFINE !macro(x=!TOKENS 4) !ENDDEFINE.
+ | ^
+
+define.sps:11.25: error: DEFINE: Syntax error expecting positive integer for !
+TOKENS.
+ 11 | DEFINE !macro(x=!TOKENS(x)) !ENDDEFINE.
+ | ^
+
+define.sps:12.27: error: DEFINE: Syntax error expecting `@:}@'.
+ 12 | DEFINE !macro(x=!TOKENS(1 5)) !ENDDEFINE.
+ | ^
+
+define.sps:13.26: error: DEFINE: Syntax error expecting `@{:@'.
+ 13 | DEFINE !macro(x=!ENCLOSE 6) !ENDDEFINE.
+ | ^
+
+define.sps:14.30: error: DEFINE: Syntax error expecting `,'.
+ 14 | DEFINE !macro(x=!ENCLOSE('x' y)) !ENDDEFINE.
+ | ^
+
+define.sps:15.30: error: DEFINE: Syntax error expecting string.
+ 15 | DEFINE !macro(x=!ENCLOSE('x',y)) !ENDDEFINE.
+ | ^
+
+define.sps:16.34: error: DEFINE: Syntax error expecting `@:}@'.
+ 16 | DEFINE !macro(x=!ENCLOSE('x','y' z)) !ENDDEFINE.
+ | ^
+
+define.sps:17.26: error: DEFINE: Syntax error expecting `@{:@'.
+ 17 | DEFINE !macro(x=!CHAREND 7) !ENDDEFINE.
+ | ^
+
+define.sps:18.26: error: DEFINE: Syntax error expecting string.
+ 18 | DEFINE !macro(x=!CHAREND(8)) !ENDDEFINE.
+ | ^
+
+define.sps:19.30: error: DEFINE: Syntax error expecting `@:}@'.
+ 19 | DEFINE !macro(x=!CHAREND('x' 9)) !ENDDEFINE.
+ | ^
+
+define.sps:20.17-20.20: error: DEFINE: Syntax error expecting !TOKENS, !
+CHAREND, !ENCLOSE, or !CMDEND.
+ 20 | DEFINE !macro(x=!WTF) !ENDDEFINE.
+ | ^~~~
+
+define.sps:21.28: error: DEFINE: Syntax error expecting `/'.
+ 21 | DEFINE !macro(x=!TOKENS(1) x) !ENDDEFINE.
+ | ^
+
+define.sps:22.28-22.35: error: DEFINE: !DEFAULT is allowed only once per
+argument.
+ 22 | DEFINE !macro(x=!DEFAULT() !DEFAULT()) !ENDDEFINE.
+ | ^~~~~~~~
+
+define.sps:23.28-23.34: error: DEFINE: Only one of !TOKENS, !CHAREND, !ENCLOSE,
+or !CMDEND is allowed.
+ 23 | DEFINE !macro(x=!TOKENS(1) !CMDEND) !ENDDEFINE.
+ | ^~~~~~~
+
+define.sps:25.1: error: DEFINE: Syntax error expecting macro body or !
+ENDDEFINE.
+ 25 |
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([macro expansion with token merging])
+AT_DATA([define.sps], [dnl
+DEFINE !foo() "foo" !ENDDEFINE.
+DEFINE !bar() "bar" !ENDDEFINE.
+DEFINE !plus() + !ENDDEFINE.
+DEFINE !minus() - !ENDDEFINE.
+DEFINE !one() 1 !ENDDEFINE.
+ECHO "foo" + "bar".
+ECHO !foo.
+ECHO !bar.
+ECHO !foo + "quux".
+ECHO "baz" + !bar.
+ECHO !foo + !bar.
+ECHO !foo !plus !bar.
+ECHO "two" "strings".
+N OF CASES -/**/1.
+N OF CASES !minus 1.
+N OF CASES - !one.
+N OF CASES !minus !one.
+])
+AT_CHECK([pspp define.sps], [1], [dnl
+foobar
+
+foo
+
+bar
+
+fooquux
+
+bazbar
+
+foobar
+
+foobar
+
+two
+
+define.sps:13.12-13.20: error: ECHO: Syntax error expecting end of command.
+ 13 | ECHO "two" "strings".
+ | ^~~~~~~~~
+
+define.sps:14.12-14.17: error: N OF CASES: Syntax error expecting positive
+integer for N OF CASES.
+ 14 | N OF CASES -/**/1.
+ | ^~~~~~
+
+define.sps:15.12-15.19: error: N OF CASES: Syntax error expecting positive
+integer for N OF CASES.
+ 15 | N OF CASES !minus 1.
+ | ^~~~~~~~
+
+define.sps:16.12-16.17: error: N OF CASES: Syntax error expecting positive
+integer for N OF CASES.
+ 16 | N OF CASES - !one.
+ | ^~~~~~
+
+define.sps:17.12-17.22: error: N OF CASES: Syntax error expecting positive
+integer for N OF CASES.
+ 17 | N OF CASES !minus !one.
+ | ^~~~~~~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([one macro calls another])
+AT_DATA([define.sps], [dnl
+DEFINE !a(!pos !enclose('(',')')) [[!1]] !ENDDEFINE.
+DEFINE !b(!pos !enclose('{','}')) !a(x !1 z) !ENDDEFINE.
+DEFINE !c(!pos !enclose('{','}')) !let !tmp=!quote(!concat('<',!1,'>')) !a(!tmp) !ENDDEFINE.
+DEBUG EXPAND.
+!b{y}.
+!c{y}.
+])
+AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
+[[x y z]]
+
+[[ < y > ]]
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DELETE VARIABLES])
+
+dnl Checks for regressions against a crash reported in bug #38843.
+AT_SETUP([DELETE VARIABLES with FILTER])
+AT_DATA([delete-variables.sps], [dnl
+DATA LIST LIST /a b.
+BEGIN DATA.
+1 3
+4 6
+7 9
+END DATA.
+
+FILTER BY b.
+DELETE VARIABLES a.
+LIST.
+])
+AT_CHECK([pspp -O format=csv delete-variables.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+a,F8.0
+b,F8.0
+
+Table: Data List
+b
+3.00
+6.00
+9.00
+])
+AT_CLEANUP
+
+dnl Checks for regression against a crash reported on pspp-users:
+dnl https://lists.gnu.org/archive/html/pspp-users/2021-03/msg00025.html
+AT_SETUP([DELETE VARIABLES with string variables])
+AT_DATA([delete-variables.sps], [dnl
+DATA LIST NOTABLE /s1 TO s2 1-2(A).
+BEGIN DATA
+12
+END DATA.
+DELETE VARIABLES s1.
+NUMERIC n1.
+LIST.
+])
+AT_CHECK([pspp -O format=csv delete-variables.sps], [0], [dnl
+Table: Data List
+s2,n1
+2,. @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([DELETE VARIABLES syntax errors])
+AT_DATA([delete-variables.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+BEGIN DATA.
+1 2 3
+END DATA.
+TEMPORARY.
+DELETE VARIABLES x.
+DELETE VARIABLES y z.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='delete-variables.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"delete-variables.sps:6.1-6.16: error: DELETE VARIABLES: DELETE VARIABLES may not be used after TEMPORARY. Temporary transformations will be made permanent.
+ 6 | DELETE VARIABLES x.
+ | ^~~~~~~~~~~~~~~~"
+
+"delete-variables.sps:7.1-7.20: error: DELETE VARIABLES: DELETE VARIABLES may not be used to delete all variables from the active dataset dictionary. Use NEW FILE instead.
+ 7 | DELETE VARIABLES y z.
+ | ^~~~~~~~~~~~~~~~~~~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DESCRIPTIVES procedure])
+
+AT_SETUP([DESCRIPTIVES basics])
+AT_DATA([descriptives.sps],
+ [title 'Test DESCRIPTIVES procedure'.
+
+data list / V0 to V16 1-17.
+begin data.
+12128989012389023
+34128080123890128
+56127781237893217
+78127378123793112
+90913781237892318
+37978547878935789
+52878237892378279
+12377912789378932
+26787654347894348
+29137178947891888
+end data.
+
+descript all/stat=all/format=serial.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+V0,1,1-1,F1.0
+V1,1,2-2,F1.0
+V2,1,3-3,F1.0
+V3,1,4-4,F1.0
+V4,1,5-5,F1.0
+V5,1,6-6,F1.0
+V6,1,7-7,F1.0
+V7,1,8-8,F1.0
+V8,1,9-9,F1.0
+V9,1,10-10,F1.0
+V10,1,11-11,F1.0
+V11,1,12-12,F1.0
+V12,1,13-13,F1.0
+V13,1,14-14,F1.0
+V14,1,15-15,F1.0
+V15,1,16-16,F1.0
+V16,1,17-17,F1.0
+
+Table: Descriptive Statistics
+,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
+V0,10,3.80,.84,2.66,7.07,-.03,1.33,.89,.69,8.00,1,9,38.00
+V1,10,4.60,.96,3.03,9.16,-1.39,1.33,-.03,.69,9.00,0,9,46.00
+V2,10,4.10,1.16,3.67,13.43,-2.02,1.33,.48,.69,8.00,1,9,41.00
+V3,10,4.10,.87,2.77,7.66,-2.05,1.33,.42,.69,7.00,1,8,41.00
+V4,10,7.00,.47,1.49,2.22,7.15,1.33,-2.52,.69,5.00,3,8,70.00
+V5,10,4.90,1.03,3.25,10.54,-1.40,1.33,-.20,.69,9.00,0,9,49.00
+V6,10,5.90,.80,2.51,6.32,-.29,1.33,-.96,.69,7.00,1,8,59.00
+V7,10,4.70,1.10,3.47,12.01,-1.99,1.33,-.16,.69,9.00,0,9,47.00
+V8,10,4.10,1.10,3.48,12.10,-1.93,1.33,.37,.69,9.00,0,9,41.00
+V9,10,4.30,.87,2.75,7.57,-.87,1.33,.73,.69,8.00,1,9,43.00
+V10,10,5.50,.85,2.68,7.17,-1.84,1.33,-.33,.69,7.00,2,9,55.00
+V11,10,6.50,.78,2.46,6.06,-1.28,1.33,-.89,.69,6.00,3,9,65.00
+V12,10,7.90,.60,1.91,3.66,5.24,1.33,-2.21,.69,6.00,3,9,79.00
+V13,10,4.30,.99,3.13,9.79,-1.25,1.33,.33,.69,9.00,0,9,43.00
+V14,10,3.60,1.01,3.20,10.27,-.96,1.33,.81,.69,9.00,0,9,36.00
+V15,10,3.70,.92,2.91,8.46,-1.35,1.33,.71,.69,7.00,1,8,37.00
+V16,10,6.40,.91,2.88,8.27,-1.14,1.33,-.92,.69,7.00,2,9,64.00
+Valid N (listwise),10,,,,,,,,,,,,
+Missing N (listwise),0,,,,,,,,,,,,
+])
+AT_CLEANUP
+
+m4_define([DESCRIPTIVES_MISSING_DATA],
+ [data list notable / V1 TO V3 1-3.
+mis val v1 to v3 (1).
+begin data.
+111
+
+ 1
+1 1
+112
+123
+234
+end data.
+])
+
+AT_SETUP([DESCRIPTIVES -- excluding missing data])
+AT_DATA([descriptives.sps],
+ [DESCRIPTIVES_MISSING_DATA
+descript all/stat=all/format=serial.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
+V1,1,2.00,. ,. ,. ,. ,. ,. ,. ,.00,2,2,2.00
+V2,2,2.50,.50,.71,.50,. ,. ,. ,. ,1.00,2,3,5.00
+V3,3,3.00,.58,1.00,1.00,. ,. ,.00,1.22,2.00,2,4,9.00
+Valid N (listwise),7,,,,,,,,,,,,
+Missing N (listwise),6,,,,,,,,,,,,
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES -- including missing data])
+AT_DATA([descriptives.sps],
+ [DESCRIPTIVES_MISSING_DATA
+descript all/stat=all/format=serial/missing=include.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
+V1,5,1.20,.20,.45,.20,5.00,2.00,2.24,.91,1.00,1,2,6.00
+V2,5,1.60,.40,.89,.80,.31,2.00,1.26,.91,2.00,1,3,8.00
+V3,5,2.20,.58,1.30,1.70,-1.49,2.00,.54,.91,3.00,1,4,11.00
+Valid N (listwise),7,,,,,,,,,,,,
+Missing N (listwise),3,,,,,,,,,,,,
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES -- excluding missing data listwise])
+AT_DATA([descriptives.sps],
+ [DESCRIPTIVES_MISSING_DATA
+descript all/stat=all/format=serial/missing=listwise.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
+V1,1,2.00,. ,. ,. ,. ,. ,. ,. ,.00,2,2,2.00
+V2,1,3.00,. ,. ,. ,. ,. ,. ,. ,.00,3,3,3.00
+V3,1,4.00,. ,. ,. ,. ,. ,. ,. ,.00,4,4,4.00
+Valid N (listwise),1,,,,,,,,,,,,
+Missing N (listwise),6,,,,,,,,,,,,
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES -- including missing data listwise])
+AT_DATA([descriptives.sps],
+ [DESCRIPTIVES_MISSING_DATA
+descript all/stat=all/format=serial/missing=listwise include.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
+V1,4,1.25,.25,.50,.25,4.00,2.62,2.00,1.01,1.00,1,2,5.00
+V2,4,1.75,.48,.96,.92,-1.29,2.62,.85,1.01,2.00,1,3,7.00
+V3,4,2.50,.65,1.29,1.67,-1.20,2.62,.00,1.01,3.00,1,4,10.00
+Valid N (listwise),4,,,,,,,,,,,,
+Missing N (listwise),3,,,,,,,,,,,,
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES bug calculating mean only])
+AT_DATA([descriptives.sps],
+ [SET FORMAT F8.3.
+
+data list notable / X 1.
+begin data.
+0
+1
+2
+3
+4
+5
+end data.
+
+descript all/stat=mean.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean
+X,6,2.500
+Valid N (listwise),6,
+Missing N (listwise),0,
+])
+AT_CLEANUP
+
+dnl Git history shows that this was probably a bug in the PSPP
+dnl core regarding multipass procedures, not anything specific
+dnl to DESCRIPTIVES.
+AT_SETUP([DESCRIPTIVES bug with TEMPORARY])
+AT_DATA([descriptives.sps], [dnl
+DATA LIST LIST NOTABLE /id * abc *.
+BEGIN DATA.
+1 3.5
+2 2.0
+3 2.0
+4 3.5
+5 3.0
+6 4.0
+7 5.0
+END DATA.
+
+TEMPORARY.
+SELECT IF id < 7 .
+
+DESCRIPTIVES /VAR=abc.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+abc,6,3.00,.84,2.00,4.00
+Valid N (listwise),6,,,,
+Missing N (listwise),0,,,,
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES -- Z scores])
+AT_DATA([descriptives.sps], [dnl
+DATA LIST LIST NOTABLE /a b.
+BEGIN DATA.
+1 50
+2 60
+3 70
+END DATA.
+
+DESCRIPTIVES /VAR=a b /SAVE.
+LIST.
+])
+AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
+Table: Mapping of Variables to Z-scores
+Source,Target
+a,Za
+b,Zb
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+a,3,2.00,1.00,1.00,3.00
+b,3,60.00,10.00,50.00,70.00
+Valid N (listwise),3,,,,
+Missing N (listwise),0,,,,
+
+Table: Data List
+a,b,Za,Zb
+1.00,50.00,-1.00,-1.00
+2.00,60.00,.00,.00
+3.00,70.00,1.00,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES -- Z scores with SPLIT FILE])
+AT_DATA([descriptives.sps], [dnl
+DATA LIST LIST NOTABLE /group a b.
+BEGIN DATA.
+1 1 50
+1 2 60
+1 3 70
+2 100 6000
+2 200 7000
+2 400 9000
+2 500 10000
+END DATA.
+
+SPLIT FILE BY group.
+DESCRIPTIVES /VAR=a b /SAVE.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt descriptives.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Mapping of Variables to Z-scores
+Source,Target
+a,Za
+b,Zb
+
+Table: Split Values
+Variable,Value
+group,1.00
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+a,3,2.00,1.00,1.00,3.00
+b,3,60.00,10.00,50.00,70.00
+Valid N (listwise),3,,,,
+Missing N (listwise),0,,,,
+
+Table: Split Values
+Variable,Value
+group,2.00
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+a,4,300.00,182.57,100.00,500.00
+b,4,8000.00,1825.74,6000.00,10000.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+
+Table: Split Values
+Variable,Value
+group,1.00
+
+Table: Data List
+group,a,b,Za,Zb
+1.00,1.00,50.00,-1.00,-1.00
+1.00,2.00,60.00,.00,.00
+1.00,3.00,70.00,1.00,1.00
+
+Table: Split Values
+Variable,Value
+group,2.00
+
+Table: Data List
+group,a,b,Za,Zb
+2.00,100.00,6000.00,-1.10,-1.10
+2.00,200.00,7000.00,-.55,-.55
+2.00,400.00,9000.00,.55,.55
+2.00,500.00,10000.00,1.10,1.10
+])
+AT_CLEANUP
+
+dnl Ideally DESCRIPTIVES would not make temporary transformations permanent
+dnl as it does now (bug #38786), so these results are imperfect. However,
+dnl this test does verify that DESCRIPTIVES does not crash in this situation
+dnl (as it once did).
+AT_SETUP([DESCRIPTIVES -- Z scores bug with TEMPORARY])
+AT_DATA([descriptives.sps], [dnl
+DATA LIST LIST NOTABLE /id abc.
+BEGIN DATA.
+1 3.5
+2 2.0
+3 2.0
+4 3.5
+5 3.0
+6 4.0
+7 5.0
+END DATA.
+
+TEMPORARY.
+SELECT IF id < 7 .
+
+DESCRIPTIVES /VAR=abc/SAVE.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt descriptives.sps], [0], [dnl
+descriptives.sps:15.23-15.26: warning: DESCRIPTIVES: DESCRIPTIVES with Z scores ignores TEMPORARY. Temporary transformations will be made permanent.
+ 15 | DESCRIPTIVES /VAR=abc/SAVE.
+ | ^~~~
+])
+AT_CHECK([cat pspp.csv], [0], [dnl
+"descriptives.sps:15.23-15.26: warning: DESCRIPTIVES: DESCRIPTIVES with Z scores ignores TEMPORARY. Temporary transformations will be made permanent.
+ 15 | DESCRIPTIVES /VAR=abc/SAVE.
+ | ^~~~"
+
+Table: Mapping of Variables to Z-scores
+Source,Target
+abc,Zabc
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+abc,6,3.00,.84,2.00,4.00
+Valid N (listwise),6,,,,
+Missing N (listwise),0,,,,
+
+Table: Data List
+id,abc,Zabc
+1.00,3.50,.60
+2.00,2.00,-1.20
+3.00,2.00,-1.20
+4.00,3.50,.60
+5.00,3.00,.00
+6.00,4.00,1.20
+])
+AT_CLEANUP
+
+dnl This test was supplied by Mindaugus as part of the report for bug #42012.
+AT_SETUP([DESCRIPTIVES -- Z scores with FILTER])
+AT_DATA([descriptives.sps], [dnl
+DATA LIST LIST/filter1 filter2 x.
+BEGIN DATA.
+0,0,300
+0,1,200
+0,1,100
+1,0,5
+1,0,4
+1,1,3
+1,1,2
+1,1,1
+END DATA.
+
+FILTER OFF.
+SPLIT FILE OFF.
+DESCRIPTIVES /VARIABLES=X /SAVE.
+
+FILTER BY filter1.
+SPLIT FILE OFF.
+DESCRIPTIVES /VARIABLES=X /SAVE.
+
+FILTER OFF.
+SORT CASES BY filter1.
+SPLIT FILE BY filter1.
+DESCRIPTIVES /VARIABLES=X /SAVE.
+
+FILTER BY filter2.
+SPLIT FILE BY filter1.
+DESCRIPTIVES /VARIABLES=X /SAVE.
+
+FILTER OFF.
+SORT CASES BY filter1 filter2.
+SPLIT FILE BY filter1 filter2.
+DESCRIPTIVES /VARIABLES=X /SAVE.
+EXECUTE.
+
+SPLIT FILE OFF.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv descriptives.sps])
+AT_CHECK([sed -n '/Table: Data List/,$p' < pspp.csv], [0], [dnl
+Table: Data List
+filter1,filter2,x,Zx,ZSC001,ZSC002,ZSC003,ZSC004
+.00,.00,300.00,1.94,. ,1.00,. ,. @&t@
+.00,1.00,200.00,1.07,. ,.00,.71,.71
+.00,1.00,100.00,.20,. ,-1.00,-.71,-.71
+1.00,.00,5.00,-.62,1.26,1.26,. ,.71
+1.00,.00,4.00,-.63,.63,.63,. ,-.71
+1.00,1.00,3.00,-.64,.00,.00,1.00,1.00
+1.00,1.00,2.00,-.65,-.63,-.63,.00,.00
+1.00,1.00,1.00,-.66,-1.26,-1.26,-1.00,-1.00
+])
+AT_CLEANUP
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([DESCRIPTIVES tutorial example])
+cp $top_srcdir/examples/physiology.sav .
+AT_DATA([descriptives.sps], [dnl
+GET FILE='physiology.sav'.
+DESCRIPTIVES sex, weight, height.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt descriptives.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+Sex of subject ,40,.45,.50,Male,Female
+Weight in kilograms ,40,72.12,26.70,-55.6,92.1
+Height in millimeters ,40,1677.12,262.87,179,1903
+Valid N (listwise),40,,,,
+Missing N (listwise),0,,,,
+])
+AT_CLEANUP
+
+AT_SETUP([DESCRIPTIVES syntax errors])
+AT_DATA([descriptives.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+DESCRIPTIVES MISSING=**.
+DESCRIPTIVES FORMAT=**.
+DESCRIPTIVES STATISTICS=**.
+DESCRIPTIVES SORT=**.
+DESCRIPTIVES SORT=NAME (**).
+DESCRIPTIVES SORT=NAME (A **).
+DESCRIPTIVES **.
+DESCRIPTIVES x/ **.
+DESCRIPTIVES MISSING=INCLUDE.
+TEMPORARY.
+NUMERIC Zx ZSC001 TO ZSC099 STDZ01 TO STDZ09 ZZZZ01 TO ZZZZ09 ZQZQ01 TO ZQZQ09.
+DESCRIPTIVES x/SAVE.
+])
+AT_CHECK([pspp descriptives.sps], [1], [dnl
+descriptives.sps:2.22-2.23: error: DESCRIPTIVES: Syntax error expecting
+VARIABLE, LISTWISE, or INCLUDE.
+ 2 | DESCRIPTIVES MISSING=**.
+ | ^~
+
+descriptives.sps:3.21-3.22: error: DESCRIPTIVES: Syntax error expecting LABELS,
+NOLABELS, INDEX, NOINDEX, LINE, or SERIAL.
+ 3 | DESCRIPTIVES FORMAT=**.
+ | ^~
+
+descriptives.sps:5.19-5.20: error: DESCRIPTIVES: Syntax error expecting
+variable name.
+ 5 | DESCRIPTIVES SORT=**.
+ | ^~
+
+descriptives.sps:6.25-6.26: error: DESCRIPTIVES: Syntax error expecting A or D.
+ 6 | DESCRIPTIVES SORT=NAME (**).
+ | ^~
+
+descriptives.sps:7.27-7.28: error: DESCRIPTIVES: Syntax error expecting `)'.
+ 7 | DESCRIPTIVES SORT=NAME (A **).
+ | ^~
+
+descriptives.sps:8.14-8.15: error: DESCRIPTIVES: Syntax error expecting
+variable name.
+ 8 | DESCRIPTIVES **.
+ | ^~
+
+descriptives.sps:9.17-9.18: error: DESCRIPTIVES: Syntax error expecting
+MISSING, SAVE, FORMAT, STATISTICS, SORT, or VARIABLES.
+ 9 | DESCRIPTIVES x/ **.
+ | ^~
+
+descriptives.sps:10: error: DESCRIPTIVES: No variables specified.
+
+descriptives.sps:13: error: DESCRIPTIVES: Ran out of generic names for Z-score
+variables. There are only 126 generic names: ZSC001-ZSC099, STDZ01-STDZ09,
+ZZZZ01-ZZZZ09, ZQZQ01-ZQZQ09.
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DO IF])
+
+AT_SETUP([DO IF])
+(for a in 0 1 ' '; do
+ for b in 0 1 ' '; do
+ for c in 0 1 ' '; do
+ for d in 0 1 ' '; do
+ abcd=$a$b$c$d
+ echo "$abcd" 1>&3
+ if test "$a" = "1"; then
+ echo " $abcd A"
+ elif test "$a" = " "; then
+ :
+ elif test "$b" = "1"; then
+ echo " $abcd B"
+ elif test "$b" = " "; then
+ :
+ elif test "$c" = "1"; then
+ echo " $abcd C"
+ elif test "$c" = " "; then
+ :
+ elif test "$d" = "1"; then
+ echo " $abcd D"
+ elif test "$d" = " "; then
+ :
+ else
+ echo " $abcd E"
+ fi
+ done
+ done
+ done
+done) >expout 3>do-if.txt || exit 99
+AT_DATA([do-if.sps], [dnl
+DATA LIST FILE="do-if.txt"/A B C D 1-4 ABCD 1-4 (A).
+DO IF A.
+PRINT OUTFILE="do-if.out"/ABCD 'A'.
+ELSE IF B.
+PRINT OUTFILE="do-if.out"/ABCD 'B'.
+ELSE IF C.
+PRINT OUTFILE="do-if.out"/ABCD 'C'.
+ELSE IF D.
+PRINT OUTFILE="do-if.out"/ABCD 'D'.
+ELSE.
+PRINT OUTFILE="do-if.out"/ABCD 'E'.
+END IF.
+EXECUTE.
+])
+AT_CHECK([pspp do-if.sps], [0], [ignore])
+AT_CHECK([cat do-if.out], [0], [expout])
+AT_CLEANUP
+
+AT_SETUP([DO IF - negative])
+AT_DATA([do-if.sps], [dnl
+DATA LIST LIST NOTABLE/a b c.
+BEGIN DATA.
+1 2 3
+END DATA.
+
+END IF.
+ELSE.
+ELSE IF 1.
+
+DO IF 0.
+ELSE.
+ELSE.
+END IF.
+
+DO IF 0.
+ELSE.
+ELSE IF 0.
+END IF.
+
+DO IF !.
+END IF.
+
+DO IF 0.
+])
+AT_CHECK([pspp -O format=csv do-if.sps], [1], [dnl
+"do-if.sps:6.1-6.6: error: END IF: This command cannot appear outside DO IF...END IF.
+ 6 | END IF.
+ | ^~~~~~"
+
+"do-if.sps:7.1-7.4: error: ELSE: This command cannot appear outside DO IF...END IF.
+ 7 | ELSE.
+ | ^~~~"
+
+"do-if.sps:8.1-8.7: error: ELSE IF: This command cannot appear outside DO IF...END IF.
+ 8 | ELSE IF 1.
+ | ^~~~~~~"
+
+"do-if.sps:12.1-12.4: error: DO IF: Only one ELSE is allowed within DO IF...END IF.
+ 12 | ELSE.
+ | ^~~~"
+
+"do-if.sps:11.1-11.5: note: DO IF: This is the location of the previous ELSE clause.
+ 11 | ELSE.
+ | ^~~~~"
+
+"do-if.sps:10.1-10.8: note: DO IF: This is the location of the DO IF command.
+ 10 | DO IF 0.
+ | ^~~~~~~~"
+
+"do-if.sps:17.1-17.7: error: DO IF: ELSE IF is not allowed following ELSE within DO IF...END IF.
+ 17 | ELSE IF 0.
+ | ^~~~~~~"
+
+"do-if.sps:16.1-16.5: note: DO IF: This is the location of the previous ELSE clause.
+ 16 | ELSE.
+ | ^~~~~"
+
+"do-if.sps:15.1-15.8: note: DO IF: This is the location of the DO IF command.
+ 15 | DO IF 0.
+ | ^~~~~~~~"
+
+"do-if.sps:20.7: error: DO IF: Syntax error parsing expression.
+ 20 | DO IF !.
+ | ^"
+
+error: DO IF: At end of input: Syntax error expecting END IF.
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([DO REPEAT])
+
+AT_SETUP([DO REPEAT -- simple])
+AT_DATA([do-repeat.sps], [dnl
+INPUT PROGRAM.
+STRING y(A1).
+DO REPEAT xval = 1 2 3 / yval = 'a' 'b' 'c' / var = a b c.
+COMPUTE x=xval.
+COMPUTE y=yval.
+COMPUTE var=xval.
+END CASE.
+END REPEAT PRINT.
+END FILE.
+END INPUT PROGRAM.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv do-repeat.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+COMPUTE x=3.
+COMPUTE y='c'.
+COMPUTE c=3.
+END CASE.
+
+COMPUTE x=2.
+COMPUTE y='b'.
+COMPUTE b=2.
+END CASE.
+
+COMPUTE x=1.
+COMPUTE y='a'.
+COMPUTE a=1.
+END CASE.
+
+Table: Data List
+y,x,a,b,c
+a,1.00,1.00,. ,. @&t@
+b,2.00,. ,2.00,. @&t@
+c,3.00,. ,. ,3.00
+])
+AT_CLEANUP
+
+AT_SETUP([DO REPEAT -- containing BEGIN DATA])
+AT_DATA([do-repeat.sps], [dnl
+DO REPEAT offset = 1 2 3.
+DATA LIST NOTABLE /x 1-2.
+BEGIN DATA.
+10
+20
+30
+END DATA.
+COMPUTE x = x + offset.
+LIST.
+END REPEAT.
+])
+AT_CHECK([pspp -o pspp.csv do-repeat.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+x
+11
+21
+31
+
+Table: Data List
+x
+12
+22
+32
+
+Table: Data List
+x
+13
+23
+33
+])
+AT_CLEANUP
+
+AT_SETUP([DO REPEAT -- dummy vars not expanded in include files])
+AT_DATA([include.sps], [dnl
+COMPUTE y = y + x + 10.
+])
+AT_DATA([do-repeat.sps], [dnl
+INPUT PROGRAM.
+COMPUTE x = 0.
+COMPUTE y = 0.
+END CASE.
+END FILE.
+END INPUT PROGRAM.
+
+DO REPEAT x = 1 2 3.
+INCLUDE include.sps.
+END REPEAT.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv do-repeat.sps], [0], [dnl
+do-repeat.sps:8.11: warning: DO REPEAT: Dummy variable name `x' hides dictionary variable `x'.
+ 8 | DO REPEAT x = 1 2 3.
+ | ^
+])
+AT_CHECK([cat pspp.csv], [0], [dnl
+"do-repeat.sps:8.11: warning: DO REPEAT: Dummy variable name `x' hides dictionary variable `x'.
+ 8 | DO REPEAT x = 1 2 3.
+ | ^"
+
+Table: Data List
+x,y
+.00,30.00
+])
+AT_CLEANUP
+
+AT_SETUP([DO REPEAT -- nested])
+AT_DATA([do-repeat.sps], [dnl
+DATA LIST NOTABLE /a 1.
+BEGIN DATA.
+0
+END DATA.
+
+DO REPEAT h = h0 TO h3 / x = 0 TO 3 / y = 8, 7.5, 6, 5.
+ COMPUTE h = x + y.
+END REPEAT.
+
+VECTOR v(6).
+COMPUTE #idx = 0.
+DO REPEAT i = 1 TO 2.
+ DO REPEAT j = 3 TO 5.
+ COMPUTE #x = i + j.
+ COMPUTE #idx = #idx + 1.
+ COMPUTE v(#idx) = #x.
+ END REPEAT.
+END REPEAT.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv do-repeat.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+a,h0,h1,h2,h3,v1,v2,v3,v4,v5,v6
+0,8.00,8.50,8.00,8.00,4.00,5.00,6.00,5.00,6.00,7.00
+])
+AT_CLEANUP
+
+dnl This program tests for a bug that crashed PSPP given an empty DO
+dnl REPEAT...END REPEAT block. See bug #18407.
+AT_SETUP([DO REPEAT -- empty])
+AT_DATA([do-repeat.sps], [dnl
+DATA LIST NOTABLE /a 1.
+BEGIN DATA.
+0
+END DATA.
+
+DO REPEAT h = a.
+END REPEAT.
+])
+AT_CHECK([pspp -o pspp.csv do-repeat.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+])
+AT_CLEANUP
+
+dnl This program tests for a bug that crashed PSPP when END REPEAT
+dnl was missing. See bug #31016.
+AT_SETUP([DO REPEAT -- missing END REPEAT])
+AT_DATA([do-repeat.sps], [dnl
+DATA LIST NOTABLE /x 1.
+DO REPEAT y = 1 TO 10.
+])
+AT_CHECK([pspp -O format=csv do-repeat.sps], [1], [dnl
+error: DO REPEAT: At end of input: Syntax error expecting END REPEAT.
+])
+AT_CLEANUP
+
+AT_SETUP([DO REPEAT -- syntax errors])
+AT_DATA([do-repeat.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+DO REPEAT **.
+END REPEAT.
+DO REPEAT x **.
+END REPEAT.
+DO REPEAT y=1/y=2.
+END REPEAT.
+DO REPEAT y=a b c **.
+END REPEAT.
+DO REPEAT y=1 2 **.
+END REPEAT.
+DO REPEAT y='a' 'b' **.
+END REPEAT.
+DO REPEAT y=**.
+END REPEAT.
+DO REPEAT y=1 2 3/z=4.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='do-repeat.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"do-repeat.sps:2.11-2.12: error: DO REPEAT: Syntax error expecting identifier.
+ 2 | DO REPEAT **.
+ | ^~"
+
+"do-repeat.sps:4.11: warning: DO REPEAT: Dummy variable name `x' hides dictionary variable `x'.
+ 4 | DO REPEAT x **.
+ | ^"
+
+"do-repeat.sps:4.13-4.14: error: DO REPEAT: Syntax error expecting `='.
+ 4 | DO REPEAT x **.
+ | ^~"
+
+"do-repeat.sps:6.15: error: DO REPEAT: Dummy variable name `y' is given twice.
+ 6 | DO REPEAT y=1/y=2.
+ | ^"
+
+"do-repeat.sps:8.19-8.20: error: DO REPEAT: Syntax error expecting `/' or end of command.
+ 8 | DO REPEAT y=a b c **.
+ | ^~"
+
+"do-repeat.sps:10.17-10.18: error: DO REPEAT: Syntax error expecting number.
+ 10 | DO REPEAT y=1 2 **.
+ | ^~"
+
+"do-repeat.sps:12.21-12.22: error: DO REPEAT: Syntax error expecting string.
+ 12 | DO REPEAT y='a' 'b' **.
+ | ^~"
+
+"do-repeat.sps:14.13-14.14: error: DO REPEAT: Syntax error expecting substitution values.
+ 14 | DO REPEAT y=**.
+ | ^~"
+
+do-repeat.sps:16: error: DO REPEAT: Each dummy variable must have the same number of substitutions.
+
+"do-repeat.sps:16.11-16.17: note: DO REPEAT: Dummy variable y had 3 substitutions.
+ 16 | DO REPEAT y=1 2 3/z=4.
+ | ^~~~~~~"
+
+"do-repeat.sps:16.19-16.21: note: DO REPEAT: Dummy variable z had 1 substitution.
+ 16 | DO REPEAT y=1 2 3/z=4.
+ | ^~~"
+
+error: DO REPEAT: At end of input: Syntax error expecting END REPEAT.
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017, 2019 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([EXAMINE])
+
+AT_SETUP([EXAMINE])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [
+DATA LIST LIST /QUALITY * W * BRAND * .
+BEGIN DATA
+3 1 1
+2 2 1
+1 2 1
+1 1 1
+4 1 1
+4 1 1
+5 1 2
+2 1 2
+4 4 2
+2 1 2
+3 1 2
+7 1 3
+4 2 3
+5 3 3
+3 1 3
+6 1 3
+END DATA
+
+WEIGHT BY w.
+
+VARIABLE LABELS brand 'Manufacturer'.
+VARIABLE LABELS quality 'Breaking Strain'.
+
+VALUE LABELS /brand 1 'Aspeger' 2 'Bloggs' 3 'Charlies'.
+
+LIST /FORMAT=NUMBERED.
+
+EXAMINE
+ quality BY brand
+ /STATISTICS descriptives extreme(3)
+ .
+])
+
+
+dnl In the following data, only the extreme values have been checked.
+dnl The descriptives have been blindly pasted.
+AT_CHECK([pspp -O format=csv examine.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+QUALITY,F8.0
+W,F8.0
+BRAND,F8.0
+
+Table: Data List
+Case Number,QUALITY,W,BRAND
+1,3.00,1.00,1.00
+2,2.00,2.00,1.00
+3,1.00,2.00,1.00
+4,1.00,1.00,1.00
+5,4.00,1.00,1.00
+6,4.00,1.00,1.00
+7,5.00,1.00,2.00
+8,2.00,1.00,2.00
+9,4.00,4.00,2.00
+10,2.00,1.00,2.00
+11,3.00,1.00,2.00
+12,7.00,1.00,3.00
+13,4.00,2.00,3.00
+14,5.00,3.00,3.00
+15,3.00,1.00,3.00
+16,6.00,1.00,3.00
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+Breaking Strain,24.00,100.0%,.00,.0%,24.00,100.0%
+
+Table: Extreme Values
+,,,Case Number,Value
+Breaking Strain,Highest,1,12,7.00
+,,2,16,6.00
+,,3,14,5.00
+,Lowest,1,3,1.00
+,,2,4,1.00
+,,3,2,2.00
+
+Table: Descriptives
+,,,Statistic,Std. Error
+Breaking Strain,Mean,,3.54,.32
+,95% Confidence Interval for Mean,Lower Bound,2.87,
+,,Upper Bound,4.21,
+,5% Trimmed Mean,,3.50,
+,Median,,4.00,
+,Variance,,2.52,
+,Std. Deviation,,1.59,
+,Minimum,,1.00,
+,Maximum,,7.00,
+,Range,,6.00,
+,Interquartile Range,,2.75,
+,Skewness,,.06,.47
+,Kurtosis,,-.36,.92
+
+Table: Case Processing Summary
+,Manufacturer,Cases,,,,,
+,,Valid,,Missing,,Total,
+,,N,Percent,N,Percent,N,Percent
+Breaking Strain,Aspeger,8.00,100.0%,.00,.0%,8.00,100.0%
+,Bloggs,8.00,100.0%,.00,.0%,8.00,100.0%
+,Charlies,8.00,100.0%,.00,.0%,8.00,100.0%
+
+Table: Extreme Values
+,Manufacturer,,,Case Number,Value
+Breaking Strain,Aspeger,Highest,1,6,4.00
+,,,2,5,4.00
+,,,3,1,3.00
+,,Lowest,1,3,1.00
+,,,2,4,1.00
+,,,3,2,2.00
+,Bloggs,Highest,1,7,5.00
+,,,2,9,4.00
+,,,3,11,3.00
+,,Lowest,1,8,2.00
+,,,2,10,2.00
+,,,3,11,3.00
+,Charlies,Highest,1,12,7.00
+,,,2,16,6.00
+,,,3,14,5.00
+,,Lowest,1,15,3.00
+,,,2,13,4.00
+,,,3,14,5.00
+
+Table: Descriptives
+,Manufacturer,,,Statistic,Std. Error
+Breaking Strain,Aspeger,Mean,,2.25,.45
+,,95% Confidence Interval for Mean,Lower Bound,1.18,
+,,,Upper Bound,3.32,
+,,5% Trimmed Mean,,2.22,
+,,Median,,2.00,
+,,Variance,,1.64,
+,,Std. Deviation,,1.28,
+,,Minimum,,1.00,
+,,Maximum,,4.00,
+,,Range,,3.00,
+,,Interquartile Range,,2.75,
+,,Skewness,,.47,.75
+,,Kurtosis,,-1.55,1.48
+,Bloggs,Mean,,3.50,.38
+,,95% Confidence Interval for Mean,Lower Bound,2.61,
+,,,Upper Bound,4.39,
+,,5% Trimmed Mean,,3.50,
+,,Median,,4.00,
+,,Variance,,1.14,
+,,Std. Deviation,,1.07,
+,,Minimum,,2.00,
+,,Maximum,,5.00,
+,,Range,,3.00,
+,,Interquartile Range,,1.75,
+,,Skewness,,-.47,.75
+,,Kurtosis,,-.83,1.48
+,Charlies,Mean,,4.88,.44
+,,95% Confidence Interval for Mean,Lower Bound,3.83,
+,,,Upper Bound,5.92,
+,,5% Trimmed Mean,,4.86,
+,,Median,,5.00,
+,,Variance,,1.55,
+,,Std. Deviation,,1.25,
+,,Minimum,,3.00,
+,,Maximum,,7.00,
+,,Range,,4.00,
+,,Interquartile Range,,1.75,
+,,Skewness,,.30,.75
+,,Kurtosis,,.15,1.48
+])
+
+AT_CLEANUP
+
+AT_SETUP([EXAMINE -- extremes])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+data list free /V1 W
+begin data.
+1 1
+2 1
+3 2
+3 1
+4 1
+5 1
+6 1
+7 1
+8 1
+9 1
+10 1
+11 1
+12 1
+13 1
+14 1
+15 1
+16 1
+17 1
+18 2
+19 1
+20 1
+end data.
+
+weight by w.
+
+examine v1
+ /statistics=extreme(6)
+ .
+])
+
+AT_CHECK([pspp -O format=csv examine.sps], [0],[dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+V1,23.00,100.0%,.00,.0%,23.00,100.0%
+
+Table: Extreme Values
+,,,Case Number,Value
+V1,Highest,1,21,20.00
+,,2,20,19.00
+,,3,19,18.00
+,,4,18,17.00
+,,5,17,16.00
+,,6,16,15.00
+,Lowest,1,1,1.00
+,,2,2,2.00
+,,3,3,3.00
+,,4,4,3.00
+,,5,5,4.00
+,,6,6,5.00
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([EXAMINE -- extremes with fractional weights])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([extreme.sps], [dnl
+set format=F20.3.
+data list notable list /w * x *.
+begin data.
+ 0.88 300000
+ 0.86 320000
+ 0.98 480000
+ 0.93 960000
+ 1.35 960000
+ 1.31 960000
+ 0.88 960000
+ 0.88 1080000
+ 0.88 1080000
+ 0.95 1200000
+ 1.47 1200000
+ 0.93 1200000
+ 0.98 1320000
+ 1.31 1380000
+ 0.93 1440000
+ 0.88 1560000
+ 1.56 1560000
+ 1.47 1560000
+end data.
+
+weight by w.
+
+
+EXAMINE
+ x
+ /STATISTICS = DESCRIPTIVES EXTREME (5)
+ .
+])
+
+AT_CHECK([pspp -O format=csv extreme.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x,19.430,100.0%,.000,.0%,19.430,100.0%
+
+Table: Extreme Values
+,,,Case Number,Value
+x,Highest,1,18,1560000.000
+,,2,17,1560000.000
+,,3,16,1560000.000
+,,4,15,1440000.000
+,,5,14,1380000.000
+,Lowest,1,1,300000.000
+,,2,2,320000.000
+,,3,3,480000.000
+,,4,4,960000.000
+,,5,5,960000.000
+
+Table: Descriptives
+,,,Statistic,Std. Error
+x,Mean,,1120010.293,86222.178
+,95% Confidence Interval for Mean,Lower Bound,939166.693,
+,,Upper Bound,1300853.894,
+,5% Trimmed Mean,,1141017.899,
+,Median,,1200000.000,
+,Variance,,144447748124.869,
+,Std. Deviation,,380062.821,
+,Minimum,,300000.000,
+,Maximum,,1560000.000,
+,Range,,1260000.000,
+,Interquartile Range,,467258.065,
+,Skewness,,-.887,.519
+,Kurtosis,,.340,1.005
+])
+
+AT_CLEANUP
+
+dnl Test the PERCENTILES subcommand of the EXAMINE command.
+dnl In particular test that it behaves properly when there are only
+dnl a few cases.
+AT_SETUP([EXAMINE -- percentiles])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST /X *.
+BEGIN DATA.
+2.00
+8.00
+5.00
+END DATA.
+
+EXAMINE /x
+ /PERCENTILES=HAVERAGE.
+
+EXAMINE /x
+ /PERCENTILES=WAVERAGE.
+
+EXAMINE /x
+ /PERCENTILES=ROUND.
+
+EXAMINE /x
+ /PERCENTILES=EMPIRICAL.
+
+EXAMINE /x
+ /PERCENTILES=AEMPIRICAL.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt examine.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+X,F8.0
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,3,100.0%,0,.0%,3,100.0%
+
+Table: Percentiles
+,,Percentiles,,,,,,
+,,5,10,25,50,75,90,95
+X,Weighted Average,.40,.80,2.00,5.00,8.00,8.00,8.00
+,Tukey's Hinges,,,3.50,5.00,6.50,,
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,3,100.0%,0,.0%,3,100.0%
+
+Table: Percentiles
+,,Percentiles,,,,,,
+,,5,10,25,50,75,90,95
+X,Weighted Average,.30,.60,1.50,3.50,5.75,7.10,7.55
+,Tukey's Hinges,,,3.50,5.00,6.50,,
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,3,100.0%,0,.0%,3,100.0%
+
+Table: Percentiles
+,,Percentiles,,,,,,
+,,5,10,25,50,75,90,95
+X,Weighted Average,.00,.00,2.00,5.00,5.00,8.00,8.00
+,Tukey's Hinges,,,3.50,5.00,6.50,,
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,3,100.0%,0,.0%,3,100.0%
+
+Table: Percentiles
+,,Percentiles,,,,,,
+,,5,10,25,50,75,90,95
+X,Weighted Average,2.00,2.00,2.00,5.00,8.00,8.00,8.00
+,Tukey's Hinges,,,3.50,5.00,6.50,,
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,3,100.0%,0,.0%,3,100.0%
+
+Table: Percentiles
+,,Percentiles,,,,,,
+,,5,10,25,50,75,90,95
+X,Weighted Average,2.00,2.00,2.00,5.00,8.00,8.00,8.00
+,Tukey's Hinges,,,3.50,5.00,6.50,,
+])
+AT_CLEANUP
+
+AT_SETUP([EXAMINE -- missing values])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST /x * y *.
+BEGIN DATA.
+1 1
+2 1
+3 1
+4 1
+5 2
+6 2
+. 2
+END DATA
+
+EXAMINE /x by y
+ /MISSING = PAIRWISE
+ .
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+y,F8.0
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x,6,85.7%,1,14.3%,7,100.0%
+
+Table: Case Processing Summary
+,y,Cases,,,,,
+,,Valid,,Missing,,Total,
+,,N,Percent,N,Percent,N,Percent
+x,1.00,4,100.0%,0,.0%,4,100.0%
+,2.00,2,66.7%,1,33.3%,3,100.0%
+])
+AT_CLEANUP
+
+
+AT_SETUP([EXAMINE -- user missing values])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine-m.sps], [dnl
+DATA LIST notable LIST /x * y *.
+BEGIN DATA.
+1 2
+9999999999 2
+9999999999 99
+END DATA.
+
+MISSING VALUES x (9999999999).
+MISSING VALUES y (99).
+
+EXAMINE
+ /VARIABLES= x y
+ /MISSING=PAIRWISE.
+])
+AT_CHECK([pspp -O format=csv examine-m.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x,1,33.3%,2,66.7%,3,100.0%
+y,2,66.7%,1,33.3%,3,100.0%
+])
+AT_CLEANUP
+
+AT_SETUP([EXAMINE -- missing values and percentiles])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST /X *.
+BEGIN DATA.
+99
+99
+5.00
+END DATA.
+
+MISSING VALUE X (99).
+
+EXAMINE /x
+ /PERCENTILES=HAVERAGE.
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Tests the trimmed mean calculation in the case
+dnl where the data is weighted towards the centre.
+AT_SETUP([EXAMINE -- trimmed mean])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST /X * C *.
+BEGIN DATA.
+1 1
+2 49
+3 2
+END DATA.
+
+WEIGHT BY c.
+
+EXAMINE
+ x
+ /STATISTICS=DESCRIPTIVES
+ .
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+X,F8.0
+C,F8.0
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,52.00,100.0%,.00,.0%,52.00,100.0%
+
+Table: Descriptives
+,,,Statistic,Std. Error
+X,Mean,,2.02,.03
+,95% Confidence Interval for Mean,Lower Bound,1.95,
+,,Upper Bound,2.09,
+,5% Trimmed Mean,,2.00,
+,Median,,2.00,
+,Variance,,.06,
+,Std. Deviation,,.24,
+,Minimum,,1.00,
+,Maximum,,3.00,
+,Range,,2.00,
+,Interquartile Range,,.00,
+,Skewness,,1.19,.33
+,Kurtosis,,15.73,.65
+])
+AT_CLEANUP
+
+AT_SETUP([EXAMINE -- crash bug])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+data list list /a * x * y *.
+begin data.
+3 1 3
+5 1 4
+7 2 3
+end data.
+
+examine a by x by y
+ /statistics=DESCRIPTIVES
+ .
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Test that two consecutive EXAMINE commands don't crash PSPP.
+AT_SETUP([EXAMINE -- consecutive runs don't crash])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+data list list /y * z *.
+begin data.
+6 4
+5 3
+7 6
+end data.
+
+EXAMINE /VARIABLES= z BY y.
+
+EXAMINE /VARIABLES= z.
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Test that /DESCRIPTIVES does not crash in presence of missing values.
+AT_SETUP([EXAMINE -- missing values don't crash])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+data list list /x * y *.
+begin data.
+1 0
+2 0
+. 0
+3 1
+4 1
+end data.
+examine x by y /statistics=descriptives.
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Test that having only a single case doesn't crash.
+AT_SETUP([EXAMINE -- single case doesn't crash])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST /quality * .
+BEGIN DATA
+3
+END DATA
+
+
+EXAMINE
+ quality
+ /STATISTICS descriptives
+ /PLOT = histogram
+ .
+])
+AT_CHECK([pspp -o pspp.csv examine.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Test that all-missing data doesn't crash.
+AT_SETUP([EXAMINE -- all-missing data doesn't crash])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST /x *.
+BEGIN DATA.
+.
+.
+.
+.
+END DATA.
+
+EXAMINE /x
+ PLOT=HISTOGRAM BOXPLOT NPPLOT SPREADLEVEL(1) ALL
+ /ID=x
+ /STATISTICS = DESCRIPTIVES EXTREME (5) ALL
+ /PERCENTILE=AEMPIRICAL
+ .
+])
+AT_CHECK([pspp -o pspp.csv examine.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Test that big input doesn't crash (bug 11307).
+AT_SETUP([EXAMINE -- big input doesn't crash])
+AT_KEYWORDS([categorical categoricals slow])
+AT_DATA([examine.sps], [dnl
+INPUT PROGRAM.
+ LOOP #I=1 TO 50000.
+ COMPUTE X=NORMAL(10).
+ END CASE.
+ END LOOP.
+ END FILE.
+END INPUT PROGRAM.
+
+
+EXAMINE /x
+ /STATISTICS=DESCRIPTIVES.
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+dnl Another test that big input doesn't crash.
+dnl The actual bug that this checks for has been lost.
+AT_SETUP([EXAMINE -- big input doesn't crash 2])
+AT_KEYWORDS([categorical categoricals slow])
+AT_CHECK([$PYTHON3 -c '
+for i in range(100000): print("AB12")
+for i in range(100000): print("AB04")
+' > large.txt])
+AT_DATA([examine.sps], [dnl
+DATA LIST FILE='large.txt' /S 1-2 (A) X 3 .
+
+
+AGGREGATE OUTFILE=* /BREAK=X /A=N.
+
+
+EXAMINE /A BY X.
+])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CHECK([$PYTHON3 -c 'for i in range(25000): print("AB04\nAB12")' >> large.txt])
+AT_CHECK([pspp -o pspp.csv examine.sps])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+
+dnl Test that the ID command works with non-numberic variables
+AT_SETUP([EXAMINE -- non-numeric ID])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([examine-id.sps], [dnl
+data list notable list /x * y (a12).
+begin data.
+1 one
+2 two
+3 three
+4 four
+5 five
+6 six
+7 seven
+8 eight
+9 nine
+10 ten
+11 eleven
+12 twelve
+30 thirty
+300 threehundred
+end data.
+
+set small=0.
+examine x
+ /statistics = extreme
+ /id = y
+ /plot = boxplot
+ .
+])
+
+AT_CHECK([pspp -O format=csv examine-id.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x,14,100.0%,0,.0%,14,100.0%
+
+Table: Extreme Values
+,,,y,Value
+x,Highest,1,threehundred,300.00
+,,2,thirty,30.00
+,,3,twelve,12.00
+,,4,eleven,11.00
+,,5,ten,10.00
+,Lowest,1,one,1.00
+,,2,two,2.00
+,,3,three,3.00
+,,4,four,4.00
+,,5,five,5.00
+
+Table: Tests of Normality
+,Shapiro-Wilk,,
+,Statistic,df,Sig.
+x,.37,14,.00
+])
+
+AT_CLEANUP
+
+dnl Test for a crash which happened on cleanup from a bad input syntax
+AT_SETUP([EXAMINE -- Bad Input])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([examine-bad.sps], [dnl
+data list list /h * g *.
+begin data.
+1 1
+2 1
+3 1
+4 1
+5 2
+6 2
+7 2
+8 2
+9 2
+end data.
+
+EXAMINE
+ /VARIABLES= h
+ BY g
+ /STATISTICS = DESCRIPTIVES EXTREME
+ /PLOT = lkajsdas
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv examine-bad.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+dnl Check the MISSING=REPORT option
+AT_SETUP([EXAMINE -- MISSING=REPORT])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([examine-report.sps], [dnl
+set format = F22.0.
+data list list /x * g *.
+begin data.
+1 1
+2 1
+3 1
+4 1
+5 1
+6 1
+7 1
+8 1
+9 1
+10 2
+20 2
+30 2
+40 2
+50 2
+60 2
+70 2
+80 2
+90 2
+101 9
+201 9
+301 9
+401 9
+501 99
+601 99
+701 99
+801 99
+901 99
+1001 .
+2002 .
+3003 .
+4004 .
+end data.
+
+MISSING VALUES g (9, 99, 999).
+
+EXAMINE
+ /VARIABLES = x
+ BY g
+ /STATISTICS = EXTREME
+ /NOTOTAL
+ /MISSING = REPORT.
+])
+
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt examine-report.sps])
+AT_CHECK([cat pspp.csv], [0],
+ [[Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+g,F8.0
+
+Table: Case Processing Summary
+,g,Cases,,,,,
+,,Valid,,Missing,,Total,
+,,N,Percent,N,Percent,N,Percent
+x,.,4,100.0%,0,.0%,4,100.0%
+,1,9,100.0%,0,.0%,9,100.0%
+,2,9,100.0%,0,.0%,9,100.0%
+,9[a],4,100.0%,0,.0%,4,100.0%
+,99[a],5,100.0%,0,.0%,5,100.0%
+Footnote: a. User-missing value.
+
+Table: Extreme Values
+,g,,,Case Number,Value
+x,.,Highest,1,31,4004
+,,,2,30,3003
+,,,3,29,2002
+,,,4,28,1001
+,,,5,0,0
+,,Lowest,1,28,1001
+,,,2,29,2002
+,,,3,30,3003
+,,,4,31,4004
+,,,5,31,4004
+,1,Highest,1,9,9
+,,,2,8,8
+,,,3,7,7
+,,,4,6,6
+,,,5,5,5
+,,Lowest,1,1,1
+,,,2,2,2
+,,,3,3,3
+,,,4,4,4
+,,,5,5,5
+,2,Highest,1,18,90
+,,,2,17,80
+,,,3,16,70
+,,,4,15,60
+,,,5,14,50
+,,Lowest,1,10,10
+,,,2,11,20
+,,,3,12,30
+,,,4,13,40
+,,,5,14,50
+,9[a],Highest,1,22,401
+,,,2,21,301
+,,,3,20,201
+,,,4,19,101
+,,,5,0,0
+,,Lowest,1,19,101
+,,,2,20,201
+,,,3,21,301
+,,,4,22,401
+,,,5,22,401
+,99[a],Highest,1,27,901
+,,,2,26,801
+,,,3,25,701
+,,,4,24,601
+,,,5,23,501
+,,Lowest,1,23,501
+,,,2,24,601
+,,,3,25,701
+,,,4,26,801
+,,,5,27,901
+Footnote: a. User-missing value.
+]])
+
+AT_CLEANUP
+
+
+dnl Run a test of the basic STATISTICS using a "real"
+dnl dataset and comparing with "real" results kindly
+dnl provided by Olaf Nöhring
+AT_SETUP([EXAMINE -- sample unweighted])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([sample.sps], [dnl
+set format = F22.4.
+DATA LIST notable LIST /X *
+BEGIN DATA.
+461.19000000
+466.38000000
+479.46000000
+480.10000000
+483.43000000
+488.30000000
+489.00000000
+491.62000000
+505.62000000
+511.30000000
+521.53000000
+526.70000000
+528.25000000
+538.70000000
+540.22000000
+540.58000000
+546.10000000
+548.17000000
+553.99000000
+566.21000000
+575.90000000
+584.38000000
+593.40000000
+357.05000000
+359.73000000
+360.48000000
+373.98000000
+374.13000000
+381.45000000
+383.72000000
+390.00000000
+400.34000000
+415.32000000
+415.91000000
+418.30000000
+421.03000000
+422.43000000
+426.93000000
+433.25000000
+436.89000000
+445.33000000
+446.33000000
+446.55000000
+456.44000000
+689.49000000
+691.92000000
+695.00000000
+695.36000000
+698.21000000
+699.46000000
+706.61000000
+710.69000000
+715.82000000
+715.82000000
+741.39000000
+752.27000000
+756.73000000
+757.74000000
+759.57000000
+796.07000000
+813.78000000
+817.25000000
+825.48000000
+831.28000000
+849.24000000
+890.00000000
+894.78000000
+935.65000000
+935.90000000
+945.90000000
+1012.8600000
+1022.6000000
+1061.8100000
+1063.5000000
+1077.2300000
+1151.6300000
+1355.2800000
+598.88000000
+606.91000000
+621.60000000
+624.80000000
+636.13000000
+637.38000000
+640.32000000
+649.35000000
+656.51000000
+662.55000000
+664.69000000
+106.22000000
+132.24000000
+174.76000000
+204.85000000
+264.93000000
+264.99000000
+269.84000000
+325.12000000
+331.67000000
+337.26000000
+347.68000000
+354.91000000
+END DATA.
+
+EXAMINE
+ x
+ /STATISTICS=DESCRIPTIVES
+ .
+])
+
+AT_CHECK([pspp -O format=csv sample.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+X,100,100.0%,0,.0%,100,100.0%
+
+Table: Descriptives
+,,,Statistic,Std. Error
+X,Mean,,587.6603,23.2665
+,95% Confidence Interval for Mean,Lower Bound,541.4946,
+,,Upper Bound,633.8260,
+,5% Trimmed Mean,,579.7064,
+,Median,,547.1350,
+,Variance,,54132.8466,
+,Std. Deviation,,232.6647,
+,Minimum,,106.2200,
+,Maximum,,1355.2800,
+,Range,,1249.0600,
+,Interquartile Range,,293.1575,
+,Skewness,,.6331,.2414
+,Kurtosis,,.5300,.4783
+])
+
+AT_CLEANUP
+
+
+
+dnl Test for a crash which happened on bad input syntax
+AT_SETUP([EXAMINE -- Empty Parentheses])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([examine-empty-parens.sps], [dnl
+DATA LIST notable LIST /X *
+BEGIN DATA.
+2
+3
+END DATA.
+
+
+EXAMINE
+ x
+ /PLOT = SPREADLEVEL()
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv examine-empty-parens.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+
+dnl Test for another crash which happened on bad input syntax
+AT_SETUP([EXAMINE -- Bad variable])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([examine-bad-variable.sps], [dnl
+data list list /h * g *.
+begin data.
+3 1
+4 1
+5 2
+end data.
+
+EXAMINE
+ /VARIABLES/ h
+ BY g
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv examine-bad-variable.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+dnl Test for yet another crash. This time for extremes vs. missing weight values.\0
+AT_SETUP([EXAMINE -- Extremes vs. Missing Weights])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([examine-missing-weights.sps], [dnl
+data list notable list /h * g *.
+begin data.
+3 1
+4 .
+5 1
+2 1
+end data.
+
+WEIGHT BY g.
+
+EXAMINE h
+ /STATISTICS extreme(3)
+ .
+])
+
+AT_CHECK([pspp -O format=csv examine-missing-weights.sps], [0], [dnl
+"examine-missing-weights.sps:13: warning: EXAMINE: At least one case in the data file had a weight value that was user-missing, system-missing, zero, or negative. These case(s) were ignored."
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+h,3.00,100.0%,.00,.0%,3.00,100.0%
+
+Table: Extreme Values
+,,,Case Number,Value
+h,Highest,1,3,5.00
+,,2,2,4.00
+,,3,1,3.00
+,Lowest,1,4,2.00
+,,2,1,3.00
+,,3,2,4.00
+])
+
+AT_CLEANUP
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([EXAMINE tutorial example 1])
+cp $top_srcdir/examples/repairs.sav .
+AT_DATA([repairs.sps], [dnl
+GET FILE='repairs.sav'.
+EXAMINE mtbf /STATISTICS=DESCRIPTIVES.
+COMPUTE mtbf_ln = LN (mtbf).
+EXAMINE mtbf_ln /STATISTICS=DESCRIPTIVES.
+])
+
+AT_CHECK([pspp -O format=csv repairs.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+Mean time between failures (months) ,30,100.0%,0,.0%,30,100.0%
+
+Table: Descriptives
+,,,Statistic,Std. Error
+Mean time between failures (months) ,Mean,,8.78,1.10
+,95% Confidence Interval for Mean,Lower Bound,6.53,
+,,Upper Bound,11.04,
+,5% Trimmed Mean,,8.20,
+,Median,,8.29,
+,Variance,,36.34,
+,Std. Deviation,,6.03,
+,Minimum,,1.63,
+,Maximum,,26.47,
+,Range,,24.84,
+,Interquartile Range,,6.03,
+,Skewness,,1.65,.43
+,Kurtosis,,3.41,.83
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+mtbf_ln,30,100.0%,0,.0%,30,100.0%
+
+Table: Descriptives
+,,,Statistic,Std. Error
+mtbf_ln,Mean,,1.95,.13
+,95% Confidence Interval for Mean,Lower Bound,1.69,
+,,Upper Bound,2.22,
+,5% Trimmed Mean,,1.96,
+,Median,,2.11,
+,Variance,,.49,
+,Std. Deviation,,.70,
+,Minimum,,.49,
+,Maximum,,3.28,
+,Range,,2.79,
+,Interquartile Range,,.88,
+,Skewness,,-.37,.43
+,Kurtosis,,.01,.83
+])
+
+AT_CLEANUP
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([EXAMINE tutorial example 2])
+cp $top_srcdir/examples/physiology.sav .
+AT_DATA([examine.sps], [dnl
+GET FILE='physiology.sav'.
+EXAMINE height, weight /STATISTICS=EXTREME(3).
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt examine.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+Height in millimeters ,40,100.0%,0,.0%,40,100.0%
+Weight in kilograms ,40,100.0%,0,.0%,40,100.0%
+
+Table: Extreme Values
+,,,Case Number,Value
+Height in millimeters ,Highest,1,14,1903
+,,2,15,1884
+,,3,12,1802
+,Lowest,1,30,179
+,,2,31,1598
+,,3,28,1601
+Weight in kilograms ,Highest,1,13,92.1
+,,2,5,92.1
+,,3,17,91.7
+,Lowest,1,38,-55.6
+,,2,39,54.5
+,,3,33,55.4
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([EXAMINE -- Crash on unrepresentable graphs])
+AT_DATA([examine.sps], [dnl
+data list notable list /x * g *.
+begin data.
+96 1
+end data.
+
+examine x by g
+ /nototal
+ /plot = all.
+])
+dnl This bug only manifested itself on cairo based drivers.
+AT_CHECK([pspp -O format=pdf examine.sps], [0], [ignore], [ignore])
+AT_CLEANUP
+
+
+dnl This example comes from the web site:
+dnl https://www.spsstests.com/2018/11/shapiro-wilk-normality-test-spss.html
+AT_SETUP([EXAMINE -- shapiro-wilk 1])
+AT_KEYWORDS([shapiro wilk])
+AT_DATA([shapiro-wilk.sps], [dnl
+data list notable list /x * g *.
+begin data.
+96 1
+98 1
+95 1
+89 1
+90 1
+92 1
+94 1
+93 1
+97 1
+100 1
+99 2
+96 2
+80 2
+89 2
+91 2
+92 2
+93 2
+94 2
+99 2
+80 2
+end data.
+
+set format F22.3.
+
+examine x by g
+ /nototal
+ /plot = all.
+])
+
+AT_CHECK([pspp -O format=csv shapiro-wilk.sps], [0],[dnl
+Table: Case Processing Summary
+,g,Cases,,,,,
+,,Valid,,Missing,,Total,
+,,N,Percent,N,Percent,N,Percent
+x,1.00,10,100.0%,0,.0%,10,100.0%
+,2.00,10,100.0%,0,.0%,10,100.0%
+
+Table: Tests of Normality
+,g,Shapiro-Wilk,,
+,,Statistic,df,Sig.
+x,1.00,.984,10,.983
+,2.00,.882,10,.136
+])
+
+AT_CLEANUP
+
+
+dnl This example comes from the web site:
+dnl http://www.real-statistics.com/tests-normality-and-symmetry/statistical-tests-normality-symmetry/shapiro-wilk-expanded-test/
+dnl It uses a dataset larger than 11 samples. Hence the alternative method for
+dnl signficance is used.
+AT_SETUP([EXAMINE -- shapiro-wilk 2])
+AT_KEYWORDS([shapiro wilk])
+AT_DATA([shapiro-wilk2.sps], [dnl
+data list notable list /x *.
+begin data.
+65
+61
+63
+86
+70
+55
+74
+35
+72
+68
+45
+58
+end data.
+
+set format F22.3.
+
+examine x
+ /plot = boxplot.
+])
+
+AT_CHECK([pspp -O format=csv shapiro-wilk2.sps], [0],[dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Valid,,Missing,,Total,
+,N,Percent,N,Percent,N,Percent
+x,12,100.0%,0,.0%,12,100.0%
+
+Table: Tests of Normality
+,Shapiro-Wilk,,
+,Statistic,df,Sig.
+x,.971,12,.922
+])
+
+AT_CLEANUP
+
+AT_SETUP([EXAMINE syntax errors])
+AT_DATA([examine.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+EXAMINE VARIABLES **.
+EXAMINE **.
+EXAMINE x BY **.
+EXAMINE x/STATISTICS EXTREME (**).
+EXAMINE x/STATISTICS EXTREME (5 **).
+EXAMINE x/STATISTICS **.
+EXAMINE x/PERCENTILES(111).
+EXAMINE x/PERCENTILES(**).
+EXAMINE x/PERCENTILES **.
+EXAMINE x/MISSING **.
+EXAMINE x/COMPARE **.
+EXAMINE x/PLOT SPREADLEVEL(**).
+EXAMINE x/PLOT SPREADLEVEL(123 **).
+EXAMINE x/PLOT **.
+EXAMINE x/CINTERVAL **.
+EXAMINE x/ **.
+EXAMINE x/TOTAL/NOTOTAL.
+])
+AT_CHECK([pspp -O format=csv examine.sps], [1], [dnl
+"examine.sps:2.19-2.20: error: EXAMINE: Syntax error expecting `='.
+ 2 | EXAMINE VARIABLES **.
+ | ^~"
+
+"examine.sps:3.9-3.10: error: EXAMINE: Syntax error expecting variable name.
+ 3 | EXAMINE **.
+ | ^~"
+
+"examine.sps:4.14-4.15: error: EXAMINE: Syntax error expecting one of the following: STATISTICS, PERCENTILES, TOTAL, NOTOTAL, MISSING, COMPARE, PLOT, CINTERVAL, ID.
+ 4 | EXAMINE x BY **.
+ | ^~"
+
+"examine.sps:5.31-5.32: error: EXAMINE: Syntax error expecting non-negative integer for EXTREME.
+ 5 | EXAMINE x/STATISTICS EXTREME (**).
+ | ^~"
+
+"examine.sps:6.33-6.34: error: EXAMINE: Syntax error expecting `@:}@'.
+ 6 | EXAMINE x/STATISTICS EXTREME (5 **).
+ | ^~"
+
+"examine.sps:7.22-7.23: error: EXAMINE: Syntax error expecting DESCRIPTIVES, EXTREME, NONE, or ALL.
+ 7 | EXAMINE x/STATISTICS **.
+ | ^~"
+
+"examine.sps:8.23-8.25: error: EXAMINE: Syntax error expecting number in (0,100) for PERCENTILES.
+ 8 | EXAMINE x/PERCENTILES(111).
+ | ^~~"
+
+"examine.sps:9.23-9.24: error: EXAMINE: Syntax error expecting `@:}@'.
+ 9 | EXAMINE x/PERCENTILES(**).
+ | ^~"
+
+"examine.sps:10.23-10.24: error: EXAMINE: Syntax error expecting HAVERAGE, WAVERAGE, ROUND, EMPIRICAL, AEMPIRICAL, or NONE.
+ 10 | EXAMINE x/PERCENTILES **.
+ | ^~"
+
+"examine.sps:11.19-11.20: error: EXAMINE: Syntax error expecting LISTWISE, PAIRWISE, EXCLUDE, INCLUDE, REPORT, or NOREPORT.
+ 11 | EXAMINE x/MISSING **.
+ | ^~"
+
+"examine.sps:12.19-12.20: error: EXAMINE: Syntax error expecting VARIABLES or GROUPS.
+ 12 | EXAMINE x/COMPARE **.
+ | ^~"
+
+"examine.sps:13.28-13.29: error: EXAMINE: Syntax error expecting number.
+ 13 | EXAMINE x/PLOT SPREADLEVEL(**).
+ | ^~"
+
+"examine.sps:13.28-13.29: error: EXAMINE: Syntax error expecting BOXPLOT, NPPLOT, HISTOGRAM, SPREADLEVEL, NONE, or ALL.
+ 13 | EXAMINE x/PLOT SPREADLEVEL(**).
+ | ^~"
+
+"examine.sps:14.32-14.33: error: EXAMINE: Syntax error expecting `@:}@'.
+ 14 | EXAMINE x/PLOT SPREADLEVEL(123 **).
+ | ^~"
+
+"examine.sps:15.16-15.17: error: EXAMINE: Syntax error expecting BOXPLOT, NPPLOT, HISTOGRAM, SPREADLEVEL, NONE, or ALL.
+ 15 | EXAMINE x/PLOT **.
+ | ^~"
+
+"examine.sps:16.21-16.22: error: EXAMINE: Syntax error expecting number.
+ 16 | EXAMINE x/CINTERVAL **.
+ | ^~"
+
+"examine.sps:17.12-17.13: error: EXAMINE: Syntax error expecting one of the following: STATISTICS, PERCENTILES, TOTAL, NOTOTAL, MISSING, COMPARE, PLOT, CINTERVAL, ID.
+ 17 | EXAMINE x/ **.
+ | ^~"
+
+"examine.sps:18.17-18.23: error: EXAMINE: TOTAL and NOTOTAL are mutually exclusive.
+ 18 | EXAMINE x/TOTAL/NOTOTAL.
+ | ^~~~~~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([FACTOR procedure])
+
+AT_SETUP([FACTOR extraction=paf method=correlation])
+dnl This example is based on data from http://www.ats.ucla.edu/stat/Spss/output/factor1.htm
+
+AT_DATA([factor.sps],
+ [set format = F11.3.
+
+data list notable fixed /question13 to question24 1-12.
+begin data.
+555555535543
+544453434443
+545555544444
+444442444433
+554545554554
+554455454455
+555554555244
+554455544443
+555554434344
+544454544344
+555545555555
+555454544455
+555445544455
+455544443343
+544454344344
+555555555455
+555554454455
+555555554445
+555555554555
+545534553343
+555555535554
+555544444445
+545544334433
+554555434443
+555555544454
+555445545453
+555554434244
+444444433233
+545555454443
+554443434243
+444534334333
+454534444332
+555455353444
+555544443243
+555554543243
+555544444343
+445444434443
+555555555544
+444444434340
+455044434334
+555555533433
+554554535040
+434533334232
+443232444432
+555555555555
+555555554544
+555544444445
+444224343344
+444554454355
+444434332433
+555555555555
+043243432433
+444443334333
+453443433434
+443342332232
+554434434533
+444344434443
+444444434443
+554552434133
+453334332432
+444445554444
+431232332223
+555555555544
+544445543443
+444455535543
+444444444433
+444444543243
+555431443333
+444443433433
+444433433443
+454334444433
+111111111544
+444423442433
+555443333353
+555543334344
+055454400000
+555454444355
+555555555555
+055544533333
+555554555554
+555555535554
+555555545355
+555555555455
+555544545543
+555554404455
+555454435454
+555555535554
+555555555555
+555443433544
+555554435454
+555555545344
+555555535454
+555445535453
+444444333544
+555554434444
+455454434454
+555555535555
+554545534455
+555555555443
+454443434444
+555553334444
+555554545454
+555555555555
+555554554454
+555555555555
+544545534544
+555555534454
+555555555455
+555554535544
+555555535555
+555451234443
+555444444544
+555544434354
+555545533444
+555554534443
+545554434554
+554433444433
+544432233524
+411111111111
+445423442233
+555543433344
+444443342233
+555555534455
+545442434223
+554553352333
+544554554445
+555555435455
+444334304234
+455453444434
+444443443245
+555552232132
+555434324345
+544444434344
+534344344444
+445555555344
+444343442132
+444444434344
+444444554334
+444545444333
+434442343224
+443443433233
+555551555554
+544544434444
+454544434433
+555555535434
+555555555555
+455544444444
+454444434233
+555555544344
+454445544445
+444444554434
+555455455443
+555454425444
+444454434443
+544443433233
+444543434433
+555553545354
+544444444433
+444445434433
+555533353333
+555434234333
+444314222411
+555555555555
+455545534443
+455045534433
+555545444444
+455544243543
+444421113343
+554444534444
+555555544554
+555334434452
+555544543455
+555554434554
+555445554454
+555555545344
+555555555555
+554543334245
+554441233333
+555554334545
+555555535544
+555555554554
+555445444543
+444424544432
+444425544333
+555434344443
+555533233223
+544433442343
+555555555555
+555445452234
+555444555444
+555444544455
+555544354554
+545445555555
+555555555555
+455443342232
+555555544454
+455534443455
+555555555055
+444554333244
+444445434543
+555554544455
+444443444434
+444444434445
+334231314323
+444444434433
+555554444443
+444444434443
+555455434444
+444444444444
+555455435455
+444444434344
+444543433232
+544443334454
+555544433244
+534443324224
+555555544444
+444443434444
+555553444344
+443434443333
+444444434333
+554445444343
+444443344434
+445555544543
+555554443443
+322232403322
+444444444433
+444445444443
+444454534445
+544344444344
+554445534544
+555555555554
+555544432333
+444444444443
+555555555554
+455555555554
+555555555543
+555555555544
+555554445555
+444335544455
+555555555555
+555454444454
+555455544454
+444445444444
+555555555555
+555455454554
+555454544554
+555555555455
+555555554444
+444444544434
+334334443333
+422224222211
+121512011111
+443444432332
+434335533344
+444443543433
+554454443453
+555555545454
+555555555344
+555555555554
+555555334555
+454445554444
+454545534444
+555554555444
+455444534455
+555544554544
+444444444444
+555424423133
+222433443224
+544544532344
+545554544544
+555532544144
+444432343433
+434545554545
+444344505443
+555555555544
+555554555344
+444531113112
+555554333133
+455433233233
+354354434345
+454534433433
+433112332321
+555445433333
+454343434143
+555554445555
+455423334322
+443333332222
+444443444443
+553432224134
+444223432233
+454324425444
+344434514443
+555552415255
+354332543353
+444531333233
+443433334133
+444444444433
+444444445533
+445133212223
+343433432213
+442333333332
+345455535244
+443211121122
+555445444444
+555555544344
+453243332232
+443543422533
+554444434344
+554444333444
+555555335343
+444231121133
+454433332233
+455524434244
+554433224354
+555455555444
+454444424444
+554242232134
+555553315443
+555553333454
+455421434211
+111111151111
+454443203123
+454243334132
+355332213144
+554534334134
+555543434355
+555543433255
+444441223334
+555443143255
+444444444454
+444442323544
+454443434343
+445453434444
+454455554433
+554532342234
+444442243233
+444442343334
+555443433444
+555543253444
+554554444455
+555543355344
+555444445444
+434443204222
+444432444234
+222142441111
+111111141111
+334334454433
+555354552543
+433411122112
+455534504444
+543211224233
+344333332532
+344443233333
+444424432434
+555555544454
+555555554544
+555444444444
+555443533443
+555554544344
+555554444444
+544543433343
+444445434444
+555555555555
+555443333233
+545444534454
+555454434343
+555453434544
+544334444333
+555443443444
+555555534544
+444444444433
+555543433343
+555444445334
+555543442433
+455444334443
+554443434443
+555254500544
+555444433344
+555555535544
+444443532232
+544443433433
+555555535544
+555455454444
+555455445544
+544444534433
+555555555554
+555553533444
+555555555554
+555554434343
+555455544443
+555554444443
+555454444445
+333222333223
+444443544233
+543443534433
+555545454533
+444444404033
+455454504543
+455555555454
+454443344343
+555553435244
+555543544444
+555553343044
+444443444433
+445543434434
+555554433545
+555554455445
+455553333234
+555552344243
+444141212213
+444443504234
+445544253444
+555554354555
+534552234543
+555554544544
+444233404224
+555554534444
+444443444344
+422442434324
+554434434344
+444444434334
+443433432444
+555454435344
+252423332214
+454544434434
+444444444244
+555554544445
+443222432333
+555544444332
+545555543445
+554544334444
+555445555544
+554343434433
+555555535554
+444554444333
+553544343534
+555553254433
+555555554554
+544443443344
+555443444344
+544432144123
+555555555555
+555555555354
+555555535545
+455454434444
+455455534445
+455555554455
+555553545445
+445545544444
+545345553555
+444445444433
+443435433433
+555554545353
+444443232223
+444444434433
+555555545454
+555554444343
+555554444443
+434544333422
+444443434343
+555555455443
+555544434343
+455545534444
+555555555553
+444443343434
+555555555445
+433444444434
+344221512132
+333421132223
+333444443444
+435544545533
+444335544443
+333323223323
+334434332333
+444422323213
+324433443423
+555555554555
+555452424444
+555544434444
+555544443343
+555445545453
+555555344453
+555544545550
+454443434334
+444332233344
+555554332344
+554444433444
+555554555554
+555555543544
+444442443132
+555555545555
+544553435533
+044044444444
+444443434533
+555454433434
+555555555454
+555555555555
+454544434444
+444444434333
+554555455554
+555555543444
+423331124132
+445445534455
+555555555554
+444535554434
+555555555554
+354443422232
+555545535443
+555555555555
+555455444453
+455434443333
+455444443233
+343322434411
+555555555534
+453442432333
+445554434544
+345444443333
+454554554355
+444434453434
+555555454443
+555443343341
+545553343433
+222343222201
+444433434443
+535555555544
+552541442423
+545433343334
+455445433443
+444444433433
+455543343433
+434444432333
+444545444444
+555554534444
+452444434433
+454443432533
+555453334433
+444442233432
+555555553433
+545555545445
+335543222333
+554554344445
+243424442212
+133222432411
+343434534233
+222222212211
+445455554434
+455554453344
+223334424434
+555355555544
+455544554434
+544455555444
+444444444444
+444444444344
+444334444444
+444444444434
+555455534243
+555555545555
+555555555555
+444443534343
+555554544444
+555555555454
+444434433433
+455445544444
+555544544455
+444333433333
+455443445343
+444432442444
+443334451543
+555554455555
+555444444444
+555555555455
+555555555554
+555454345154
+555555555555
+555555555455
+555554454544
+454444454544
+554443334544
+455555402535
+555554531534
+455545544554
+444423444223
+555444334533
+455554445243
+453444334344
+555555555554
+555555555455
+555555555554
+455443334344
+555555253555
+555554433454
+555444344455
+550030034433
+555444233343
+555343222133
+555555555554
+555555243243
+555555555355
+555554345555
+555443434454
+545543133133
+555443334154
+444444344454
+555555242254
+555554444344
+545443334454
+554444132454
+555455143154
+555554453044
+555555555455
+555534335454
+555555245555
+545543333444
+555555355454
+555353145133
+555553043454
+555555354554
+555543434454
+555444324454
+444444203443
+555552233355
+555555445455
+555500034354
+555354354444
+555555555555
+555543334144
+555555005254
+454444344254
+555555555554
+555555555255
+555555555455
+444444242243
+555554445154
+444444234333
+555555553455
+444422224243
+555545443344
+545552133143
+555455044344
+555555455555
+555555545454
+454433343144
+555555555555
+555455255155
+555555454455
+555555424455
+555555355555
+555444444455
+555555454455
+544411211314
+223322441123
+444223434233
+444441121232
+555555535555
+454445533444
+434442434433
+545355554454
+555542544333
+445545555444
+321000001011
+444444444444
+333223321322
+444232433233
+425432523122
+455555555544
+555555545555
+555555534354
+554554444243
+555554343443
+444443434333
+555224252443
+555544433433
+344544434423
+555554434344
+555542134233
+444344434444
+445443433233
+455343333434
+455443333445
+355344434433
+234234433333
+445444444445
+455535535545
+443423234443
+455544334544
+345441333323
+445444533433
+455554443355
+445444433243
+455454453444
+444244444444
+554244544154
+555555555555
+454343444444
+555444444344
+545455534454
+555555555555
+555554534454
+555254555444
+544354544453
+553454534445
+555454445355
+545253554454
+433342322233
+544444444443
+555455445544
+424322433233
+444424222233
+555355555544
+000000000000
+555455345344
+000055505450
+434333444333
+444445444444
+555555555545
+555554455545
+555455435533
+544544444243
+444331232323
+555555554444
+443332323233
+444224342433
+555555555544
+433443342333
+555445504554
+555545555544
+555555535555
+555555535345
+454443433333
+555444444555
+444443232435
+555433444443
+555443434543
+555555535555
+555555533544
+555543234444
+433432332221
+443433544233
+443443333334
+444424433444
+552444333123
+233332232211
+223422221122
+434433414133
+332323333102
+552544223222
+542423343232
+555552534132
+455554544134
+433523533132
+433333433433
+435434543333
+434553433444
+555544434345
+443543443433
+555555542344
+544444553144
+555544544243
+535443441342
+344555444333
+444444443333
+443443433432
+545554534544
+443533433433
+333443432223
+333333433123
+322432122213
+555555555555
+554555444433
+444543443234
+444444433433
+452555534433
+244444444233
+433442422232
+555555535555
+555555554455
+555554545244
+555555355455
+555555555555
+555555555555
+444433323233
+555455455455
+434445444433
+555554444455
+454444543445
+555555535455
+555545554455
+555555555555
+555555555555
+555443344353
+455543304132
+444443444433
+555555555355
+434433443333
+444434444433
+444444444433
+555555555555
+445553443323
+445444444444
+555554444054
+455555554543
+555555535555
+555554445454
+555444444443
+555555555454
+555554344455
+555555454454
+554444444454
+555555555353
+555545544454
+553545332223
+444424444332
+545555543433
+555444433444
+555555444445
+444444424433
+444444543434
+124113531311
+555554534543
+555343333333
+545444544344
+555444534444
+555544543444
+555455544443
+444324402121
+555554534544
+555455544444
+555555544454
+444334404433
+555443534444
+555545554444
+555555555555
+455333233433
+455444433433
+455444444444
+444235442443
+555443343433
+444445453444
+454345453432
+555555453444
+344433322323
+444443444244
+444442343133
+554445432233
+555555544444
+555555534444
+555554455554
+454443334244
+544443333233
+444445534445
+444432134121
+555555332243
+555555544433
+555554434444
+454543534233
+454432432343
+444424432433
+545553335344
+555443434344
+454443433333
+555553534444
+554544434355
+552532421235
+454543433434
+544544343234
+555552334125
+555543455555
+443442334222
+554443444344
+555554543334
+555552342444
+554443433333
+443444434445
+555554533344
+442412242121
+454543343244
+445554433344
+444443333433
+444443433333
+444433333334
+444444334444
+432321102223
+444444434443
+444444434343
+454531432331
+445543433434
+554554434554
+334253232333
+444443434244
+444433443234
+444433334334
+444443433333
+553434303222
+454443434244
+444445544444
+443441133433
+444432232133
+444444404344
+444444333243
+455543124243
+555544532344
+444432333132
+554553434244
+454443443333
+433111121111
+555555555544
+444432433222
+444443433233
+443332332133
+445344453243
+444444405434
+554554434343
+455344534443
+444444400434
+444444344344
+544554533443
+555554443455
+555555544544
+555554534444
+323123232311
+444344344443
+555554555544
+343323332333
+444443434444
+444442214340
+555434434444
+455543343444
+445432434433
+455553434455
+243321332322
+444422332332
+555533454444
+555544443433
+354422431422
+333322421211
+444443432434
+344422431322
+333342222321
+444443454433
+443443444433
+553434531334
+554434552343
+545455553544
+554455554443
+555555555533
+555555554543
+454454544433
+555444532143
+554545544443
+444233442434
+544444434234
+554344432233
+555345533355
+554554544433
+455444444344
+555554554554
+555554545555
+444433534434
+444444421134
+334333333333
+334443443343
+122333441413
+434444333333
+444344433233
+444333332143
+555154344133
+324344333223
+244444402233
+454443433543
+444344433344
+455555445555
+555555544433
+454544434443
+344535554533
+333435443433
+444444554544
+343434443333
+544553544455
+555444444455
+244333332222
+333441232233
+544433433433
+555544343344
+344211142124
+442442232113
+433432332223
+333424322222
+444443333233
+344321232223
+442434342422
+545555535555
+455454434454
+455355544444
+454444444445
+555554432430
+555444344144
+455534342234
+555555554354
+213332443111
+555545434433
+555554424444
+443434443433
+555555554443
+555555555544
+435535554433
+555455545443
+555555554533
+554545443455
+555553333233
+434432232323
+443443333433
+443544231534
+434533334334
+555555535544
+555544444445
+555555555555
+555555555555
+555555555555
+555555555555
+544555544544
+555555545555
+545555534354
+445444344344
+555555555555
+444443343334
+555444440000
+555555555554
+555555545455
+454434444345
+555445544443
+555554535433
+454445555555
+555444444355
+555555555555
+555555555555
+555555555555
+555555554555
+555455554455
+454435544255
+545543342243
+555544355345
+555555455354
+435553244333
+555555443454
+444444433445
+121422433111
+555554543244
+555444554444
+444203444433
+344342553322
+554445554344
+545445454454
+444245504233
+334335555533
+554355544444
+444445545444
+555555555344
+454544543233
+444455532434
+555544354243
+535444554433
+444444444444
+455555553243
+244442343235
+554544504043
+444435553433
+455553434354
+555444343314
+555553344453
+555555555544
+544444444433
+434434550033
+555455544444
+455445534344
+454445444554
+555555555555
+555555555555
+555555555454
+555455545412
+334433343132
+555454455455
+555554454443
+555555555555
+555555553433
+555555555555
+555555445555
+555545445455
+555555545554
+555555555555
+445455554443
+555445554433
+555555554445
+555553333144
+555554455143
+554454445444
+555555554533
+334422433422
+555554434444
+433531133222
+443432342224
+555544554433
+553434333333
+555543334443
+444443344323
+555555555555
+555554545344
+555555545554
+555342434333
+444443333233
+444544435444
+555555555555
+555455554354
+555555405054
+555555555555
+554555545545
+555555555544
+555545454344
+555555455555
+433444444343
+454555554444
+453555554544
+444445554444
+544455555455
+555343434343
+555553444454
+555444434443
+555555555444
+555555545443
+555444534455
+555555555555
+444443443434
+555445533543
+555555555444
+555544444354
+445444544243
+555554555455
+555555455444
+555555555544
+555555555454
+554445545454
+555555555543
+344444434443
+555555454453
+455444443433
+344244434433
+355234452132
+555445545455
+444444444444
+455444454433
+555555555555
+344233341155
+333334433233
+444433434333
+444233443334
+454344544444
+555554245253
+444444333344
+554544434333
+555444443343
+555554555445
+005555555544
+554444445244
+555455555555
+555555555444
+444444444433
+555555555444
+555555544454
+555555555544
+444444434444
+555555535553
+545534543334
+554554534533
+555555555554
+554554544544
+454541231221
+555555535544
+445344334432
+444444344333
+555555534444
+555555545555
+443434544232
+545544433343
+343234434333
+444444443233
+555454444455
+555455535455
+554433442243
+444444544454
+554545554544
+454444444444
+554455534455
+555555544355
+555445555555
+555444534444
+455454534444
+555555555544
+444444444455
+454455554344
+443244442233
+350554554434
+455444535343
+344233443433
+454444535545
+244222232232
+245345554344
+355344444443
+555455555444
+444434444233
+334323444322
+333234443233
+455455554454
+555554544434
+555355555444
+444444544443
+555554555544
+344231224131
+444443434433
+454344444431
+555554555553
+454544444443
+444443443532
+334323423222
+455555545554
+555542434443
+554444444443
+554444534344
+555443454555
+555555534554
+444434443343
+444234432233
+334323311333
+444443443233
+355424552242
+233335323322
+234233443333
+123353532334
+444345555244
+332222433422
+545445555443
+555555555555
+555554444443
+455445554455
+455455555455
+555555555555
+444443433445
+555455555544
+355453434232
+555555554344
+444433433433
+434322242112
+444444433433
+444445444544
+555444535444
+544444444443
+454544344334
+454444334333
+434433433332
+334434423423
+444455552233
+442442342233
+445433433343
+555555444355
+555555555555
+455554555545
+555445544444
+544444434443
+555555535333
+554444444344
+555554554445
+345433334121
+555555555554
+545444444233
+555555544455
+555554545555
+555555455555
+555555545555
+555554544455
+444444444333
+544544544445
+555445544544
+555455444455
+555455555454
+554455535444
+453423442244
+444443444444
+454444333333
+555555555542
+555555555543
+555555555544
+555555535534
+555555555444
+444443423333
+555444444444
+555445544453
+444444444443
+555555443444
+444443444333
+554554154344
+555543353333
+554445443333
+555555553455
+554534444243
+554555554545
+555544443443
+555554555555
+555425552422
+555555555555
+454421121321
+555454453433
+555555554443
+244224431223
+455444453444
+454345544455
+344235545044
+555555555544
+555553325554
+554554444244
+555545544544
+454555554545
+444444545444
+545544444455
+454344445443
+545555554453
+444444444443
+545554554453
+555453434444
+005434434454
+455555455544
+555555555554
+455542444433
+545543444555
+445545445444
+555553325454
+555554555554
+444544444443
+555555444454
+443332443222
+454444443444
+344333433332
+555455444455
+555555555445
+555555534454
+445433432343
+555554434444
+555555545444
+544545434455
+454435543444
+555555555555
+555455554555
+555555544554
+555553554455
+555555555555
+545454545444
+555555445555
+554534343444
+555545443343
+545454334444
+554445544544
+555455354344
+555555555355
+554445544354
+455554444444
+555555553543
+555554444444
+555543233444
+555554344433
+333334441223
+543554344434
+055541243244
+555555555544
+555555533444
+555445444544
+444335343433
+544434344333
+544435533333
+444444443333
+555555555554
+555552343233
+444444444433
+544444434444
+555555545555
+555555555554
+445244434444
+445444434444
+445555554521
+443444543343
+444433343434
+355444433442
+555543444455
+555444334544
+555555555555
+554555545555
+555555554555
+555555555555
+454444343445
+444444433444
+354445443444
+555334242132
+455445555543
+254153343433
+354244443333
+554455444344
+343255535444
+455454555553
+555455545555
+444343433343
+343323443323
+455444444424
+555544444455
+343434543444
+555555555544
+555554534243
+555554543344
+555455555544
+555443344343
+444445533133
+555543534555
+444554444444
+444002323320
+444232322222
+244344424441
+555443344334
+555555555555
+555444534443
+555555555555
+555555555550
+455555554555
+555555555555
+
+end data.
+
+missing values
+ question13 question14 question15 question16 question17 question18 question19 question20 question21 question22 question23 question24 (0).
+
+factor
+ /variables question13 question14 question15 question16 question17 question18 question19 question20 question21 question22 question23 question24
+ /analysis all
+ /print univariate det correlation
+ /format blank(.30)
+ /plot eigen
+ /criteria factors(3)
+ /extraction paf
+ /method = correlation.
+])
+
+AT_CHECK([pspp -O format=csv factor.sps], [0],
+ [Table: Descriptive Statistics
+,Mean,Std. Deviation,Analysis N
+question13,4.462,.729,1365
+question14,4.525,.700,1365
+question15,4.445,.732,1365
+question16,4.281,.829,1365
+question17,4.166,.895,1365
+question18,3.930,1.034,1365
+question19,4.077,.963,1365
+question20,3.777,.909,1365
+question21,3.774,.984,1365
+question22,3.607,1.116,1365
+question23,3.813,.957,1365
+question24,3.666,.926,1365
+
+Table: Correlation Matrix
+,,question13,question14,question15,question16,question17,question18,question19,question20,question21,question22,question23,question24
+Correlation,question13,1.000,.661,.600,.566,.577,.409,.286,.304,.476,.333,.564,.454
+,question14,.661,1.000,.635,.500,.552,.433,.320,.315,.449,.333,.565,.443
+,question15,.600,.635,1.000,.505,.587,.457,.359,.356,.509,.369,.582,.435
+,question16,.566,.500,.505,1.000,.586,.405,.335,.317,.452,.363,.459,.430
+,question17,.577,.552,.587,.586,1.000,.555,.449,.417,.595,.450,.613,.521
+,question18,.409,.433,.457,.405,.555,1.000,.627,.521,.554,.536,.569,.474
+,question19,.286,.320,.359,.335,.449,.627,1.000,.446,.499,.484,.444,.374
+,question20,.304,.315,.356,.317,.417,.521,.446,1.000,.425,.383,.410,.357
+,question21,.476,.449,.509,.452,.595,.554,.499,.425,1.000,.507,.598,.500
+,question22,.333,.333,.369,.363,.450,.536,.484,.383,.507,1.000,.493,.444
+,question23,.564,.565,.582,.459,.613,.569,.444,.410,.598,.493,1.000,.705
+,question24,.454,.443,.435,.430,.521,.474,.374,.357,.500,.444,.705,1.000
+Caption: Determinant: 0.00
+
+Table: Factor Matrix
+,Factor,,
+,1,2,3
+question13,.713,.398,
+question14,.703,.339,
+question15,.721,,
+question16,.648,,
+question17,.783,,
+question18,.740,-.345,
+question19,.616,-.415,
+question20,.550,,
+question21,.732,,
+question22,.613,,
+question23,.819,,.345
+question24,.695,,.386
+
+Table: Rotated Factor Matrix
+,Factor,,
+,1,2,3
+question13,.771,,
+question14,.726,,
+question15,.676,,
+question16,.591,,
+question17,.587,.446,
+question18,,.739,
+question19,,.727,
+question20,,.540,
+question21,.402,.533,.321
+question22,,.559,
+question23,.449,.377,.668
+question24,.324,.321,.652
+])
+
+AT_CLEANUP
+
+AT_SETUP([FACTOR extraction=pc method=correlation])
+dnl This example is from http://www.ats.ucla.edu/stat/spss/whatstat/whatstat.htm
+
+AT_DATA([factor2.sps],
+ [set format = F11.3.
+
+
+data list notable list /id female race ses schtyp prog read write math science socst.
+begin data.
+ 70.00 .00 4.00 1.00 1.00 1.00 57.00 52.00 41.00 47.00 57.00
+121.00 1.00 4.00 2.00 1.00 3.00 68.00 59.00 53.00 63.00 61.00
+ 86.00 .00 4.00 3.00 1.00 1.00 44.00 33.00 54.00 58.00 31.00
+141.00 .00 4.00 3.00 1.00 3.00 63.00 44.00 47.00 53.00 56.00
+172.00 .00 4.00 2.00 1.00 2.00 47.00 52.00 57.00 53.00 61.00
+113.00 .00 4.00 2.00 1.00 2.00 44.00 52.00 51.00 63.00 61.00
+ 50.00 .00 3.00 2.00 1.00 1.00 50.00 59.00 42.00 53.00 61.00
+ 11.00 .00 1.00 2.00 1.00 2.00 34.00 46.00 45.00 39.00 36.00
+ 84.00 .00 4.00 2.00 1.00 1.00 63.00 57.00 54.00 58.00 51.00
+ 48.00 .00 3.00 2.00 1.00 2.00 57.00 55.00 52.00 50.00 51.00
+ 75.00 .00 4.00 2.00 1.00 3.00 60.00 46.00 51.00 53.00 61.00
+ 60.00 .00 4.00 2.00 1.00 2.00 57.00 65.00 51.00 63.00 61.00
+ 95.00 .00 4.00 3.00 1.00 2.00 73.00 60.00 71.00 61.00 71.00
+104.00 .00 4.00 3.00 1.00 2.00 54.00 63.00 57.00 55.00 46.00
+ 38.00 .00 3.00 1.00 1.00 2.00 45.00 57.00 50.00 31.00 56.00
+115.00 .00 4.00 1.00 1.00 1.00 42.00 49.00 43.00 50.00 56.00
+ 76.00 .00 4.00 3.00 1.00 2.00 47.00 52.00 51.00 50.00 56.00
+195.00 .00 4.00 2.00 2.00 1.00 57.00 57.00 60.00 58.00 56.00
+114.00 .00 4.00 3.00 1.00 2.00 68.00 65.00 62.00 55.00 61.00
+ 85.00 .00 4.00 2.00 1.00 1.00 55.00 39.00 57.00 53.00 46.00
+167.00 .00 4.00 2.00 1.00 1.00 63.00 49.00 35.00 66.00 41.00
+143.00 .00 4.00 2.00 1.00 3.00 63.00 63.00 75.00 72.00 66.00
+ 41.00 .00 3.00 2.00 1.00 2.00 50.00 40.00 45.00 55.00 56.00
+ 20.00 .00 1.00 3.00 1.00 2.00 60.00 52.00 57.00 61.00 61.00
+ 12.00 .00 1.00 2.00 1.00 3.00 37.00 44.00 45.00 39.00 46.00
+ 53.00 .00 3.00 2.00 1.00 3.00 34.00 37.00 46.00 39.00 31.00
+154.00 .00 4.00 3.00 1.00 2.00 65.00 65.00 66.00 61.00 66.00
+178.00 .00 4.00 2.00 2.00 3.00 47.00 57.00 57.00 58.00 46.00
+196.00 .00 4.00 3.00 2.00 2.00 44.00 38.00 49.00 39.00 46.00
+ 29.00 .00 2.00 1.00 1.00 1.00 52.00 44.00 49.00 55.00 41.00
+126.00 .00 4.00 2.00 1.00 1.00 42.00 31.00 57.00 47.00 51.00
+103.00 .00 4.00 3.00 1.00 2.00 76.00 52.00 64.00 64.00 61.00
+192.00 .00 4.00 3.00 2.00 2.00 65.00 67.00 63.00 66.00 71.00
+150.00 .00 4.00 2.00 1.00 3.00 42.00 41.00 57.00 72.00 31.00
+199.00 .00 4.00 3.00 2.00 2.00 52.00 59.00 50.00 61.00 61.00
+144.00 .00 4.00 3.00 1.00 1.00 60.00 65.00 58.00 61.00 66.00
+200.00 .00 4.00 2.00 2.00 2.00 68.00 54.00 75.00 66.00 66.00
+ 80.00 .00 4.00 3.00 1.00 2.00 65.00 62.00 68.00 66.00 66.00
+ 16.00 .00 1.00 1.00 1.00 3.00 47.00 31.00 44.00 36.00 36.00
+153.00 .00 4.00 2.00 1.00 3.00 39.00 31.00 40.00 39.00 51.00
+176.00 .00 4.00 2.00 2.00 2.00 47.00 47.00 41.00 42.00 51.00
+177.00 .00 4.00 2.00 2.00 2.00 55.00 59.00 62.00 58.00 51.00
+168.00 .00 4.00 2.00 1.00 2.00 52.00 54.00 57.00 55.00 51.00
+ 40.00 .00 3.00 1.00 1.00 1.00 42.00 41.00 43.00 50.00 41.00
+ 62.00 .00 4.00 3.00 1.00 1.00 65.00 65.00 48.00 63.00 66.00
+169.00 .00 4.00 1.00 1.00 1.00 55.00 59.00 63.00 69.00 46.00
+ 49.00 .00 3.00 3.00 1.00 3.00 50.00 40.00 39.00 49.00 47.00
+136.00 .00 4.00 2.00 1.00 2.00 65.00 59.00 70.00 63.00 51.00
+189.00 .00 4.00 2.00 2.00 2.00 47.00 59.00 63.00 53.00 46.00
+ 7.00 .00 1.00 2.00 1.00 2.00 57.00 54.00 59.00 47.00 51.00
+ 27.00 .00 2.00 2.00 1.00 2.00 53.00 61.00 61.00 57.00 56.00
+128.00 .00 4.00 3.00 1.00 2.00 39.00 33.00 38.00 47.00 41.00
+ 21.00 .00 1.00 2.00 1.00 1.00 44.00 44.00 61.00 50.00 46.00
+183.00 .00 4.00 2.00 2.00 2.00 63.00 59.00 49.00 55.00 71.00
+132.00 .00 4.00 2.00 1.00 2.00 73.00 62.00 73.00 69.00 66.00
+ 15.00 .00 1.00 3.00 1.00 3.00 39.00 39.00 44.00 26.00 42.00
+ 67.00 .00 4.00 1.00 1.00 3.00 37.00 37.00 42.00 33.00 32.00
+ 22.00 .00 1.00 2.00 1.00 3.00 42.00 39.00 39.00 56.00 46.00
+185.00 .00 4.00 2.00 2.00 2.00 63.00 57.00 55.00 58.00 41.00
+ 9.00 .00 1.00 2.00 1.00 3.00 48.00 49.00 52.00 44.00 51.00
+181.00 .00 4.00 2.00 2.00 2.00 50.00 46.00 45.00 58.00 61.00
+170.00 .00 4.00 3.00 1.00 2.00 47.00 62.00 61.00 69.00 66.00
+134.00 .00 4.00 1.00 1.00 1.00 44.00 44.00 39.00 34.00 46.00
+108.00 .00 4.00 2.00 1.00 1.00 34.00 33.00 41.00 36.00 36.00
+197.00 .00 4.00 3.00 2.00 2.00 50.00 42.00 50.00 36.00 61.00
+140.00 .00 4.00 2.00 1.00 3.00 44.00 41.00 40.00 50.00 26.00
+171.00 .00 4.00 2.00 1.00 2.00 60.00 54.00 60.00 55.00 66.00
+107.00 .00 4.00 1.00 1.00 3.00 47.00 39.00 47.00 42.00 26.00
+ 81.00 .00 4.00 1.00 1.00 2.00 63.00 43.00 59.00 65.00 44.00
+ 18.00 .00 1.00 2.00 1.00 3.00 50.00 33.00 49.00 44.00 36.00
+155.00 .00 4.00 2.00 1.00 1.00 44.00 44.00 46.00 39.00 51.00
+ 97.00 .00 4.00 3.00 1.00 2.00 60.00 54.00 58.00 58.00 61.00
+ 68.00 .00 4.00 2.00 1.00 2.00 73.00 67.00 71.00 63.00 66.00
+157.00 .00 4.00 2.00 1.00 1.00 68.00 59.00 58.00 74.00 66.00
+ 56.00 .00 4.00 2.00 1.00 3.00 55.00 45.00 46.00 58.00 51.00
+ 5.00 .00 1.00 1.00 1.00 2.00 47.00 40.00 43.00 45.00 31.00
+159.00 .00 4.00 3.00 1.00 2.00 55.00 61.00 54.00 49.00 61.00
+123.00 .00 4.00 3.00 1.00 1.00 68.00 59.00 56.00 63.00 66.00
+164.00 .00 4.00 2.00 1.00 3.00 31.00 36.00 46.00 39.00 46.00
+ 14.00 .00 1.00 3.00 1.00 2.00 47.00 41.00 54.00 42.00 56.00
+127.00 .00 4.00 3.00 1.00 2.00 63.00 59.00 57.00 55.00 56.00
+165.00 .00 4.00 1.00 1.00 3.00 36.00 49.00 54.00 61.00 36.00
+174.00 .00 4.00 2.00 2.00 2.00 68.00 59.00 71.00 66.00 56.00
+ 3.00 .00 1.00 1.00 1.00 2.00 63.00 65.00 48.00 63.00 56.00
+ 58.00 .00 4.00 2.00 1.00 3.00 55.00 41.00 40.00 44.00 41.00
+146.00 .00 4.00 3.00 1.00 2.00 55.00 62.00 64.00 63.00 66.00
+102.00 .00 4.00 3.00 1.00 2.00 52.00 41.00 51.00 53.00 56.00
+117.00 .00 4.00 3.00 1.00 3.00 34.00 49.00 39.00 42.00 56.00
+133.00 .00 4.00 2.00 1.00 3.00 50.00 31.00 40.00 34.00 31.00
+ 94.00 .00 4.00 3.00 1.00 2.00 55.00 49.00 61.00 61.00 56.00
+ 24.00 .00 2.00 2.00 1.00 2.00 52.00 62.00 66.00 47.00 46.00
+149.00 .00 4.00 1.00 1.00 1.00 63.00 49.00 49.00 66.00 46.00
+ 82.00 1.00 4.00 3.00 1.00 2.00 68.00 62.00 65.00 69.00 61.00
+ 8.00 1.00 1.00 1.00 1.00 2.00 39.00 44.00 52.00 44.00 48.00
+129.00 1.00 4.00 1.00 1.00 1.00 44.00 44.00 46.00 47.00 51.00
+173.00 1.00 4.00 1.00 1.00 1.00 50.00 62.00 61.00 63.00 51.00
+ 57.00 1.00 4.00 2.00 1.00 2.00 71.00 65.00 72.00 66.00 56.00
+100.00 1.00 4.00 3.00 1.00 2.00 63.00 65.00 71.00 69.00 71.00
+ 1.00 1.00 1.00 1.00 1.00 3.00 34.00 44.00 40.00 39.00 41.00
+194.00 1.00 4.00 3.00 2.00 2.00 63.00 63.00 69.00 61.00 61.00
+ 88.00 1.00 4.00 3.00 1.00 2.00 68.00 60.00 64.00 69.00 66.00
+ 99.00 1.00 4.00 3.00 1.00 1.00 47.00 59.00 56.00 66.00 61.00
+ 47.00 1.00 3.00 1.00 1.00 2.00 47.00 46.00 49.00 33.00 41.00
+120.00 1.00 4.00 3.00 1.00 2.00 63.00 52.00 54.00 50.00 51.00
+166.00 1.00 4.00 2.00 1.00 2.00 52.00 59.00 53.00 61.00 51.00
+ 65.00 1.00 4.00 2.00 1.00 2.00 55.00 54.00 66.00 42.00 56.00
+101.00 1.00 4.00 3.00 1.00 2.00 60.00 62.00 67.00 50.00 56.00
+ 89.00 1.00 4.00 1.00 1.00 3.00 35.00 35.00 40.00 51.00 33.00
+ 54.00 1.00 3.00 1.00 2.00 1.00 47.00 54.00 46.00 50.00 56.00
+180.00 1.00 4.00 3.00 2.00 2.00 71.00 65.00 69.00 58.00 71.00
+162.00 1.00 4.00 2.00 1.00 3.00 57.00 52.00 40.00 61.00 56.00
+ 4.00 1.00 1.00 1.00 1.00 2.00 44.00 50.00 41.00 39.00 51.00
+131.00 1.00 4.00 3.00 1.00 2.00 65.00 59.00 57.00 46.00 66.00
+125.00 1.00 4.00 1.00 1.00 2.00 68.00 65.00 58.00 59.00 56.00
+ 34.00 1.00 1.00 3.00 2.00 2.00 73.00 61.00 57.00 55.00 66.00
+106.00 1.00 4.00 2.00 1.00 3.00 36.00 44.00 37.00 42.00 41.00
+130.00 1.00 4.00 3.00 1.00 1.00 43.00 54.00 55.00 55.00 46.00
+ 93.00 1.00 4.00 3.00 1.00 2.00 73.00 67.00 62.00 58.00 66.00
+163.00 1.00 4.00 1.00 1.00 2.00 52.00 57.00 64.00 58.00 56.00
+ 37.00 1.00 3.00 1.00 1.00 3.00 41.00 47.00 40.00 39.00 51.00
+ 35.00 1.00 1.00 1.00 2.00 1.00 60.00 54.00 50.00 50.00 51.00
+ 87.00 1.00 4.00 2.00 1.00 1.00 50.00 52.00 46.00 50.00 56.00
+ 73.00 1.00 4.00 2.00 1.00 2.00 50.00 52.00 53.00 39.00 56.00
+151.00 1.00 4.00 2.00 1.00 3.00 47.00 46.00 52.00 48.00 46.00
+ 44.00 1.00 3.00 1.00 1.00 3.00 47.00 62.00 45.00 34.00 46.00
+152.00 1.00 4.00 3.00 1.00 2.00 55.00 57.00 56.00 58.00 61.00
+105.00 1.00 4.00 2.00 1.00 2.00 50.00 41.00 45.00 44.00 56.00
+ 28.00 1.00 2.00 2.00 1.00 1.00 39.00 53.00 54.00 50.00 41.00
+ 91.00 1.00 4.00 3.00 1.00 3.00 50.00 49.00 56.00 47.00 46.00
+ 45.00 1.00 3.00 1.00 1.00 3.00 34.00 35.00 41.00 29.00 26.00
+116.00 1.00 4.00 2.00 1.00 2.00 57.00 59.00 54.00 50.00 56.00
+ 33.00 1.00 2.00 1.00 1.00 2.00 57.00 65.00 72.00 54.00 56.00
+ 66.00 1.00 4.00 2.00 1.00 3.00 68.00 62.00 56.00 50.00 51.00
+ 72.00 1.00 4.00 2.00 1.00 3.00 42.00 54.00 47.00 47.00 46.00
+ 77.00 1.00 4.00 1.00 1.00 2.00 61.00 59.00 49.00 44.00 66.00
+ 61.00 1.00 4.00 3.00 1.00 2.00 76.00 63.00 60.00 67.00 66.00
+190.00 1.00 4.00 2.00 2.00 2.00 47.00 59.00 54.00 58.00 46.00
+ 42.00 1.00 3.00 2.00 1.00 3.00 46.00 52.00 55.00 44.00 56.00
+ 2.00 1.00 1.00 2.00 1.00 3.00 39.00 41.00 33.00 42.00 41.00
+ 55.00 1.00 3.00 2.00 2.00 2.00 52.00 49.00 49.00 44.00 61.00
+ 19.00 1.00 1.00 1.00 1.00 1.00 28.00 46.00 43.00 44.00 51.00
+ 90.00 1.00 4.00 3.00 1.00 2.00 42.00 54.00 50.00 50.00 52.00
+142.00 1.00 4.00 2.00 1.00 3.00 47.00 42.00 52.00 39.00 51.00
+ 17.00 1.00 1.00 2.00 1.00 2.00 47.00 57.00 48.00 44.00 41.00
+122.00 1.00 4.00 2.00 1.00 2.00 52.00 59.00 58.00 53.00 66.00
+191.00 1.00 4.00 3.00 2.00 2.00 47.00 52.00 43.00 48.00 61.00
+ 83.00 1.00 4.00 2.00 1.00 3.00 50.00 62.00 41.00 55.00 31.00
+182.00 1.00 4.00 2.00 2.00 2.00 44.00 52.00 43.00 44.00 51.00
+ 6.00 1.00 1.00 1.00 1.00 2.00 47.00 41.00 46.00 40.00 41.00
+ 46.00 1.00 3.00 1.00 1.00 2.00 45.00 55.00 44.00 34.00 41.00
+ 43.00 1.00 3.00 1.00 1.00 2.00 47.00 37.00 43.00 42.00 46.00
+ 96.00 1.00 4.00 3.00 1.00 2.00 65.00 54.00 61.00 58.00 56.00
+138.00 1.00 4.00 2.00 1.00 3.00 43.00 57.00 40.00 50.00 51.00
+ 10.00 1.00 1.00 2.00 1.00 1.00 47.00 54.00 49.00 53.00 61.00
+ 71.00 1.00 4.00 2.00 1.00 1.00 57.00 62.00 56.00 58.00 66.00
+139.00 1.00 4.00 2.00 1.00 2.00 68.00 59.00 61.00 55.00 71.00
+110.00 1.00 4.00 2.00 1.00 3.00 52.00 55.00 50.00 54.00 61.00
+148.00 1.00 4.00 2.00 1.00 3.00 42.00 57.00 51.00 47.00 61.00
+109.00 1.00 4.00 2.00 1.00 1.00 42.00 39.00 42.00 42.00 41.00
+ 39.00 1.00 3.00 3.00 1.00 2.00 66.00 67.00 67.00 61.00 66.00
+147.00 1.00 4.00 1.00 1.00 2.00 47.00 62.00 53.00 53.00 61.00
+ 74.00 1.00 4.00 2.00 1.00 2.00 57.00 50.00 50.00 51.00 58.00
+198.00 1.00 4.00 3.00 2.00 2.00 47.00 61.00 51.00 63.00 31.00
+161.00 1.00 4.00 1.00 1.00 2.00 57.00 62.00 72.00 61.00 61.00
+112.00 1.00 4.00 2.00 1.00 2.00 52.00 59.00 48.00 55.00 61.00
+ 69.00 1.00 4.00 1.00 1.00 3.00 44.00 44.00 40.00 40.00 31.00
+156.00 1.00 4.00 2.00 1.00 2.00 50.00 59.00 53.00 61.00 61.00
+111.00 1.00 4.00 1.00 1.00 1.00 39.00 54.00 39.00 47.00 36.00
+186.00 1.00 4.00 2.00 2.00 2.00 57.00 62.00 63.00 55.00 41.00
+ 98.00 1.00 4.00 1.00 1.00 3.00 57.00 60.00 51.00 53.00 37.00
+119.00 1.00 4.00 1.00 1.00 1.00 42.00 57.00 45.00 50.00 43.00
+ 13.00 1.00 1.00 2.00 1.00 3.00 47.00 46.00 39.00 47.00 61.00
+ 51.00 1.00 3.00 3.00 1.00 1.00 42.00 36.00 42.00 31.00 39.00
+ 26.00 1.00 2.00 3.00 1.00 2.00 60.00 59.00 62.00 61.00 51.00
+ 36.00 1.00 3.00 1.00 1.00 1.00 44.00 49.00 44.00 35.00 51.00
+135.00 1.00 4.00 1.00 1.00 2.00 63.00 60.00 65.00 54.00 66.00
+ 59.00 1.00 4.00 2.00 1.00 2.00 65.00 67.00 63.00 55.00 71.00
+ 78.00 1.00 4.00 2.00 1.00 2.00 39.00 54.00 54.00 53.00 41.00
+ 64.00 1.00 4.00 3.00 1.00 3.00 50.00 52.00 45.00 58.00 36.00
+ 63.00 1.00 4.00 1.00 1.00 1.00 52.00 65.00 60.00 56.00 51.00
+ 79.00 1.00 4.00 2.00 1.00 2.00 60.00 62.00 49.00 50.00 51.00
+193.00 1.00 4.00 2.00 2.00 2.00 44.00 49.00 48.00 39.00 51.00
+ 92.00 1.00 4.00 3.00 1.00 1.00 52.00 67.00 57.00 63.00 61.00
+160.00 1.00 4.00 2.00 1.00 2.00 55.00 65.00 55.00 50.00 61.00
+ 32.00 1.00 2.00 3.00 1.00 3.00 50.00 67.00 66.00 66.00 56.00
+ 23.00 1.00 2.00 1.00 1.00 2.00 65.00 65.00 64.00 58.00 71.00
+158.00 1.00 4.00 2.00 1.00 1.00 52.00 54.00 55.00 53.00 51.00
+ 25.00 1.00 2.00 2.00 1.00 1.00 47.00 44.00 42.00 42.00 36.00
+188.00 1.00 4.00 3.00 2.00 2.00 63.00 62.00 56.00 55.00 61.00
+ 52.00 1.00 3.00 1.00 1.00 2.00 50.00 46.00 53.00 53.00 66.00
+124.00 1.00 4.00 1.00 1.00 3.00 42.00 54.00 41.00 42.00 41.00
+175.00 1.00 4.00 3.00 2.00 1.00 36.00 57.00 42.00 50.00 41.00
+184.00 1.00 4.00 2.00 2.00 3.00 50.00 52.00 53.00 55.00 56.00
+ 30.00 1.00 2.00 3.00 1.00 2.00 41.00 59.00 42.00 34.00 51.00
+179.00 1.00 4.00 2.00 2.00 2.00 47.00 65.00 60.00 50.00 56.00
+ 31.00 1.00 2.00 2.00 2.00 1.00 55.00 59.00 52.00 42.00 56.00
+145.00 1.00 4.00 2.00 1.00 3.00 42.00 46.00 38.00 36.00 46.00
+187.00 1.00 4.00 2.00 2.00 1.00 57.00 41.00 57.00 55.00 52.00
+118.00 1.00 4.00 2.00 1.00 1.00 55.00 62.00 58.00 58.00 61.00
+137.00 1.00 4.00 3.00 1.00 2.00 63.00 65.00 65.00 53.00 61.00
+end data.
+
+factor
+ /variables read write math science socst
+ /analysis read write math science socst
+ /extraction pc
+ /plot eigen
+ /criteria mineigen (.557)
+ .
+])
+
+AT_CHECK([pspp -O format=csv factor2.sps], [0],
+ [Table: Communalities
+,Initial,Extraction
+read,1.000,.736
+write,1.000,.704
+math,1.000,.750
+science,1.000,.849
+socst,1.000,.900
+
+Table: Total Variance Explained
+,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,,Rotation Sums of Squared Loadings,,
+,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
+1,3.381,67.6%,67.6%,3.381,67.6%,67.6%,2.113,42.3%,42.3%
+2,.557,11.1%,78.8%,.557,11.1%,78.8%,1.825,36.5%,78.8%
+3,.407,8.1%,86.9%,,,,,,
+4,.356,7.1%,94.0%,,,,,,
+5,.299,6.0%,100.0%,,,,,,
+
+Table: Component Matrix
+,Component,
+,1,2
+read,.858,.020
+write,.824,-.155
+math,.844,.195
+science,.801,.456
+socst,.783,-.536
+
+Table: Rotated Component Matrix
+,Component,
+,1,2
+read,.650,.559
+write,.508,.667
+math,.757,.421
+science,.900,.198
+socst,.222,.922
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([FACTOR empty dataset])
+dnl Test that something sane happens when the dataset contains no complete observations
+
+AT_DATA([factor-empty.sps],
+ [data list notable list /x * y * z *.
+begin data.
+3.4 . 92.9
+. 32.09 34.2
+1.00 19.80 .
+2.00 . 3.6
+end data.
+
+factor /variables = ALL.
+])
+
+AT_CHECK([pspp -O format=csv factor-empty.sps], [0], [ignore])
+AT_CLEANUP
+
+
+dnl Fixes a crash reported at
+dnl http://lists.gnu.org/archive/html/bug-gnu-pspp/2012-04/msg00001.html
+AT_SETUP([FACTOR /ROTATION=NOROTATE])
+AT_DATA([factor-norotate.sps], [dnl
+DATA LIST FREE / TRAIT1 TO TRAIT5 (F8.2).
+BEGIN DATA
+1 5 5 1 1
+8 9 7 9 8
+9 8 9 9 8
+9 9 9 9 9
+1 9 1 1 9
+9 7 7 9 9
+9 7 9 9 7
+END DATA
+
+SET SMALL=0.
+FACTOR /VARIABLES=TRAIT1 TO TRAIT5
+ /ROTATION=NOROTATE /* NOROTATE may have caused the problem. */
+ /EXTRACTION=PC
+ /PRINT=DEFAULT DET UNIVARIATE ROTATION SIG CORRELATION.
+])
+
+AT_CHECK([pspp -O format=csv factor-norotate.sps], [0], [dnl
+Table: Descriptive Statistics
+,Mean,Std. Deviation,Analysis N
+TRAIT1,6.57,3.54,7
+TRAIT2,7.71,1.39,7
+TRAIT3,6.71,2.71,7
+TRAIT4,6.71,3.61,7
+TRAIT5,7.29,2.66,7
+
+Table: Correlation Matrix
+,,TRAIT1,TRAIT2,TRAIT3,TRAIT4,TRAIT5
+Correlation,TRAIT1,1.000,.296,.881,.995,.545
+,TRAIT2,.296,1.000,-.022,.326,.837
+,TRAIT3,.881,-.022,1.000,.867,.130
+,TRAIT4,.995,.326,.867,1.000,.544
+,TRAIT5,.545,.837,.130,.544,1.000
+Sig. (1-tailed),TRAIT1,,.260,.004,.000,.103
+,TRAIT2,.260,,.482,.238,.009
+,TRAIT3,.004,.482,,.006,.390
+,TRAIT4,.000,.238,.006,,.103
+,TRAIT5,.103,.009,.390,.103,
+Caption: Determinant: 0.00
+
+Table: Communalities
+,Initial,Extraction
+TRAIT1,1.00,1.00
+TRAIT2,1.00,1.00
+TRAIT3,1.00,.99
+TRAIT4,1.00,.99
+TRAIT5,1.00,.99
+
+Table: Total Variance Explained
+,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,
+,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
+1,3.26,65.3%,65.3%,3.26,65.3%,65.3%
+2,1.54,30.8%,96.0%,1.54,30.8%,96.0%
+3,.17,3.4%,99.4%,.17,3.4%,99.4%
+4,.03,.6%,100.0%,.03,.6%,100.0%
+5,.00,.0%,100.0%,,,
+
+Table: Component Matrix
+,Component,,,
+,1,2,3,4
+TRAIT1,.97,.23,-.08,.00
+TRAIT2,.52,-.81,.28,.00
+TRAIT3,.78,.59,.17,.00
+TRAIT4,.97,.21,-.04,.00
+TRAIT5,.70,-.67,-.23,.00
+])
+AT_CLEANUP
+
+
+
+dnl Fixes a bug in the way that the /CRITERIA = ITERATE option was interpreted.
+dnl http://lists.gnu.org/archive/html/bug-gnu-pspp/2013-09/msg00036.html
+AT_SETUP([FACTOR /CRITERIA=ITERATE])
+AT_DATA([factor-iterate.sps], [dnl
+set format = F20.3.
+data list notable list /x y z *.
+begin data.
+1.00 5.00 3.00
+2.00 2.00 2.00
+3.00 1.00 1.00
+4.00 4.00 5.00
+5.00 3.00 9.00
+6.00 6.00 4.00
+7.00 7.00 6.00
+8.00 8.00 8.00
+9.00 9.00 7.00
+end data.
+
+FACTOR
+ /VARIABLES= x y z
+ /CRITERIA = MINEIGEN (1) ITERATE (25)
+ /EXTRACTION =PAF
+ /METHOD = CORRELATION
+ /PRINT = INITIAL EXTRACTION
+ /CRITERIA = ITERATE (0)
+ /ROTATION = NOROTATE.
+])
+
+AT_CHECK([pspp -O format=csv factor-iterate.sps], [0], [dnl
+Table: Communalities
+,Initial,Extraction
+x,.735,.979
+y,.640,.653
+z,.514,.523
+
+Table: Total Variance Explained
+,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,
+,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
+1,2.404,80.1%,80.1%,2.155,71.8%,71.8%
+2,.425,14.2%,94.3%,,,
+3,.171,5.7%,100.0%,,,
+
+Table: Factor Matrix
+,Factor
+,1
+x,.990
+y,.808
+z,.723
+])
+AT_CLEANUP
+
+
+AT_SETUP([FACTOR promax])
+AT_DATA([factor-promax.sps], [dnl
+set decimal=dot.
+set format=F22.3.
+
+get file='llz.zsav'.
+
+factor
+ /variables pz pn ps nz nn ns tz tn ts oz on os sz sn ss zz zn zs
+ /missing listwise
+ /print initial extraction rotation
+ /criteria mineigen(1) iterate(25)
+ /extraction paf
+ /method correlation
+ /rotation promax (5).
+])
+
+AT_CHECK([ln -s $top_srcdir/tests/language/commands/llz.zsav .], [0], [ignore])
+
+AT_CHECK([pspp -O format=csv factor-promax.sps], [0], [dnl
+Table: Communalities
+,Initial,Extraction
+PZ,.191,.375
+PN,.042,.102
+PS,.458,.623
+NZ,.100,.163
+NN,.065,.079
+NS,.129,.148
+TZ,.181,.344
+TN,.102,.142
+TS,.310,.372
+OZ,.097,.158
+ON,.323,.410
+OS,.469,.617
+SZ,.104,.170
+SN,.154,.267
+SS,.081,.180
+ZZ,.123,.192
+ZN,.208,.412
+ZS,.130,.158
+
+Table: Total Variance Explained
+,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,,Rotation Sums of Squared Loadings,,
+,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
+1,2.968,16.5%,16.5%,2.411,13.4%,13.4%,. ,. ,. @&t@
+2,2.026,11.3%,27.7%,1.271,7.1%,20.5%,. ,. ,-Infinity
+3,1.622,9.0%,36.8%,.948,5.3%,25.7%,. ,. ,-Infinity
+4,1.086,6.0%,42.8%,.283,1.6%,27.3%,. ,. ,-Infinity
+5,.996,5.5%,48.3%,,,,,,
+6,.923,5.1%,53.5%,,,,,,
+7,.873,4.9%,58.3%,,,,,,
+8,.856,4.8%,63.1%,,,,,,
+9,.836,4.6%,67.7%,,,,,,
+10,.816,4.5%,72.2%,,,,,,
+11,.785,4.4%,76.6%,,,,,,
+12,.740,4.1%,80.7%,,,,,,
+13,.713,4.0%,84.7%,,,,,,
+14,.653,3.6%,88.3%,,,,,,
+15,.633,3.5%,91.8%,,,,,,
+16,.604,3.4%,95.2%,,,,,,
+17,.484,2.7%,97.9%,,,,,,
+18,.386,2.1%,100.0%,,,,,,
+
+Table: Factor Matrix
+,Factor,,,
+,1,2,3,4
+PZ,-.276,.154,.510,.124
+PN,.096,.129,-.091,.261
+PS,.746,-.085,.234,.063
+NZ,-.111,.323,.206,-.058
+NN,.007,.260,-.083,-.069
+NS,.366,.096,.046,.051
+TZ,-.228,.172,.509,.059
+TN,.131,.345,-.074,.029
+TS,.601,-.005,.098,.030
+OZ,-.145,.166,.322,-.081
+ON,.607,.082,.073,-.173
+OS,.757,-.059,.171,-.104
+SZ,-.142,.307,.226,-.066
+SN,.175,.436,-.183,.115
+SS,.199,.206,-.083,.302
+ZZ,-.074,.411,-.080,-.104
+ZN,.015,.580,-.252,-.114
+ZS,.365,.156,-.004,.015
+
+Table: Pattern Matrix
+,Factor,,,
+,1,2,3,4
+PZ,-.063,-.126,.599,.085
+PN,-.035,.000,-.033,.325
+PS,.762,-.175,.058,.081
+NZ,.027,.230,.327,-.044
+NN,.008,.289,.008,-.026
+NS,.344,.044,.015,.091
+TZ,.004,-.074,.589,.020
+TN,.097,.307,.033,.103
+TS,.585,-.043,-.017,.062
+OZ,.046,.067,.382,-.109
+ON,.654,.151,-.029,-.145
+OS,.803,-.037,-.009,-.092
+SZ,.009,.213,.345,-.060
+SN,.065,.376,-.036,.227
+SS,.054,.042,-.013,.388
+ZZ,-.044,.434,.078,-.046
+ZN,-.025,.646,-.041,-.006
+ZS,.337,.133,-.013,.067
+
+Table: Structure Matrix
+,Factor,,,
+,1,2,3,4
+PZ,-.177,-.058,.598,-.022
+PN,.068,.110,-.049,.317
+PS,.771,-.138,-.136,.240
+NZ,-.060,.236,.339,.019
+NN,.000,.281,.027,.076
+NS,.368,.080,-.068,.207
+TZ,-.127,-.028,.582,-.049
+TN,.122,.345,.023,.235
+TS,.607,-.018,-.160,.221
+OZ,-.074,.055,.384,-.101
+ON,.619,.104,-.160,.102
+OS,.778,-.064,-.190,.132
+SZ,-.086,.215,.361,-.009
+SN,.143,.453,-.044,.380
+SS,.171,.176,-.052,.420
+ZZ,-.073,.422,.120,.085
+ZN,-.013,.641,.008,.214
+ZS,.361,.158,-.088,.213
+
+Table: Factor Correlation Matrix
+Factor,1,2,3,4
+1,1.000,.008,-.232,.294
+2,.008,1.000,.065,.347
+3,-.232,.065,1.000,-.076
+4,.294,.347,-.076,1.000
+])
+
+
+AT_CLEANUP
+
+
+
+
+
+
+AT_SETUP([FACTOR covariance matrix])
+
+AT_DATA([covariance-matrix.sps], [dnl
+set format = F10.3.
+matrix data
+ variables = rowtype_ var01 var02 var03 var04 var05 var06 var07 var08
+ / format = lower diagonal .
+begin data
+mean 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+sd 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+n 92 92 92 92 92 92 92 92
+cov 32.490000
+cov 1.539000 2.250000
+cov -29.469000 -5.992500 552.250000
+cov 11.901600 2.697000 -19.082000 33.640000
+cov 4.309200 0.672000 -7.896000 3.572800 7.840000
+cov 8.464500 1.012500 -17.977500 6.264000 2.646000 20.250000
+cov 15.390000 2.349000 -25.380000 10.022400 1.814400 9.234000 29.160000
+cov 1.453500 0.652500 -1.762500 1.740000 1.134000 1.350000 0.324000 2.250000
+end data.
+
+factor matrix in (cov = *)
+ / method = covariance
+ / print = initial covariance
+ / extraction = pc
+ / rotation = norotate.
+])
+
+AT_CHECK([pspp -O format=csv covariance-matrix.sps], [0], [dnl
+Table: Covariance Matrix
+,var01,var02,var03,var04,var05,var06,var07,var08
+var01,32.490,1.539,-29.469,11.902,4.309,8.464,15.390,1.454
+var02,1.539,2.250,-5.992,2.697,.672,1.013,2.349,.653
+var03,-29.469,-5.992,552.250,-19.082,-7.896,-17.977,-25.380,-1.763
+var04,11.902,2.697,-19.082,33.640,3.573,6.264,10.022,1.740
+var05,4.309,.672,-7.896,3.573,7.840,2.646,1.814,1.134
+var06,8.464,1.013,-17.977,6.264,2.646,20.250,9.234,1.350
+var07,15.390,2.349,-25.380,10.022,1.814,9.234,29.160,.324
+var08,1.454,.653,-1.763,1.740,1.134,1.350,.324,2.250
+
+Table: Communalities
+,Initial
+var01,32.490
+var02,2.250
+var03,552.250
+var04,33.640
+var05,7.840
+var06,20.250
+var07,29.160
+var08,2.250
+
+Table: Total Variance Explained
+,Initial Eigenvalues,,
+,Total,% of Variance,Cumulative %
+1,556.895,81.9%,81.9%
+2,57.963,8.5%,90.4%
+3,23.576,3.5%,93.9%
+4,16.446,2.4%,96.3%
+5,14.603,2.1%,98.4%
+6,6.831,1.0%,99.4%
+7,2.375,.3%,99.8%
+8,1.440,.2%,100.0%
+
+Table: Component Matrix
+,Component,,,,,,
+,1,2,3,4,5,6,7
+var01,1.394,4.388,1.513,-2.851,.849,.396,.033
+var02,.269,.460,-.173,.147,-.146,-.213,.872
+var03,-23.489,.725,.058,.003,.022,-.012,.006
+var04,.926,4.007,-4.068,.241,-.253,.218,-.026
+var05,.363,.829,-.172,-.255,.805,-2.492,.058
+var06,.843,2.354,.971,2.425,2.649,.392,.046
+var07,1.205,3.948,1.926,1.515,-2.450,-.317,-.087
+var08,.085,.319,-.157,-.011,.353,-.341,-.816
+])
+
+AT_CLEANUP
+
+
+
+dnl A more realistic example of factor analysis usage.
+AT_SETUP([FACTOR correlation matrix])
+
+AT_DATA([correlation-matrix.sps], [dnl
+set format = F10.3.
+
+matrix data
+ variables = rowtype_
+ cdi_actws_16 cdi_maxzin_16 rdls_passws_16 rdls_actws_16 cdi_actws_20 cdi_maxzin_20 cdi_actws_26 cdi_maxzin_26 rdls_passws_26 rdls_actws_26
+ nepsy_passws_36 morf_verv_36 bnt_actws_36 klankgr_id_36 snelnoe_36 letters_36 ppvt_passws_50 morf_verv_50
+ nepsy_passws_56 bnt_actws_56 klank_gr_weg_56 snelnoe_56 letters_56
+ leesacc_wo_owo_811 leesacc_tekst_811 leesacc_otekst_811 leessne_wo_owo_811 leesvl_tekst_811 leesvl_otekst_811 leessne_wo_811 spel_wo_owo_811
+ / format = upper diagonal .
+begin data
+mean 64.44 1.74 15.30 11.50 269.03 5.37 441.90 8.57 36.59 33.99 11.68 14.74 18.67 6.70 71.57 2.28 70.45 51.82 18.82 34.57 11.68 45.63 12.94 35.08 92.60 79.28 2.78 61.71 29.44 9.46 13.17
+sd 74.93 1.36 5.51 4.17 159.26 2.76 128.77 3.50 6.20 6.50 3.55 8.37 5.90 3.01 24.81 4.09 24.44 18.55 2.90 6.46 3.01 14.06 7.69 4.36 7.10 17.57 1.27 25.68 11.75 3.36 4.13
+n 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150
+corr 1.00 .784 .397 .862 .692 .625 .490 .374 .406 .371 .260 .257 .306 .118 -.148 .072 .202 .234 .198 .241 .205 -.054 .246 .166 .143 .155 -.122 .144 -.010 .135 .241
+corr 1.00 .333 .751 .549 .553 .447 .313 .304 .377 .204 .249 .258 .193 -.158 .119 .150 .216 .127 .209 .242 .046 .233 .120 .155 .107 -.126 .147 -.009 .134 .208
+corr 1.00 .469 .433 .381 .442 .307 .462 .391 .378 .293 .369 .191 -.306 .238 .204 .215 .295 .285 .157 .069 .241 .029 .060 .054 -.043 .124 -.069 .054 .136
+corr 1.00 .708 .663 .509 .419 .434 .432 .267 .255 .342 .132 -.192 .142 .228 .203 .248 .260 .200 -.051 .254 .136 .156 .109 -.126 .172 -.004 .157 .268
+corr 1.00 .787 .710 .567 .402 .511 .274 .285 .332 .154 -.096 .247 .253 .235 .245 .257 .261 -.048 .243 .119 .194 .164 -.108 .184 .011 .157 .235
+corr 1.00 .590 .646 .449 .505 .313 .322 .405 .148 -.117 .152 .294 .322 .252 .321 .267 -.055 .255 .118 .178 .137 -.110 .182 .004 .146 .216
+corr 1.00 .548 .343 .619 .296 .260 .456 .149 -.098 .252 .279 .267 .342 .361 .186 -.066 .215 .107 .148 .059 -.114 .156 -.035 .095 .220
+corr 1.00 .406 .509 .397 .236 .416 .037 -.179 .192 .334 .293 .277 .367 .162 -.150 .306 .171 .307 .173 -.128 .255 .075 .224 .315
+corr 1.00 .410 .497 .560 .574 .240 -.301 .204 .508 .351 .457 .428 .242 -.117 .367 .136 .191 .191 -.102 .215 .053 .185 .273
+corr 1.00 .328 .258 .534 .236 -.202 .200 .333 .209 .352 .375 .302 -.119 .272 .062 .203 .042 -.092 .220 .020 .158 .227
+corr 1.00 .439 .488 .323 -.213 .287 .507 .427 .493 .522 .298 -.142 .371 .109 .215 .213 -.048 .228 .009 .133 .267
+corr 1.00 .437 .381 -.158 .153 .403 .430 .383 .379 .150 -.141 .303 .115 .131 .155 -.170 .206 .039 .193 .254
+corr 1.00 .247 -.143 .182 .521 .364 .415 .688 .304 -.185 .327 .188 .211 .202 -.111 .272 .122 .226 .301
+corr 1.00 -.150 .229 .296 .249 .329 .255 .210 -.036 .252 .141 .230 .112 -.195 .309 .135 .250 .195
+corr 1.00 -.132 -.204 -.162 -.284 -.166 -.189 .294 -.339 -.094 -.218 -.144 .153 -.246 -.128 -.192 -.239
+corr 1.00 .151 .132 .166 .195 .387 -.214 .476 .154 .187 .167 -.236 .410 .316 .370 .245
+corr 1.00 .388 .479 .591 .294 -.171 .351 .102 .245 .180 .003 .274 .059 .178 .236
+corr 1.00 .408 .437 .276 -.153 .353 .251 .318 .229 -.111 .263 .042 .203 .349
+corr 1.00 .467 .234 -.249 .382 .199 .313 .197 -.117 .263 .047 .215 .318
+corr 1.00 .368 -.199 .441 .198 .224 .197 -.099 .329 .105 .256 .322
+corr 1.00 -.211 .473 .233 .253 .268 -.198 .397 .229 .309 .277
+corr 1.00 -.310 -.217 -.312 -.203 .227 -.296 -.260 -.276 -.321
+corr 1.00 .368 .350 .311 -.313 .578 .338 .521 .458
+corr 1.00 .415 .580 -.588 .545 .497 .635 .683
+corr 1.00 .570 -.386 .494 .340 .538 .524
+corr 1.00 -.366 .427 .299 .498 .506
+corr 1.00 -.684 -.620 -.746 -.568
+corr 1.00 .759 .900 .555
+corr 1.00 .814 .400
+corr 1.00 .621
+corr 1.00
+end data .
+
+factor matrix in (cor = *)
+ / analysis = cdi_actws_16 rdls_actws_16 cdi_actws_20 cdi_actws_26 rdls_actws_26 bnt_actws_36 bnt_actws_56
+ / format = default
+ / criteria = factors (1)
+ / extraction = pc
+ / rotation = norotate
+ / print = initial extraction .
+
+])
+
+AT_CHECK([pspp -O format=csv correlation-matrix.sps], [0], [dnl
+Table: Communalities
+,Initial,Extraction
+cdi_actws_16,1.000,.614
+rdls_actws_16,1.000,.660
+cdi_actws_20,1.000,.695
+cdi_actws_26,1.000,.650
+rdls_actws_26,1.000,.536
+bnt_actws_36,1.000,.443
+bnt_actws_56,1.000,.316
+
+Table: Total Variance Explained
+,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,
+,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
+1,3.914,55.9%,55.9%,3.914,55.9%,55.9%
+2,1.320,18.9%,74.8%,,,
+3,.716,10.2%,85.0%,,,
+4,.422,6.0%,91.0%,,,
+5,.278,4.0%,95.0%,,,
+6,.216,3.1%,98.1%,,,
+7,.135,1.9%,100.0%,,,
+
+Table: Component Matrix
+,Component
+,1
+cdi_actws_16,.784
+rdls_actws_16,.812
+cdi_actws_20,.834
+cdi_actws_26,.806
+rdls_actws_26,.732
+bnt_actws_36,.666
+bnt_actws_56,.562
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([FACTOR bad input])
+
+dnl Test for a crash
+AT_DATA([bad-input.sps], [dnl
+set format = F10.3.
+MATRIX DATA VARIABLES S1 ROWTYPE_ V1 TO V3 /SPLIT=S1.
+BEGIN DATA
+0 MEAN 2 4 3
+0 SD 1 2 3
+0 N 9 9 9
+0 KORR 1
+0 CORV .6 1
+0 CORR .7 .8 1
+1 MEAN 9 8 7
+1 SD 5 6 7
+1 N 9 9 9
+1 CORR 1
+X CORR .4 1
+1 CORR .3 .2 1
+END DATA.
+
+EXECUTE.
+
+FACTOR MATRIX IN (CORR =!*)
+ /PRINT = CORRELATION
+ .
+])
+
+AT_CHECK([pspp -O format=csv bad-input.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([FACTOR anti-image matrix])
+
+AT_DATA([anti-image-matrix.sps], [dnl
+SET FORMAT=F20.3 .
+matrix data
+ variables = rowtype_ viq piq pa ran piatwr piatc
+ / n = 476
+ / format = lower diagonal .
+begin data
+mean 96.88 100.51 -1.73 -0.94 -2.52 -1.85
+sd 10.97 11.19 1.19 0.88 0.85 0.97
+corr 1.00
+corr 0.38 1.00
+corr 0.26 0.24 1.00
+corr 0.16 0.17 0.34 1.00
+corr 0.25 0.07 0.46 0.38 1.00
+corr 0.37 0.22 0.39 0.30 0.59 1.00
+end data.
+
+factor matrix = in (cor = *)
+ / analysis = viq piq pa ran piatwr piatc
+ / format = sort
+ / extraction = pc
+ / rotation = norotate
+ / print = aic
+])
+
+AT_CHECK([pspp -O format=csv anti-image-matrix.sps], [0], [dnl
+Table: Anti-Image Matrices
+,,viq,piq,pa,ran,piatwr,piatc
+Anti-image Covariance,viq,.762,-.248,-.048,.008,-.031,-.143
+,piq,-.248,.807,-.117,-.081,.108,-.071
+,pa,-.048,-.117,.711,-.125,-.173,-.060
+,ran,.008,-.081,-.125,.808,-.143,-.035
+,piatwr,-.031,.108,-.173,-.143,.551,-.265
+,piatc,-.143,-.071,-.060,-.035,-.265,.581
+Anti-image Correlation,viq,.741,-.316,-.066,.011,-.048,-.215
+,piq,-.316,.624,-.154,-.100,.163,-.103
+,pa,-.066,-.154,.811,-.165,-.277,-.093
+,ran,.011,-.100,-.165,.825,-.214,-.051
+,piatwr,-.048,.163,-.277,-.214,.675,-.469
+,piatc,-.215,-.103,-.093,-.051,-.469,.729
+
+Table: Component Matrix
+,Component,,,,
+,1,2,3,4,5
+piatc,.774,.122,-.368,.365,-.322
+piatwr,.754,.418,.442,.219,-.115
+pa,.707,.124,-.117,-.161,.256
+piq,.456,-.733,.122,-.289,-.377
+viq,.589,-.539,.033,.298,.457
+ran,.592,.262,-.069,-.638,.096
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([FACTOR Kaiser-Meyer-Olkin])
+
+AT_DATA([kmo.sps], [dnl
+SET FORMAT=F20.3 .
+matrix data
+ variables = rowtype_ viq piq pa ran piatwr piatc
+ / n = 476
+ / format = lower diagonal .
+begin data
+mean 96.88 100.51 -1.73 -0.94 -2.52 -1.85
+sd 10.97 11.19 1.19 0.88 0.85 0.97
+corr 1.00
+corr 0.38 1.00
+corr 0.26 0.24 1.00
+corr 0.16 0.17 0.34 1.00
+corr 0.25 0.07 0.46 0.38 1.00
+corr 0.37 0.22 0.39 0.30 0.59 1.00
+end data.
+
+factor matrix = in (cor = *)
+ / analysis = viq piq pa ran piatwr piatc
+ / extraction = pc
+ / rotation = norotate
+ / print = kmo
+])
+
+AT_CHECK([pspp -O format=csv kmo.sps], [0], [dnl
+Table: KMO and Bartlett's Test
+Kaiser-Meyer-Olkin Measure of Sampling Adequacy,,.730
+Bartlett's Test of Sphericity,Approx. Chi-Square,602.673
+,df,15
+,Sig.,.000
+
+Table: Component Matrix
+,Component,,,,
+,1,2,3,4,5
+viq,.589,-.539,.033,.298,.457
+piq,.456,-.733,.122,-.289,-.377
+pa,.707,.124,-.117,-.161,.256
+ran,.592,.262,-.069,-.638,.096
+piatwr,.754,.418,.442,.219,-.115
+piatc,.774,.122,-.368,.365,-.322
+])
+
+AT_CLEANUP
+
+AT_SETUP([FACTOR syntax errors])
+AT_DATA([factor.sps], [dnl
+DATA LIST LIST NOTABLE /x y.
+FACTOR VARIABLES=**.
+FACTOR MATRIX=**.
+FACTOR MATRIX=IN **.
+FACTOR MATRIX=IN(**).
+FACTOR MATRIX=IN(CORR **).
+FACTOR MATRIX=IN(CORR=**).
+FACTOR MATRIX=IN(CORR=* **).
+FACTOR **.
+FACTOR VARIABLES=x/ANALYSIS=**.
+FACTOR VARIABLES=x/PLOT=**.
+FACTOR VARIABLES=x/METHOD=**.
+FACTOR VARIABLES=x/ROTATION=PROMAX(**).
+FACTOR VARIABLES=x/ROTATION=PROMAX(123 **).
+FACTOR VARIABLES=x/ROTATION=**.
+FACTOR VARIABLES=x/CRITERIA=FACTORS **.
+FACTOR VARIABLES=x/CRITERIA=FACTORS(**).
+FACTOR VARIABLES=x/CRITERIA=FACTORS(123 **).
+FACTOR VARIABLES=x/CRITERIA=MINEIGEN **.
+FACTOR VARIABLES=x/CRITERIA=MINEIGEN(**).
+FACTOR VARIABLES=x/CRITERIA=MINEIGEN(123 **).
+FACTOR VARIABLES=x/CRITERIA=ECONVERGE **.
+FACTOR VARIABLES=x/CRITERIA=ECONVERGE(**).
+FACTOR VARIABLES=x/CRITERIA=ECONVERGE(123 **).
+FACTOR VARIABLES=x/CRITERIA=RCONVERGE **.
+FACTOR VARIABLES=x/CRITERIA=RCONVERGE(**).
+FACTOR VARIABLES=x/CRITERIA=RCONVERGE(123 **).
+FACTOR VARIABLES=x/CRITERIA=**.
+FACTOR VARIABLES=x/EXTRACTION=**.
+FACTOR VARIABLES=x/FORMAT=BLANK **.
+FACTOR VARIABLES=x/FORMAT=BLANK(**).
+FACTOR VARIABLES=x/FORMAT=BLANK(123 **).
+FACTOR VARIABLES=x/FORMAT=**.
+FACTOR VARIABLES=x/PRINT=**.
+FACTOR VARIABLES=x/MISSING=**.
+FACTOR VARIABLES=x/ **.
+FACTOR VARIABLES=x.
+])
+AT_CHECK([pspp -O format=csv factor.sps], [1], [dnl
+"factor.sps:2.18-2.19: error: FACTOR: Syntax error expecting variable name.
+ 2 | FACTOR VARIABLES=**.
+ | ^~"
+
+"factor.sps:3.15-3.16: error: FACTOR: Syntax error expecting `IN('.
+ 3 | FACTOR MATRIX=**.
+ | ^~"
+
+"factor.sps:4.15-4.19: error: FACTOR: Syntax error expecting `IN('.
+ 4 | FACTOR MATRIX=IN **.
+ | ^~~~~"
+
+"factor.sps:5.18-5.19: error: FACTOR: Matrix input for FACTOR must be either COV or CORR.
+ 5 | FACTOR MATRIX=IN(**).
+ | ^~"
+
+"factor.sps:6.23-6.24: error: FACTOR: Syntax error expecting `='.
+ 6 | FACTOR MATRIX=IN(CORR **).
+ | ^~"
+
+"factor.sps:7.23-7.24: error: FACTOR: Syntax error expecting a file name or handle name.
+ 7 | FACTOR MATRIX=IN(CORR=**).
+ | ^~"
+
+"factor.sps:8.25-8.26: error: FACTOR: Syntax error expecting `)'.
+ 8 | FACTOR MATRIX=IN(CORR=* **).
+ | ^~"
+
+"factor.sps:10.29-10.30: error: FACTOR: Syntax error expecting variable name.
+ 10 | FACTOR VARIABLES=x/ANALYSIS=**.
+ | ^~"
+
+"factor.sps:11.25-11.26: error: FACTOR: Syntax error expecting EIGEN.
+ 11 | FACTOR VARIABLES=x/PLOT=**.
+ | ^~"
+
+"factor.sps:12.27-12.28: error: FACTOR: Syntax error expecting COVARIANCE or CORRELATION.
+ 12 | FACTOR VARIABLES=x/METHOD=**.
+ | ^~"
+
+"factor.sps:13.36-13.37: error: FACTOR: Syntax error expecting integer.
+ 13 | FACTOR VARIABLES=x/ROTATION=PROMAX(**).
+ | ^~"
+
+"factor.sps:14.40-14.41: error: FACTOR: Syntax error expecting `)'.
+ 14 | FACTOR VARIABLES=x/ROTATION=PROMAX(123 **).
+ | ^~"
+
+"factor.sps:15.29-15.30: error: FACTOR: Syntax error expecting DEFAULT, VARIMAX, EQUAMAX, QUARTIMAX, PROMAX, or NOROTATE.
+ 15 | FACTOR VARIABLES=x/ROTATION=**.
+ | ^~"
+
+"factor.sps:16.37-16.38: error: FACTOR: Syntax error expecting `('.
+ 16 | FACTOR VARIABLES=x/CRITERIA=FACTORS **.
+ | ^~"
+
+"factor.sps:17.37-17.38: error: FACTOR: Syntax error expecting integer.
+ 17 | FACTOR VARIABLES=x/CRITERIA=FACTORS(**).
+ | ^~"
+
+"factor.sps:18.41-18.42: error: FACTOR: Syntax error expecting `)'.
+ 18 | FACTOR VARIABLES=x/CRITERIA=FACTORS(123 **).
+ | ^~"
+
+"factor.sps:19.38-19.39: error: FACTOR: Syntax error expecting `('.
+ 19 | FACTOR VARIABLES=x/CRITERIA=MINEIGEN **.
+ | ^~"
+
+"factor.sps:20.38-20.39: error: FACTOR: Syntax error expecting number.
+ 20 | FACTOR VARIABLES=x/CRITERIA=MINEIGEN(**).
+ | ^~"
+
+"factor.sps:21.42-21.43: error: FACTOR: Syntax error expecting `)'.
+ 21 | FACTOR VARIABLES=x/CRITERIA=MINEIGEN(123 **).
+ | ^~"
+
+"factor.sps:22.39-22.40: error: FACTOR: Syntax error expecting `('.
+ 22 | FACTOR VARIABLES=x/CRITERIA=ECONVERGE **.
+ | ^~"
+
+"factor.sps:23.39-23.40: error: FACTOR: Syntax error expecting number.
+ 23 | FACTOR VARIABLES=x/CRITERIA=ECONVERGE(**).
+ | ^~"
+
+"factor.sps:24.43-24.44: error: FACTOR: Syntax error expecting `)'.
+ 24 | FACTOR VARIABLES=x/CRITERIA=ECONVERGE(123 **).
+ | ^~"
+
+"factor.sps:25.39-25.40: error: FACTOR: Syntax error expecting `('.
+ 25 | FACTOR VARIABLES=x/CRITERIA=RCONVERGE **.
+ | ^~"
+
+"factor.sps:26.39-26.40: error: FACTOR: Syntax error expecting number.
+ 26 | FACTOR VARIABLES=x/CRITERIA=RCONVERGE(**).
+ | ^~"
+
+"factor.sps:27.43-27.44: error: FACTOR: Syntax error expecting `)'.
+ 27 | FACTOR VARIABLES=x/CRITERIA=RCONVERGE(123 **).
+ | ^~"
+
+"factor.sps:28.29-28.30: error: FACTOR: Syntax error expecting FACTORS, MINEIGEN, ECONVERGE, RCONVERGE, ITERATE, or DEFAULT.
+ 28 | FACTOR VARIABLES=x/CRITERIA=**.
+ | ^~"
+
+"factor.sps:29.31-29.32: error: FACTOR: Syntax error expecting PAF, PC, PA1, or DEFAULT.
+ 29 | FACTOR VARIABLES=x/EXTRACTION=**.
+ | ^~"
+
+"factor.sps:30.33-30.34: error: FACTOR: Syntax error expecting `('.
+ 30 | FACTOR VARIABLES=x/FORMAT=BLANK **.
+ | ^~"
+
+"factor.sps:31.33-31.34: error: FACTOR: Syntax error expecting number.
+ 31 | FACTOR VARIABLES=x/FORMAT=BLANK(**).
+ | ^~"
+
+"factor.sps:32.37-32.38: error: FACTOR: Syntax error expecting `)'.
+ 32 | FACTOR VARIABLES=x/FORMAT=BLANK(123 **).
+ | ^~"
+
+"factor.sps:33.27-33.28: error: FACTOR: Syntax error expecting SORT, BLANK, or DEFAULT.
+ 33 | FACTOR VARIABLES=x/FORMAT=**.
+ | ^~"
+
+"factor.sps:34.26-34.27: error: FACTOR: Syntax error expecting one of the following: UNIVARIATE, DET, AIC, SIG, CORRELATION, COVARIANCE, ROTATION, EXTRACTION, INITIAL, KMO, ALL, DEFAULT.
+ 34 | FACTOR VARIABLES=x/PRINT=**.
+ | ^~"
+
+"factor.sps:35.28-35.29: error: FACTOR: Syntax error expecting INCLUDE, EXCLUDE, LISTWISE, PAIRRWISE, or MEANSUB.
+ 35 | FACTOR VARIABLES=x/MISSING=**.
+ | ^~"
+
+"factor.sps:36.21-36.22: error: FACTOR: Syntax error expecting one of the following: ANALYSIS, PLOT, METHOD, ROTATION, CRITERIA, EXTRACTION, FORMAT, PRINT, MISSING.
+ 36 | FACTOR VARIABLES=x/ **.
+ | ^~"
+
+"factor.sps:37.18: warning: FACTOR: Factor analysis on a single variable is not useful.
+ 37 | FACTOR VARIABLES=x.
+ | ^"
+
+error: FACTOR: At end of input: Syntax error expecting `BEGIN DATA'.
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([FILE HANDLE])
+
+AT_SETUP([FILE HANDLE])
+AT_DATA([wiggle.txt], [dnl
+1
+2
+5
+109
+])
+AT_DATA([file-handle.sps], [dnl
+FILE HANDLE myhandle /NAME='wiggle.txt'.
+DATA LIST LIST FILE=myhandle /x *.
+LIST.
+])
+AT_CHECK([pspp -O format=csv file-handle.sps], [0], [dnl
+Table: Reading free-form data from myhandle.
+Variable,Format
+x,F8.0
+
+Table: Data List
+x
+1.00
+2.00
+5.00
+109.00
+])
+AT_CLEANUP
+
+AT_SETUP([FILE HANDLE syntax errors])
+AT_DATA([file-handle.sps], [dnl
+FILE HANDLE **.
+FILE HANDLE x/NAME='x.txt'.
+FILE HANDLE x/NAME='x.txt'.
+FILE HANDLE y **.
+FILE HANDLE y/NAME=**.
+FILE HANDLE y/LRECL=8/LRECL=8.
+FILE HANDLE y/LRECL=**.
+FILE HANDLE y/TABWIDTH=8/TABWIDTH=8.
+FILE HANDLE y/TABWIDTH=**.
+FILE HANDLE y/MODE=CHARACTER/MODE=CHARACTER.
+FILE HANDLE y/MODE=**.
+FILE HANDLE y/ENDS=LF/ENDS=LF.
+FILE HANDLE y/ENDS=**.
+FILE HANDLE y/RECFORM=FIXED/RECFORM=FIXED.
+FILE HANDLE y/RECFORM=**.
+FILE HANDLE y/ENCODING='UTF-8'/ENCODING='UTF-8'.
+FILE HANDLE y/ENCODING=**.
+FILE HANDLE y/TABWIDTH=8.
+FILE HANDLE y/NAME='x.txt'/MODE=360.
+FILE HANDLE y/NAME='x.txt'/MODE=FIXED.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='file-handle.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"file-handle.sps:1.13-1.14: error: FILE HANDLE: Syntax error expecting identifier.
+ 1 | FILE HANDLE **.
+ | ^~"
+
+"file-handle.sps:3.13: error: FILE HANDLE: File handle x is already defined. Use CLOSE FILE HANDLE before redefining a file handle.
+ 3 | FILE HANDLE x/NAME='x.txt'.
+ | ^"
+
+"file-handle.sps:4.15-4.16: error: FILE HANDLE: Syntax error expecting `/'.
+ 4 | FILE HANDLE y **.
+ | ^~"
+
+"file-handle.sps:5.20-5.21: error: FILE HANDLE: Syntax error expecting string.
+ 5 | FILE HANDLE y/NAME=**.
+ | ^~"
+
+"file-handle.sps:6.23-6.27: error: FILE HANDLE: Subcommand LRECL may only be specified once.
+ 6 | FILE HANDLE y/LRECL=8/LRECL=8.
+ | ^~~~~"
+
+"file-handle.sps:7.21-7.22: error: FILE HANDLE: Syntax error expecting positive integer for LRECL.
+ 7 | FILE HANDLE y/LRECL=**.
+ | ^~"
+
+"file-handle.sps:8.26-8.33: error: FILE HANDLE: Subcommand TABWIDTH may only be specified once.
+ 8 | FILE HANDLE y/TABWIDTH=8/TABWIDTH=8.
+ | ^~~~~~~~"
+
+"file-handle.sps:9.24-9.25: error: FILE HANDLE: Syntax error expecting positive integer for TABWIDTH.
+ 9 | FILE HANDLE y/TABWIDTH=**.
+ | ^~"
+
+"file-handle.sps:10.30-10.33: error: FILE HANDLE: Subcommand MODE may only be specified once.
+ 10 | FILE HANDLE y/MODE=CHARACTER/MODE=CHARACTER.
+ | ^~~~"
+
+"file-handle.sps:11.20-11.21: error: FILE HANDLE: Syntax error expecting CHARACTER, BINARY, IMAGE, or 360.
+ 11 | FILE HANDLE y/MODE=**.
+ | ^~"
+
+"file-handle.sps:12.23-12.26: error: FILE HANDLE: Subcommand ENDS may only be specified once.
+ 12 | FILE HANDLE y/ENDS=LF/ENDS=LF.
+ | ^~~~"
+
+"file-handle.sps:13.20-13.21: error: FILE HANDLE: Syntax error expecting LF or CRLF.
+ 13 | FILE HANDLE y/ENDS=**.
+ | ^~"
+
+"file-handle.sps:14.29-14.35: error: FILE HANDLE: Subcommand RECFORM may only be specified once.
+ 14 | FILE HANDLE y/RECFORM=FIXED/RECFORM=FIXED.
+ | ^~~~~~~"
+
+"file-handle.sps:15.23-15.24: error: FILE HANDLE: Syntax error expecting FIXED, VARIABLE, or SPANNED.
+ 15 | FILE HANDLE y/RECFORM=**.
+ | ^~"
+
+"file-handle.sps:16.32-16.39: error: FILE HANDLE: Subcommand ENCODING may only be specified once.
+ 16 | FILE HANDLE y/ENCODING='UTF-8'/ENCODING='UTF-8'.
+ | ^~~~~~~~"
+
+"file-handle.sps:17.24-17.25: error: FILE HANDLE: Syntax error expecting string.
+ 17 | FILE HANDLE y/ENCODING=**.
+ | ^~"
+
+"file-handle.sps:18.1-18.25: error: FILE HANDLE: Required subcommand NAME was not specified.
+ 18 | FILE HANDLE y/TABWIDTH=8.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"file-handle.sps:19.28-19.35: error: FILE HANDLE: RECFORM must be specified with MODE=360.
+ 19 | FILE HANDLE y/NAME='x.txt'/MODE=360.
+ | ^~~~~~~~"
+
+"file-handle.sps:20.33-20.37: error: FILE HANDLE: Syntax error expecting CHARACTER, BINARY, IMAGE, or 360.
+ 20 | FILE HANDLE y/NAME='x.txt'/MODE=FIXED.
+ | ^~~~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([FLIP command])
+
+AT_SETUP([FLIP with NEWNAMES])
+AT_DATA([flip.sps], [dnl
+data list notable /N 1 (a) a b c d 2-9.
+list.
+begin data.
+v 1 2 3 4 5
+w 6 7 8 910
+x1112131415
+y1617181920
+z2122232425
+end data.
+temporary.
+compute e = a.
+flip newnames=n.
+list.
+flip.
+list.
+])
+AT_CHECK([pspp -O format=csv flip.sps], [0], [dnl
+Table: Data List
+N,a,b,c,d
+v,1,2,3,4
+w,6,7,8,9
+x,11,12,13,14
+y,16,17,18,19
+z,21,22,23,24
+
+"flip.sps:12.1-12.4: warning: FLIP: FLIP ignores TEMPORARY. Temporary transformations will be made permanent.
+ 12 | flip newnames=n.
+ | ^~~~"
+
+Table: Data List
+CASE_LBL,v,w,x,y,z
+a,1.00,6.00,11.00,16.00,21.00
+b,2.00,7.00,12.00,17.00,22.00
+c,3.00,8.00,13.00,18.00,23.00
+d,4.00,9.00,14.00,19.00,24.00
+e,1.00,6.00,11.00,16.00,21.00
+
+Table: Data List
+CASE_LBL,a,b,c,d,e
+v,1.00,2.00,3.00,4.00,1.00
+w,6.00,7.00,8.00,9.00,6.00
+x,11.00,12.00,13.00,14.00,11.00
+y,16.00,17.00,18.00,19.00,16.00
+z,21.00,22.00,23.00,24.00,21.00
+])
+AT_CLEANUP
+
+AT_SETUP([FLIP without NEWNAMES])
+AT_DATA([flip.sps], [dnl
+data list list notable /v1 to v10.
+format all(f2).
+begin data.
+1 2 3 4 5 6 7 8 9 10
+4 5 6 7 8 9 10 11 12 13
+end data.
+
+list.
+
+flip.
+list.
+])
+AT_CHECK([pspp -O format=csv flip.sps], [0], [dnl
+Table: Data List
+v1,v2,v3,v4,v5,v6,v7,v8,v9,v10
+1,2,3,4,5,6,7,8,9,10
+4,5,6,7,8,9,10,11,12,13
+
+Table: Data List
+CASE_LBL,VAR000,VAR001
+v1,1.00,4.00
+v2,2.00,5.00
+v3,3.00,6.00
+v4,4.00,7.00
+v5,5.00,8.00
+v6,6.00,9.00
+v7,7.00,10.00
+v8,8.00,11.00
+v9,9.00,12.00
+v10,10.00,13.00
+])
+AT_CLEANUP
+
+
+
+
+AT_SETUP([FLIP badly formed])
+
+AT_DATA([flip.sps], [dnl
+data list notable /N 1 (a) a b c d 2-9.
+
+flip newnames=n.
+list.
+flip.
+])
+
+AT_CHECK([pspp -O format=csv flip.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([FLIP with invalid variable names])
+
+AT_DATA([flip.sps], [dnl
+data list notable list /N (a3) a b c d *.
+begin data.
+"" 1 2 3 4
+BY 1 2 3 4
+end data.
+
+flip newnames=n.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv flip.sps], [0], [dnl
+Table: Data List
+CASE_LBL,v,BY1
+a,1.00,1.00
+b,2.00,2.00
+c,3.00,3.00
+d,4.00,4.00
+])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([FORMATS])
+
+AT_SETUP([FORMATS positive tests])
+AT_DATA([formats.sps], [dnl
+DATA LIST LIST /a b c * x (A1) y (A2) z (A3).
+DISPLAY VARIABLES.
+FORMATS /a (COMMA10) b (N4).
+DISPLAY VARIABLES.
+FORMATS c (E8.1) x (A1) /y (AHEX4) z (A3).
+DISPLAY VARIABLES.
+])
+AT_CHECK([pspp -o pspp.csv formats.sps])
+AT_CHECK([grep -E -v 'Measure|Display' pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+a,F8.0
+b,F8.0
+c,F8.0
+x,A1
+y,A2
+z,A3
+
+Table: Variables
+Name,Position,Print Format,Write Format
+a,1,F8.2,F8.2
+b,2,F8.2,F8.2
+c,3,F8.2,F8.2
+x,4,A1,A1
+y,5,A2,A2
+z,6,A3,A3
+
+Table: Variables
+Name,Position,Print Format,Write Format
+a,1,COMMA10.0,COMMA10.0
+b,2,N4.0,N4.0
+c,3,F8.2,F8.2
+x,4,A1,A1
+y,5,A2,A2
+z,6,A3,A3
+
+Table: Variables
+Name,Position,Print Format,Write Format
+a,1,COMMA10.0,COMMA10.0
+b,2,N4.0,N4.0
+c,3,E8.1,E8.1
+x,4,A1,A1
+y,5,AHEX4,AHEX4
+z,6,A3,A3
+])
+AT_CLEANUP
+
+AT_SETUP([FORMATS negative tests])
+AT_DATA([formats.sps], [dnl
+DATA LIST LIST /a b c * x (A1) y (A2) z (A3).
+FORMATS a (E6.1).
+FORMATS a y (F4).
+FORMATS x (A2).
+FORMATS y (AHEX2).
+FORMATS x y (A2).
+])
+AT_CHECK([pspp -O format=csv formats.sps], [1], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+a,F8.0
+b,F8.0
+c,F8.0
+x,A1
+y,A2
+z,A3
+
+"formats.sps:2.12-2.15: error: FORMATS: Output format E6.1 specifies 1 decimal place, but width 6 does not allow for any decimals.
+ 2 | FORMATS a (E6.1).
+ | ^~~~"
+
+"formats.sps:3.11: error: FORMATS: a and y are not the same type. All variables in this variable list must be of the same type. y will be omitted from the list.
+ 3 | FORMATS a y (F4).
+ | ^"
+
+"formats.sps:4.12-4.13: error: FORMATS: String variable x with width 1 is not compatible with format A2. Use format A1 instead.
+ 4 | FORMATS x (A2).
+ | ^~"
+
+"formats.sps:5.12-5.16: error: FORMATS: String variable y with width 2 is not compatible with format AHEX2. Use format AHEX4 instead.
+ 5 | FORMATS y (AHEX2).
+ | ^~~~~"
+
+"formats.sps:6.11: error: FORMATS: x and y are string variables with different widths. All variables in this variable list must have the same width. y will be omitted from the list.
+ 6 | FORMATS x y (A2).
+ | ^"
+
+"formats.sps:6.14-6.15: error: FORMATS: String variable x with width 1 is not compatible with format A2. Use format A1 instead.
+ 6 | FORMATS x y (A2).
+ | ^~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([FREQUENCIES procedure])
+
+AT_SETUP([FREQUENCIES string variable])
+AT_DATA([frequencies.sps],
+ [DATA LIST FREE/
+ name (A8) value * quantity .
+BEGIN DATA.
+foo 1 5
+bar 2 6
+baz 1 9
+quux 3 1
+bar 1 2
+baz 4 3
+baz 1 4
+baz 1 1
+foo 6 0
+quux 5 8
+END DATA.
+EXECUTE.
+
+FREQUENCIES /VAR = name/ORDER=ANALYSIS.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: name
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,bar,2,20.0%,20.0%,20.0%
+,baz,4,40.0%,40.0%,60.0%
+,foo,2,20.0%,20.0%,80.0%
+,quux,2,20.0%,20.0%,100.0%
+Total,,10,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES with SPLIT FILE - LAYERED])
+AT_DATA([frequencies.sps], [dnl
+DATA LIST LIST NOTABLE/name (A8) value quantity.
+BEGIN DATA.
+foo 1 5
+bar 2 6
+baz 1 9
+quux 3 1
+bar 1 2
+baz 4 3
+baz 1 4
+baz 1 1
+foo 6 0
+quux 5 8
+END DATA.
+EXECUTE.
+
+SORT CASES BY name.
+SPLIT FILE BY name.
+FREQUENCIES /VARIABLES=value quantity /FORMAT NOTABLE.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,name,,,,,,,
+,,bar,,baz,,foo,,quux,
+,,value,quantity,value,quantity,value,quantity,value,quantity
+N,Valid,2,2,4,4,2,2,2,2
+,Missing,0,0,0,0,0,0,0,0
+Mean,,1.50,4.00,1.75,4.25,3.50,2.50,4.00,4.50
+Std Dev,,.71,2.83,1.50,3.40,3.54,3.54,1.41,4.95
+Minimum,,1.00,2.00,1.00,1.00,1.00,.00,3.00,1.00
+Maximum,,2.00,6.00,4.00,9.00,6.00,5.00,5.00,8.00
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES with SPLIT FILE - SEPARATE])
+AT_DATA([frequencies.sps], [dnl
+DATA LIST LIST NOTABLE/name (A8) value quantity.
+BEGIN DATA.
+foo 1 5
+bar 2 6
+baz 1 9
+quux 3 1
+bar 1 2
+baz 4 3
+baz 1 4
+baz 1 1
+foo 6 0
+quux 5 8
+END DATA.
+EXECUTE.
+
+SORT CASES BY name.
+SPLIT FILE SEPARATE BY name.
+FREQUENCIES /VARIABLES=value quantity /FORMAT NOTABLE.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Split Values
+Variable,Value
+name,bar
+
+Table: Statistics
+,,value,quantity
+N,Valid,2,2
+,Missing,0,0
+Mean,,1.50,4.00
+Std Dev,,.71,2.83
+Minimum,,1.00,2.00
+Maximum,,2.00,6.00
+
+Table: Split Values
+Variable,Value
+name,baz
+
+Table: Statistics
+,,value,quantity
+N,Valid,4,4
+,Missing,0,0
+Mean,,1.75,4.25
+Std Dev,,1.50,3.40
+Minimum,,1.00,1.00
+Maximum,,4.00,9.00
+
+Table: Split Values
+Variable,Value
+name,foo
+
+Table: Statistics
+,,value,quantity
+N,Valid,2,2
+,Missing,0,0
+Mean,,3.50,2.50
+Std Dev,,3.54,3.54
+Minimum,,1.00,.00
+Maximum,,6.00,5.00
+
+Table: Split Values
+Variable,Value
+name,quux
+
+Table: Statistics
+,,value,quantity
+N,Valid,2,2
+,Missing,0,0
+Mean,,4.00,4.50
+Std Dev,,1.41,4.95
+Minimum,,3.00,1.00
+Maximum,,5.00,8.00
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES with SPLIT FILE - LAYERED - unsorted data])
+AT_DATA([frequencies.sps], [dnl
+DATA LIST LIST NOTABLE/name (A8) value quantity.
+BEGIN DATA.
+foo 1 5
+bar 2 6
+baz 1 9
+quux 3 1
+baz 4 3
+bar 1 2
+baz 1 1
+foo 6 0
+baz 1 4
+quux 5 8
+END DATA.
+EXECUTE.
+
+SPLIT FILE BY name.
+FREQUENCIES /VARIABLES=value quantity /FORMAT NOTABLE.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = baz "
+
+"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = bar "
+
+"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = baz "
+
+"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = foo "
+
+"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = baz "
+
+Table: Statistics
+,,name,,,,,,,,,,,,,,,,,,,
+,,foo,,bar,,baz,,quux,,baz,,bar,,baz,,foo,,baz,,quux,
+,,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity
+N,Valid,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
+,Missing,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
+Mean,,1.00,5.00,2.00,6.00,1.00,9.00,3.00,1.00,4.00,3.00,1.00,2.00,1.00,1.00,6.00,.00,1.00,4.00,5.00,8.00
+Std Dev,,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN
+Minimum,,1.00,5.00,2.00,6.00,1.00,9.00,3.00,1.00,4.00,3.00,1.00,2.00,1.00,1.00,6.00,.00,1.00,4.00,5.00,8.00
+Maximum,,1.00,5.00,2.00,6.00,1.00,9.00,3.00,1.00,4.00,3.00,1.00,2.00,1.00,1.00,6.00,.00,1.00,4.00,5.00,8.00
+
+frequencies.sps:17: warning: FREQUENCIES: Suppressed 1 additional warning about duplicate split values.
+])
+AT_CLEANUP
+
+# Tests for a bug where pspp would crash if two FREQUENCIES commands
+# existed in a input file.
+AT_SETUP([FREQUENCIES two runs crash])
+AT_DATA([frequencies.sps],
+ [data list free /v1 v2.
+begin data.
+0 1
+2 3
+4 5
+3 4
+end data.
+
+frequencies v1 v2/statistics=none/ORDER=VARIABLE.
+frequencies v1 v2/statistics=none.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: v1
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,.00,1,25.0%,25.0%,25.0%
+,2.00,1,25.0%,25.0%,50.0%
+,3.00,1,25.0%,25.0%,75.0%
+,4.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+
+Table: v2
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1,25.0%,25.0%,25.0%
+,3.00,1,25.0%,25.0%,50.0%
+,4.00,1,25.0%,25.0%,75.0%
+,5.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+
+Table: v1
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,.00,1,25.0%,25.0%,25.0%
+,2.00,1,25.0%,25.0%,50.0%
+,3.00,1,25.0%,25.0%,75.0%
+,4.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+
+Table: v2
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1,25.0%,25.0%,25.0%
+,3.00,1,25.0%,25.0%,50.0%
+,4.00,1,25.0%,25.0%,75.0%
+,5.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+])
+AT_CLEANUP
+
+# Test that the LIMIT specification works.
+AT_SETUP([FREQUENCIES with LIMIT])
+AT_DATA([frequencies.sps],
+ [data list free /v1 v2.
+begin data.
+0 1
+2 5
+4 3
+3 5
+end data.
+
+frequencies v1 v2/statistics=none/FORMAT=LIMIT(3).
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: v2
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1,25.0%,25.0%,25.0%
+,3.00,1,25.0%,25.0%,50.0%
+,5.00,2,50.0%,50.0%,100.0%
+Total,,4,100.0%,,
+])
+AT_CLEANUP
+
+# Tests for a bug where PSPP would crash when a FREQUENCIES command
+# was used with the HTML output driver.
+AT_SETUP([FREQUENCIES HTML output crash])
+AT_DATA([frequencies.sps],
+ [data list free /v1 v2.
+begin data.
+0 1
+2 3
+4 5
+3 4
+end data.
+
+list.
+
+frequencies v1/statistics=none.
+])
+AT_CHECK([pspp -o - -O format=csv -o pspp.html frequencies.sps], [0],
+ [Table: Data List
+v1,v2
+.00,1.00
+2.00,3.00
+4.00,5.00
+3.00,4.00
+
+Table: v1
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,.00,1,25.0%,25.0%,25.0%
+,2.00,1,25.0%,25.0%,50.0%
+,3.00,1,25.0%,25.0%,75.0%
+,4.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+])
+AT_CHECK([test -s pspp.html])
+AT_CLEANUP
+
+# Tests for a bug which crashed PSPP when a piechart with too many
+# segments was requested.
+AT_SETUP([FREQUENCIES pie chart crash])
+AT_DATA([frequencies.sps],
+ [data list list /x * w *.
+begin data.
+1 4
+34 10
+-9 15
+232 6
+11 4
+134 1
+9 5
+32 16
+-2 6
+2 16
+20 6
+end data.
+
+weight by w.
+
+frequencies /x /format=notable /statistics=none
+ /piechart.
+])
+# Cannot use the CSV driver for this because it does not output charts
+# at all.
+AT_CHECK([pspp frequencies.sps], [0], [dnl
+Reading free-form data from INLINE.
++--------+------+
+|Variable|Format|
++--------+------+
+|x |F8.0 |
+|w |F8.0 |
++--------+------+
+])
+AT_CLEANUP
+
+dnl Check that histogram subcommand runs wihout crashing
+AT_SETUP([FREQUENCIES histogram crash])
+AT_DATA([frequencies.sps],
+ [data list notable list /x * w *.
+begin data.
+1 4
+34 10
+-9 15
+232 6
+11 4
+134 1
+9 5
+32 16
+-2 6
+2 16
+20 6
+end data.
+
+weight by w.
+
+frequencies /x
+ /format=notable
+ /statistics=none
+ /histogram=minimum(0) maximum(50) percent(5) normal.
+])
+# Cannot use the CSV driver for this because it does not output charts
+# at all.
+AT_CHECK([pspp -O format=pdf frequencies.sps], [0], [ignore], [ignore])
+AT_CLEANUP
+
+# Tests for a bug which crashed PSPP when the median and a histogram
+# were both requested.
+AT_SETUP([FREQUENCIES median with histogram crash])
+AT_DATA([frequencies.sps], [dnl
+data list list notable /x.
+begin data.
+1
+end data.
+
+frequencies /x /histogram /STATISTICS=median.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [ignore])
+dnl Ignore output - No crash test.
+AT_CLEANUP
+
+# Tests for a bug which caused FREQUENCIES following TEMPORARY to
+# crash (bug #11492).
+AT_SETUP([FREQUENCIES crash after TEMPORARY])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST /SEX (A1) X *.
+BEGIN DATA.
+M 31
+F 21
+M 41
+F 31
+M 13
+F 12
+M 14
+F 13
+END DATA.
+
+
+TEMPORARY
+SELECT IF SEX EQ 'F'
+FREQUENCIES /X .
+
+FINISH
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt frequencies.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+SEX,A1
+X,F8.0
+
+Table: Statistics
+,,X
+N,Valid,4
+,Missing,0
+Mean,,19.25
+Std Dev,,8.81
+Minimum,,12.00
+Maximum,,31.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,12.00,1,25.0%,25.0%,25.0%
+,13.00,1,25.0%,25.0%,50.0%
+,21.00,1,25.0%,25.0%,75.0%
+,31.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+])
+AT_CLEANUP
+
+m4_define([FREQUENCIES_NTILES_OUTPUT], [dnl
+Table: Statistics
+,,x,y
+N,Valid,5,5
+,Missing,0,0
+Mean,,3.00,30.00
+Std Dev,,1.58,15.81
+Minimum,,1.00,10.00
+Maximum,,5.00,50.00
+Percentiles,0,1.00,10.00
+,25,2.00,20.00
+,33,2.33,23.33
+,50,3.00,30.00
+,67,3.67,36.67
+,75,4.00,40.00
+,100,5.00,50.00
+])
+AT_SETUP([FREQUENCIES basic percentiles])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /x y.
+BEGIN DATA.
+1 10
+2 20
+3 30
+4 40
+5 50
+END DATA.
+
+FREQUENCIES
+ VAR=x y
+ /FORMAT=NOTABLE
+ /PERCENTILES = 0 25 33.333 50 66.666 75 100.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0],
+ [FREQUENCIES_NTILES_OUTPUT])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES basic n-tiles])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /x y.
+BEGIN DATA.
+1 10
+2 20
+3 30
+4 40
+5 50
+END DATA.
+
+FREQUENCIES
+ VAR=x y
+ /FORMAT=NOTABLE
+ /NTILES = 3
+ /NTILES = 4.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0],
+ [FREQUENCIES_NTILES_OUTPUT])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES compatibility percentiles])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /X * .
+BEGIN DATA.
+1
+2
+3
+4
+5
+END DATA.
+
+FREQUENCIES
+ VAR=x
+ /ALGORITHM=COMPATIBLE
+ /PERCENTILES = 0 25 50 75 100.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,X
+N,Valid,5
+,Missing,0
+Mean,,3.00
+Std Dev,,1.58
+Minimum,,1.00
+Maximum,,5.00
+Percentiles,0,1.00
+,25,1.50
+,50,3.00
+,75,4.50
+,100,5.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1,20.0%,20.0%,20.0%
+,2.00,1,20.0%,20.0%,40.0%
+,3.00,1,20.0%,20.0%,60.0%
+,4.00,1,20.0%,20.0%,80.0%
+,5.00,1,20.0%,20.0%,100.0%
+Total,,5,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES enhanced percentiles])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /X * .
+BEGIN DATA.
+1
+2
+3
+4
+5
+END DATA.
+
+FREQUENCIES
+ VAR=x
+ /PERCENTILES = 0 25 50 75 100.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,X
+N,Valid,5
+,Missing,0
+Mean,,3.00
+Std Dev,,1.58
+Minimum,,1.00
+Maximum,,5.00
+Percentiles,0,1.00
+,25,2.00
+,50,3.00
+,75,4.00
+,100,5.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1,20.0%,20.0%,20.0%
+,2.00,1,20.0%,20.0%,40.0%
+,3.00,1,20.0%,20.0%,60.0%
+,4.00,1,20.0%,20.0%,80.0%
+,5.00,1,20.0%,20.0%,100.0%
+Total,,5,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES enhanced percentiles, weighted])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /X * F *.
+BEGIN DATA.
+1 2
+2 2
+3 2
+4 1
+4 1
+5 1
+5 1
+END DATA.
+
+WEIGHT BY f.
+
+FREQUENCIES
+ VAR=x
+ /PERCENTILES = 0 25 50 75 100.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,X
+N,Valid,10.00
+,Missing,.00
+Mean,,3.00
+Std Dev,,1.49
+Minimum,,1.00
+Maximum,,5.00
+Percentiles,0,1.00
+,25,2.00
+,50,3.00
+,75,4.00
+,100,5.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,2.00,20.0%,20.0%,20.0%
+,2.00,2.00,20.0%,20.0%,40.0%
+,3.00,2.00,20.0%,20.0%,60.0%
+,4.00,2.00,20.0%,20.0%,80.0%
+,5.00,2.00,20.0%,20.0%,100.0%
+Total,,10.00,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES enhanced percentiles, weighted (2)])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /X * F *.
+BEGIN DATA.
+1 1
+3 2
+4 1
+5 1
+5 1
+END DATA.
+
+WEIGHT BY f.
+
+FREQUENCIES
+ VAR=x
+ /PERCENTILES = 0 25 50 75 100.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,X
+N,Valid,6.00
+,Missing,.00
+Mean,,3.50
+Std Dev,,1.52
+Minimum,,1.00
+Maximum,,5.00
+Percentiles,0,1.00
+,25,3.00
+,50,3.50
+,75,4.75
+,100,5.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1.00,16.7%,16.7%,16.7%
+,3.00,2.00,33.3%,33.3%,50.0%
+,4.00,1.00,16.7%,16.7%,66.7%
+,5.00,2.00,33.3%,33.3%,100.0%
+Total,,6.00,100.0%,,
+])
+AT_CLEANUP
+
+dnl Data for this test case from Fabio Bordignon <bordignon@demos.it>.
+AT_SETUP([FREQUENCIES enhanced percentiles, weighted (3)])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /X * F *.
+BEGIN DATA.
+1 7
+2 16
+3 12
+4 5
+END DATA.
+
+WEIGHT BY f.
+
+FREQUENCIES
+ VAR=x
+ /PERCENTILES = 0 25 50 75 100.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,X
+N,Valid,40.00
+,Missing,.00
+Mean,,2.38
+Std Dev,,.93
+Minimum,,1.00
+Maximum,,4.00
+Percentiles,0,1.00
+,25,2.00
+,50,2.00
+,75,3.00
+,100,4.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,7.00,17.5%,17.5%,17.5%
+,2.00,16.00,40.0%,40.0%,57.5%
+,3.00,12.00,30.0%,30.0%,87.5%
+,4.00,5.00,12.5%,12.5%,100.0%
+Total,,40.00,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES enhanced percentiles, weighted, missing values])
+AT_DATA([frequencies.sps],
+ [DATA LIST LIST notable /X * F *.
+BEGIN DATA.
+1 1
+3 2
+4 1
+5 1
+5 1
+99 4
+END DATA.
+
+MISSING VALUE x (99.0) .
+WEIGHT BY f.
+
+FREQUENCIES
+ VAR=x
+ /PERCENTILES = 0 25 50 75 100.
+])
+
+AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
+Table: Statistics
+,,X
+N,Valid,6.00
+,Missing,4.00
+Mean,,3.50
+Std Dev,,1.52
+Minimum,,1.00
+Maximum,,5.00
+Percentiles,0,1.00
+,25,3.00
+,50,3.50
+,75,4.75
+,100,5.00
+
+Table: X
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1.00,10.0%,16.7%,16.7%
+,3.00,2.00,20.0%,33.3%,50.0%
+,4.00,1.00,10.0%,16.7%,66.7%
+,5.00,2.00,20.0%,33.3%,100.0%
+Missing,99.00,4.00,40.0%,,
+Total,,10.00,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES dichotomous histogram])
+AT_DATA([frequencies.sps], [dnl
+data list notable list /d4 *.
+begin data.
+0
+0
+0
+1
+0
+0
+0
+0
+1
+0
+0
+0
+0
+0
+1
+2
+0
+end data.
+
+FREQUENCIES
+ /VARIABLES = d4
+ /FORMAT=AVALUE TABLE
+ /HISTOGRAM=NORMAL
+ .
+])
+
+AT_CHECK([pspp frequencies.sps], [0], [ignore])
+AT_CLEANUP
+
+
+AT_SETUP([FREQUENCIES median])
+AT_DATA([median.sps], [dnl
+data list notable list /x *.
+begin data.
+1
+2
+3000000
+end data.
+
+FREQUENCIES
+ /VARIABLES = x
+ /STATISTICS = MEDIAN
+ .
+])
+
+AT_CHECK([pspp median.sps -O format=csv], [0], [dnl
+Table: Statistics
+,,x
+N,Valid,3
+,Missing,0
+Median,,2.00
+
+Table: x
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,1,33.3%,33.3%,33.3%
+,2.00,1,33.3%,33.3%,66.7%
+,3000000,1,33.3%,33.3%,100.0%
+Total,,3,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES variance])
+AT_DATA([variance.sps], [dnl
+data list notable list /forename (A12) height.
+begin data.
+Ahmed 188
+bertram 167
+Catherine 134
+David 109
+end data.
+
+FREQUENCIES
+ /VARIABLES = height
+ /STATISTICS = VARIANCE.
+])
+
+AT_CHECK([pspp variance.sps -O format=csv], [0], [dnl
+Table: Statistics
+,,height
+N,Valid,4
+,Missing,0
+Variance,,1223.00
+
+Table: height
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,109.00,1,25.0%,25.0%,25.0%
+,134.00,1,25.0%,25.0%,50.0%
+,167.00,1,25.0%,25.0%,75.0%
+,188.00,1,25.0%,25.0%,100.0%
+Total,,4,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES default statistics])
+AT_DATA([median.sps], [dnl
+data list notable list /x *.
+begin data.
+10
+20
+3000000
+end data.
+
+FREQUENCIES
+ /VARIABLES = x
+ /STATISTICS
+ .
+
+FREQUENCIES
+ /VARIABLES = x
+ /STATISTICS = DEFAULT
+ .
+])
+
+AT_CHECK([pspp median.sps -o pspp.csv -o pspp.txt])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Statistics
+,,x
+N,Valid,3
+,Missing,0
+Mean,,1000010
+Std Dev,,1732042
+Minimum,,10.00
+Maximum,,3000000
+
+Table: x
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,10.00,1,33.3%,33.3%,33.3%
+,20.00,1,33.3%,33.3%,66.7%
+,3000000,1,33.3%,33.3%,100.0%
+Total,,3,100.0%,,
+
+Table: Statistics
+,,x
+N,Valid,3
+,Missing,0
+Mean,,1000010
+Std Dev,,1732042
+Minimum,,10.00
+Maximum,,3000000
+
+Table: x
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,10.00,1,33.3%,33.3%,33.3%
+,20.00,1,33.3%,33.3%,66.7%
+,3000000,1,33.3%,33.3%,100.0%
+Total,,3,100.0%,,
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([FREQUENCIES no valid data])
+AT_DATA([empty.sps], [dnl
+data list notable list /x *.
+begin data.
+.
+.
+.
+end data.
+
+FREQUENCIES
+ /VARIABLES = x
+ /STATISTICS = ALL
+ .
+])
+
+AT_CHECK([pspp empty.sps -O format=csv], [0], [dnl
+Table: Statistics
+,,x
+N,Valid,0
+,Missing,3
+Mean,,. @&t@
+S.E. Mean,,. @&t@
+Median,,. @&t@
+Mode,,. @&t@
+Std Dev,,. @&t@
+Variance,,. @&t@
+Kurtosis,,. @&t@
+S.E. Kurt,,. @&t@
+Skewness,,. @&t@
+S.E. Skew,,. @&t@
+Range,,. @&t@
+Minimum,,. @&t@
+Maximum,,. @&t@
+Sum,,. @&t@
+
+Table: x
+,,Frequency,Percent
+Missing,. ,3,100.0%
+Total,,3,.0%
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([FREQUENCIES histogram no valid cases])
+AT_DATA([empty.sps], [dnl
+data list notable list /x w *.
+begin data.
+1 .
+2 .
+3 .
+end data.
+
+weight by w.
+
+FREQUENCIES
+ /VARIABLES = x
+ /histogram
+ .
+])
+
+AT_CHECK([pspp empty.sps -O format=csv], [0], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES percentiles + histogram bug#48128])
+AT_DATA([bug.sps], [dnl
+SET FORMAT=F8.0.
+
+INPUT PROGRAM.
+ LOOP I=1 TO 10.
+ COMPUTE SCORE=EXP(NORMAL(1)).
+ END CASE.
+ END LOOP.
+ END FILE.
+END INPUT PROGRAM.
+
+FREQUENCIES VARIABLES=SCORE
+/FORMAT=NOTABLE
+/STATISTICS=ALL
+/PERCENTILES=1 10 20 30 40 50 60 70 80 90 99
+/HISTOGRAM.
+
+])
+
+AT_CHECK([pspp bug.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([FREQUENCIES vs. missing weights])
+AT_DATA([warn.sps], [dnl
+data list notable list /x w .
+begin data.
+1 1
+2 1
+1 1
+3 1
+3 .
+4 .
+end data.
+
+weight by w.
+
+frequencies /variables=x.
+])
+
+AT_CHECK([pspp warn.sps -O format=csv], [0], [dnl
+"warn.sps:13: warning: FREQUENCIES: At least one case in the data file had a weight value that was user-missing, system-missing, zero, or negative. These case(s) were ignored."
+
+Table: Statistics
+,,x
+N,Valid,4.00
+,Missing,.00
+Mean,,1.75
+Std Dev,,.96
+Minimum,,1.00
+Maximum,,4.00
+
+Table: x
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,1.00,2.00,50.0%,50.0%,50.0%
+,2.00,1.00,25.0%,25.0%,75.0%
+,3.00,1.00,25.0%,25.0%,100.0%
+,4.00,.00,.0%,.0%,100.0%
+Total,,4.00,100.0%,,
+])
+
+AT_CLEANUP
+
+AT_SETUP([FREQUENCIES syntax errors])
+AT_DATA([frequencies.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+FREQUENCIES VARIABLES **.
+FREQUENCIES **.
+FREQUENCIES x/STATISTICS **.
+FREQUENCIES x/PERCENTILES **.
+FREQUENCIES x/FORMAT LIMIT **.
+FREQUENCIES x/FORMAT LIMIT(**).
+FREQUENCIES x/FORMAT LIMIT(5 **).
+FREQUENCIES x/FORMAT **.
+FREQUENCIES x/NTILES **.
+FREQUENCIES x/ALGORITHM **.
+FREQUENCIES x/HISTOGRAM FREQ(**).
+FREQUENCIES x/HISTOGRAM FREQ(5 **).
+FREQUENCIES x/HISTOGRAM PERCENT(**).
+FREQUENCIES x/HISTOGRAM PERCENT(5 **).
+FREQUENCIES x/HISTOGRAM MINIMUM(**).
+FREQUENCIES x/HISTOGRAM MINIMUM(5 **).
+FREQUENCIES x/HISTOGRAM MAXIMUM(**).
+FREQUENCIES x/HISTOGRAM MAXIMUM(5 **).
+FREQUENCIES x/HISTOGRAM MINIMUM(5) MAXIMUM(1).
+FREQUENCIES x/HISTOGRAM MAXIMUM(5) MINIMUM(10).
+FREQUENCIES x/HISTOGRAM **.
+FREQUENCIES x/PIECHART MINIMUM(**).
+FREQUENCIES x/PIECHART MINIMUM(5 **).
+FREQUENCIES x/PIECHART MAXIMUM(**).
+FREQUENCIES x/PIECHART MAXIMUM(5 **).
+FREQUENCIES x/PIECHART MINIMUM(5) MAXIMUM(1).
+FREQUENCIES x/PIECHART MAXIMUM(5) MINIMUM(10).
+FREQUENCIES x/PIECHART **.
+FREQUENCIES x/BARCHART FREQ(**).
+FREQUENCIES x/BARCHART FREQ(5 **).
+FREQUENCIES x/BARCHART PERCENT(**).
+FREQUENCIES x/BARCHART PERCENT(5 **).
+FREQUENCIES x/BARCHART MINIMUM(**).
+FREQUENCIES x/BARCHART MINIMUM(5 **).
+FREQUENCIES x/BARCHART MAXIMUM(**).
+FREQUENCIES x/BARCHART MAXIMUM(5 **).
+FREQUENCIES x/BARCHART MINIMUM(5) MAXIMUM(1).
+FREQUENCIES x/BARCHART MAXIMUM(5) MINIMUM(10).
+FREQUENCIES x/BARCHART **.
+FREQUENCIES x/MISSING **.
+FREQUENCIES x/ORDER **.
+FREQUENCIES x/ **.
+])
+AT_CHECK([pspp -O format=csv frequencies.sps], [1], [dnl
+"frequencies.sps:2.23-2.24: error: FREQUENCIES: Syntax error expecting `='.
+ 2 | FREQUENCIES VARIABLES **.
+ | ^~"
+
+"frequencies.sps:3.13-3.14: error: FREQUENCIES: Syntax error expecting variable name.
+ 3 | FREQUENCIES **.
+ | ^~"
+
+"frequencies.sps:4.26-4.27: error: FREQUENCIES: Syntax error expecting one of the following: MEAN, SEMEAN, MEDIAN, MODE, STDDEV, VARIANCE, KURTOSIS, SEKURTOSIS, SKEWNESS, SESKEWNESS, RANGE, MINIMUM, MAXIMUM, SUM, DEFAULT, ALL, NONE.
+ 4 | FREQUENCIES x/STATISTICS **.
+ | ^~"
+
+"frequencies.sps:5.27-5.28: error: FREQUENCIES: Syntax error expecting number between 0 and 100 for PERCENTILES.
+ 5 | FREQUENCIES x/PERCENTILES **.
+ | ^~"
+
+"frequencies.sps:6.28-6.29: error: FREQUENCIES: Syntax error expecting `('.
+ 6 | FREQUENCIES x/FORMAT LIMIT **.
+ | ^~"
+
+"frequencies.sps:7.28-7.29: error: FREQUENCIES: Syntax error expecting non-negative integer for LIMIT.
+ 7 | FREQUENCIES x/FORMAT LIMIT(**).
+ | ^~"
+
+"frequencies.sps:8.30-8.31: error: FREQUENCIES: Syntax error expecting `)'.
+ 8 | FREQUENCIES x/FORMAT LIMIT(5 **).
+ | ^~"
+
+"frequencies.sps:9.22-9.23: error: FREQUENCIES: Syntax error expecting TABLE, NOTABLE, LIMIT, AVALUE, DVALUE, AFREQ, or DFREQ.
+ 9 | FREQUENCIES x/FORMAT **.
+ | ^~"
+
+"frequencies.sps:10.22-10.23: error: FREQUENCIES: Syntax error expecting non-negative integer for NTILES.
+ 10 | FREQUENCIES x/NTILES **.
+ | ^~"
+
+"frequencies.sps:11.25-11.26: error: FREQUENCIES: Syntax error expecting COMPATIBLE or ENHANCED.
+ 11 | FREQUENCIES x/ALGORITHM **.
+ | ^~"
+
+"frequencies.sps:12.30-12.31: error: FREQUENCIES: Syntax error expecting positive integer for FREQ.
+ 12 | FREQUENCIES x/HISTOGRAM FREQ(**).
+ | ^~"
+
+"frequencies.sps:13.32-13.33: error: FREQUENCIES: Syntax error expecting `)'.
+ 13 | FREQUENCIES x/HISTOGRAM FREQ(5 **).
+ | ^~"
+
+"frequencies.sps:14.33-14.34: error: FREQUENCIES: Syntax error expecting positive integer for PERCENT.
+ 14 | FREQUENCIES x/HISTOGRAM PERCENT(**).
+ | ^~"
+
+"frequencies.sps:15.35-15.36: error: FREQUENCIES: Syntax error expecting `)'.
+ 15 | FREQUENCIES x/HISTOGRAM PERCENT(5 **).
+ | ^~"
+
+"frequencies.sps:16.33-16.34: error: FREQUENCIES: Syntax error expecting number for MINIMUM.
+ 16 | FREQUENCIES x/HISTOGRAM MINIMUM(**).
+ | ^~"
+
+"frequencies.sps:17.35-17.36: error: FREQUENCIES: Syntax error expecting `)'.
+ 17 | FREQUENCIES x/HISTOGRAM MINIMUM(5 **).
+ | ^~"
+
+"frequencies.sps:18.33-18.34: error: FREQUENCIES: Syntax error expecting number for MAXIMUM.
+ 18 | FREQUENCIES x/HISTOGRAM MAXIMUM(**).
+ | ^~"
+
+"frequencies.sps:19.35-19.36: error: FREQUENCIES: Syntax error expecting `)'.
+ 19 | FREQUENCIES x/HISTOGRAM MAXIMUM(5 **).
+ | ^~"
+
+"frequencies.sps:20.44: error: FREQUENCIES: Syntax error expecting number 5 or greater for MAXIMUM.
+ 20 | FREQUENCIES x/HISTOGRAM MINIMUM(5) MAXIMUM(1).
+ | ^"
+
+"frequencies.sps:21.44-21.45: error: FREQUENCIES: Syntax error expecting number less than or equal to 5 for MINIMUM.
+ 21 | FREQUENCIES x/HISTOGRAM MAXIMUM(5) MINIMUM(10).
+ | ^~"
+
+"frequencies.sps:22.25-22.26: error: FREQUENCIES: Syntax error expecting NORMAL, NONORMAL, FREQ, PERCENT, MINIMUM, or MAXIMUM.
+ 22 | FREQUENCIES x/HISTOGRAM **.
+ | ^~"
+
+"frequencies.sps:23.32-23.33: error: FREQUENCIES: Syntax error expecting number for MINIMUM.
+ 23 | FREQUENCIES x/PIECHART MINIMUM(**).
+ | ^~"
+
+"frequencies.sps:24.34-24.35: error: FREQUENCIES: Syntax error expecting `)'.
+ 24 | FREQUENCIES x/PIECHART MINIMUM(5 **).
+ | ^~"
+
+"frequencies.sps:25.32-25.33: error: FREQUENCIES: Syntax error expecting number for MAXIMUM.
+ 25 | FREQUENCIES x/PIECHART MAXIMUM(**).
+ | ^~"
+
+"frequencies.sps:26.34-26.35: error: FREQUENCIES: Syntax error expecting `)'.
+ 26 | FREQUENCIES x/PIECHART MAXIMUM(5 **).
+ | ^~"
+
+"frequencies.sps:27.43: error: FREQUENCIES: Syntax error expecting number 5 or greater for MAXIMUM.
+ 27 | FREQUENCIES x/PIECHART MINIMUM(5) MAXIMUM(1).
+ | ^"
+
+"frequencies.sps:28.43-28.44: error: FREQUENCIES: Syntax error expecting number less than or equal to 5 for MINIMUM.
+ 28 | FREQUENCIES x/PIECHART MAXIMUM(5) MINIMUM(10).
+ | ^~"
+
+"frequencies.sps:29.24-29.25: error: FREQUENCIES: Syntax error expecting MINIMUM, MAXIMUM, MISSING, or NOMISSING.
+ 29 | FREQUENCIES x/PIECHART **.
+ | ^~"
+
+"frequencies.sps:30.29-30.30: error: FREQUENCIES: Syntax error expecting positive number for FREQ.
+ 30 | FREQUENCIES x/BARCHART FREQ(**).
+ | ^~"
+
+"frequencies.sps:31.31-31.32: error: FREQUENCIES: Syntax error expecting `)'.
+ 31 | FREQUENCIES x/BARCHART FREQ(5 **).
+ | ^~"
+
+"frequencies.sps:32.32-32.33: error: FREQUENCIES: Syntax error expecting positive number for PERCENT.
+ 32 | FREQUENCIES x/BARCHART PERCENT(**).
+ | ^~"
+
+"frequencies.sps:33.34-33.35: error: FREQUENCIES: Syntax error expecting `)'.
+ 33 | FREQUENCIES x/BARCHART PERCENT(5 **).
+ | ^~"
+
+"frequencies.sps:34.32-34.33: error: FREQUENCIES: Syntax error expecting number for MINIMUM.
+ 34 | FREQUENCIES x/BARCHART MINIMUM(**).
+ | ^~"
+
+"frequencies.sps:35.34-35.35: error: FREQUENCIES: Syntax error expecting `)'.
+ 35 | FREQUENCIES x/BARCHART MINIMUM(5 **).
+ | ^~"
+
+"frequencies.sps:36.32-36.33: error: FREQUENCIES: Syntax error expecting number for MAXIMUM.
+ 36 | FREQUENCIES x/BARCHART MAXIMUM(**).
+ | ^~"
+
+"frequencies.sps:37.34-37.35: error: FREQUENCIES: Syntax error expecting `)'.
+ 37 | FREQUENCIES x/BARCHART MAXIMUM(5 **).
+ | ^~"
+
+"frequencies.sps:38.43: error: FREQUENCIES: Syntax error expecting number 5 or greater for MAXIMUM.
+ 38 | FREQUENCIES x/BARCHART MINIMUM(5) MAXIMUM(1).
+ | ^"
+
+"frequencies.sps:39.43-39.44: error: FREQUENCIES: Syntax error expecting number less than or equal to 5 for MINIMUM.
+ 39 | FREQUENCIES x/BARCHART MAXIMUM(5) MINIMUM(10).
+ | ^~"
+
+"frequencies.sps:40.24-40.25: error: FREQUENCIES: Syntax error expecting MINIMUM, MAXIMUM, FREQ, or PERCENT.
+ 40 | FREQUENCIES x/BARCHART **.
+ | ^~"
+
+"frequencies.sps:41.23-41.24: error: FREQUENCIES: Syntax error expecting EXCLUDE or INCLUDE.
+ 41 | FREQUENCIES x/MISSING **.
+ | ^~"
+
+"frequencies.sps:42.21-42.22: error: FREQUENCIES: Syntax error expecting ANALYSIS or VARIABLE.
+ 42 | FREQUENCIES x/ORDER **.
+ | ^~"
+
+"frequencies.sps:43.16-43.17: error: FREQUENCIES: Syntax error expecting one of the following: STATISTICS, PERCENTILES, FORMAT, NTILES, ALGORITHM, HISTOGRAM, PIECHART, BARCHART, MISSING, ORDER.
+ 43 | FREQUENCIES x/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([GET DATA /TYPE=PSQL])
+
+m4_define([INIT_PSQL],
+ [AT_SKIP_IF([test "$PSQL_SUPPORT" = no])
+ PGDATA=`pwd`/cluster
+ export PGDATA
+ PGPORT=$PG_PORT
+ export PGPORT
+ socket_dir=`mktemp -d`
+ PGHOST="$socket_dir"
+ export PGHOST
+ AT_CHECK([PATH=$PG_PATH:$PATH initdb -A trust], [0], [ignore])
+ AT_CHECK([PATH=$PG_PATH:$PATH pg_ctl start -w -o "-k $socket_dir -h ''"], [0], [ignore])
+ trap 'CLEANUP_PSQL' 0
+ AT_CHECK([PATH=$PG_PATH:$PATH createdb -h "$socket_dir" -p $PG_PORT $PG_DBASE],
+ [0], [ignore], [ignore])
+ AT_DATA([populate.sql],
+ [CREATE TABLE empty (a int, b date, c numeric(23, 4));
+
+ -- a largeish table to check big queries work ok.
+ CREATE TABLE large (x int);
+ INSERT INTO large (select * from generate_series(1, 1000));
+
+
+ CREATE TABLE thing (
+ bool bool ,
+ bytea bytea ,
+ char char ,
+ int8 int8 ,
+ int2 int2 ,
+ int4 int4 ,
+ numeric numeric(50,6) ,
+ text text ,
+ oid oid ,
+ float4 float4 ,
+ float8 float8 ,
+ money money ,
+ pbchar bpchar ,
+ varchar varchar ,
+ date date ,
+ time time ,
+ timestamp timestamp ,
+ timestamptz timestamptz ,
+ interval interval ,
+ timetz timetz
+ );
+
+ INSERT INTO thing VALUES (
+ false,
+ '0',
+ 'a',
+ '0',
+ 0,
+ 0,
+ -256.098,
+ 'this-long-text',
+ 0,
+ 0,
+ 0,
+ '0.01',
+ 'a',
+ 'A',
+ '1-Jan-2000',
+ '00:00',
+ 'January 8 04:05:06 1999',
+ 'January 8 04:05:06 1999 PST',
+ '1 minutes',
+ '10:09 UTC+4'
+ );
+
+ INSERT INTO thing VALUES (
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null,
+ null
+ );
+
+ INSERT INTO thing VALUES (
+ true,
+ '1',
+ 'b',
+ '1',
+ 1,
+ 1,
+ 65535.00001,
+ 'that-long-text',
+ 1,
+ 1,
+ 1,
+ '1.23',
+ 'b',
+ 'B',
+ '10-Jan-1963',
+ '01:05:02',
+ '10-Jan-1963 23:58:00',
+ '10-Jan-1963 23:58:00 CET',
+ '2 year 1 month 12 days 1 hours 3 minutes 4 seconds',
+ '01:05:02 UTC-7'
+ );
+])
+
+ # On Debian, the psql binary in the postgres bindir won't work because
+ # it needs libreadline to be LD_PRELOADed into it. The psql in the
+ # normal $PATH works fine though.
+ if (PATH=$PG_PATH:$PATH psql -V) >/dev/null 2>&1; then
+ psql () {
+ PATH=$PG_PATH:$PATH command psql "$$@@"
+ }
+ fi
+ AT_CHECK([psql -h "$socket_dir" -p $PG_PORT $PG_DBASE < populate.sql],
+ [0], [ignore])])
+
+m4_define([CLEANUP_PSQL], [PATH=$PG_PATH:$PATH pg_ctl stop -W -o "-k $socket_dir -h ''"])
+
+AT_SETUP([GET DATA /TYPE=PSQL])
+AT_KEYWORDS([slow])
+INIT_PSQL
+
+dnl Test with an ordinary query.
+AT_CHECK([cat > ordinary-query.sps <<EOF
+GET DATA /TYPE=psql
+ /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
+ /UNENCRYPTED
+ /SQL="select * from thing".
+
+DISPLAY DICTIONARY.
+
+LIST.
+EOF
+])
+AT_CHECK([pspp -o pspp.csv ordinary-query.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+bool,1,Scale,Input,8,Right,F8.2,F8.2
+bytea,2,Nominal,Input,1,Left,AHEX2,AHEX2
+char,3,Nominal,Input,8,Left,A8,A8
+int8,4,Scale,Input,8,Right,F8.2,F8.2
+int2,5,Scale,Input,8,Right,F8.2,F8.2
+int4,6,Scale,Input,8,Right,F8.2,F8.2
+numeric,7,Scale,Input,8,Right,E40.6,E40.6
+text,8,Nominal,Input,16,Left,A16,A16
+oid,9,Scale,Input,8,Right,F8.2,F8.2
+float4,10,Scale,Input,8,Right,F8.2,F8.2
+float8,11,Scale,Input,8,Right,F8.2,F8.2
+money,12,Scale,Input,8,Right,DOLLAR8.2,DOLLAR8.2
+pbchar,13,Nominal,Input,8,Left,A8,A8
+varchar,14,Nominal,Input,8,Left,A8,A8
+date,15,Scale,Input,8,Right,DATE11,DATE11
+time,16,Scale,Input,8,Right,TIME11.0,TIME11.0
+timestamp,17,Scale,Input,8,Right,DATETIME22.0,DATETIME22.0
+timestamptz,18,Scale,Input,8,Right,DATETIME22.0,DATETIME22.0
+interval,19,Scale,Input,8,Right,DTIME13.0,DTIME13.0
+interval_months,20,Scale,Input,8,Right,F3.0,F3.0
+timetz,21,Scale,Input,8,Right,TIME11.0,TIME11.0
+timetz_zone,22,Scale,Input,8,Right,F8.2,F8.2
+
+Table: Data List
+bool,bytea,char,int8,int2,int4,numeric,text,oid,float4,float8,money,pbchar,varchar,date,time,timestamp,timestamptz,interval,interval_months,timetz,timetz_zone
+.00,30,a,.00,.00,.00,-2.560980E+002,this-long-text,.00,.00,.00,$.01,a,A,01-JAN-2000,00:00:00,08-JAN-1999 04:05:06,08-JAN-1999 12:05:06,0 00:01:00,0,10:09:00,4.00
+. ,,,. ,. ,. ,. ,,. ,. ,. ,. ,,,.,.,.,.,.,.,.,. @&t@
+1.00,31,b,1.00,1.00,1.00,6.553500E+004,that-long-text,.00,1.00,1.00,$1.23,b,B,10-JAN-1963,01:05:02,10-JAN-1963 23:58:00,10-JAN-1963 22:58:00,12 01:03:04,25,01:05:02,-7.00
+])
+
+dnl Test query with empty result set.
+AT_CHECK([cat > empty-result.sps <<EOF
+GET DATA /TYPE=psql
+ /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
+ /UNENCRYPTED
+ /SQL="select * from empty".
+
+DISPLAY DICTIONARY.
+
+LIST.
+EOF
+])
+AT_CHECK([pspp -o pspp.csv empty-result.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+a,1,Scale,Input,8,Right,F8.2,F8.2
+b,2,Scale,Input,8,Right,DATE11,DATE11
+c,3,Scale,Input,8,Right,E40.2,E40.2
+])
+
+dnl Test query with large result set.
+AT_CHECK([cat > large-result.sps <<EOF
+GET DATA /TYPE=psql
+ /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
+ /UNENCRYPTED
+ /SQL="select * from large".
+
+NUMERIC diff.
+COMPUTE diff = x - lag (x).
+
+TEMPORARY.
+SELECT IF (diff <> 1).
+LIST.
+
+TEMPORARY.
+N OF CASES 6.
+LIST.
+
+SORT CASES BY x (D).
+
+TEMPORARY.
+N OF CASES 6.
+LIST.
+EOF
+])
+AT_CHECK([pspp -o pspp.csv large-result.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+x,diff
+1.00,. @&t@
+2.00,1.00
+3.00,1.00
+4.00,1.00
+5.00,1.00
+6.00,1.00
+
+Table: Data List
+x,diff
+1000.00,1.00
+999.00,1.00
+998.00,1.00
+997.00,1.00
+996.00,1.00
+995.00,1.00
+])
+
+dnl Check for a bug caused by having string variables in the database,
+dnl all of which are null.
+AT_DATA([all-null-string.sql],
+ [-- a table which has a text field containing only null, or zero
+ -- length entries.
+
+ CREATE TABLE foo (int4 int4, text text);
+
+ INSERT INTO foo VALUES ('12', '');
+
+ INSERT INTO foo VALUES (null, '');
+])
+AT_CHECK([psql -h "$socket_dir" -p $PG_PORT $PG_DBASE < all-null-string.sql],
+ [0], [ignore])
+AT_CAPTURE_FILE([get-data.sps])
+AT_CHECK([cat > get-data.sps <<EOF
+GET DATA /TYPE=psql
+ /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
+ /UNENCRYPTED
+ /SQL="select * from foo".
+
+DISPLAY DICTIONARY.
+
+LIST.
+EOF
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CAPTURE_FILE([pspp.csv])
+rm -rf "$socket_dir"
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+m4_define([SPREADSHEET_TEST_PREP],[dnl
+ AT_KEYWORDS([spreadsheet])
+ m4_if($1,[GNM],[dnl
+ AT_CHECK([gzip -c $top_srcdir/tests/language/commands/Book1.gnm.unzipped > Book1.gnumeric])dnl
+ m4_define([testsheet],[Book1.gnumeric])dnl
+ ]) dnl
+ m4_if($1,[ODS],[dnl
+ AT_CHECK([cp $top_srcdir/tests/language/commands/test.ods test.ods])dnl
+ m4_define([testsheet],[test.ods])dnl
+ ])dnl
+])
+
+m4_define([CHECK_SPREADSHEET_READER],
+ [dnl
+AT_SETUP([GET DATA /TYPE=$1 with CELLRANGE])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+GET DATA /TYPE=$1 /FILE='testsheet' /READNAMES=off /SHEET=name 'This' /CELLRANGE=range 'g9:i13' .
+DISPLAY VARIABLES.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Print Format,Write Format
+VAR001,1,F8.2,F8.2
+VAR002,2,A8,A8
+VAR003,3,F8.2,F8.2
+
+Table: Data List
+VAR001,VAR002,VAR003
+.00,fred,20.00
+1.00,11,21.00
+2.00,twelve,22.00
+3.00,13,23.00
+4.00,14,24.00
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=$1 with CELLRANGE and READNAMES])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+GET DATA /TYPE=$1 /FILE='testsheet' /READNAMES=on /SHEET=name 'This' /CELLRANGE=range 'g8:i13' .
+DISPLAY VARIABLES.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Print Format,Write Format
+V1,1,F8.2,F8.2
+V2,2,A8,A8
+VAR001,3,F8.2,F8.2
+
+Table: Data List
+V1,V2,VAR001
+.00,fred,20.00
+1.00,11,21.00
+2.00,twelve,22.00
+3.00,13,23.00
+4.00,14,24.00
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=$1 without CELLRANGE])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+GET DATA /TYPE=$1 /FILE='testsheet' /SHEET=index 3.
+DISPLAY VARIABLES.
+LIST.
+])
+AT_CHECK([pspp -O format=csv get-data.sps], [0], [dnl
+Table: Variables
+Name,Position,Print Format,Write Format
+name,1,A8,A8
+id,2,F8.2,F8.2
+height,3,F8.2,F8.2
+
+warning: Cannot convert the value in the spreadsheet cell C4 to format (F8.2): Field contents are not numeric.
+
+Table: Data List
+name,id,height
+fred,.00,23.40
+bert,1.00,.56
+charlie,2.00,. @&t@
+dick,3.00,-34.09
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=$1 with missing data])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+* This sheet has no data in one of its variables
+GET DATA /TYPE=$1 /FILE='testsheet' /READNAMES=on /SHEET=index 5.
+DISPLAY VARIABLES.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Print Format,Write Format
+vone,1,F8.2,F8.2
+vtwo,2,F8.2,F8.2
+vthree,3,A8,A8
+v4,4,F8.2,F8.2
+
+Table: Data List
+vone,vtwo,vthree,v4
+1.00,3.00,,5.00
+2.00,4.00,,6.00
+])
+AT_CLEANUP
+
+dnl This syntax doesn't do anything particularly useful.
+dnl It has been seen to cause a few crashes, so we check here that it
+dnl doesn't do anthing bad.
+AT_SETUP([GET DATA /TYPE=$1 with no options])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+* This sheet is empty
+GET DATA /TYPE=$1 /FILE='testsheet'.
+DISPLAY DICTIONARY.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [ignore])
+AT_CLEANUP
+
+
+
+AT_SETUP([GET DATA /TYPE=$1 with empty sheet])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+* This sheet is empty
+GET DATA /TYPE=$1 /FILE='testsheet' /SHEET=name 'Empty'.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [dnl
+warning: Selected sheet or range of spreadsheet `testsheet' is empty.
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=$1 with nonexistent sheet])
+SPREADSHEET_TEST_PREP($1)
+AT_DATA([get-data.sps], [dnl
+* This sheet doesnt exist.
+GET DATA /TYPE=$1 /FILE='testsheet' /SHEET=name 'foobarxx'.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [dnl
+warning: Selected sheet or range of spreadsheet `testsheet' is empty.
+])
+AT_CLEANUP
+])
+
+
+AT_BANNER([GET DATA Spreadsheet /TYPE=GNM])
+
+CHECK_SPREADSHEET_READER([GNM])
+
+dnl Check for a bug where gnumeric files were interpreted incorrectly
+AT_SETUP([GET DATA /TYPE=GNM sheet index bug])
+AT_KEYWORDS([spreadsheet])
+AT_DATA([minimal3.gnumeric],[dnl
+<?xml version="1.0" encoding="UTF-8"?>
+<gnm:Workbook xmlns:gnm="http://www.gnumeric.org/v10.dtd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gnumeric.org/v9.xsd">
+ <gnm:Version Epoch="1" Major="10" Minor="8" Full="1.10.8"/>
+ <gnm:SheetNameIndex>
+ <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet1</gnm:SheetName>
+ <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet2</gnm:SheetName>
+ <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet3</gnm:SheetName>
+ </gnm:SheetNameIndex>
+ <gnm:Sheets>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
+ <gnm:Name>Sheet1</gnm:Name>
+ <gnm:MaxCol>2</gnm:MaxCol>
+ <gnm:MaxRow>3</gnm:MaxRow>
+ <gnm:Names>
+ <gnm:Name>
+ <gnm:name>Print_Area</gnm:name>
+ <gnm:value>#REF!</gnm:value>
+ <gnm:position>A1</gnm:position>
+ </gnm:Name>
+ <gnm:Name>
+ <gnm:name>Sheet_Title</gnm:name>
+ <gnm:value>"Sheet1"</gnm:value>
+ <gnm:position>A1</gnm:position>
+ </gnm:Name>
+ </gnm:Names>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="94.5" HardSize="1"/>
+ <gnm:ColInfo No="1" Unit="48" Count="2"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="13.5" Count="4"/>
+ </gnm:Rows>
+ <gnm:Cells>
+ <gnm:Cell Row="0" Col="0" ValueType="60">Name</gnm:Cell>
+ <gnm:Cell Row="0" Col="1" ValueType="60">x</gnm:Cell>
+ <gnm:Cell Row="0" Col="2" ValueType="60">y</gnm:Cell>
+ <gnm:Cell Row="1" Col="0" ValueType="60">Sheet One</gnm:Cell>
+ <gnm:Cell Row="1" Col="1" ValueType="40">1</gnm:Cell>
+ <gnm:Cell Row="1" Col="2" ValueType="40">2</gnm:Cell>
+ <gnm:Cell Row="2" Col="0" ValueType="60">foo</gnm:Cell>
+ <gnm:Cell Row="2" Col="1" ValueType="40">3</gnm:Cell>
+ <gnm:Cell Row="2" Col="2" ValueType="40">4</gnm:Cell>
+ <gnm:Cell Row="3" Col="0" ValueType="60">bar</gnm:Cell>
+ <gnm:Cell Row="3" Col="1" ValueType="40">5</gnm:Cell>
+ <gnm:Cell Row="3" Col="2" ValueType="40">6</gnm:Cell>
+ </gnm:Cells>
+ </gnm:Sheet>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
+ <gnm:Name>Sheet2</gnm:Name>
+ <gnm:MaxCol>2</gnm:MaxCol>
+ <gnm:MaxRow>2</gnm:MaxRow>
+ <gnm:Names>
+ <gnm:Name>
+ <gnm:name>Print_Area</gnm:name>
+ <gnm:value>#REF!</gnm:value>
+ <gnm:position>A1</gnm:position>
+ </gnm:Name>
+ <gnm:Name>
+ <gnm:name>Sheet_Title</gnm:name>
+ <gnm:value>"Sheet2"</gnm:value>
+ <gnm:position>A1</gnm:position>
+ </gnm:Name>
+ </gnm:Names>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="48"/>
+ <gnm:ColInfo No="1" Unit="57.75"/>
+ <gnm:ColInfo No="2" Unit="54.75"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="13.5" Count="3"/>
+ </gnm:Rows>
+ <gnm:Cells>
+ <gnm:Cell Row="0" Col="0" ValueType="60">Comment</gnm:Cell>
+ <gnm:Cell Row="0" Col="1" ValueType="60">DOB</gnm:Cell>
+ <gnm:Cell Row="0" Col="2" ValueType="60">wealth</gnm:Cell>
+ <gnm:Cell Row="1" Col="0" ValueType="60">Sheet Two</gnm:Cell>
+ <gnm:Cell Row="1" Col="1" ValueType="60">24/5/1966</gnm:Cell>
+ <gnm:Cell Row="1" Col="2" ValueType="40" ValueFormat="_($* 0.00_);_($* (0.00);_($* "-"??_);_(@_)">0.02</gnm:Cell>
+ <gnm:Cell Row="2" Col="0" ValueType="60">wee</gnm:Cell>
+ <gnm:Cell Row="2" Col="1" ValueType="40" ValueFormat="dd/mm/yyyy">37145</gnm:Cell>
+ <gnm:Cell Row="2" Col="2" ValueType="40" ValueFormat="_($* 0.00_);_($* (0.00);_($* "-"??_);_(@_)">3000</gnm:Cell>
+ </gnm:Cells>
+ </gnm:Sheet>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
+ <gnm:Name>Sheet3</gnm:Name>
+ <gnm:MaxCol>2</gnm:MaxCol>
+ <gnm:MaxRow>2</gnm:MaxRow>
+ <gnm:Names>
+ <gnm:Name>
+ <gnm:name>Print_Area</gnm:name>
+ <gnm:value>#REF!</gnm:value>
+ <gnm:position>A1</gnm:position>
+ </gnm:Name>
+ <gnm:Name>
+ <gnm:name>Sheet_Title</gnm:name>
+ <gnm:value>"Sheet3"</gnm:value>
+ <gnm:position>A1</gnm:position>
+ </gnm:Name>
+ </gnm:Names>
+ <gnm:Cols DefaultSizePts="48">
+ <gnm:ColInfo No="0" Unit="48" Count="3"/>
+ </gnm:Cols>
+ <gnm:Rows DefaultSizePts="12.75">
+ <gnm:RowInfo No="0" Unit="13.5"/>
+ <gnm:RowInfo No="1" Unit="12.75" Count="2"/>
+ </gnm:Rows>
+ <gnm:Cells>
+ <gnm:Cell Row="0" Col="0" ValueType="40">3</gnm:Cell>
+ <gnm:Cell Row="0" Col="1" ValueType="40">4</gnm:Cell>
+ <gnm:Cell Row="0" Col="2" ValueType="40">5</gnm:Cell>
+ <gnm:Cell Row="1" Col="0" ValueType="40">6</gnm:Cell>
+ <gnm:Cell Row="1" Col="1" ValueType="40">7</gnm:Cell>
+ <gnm:Cell Row="1" Col="2" ValueType="40">8</gnm:Cell>
+ <gnm:Cell Row="2" Col="0" ValueType="40">9</gnm:Cell>
+ <gnm:Cell Row="2" Col="1" ValueType="40">10</gnm:Cell>
+ <gnm:Cell Row="2" Col="2" ValueType="40">11</gnm:Cell>
+ </gnm:Cells>
+ </gnm:Sheet>
+ </gnm:Sheets>
+</gnm:Workbook>
+])
+
+AT_DATA([gnum.sps], [dnl
+GET DATA
+ /TYPE=GNM
+ /FILE='minimal3.gnumeric'
+ /SHEET=index 3
+ /READNAMES=off
+ .
+
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv gnum.sps], [0], [dnl
+Table: Data List
+VAR001,VAR002,VAR003
+3,4.00,5.00
+6,7.00,8.00
+9,10.00,11.00
+])
+
+
+AT_CLEANUP
+
+
+dnl Check for a bug where certain gnumeric files failed an assertion
+AT_SETUP([GET DATA /TYPE=GNM assert-fail])
+AT_KEYWORDS([spreadsheet])
+AT_DATA([read.sps],[dnl
+GET DATA
+ /TYPE=GNM
+ /FILE='crash.gnumeric'
+ .
+list.
+])
+
+
+AT_DATA([crash.gnumeric],[dnl
+<?xml version="1.0" encoding="UTF-8"?>
+<gnm:Workbook xmlns:gnm="http://www.gnumeric.org/v10.dtd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gnumeric.org/v9.xsd">
+ <office:document-meta xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:meta="urn:oasis:names:tc:opendocument:xmlns:meta:1.0" xmlns:ooo="http://openoffice.org/2004/office" office:version="1.1">
+ </office:document-meta>
+ <gnm:SheetNameIndex>
+ <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet1</gnm:SheetName>
+ </gnm:SheetNameIndex>
+ <gnm:Sheets>
+ <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
+ <gnm:Name>Sheet1</gnm:Name>
+ <gnm:MaxCol>2</gnm:MaxCol>
+ <gnm:MaxRow>4</gnm:MaxRow>
+ <gnm:Styles>
+ <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
+ <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
+ </gnm:Style>
+ </gnm:StyleRegion>
+ </gnm:Styles>
+ <gnm:Cells>
+ <gnm:Cell Row="1" Col="1" ValueType="60">one</gnm:Cell>
+ <gnm:Cell Row="1" Col="2" ValueType="60">two</gnm:Cell>
+ <gnm:Cell Row="2" Col="1" ValueType="40">1</gnm:Cell>
+ <gnm:Cell Row="2" Col="2" ValueType="40">2</gnm:Cell>
+ <gnm:Cell Row="3" Col="1" ValueType="40">1</gnm:Cell>
+ <gnm:Cell Row="3" Col="2" ValueType="40">2</gnm:Cell>
+ <gnm:Cell Row="4" Col="1" ValueType="40">1</gnm:Cell>
+ <gnm:Cell Row="4" Col="2" ValueType="40">2</gnm:Cell>
+ </gnm:Cells>
+ </gnm:Sheet>
+ </gnm:Sheets>
+</gnm:Workbook>
+])
+
+AT_CHECK([pspp -O format=csv read.sps], [0], [ignore])
+
+
+AT_CLEANUP
+
+
+
+AT_BANNER([GET DATA Spreadsheet /TYPE=ODS])
+
+CHECK_SPREADSHEET_READER([ODS])
+
+
+AT_SETUP([GET DATA /TYPE=ODS crash])
+AT_KEYWORDS([spreadsheet])
+
+
+AT_CHECK([cp $top_srcdir/tests/language/commands/newone.ods this.ods])dnl
+
+AT_DATA([crash.sps],[dnl
+GET DATA /TYPE=ODS /FILE='this.ods' /CELLRANGE=RANGE 'A1:C8' /READNAMES=ON
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv crash.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([GET DATA /TYPE=ODS readnames])
+AT_KEYWORDS([spreadsheet])
+
+dnl Check for a bug where in the ODS reader /READNAMES incorrectly
+dnl dealt with repeated names.
+AT_CHECK([cp $top_srcdir/tests/language/commands/readnames.ods this.ods])dnl
+
+AT_DATA([readnames.sps],[dnl
+GET DATA /TYPE=ODS /FILE='this.ods' /CELLRANGE=RANGE 'A1:H8' /READNAMES=ON
+EXECUTE.
+DISPLAY DICTIONARY.
+LIST.
+])
+
+
+AT_CHECK([pspp -O format=csv readnames.sps], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+freda,1,Nominal,Input,8,Right,F8.2,F8.2
+fred,2,Nominal,Input,8,Right,F8.2,F8.2
+fred_A,3,Nominal,Input,8,Right,F8.2,F8.2
+fred_B,4,Nominal,Input,8,Right,F8.2,F8.2
+fred_C,5,Nominal,Input,8,Right,F8.2,F8.2
+fred_D,6,Nominal,Input,8,Right,F8.2,F8.2
+fred_E,7,Nominal,Input,8,Right,F8.2,F8.2
+
+Table: Data List
+freda,fred,fred_A,fred_B,fred_C,fred_D,fred_E
+1.00,2.00,3.00,4.00,5.00,6.00,7.00
+8.00,9.00,10.00,11.00,12.00,13.00,14.00
+])
+
+AT_CLEANUP
+
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([GET DATA /TYPE=TXT])
+
+dnl These tests exercise features of GET DATA /TYPE=TXT that
+dnl it has in common with DATA LIST, using tests drawn from
+dnl DATA LIST.
+
+AT_SETUP([GET DATA /TYPE=TXT with explicit delimiters])
+AT_DATA([get-data.sps], [dnl
+get data /type=txt /file=inline /delimiters="|X"
+ /variables=A f7.2 B f7.2 C f7.2 D f7.2.
+begin data.
+1|23X45|2.03
+2X22|34|23|
+3|34|34X34
+end data.
+
+list.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+A,B,C,D
+1.00,23.00,45.00,2.03
+2.00,22.00,34.00,23.00
+3.00,34.00,34.00,34.00
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=TXT with FIRSTCASE])
+AT_DATA([get-data.sps], [dnl
+get data /type=txt /file=inline /delimiters=', ' /delcase=variables 4
+ /firstcase=2 /variables=A f7.2 B f7.2 C f7.2 D f7.2.
+begin data.
+# This record is ignored.
+,1,2,3
+,4,,5
+6
+7,
+
+8 9
+0,1,,,
+
+,,,,
+
+2
+
+3
+4
+5
+end data.
+list.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [ignore])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+A,B,C,D
+. ,1.00,2.00,3.00
+. ,4.00,. ,5.00
+6.00,7.00,. ,8.00
+9.00,.00,1.00,. @&t@
+. ,. ,. ,. @&t@
+. ,. ,. ,2.00
+. ,3.00,4.00,5.00
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=TXT with FIRSTCASE and tab delimiter])
+AT_DATA([get-data.sps], [dnl
+get data /type=txt /file=inline /delimiters='\t' /delcase=variables 4
+ /firstcase=3 /variables=A f7.2 B f7.2 C f7.2 D f7.2.
+begin data.
+# These records
+# are skipped.
+1 2 3 4
+1 2 3 4 @&t@
+1 2 4
+1 2 4 @&t@
+1 3 4
+1 3 4 @&t@
+1 4
+1 4 @&t@
+ 2 3 4
+ 2 3 4 @&t@
+ 2 4
+ 2 4 @&t@
+ 3 4
+ 3 4 @&t@
+ 4
+ 4 @&t@
+end data.
+list.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+A,B,C,D
+1.00,2.00,3.00,4.00
+1.00,2.00,3.00,4.00
+1.00,2.00,. ,4.00
+1.00,2.00,. ,4.00
+1.00,. ,3.00,4.00
+1.00,. ,3.00,4.00
+1.00,. ,. ,4.00
+1.00,. ,. ,4.00
+. ,2.00,3.00,4.00
+. ,2.00,3.00,4.00
+. ,2.00,. ,4.00
+. ,2.00,. ,4.00
+. ,. ,3.00,4.00
+. ,. ,3.00,4.00
+. ,. ,. ,4.00
+. ,. ,. ,4.00
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=TXT with multiple records per case])
+AT_DATA([get-data.sps], [dnl
+get data /type=txt /file=inline /arrangement=fixed /fixcase=3 /variables=
+ /1 start 0-19 adate8
+ /2 end 0-19 adate
+ /3 count 0-2 f.
+begin data.
+07-22-2007
+10-06-2007
+321
+07-14-1789
+08-26-1789
+4
+01-01-1972
+12-31-1999
+682
+end data.
+list.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+start,end,count
+07/22/07,10/06/2007,321
+********,08/26/1789,4
+01/01/72,12/31/1999,682
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=TXT with empty trailing record])
+AT_DATA([get-data.sps], [dnl
+get data /type=txt /file=inline /arrangement=fixed /fixcase=2 /variables=
+ /1 x 0 f
+ y 1 f.
+begin data.
+12
+
+34
+
+56
+
+78
+
+90
+
+end data.
+list.
+])
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+x,y
+1,2
+3,4
+5,6
+7,8
+9,0
+])
+AT_CLEANUP
+
+dnl This test is a copy of an example given in the manual
+dnl in doc/files.texi.
+AT_SETUP([GET DATA /TYPE=TXT password example])
+AT_DATA([passwd.data], [dnl
+root:$1$nyeSP5gD$pDq/:0:0:,,,:/root:/bin/bash
+blp:$1$BrP/pFg4$g7OG:1000:1000:Ben Pfaff,,,:/home/blp:/bin/bash
+john:$1$JBuq/Fioq$g4A:1001:1001:John Darrington,,,:/home/john:/bin/bash
+jhs:$1$D3li4hPL$88X1:1002:1002:Jason Stover,,,:/home/jhs:/bin/csh
+])
+AT_DATA([passwd.sps], [dnl
+GET DATA /TYPE=TXT /FILE='passwd.data' /DELIMITERS=':'
+ /VARIABLES=username A20
+ password A40
+ uid F10
+ gid F10
+ gecos A40
+ home A40
+ shell A40.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv passwd.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+username,password,uid,gid,gecos,home,shell
+root,$1$nyeSP5gD$pDq/,0,0,",,,",/root,/bin/bash
+blp,$1$BrP/pFg4$g7OG,1000,1000,"Ben Pfaff,,,",/home/blp,/bin/bash
+john,$1$JBuq/Fioq$g4A,1001,1001,"John Darrington,,,",/home/john,/bin/bash
+jhs,$1$D3li4hPL$88X1,1002,1002,"Jason Stover,,,",/home/jhs,/bin/csh
+])
+AT_CLEANUP
+
+dnl This test is a copy of an example given in the manual
+dnl in doc/files.texi.
+AT_SETUP([GET DATA /TYPE=TXT cars example])
+AT_DATA([cars.data], [dnl
+model year mileage price type age
+Civic 2002 29883 15900 Si 2
+Civic 2003 13415 15900 EX 1
+Civic 1992 107000 3800 n/a 12
+Accord 2002 26613 17900 EX 1
+])
+AT_DATA([cars.sps], [dnl
+GET DATA /TYPE=TXT /FILE='cars.data' /DELIMITERS=' ' /FIRSTCASE=2
+ /VARIABLES=model A8
+ year F4
+ mileage F6
+ price F5
+ type A4
+ age F2.
+LIST.
+
+GET DATA /TYPE=TXT /FILE='cars.data' /ARRANGEMENT=FIXED /FIRSTCASE=2
+ /VARIABLES=model 0-7 A
+ year 8-15 F
+ mileage 16-23 F
+ price 24-31 F
+ type 32-39 A
+ age 40-47 F.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv cars.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+model,year,mileage,price,type,age
+Civic,2002,29883,15900,Si,2
+Civic,2003,13415,15900,EX,1
+Civic,1992,107000,3800,n/a,12
+Accord,2002,26613,17900,EX,1
+
+Table: Data List
+model,year,mileage,price,type,age
+Civic,2002,29883,15900,Si,2
+Civic,2003,13415,15900,EX,1
+Civic,1992,107000,3800,n/a,12
+Accord,2002,26613,17900,EX,1
+])
+AT_CLEANUP
+
+dnl This test is a copy of an example given in the manual
+dnl in doc/files.texi.
+AT_SETUP([GET DATA /TYPE=TXT pets example])
+AT_DATA([pets.data], [dnl
+'Pet''s Name', "Age", "Color", "Date Received", "Price", "Height", "Type"
+, (Years), , , (Dollars), ,
+"Rover", 4.5, Brown, "12 Feb 2004", 80, '1''4"', "Dog"
+"Charlie", , Gold, "5 Apr 2007", 12.3, "3""", "Fish"
+"Molly", 2, Black, "12 Dec 2006", 25, '5"', "Cat"
+"Gilly", , White, "10 Apr 2007", 10, "3""", "Guinea Pig"
+])
+AT_DATA([pets.sps], [dnl
+GET DATA /TYPE=TXT /FILE='pets.data' /DELIMITERS=', ' /QUALIFIER='''"'
+ /FIRSTCASE=3
+ /VARIABLES=name A10
+ age F3.1
+ color A5
+ received EDATE10
+ price F5.2
+ height a5
+ type a10.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv pets.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+name,age,color,received,price,height,type
+Rover,4.5,Brown,12.02.2004,80.00,"1'4""",Dog
+Charlie,. ,Gold,05.04.2007,12.30,"3""",Fish
+Molly,2.0,Black,12.12.2006,25.00,"5""",Cat
+Gilly,. ,White,10.04.2007,10.00,"3""",Guinea Pig
+])
+AT_CLEANUP
+dnl " (fixes Emacs highlighting)
+
+AT_SETUP([GET DATA /TYPE=TXT with IMPORTCASE])
+AT_CHECK([$PYTHON3 > test.data -c '
+for i in range(1, 101):
+ print("%02d" % i)
+'])
+AT_DATA([get-data.sps], [dnl
+get data /type=txt /file='test.data' /importcase=first 10 /variables x f8.0.
+get data /type=txt /file='test.data' /importcase=percent 1 /variables x f8.0.
+get data /type=txt /file='test.data' /importcase=percent 35 /variables x f8.0.
+get data /type=txt /file='test.data' /importcase=percent 95 /variables x f8.0.
+get data /type=txt /file='test.data' /importcase=percent 100 /variables x f8.0.
+])
+AT_CHECK([pspp -O format=csv get-data.sps], [0], [dnl
+"get-data.sps:1.39-1.57: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
+ 1 | get data /type=txt /file='test.data' /importcase=first 10 /variables x f8.0.
+ | ^~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:2.39-2.58: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
+ 2 | get data /type=txt /file='test.data' /importcase=percent 1 /variables x f8.0.
+ | ^~~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:3.39-3.59: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
+ 3 | get data /type=txt /file='test.data' /importcase=percent 35 /variables x f8.0.
+ | ^~~~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:4.39-4.59: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
+ 4 | get data /type=txt /file='test.data' /importcase=percent 95 /variables x f8.0.
+ | ^~~~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:5.39-5.60: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
+ 5 | get data /type=txt /file='test.data' /importcase=percent 100 /variables x f8.0.
+ | ^~~~~~~~~~~~~~~~~~~~~~"
+])
+AT_CLEANUP
+
+AT_SETUP([GET DATA /TYPE=TXT with ENCODING subcommand])
+AT_CHECK([i18n-test supports_encodings UTF-8 ISO-8859-1])
+AT_DATA([get-data.sps], [dnl
+set locale='utf-8'
+get data /type=txt /file='data.txt' /encoding='iso-8859-1'
+ /delimiters="," /variables=s a8.
+list.
+])
+printf '\351' > data.txt # é in ISO-8859-1.
+AT_CHECK([pspp -o pspp.csv get-data.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+s
+é
+])
+AT_CLEANUP
+
+
+AT_SETUP([GET DATA /TYPE= truncated])
+
+AT_DATA([x.sps], [dnl
+GET DATA /TYPE=
+.
+])
+
+AT_CHECK([pspp -o pspp.csv x.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([GET DATA /TYPE=txt bug])
+
+
+AT_DATA([thing.txt], [dnl
+foo, title, last
+1, this, 1
+2, that, 2
+3, other, 3
+])
+
+AT_DATA([x.sps], [dnl
+GET DATA
+ /TYPE=TXT
+ /FILE="thing.txt"
+ /ARRANGEMENT=DELIMITED
+ /DELCASE=LINE
+ /FIRSTCASE=2
+ /DELIMITERS=","
+ /VARIABLES=foo F1.0
+ title A8
+ last F2.0.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv x.sps], [0], [dnl
+Table: Data List
+foo,title,last
+1,this,1
+2,that,2
+3,other,3
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([GET DATA /TYPE=txt another bug])
+
+AT_DATA([crash.sps], [dnl
+get data /type=txt /file=inline /variables=C f7.2 D f7>2.
+begin data.
+3 2
+4 2
+5 2
+end data.
+])
+
+AT_CHECK([pspp -O format=csv crash.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+
+
+
--- /dev/null
+AT_BANNER([GET DATA])
+
+AT_SETUP([GET DATA syntax errors])
+AT_DATA([get-data.sps], [dnl
+GET DATA **.
+GET DATA / **.
+GET DATA /TYPE **.
+
+GET DATA /TYPE=TXT **.
+GET DATA /TYPE=TXT/ **.
+GET DATA /TYPE=TXT/FILE **.
+GET DATA /TYPE=TXT/FILE='x.txt' **.
+GET DATA /TYPE=TXT/FILE='x.txt' /ENCODING=**.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=**.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /ARRANGEMENT=DELIMITED.
+GET DATA /TYPE=TXT/FILE='x.txt' /FIRSTCASE=**.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELCASE=LINE.
+GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=VARIABLES **.
+GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=**.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=DELIMITED /FIXCASE=1.
+GET DATA /TYPE=TXT/FILE='x.txt' /FIXCASE=**.
+GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=FIRST **.
+GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=PERCENT **.
+GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=ALL.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELIMITERS=' '.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /QUALIFIER='"'.
+GET DATA /TYPE=TXT/FILE='x.txt' /QUALIFIER='"' + "'".
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES / **.
+GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES **.
+GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES a_very_long_name_that_exceeds_the_64_byte_limit_for_variable_names.
+GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x **.
+GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x F1.2.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x **.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 **.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 FOO.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DATE.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DOLLAR1.2.
+GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 F x 6-10 F.
+
+GET DATA /TYPE=PSQL **.
+GET DATA /TYPE=PSQL/ **.
+GET DATA /TYPE=PSQL/CONNECT **.
+GET DATA /TYPE=PSQL/CONNECT='db'/ASSUMEDSTRWIDTH=**.
+GET DATA /TYPE=PSQL/CONNECT='db'/BSIZE=**.
+GET DATA /TYPE=PSQL/CONNECT='db'/SQL=**.
+
+GET DATA /TYPE=GNM **.
+GET DATA /TYPE=GNM/ **.
+GET DATA /TYPE=GNM/FILE **.
+GET DATA /TYPE=GNM/FILE= **.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/ASSUMEDSTRWIDTH=**.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=NAME **.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=INDEX **.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=**.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=RANGE **.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=**.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/READNAMES=**.
+GET DATA /TYPE=GNM/FILE='x.gnumeric'/ **.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='get-data.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp -x compatible --testing-mode -O format=csv insert.sps], [1], [dnl
+"get-data.sps:1.10-1.11: error: GET DATA: Syntax error expecting `/TYPE='.
+ 1 | GET DATA **.
+ | ^~"
+
+"get-data.sps:2.10-2.13: error: GET DATA: Syntax error expecting `/TYPE='.
+ 2 | GET DATA / **.
+ | ^~~~"
+
+"get-data.sps:3.10-3.17: error: GET DATA: Syntax error expecting `/TYPE='.
+ 3 | GET DATA /TYPE **.
+ | ^~~~~~~~"
+
+"get-data.sps:5.20-5.21: error: GET DATA: Syntax error expecting `/FILE='.
+ 5 | GET DATA /TYPE=TXT **.
+ | ^~"
+
+"get-data.sps:6.19-6.22: error: GET DATA: Syntax error expecting `/FILE='.
+ 6 | GET DATA /TYPE=TXT/ **.
+ | ^~~~"
+
+"get-data.sps:7.19-7.26: error: GET DATA: Syntax error expecting `/FILE='.
+ 7 | GET DATA /TYPE=TXT/FILE **.
+ | ^~~~~~~~"
+
+"get-data.sps:8.33-8.34: error: GET DATA: Syntax error expecting `/'.
+ 8 | GET DATA /TYPE=TXT/FILE='x.txt' **.
+ | ^~"
+
+"get-data.sps:9.43-9.44: error: GET DATA: Syntax error expecting string.
+ 9 | GET DATA /TYPE=TXT/FILE='x.txt' /ENCODING=**.
+ | ^~"
+
+"get-data.sps:10.46-10.47: error: GET DATA: Syntax error expecting FIXED or DELIMITED.
+ 10 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=**.
+ | ^~"
+
+get-data.sps:11: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
+
+"get-data.sps:11.53-11.73: note: GET DATA: This syntax requires DELIMITED arrangement.
+ 11 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /ARRANGEMENT=DELIMITED.
+ | ^~~~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:11.34-11.50: note: GET DATA: This syntax requires FIXED arrangement.
+ 11 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /ARRANGEMENT=DELIMITED.
+ | ^~~~~~~~~~~~~~~~~"
+
+"get-data.sps:12.44-12.45: error: GET DATA: Syntax error expecting positive integer for FIRSTCASE.
+ 12 | GET DATA /TYPE=TXT/FILE='x.txt' /FIRSTCASE=**.
+ | ^~"
+
+get-data.sps:13: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
+
+"get-data.sps:13.53-13.59: note: GET DATA: This syntax requires DELIMITED arrangement.
+ 13 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELCASE=LINE.
+ | ^~~~~~~"
+
+"get-data.sps:13.34-13.50: note: GET DATA: This syntax requires FIXED arrangement.
+ 13 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELCASE=LINE.
+ | ^~~~~~~~~~~~~~~~~"
+
+"get-data.sps:14.52-14.53: error: GET DATA: Syntax error expecting integer.
+ 14 | GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=VARIABLES **.
+ | ^~"
+
+"get-data.sps:15.42-15.43: error: GET DATA: Syntax error expecting LINE or VARIABLES.
+ 15 | GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=**.
+ | ^~"
+
+get-data.sps:16: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
+
+"get-data.sps:16.57-16.63: note: GET DATA: This syntax requires FIXED arrangement.
+ 16 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=DELIMITED /FIXCASE=1.
+ | ^~~~~~~"
+
+"get-data.sps:16.34-16.54: note: GET DATA: This syntax requires DELIMITED arrangement.
+ 16 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=DELIMITED /FIXCASE=1.
+ | ^~~~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:17.42-17.43: error: GET DATA: Syntax error expecting positive integer for FIXCASE.
+ 17 | GET DATA /TYPE=TXT/FILE='x.txt' /FIXCASE=**.
+ | ^~"
+
+"get-data.sps:18.52-18.53: error: GET DATA: Syntax error expecting integer.
+ 18 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=FIRST **.
+ | ^~"
+
+"get-data.sps:19.54-19.55: error: GET DATA: Syntax error expecting integer.
+ 19 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=PERCENT **.
+ | ^~"
+
+"get-data.sps:20.34-20.48: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
+ 20 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=ALL.
+ | ^~~~~~~~~~~~~~~"
+
+"get-data.sps:20.49: error: GET DATA: Syntax error expecting `/'.
+ 20 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=ALL.
+ | ^"
+
+get-data.sps:21: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
+
+"get-data.sps:21.53-21.62: note: GET DATA: This syntax requires DELIMITED arrangement.
+ 21 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELIMITERS=' '.
+ | ^~~~~~~~~~"
+
+"get-data.sps:21.34-21.50: note: GET DATA: This syntax requires FIXED arrangement.
+ 21 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELIMITERS=' '.
+ | ^~~~~~~~~~~~~~~~~"
+
+get-data.sps:22: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
+
+"get-data.sps:22.53-22.61: note: GET DATA: This syntax requires DELIMITED arrangement.
+ 22 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /QUALIFIER='""'.
+ | ^~~~~~~~~"
+
+"get-data.sps:22.34-22.50: note: GET DATA: This syntax requires FIXED arrangement.
+ 22 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /QUALIFIER='""'.
+ | ^~~~~~~~~~~~~~~~~"
+
+"get-data.sps:23.44-23.52: error: GET DATA: In compatible syntax mode, the QUALIFIER string must contain exactly one character.
+ 23 | GET DATA /TYPE=TXT/FILE='x.txt' /QUALIFIER='""' + ""'"".
+ | ^~~~~~~~~"
+
+"get-data.sps:24.65-24.66: error: GET DATA: Syntax error expecting integer.
+ 24 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES / **.
+ | ^~"
+
+"get-data.sps:25.44-25.45: error: GET DATA: Syntax error expecting identifier.
+ 25 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES **.
+ | ^~"
+
+"get-data.sps:26.44-26.109: error: GET DATA: Identifier `a_very_long_name_that_exceeds_the_64_byte_limit_for_variable_names' exceeds 64-byte limit.
+ 26 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES a_very_long_name_that_exceeds_the_64_byte_limit_for_variable_names.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"get-data.sps:27.46-27.47: error: GET DATA: Syntax error expecting valid format specifier.
+ 27 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x **.
+ | ^~"
+
+"get-data.sps:28.46-28.49: error: GET DATA: Input format F1.2 specifies 2 decimal places, but width 1 allows at most 1 decimals.
+ 28 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x F1.2.
+ | ^~~~"
+
+"get-data.sps:29.65-29.66: error: GET DATA: Syntax error expecting integer.
+ 29 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x **.
+ | ^~"
+
+"get-data.sps:30.69-30.70: error: GET DATA: Syntax error expecting valid format specifier.
+ 30 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 **.
+ | ^~"
+
+"get-data.sps:31.69-31.71: error: GET DATA: Unknown format type `FOO'.
+ 31 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 FOO.
+ | ^~~"
+
+"get-data.sps:32.65-32.72: error: GET DATA: Input format DATE5 specifies width 5, but DATE requires a width between 8 and 40.
+ 32 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DATE.
+ | ^~~~~~~~"
+
+"get-data.sps:33.65-33.77: error: GET DATA: Output format DOLLAR1.2 specifies width 1, but DOLLAR requires a width between 2 and 40.
+ 33 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DOLLAR1.2.
+ | ^~~~~~~~~~~~~"
+
+"get-data.sps:34.71: error: GET DATA: x is a duplicate variable name.
+ 34 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 F x 6-10 F.
+ | ^"
+
+"get-data.sps:36.21-36.22: error: GET DATA: Syntax error expecting `/CONNECT='.
+ 36 | GET DATA /TYPE=PSQL **.
+ | ^~"
+
+"get-data.sps:37.20-37.23: error: GET DATA: Syntax error expecting `/CONNECT='.
+ 37 | GET DATA /TYPE=PSQL/ **.
+ | ^~~~"
+
+"get-data.sps:38.20-38.30: error: GET DATA: Syntax error expecting `/CONNECT='.
+ 38 | GET DATA /TYPE=PSQL/CONNECT **.
+ | ^~~~~~~~~~~"
+
+"get-data.sps:39.50-39.51: error: GET DATA: Syntax error expecting integer between 1 and 32767 for ASSUMEDSTRWIDTH.
+ 39 | GET DATA /TYPE=PSQL/CONNECT='db'/ASSUMEDSTRWIDTH=**.
+ | ^~"
+
+"get-data.sps:40.40-40.41: error: GET DATA: Syntax error expecting positive integer for BSIZE.
+ 40 | GET DATA /TYPE=PSQL/CONNECT='db'/BSIZE=**.
+ | ^~"
+
+"get-data.sps:41.38-41.39: error: GET DATA: Syntax error expecting string.
+ 41 | GET DATA /TYPE=PSQL/CONNECT='db'/SQL=**.
+ | ^~"
+
+"get-data.sps:43.20-43.21: error: GET DATA: Syntax error expecting `/FILE='.
+ 43 | GET DATA /TYPE=GNM **.
+ | ^~"
+
+"get-data.sps:44.19-44.22: error: GET DATA: Syntax error expecting `/FILE='.
+ 44 | GET DATA /TYPE=GNM/ **.
+ | ^~~~"
+
+"get-data.sps:45.19-45.26: error: GET DATA: Syntax error expecting `/FILE='.
+ 45 | GET DATA /TYPE=GNM/FILE **.
+ | ^~~~~~~~"
+
+"get-data.sps:46.26-46.27: error: GET DATA: Syntax error expecting string.
+ 46 | GET DATA /TYPE=GNM/FILE= **.
+ | ^~"
+
+"get-data.sps:47.54-47.55: error: GET DATA: Syntax error expecting integer between 1 and 32767 for ASSUMEDSTRWIDTH.
+ 47 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/ASSUMEDSTRWIDTH=**.
+ | ^~"
+
+"get-data.sps:48.49-48.50: error: GET DATA: Syntax error expecting string.
+ 48 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=NAME **.
+ | ^~"
+
+"get-data.sps:49.50-49.51: error: GET DATA: Syntax error expecting positive integer for INDEX.
+ 49 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=INDEX **.
+ | ^~"
+
+"get-data.sps:50.44-50.45: error: GET DATA: Syntax error expecting NAME or INDEX.
+ 50 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=**.
+ | ^~"
+
+"get-data.sps:51.54-51.55: error: GET DATA: Syntax error expecting string.
+ 51 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=RANGE **.
+ | ^~"
+
+"get-data.sps:52.48-52.49: error: GET DATA: Syntax error expecting FULL or RANGE.
+ 52 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=**.
+ | ^~"
+
+"get-data.sps:53.48-53.49: error: GET DATA: Syntax error expecting ON or OFF.
+ 53 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/READNAMES=**.
+ | ^~"
+
+"get-data.sps:54.39-54.40: error: GET DATA: Syntax error expecting ASSUMEDSTRWIDTH, SHEET, CELLRANGE, or READNAMES.
+ 54 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([GET])
+
+dnl Tests for a bug which caused the second procedure
+dnl after GET to have corrupt input.
+AT_SETUP([GET data works in multiple procedures])
+AT_DATA([get.sps], [dnl
+DATA LIST LIST NOTABLE /LOCATION * EDITOR * SHELL * FREQ * .
+BEGIN DATA.
+ 1.00 1.00 1.0 2.00
+ 1.00 1.00 2.0 30.00
+ 1.00 2.00 1.0 8.00
+ 1.00 2.00 2.0 20.00
+ 2.00 1.00 1.0 2.00
+ 2.00 1.00 2.0 22.00
+ 2.00 2.00 1.0 1.00
+ 2.00 2.00 2.0 3.00
+END DATA.
+
+SAVE /OUTFILE='foo.sav'.
+
+GET /FILE='foo.sav'.
+
+* This one's ok
+LIST.
+
+* But this one get rubbish
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv get.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+LOCATION,EDITOR,SHELL,FREQ
+1.00,1.00,1.00,2.00
+1.00,1.00,2.00,30.00
+1.00,2.00,1.00,8.00
+1.00,2.00,2.00,20.00
+2.00,1.00,1.00,2.00
+2.00,1.00,2.00,22.00
+2.00,2.00,1.00,1.00
+2.00,2.00,2.00,3.00
+
+Table: Data List
+LOCATION,EDITOR,SHELL,FREQ
+1.00,1.00,1.00,2.00
+1.00,1.00,2.00,30.00
+1.00,2.00,1.00,8.00
+1.00,2.00,2.00,20.00
+2.00,1.00,1.00,2.00
+2.00,1.00,2.00,22.00
+2.00,2.00,1.00,1.00
+2.00,2.00,2.00,3.00
+])
+AT_CLEANUP
+
+dnl Tests for a bug that crashed when GET specified a nonexistent file.
+AT_SETUP([GET nonexistent file doesn't crash])
+dnl We use stdin here, because the bug seems to manifest itself only in
+dnl interactive mode.
+AT_CHECK([echo "GET /FILE='nonexistent.sav'." | pspp -O format=csv], [1], [dnl
+error: An error occurred while opening `nonexistent.sav': No such file or directory.
+])
+AT_CLEANUP
+
+dnl Tests for bug #15766 (/KEEP subcommand on SAVE doesn't
+dnl fully support ALL) and underlying problems.
+m4_define([GET_KEEP_ALL],
+ [AT_SETUP([GET with /KEEP=ALL crashes -- $1])
+ AT_DATA([get.sps], [dnl
+DATA LIST LIST NOTABLE
+ /a b c d e f g h i j k l m n o p q r s t u v w x y z (F2.0).
+BEGIN DATA.
+1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
+END DATA.
+LIST.
+SAVE OUTFILE='test.sav'/$1.
+GET FILE='test.sav'/KEEP=x y z all.
+LIST.
+])
+ AT_CHECK([pspp -o pspp.csv get.sps])
+ AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
+1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26
+
+Table: Data List
+x,y,z,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w
+24,25,26,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23
+])
+ AT_CLEANUP])
+GET_KEEP_ALL([uncompressed])
+GET_KEEP_ALL([compressed])
+
+
+dnl Test for a crash when no /TYPE was provided
+AT_SETUP([GET data no type])
+AT_DATA([get.sps], [dnl
+get data /file='anything'.
+])
+
+AT_CHECK([pspp get.sps], [1], [ignore])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([GLM procedure])
+
+AT_SETUP([GLM latin square design])
+AT_KEYWORDS([categorical categoricals])
+
+dnl This example comes from :
+dnl http://ssnds.uwo.ca/statsexamples/spssanova/latinsquareresults.html
+AT_DATA([latin.sps], [dnl
+set format = F20.3.
+data list notable fixed /a 1 b 3 c 5 y 7-10(2).
+begin data.
+1 1 6 3.5
+1 2 2 8.9
+1 3 3 9.6
+1 4 4 10.5
+1 5 5 3.1
+1 6 1 5.9
+2 1 2 4.2
+2 2 6 1.9
+2 3 5 3.7
+2 4 3 10.2
+2 5 1 7.2
+2 6 4 7.6
+3 1 1 6.7
+3 2 4 5.8
+3 3 6 -2.7
+3 4 2 4.6
+3 5 3 4.0
+3 6 5 -0.7
+4 1 4 6.6
+4 2 1 4.5
+4 3 2 3.7
+4 4 5 3.7
+4 5 6 -3.3
+4 6 3 3.0
+5 1 3 4.1
+5 2 5 2.4
+5 3 4 6.0
+5 4 1 5.1
+5 5 2 3.5
+5 6 6 4.0
+6 1 5 3.8
+6 2 3 5.8
+6 3 1 7.0
+6 4 6 3.8
+6 5 4 5.0
+6 6 2 8.6
+end data.
+
+variable labels a 'Factor A' b 'Factor B' c 'Factor C' y 'Criterion'.
+
+glm y by b a c
+ /intercept=include
+ /criteria=alpha(.05)
+ /design = a b c
+ .
+])
+
+AT_CHECK([pspp -O format=csv latin.sps | sed 's/329.62[[678]]/329.62/'], [0],
+ [dnl
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,263.064,15,17.538,5.269,.000
+Intercept,815.103,1,815.103,244.910,.000
+Factor A,78.869,5,15.774,4.739,.005
+Factor B,28.599,5,5.720,1.719,.176
+Factor C,155.596,5,31.119,9.350,.000
+Error,66.563,20,3.328,,
+Total,1144.730,36,,,
+Corrected Total,329.62,35,,,
+])
+
+AT_CLEANUP
+
+AT_SETUP([GLM 2 by 2 factorial design])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([2by2.sps], [dnl
+set format = F20.3.
+data list notable list /Factor0 * Factor1 * points (F10).
+begin data.
+1 4 332
+1 4 380
+1 4 371
+1 4 366
+1 4 354
+1 0 259.5
+1 0 302.5
+1 0 296
+1 0 349
+1 0 309
+2 4 354.67
+2 4 353.5
+2 4 304
+2 4 365
+2 4 339
+2 0 306
+2 0 339
+2 0 353
+2 0 351
+2 0 333
+end data.
+
+glm points by Factor0 Factor1
+ /intercept=include
+ /criteria=alpha(.05)
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv 2by2.sps ], [0],
+ [dnl
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,8667.053,3,2889.018,5.043,.012
+Intercept,2256018.640,1,2256018.640,3937.693,.000
+Factor0,313.394,1,313.394,.547,.470
+Factor1,5157.508,1,5157.508,9.002,.008
+Factor0 × Factor1,3196.150,1,3196.150,5.579,.031
+Error,9166.865,16,572.929,,
+Total,2273852.559,20,,,
+Corrected Total,17833.918,19,,,
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([GLM Type I and II Sums of Squares])
+AT_KEYWORDS([categorical categoricals])
+
+dnl The following example comes from
+dnl http://www.uvm.edu/~dhowell/StatPages/More_Stuff/Type1-3.pdf
+AT_DATA([data-inc.sps], [dnl
+set decimal = dot.
+set format=F20.3.
+data list notable list /dv * Agrp * B0 * B1 * B2 * i0 * i1 * i2 * sss *.
+begin data.
+5 1 1 0 0 1 0 0 1.00
+7 1 1 0 0 1 0 0 1.00
+9 1 1 0 0 1 0 0 1.00
+8 1 1 0 0 1 0 0 1.00
+2 1 0 1 0 0 1 0 1.00
+5 1 0 1 0 0 1 0 1.00
+7 1 0 1 0 0 1 0 1.00
+3 1 0 1 0 0 1 0 1.00
+9 1 0 1 0 0 1 0 1.00
+8 1 0 0 1 0 0 1 1.00
+11 1 0 0 1 0 0 1 1.00
+12 1 0 0 1 0 0 1 1.00
+14 1 0 0 1 0 0 1 1.00
+11 1 -1 -1 -1 -1 -1 -1 1.00
+15 1 -1 -1 -1 -1 -1 -1 1.00
+16 1 -1 -1 -1 -1 -1 -1 1.00
+10 1 -1 -1 -1 -1 -1 -1 1.00
+9 1 -1 -1 -1 -1 -1 -1 1.00
+7 -1 1 0 0 -1 0 0 2.00
+9 -1 1 0 0 -1 0 0 2.00
+10 -1 1 0 0 -1 0 0 2.00
+9 -1 1 0 0 -1 0 0 2.00
+3 -1 0 1 0 0 -1 0 2.00
+8 -1 0 1 0 0 -1 0 2.00
+9 -1 0 1 0 0 -1 0 2.00
+11 -1 0 1 0 0 -1 0 2.00
+9 -1 0 0 1 0 0 -1 2.00
+12 -1 0 0 1 0 0 -1 2.00
+14 -1 0 0 1 0 0 -1 2.00
+8 -1 0 0 1 0 0 -1 2.00
+7 -1 0 0 1 0 0 -1 2.00
+11 -1 -1 -1 -1 1 1 1 2.00
+14 -1 -1 -1 -1 1 1 1 2.00
+10 -1 -1 -1 -1 1 1 1 2.00
+12 -1 -1 -1 -1 1 1 1 2.00
+13 -1 -1 -1 -1 1 1 1 2.00
+11 -1 -1 -1 -1 1 1 1 2.00
+12 -1 -1 -1 -1 1 1 1 2.00
+end data.
+
+do if B0 = -1 AND B1 = -1 AND B2 = -1.
+compute Bgrp = 4.
+end if.
+
+do if B0 = 0 AND B1 = 0 AND B2 = 1.
+compute Bgrp = 3.
+end if.
+
+do if B0 = 0 AND B1 = 1 AND B2 = 0.
+compute Bgrp = 2.
+end if.
+
+do if B0 = 1 AND B1 = 0 AND B2 = 0.
+compute Bgrp = 1.
+end if.
+])
+
+AT_DATA([type1.sps], [dnl
+include 'data-inc.sps'.
+
+glm dv by Agrp Bgrp
+ /method = sstype (1)
+ .
+
+glm dv by Agrp Bgrp
+ /method = sstype (1)
+ /design Bgrp Agrp Bgrp * Agrp
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv type1.sps], [0],
+ [dnl
+Table: Tests of Between-Subjects Effects
+,Type I Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,216.017,7,30.860,5.046,.001
+Agrp,9.579,1,9.579,1.566,.220
+Bgrp,186.225,3,62.075,10.151,.000
+Agrp × Bgrp,20.212,3,6.737,1.102,.364
+Error,183.457,30,6.115,,
+Total,3810.000,38,,,
+Corrected Total,399.474,37,,,
+
+Table: Tests of Between-Subjects Effects
+,Type I Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,216.017,7,30.860,5.046,.001
+Bgrp,193.251,3,64.417,10.534,.000
+Agrp,2.553,1,2.553,.418,.523
+Bgrp × Agrp,20.212,3,6.737,1.102,.364
+Error,183.457,30,6.115,,
+Total,3810.000,38,,,
+Corrected Total,399.474,37,,,
+])
+
+
+AT_DATA([type2.sps], [dnl
+include 'data-inc.sps'.
+
+glm dv by Agrp Bgrp
+ /method = sstype (2)
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv type2.sps], [0],
+ [dnl
+Table: Tests of Between-Subjects Effects
+,Type II Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,216.017,7,30.860,5.046,.001
+Agrp,2.553,1,2.553,.418,.523
+Bgrp,186.225,3,62.075,10.151,.000
+Agrp × Bgrp,20.212,3,6.737,1.102,.364
+Error,183.457,30,6.115,,
+Total,3810.000,38,,,
+Corrected Total,399.474,37,,,
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([GLM excluded intercept])
+AT_KEYWORDS([categorical categoricals])
+
+dnl The following example comes from
+dnl
+dnl Rudolf N. Cardinal
+dnl Graduate-level statistics for psychology and neuroscience
+dnl ANOVA in practice, and complex ANOVA designs
+dnl Version of 2 May 2004
+dnl
+dnl Downloaded from: http://egret.psychol.cam.ac.uk/psychology/graduate/Guide_to_ANOVA.pdf
+
+AT_DATA([intercept-exclude.sps], [dnl
+set format = F20.3.
+
+data list notable list /depvar * A *.
+begin data.
+10 1
+14 1
+8 1
+7 1
+2 1
+10 1
+1 1
+3 1
+2 1
+8.5 1
+14.29 2
+18.49 2
+12.46 2
+11.63 2
+6.66 2
+14.02 2
+5.66 2
+7.06 2
+6.37 2
+13.26 2
+end data.
+
+GLM depvar by A
+ /intercept = exclude
+ .
+
+
+GLM depvar by A
+ /intercept = include
+ .
+
+])
+
+AT_CHECK([pspp -O format=csv intercept-exclude.sps], [0],
+ [dnl
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Model,1636.826,2,818.413,43.556,.000
+A,1636.826,2,818.413,43.556,.000
+Error,338.216,18,18.790,,
+Total,1975.042,20,,,
+
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,98.568,1,98.568,5.246,.034
+Intercept,1538.258,1,1538.258,81.867,.000
+A,98.568,1,98.568,5.246,.034
+Error,338.216,18,18.790,,
+Total,1975.042,20,,,
+Corrected Total,436.784,19,,,
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([GLM missing values])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([glm.data], [dnl
+1 1 6 3.5
+1 2 2 8.9
+1 3 3 9.6
+1 4 4 10.5
+1 5 5 3.1
+1 6 1 5.9
+2 1 2 4.2
+2 2 6 1.9
+2 3 5 3.7
+2 4 3 10.2
+2 5 1 7.2
+2 6 4 7.6
+3 1 1 6.7
+3 2 4 5.8
+3 3 6 -2.7
+3 4 2 4.6
+3 5 3 4.0
+3 6 5 -0.7
+4 1 4 6.6
+4 2 1 4.5
+4 3 2 3.7
+4 4 5 3.7
+4 5 6 -3.3
+4 6 3 3.0
+5 1 3 4.1
+5 2 5 2.4
+5 3 4 6.0
+5 4 1 5.1
+5 5 2 3.5
+5 6 6 4.0
+6 1 5 3.8
+6 2 3 5.8
+6 3 1 7.0
+6 4 6 3.8
+6 5 4 5.0
+6 6 2 8.6
+])
+
+AT_DATA([glm-miss.sps], [dnl
+set format = F20.3.
+data list file='glm.data' notable fixed /a 1 b 3 c 5 y 7-10(2).
+
+do if a=6.
+recode y (else=SYSMIS).
+end if.
+
+glm y by b a c
+ /criteria=alpha(.05)
+ /design = a b c
+ .
+])
+
+AT_CHECK([pspp -O format=csv glm-miss.sps], [0], [dnl
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,251.621,14,17.973,4.969,.002
+Intercept,628.376,1,628.376,173.737,.000
+a,72.929,4,18.232,5.041,.009
+b,20.703,5,4.141,1.145,.380
+c,135.179,5,27.036,7.475,.001
+Error,54.253,15,3.617,,
+Total,934.250,30,,,
+Corrected Total,305.874,29,,,
+])
+
+
+
+AT_DATA([glm-miss2.sps], [dnl
+set format = F20.3.
+data list file='glm.data' notable fixed /a 1 b 3 c 5 y 7-10(2).
+
+select if a <> 6.
+
+glm y by b a c
+ /criteria=alpha(.05)
+ /design = a b c
+ .
+])
+
+AT_CHECK([pspp -O format=csv glm-miss2.sps], [0], [dnl
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,251.621,14,17.973,4.969,.002
+Intercept,628.376,1,628.376,173.737,.000
+a,72.929,4,18.232,5.041,.009
+b,20.703,5,4.141,1.145,.380
+c,135.179,5,27.036,7.475,.001
+Error,54.253,15,3.617,,
+Total,934.250,30,,,
+Corrected Total,305.874,29,,,
+])
+
+
+dnl Now for some missing values in the factor variables.
+
+AT_DATA([glm-miss3.sps], [dnl
+set format = F20.3.
+data list file=glm.data notable fixed /a 1 b 3 c 5 y 7-10(2).
+
+do if a=6.
+recode a (else=SYSMIS).
+end if.
+
+glm y by b a c
+ /criteria=alpha(.05)
+ /design = a b c
+ .
+])
+
+AT_CHECK([pspp -O format=csv glm-miss3.sps], [0], [dnl
+Table: Tests of Between-Subjects Effects
+,Type III Sum Of Squares,df,Mean Square,F,Sig.
+Corrected Model,251.621,14,17.973,4.969,.002
+Intercept,628.376,1,628.376,173.737,.000
+a,72.929,4,18.232,5.041,.009
+b,20.703,5,4.141,1.145,.380
+c,135.179,5,27.036,7.475,.001
+Error,54.253,15,3.617,,
+Total,934.250,30,,,
+Corrected Total,305.874,29,,,
+])
+
+AT_CLEANUP
+
+AT_SETUP([GLM syntax errors])
+AT_DATA([glm.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+GLM **.
+GLM x **.
+GLM x BY **.
+GLM x BY y.
+GLM x y BY z.
+GLM x BY y/MISSING=**.
+GLM x BY y/INTERCEPT=**.
+GLM x BY y/CRITERIA=**.
+GLM x BY y/CRITERIA=ALPHA **.
+GLM x BY y/CRITERIA=ALPHA(**).
+GLM x BY y/CRITERIA=ALPHA(123 **).
+GLM x BY y/METHOD=**.
+GLM x BY y/METHOD=SSTYPE **.
+GLM x BY y/METHOD=SSTYPE(4).
+GLM x BY y/METHOD=SSTYPE(2 **).
+GLM x BY y/DESIGN=**.
+GLM x BY y/DESIGN=x(y).
+GLM x BY y/DESIGN=x WITHIN y.
+])
+AT_CHECK([pspp -O format=csv glm.sps], [1], [dnl
+"glm.sps:2.5-2.6: error: GLM: Syntax error expecting variable name.
+ 2 | GLM **.
+ | ^~"
+
+"glm.sps:3.7-3.8: error: GLM: Syntax error expecting `BY'.
+ 3 | GLM x **.
+ | ^~"
+
+"glm.sps:4.10-4.11: error: GLM: Syntax error expecting variable name.
+ 4 | GLM x BY **.
+ | ^~"
+
+"glm.sps:6.1-6.3: error: GLM: Syntax error expecting `BEGIN DATA'.
+ 6 | GLM x y BY z.
+ | ^~~"
+
+"glm.sps:6.1-6.3: error: GLM: Syntax error expecting end of command.
+ 6 | GLM x y BY z.
+ | ^~~"
+
+"glm.sps:7.20-7.21: error: GLM: Syntax error expecting INCLUDE or EXCLUDE.
+ 7 | GLM x BY y/MISSING=**.
+ | ^~"
+
+"glm.sps:8.22-8.23: error: GLM: Syntax error expecting INCLUDE or EXCLUDE.
+ 8 | GLM x BY y/INTERCEPT=**.
+ | ^~"
+
+"glm.sps:9.21-9.22: error: GLM: Syntax error expecting `ALPHA@{:@'.
+ 9 | GLM x BY y/CRITERIA=**.
+ | ^~"
+
+"glm.sps:10.21-10.28: error: GLM: Syntax error expecting `ALPHA@{:@'.
+ 10 | GLM x BY y/CRITERIA=ALPHA **.
+ | ^~~~~~~~"
+
+"glm.sps:11.27-11.28: error: GLM: Syntax error expecting number.
+ 11 | GLM x BY y/CRITERIA=ALPHA(**).
+ | ^~"
+
+"glm.sps:12.31-12.32: error: GLM: Syntax error expecting `@:}@'.
+ 12 | GLM x BY y/CRITERIA=ALPHA(123 **).
+ | ^~"
+
+"glm.sps:13.19-13.20: error: GLM: Syntax error expecting `SSTYPE@{:@'.
+ 13 | GLM x BY y/METHOD=**.
+ | ^~"
+
+"glm.sps:14.19-14.27: error: GLM: Syntax error expecting `SSTYPE@{:@'.
+ 14 | GLM x BY y/METHOD=SSTYPE **.
+ | ^~~~~~~~~"
+
+"glm.sps:15.26: error: GLM: Syntax error expecting integer between 1 and 3 for SSTYPE.
+ 15 | GLM x BY y/METHOD=SSTYPE(4).
+ | ^"
+
+"glm.sps:16.28-16.29: error: GLM: Syntax error expecting `@:}@'.
+ 16 | GLM x BY y/METHOD=SSTYPE(2 **).
+ | ^~"
+
+"glm.sps:17.19-17.20: error: GLM: Syntax error expecting variable name.
+ 17 | GLM x BY y/DESIGN=**.
+ | ^~"
+
+"glm.sps:18.20: error: GLM: Nested variables are not yet implemented.
+ 18 | GLM x BY y/DESIGN=x(y).
+ | ^"
+
+"glm.sps:19.21-19.26: error: GLM: Nested variables are not yet implemented.
+ 19 | GLM x BY y/DESIGN=x WITHIN y.
+ | ^~~~~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([GRAPH])
+
+AT_SETUP([GRAPH simple scatterplot])
+AT_DATA([scatterplot.sps],[
+* Simple Scatterplot test
+NEW FILE.
+INPUT PROGRAM.
+LOOP #i = 1 to 100.
+COMPUTE Age = RV.NORMAL(40,10).
+END CASE.
+END LOOP.
+END FILE.
+END INPUT PROGRAM.
+
+COMPUTE Size = Age * 3 + 50.
+
+GRAPH
+ /SCATTERPLOT(BIVARIATE) = Age WITH Size.
+
+])
+
+AT_CHECK([pspp -O format=csv scatterplot.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([GRAPH Scatter and Histogram])
+AT_KEYWORDS([slow])
+AT_DATA([scatterlong.sps],[
+NEW FILE.
+INPUT PROGRAM.
+LOOP #i = 1 to 10000.
+COMPUTE Age = RV.NORMAL(40,10).
+COMPUTE CityNum = TRUNC(UNIFORM(2.95)).
+END CASE.
+END LOOP.
+END FILE.
+END INPUT PROGRAM.
+
+COMPUTE Size = Age * 3 + 50 + 50*CityNum.
+
+STRING City (a20).
+
+Recode CityNum
+ (0 = "Madrid")
+ (1 = "Paris")
+ (ELSE = "Stockholm")
+ into City.
+
+ GRAPH
+ /SCATTERPLOT(BIVARIATE) = Age WITH Size
+
+ GRAPH
+ /SCATTERPLOT(BIVARIATE) = Age WITH CityNum
+
+ GRAPH
+ /SCATTERPLOT = CityNum WITH Age
+
+ GRAPH
+ /SCATTERPLOT = CityNum WITH Size
+
+ GRAPH
+ /SCATTERPLOT(BIVARIATE) = Age WITH Size BY City
+
+ GRAPH
+ /SCATTERPLOT(BIVARIATE) = Age WITH Size BY CityNum
+
+ ADD VALUE LABELS
+ /CityNum 1 'Rio' 2 'Tokyo' 0 'Mumbai'.
+
+ GRAPH
+ /SCATTERPLOT(BIVARIATE) = Age WITH Size BY CityNum
+
+ GRAPH
+ /HISTOGRAM = Age.
+
+])
+
+AT_CHECK([pspp -O format=pdf scatterlong.sps], [0], [ignore], [ignore])
+AT_CLEANUP
+
+AT_SETUP([GRAPH missing values don't crash])
+AT_DATA([scatter.sps], [dnl
+data list list /x * y *.
+begin data.
+1 0
+2 0
+. 0
+3 1
+4 1
+5 .
+6 1
+end data.
+graph
+ /scatterplot = x with y.
+graph
+ /histogram = x.
+])
+AT_CHECK([pspp -o pspp.pdf scatter.sps], [], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+AT_SETUP([GRAPH missing=VARIABLE no crash])
+AT_DATA([scatter.sps], [dnl
+data list list /x * y *.
+begin data.
+1 0
+2 0
+. 0
+3 1
+4 1
+5 .
+6 1
+end data.
+graph
+ /scatterplot = x with y
+ /missing = VARIABLE.
+graph
+ /histogram = x
+ /missing = VARIABLE.
+])
+AT_CHECK([pspp -o pspp.pdf scatter.sps], [], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+AT_SETUP([GRAPH missing value in by variable])
+AT_DATA([scatter.sps], [dnl
+data list list /x * y * z *.
+begin data.
+1 0 9
+2 0 9
+. 0 9
+3 1 .
+4 1 8
+5 . 8
+6 1 8
+end data.
+graph
+ /scatterplot = x with y by z
+ /missing = VARIABLE.
+
+graph
+ /scatterplot = x with y by z.
+
+])
+AT_CHECK([pspp -o pspp.pdf scatter.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+
+AT_SETUP([GRAPH histogram with null data])
+AT_DATA([null-hist.sps], [dnl
+data list list /x *.
+begin data.
+1109
+.
+end data.
+
+graph
+ /histogram = x.
+
+])
+
+AT_CHECK([pspp -o pspp.pdf null-hist.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+
+AT_SETUP([GRAPH histogram all missing])
+AT_DATA([null-hist.sps], [dnl
+data list list /x *.
+begin data.
+.
+end data.
+
+graph
+ /histogram = x.
+
+])
+
+AT_CHECK([pspp null-hist.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+AT_CLEANUP
+
+
+
+
+AT_SETUP([GRAPH barcharts])
+AT_CHECK([ln -s $top_srcdir/examples/physiology.sav .], [0])
+AT_CHECK([ln -s $top_srcdir/examples/repairs.sav .], [0])
+
+AT_DATA([barchart.sps], [dnl
+GET FILE="physiology.sav".
+
+GRAPH /BAR = COUNT BY SEX.
+
+GRAPH /BAR = MEAN(height) BY SEX.
+
+NEW FILE.
+
+GET FILE="repairs.sav".
+
+GRAPH /BAR = MEAN (mtbf) BY factory.
+
+COMPUTE R = TRUNC(RV.UNIFORM(1,5)).
+
+GRAPH /BAR = MEAN (mtbf) BY factory BY R.
+])
+
+AT_CHECK([pspp -o pspp.pdf barchart.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+
+AT_CLEANUP
+
+
+
+AT_SETUP([GRAPH barchart arity])
+
+AT_DATA([barchart.sps], [dnl
+data list notable list /x y z*.
+begin data
+1 1 3
+2 1 4
+3 1 3
+4 1 4
+5 . 3
+6 2 4
+7 2 3
+8 2 4
+9 2 3
+10 2 4
+end data.
+
+* This line is invalid
+GRAPH /BAR = COUNT(x) BY y.
+])
+
+AT_CHECK([pspp -o pspp.pdf barchart.sps], [1], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+
+AT_CLEANUP
+
+
+
+
+AT_SETUP([GRAPH barchart bad syntax])
+
+AT_DATA([barchart.sps], [dnl
+data list notable list /x y z*.
+begin data
+1 1 3
+2 1 4
+3 1 3
+4 1 4
+5 . 3
+6 2 4
+7 2 3
+8 2 4
+9 2 3
+10 2 4
+end data.
+
+* This line is invalid
+GRAPH /BAR = SCROD BY y.
+])
+
+AT_CHECK([pspp -o pspp.pdf barchart.sps], [1], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+
+AT_CLEANUP
+
+
+
+AT_SETUP([GRAPH barchart full])
+
+AT_DATA([barchart.sps], [dnl
+data list notable list /x y z*.
+begin data
+1 1 3
+2 1 4
+3 1 3
+4 1 4
+5 . 3
+6 2 4
+7 2 3
+8 2 4
+9 2 3
+10 2 4
+end data.
+
+* This line is invalid
+GRAPH /BAR = COUNT by z.
+GRAPH /BAR = CUFREQ by z.
+GRAPH /BAR = PCT by z.
+GRAPH /BAR = CUPCT by z.
+
+GRAPH /BAR = MEAN(y) BY z.
+GRAPH /BAR = SUM(y) BY z.
+GRAPH /BAR = MAXIMUM(y) BY z.
+GRAPH /BAR = MINIMUM(y) BY z.
+
+GRAPH /BAR = MEAN(y) BY z BY y.
+GRAPH /BAR = SUM(y) BY z BY y.
+GRAPH /BAR = MAXIMUM(y) BY z BY y.
+GRAPH /BAR = MINIMUM(y) BY z BY y.
+])
+
+AT_CHECK([pspp -o pspp.pdf barchart.sps], [0], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+
+AT_CLEANUP
+
+
+
+
+
+AT_SETUP([GRAPH buggy syntax])
+
+AT_DATA([barchart.sps], [dnl
+data list notable list /x y z*.
+begin data
+1 1 3
+2 1 4
+10 2 4
+end data.
+
+GRAPH /BAR = MINIMUM({) BY z BY y.
+])
+
+AT_CHECK([pspp -o pspp.pdf barchart.sps], [1], [ignore])
+dnl Ignore output -- this is just a no-crash check.
+
+AT_CLEANUP
+
+
+dnl Check that percentages are calculated with respect to the
+dnl proper total. See bug #56247
+AT_SETUP([GRAPH barchart percentage sub-categorical])
+AT_DATA([barchart.sps], [dnl
+data list list notable /penalty_favopp_x * XYdem_racethW8 * w *.
+begin data.
+1 0 1960
+1 1 376
+2 0 678
+2 1 147
+4 0 368
+4 1 164
+5 0 427
+5 1 274
+. . 1522
+end data.
+
+weight by w.
+
+* crosstabs
+* /tables=penalty_favopp_x by XYdem_racethW8
+* /format=AVALUE TABLES PIVOT
+* /statistics=CHISQ
+* /cells COUNT COLUMN TOTAL.
+
+graph
+ /bar=pct by penalty_favopp_x
+ .
+
+graph
+ /bar=pct by penalty_favopp_x by XYdem_racethW8
+ .
+])
+
+AT_CHECK([pspp --testing barchart.sps], [0], [dnl
+Graphic: Barchart
+Percentage: 0
+Total Categories: 4
+Primary Categories: 4
+Largest Category: 53.1634
+Total Count: 100
+Y Label: "Percentage"
+Categorical Variables:
+ Var: "penalty_favopp_x"
+Categories:
+ 0 " 1.00"
+ 1 " 2.00"
+ 2 " 4.00"
+ 3 " 5.00"
+All Categories:
+Count: 53.1634; Cat: " 1.00"
+Count: 18.7756; Cat: " 2.00"
+Count: 12.1074; Cat: " 4.00"
+Count: 15.9536; Cat: " 5.00"
+
+Graphic: Barchart
+Percentage: 0
+Total Categories: 8
+Primary Categories: 4
+Largest Category: 57.0929
+Total Count: 200
+Y Label: "Percentage"
+Categorical Variables:
+ Var: "penalty_favopp_x"
+ Var: "XYdem_racethW8"
+Categories:
+ 0 " 1.00"
+ 1 " 2.00"
+ 2 " 4.00"
+ 3 " 5.00"
+Sub-categories:
+ 0 " .00"
+ 1 " 1.00"
+All Categories:
+Count: 57.0929; Cat: " 1.00", " .00"
+Count: 39.1259; Cat: " 1.00", " 1.00"
+Count: 19.7495; Cat: " 2.00", " .00"
+Count: 15.2966; Cat: " 2.00", " 1.00"
+Count: 10.7195; Cat: " 4.00", " .00"
+Count: 17.0656; Cat: " 4.00", " 1.00"
+Count: 12.4381; Cat: " 5.00", " .00"
+Count: 28.512; Cat: " 5.00", " 1.00"
+
+])
+
+AT_CLEANUP
+
+AT_SETUP([GRAPH syntax errors])
+AT_DATA([graph.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+GRAPH/HISTOGRAM=x/HISTOGRAM=y.
+GRAPH/HISTOGRAM(**).
+GRAPH/HISTOGRAM(NORMAL **).
+GRAPH/HISTOGRAM=**.
+GRAPH/HISTOGRAM=x y z.
+GRAPH/HISTOGRAM=x/BAR=y.
+GRAPH/BAR(GROUPED).
+GRAPH/BAR(STACKED).
+GRAPH/BAR(RANGE).
+GRAPH/BAR(**).
+GRAPH/BAR **.
+GRAPH/BAR=**.
+GRAPH/BAR=MEAN **.
+GRAPH/BAR=MEAN(**).
+GRAPH/BAR=MEAN(x**).
+GRAPH/BAR=MEAN(x) **.
+GRAPH/BAR=MEAN(x) BY **.
+GRAPH/BAR=MEAN(x) BY y BY **.
+GRAPH/HISTOGRAM=x/SCATTERPLOT=y.
+GRAPH/SCATTERPLOT(OVERLAY).
+GRAPH/SCATTERPLOT(MATRIX).
+GRAPH/SCATTERPLOT(XYZ).
+GRAPH/SCATTERPLOT(**).
+GRAPH/SCATTERPLOT(BIVARIATE **).
+GRAPH/SCATTERPLOT **.
+GRAPH/SCATTERPLOT=**.
+GRAPH/SCATTERPLOT=x y z.
+GRAPH/SCATTERPLOT=x **.
+GRAPH/SCATTERPLOT=x WITH **.
+GRAPH/SCATTERPLOT=x WITH y z.
+GRAPH/SCATTERPLOT=x WITH y BY **.
+GRAPH/LINE.
+GRAPH/PIE.
+GRAPH/ERRORBAR.
+GRAPH/PARETO.
+GRAPH/TITLE.
+GRAPH/SUBTITLE.
+GRAPH/FOOTNOTE.
+GRAPH/MISSING=**.
+GRAPH/ **.
+])
+AT_CHECK([pspp -O format=csv graph.sps], [1], [dnl
+"graph.sps:2.19-2.27: error: GRAPH: Only one chart type is allowed.
+ 2 | GRAPH/HISTOGRAM=x/HISTOGRAM=y.
+ | ^~~~~~~~~"
+
+"graph.sps:3.17-3.18: error: GRAPH: Syntax error expecting `NORMAL)'.
+ 3 | GRAPH/HISTOGRAM(**).
+ | ^~"
+
+"graph.sps:4.17-4.25: error: GRAPH: Syntax error expecting `NORMAL)'.
+ 4 | GRAPH/HISTOGRAM(NORMAL **).
+ | ^~~~~~~~~"
+
+"graph.sps:5.17-5.18: error: GRAPH: Syntax error expecting variable name.
+ 5 | GRAPH/HISTOGRAM=**.
+ | ^~"
+
+"graph.sps:6.17-6.21: error: GRAPH: Only one variable is allowed.
+ 6 | GRAPH/HISTOGRAM=x y z.
+ | ^~~~~"
+
+"graph.sps:7.19-7.21: error: GRAPH: Only one chart type is allowed.
+ 7 | GRAPH/HISTOGRAM=x/BAR=y.
+ | ^~~"
+
+"graph.sps:8.11-8.17: error: GRAPH: GROUPED is not yet implemented.
+ 8 | GRAPH/BAR(GROUPED).
+ | ^~~~~~~"
+
+"graph.sps:9.11-9.17: error: GRAPH: STACKED is not yet implemented.
+ 9 | GRAPH/BAR(STACKED).
+ | ^~~~~~~"
+
+"graph.sps:10.11-10.15: error: GRAPH: RANGE is not yet implemented.
+ 10 | GRAPH/BAR(RANGE).
+ | ^~~~~"
+
+"graph.sps:11.11-11.12: error: GRAPH: Syntax error expecting SIMPLE, GROUPED, STACKED, or RANGE.
+ 11 | GRAPH/BAR(**).
+ | ^~"
+
+"graph.sps:12.11-12.12: error: GRAPH: Syntax error expecting `='.
+ 12 | GRAPH/BAR **.
+ | ^~"
+
+"graph.sps:13.11-13.12: error: GRAPH: Syntax error expecting COUNT, PCT, CUFREQ, CUPCT, MEAN, SUM, MAXIMUM, or MINIMUM.
+ 13 | GRAPH/BAR=**.
+ | ^~"
+
+"graph.sps:14.16-14.17: error: GRAPH: Syntax error expecting `('.
+ 14 | GRAPH/BAR=MEAN **.
+ | ^~"
+
+"graph.sps:15.16-15.17: error: GRAPH: Syntax error expecting variable name.
+ 15 | GRAPH/BAR=MEAN(**).
+ | ^~"
+
+"graph.sps:16.17-16.18: error: GRAPH: Syntax error expecting `)'.
+ 16 | GRAPH/BAR=MEAN(x**).
+ | ^~"
+
+"graph.sps:17.19-17.20: error: GRAPH: Syntax error expecting `BY'.
+ 17 | GRAPH/BAR=MEAN(x) **.
+ | ^~"
+
+"graph.sps:18.22-18.23: error: GRAPH: Syntax error expecting variable name.
+ 18 | GRAPH/BAR=MEAN(x) BY **.
+ | ^~"
+
+"graph.sps:19.27-19.28: error: GRAPH: Syntax error expecting variable name.
+ 19 | GRAPH/BAR=MEAN(x) BY y BY **.
+ | ^~"
+
+"graph.sps:20.19-20.29: error: GRAPH: Only one chart type is allowed.
+ 20 | GRAPH/HISTOGRAM=x/SCATTERPLOT=y.
+ | ^~~~~~~~~~~"
+
+"graph.sps:21.19-21.25: error: GRAPH: OVERLAY is not yet implemented.
+ 21 | GRAPH/SCATTERPLOT(OVERLAY).
+ | ^~~~~~~"
+
+"graph.sps:22.19-22.24: error: GRAPH: MATRIX is not yet implemented.
+ 22 | GRAPH/SCATTERPLOT(MATRIX).
+ | ^~~~~~"
+
+"graph.sps:23.19-23.21: error: GRAPH: XYZ is not yet implemented.
+ 23 | GRAPH/SCATTERPLOT(XYZ).
+ | ^~~"
+
+"graph.sps:24.19-24.20: error: GRAPH: Syntax error expecting BIVARIATE, OVERLAY, MATRIX, or XYZ.
+ 24 | GRAPH/SCATTERPLOT(**).
+ | ^~"
+
+"graph.sps:25.29-25.30: error: GRAPH: Syntax error expecting `)'.
+ 25 | GRAPH/SCATTERPLOT(BIVARIATE **).
+ | ^~"
+
+"graph.sps:26.19-26.20: error: GRAPH: Syntax error expecting `='.
+ 26 | GRAPH/SCATTERPLOT **.
+ | ^~"
+
+"graph.sps:27.19-27.20: error: GRAPH: Syntax error expecting variable name.
+ 27 | GRAPH/SCATTERPLOT=**.
+ | ^~"
+
+"graph.sps:28.19-28.23: error: GRAPH: Only one variable is allowed.
+ 28 | GRAPH/SCATTERPLOT=x y z.
+ | ^~~~~"
+
+"graph.sps:29.21-29.22: error: GRAPH: Syntax error expecting `WITH'.
+ 29 | GRAPH/SCATTERPLOT=x **.
+ | ^~"
+
+"graph.sps:30.26-30.27: error: GRAPH: Syntax error expecting variable name.
+ 30 | GRAPH/SCATTERPLOT=x WITH **.
+ | ^~"
+
+"graph.sps:31.26-31.28: error: GRAPH: Only one variable is allowed.
+ 31 | GRAPH/SCATTERPLOT=x WITH y z.
+ | ^~~"
+
+"graph.sps:32.31-32.32: error: GRAPH: Syntax error expecting variable name.
+ 32 | GRAPH/SCATTERPLOT=x WITH y BY **.
+ | ^~"
+
+"graph.sps:33.7-33.10: error: GRAPH: LINE is not yet implemented.
+ 33 | GRAPH/LINE.
+ | ^~~~"
+
+"graph.sps:34.7-34.9: error: GRAPH: PIE is not yet implemented.
+ 34 | GRAPH/PIE.
+ | ^~~"
+
+"graph.sps:35.7-35.14: error: GRAPH: ERRORBAR is not yet implemented.
+ 35 | GRAPH/ERRORBAR.
+ | ^~~~~~~~"
+
+"graph.sps:36.7-36.12: error: GRAPH: PARETO is not yet implemented.
+ 36 | GRAPH/PARETO.
+ | ^~~~~~"
+
+"graph.sps:37.7-37.11: error: GRAPH: TITLE is not yet implemented.
+ 37 | GRAPH/TITLE.
+ | ^~~~~"
+
+"graph.sps:38.7-38.14: error: GRAPH: SUBTITLE is not yet implemented.
+ 38 | GRAPH/SUBTITLE.
+ | ^~~~~~~~"
+
+"graph.sps:39.7-39.14: error: GRAPH: FOOTNOTE is not yet implemented.
+ 39 | GRAPH/FOOTNOTE.
+ | ^~~~~~~~"
+
+"graph.sps:40.15-40.16: error: GRAPH: Syntax error expecting LISTWISE, VARIABLE, EXCLUDE, INCLUDE, REPORT, or NOREPORT.
+ 40 | GRAPH/MISSING=**.
+ | ^~"
+
+"graph.sps:41.8-41.9: error: GRAPH: Syntax error expecting one of the following: HISTOGRAM, BAR, SCATTERPLOT, LINE, PIE, ERRORBAR, PARETO, TITLE, SUBTITLE, FOOTNOTE, MISSING.
+ 41 | GRAPH/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([HOST - portable tests])
+
+AT_SETUP([HOST - one command])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['echo "hi there" > file']].
+])
+AT_CHECK([pspp -O format=csv host.sps])
+AT_CHECK([cat file], [0], [hi there
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - two commands])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['echo a > a' 'echo b > b']].
+])
+AT_CHECK([pspp -O format=csv host.sps])
+AT_CHECK([cat a], [0], [a
+])
+AT_CHECK([cat b], [0], [b
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - time limit])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['sleep 10']] TIMELIMIT=0.1.
+])
+if $MINGW; then
+ AT_CHECK([pspp -O format=csv host.sps], [1], [dnl
+host.sps:1: error: HOST: Time limit not supported on this platform.
+])
+else
+ AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
+"host.sps:1: warning: HOST: Command ""sleep 10"" timed out."
+])
+fi
+AT_CLEANUP
+
+AT_BANNER([HOST - Unix-like OSes only])
+
+AT_SETUP([HOST - command failure])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['exit 1' 'echo "Not reached"']].
+])
+AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
+"host.sps:1: warning: HOST: Command ""exit 1"" exited with status 1."
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - nonexistent shell])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['echo hi']].
+])
+AT_CHECK([SHELL=/nonexistent pspp -O format=csv host.sps], [0], [dnl
+"host.sps:1: warning: HOST: Command ""echo hi"" exited with status 127 (Command or shell not found)."
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - nonexistent command])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['/nonexistent']].
+])
+AT_CHECK([pspp -O format=csv host.sps | head -1], [0], [dnl
+"host.sps:1: warning: HOST: Command ""/nonexistent"" exited with status 127 (Command or shell not found)."
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - output to stdout])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['echo hi']].
+])
+AT_CHECK([pspp -O format=csv host.sps], [0], [hi
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - output to stderr])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['echo hi 2>&1']].
+])
+AT_CHECK([pspp -O format=csv host.sps], [0], [hi
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - input from stdin])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['cat && echo ok || echo fail']] TIMELIMIT=5.
+])
+AT_CHECK([pspp -O format=csv host.sps], [0], [ok
+])
+AT_CLEANUP
+
+dnl This is a special case inside run_command().
+AT_SETUP([HOST - zero time limit])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['sleep 10']] TIMELIMIT=0.
+])
+AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
+"host.sps:1: warning: HOST: Command ""sleep 10"" timed out."
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - signal termination])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+HOST COMMAND=[['kill -ABRT $$' 'echo "Not reached"']].
+])
+AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
+"host.sps:1: warning: HOST: Command ""kill -ABRT $$"" terminated by signal 6."
+])
+AT_CLEANUP
+
+AT_SETUP([HOST - SAFER])
+AT_SKIP_IF([$MINGW])
+AT_DATA([host.sps], [dnl
+SET SAFER=ON.
+HOST COMMAND=[['sleep 10']] TIMELIMIT=0.1.
+])
+AT_CHECK([pspp -O format=csv host.sps], [1], [dnl
+"host.sps:2.1-2.4: error: HOST: This command not allowed when the SAFER option is set.
+ 2 | HOST COMMAND=[['sleep 10']] TIMELIMIT=0.1.
+ | ^~~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([INPUT PROGRAM])
+
+dnl Tests for a bug which caused a crash when
+dnl reading invalid INPUT PROGRAM syntax.
+AT_SETUP([INPUT PROGRAM invalid syntax crash])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+DATA LIST NOTABLE /a 1-9.
+BEGIN DATA
+123456789
+END DATA.
+END INPUT PROGRAM.
+])
+AT_CHECK([pspp -O format=csv input-program.sps], [1], [dnl
+"input-program.sps:3.1-3.10: error: BEGIN DATA: BEGIN DATA is not allowed inside INPUT PROGRAM.
+ 3 | BEGIN DATA
+ | ^~~~~~~~~~"
+])
+AT_CLEANUP
+
+dnl Tests for bug #21108, a crash when
+dnl reading invalid INPUT PROGRAM syntax.
+AT_SETUP([INPUT PROGRAM invalid syntax crash])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+DATA LIST LIST NOTABLE /x.
+END FILE.
+END INPUT PROGRAM.
+
+DESCRIPTIVES x.
+])
+AT_CHECK([pspp -O format=csv input-program.sps], [1], [dnl
+error: DESCRIPTIVES: At end of input: Syntax error expecting `BEGIN DATA'.
+])
+AT_CLEANUP
+
+dnl Tests for bug #38782, an infinite loop processing an empty input program.
+AT_SETUP([INPUT PROGRAM infinite loop])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+STRING firstname lastname (a24) / address (a80).
+END INPUT PROGRAM.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv input-program.sps], [1], [dnl
+"input-program.sps:1.1-3.17: error: INPUT PROGRAM: Input program does not contain DATA LIST or END FILE.
+ 1 | INPUT PROGRAM.
+ 2 | STRING firstname lastname (a24) / address (a80).
+ 3 | END INPUT PROGRAM."
+
+"input-program.sps:4.1-4.7: error: EXECUTE: EXECUTE is allowed only after the active dataset has been defined.
+ 4 | EXECUTE.
+ | ^~~~~~~"
+])
+AT_CLEANUP
+
+dnl Tests for bug #39097, a bug when an INPUT PROGRAM used VECTOR, was
+dnl followed immediately by a call to proc_execute() (here via DATASET
+dnl COPY), and then the input was actually used.
+AT_SETUP([INPUT PROGRAM with VECTOR and EXECUTE])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+VECTOR vec(5).
+LOOP #c = 1 to 10.
+ LOOP #v = 1 to 5.
+ COMPUTE vec(#v) = #v.
+ END LOOP.
+ END CASE.
+END LOOP.
+END FILE.
+END INPUT PROGRAM.
+DATASET COPY x.
+LIST.
+])
+AT_CHECK([pspp -O format=csv input-program.sps], [0], [dnl
+Table: Data List
+vec1,vec2,vec3,vec4,vec5
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+1.00,2.00,3.00,4.00,5.00
+])
+AT_CLEANUP
+
+AT_SETUP([INPUT PROGRAM taking shorter of two files])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+ DATA LIST NOTABLE FILE='a.txt'/X 1-10.
+ DATA LIST NOTABLE FILE='b.txt'/Y 1-10.
+END INPUT PROGRAM.
+LIST.
+])
+AT_DATA([short.txt], [dnl
+1
+2
+3
+])
+AT_DATA([long.txt], [dnl
+4
+5
+6
+7
+])
+
+cp short.txt a.txt
+cp long.txt b.txt
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X,Y
+1,4
+2,5
+3,6
+])
+
+cp short.txt b.txt
+cp long.txt a.txt
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X,Y
+4,1
+5,2
+6,3
+])
+AT_CLEANUP
+
+AT_SETUP([INPUT PROGRAM taking longer of two files])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+ NUMERIC #A #B.
+
+ DO IF NOT #A.
+ DATA LIST NOTABLE END=#A FILE='a.txt'/X 1-10.
+ END IF.
+ DO IF NOT #B.
+ DATA LIST NOTABLE END=#B FILE='b.txt'/Y 1-10.
+ END IF.
+ DO IF #A AND #B.
+ END FILE.
+ END IF.
+ END CASE.
+END INPUT PROGRAM.
+LIST.
+])
+AT_DATA([short.txt], [dnl
+1
+2
+3
+])
+AT_DATA([long.txt], [dnl
+4
+5
+6
+7
+8
+])
+
+cp short.txt a.txt
+cp long.txt b.txt
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X,Y
+1,4
+2,5
+3,6
+.,7
+.,8
+])
+
+cp short.txt b.txt
+cp long.txt a.txt
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X,Y
+4,1
+5,2
+6,3
+7,.
+8,.
+])
+AT_CLEANUP
+
+AT_SETUP([INPUT PROGRAM concatenating two files - version 1])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+ NUMERIC #A #B.
+
+ DO IF #A.
+ DATA LIST NOTABLE END=#B FILE='b.txt'/X 1-10.
+ DO IF #B.
+ END FILE.
+ ELSE.
+ END CASE.
+ END IF.
+ ELSE.
+ DATA LIST NOTABLE END=#A FILE='a.txt'/X 1-10.
+ DO IF NOT #A.
+ END CASE.
+ END IF.
+ END IF.
+END INPUT PROGRAM.
+LIST.
+])
+AT_DATA([a.txt], [dnl
+1
+2
+3
+])
+AT_DATA([b.txt], [dnl
+4
+5
+6
+7
+8
+])
+
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X
+1
+2
+3
+4
+5
+6
+7
+8
+])
+AT_CLEANUP
+
+AT_SETUP([INPUT PROGRAM concatenating two files - version 2])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+ NUMERIC #EOF.
+
+ LOOP IF NOT #EOF.
+ DATA LIST NOTABLE END=#EOF FILE='a.txt'/X 1-10.
+ DO IF NOT #EOF.
+ END CASE.
+ END IF.
+ END LOOP.
+
+ COMPUTE #EOF = 0.
+ LOOP IF NOT #EOF.
+ DATA LIST NOTABLE END=#EOF FILE='b.txt'/X 1-10.
+ DO IF NOT #EOF.
+ END CASE.
+ END IF.
+ END LOOP.
+
+ END FILE.
+END INPUT PROGRAM.
+LIST.
+])
+AT_DATA([a.txt], [dnl
+1
+2
+3
+])
+AT_DATA([b.txt], [dnl
+4
+5
+6
+7
+8
+])
+
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X
+1
+2
+3
+4
+5
+6
+7
+8
+])
+AT_CLEANUP
+
+AT_SETUP([INPUT PROGRAM generating data])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+ LOOP #I=1 TO 10.
+ COMPUTE X=#I.
+ END CASE.
+ END LOOP.
+ END FILE.
+END INPUT PROGRAM.
+FORMAT X(F2).
+LIST.
+])
+AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
+Table: Data List
+X
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+])
+AT_CLEANUP
+
+AT_SETUP([INPUT PROGRAM unexpected end of file])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+])
+AT_CHECK([pspp input-program.sps], 1, [dnl
+error: INPUT PROGRAM: Unexpected end-of-file within INPUT PROGRAM.
+])
+AT_CLEANUP
+
+
+AT_SETUP([INPUT PROGRAM no variables])
+AT_DATA([input-program.sps], [dnl
+INPUT PROGRAM.
+END FILE.
+END INPUT PROGRAM.
+])
+AT_CHECK([pspp input-program.sps], 1, [dnl
+input-program.sps:1.1-3.17: error: INPUT PROGRAM: Input program did not create
+any variables.
+ 1 | INPUT PROGRAM.
+ 2 | END FILE.
+ 3 | END INPUT PROGRAM.
+])
+AT_CLEANUP
+
+AT_SETUP([REREAD syntax errors])
+AT_DATA([reread.sps], [dnl
+INPUT PROGRAM.
+REREAD COLUMN=1 COLUMN=**.
+END INPUT PROGRAM.
+
+INPUT PROGRAM.
+REREAD COLUMN=**.
+END INPUT PROGRAM.
+
+INPUT PROGRAM.
+REREAD FILE=**.
+END INPUT PROGRAM.
+
+INPUT PROGRAM.
+REREAD ENCODING=**.
+END INPUT PROGRAM.
+
+INPUT PROGRAM.
+REREAD **.
+END INPUT PROGRAM.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='reread.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"reread.sps:2.17-2.22: error: REREAD: Subcommand COLUMN may only be specified once.
+ 2 | REREAD COLUMN=1 COLUMN=**.
+ | ^~~~~~"
+
+"reread.sps:6.15-6.16: error: REREAD: Syntax error parsing expression.
+ 6 | REREAD COLUMN=**.
+ | ^~"
+
+"reread.sps:10.13-10.14: error: REREAD: Syntax error expecting a file name or handle name.
+ 10 | REREAD FILE=**.
+ | ^~"
+
+"reread.sps:14.17-14.18: error: REREAD: Syntax error expecting string.
+ 14 | REREAD ENCODING=**.
+ | ^~"
+
+"reread.sps:18.8-18.9: error: REREAD: Syntax error expecting COLUMN, FILE, or ENCODING.
+ 18 | REREAD **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([INSERT])
+
+dnl Create a file "batch.sps" that is valid syntax only in batch mode.
+m4_define([CREATE_BATCH_SPS],
+ [AT_DATA([batch.sps], [dnl
+input program
+loop #i = 1 to 5
++ compute z = #i
++ end case
+end loop
+end file
+end input program
+])])
+
+AT_SETUP([INSERT SYNTAX=INTERACTIVE])
+CREATE_BATCH_SPS
+AT_DATA([insert.sps], [dnl
+INSERT
+ FILE='batch.sps'
+ SYNTAX=interactive.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
+batch.sps:2.1-2.4: error: INPUT PROGRAM: Syntax error expecting end of command.
+ 2 | loop #i = 1 to 5
+ | ^~~~
+batch.sps:3.4-3.10: error: COMPUTE: COMPUTE is allowed only after the active dataset has been defined or inside INPUT PROGRAM.
+ 3 | + compute z = #i
+ | ^~~~~~~
+batch.sps:4.4-4.11: error: END CASE: END CASE is allowed only inside INPUT PROGRAM.
+ 4 | + end case
+ | ^~~~~~~~
+insert.sps:4.1-4.4: error: LIST: LIST is allowed only after the active dataset has been defined.
+ 4 | LIST.
+ | ^~~~
+])
+AT_CLEANUP
+
+AT_SETUP([INSERT SYNTAX=BATCH])
+CREATE_BATCH_SPS
+AT_DATA([insert.sps], [dnl
+INSERT
+ FILE='batch.sps'
+ SYNTAX=BATCH.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv insert.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+z
+1.00
+2.00
+3.00
+4.00
+5.00
+])
+AT_CLEANUP
+
+AT_SETUP([INSERT CD=NO])
+AT_DATA([insert.sps], [INSERT FILE='Dir1/foo.sps'.
+LIST.
+])
+mkdir Dir1
+AT_DATA([Dir1/foo.sps], [INSERT FILE='bar.sps' CD=NO.
+])
+AT_DATA([Dir1/bar.sps],
+ [DATA LIST LIST /x *.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+])
+AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
+Dir1/foo.sps:1: error: INSERT: Can't find `bar.sps' in include file search path.
+insert.sps:2.1-2.4: error: LIST: LIST is allowed only after the active dataset has been defined.
+ 2 | LIST.
+ | ^~~~
+])
+AT_CLEANUP
+
+AT_SETUP([INSERT CD=YES])
+AT_DATA([insert.sps], [INSERT FILE='Dir1/foo.sps' CD=YES.
+LIST.
+])
+mkdir Dir1
+AT_DATA([Dir1/foo.sps], [INSERT FILE='bar.sps'.
+])
+AT_DATA([Dir1/bar.sps],
+ [DATA LIST LIST /x *.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+])
+AT_CHECK([pspp -o pspp.csv insert.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+
+Table: Data List
+x
+1.00
+2.00
+3.00
+])
+AT_CLEANUP
+
+m4_define([CREATE_ERROR_SPS],
+ [AT_DATA([error.sps], [dnl
+DATA LIST NOTABLE LIST /x *.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+
+* The following line is erroneous
+
+DISPLAY AKSDJ.
+LIST.
+])])
+
+AT_SETUP([INSERT ERROR=STOP])
+CREATE_ERROR_SPS
+AT_DATA([insert.sps], [INSERT FILE='error.sps' ERROR=STOP.
+])
+AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
+error.sps:10.9-10.13: error: DISPLAY: AKSDJ is not a variable name.
+ 10 | DISPLAY AKSDJ.
+ | ^~~~~
+warning: Error encountered while ERROR=STOP is effective.
+])
+AT_CLEANUP
+
+AT_SETUP([INSERT ERROR=CONTINUE])
+CREATE_ERROR_SPS
+AT_DATA([insert.sps], [INSERT FILE='error.sps' ERROR=CONTINUE.
+])
+AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
+error.sps:10.9-10.13: error: DISPLAY: AKSDJ is not a variable name.
+ 10 | DISPLAY AKSDJ.
+ | ^~~~~
+])
+AT_CHECK([cat pspp.csv], [0], [dnl
+"error.sps:10.9-10.13: error: DISPLAY: AKSDJ is not a variable name.
+ 10 | DISPLAY AKSDJ.
+ | ^~~~~"
+
+Table: Data List
+x
+1.00
+2.00
+3.00
+])
+AT_CLEANUP
+
+dnl Test for regression against bug #24569 in which PSPP crashed
+dnl upon attempt to insert a nonexistent file.
+AT_SETUP([INSERT nonexistent file])
+AT_DATA([insert.sps], [dnl
+INSERT
+ FILE='nonexistent'
+ ERROR=CONTINUE.
+ .
+
+LIST.
+])
+AT_CHECK([pspp -O format=csv insert.sps], [1], [dnl
+insert.sps:2: error: INSERT: Can't find `nonexistent' in include file search path.
+
+"insert.sps:6.1-6.4: error: LIST: LIST is allowed only after the active dataset has been defined.
+ 6 | LIST.
+ | ^~~~"
+])
+AT_CLEANUP
+
+
+dnl A test to check the INCLUDE command complete with the
+dnl syntax and function of the ENCODING subcommand.
+AT_SETUP([INCLUDE full check])
+AT_DATA([two-utf8.sps], [dnl
+echo 'Äpfelfölfaß'.
+])
+
+AT_DATA([include.sps], [dnl
+echo 'ONE'.
+
+include FILE='two-latin1.sps' ENCODING='ISO_8859-1'.
+])
+
+AT_CHECK([iconv -f UTF-8 -t iso-8859-1 two-utf8.sps > two-latin1.sps], [0], [])
+
+AT_CHECK([pspp -O format=csv include.sps], [0], [dnl
+ONE
+
+Äpfelfölfaß
+])
+AT_CLEANUP
+
+
+
+
+dnl Test for a bug where insert crashed on an unterminated string input
+AT_SETUP([INSERT unterminated string])
+
+AT_DATA([insert.sps], [INSERT FILE=7bar.sps' CD=NO.
+])
+
+AT_CHECK([pspp -O format=csv insert.sps], [1], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([INSERT syntax errors])
+AT_KEYWORDS([INCLUDE])
+: >inner.sps
+AT_DATA([insert.sps], [dnl
+INSERT **.
+INSERT 'nonexistent.sps'.
+INSERT 'inner.sps' ENCODING=**.
+INSERT 'inner.sps' SYNTAX=**.
+INSERT 'inner.sps' CD=**.
+INSERT 'inner.sps' ERROR=**.
+INSERT 'inner.sps' **.
+INCLUDE 'inner.sps' **.
+])
+AT_CHECK([pspp -O format=csv insert.sps], [1], [dnl
+"insert.sps:1.8-1.9: error: INSERT: Syntax error expecting string.
+ 1 | INSERT **.
+ | ^~"
+
+insert.sps:2: error: INSERT: Can't find `nonexistent.sps' in include file search path.
+
+"insert.sps:3.29-3.30: error: INSERT: Syntax error expecting string.
+ 3 | INSERT 'inner.sps' ENCODING=**.
+ | ^~"
+
+"insert.sps:4.27-4.28: error: INSERT: Syntax error expecting BATCH, INTERACTIVE, or AUTO.
+ 4 | INSERT 'inner.sps' SYNTAX=**.
+ | ^~"
+
+"insert.sps:5.23-5.24: error: INSERT: Syntax error expecting YES or NO.
+ 5 | INSERT 'inner.sps' CD=**.
+ | ^~"
+
+"insert.sps:6.26-6.27: error: INSERT: Syntax error expecting CONTINUE or STOP.
+ 6 | INSERT 'inner.sps' ERROR=**.
+ | ^~"
+
+"insert.sps:7.20-7.21: error: INSERT: Syntax error expecting ENCODING, SYNTAX, CD, or ERROR.
+ 7 | INSERT 'inner.sps' **.
+ | ^~"
+
+"insert.sps:8.21-8.22: error: INCLUDE: Syntax error expecting ENCODING.
+ 8 | INCLUDE 'inner.sps' **.
+ | ^~"
+])
+AT_CLEANUP
--- /dev/null
+AT_BANNER([LEAVE])
+
+AT_SETUP([LEAVE])
+AT_DATA([leave.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+LEAVE x y.
+])
+AT_CHECK([pspp -O format=csv leave.sps])
+AT_CLEANUP
+
+AT_SETUP([LEAVE syntax errors])
+AT_DATA([leave.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+LEAVE **.
+LEAVE x **.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='leave.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"leave.sps:2.7-2.8: error: LEAVE: Syntax error expecting variable name.
+ 2 | LEAVE **.
+ | ^~"
+
+"leave.sps:3.9-3.10: error: LEAVE: Syntax error expecting end of command.
+ 3 | LEAVE x **.
+ | ^~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([LIST])
+
+AT_SETUP([LIST plain cases])
+AT_DATA([data.txt], [dnl
+ 18 1
+ 19 7
+ 20 26
+ 21 76
+ 22 57
+ 23 58
+ 24 38
+ 25 38
+ 26 30
+ 27 21
+ 28 23
+])
+AT_DATA([list.sps], [dnl
+DATA LIST FILE='data.txt'/avar 1-5 bvar 6-10.
+WEIGHT BY bvar.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading 1 record from `data.txt'.
+Variable,Record,Columns,Format
+avar,1,1-5,F5.0
+bvar,1,6-10,F5.0
+
+Table: Data List
+avar,bvar
+18,1
+19,7
+20,26
+21,76
+22,57
+23,58
+24,38
+25,38
+26,30
+27,21
+28,23
+])
+AT_CLEANUP
+
+AT_SETUP([LIST numbered cases])
+AT_DATA([data.txt], [dnl
+ 18 1
+ 19 7
+ 20 26
+ 21 76
+ 22 57
+ 23 58
+ 24 38
+ 25 38
+ 26 30
+ 27 21
+ 28 23
+])
+AT_DATA([list.sps], [dnl
+DATA LIST FILE='data.txt'/avar 1-5 bvar 6-10.
+WEIGHT BY bvar.
+LIST/FORMAT NUMBERED.
+LIST/FORMAT NUMBERED/CASES FROM 2 TO 9 BY 2.
+])
+AT_CHECK([pspp -o pspp.csv list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading 1 record from `data.txt'.
+Variable,Record,Columns,Format
+avar,1,1-5,F5.0
+bvar,1,6-10,F5.0
+
+Table: Data List
+Case Number,avar,bvar
+1,18,1
+2,19,7
+3,20,26
+4,21,76
+5,22,57
+6,23,58
+7,24,38
+8,25,38
+9,26,30
+10,27,21
+11,28,23
+
+Table: Data List
+Case Number,avar,bvar
+2,19,7
+4,21,76
+6,23,58
+8,25,38
+])
+AT_CLEANUP
+
+# Checks for a crash when LIST did not include the variables from SPLIT
+# FILE in the same positions (bug #30684).
+AT_SETUP([LIST with split file])
+AT_DATA([data.txt], [dnl
+a 1
+a 2
+a 3
+b 1
+c 4
+c 5
+])
+AT_DATA([list.sps], [dnl
+DATA LIST LIST NOTABLE FILE='data.txt'/s (a1) n.
+SPLIT FILE BY s.
+LIST n.
+])
+AT_CHECK([pspp -o pspp.csv list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Split Values
+Variable,Value
+s,a
+
+Table: Data List
+n
+1.00
+2.00
+3.00
+
+Table: Split Values
+Variable,Value
+s,b
+
+Table: Data List
+n
+1.00
+
+Table: Split Values
+Variable,Value
+s,c
+
+Table: Data List
+n
+4.00
+5.00
+])
+AT_CLEANUP
+
+AT_SETUP([LIST lots of variables])
+AT_DATA([data.txt], [dnl
+767532466348513789073483106409
+888693089424177542378334186760
+492611507909187152726427852242
+819848892023195875879332001491
+452777898709563729845541516650
+239961967077732760663525115073
+])
+AT_DATA([list.sps], [dnl
+DATA LIST FILE='data.txt' NOTABLE/x01 to x30 1-30.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+x01,x02,x03,x04,x05,x06,x07,x08,x09,x10,x11,x12,x13,x14,x15,x16,x17,x18,x19,x20,x21,x22,x23,x24,x25,x26,x27,x28,x29,x30
+7,6,7,5,3,2,4,6,6,3,4,8,5,1,3,7,8,9,0,7,3,4,8,3,1,0,6,4,0,9
+8,8,8,6,9,3,0,8,9,4,2,4,1,7,7,5,4,2,3,7,8,3,3,4,1,8,6,7,6,0
+4,9,2,6,1,1,5,0,7,9,0,9,1,8,7,1,5,2,7,2,6,4,2,7,8,5,2,2,4,2
+8,1,9,8,4,8,8,9,2,0,2,3,1,9,5,8,7,5,8,7,9,3,3,2,0,0,1,4,9,1
+4,5,2,7,7,7,8,9,8,7,0,9,5,6,3,7,2,9,8,4,5,5,4,1,5,1,6,6,5,0
+2,3,9,9,6,1,9,6,7,0,7,7,7,3,2,7,6,0,6,6,3,5,2,5,1,1,5,0,7,3
+])
+AT_CLEANUP
+
+AT_SETUP([LIST selected cases])
+AT_DATA([data.txt], [dnl
+7675324663
+8886930894
+4926115079
+8198488920
+4527778987
+2399619670
+1667799691
+1623914684
+3681393233
+6418731145
+2284534083
+6617637452
+9865713582
+1163234537
+9981663637
+6821567746
+0952774952
+1641790193
+3763182871
+2046820753
+7970620091
+4841176017
+6949973797
+1396285996
+0700489524
+])
+AT_DATA([list.sps], [dnl
+DATA LIST FILE='data.txt' NOTABLE/x0 to x9 1-10.
+LIST /CASES=FROM 6 TO 20 BY 5.
+LIST /CASES=4.
+LIST /CASES=BY 10.
+LIST /CASES=FROM 25.
+LIST /CASES=FROM 26.
+])
+AT_CHECK([pspp -o pspp.csv list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
+2,3,9,9,6,1,9,6,7,0
+2,2,8,4,5,3,4,0,8,3
+6,8,2,1,5,6,7,7,4,6
+
+Table: Data List
+x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
+7,6,7,5,3,2,4,6,6,3
+8,8,8,6,9,3,0,8,9,4
+4,9,2,6,1,1,5,0,7,9
+8,1,9,8,4,8,8,9,2,0
+
+Table: Data List
+x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
+7,6,7,5,3,2,4,6,6,3
+2,2,8,4,5,3,4,0,8,3
+7,9,7,0,6,2,0,0,9,1
+
+Table: Data List
+x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
+0,7,0,0,4,8,9,5,2,4
+
+Table: Data List
+x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
+])
+AT_CLEANUP
+
+dnl This program tests for a bug which caused a buffer overflow
+dnl when the list command attempted to write very long strings.
+AT_SETUP([LIST very long string])
+AT_DATA([list.sps], [dnl
+INPUT PROGRAM.
+STRING foo (a2000).
++ COMPUTE foo=CONCAT(RPAD('A',1999, 'x'), 'Z').
+END CASE.
+END FILE.
+END INPUT PROGRAM.
+
+EXECUTE.
+
+DISPLAY VARIABLES.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Print Format,Write Format
+foo,1,A2000,A2000
+
+Table: Data List
+foo
+AxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxZ
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([LIST crash on invalid input])
+
+AT_DATA([list.sps], [dnl
+DATA LIST LIST /`$b.
+BEGIN DATA.
+1 3
+4 6
+7 9
+END DATA.
+
+LIST.
+])
+
+AT_CHECK([pspp -o pspp.csv list.sps], [1], [ignore])
+
+AT_CLEANUP
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([LIST tutorial example])
+AT_DATA([list.sps], [dnl
+data list list /forename (A12) height.
+begin data.
+Ahmed 188
+Bertram 167
+Catherine 134.231
+David 109.1
+end data
+
+list /format=numbered.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt list.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+forename,A12
+height,F8.0
+
+Table: Data List
+Case Number,forename,height
+1,Ahmed,188.00
+2,Bertram,167.00
+3,Catherine,134.23
+4,David,109.10
+])
+AT_CLEANUP
+
+AT_SETUP([LIST syntax errors])
+AT_DATA([list.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+LIST VARIABLES=**.
+LIST FORMAT=**.
+LIST CASES=FROM -1.
+LIST CASES=FROM 5 TO 4.
+LIST CASES=BY 0.
+LIST **.
+])
+AT_CHECK([pspp -O format=csv list.sps], [1], [dnl
+"list.sps:2.16-2.17: error: LIST: Syntax error expecting variable name.
+ 2 | LIST VARIABLES=**.
+ | ^~"
+
+"list.sps:3.13-3.14: error: LIST: Syntax error expecting NUMBERED or UNNUMBERED.
+ 3 | LIST FORMAT=**.
+ | ^~"
+
+"list.sps:4.17-4.18: error: LIST: Syntax error expecting positive integer for FROM.
+ 4 | LIST CASES=FROM -1.
+ | ^~"
+
+"list.sps:5.22: error: LIST: Syntax error expecting integer 5 or greater for TO.
+ 5 | LIST CASES=FROM 5 TO 4.
+ | ^"
+
+"list.sps:6.15: error: LIST: Syntax error expecting positive integer for TO.
+ 6 | LIST CASES=BY 0.
+ | ^"
+
+"list.sps:7.6-7.7: error: LIST: Syntax error expecting variable name.
+ 7 | LIST **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([LOGISTIC REGRESSION])
+
+dnl These examples are adapted from
+dnl http://www.uvm.edu/~dhowell/gradstat/psych341/lectures/Logistic%20Regression/LogisticReg1.html
+
+
+
+m4_define([LOGIT_TEST_DATA],
+ [AT_DATA([lr-data.txt], dnl
+ 105.00 1.00 33.00 3.00 2.00 .35 17.00 20.00 .50110 -2.00440 1
+ 106.00 1.00 50.00 2.00 3.00 .38 7.00 15.00 .20168 -1.25264 1
+ 107.00 1.00 91.00 3.00 2.00 .28 15.00 7.00 .00897 -1.00905 1
+ 108.00 1.00 90.00 3.00 2.00 .20 2.00 2.00 .00972 -1.00982 1
+ 109.00 1.00 70.00 3.00 3.00 .38 23.00 27.00 .04745 -1.04981 1
+ 111.00 2.00 31.00 2.00 2.00 .00 19.00 10.00 .54159 1.84640 1
+ 112.00 1.00 91.00 2.00 3.00 .18 6.00 16.00 .00897 -1.00905 1
+ 113.00 1.00 81.00 3.00 2.00 .00 3.00 9.00 .01998 -1.02039 1
+ 114.00 2.00 15.00 1.00 2.00 .13 19.00 13.00 .81241 1.23090 1
+ 116.00 2.00 1.00 1.00 2.00 .88 15.00 7.00 .93102 1.07410 1
+ 117.00 1.00 93.00 3.00 2.00 .18 9.00 15.00 .00764 -1.00770 1
+ 118.00 2.00 14.00 1.00 3.00 .15 23.00 18.00 .82447 1.21289 1
+ 120.00 1.00 91.00 2.00 2.00 .43 17.00 14.00 .00897 -1.00905 1
+ 121.00 1.00 55.00 3.00 2.00 .69 20.00 14.00 .14409 -1.16834 1
+ 122.00 1.00 70.00 2.00 3.00 .03 .00 6.00 .04745 -1.04981 1
+ 123.00 1.00 25.00 2.00 2.00 .45 4.00 10.00 .65789 -2.92301 1
+ 125.00 1.00 91.00 2.00 2.00 .13 .00 3.00 .00897 -1.00905 1
+ 126.00 1.00 91.00 3.00 3.00 .23 4.00 6.00 .00897 -1.00905 1
+ 127.00 1.00 91.00 3.00 2.00 .00 8.00 8.00 .00897 -1.00905 1
+ 128.00 2.00 13.00 2.00 2.00 .65 16.00 14.00 .83592 1.19629 1
+ 129.00 1.00 50.00 2.00 2.00 .25 20.00 23.00 .20168 -1.25264 1
+ 135.00 1.00 90.00 3.00 3.00 .03 5.00 12.00 .00972 -1.00982 1
+ 138.00 1.00 70.00 3.00 3.00 .10 1.00 6.00 .04745 -1.04981 1
+ 139.00 2.00 19.00 3.00 3.00 .10 11.00 12.00 .75787 1.31949 1
+ 149.00 2.00 50.00 3.00 2.00 .03 .00 .00 .20168 4.95826 1
+ 204.00 1.00 50.00 3.00 1.00 .13 .00 1.00 .20168 -1.25264 1
+ 205.00 1.00 91.00 3.00 3.00 .72 16.00 18.00 .00897 -1.00905 1
+ 206.00 2.00 24.00 1.00 1.00 .10 5.00 21.00 .67592 1.47947 1
+ 207.00 1.00 80.00 3.00 3.00 .13 6.00 7.00 .02164 -1.02212 1
+ 208.00 1.00 87.00 2.00 2.00 .18 9.00 20.00 .01237 -1.01253 1
+ 209.00 1.00 70.00 2.00 2.00 .53 15.00 12.00 .04745 -1.04981 1
+ 211.00 1.00 55.00 2.00 1.00 .33 8.00 5.00 .14409 -1.16834 1
+ 212.00 1.00 56.00 3.00 1.00 .30 6.00 20.00 .13436 -1.15522 1
+ 214.00 1.00 54.00 2.00 2.00 .15 .00 16.00 .15439 -1.18258 1
+ 215.00 1.00 71.00 3.00 3.00 .35 12.00 12.00 .04391 -1.04592 1
+ 217.00 2.00 36.00 1.00 1.00 .10 12.00 8.00 .44049 2.27020 1
+ 218.00 1.00 91.00 2.00 2.00 .05 11.00 25.00 .00897 -1.00905 1
+ 219.00 1.00 91.00 2.00 2.00 1.23 11.00 24.00 .00897 -1.00905 1
+ 220.00 1.00 91.00 2.00 3.00 .08 8.00 11.00 .00897 -1.00905 1
+ 221.00 1.00 91.00 2.00 2.00 .33 5.00 11.00 .00897 -1.00905 1
+ 222.00 2.00 36.00 2.00 1.00 .18 5.00 3.00 .44049 2.27020 1
+ 223.00 1.00 70.00 2.00 3.00 .18 14.00 3.00 .04745 -1.04981 1
+ 224.00 1.00 91.00 2.00 2.00 .43 2.00 10.00 .00897 -1.00905 1
+ 225.00 1.00 55.00 2.00 1.00 .18 6.00 11.00 .14409 -1.16834 1
+ 229.00 2.00 75.00 2.00 2.00 .40 30.00 25.00 .03212 31.12941 1
+ 232.00 1.00 91.00 3.00 2.00 .15 6.00 3.00 .00897 -1.00905 1
+ 233.00 1.00 70.00 2.00 1.00 .00 11.00 8.00 .04745 -1.04981 1
+ 234.00 1.00 54.00 3.00 2.00 .10 .00 .00 .15439 -1.18258 1
+ 237.00 1.00 70.00 3.00 2.00 .18 5.00 25.00 .04745 -1.04981 1
+ 241.00 1.00 19.00 2.00 3.00 .33 13.00 9.00 .75787 -4.12995 1
+ 304.00 2.00 18.00 2.00 2.00 .26 25.00 6.00 .77245 1.29458 1
+ 305.00 1.00 88.00 3.00 2.00 1.35 17.00 29.00 .01142 -1.01155 1
+ 306.00 1.00 70.00 2.00 3.00 .63 14.00 33.00 .04745 -1.04981 1
+ 307.00 1.00 85.00 2.00 2.00 2.65 18.00 14.00 .01452 -1.01474 1
+ 308.00 1.00 13.00 2.00 2.00 .23 5.00 5.00 .83592 -6.09442 1
+ 309.00 2.00 13.00 2.00 2.00 .23 7.00 17.00 .83592 1.19629 1
+ 311.00 2.00 1.00 2.00 2.00 .50 20.00 14.00 .93102 1.07410 1
+ 315.00 1.00 19.00 2.00 3.00 .18 1.00 11.00 .75787 -4.12995 1
+ 316.00 1.00 88.00 2.00 2.00 .38 12.00 11.00 .01142 -1.01155 2
+ 318.00 1.00 88.00 3.00 2.00 .03 5.00 5.00 .01142 -1.01155 3
+ 319.00 2.00 18.00 2.00 3.00 .30 15.00 16.00 .77245 1.29458 1
+ 321.00 2.00 15.00 2.00 2.00 .63 15.00 18.00 .81241 1.23090 1
+ 322.00 1.00 88.00 3.00 2.00 .40 18.00 15.00 .01142 -1.01155 1
+ 325.00 2.00 18.00 2.00 3.00 1.00 28.00 18.00 .77245 1.29458 1
+ 329.00 1.00 88.00 3.00 2.00 .03 7.00 11.00 .01142 -1.01155 4
+ 332.00 2.00 2.00 2.00 2.00 .05 8.00 9.00 .92562 1.08036 1
+)])
+
+dnl Note: In the above data cases 305, 316 318 and 329 have identical values
+dnl of the 2nd and 3rd variables. We use this for weight testing.
+
+AT_SETUP([LOGISTIC REGRESSION basic test])
+AT_KEYWORDS([categorical categoricals])
+
+LOGIT_TEST_DATA
+
+AT_DATA([lr-data.sps], [dnl
+set format = F12.3.
+set decimal dot.
+data list notable file='lr-data.txt'
+ list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
+
+logistic regression
+ variables = outcome with survrate
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt lr-data.sps], [0], [dnl
+note: Estimation terminated at iteration number 6 because parameter estimates changed by less than 0.001
+])
+AT_CHECK([cat pspp.csv], [0], [Table: Dependent Variable Encoding
+Original Value,Internal Value
+1.000,.000
+2.000,1.000
+
+Table: Case Processing Summary
+Unweighted Cases,N,Percent
+Included in Analysis,66,100.0%
+Missing Cases,0,.0%
+Total,66,100.0%
+
+note: Estimation terminated at iteration number 6 because parameter estimates changed by less than 0.001
+
+Table: Model Summary
+Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
+1,37.323,.455,.659
+
+Table: Classification Table
+,Observed,,Predicted,,
+,,,outcome,,Percentage Correct
+,,,1.000,2.000,
+Step 1,outcome,1.000,43,5,89.6%
+,,2.000,4,14,77.8%
+,Overall Percentage,,,,86.4%
+
+Table: Variables in the Equation
+,,B,S.E.,Wald,df,Sig.,Exp(B)
+Step 1,survrate,-.081,.019,17.756,1,.000,.922
+,Constant,2.684,.811,10.941,1,.001,14.639
+])
+AT_CLEANUP
+
+AT_SETUP([LOGISTIC REGRESSION missing values])
+AT_KEYWORDS([categorical categoricals])
+
+LOGIT_TEST_DATA
+
+AT_DATA([lr-data.sps], [dnl
+set format = F12.3.
+set decimal dot.
+data list notable file='lr-data.txt'
+ list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
+
+missing values survrate (999) avoid (44444) outcome (99).
+
+logistic regression
+ variables = outcome with survrate avoid
+ .
+])
+
+AT_CHECK([pspp -O format=csv lr-data.sps > run0], [0], [ignore])
+
+dnl Append some cases with missing values into the data.
+cat >> lr-data.txt << HERE
+ 105.00 1.00 999.00 3.00 2.00 .35 17.00 20.00 .50110 -2.00440 1
+ 106.00 1.00 999.00 2.00 3.00 .38 7.00 15.00 .20168 -1.25264 1
+ 107.00 1.00 5.00 3.00 2.00 .28 44444 34 .00897 -1.00905 1
+ 108.00 99 5.00 3.00 2.00 .28 4 34 .00897 -1.00905 1
+HERE
+
+AT_CHECK([pspp -O format=csv lr-data.sps > run1], [0], [ignore])
+
+dnl Only the summary information should be different
+AT_CHECK([diff run0 run1], [1], [dnl
+8,10c8,10
+< Included in Analysis,66,100.0%
+< Missing Cases,0,.0%
+< Total,66,100.0%
+---
+> Included in Analysis,66,94.3%
+> Missing Cases,4,5.7%
+> Total,70,100.0%
+])
+
+AT_CLEANUP
+
+
+
+dnl Check that a weighted dataset is interpreted correctly
+dnl To do this, the same data set is used, one weighted, one not.
+dnl The weighted dataset omits certain cases which are identical
+AT_SETUP([LOGISTIC REGRESSION weights])
+AT_KEYWORDS([categorical categoricals])
+
+LOGIT_TEST_DATA
+
+AT_DATA([lr-data-unweighted.sps], [dnl
+set format = F12.3.
+set decimal dot.
+data list notable file='lr-data.txt'
+ list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
+
+logistic regression
+ variables = outcome with survrate
+ .
+])
+
+AT_DATA([lr-data-weighted.sps], [dnl
+set format = F12.3.
+set decimal dot.
+data list notable file='lr-data.txt'
+ list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
+
+weight by w.
+
+* Omit duplicate cases.
+select if id <> 305 and id <> 316 and id <> 318.
+
+logistic regression
+ variables = outcome with survrate
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv lr-data-unweighted.sps > unweighted-result], [0], [ignore])
+AT_CHECK([pspp -O format=csv lr-data-weighted.sps > weighted-result], [0], [ignore])
+
+dnl The only difference should be the summary information, since
+dnl this displays the unweighted totals.
+AT_CHECK([diff unweighted-result weighted-result], [1], [dnl
+8c8
+< Included in Analysis,66,100.0%
+---
+> Included in Analysis,63,100.0%
+10c10
+< Total,66,100.0%
+---
+> Total,63,100.0%
+22,23c22,23
+< Step 1,outcome,1.000,43,5,89.6%
+< ,,2.000,4,14,77.8%
+---
+> Step 1,outcome,1.000,43.000,5.000,89.6%
+> ,,2.000,4.000,14.000,77.8%
+])
+
+
+AT_CLEANUP
+
+
+dnl Check that the /NOCONST option works as intended.
+dnl The results this produces are very similar to those
+dnl at the example in http://www.ats.ucla.edu/stat/SPSS/faq/logregconst.htm
+AT_SETUP([LOGISTIC REGRESSION without constant])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([non-const.sps], [dnl
+set format=F20.3.
+
+input program.
+ loop #i = 1 to 200.
+ compute female = (#i > 91).
+ end case.
+ end loop.
+end file.
+end input program.
+
+compute constant = 1.
+
+logistic regression female with constant /noconst.
+])
+
+AT_CHECK([pspp -O format=csv non-const.sps], [0], [dnl
+Table: Dependent Variable Encoding
+Original Value,Internal Value
+.00,.000
+1.00,1.000
+
+Table: Case Processing Summary
+Unweighted Cases,N,Percent
+Included in Analysis,200,100.0%
+Missing Cases,0,.0%
+Total,200,100.0%
+
+note: Estimation terminated at iteration number 2 because parameter estimates changed by less than 0.001
+
+Table: Model Summary
+Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
+1,275.637,.008,.011
+
+Table: Classification Table
+,Observed,,Predicted,,
+,,,female,,Percentage Correct
+,,,.00,1.00,
+Step 1,female,.00,0,91,.0%
+,,1.00,0,109,100.0%
+,Overall Percentage,,,,54.5%
+
+Table: Variables in the Equation
+,,B,S.E.,Wald,df,Sig.,Exp(B)
+Step 1,constant,.180,.142,1.616,1,.204,1.198
+])
+
+AT_CLEANUP
+
+
+
+dnl Check that if somebody passes a dependent variable which is not dichtomous,
+dnl then an error is raised.
+AT_SETUP([LOGISTIC REGRESSION non-dichotomous dep var])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([non-dich.sps], [dnl
+data list notable list /y x1 x2 x3 x4.
+begin data.
+1 2 3 4 5
+0 2 3 4 8
+2 3 4 5 6
+end data.
+
+logistic regression y with x1 x2 x3 x4.
+])
+
+AT_CHECK([pspp -O format=csv non-dich.sps], [1],
+ [dnl
+error: Dependent variable's values are not dichotomous.
+])
+
+AT_CLEANUP
+
+
+
+dnl An example to check the behaviour of LOGISTIC REGRESSION with a categorical
+dnl variable. This examṕle was inspired from that at:
+dnl http://www.ats.ucla.edu/stat/spss/dae/logit.htm
+AT_SETUP([LOGISTIC REGRESSION with categorical])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([lr-cat.data], [dnl
+ 620 3.07 2 4
+ 800 4.00 3 9
+ 580 3.40 2 4
+ 600 3.13 2 4
+ 540 2.70 2 4
+ 660 3.31 4 4
+ 480 3.58 1 9
+ 620 4.00 1 9
+ 680 3.98 2 9
+ 580 3.40 4 4
+ 760 3.35 3 4
+ 700 3.72 2 4
+ 460 3.64 1 9
+ 540 3.28 3 4
+ 680 3.48 3 4
+ 740 3.31 1 4
+ 460 3.77 3 4
+ 740 3.54 1 4
+ 600 3.63 3 4
+ 620 3.05 2 4
+ 560 3.04 3 4
+ 520 2.70 3 4
+ 640 3.35 3 4
+ 620 3.58 2 4
+ 660 3.70 4 9
+ 500 2.86 4 4
+ 640 3.50 2 4
+ 720 4.00 3 4
+ 720 3.94 3 4
+ 400 3.65 2 4
+ 800 2.90 2 4
+ 520 2.90 3 4
+ 440 3.24 4 4
+ 580 3.51 2 4
+ 500 3.31 3 4
+ 440 3.22 1 4
+ 540 3.17 1 9
+ 420 3.02 1 4
+ 780 3.22 2 9
+ 440 3.13 4 4
+ 800 3.66 1 9
+ 580 3.32 2 9
+ 480 2.67 2 9
+ 700 4.00 1 9
+ 740 2.97 2 9
+ 700 3.83 2 4
+ 640 3.93 2 4
+ 800 3.90 2 4
+ 400 3.38 2 4
+ 700 3.52 2 4
+ 680 3.00 4 9
+ 540 3.20 1 4
+ 580 4.00 2 4
+ 780 4.00 2 9
+ 220 2.83 3 4
+ 580 3.20 2 9
+ 580 3.50 2 4
+ 620 3.30 1 4
+ 520 3.65 4 9
+ 600 3.38 3 9
+ 660 3.77 3 4
+ 580 2.86 4 9
+ 580 3.46 2 9
+ 560 3.36 3 4
+ 740 4.00 3 9
+ 480 3.44 3 4
+ 640 3.19 4 9
+ 600 3.54 1 9
+ 540 3.38 4 4
+ 500 2.81 3 4
+ 360 2.56 3 4
+ 460 3.15 4 4
+ 460 2.63 2 4
+ 440 2.76 2 4
+ 740 3.62 4 4
+ 380 3.38 2 4
+ 640 3.63 1 9
+ 800 3.73 1 4
+ 660 3.67 2 4
+ 760 3.00 2 9
+ 420 2.96 1 4
+ 740 3.74 4 4
+ 800 3.75 2 4
+ 620 3.40 2 4
+ 660 3.67 3 9
+ 400 3.35 3 4
+ 680 3.14 2 4
+ 660 3.47 3 9
+ 660 3.63 2 9
+ 420 3.41 4 4
+ 660 4.00 1 4
+ 680 3.70 2 4
+ 620 3.23 3 9
+ 520 3.35 3 4
+ 500 4.00 3 4
+ 400 3.36 2 4
+ 700 3.56 1 9
+ 540 3.81 1 9
+ 520 2.68 3 9
+ 540 3.50 2 4
+ 700 4.00 2 4
+ 600 3.64 3 9
+ 800 3.31 3 4
+ 520 3.29 1 4
+ 580 3.69 1 4
+ 380 3.43 3 4
+ 560 3.19 3 4
+ 760 2.81 1 9
+ 540 3.13 2 4
+ 660 3.14 2 9
+ 520 3.81 1 9
+ 680 3.19 4 4
+ 540 3.78 4 4
+ 500 3.57 3 4
+ 660 3.49 2 4
+ 340 3.00 2 9
+ 400 3.15 2 9
+ 420 3.92 4 4
+ 760 3.35 2 9
+ 700 2.94 2 4
+ 540 3.04 1 4
+ 780 3.87 4 4
+ 560 3.78 2 4
+ 700 3.82 3 4
+ 400 2.93 3 4
+ 440 3.45 2 9
+ 800 3.47 3 4
+ 340 3.15 3 4
+ 520 4.00 1 9
+ 520 3.15 3 4
+ 600 2.98 2 9
+ 420 2.69 2 4
+ 460 3.44 2 4
+ 620 3.71 1 9
+ 480 3.13 2 4
+ 580 3.40 3 4
+ 540 3.39 3 9
+ 540 3.94 3 4
+ 440 2.98 3 4
+ 380 3.59 4 4
+ 500 2.97 4 4
+ 340 2.92 3 4
+ 440 3.15 2 4
+ 600 3.48 2 4
+ 420 2.67 3 4
+ 460 3.07 2 4
+ 460 3.45 3 9
+ 480 3.39 4 4
+ 480 2.78 3 4
+ 720 3.42 2 9
+ 680 3.67 2 9
+ 800 3.89 2 4
+ 360 3.00 3 4
+ 620 3.17 2 9
+ 700 3.52 4 9
+ 540 3.19 2 4
+ 580 3.30 2 4
+ 800 4.00 3 9
+ 660 3.33 2 4
+ 380 3.34 3 4
+ 720 3.84 3 4
+ 600 3.59 2 4
+ 500 3.03 3 4
+ 640 3.81 2 4
+ 540 3.49 1 9
+ 680 3.85 3 9
+ 540 3.84 2 9
+ 460 2.93 3 4
+ 380 2.94 3 4
+ 620 3.22 2 4
+ 740 3.37 4 4
+ 620 4.00 2 4
+ 800 3.74 1 9
+ 400 3.31 3 4
+ 540 3.46 4 4
+ 620 3.18 2 9
+ 480 2.91 1 9
+ 300 2.84 2 9
+ 440 2.48 4 4
+ 640 2.79 2 4
+ 400 3.23 4 9
+ 680 3.46 2 9
+ 620 3.37 1 9
+ 700 3.92 2 4
+ 620 3.37 2 9
+ 620 3.63 2 4
+ 620 3.95 3 9
+ 560 2.52 2 4
+ 520 2.62 2 4
+ 600 3.35 2 4
+ 700 4.00 1 4
+ 640 3.67 3 4
+ 640 4.00 3 4
+ 520 2.93 4 4
+ 620 3.21 4 4
+ 680 3.99 3 4
+ 660 3.34 3 4
+ 700 3.45 3 4
+ 560 3.36 1 9
+ 800 2.78 2 4
+ 500 3.88 4 4
+ 700 3.65 2 4
+ 680 3.76 3 9
+ 660 3.07 3 4
+ 580 3.46 4 4
+ 460 2.87 2 4
+ 600 3.31 4 4
+ 620 3.94 4 4
+ 400 3.05 2 4
+ 800 3.43 2 9
+ 600 3.58 1 9
+ 580 3.36 2 4
+ 540 3.16 3 4
+ 500 2.71 2 4
+ 600 3.28 3 4
+ 600 2.82 4 4
+ 460 3.58 2 4
+ 520 2.85 3 4
+ 740 3.52 4 9
+ 500 3.95 4 4
+ 560 3.61 3 4
+ 620 3.45 2 9
+ 640 3.51 2 4
+ 660 3.44 2 9
+ 660 2.91 3 9
+ 540 3.28 1 4
+ 560 2.98 1 9
+ 800 3.97 1 4
+ 720 3.77 3 4
+ 720 3.64 1 9
+ 480 3.71 4 9
+ 680 3.34 2 4
+ 680 3.11 2 4
+ 540 2.81 3 4
+ 620 3.75 2 9
+ 540 3.12 1 4
+ 560 3.48 2 9
+ 720 3.40 3 4
+ 680 3.90 1 4
+ 640 3.76 3 4
+ 560 3.16 1 4
+ 520 3.30 2 9
+ 640 3.12 3 4
+ 580 3.57 3 4
+ 540 3.55 4 9
+ 780 3.63 4 9
+ 600 3.89 1 9
+ 800 4.00 1 9
+ 580 3.29 4 4
+ 360 3.27 3 4
+ 800 4.00 2 9
+ 640 3.52 4 4
+ 720 3.45 4 4
+ 580 3.06 2 4
+ 580 3.02 2 4
+ 500 3.60 3 9
+ 580 3.12 3 9
+ 600 2.82 4 4
+ 620 3.99 3 4
+ 700 4.00 3 4
+ 480 4.00 2 4
+ 560 2.95 2 4
+ 560 4.00 3 4
+ 560 2.65 3 9
+ 400 3.08 2 4
+ 480 2.62 2 9
+ 640 3.86 3 4
+ 480 3.57 2 4
+ 540 3.51 2 4
+ 380 3.33 4 4
+ 680 3.64 3 4
+ 400 3.51 3 4
+ 340 2.90 1 4
+ 700 3.08 2 4
+ 480 3.02 1 9
+ 600 3.15 2 9
+ 780 3.80 3 9
+ 520 3.74 2 9
+ 520 3.51 2 4
+ 640 3.73 3 4
+ 560 3.32 4 4
+ 620 2.85 2 4
+ 700 3.28 1 4
+ 760 4.00 1 9
+ 800 3.60 2 4
+ 580 3.34 2 4
+ 540 3.77 2 9
+ 640 3.17 2 4
+ 540 3.02 4 4
+ 680 3.08 4 4
+ 680 3.31 2 4
+ 680 2.96 3 9
+ 700 2.88 2 4
+ 580 3.77 4 4
+ 540 3.49 2 9
+ 700 3.56 2 9
+ 600 3.56 2 9
+ 560 3.59 2 4
+ 640 2.94 2 9
+ 560 3.33 4 4
+ 620 3.69 3 4
+ 680 3.27 2 9
+ 460 3.14 3 4
+ 500 3.53 4 4
+ 620 3.33 3 4
+ 600 3.62 3 4
+ 500 3.01 4 4
+ 740 3.34 4 4
+ 560 3.69 3 9
+ 620 3.95 3 9
+ 740 3.86 2 9
+ 800 3.53 1 9
+ 620 3.78 3 4
+ 700 3.27 2 4
+ 540 3.78 2 9
+ 700 3.65 2 4
+ 800 3.22 1 9
+ 560 3.59 2 9
+ 800 3.15 4 4
+ 520 3.90 3 9
+ 520 3.74 4 9
+ 480 2.55 1 4
+ 800 4.00 4 4
+ 620 3.09 4 4
+ 560 3.49 4 4
+ 500 3.17 3 4
+ 480 3.40 2 4
+ 460 2.98 1 4
+ 580 3.58 1 9
+ 640 3.30 2 4
+ 480 3.45 2 4
+ 440 3.17 2 4
+ 660 3.32 1 4
+ 500 3.08 3 4
+ 660 3.94 2 4
+ 720 3.31 1 4
+ 460 3.64 3 9
+ 500 2.93 4 4
+ 800 3.54 3 4
+ 580 2.93 2 4
+ 620 3.61 1 9
+ 500 2.98 3 4
+ 660 4.00 2 9
+ 560 3.24 4 4
+ 560 2.42 2 4
+ 580 3.80 2 4
+ 500 3.23 4 4
+ 680 2.42 1 9
+ 580 3.46 3 4
+ 800 3.91 3 4
+ 700 2.90 4 4
+ 520 3.12 2 4
+ 300 2.92 4 4
+ 560 3.43 3 4
+ 620 3.63 3 4
+ 500 2.79 4 4
+ 360 3.14 1 4
+ 640 3.94 2 9
+ 460 3.99 3 9
+ 300 3.01 3 4
+ 520 2.73 2 4
+ 600 3.47 2 9
+ 580 3.25 1 4
+ 520 3.10 4 4
+ 620 3.43 3 4
+ 380 2.91 4 4
+ 660 3.59 3 4
+ 660 3.95 2 9
+ 540 3.33 3 4
+ 740 4.00 3 4
+ 640 3.38 3 4
+ 600 3.89 3 4
+ 720 3.88 3 4
+ 580 4.00 3 4
+ 420 2.26 4 4
+ 520 4.00 2 9
+ 800 3.70 1 9
+ 700 4.00 1 9
+ 480 3.43 2 4
+ 660 3.45 4 4
+ 520 3.25 3 4
+ 560 2.71 3 4
+ 600 3.32 2 4
+ 580 2.88 2 4
+ 660 3.88 2 9
+ 600 3.22 1 4
+ 580 4.00 1 4
+ 660 3.60 3 9
+ 500 3.35 2 4
+ 520 2.98 2 4
+ 660 3.49 2 9
+ 560 3.07 2 4
+ 500 3.13 2 9
+ 720 3.50 3 9
+ 440 3.39 2 9
+ 640 3.95 2 9
+ 380 3.61 3 4
+ 800 3.05 2 9
+ 520 3.19 3 9
+ 600 3.40 3 4
+])
+
+AT_DATA([lr-cat.sps], [dnl
+set format=F20.3.
+
+data list notable list file='lr-cat.data' /b1 b2 bcat y.
+
+logistic regression
+ y with b1 b2 bcat
+ /categorical = bcat
+ .
+])
+
+AT_CHECK([pspp -O format=csv lr-cat.sps], [0], [dnl
+Table: Dependent Variable Encoding
+Original Value,Internal Value
+4.000,.000
+9.000,1.000
+
+Table: Case Processing Summary
+Unweighted Cases,N,Percent
+Included in Analysis,400,100.0%
+Missing Cases,0,.0%
+Total,400,100.0%
+
+note: Estimation terminated at iteration number 4 because parameter estimates changed by less than 0.001
+
+Table: Model Summary
+Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
+1,458.517,.098,.138
+
+Table: Categorical Variables' Codings
+,,Frequency,Parameter coding,,
+,,,(1),(2),(3)
+bcat,1.000,61,1,0,0
+,2.000,151,0,1,0
+,3.000,121,0,0,1
+,4.000,67,0,0,0
+
+Table: Classification Table
+,Observed,,Predicted,,
+,,,y,,Percentage Correct
+,,,4.000,9.000,
+Step 1,y,4.000,254,19,93.0%
+,,9.000,97,30,23.6%
+,Overall Percentage,,,,71.0%
+
+Table: Variables in the Equation
+,,B,S.E.,Wald,df,Sig.,Exp(B)
+Step 1,b1,.002,.001,4.284,1,.038,1.002
+,b2,.804,.332,5.872,1,.015,2.235
+,bcat,,,20.895,3,.000,
+,bcat(1),1.551,.418,13.788,1,.000,4.718
+,bcat(2),.876,.367,5.706,1,.017,2.401
+,bcat(3),.211,.393,.289,1,.591,1.235
+,Constant,-5.541,1.138,23.709,1,.000,.004
+])
+AT_CLEANUP
+
+
+
+dnl This example is inspired by http://www.ats.ucla.edu/stat/spss/output/logistic.htm
+AT_SETUP([LOGISTIC REGRESSION with cat var 2])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([lr-cat2.data], [dnl
+ 60.00 1.00 8.00 50.00
+ 47.00 .00 9.00 42.00
+ 57.00 1.00 7.00 53.00
+ 60.00 .00 8.00 53.00
+ 68.00 .00 8.00 66.00
+ 63.00 .00 8.00 55.00
+ 65.00 .00 8.00 63.00
+ 52.00 .00 8.00 61.00
+ 34.00 .00 9.00 42.00
+ 37.00 .00 8.00 39.00
+ 68.00 1.00 9.00 69.00
+ 60.00 .00 9.00 61.00
+ 44.00 .00 9.00 58.00
+ 42.00 .00 8.00 47.00
+ 57.00 1.00 7.00 61.00
+ 55.00 1.00 8.00 50.00
+ 55.00 .00 9.00 58.00
+ 44.00 .00 8.00 63.00
+ 50.00 1.00 9.00 66.00
+ 44.00 .00 8.00 39.00
+ 55.00 .00 8.00 58.00
+ 44.00 .00 8.00 50.00
+ 47.00 1.00 7.00 34.00
+ 48.00 .00 8.00 44.00
+ 45.00 .00 7.00 31.00
+ 43.00 .00 8.00 50.00
+ 39.00 .00 8.00 42.00
+ 63.00 .00 9.00 50.00
+ 47.00 .00 8.00 58.00
+ 42.00 .00 7.00 50.00
+ 50.00 .00 9.00 36.00
+ 47.00 .00 7.00 33.00
+ 60.00 .00 9.00 61.00
+ 47.00 .00 7.00 42.00
+ 68.00 1.00 9.00 69.00
+ 52.00 .00 8.00 54.00
+ 63.00 1.00 9.00 61.00
+ 65.00 1.00 9.00 61.00
+ 63.00 1.00 9.00 53.00
+ 57.00 .00 8.00 51.00
+ 34.00 .00 8.00 36.00
+ 50.00 .00 8.00 39.00
+ 52.00 1.00 7.00 56.00
+ 45.00 .00 7.00 34.00
+ 47.00 1.00 7.00 53.00
+ 34.00 .00 7.00 39.00
+ 50.00 1.00 8.00 55.00
+ 60.00 .00 9.00 58.00
+ 63.00 .00 8.00 58.00
+ 35.00 .00 7.00 51.00
+ 50.00 .00 8.00 58.00
+ 68.00 .00 8.00 63.00
+ 41.00 .00 9.00 34.00
+ 47.00 .00 8.00 47.00
+ 76.00 .00 9.00 64.00
+ 44.00 .00 8.00 44.00
+ 36.00 .00 9.00 50.00
+ 68.00 1.00 9.00 55.00
+ 47.00 1.00 8.00 50.00
+ 50.00 .00 7.00 53.00
+ 68.00 .00 8.00 74.00
+ 39.00 .00 7.00 44.00
+ 50.00 .00 8.00 55.00
+ 52.00 .00 9.00 61.00
+ 47.00 .00 8.00 53.00
+ 39.00 .00 7.00 47.00
+ 55.00 1.00 9.00 49.00
+ 68.00 1.00 8.00 50.00
+ 52.00 1.00 9.00 63.00
+ 55.00 .00 8.00 58.00
+ 57.00 .00 8.00 55.00
+ 66.00 1.00 9.00 61.00
+ 65.00 1.00 7.00 58.00
+ 42.00 .00 7.00 42.00
+ 68.00 1.00 7.00 59.00
+ 60.00 1.00 9.00 61.00
+ 52.00 .00 8.00 55.00
+ 57.00 1.00 7.00 54.00
+ 42.00 .00 9.00 50.00
+ 42.00 .00 8.00 47.00
+ 57.00 .00 8.00 50.00
+ 47.00 .00 7.00 45.00
+ 44.00 .00 7.00 40.00
+ 43.00 .00 9.00 55.00
+ 31.00 .00 8.00 39.00
+ 37.00 .00 7.00 33.00
+ 63.00 1.00 7.00 63.00
+ 47.00 .00 8.00 39.00
+ 57.00 1.00 8.00 63.00
+ 52.00 .00 8.00 44.00
+ 44.00 .00 7.00 35.00
+ 52.00 .00 7.00 55.00
+ 55.00 .00 7.00 69.00
+ 52.00 .00 8.00 53.00
+ 55.00 .00 9.00 61.00
+ 65.00 1.00 9.00 63.00
+ 55.00 .00 8.00 44.00
+ 63.00 .00 7.00 65.00
+ 44.00 .00 7.00 39.00
+ 47.00 .00 7.00 36.00
+ 63.00 1.00 9.00 55.00
+ 68.00 .00 8.00 66.00
+ 34.00 .00 8.00 39.00
+ 47.00 .00 9.00 50.00
+ 50.00 .00 9.00 58.00
+ 63.00 .00 8.00 66.00
+ 44.00 .00 7.00 34.00
+ 44.00 .00 8.00 50.00
+ 50.00 .00 8.00 53.00
+ 47.00 1.00 9.00 69.00
+ 65.00 .00 9.00 58.00
+ 57.00 .00 8.00 47.00
+ 39.00 .00 8.00 39.00
+ 47.00 .00 8.00 53.00
+ 50.00 1.00 7.00 63.00
+ 50.00 .00 8.00 50.00
+ 63.00 .00 9.00 53.00
+ 73.00 1.00 9.00 61.00
+ 44.00 .00 7.00 47.00
+ 47.00 .00 8.00 42.00
+ 47.00 .00 8.00 58.00
+ 36.00 .00 7.00 61.00
+ 57.00 1.00 8.00 55.00
+ 53.00 1.00 8.00 57.00
+ 63.00 .00 7.00 66.00
+ 50.00 .00 8.00 34.00
+ 47.00 .00 9.00 48.00
+ 57.00 1.00 8.00 58.00
+ 39.00 .00 8.00 53.00
+ 42.00 .00 8.00 42.00
+ 42.00 .00 9.00 31.00
+ 42.00 .00 8.00 72.00
+ 46.00 .00 8.00 44.00
+ 55.00 .00 8.00 42.00
+ 42.00 .00 8.00 47.00
+ 50.00 .00 8.00 44.00
+ 44.00 .00 9.00 39.00
+ 73.00 1.00 8.00 69.00
+ 71.00 1.00 9.00 58.00
+ 50.00 .00 9.00 49.00
+ 63.00 1.00 7.00 54.00
+ 42.00 .00 8.00 36.00
+ 47.00 .00 7.00 42.00
+ 39.00 .00 9.00 26.00
+ 63.00 .00 8.00 58.00
+ 50.00 .00 8.00 55.00
+ 65.00 1.00 8.00 55.00
+ 76.00 1.00 9.00 67.00
+ 71.00 1.00 8.00 66.00
+ 39.00 .00 9.00 47.00
+ 47.00 1.00 9.00 63.00
+ 60.00 .00 7.00 50.00
+ 63.00 .00 9.00 55.00
+ 54.00 1.00 9.00 55.00
+ 55.00 1.00 8.00 58.00
+ 57.00 .00 8.00 61.00
+ 55.00 1.00 9.00 63.00
+ 42.00 .00 7.00 50.00
+ 50.00 .00 8.00 44.00
+ 55.00 .00 8.00 42.00
+ 42.00 .00 7.00 50.00
+ 34.00 .00 8.00 39.00
+ 65.00 .00 9.00 46.00
+ 52.00 .00 7.00 58.00
+ 44.00 .00 8.00 39.00
+ 65.00 1.00 9.00 66.00
+ 47.00 .00 8.00 42.00
+ 41.00 .00 7.00 39.00
+ 68.00 .00 9.00 63.00
+ 63.00 1.00 8.00 72.00
+ 52.00 .00 8.00 53.00
+ 57.00 .00 8.00 50.00
+ 68.00 .00 8.00 55.00
+ 42.00 .00 8.00 56.00
+ 47.00 .00 8.00 48.00
+ 73.00 1.00 9.00 58.00
+ 39.00 .00 8.00 50.00
+ 63.00 1.00 9.00 69.00
+ 60.00 .00 8.00 55.00
+ 65.00 1.00 9.00 66.00
+ 73.00 1.00 8.00 63.00
+ 52.00 .00 8.00 55.00
+ 36.00 .00 8.00 42.00
+ 28.00 .00 7.00 44.00
+ 47.00 .00 8.00 44.00
+ 57.00 .00 7.00 47.00
+ 34.00 .00 7.00 29.00
+ 47.00 .00 9.00 66.00
+ 57.00 .00 8.00 58.00
+ 60.00 1.00 9.00 50.00
+ 50.00 .00 9.00 47.00
+ 73.00 1.00 9.00 55.00
+ 52.00 1.00 8.00 47.00
+ 55.00 .00 8.00 53.00
+ 47.00 .00 8.00 53.00
+ 50.00 .00 8.00 61.00
+ 61.00 .00 7.00 44.00
+ 52.00 .00 9.00 53.00
+ 47.00 .00 7.00 40.00
+ 47.00 .00 7.00 50.00
+])
+
+AT_DATA([stringcat.sps], [dnl
+set format=F20.3 /small=0.
+data list notable file='lr-cat2.data' list /read honcomp wiz science *.
+
+string ses(a1).
+recode wiz (7 = "a") (8 = "b") (9 = "c") into ses.
+
+logistic regression honcomp with read science ses
+ /categorical = ses.
+
+])
+
+AT_CHECK([pspp -O format=csv stringcat.sps], [0], [dnl
+Table: Dependent Variable Encoding
+Original Value,Internal Value
+.000,.000
+1.000,1.000
+
+Table: Case Processing Summary
+Unweighted Cases,N,Percent
+Included in Analysis,200,100.0%
+Missing Cases,0,.0%
+Total,200,100.0%
+
+note: Estimation terminated at iteration number 5 because parameter estimates changed by less than 0.001
+
+Table: Model Summary
+Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
+1,165.701,.280,.408
+
+Table: Categorical Variables' Codings
+,,Frequency,Parameter coding,
+,,,(1),(2)
+ses,a,47,1,0
+,b,95,0,1
+,c,58,0,0
+
+Table: Classification Table
+,Observed,,Predicted,,
+,,,honcomp,,Percentage Correct
+,,,.000,1.000,
+Step 1,honcomp,.000,132,15,89.8%
+,,1.000,26,27,50.9%
+,Overall Percentage,,,,79.5%
+
+Table: Variables in the Equation
+,,B,S.E.,Wald,df,Sig.,Exp(B)
+Step 1,read,.098,.025,15.199,1,.000,1.103
+,science,.066,.027,5.867,1,.015,1.068
+,ses,,,6.690,2,.035,
+,ses(1),.058,.532,.012,1,.913,1.060
+,ses(2),-1.013,.444,5.212,1,.022,.363
+,Constant,-9.561,1.662,33.113,1,.000,.000
+])
+
+AT_CLEANUP
+
+
+dnl Check that it doesn't crash if a categorical variable
+dnl has only one distinct value
+AT_SETUP([LOGISTIC REGRESSION identical categories])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([crash.sps], [dnl
+data list notable list /y x1 x2*.
+begin data
+0 1 1
+1 2 1
+end data.
+
+logistic regression y with x1 x2
+ /categorical = x2.
+])
+
+AT_CHECK([pspp -O format=csv crash.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+dnl Test that missing values on the categorical predictors are treated
+dnl properly.
+AT_SETUP([LOGISTIC REGRESSION missing categoricals])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([data.txt], [dnl
+ .00 3.69 .00
+ .00 1.16 1.00
+ 1.00 -12.99 .00
+ .00 2.97 1.00
+ .00 20.48 .00
+ .00 4.90 .00
+ 1.00 -4.38 .00
+ .00 -1.69 1.00
+ 1.00 -5.71 .00
+ 1.00 -14.28 .00
+ .00 9.00 .00
+ .00 2.89 1.00
+ .00 13.51 1.00
+ .00 23.32 1.00
+ .00 2.31 1.00
+ .00 -2.07 1.00
+ 1.00 -4.52 1.00
+ 1.00 -5.83 .00
+ 1.00 -1.91 .00
+ 1.00 -11.12 1.00
+ .00 -1.51 .00
+ .00 6.59 1.00
+ .00 19.28 1.00
+ .00 5.94 .00
+ .00 8.21 1.00
+ .00 8.11 1.00
+ .00 2.49 .00
+ .00 9.62 .00
+ 1.00 -20.74 1.00
+ .00 -1.41 1.00
+ .00 15.15 1.00
+ .00 9.39 .00
+ 1.00 -15.14 1.00
+ 1.00 -5.86 .00
+ 1.00 -11.64 1.00
+ 1.00 -14.36 .00
+ 1.00 -8.95 1.00
+ 1.00 -16.42 1.00
+ 1.00 -1.04 1.00
+ .00 12.89 1.00
+ .00 -7.08 1.00
+ .00 4.87 1.00
+ .00 11.53 1.00
+ 1.00 -6.24 1.00
+ .00 1.25 1.00
+ .00 4.39 1.00
+ .00 3.17 .00
+ .00 19.39 1.00
+ .00 13.03 1.00
+ .00 2.43 .00
+ 1.00 -14.73 1.00
+ .00 8.25 1.00
+ 1.00 -13.28 1.00
+ .00 5.27 1.00
+ 1.00 -3.46 1.00
+ .00 13.81 1.00
+ .00 1.35 1.00
+ 1.00 -3.94 1.00
+ .00 20.73 1.00
+ 1.00 -15.40 .00
+ 1.00 -11.01 1.00
+ .00 4.56 .00
+ 1.00 -15.35 1.00
+ .00 15.21 .00
+ .00 5.34 1.00
+ 1.00 -21.55 1.00
+ .00 10.12 1.00
+ .00 -.73 1.00
+ .00 15.28 1.00
+ .00 11.08 1.00
+ 1.00 -8.24 .00
+ .00 2.46 .00
+ .00 9.60 .00
+ .00 11.24 .00
+ .00 14.13 1.00
+ .00 19.72 1.00
+ .00 5.58 .00
+ .00 26.23 1.00
+ .00 7.25 .00
+ 1.00 -.79 .00
+ .00 6.24 .00
+ 1.00 1.16 .00
+ 1.00 -7.89 1.00
+ 1.00 -1.86 1.00
+ 1.00 -10.80 1.00
+ 1.00 -5.51 .00
+ .00 7.51 .00
+ .00 11.18 .00
+ .00 8.73 .00
+ 1.00 -11.21 1.00
+ 1.00 -13.24 .00
+ .00 19.34 .00
+ .00 9.32 1.00
+ .00 17.97 1.00
+ 1.00 -1.56 1.00
+ 1.00 -3.13 .00
+ .00 3.98 .00
+ .00 -1.21 1.00
+ .00 2.37 .00
+ 1.00 -18.03 1.00
+])
+
+AT_DATA([miss.sps], [dnl
+data list notable file='data.txt' list /y x1 cat0*.
+
+logistic regression y with x1 cat0
+ /categorical = cat0.
+])
+
+AT_CHECK([pspp -O format=csv miss.sps > file1], [0], [ignore])
+
+dnl Append a case with a missing categorical.
+AT_CHECK([echo '1 34 .' >> data.txt], [0], [ignore])
+
+AT_CHECK([pspp -O format=csv miss.sps > file2], [0], [ignore])
+
+AT_CHECK([diff file1 file2], [1], [dnl
+8,10c8,10
+< Included in Analysis,100,100.0%
+< Missing Cases,0,.0%
+< Total,100,100.0%
+---
+> Included in Analysis,100,99.0%
+> Missing Cases,1,1.0%
+> Total,101,100.0%
+])
+
+AT_CLEANUP
+
+
+dnl Check that the confidence intervals are properly reported.
+dnl Use an example with categoricals, because that was buggy at
+dnl one point. The data in this example comes from:
+dnl http://people.ysu.edu/~gchang/SPSSE/SPSS_lab2Regression.pdf
+AT_SETUP([LOGISTIC REGRESSION confidence interval])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([ci.sps], [dnl
+set FORMAT=F20.3
+data list notable list /disease age sciostat sector savings *.
+begin data.
+0 33 1 1 1
+0 35 1 1 1
+0 6 1 1 0
+0 60 1 1 1
+1 18 3 1 0
+0 26 3 1 0
+0 6 3 1 0
+1 31 2 1 1
+1 26 2 1 0
+0 37 2 1 0
+0 23 1 1 0
+0 23 1 1 0
+0 27 1 1 1
+1 9 1 1 1
+1 37 1 2 1
+1 22 1 2 1
+1 67 1 2 1
+0 8 1 2 1
+1 6 1 2 1
+1 15 1 2 1
+1 21 2 2 1
+1 32 2 2 1
+1 16 1 2 1
+0 11 2 2 0
+0 14 3 2 0
+0 9 2 2 0
+0 18 2 2 0
+0 2 3 1 0
+0 61 3 1 1
+0 20 3 1 0
+0 16 3 1 0
+0 9 2 1 0
+0 35 2 1 1
+0 4 1 1 1
+0 44 3 2 0
+1 11 3 2 0
+0 3 2 2 1
+0 6 3 2 0
+1 17 2 2 0
+0 1 3 2 1
+1 53 2 2 1
+1 13 1 2 0
+0 24 1 2 0
+1 70 1 2 1
+1 16 3 2 1
+0 12 2 2 1
+1 20 3 2 1
+0 65 3 2 1
+1 40 2 2 0
+1 38 2 2 1
+1 68 2 2 1
+1 74 1 2 1
+1 14 1 2 1
+1 27 1 2 1
+0 31 1 2 1
+0 18 1 2 1
+0 39 1 2 0
+0 50 1 2 1
+0 31 1 2 1
+0 61 1 2 1
+0 18 3 1 0
+0 5 3 1 0
+0 2 3 1 1
+0 16 3 1 0
+1 59 3 1 1
+0 22 3 1 0
+0 24 1 1 1
+0 30 1 1 1
+0 46 1 1 1
+0 28 1 1 0
+0 27 1 1 1
+1 27 1 1 0
+0 28 1 1 1
+1 52 1 1 1
+0 11 3 1 1
+0 6 2 1 1
+0 46 3 1 0
+1 20 2 1 1
+0 3 1 1 1
+0 18 2 1 0
+0 25 2 1 0
+0 6 3 1 1
+1 65 3 1 1
+0 51 3 1 1
+0 39 2 1 1
+0 8 1 1 1
+0 8 2 1 0
+0 14 3 1 0
+0 6 3 1 0
+0 6 3 1 1
+0 7 3 1 0
+0 4 3 1 0
+0 8 3 1 0
+0 9 2 1 0
+1 32 3 1 0
+0 19 3 1 0
+0 11 3 1 0
+0 35 3 1 0
+0 16 1 1 0
+0 1 1 1 1
+0 6 1 1 1
+0 27 1 1 1
+0 25 1 1 1
+0 18 1 1 0
+0 37 3 1 0
+1 33 3 1 0
+0 27 2 1 0
+0 2 1 1 0
+0 8 2 1 0
+0 5 1 1 0
+0 1 1 1 1
+0 32 1 1 0
+1 25 1 1 1
+0 15 1 2 0
+0 15 1 2 1
+0 26 1 2 1
+1 42 1 2 1
+0 7 1 2 1
+0 2 1 2 0
+1 65 1 2 1
+0 33 2 2 1
+1 8 2 2 0
+0 30 2 2 0
+0 5 3 2 0
+0 15 3 2 0
+1 60 3 2 1
+1 13 3 2 1
+0 70 3 1 1
+0 5 3 1 0
+0 3 3 1 1
+0 50 2 1 1
+0 6 2 1 0
+0 12 2 1 1
+1 39 3 2 0
+0 15 2 2 1
+1 35 2 2 0
+0 2 2 2 1
+0 17 3 2 0
+1 43 3 2 1
+0 30 2 2 1
+0 11 1 2 1
+1 39 1 2 1
+0 32 1 2 1
+0 17 1 2 1
+0 3 3 2 1
+0 7 3 2 0
+0 2 2 2 0
+1 64 2 2 1
+1 13 1 2 2
+1 15 2 2 1
+0 48 2 2 1
+0 23 1 2 1
+1 48 1 2 0
+0 25 1 2 1
+0 12 1 2 1
+1 46 1 2 1
+0 79 1 2 1
+0 56 1 2 1
+0 8 1 2 1
+1 29 3 1 0
+1 35 3 1 0
+1 11 3 1 0
+0 69 3 1 1
+1 21 3 1 0
+0 13 3 1 0
+0 21 1 1 1
+1 32 1 1 1
+1 24 1 1 0
+0 24 1 1 1
+0 73 1 1 1
+0 42 1 1 1
+1 34 1 1 1
+0 30 2 1 0
+0 7 2 1 0
+1 29 3 1 0
+1 22 3 1 0
+0 38 2 1 1
+0 13 2 1 1
+0 12 2 1 1
+0 42 3 1 0
+1 17 3 1 0
+0 21 3 1 1
+0 34 1 1 1
+0 1 3 1 0
+0 14 2 1 0
+0 16 2 1 0
+0 9 3 1 0
+0 53 3 1 0
+0 27 3 1 0
+0 15 3 1 0
+0 9 3 1 0
+0 4 2 1 1
+0 10 3 1 1
+0 31 3 1 0
+0 85 3 1 1
+0 24 2 1 0
+end data.
+
+logistic regression
+ disease WITH age sciostat sector savings
+ /categorical = sciostat sector
+ /print = ci(95).
+])
+
+AT_CHECK([pspp -O format=csv ci.sps], [0], [dnl
+Table: Dependent Variable Encoding
+Original Value,Internal Value
+.000,.000
+1.000,1.000
+
+Table: Case Processing Summary
+Unweighted Cases,N,Percent
+Included in Analysis,196,100.0%
+Missing Cases,0,.0%
+Total,196,100.0%
+
+note: Estimation terminated at iteration number 4 because parameter estimates changed by less than 0.001
+
+Table: Model Summary
+Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
+1,211.195,.120,.172
+
+Table: Categorical Variables' Codings
+,,Frequency,Parameter coding,
+,,,(1),(2)
+sciostat,1.000,77,1,0
+,2.000,49,0,1
+,3.000,70,0,0
+sector,1.000,117,1,
+,2.000,79,0,
+
+Table: Classification Table
+,Observed,,Predicted,,
+,,,disease,,Percentage Correct
+,,,.000,1.000,
+Step 1,disease,.000,131,8,94.2%
+,,1.000,41,16,28.1%
+,Overall Percentage,,,,75.0%
+
+Table: Variables in the Equation
+,,B,S.E.,Wald,df,Sig.,Exp(B),95% CI for Exp(B),
+,,,,,,,,Lower,Upper
+Step 1,age,.027,.009,8.647,1,.003,1.027,1.009,1.045
+,savings,.061,.386,.025,1,.874,1.063,.499,2.264
+,sciostat,,,.440,2,.803,,,
+,sciostat(1),-.278,.434,.409,1,.522,.757,.323,1.775
+,sciostat(2),-.219,.459,.227,1,.634,.803,.327,1.976
+,sector,,,11.974,1,.001,,,
+,sector(1),-1.235,.357,11.974,1,.001,.291,.145,.586
+,Constant,-.814,.452,3.246,1,.072,.443,,
+])
+
+AT_CLEANUP
+
+AT_SETUP([LOGISTIC REGRESSION syntax errors])
+AT_DATA([logistic.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+LOGISTIC REGRESSION **.
+LOGISTIC REGRESSION x **.
+LOGISTIC REGRESSION x WITH **.
+LOGISTIC REGRESSION x WITH y/MISSING=**.
+LOGISTIC REGRESSION x WITH y/CATEGORICAL=**.
+LOGISTIC REGRESSION x WITH y/PRINT=CI **.
+LOGISTIC REGRESSION x WITH y/PRINT=CI(**).
+LOGISTIC REGRESSION x WITH y/PRINT=CI(123 **).
+LOGISTIC REGRESSION x WITH y/PRINT=**.
+LOGISTIC REGRESSION x WITH y/CRITERIA=BCON **.
+LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(**).
+LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(123 **).
+LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE **.
+LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(**).
+LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(123 **).
+LOGISTIC REGRESSION x WITH y/CRITERIA=LCON **.
+LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(**).
+LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(123 **).
+LOGISTIC REGRESSION x WITH y/CRITERIA=EPS **.
+LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(**).
+LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(123 **).
+LOGISTIC REGRESSION x WITH y/CRITERIA=CUT **.
+LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(**).
+LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(0.5 **).
+LOGISTIC REGRESSION x WITH y/CRITERIA=**.
+])
+AT_CHECK([pspp -O format=csv logistic.sps], [1], [dnl
+"logistic.sps:2.21-2.22: error: LOGISTIC REGRESSION: Syntax error expecting variable name.
+ 2 | LOGISTIC REGRESSION **.
+ | ^~"
+
+"logistic.sps:3.23-3.24: error: LOGISTIC REGRESSION: Syntax error expecting `WITH'.
+ 3 | LOGISTIC REGRESSION x **.
+ | ^~"
+
+"logistic.sps:4.28-4.29: error: LOGISTIC REGRESSION: Syntax error expecting variable name.
+ 4 | LOGISTIC REGRESSION x WITH **.
+ | ^~"
+
+"logistic.sps:5.38-5.39: error: LOGISTIC REGRESSION: Syntax error expecting INCLUDE or EXCLUDE.
+ 5 | LOGISTIC REGRESSION x WITH y/MISSING=**.
+ | ^~"
+
+"logistic.sps:6.42-6.43: error: LOGISTIC REGRESSION: Syntax error expecting variable name.
+ 6 | LOGISTIC REGRESSION x WITH y/CATEGORICAL=**.
+ | ^~"
+
+"logistic.sps:7.39-7.40: error: LOGISTIC REGRESSION: Syntax error expecting `('.
+ 7 | LOGISTIC REGRESSION x WITH y/PRINT=CI **.
+ | ^~"
+
+"logistic.sps:8.39-8.40: error: LOGISTIC REGRESSION: Syntax error expecting number.
+ 8 | LOGISTIC REGRESSION x WITH y/PRINT=CI(**).
+ | ^~"
+
+"logistic.sps:9.43-9.44: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
+ 9 | LOGISTIC REGRESSION x WITH y/PRINT=CI(123 **).
+ | ^~"
+
+"logistic.sps:10.36-10.37: error: LOGISTIC REGRESSION: Syntax error expecting DEFAULT, SUMMARY, CI, or ALL.
+ 10 | LOGISTIC REGRESSION x WITH y/PRINT=**.
+ | ^~"
+
+"logistic.sps:11.44-11.45: error: LOGISTIC REGRESSION: Syntax error expecting `('.
+ 11 | LOGISTIC REGRESSION x WITH y/CRITERIA=BCON **.
+ | ^~"
+
+"logistic.sps:12.44-12.45: error: LOGISTIC REGRESSION: Syntax error expecting number.
+ 12 | LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(**).
+ | ^~"
+
+"logistic.sps:13.48-13.49: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
+ 13 | LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(123 **).
+ | ^~"
+
+"logistic.sps:14.47-14.48: error: LOGISTIC REGRESSION: Syntax error expecting `('.
+ 14 | LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE **.
+ | ^~"
+
+"logistic.sps:15.47-15.48: error: LOGISTIC REGRESSION: Syntax error expecting non-negative integer for ITERATE.
+ 15 | LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(**).
+ | ^~"
+
+"logistic.sps:16.51-16.52: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
+ 16 | LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(123 **).
+ | ^~"
+
+"logistic.sps:17.44-17.45: error: LOGISTIC REGRESSION: Syntax error expecting `('.
+ 17 | LOGISTIC REGRESSION x WITH y/CRITERIA=LCON **.
+ | ^~"
+
+"logistic.sps:18.44-18.45: error: LOGISTIC REGRESSION: Syntax error expecting number.
+ 18 | LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(**).
+ | ^~"
+
+"logistic.sps:19.48-19.49: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
+ 19 | LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(123 **).
+ | ^~"
+
+"logistic.sps:20.43-20.44: error: LOGISTIC REGRESSION: Syntax error expecting `('.
+ 20 | LOGISTIC REGRESSION x WITH y/CRITERIA=EPS **.
+ | ^~"
+
+"logistic.sps:21.43-21.44: error: LOGISTIC REGRESSION: Syntax error expecting number.
+ 21 | LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(**).
+ | ^~"
+
+"logistic.sps:22.47-22.48: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
+ 22 | LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(123 **).
+ | ^~"
+
+"logistic.sps:23.43-23.44: error: LOGISTIC REGRESSION: Syntax error expecting `('.
+ 23 | LOGISTIC REGRESSION x WITH y/CRITERIA=CUT **.
+ | ^~"
+
+"logistic.sps:24.43-24.44: error: LOGISTIC REGRESSION: Syntax error expecting number between 0 and 1 for CUT.
+ 24 | LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(**).
+ | ^~"
+
+"logistic.sps:25.47-25.48: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
+ 25 | LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(0.5 **).
+ | ^~"
+
+"logistic.sps:26.39-26.40: error: LOGISTIC REGRESSION: Syntax error expecting BCON, ITERATE, LCON, EPS, or CUT.
+ 26 | LOGISTIC REGRESSION x WITH y/CRITERIA=**.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([LOOP])
+
+m4_define([LOOP_DATA], [dnl
+data list notable /x 1 y 2 z 3.
+begin data.
+121
+252
+393
+404
+end data.
+])
+
+AT_SETUP([LOOP with index])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+loop #i=x to y by z.
+print /#i.
+end loop.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1.00 @&t@
+2.00 @&t@
+--------
+2.00 @&t@
+4.00 @&t@
+--------
+3.00 @&t@
+6.00 @&t@
+9.00 @&t@
+--------
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with IF condition])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+compute #j=x.
+loop if #j <= y.
+print /#j.
+compute #j = #j + z.
+end loop.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1.00 @&t@
+2.00 @&t@
+--------
+2.00 @&t@
+4.00 @&t@
+--------
+3.00 @&t@
+6.00 @&t@
+9.00 @&t@
+--------
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with END IF condition])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+compute #k=x.
+loop.
+print /#k.
+compute #k = #k + z.
+end loop if #k > y.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1.00 @&t@
+2.00 @&t@
+--------
+2.00 @&t@
+4.00 @&t@
+--------
+3.00 @&t@
+6.00 @&t@
+9.00 @&t@
+--------
+4.00 @&t@
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with index and IF based on index])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+loop #m=x to y by z if #m < 4.
+print /#m.
+end loop.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1.00 @&t@
+2.00 @&t@
+--------
+2.00 @&t@
+--------
+3.00 @&t@
+--------
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with index and END IF based on index])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+loop #n=x to y by z.
+print /#n.
+end loop if #n >= 4.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1.00 @&t@
+2.00 @&t@
+--------
+2.00 @&t@
+4.00 @&t@
+--------
+3.00 @&t@
+6.00 @&t@
+--------
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with index and IF and END IF based on index])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+loop #o=x to y by z if mod(#o,2) = 0.
+print /#o.
+end loop if #o >= 4.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+--------
+2.00 @&t@
+4.00 @&t@
+--------
+--------
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with no conditions containing BREAK])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+compute #p = x.
+loop.
+print /#p.
+compute #p = #p + z.
+do if #p >= y.
+break.
+end if.
+end loop.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1.00 @&t@
+--------
+2.00 @&t@
+4.00 @&t@
+--------
+3.00 @&t@
+6.00 @&t@
+--------
+4.00 @&t@
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with no conditions that ends due to MXLOOPS])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+set mxloops=2.
+loop.
+compute #p = #p + 1.
+print /x #p.
+end loop.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1 1.00 @&t@
+1 2.00 @&t@
+--------
+2 3.00 @&t@
+2 4.00 @&t@
+--------
+3 5.00 @&t@
+3 6.00 @&t@
+--------
+4 7.00 @&t@
+4 8.00 @&t@
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP with IF condition that ends due to MXLOOPS])
+AT_DATA([loop.sps], [dnl
+LOOP_DATA
+set mxloops=3.
+compute #p = x.
+loop.
+print /x #p.
+compute #p = #p + 1.
+end loop if #p >= 6.
+print/'--------'.
+execute.
+])
+AT_CHECK([pspp -o pspp.csv loop.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+1 1.00 @&t@
+1 2.00 @&t@
+1 3.00 @&t@
+--------
+2 2.00 @&t@
+2 3.00 @&t@
+2 4.00 @&t@
+--------
+3 3.00 @&t@
+3 4.00 @&t@
+3 5.00 @&t@
+--------
+4 4.00 @&t@
+4 5.00 @&t@
+--------
+])
+AT_CLEANUP
+
+AT_SETUP([LOOP negative])
+AT_DATA([loop.sps], [dnl
+DATA LIST NOTABLE /x 1.
+BREAK.
+END LOOP.
+
+LOOP A=1 TO 5 B=1 TO 5.
+END LOOP.
+LOOP 5.
+END LOOP.
+LOOP B !.
+END LOOP.
+LOOP B=!.
+END LOOP.
+LOOP A=1 TO !.
+END LOOP.
+LOOP A=1 BY !.
+END LOOP.
+LOOP A=1 TO 5 BY 5 TO !.
+END LOOP.
+LOOP A=1 BY 5 TO 10 BY !.
+END LOOP.
+LOOP A=1.
+END LOOP.
+LOOP !.
+END LOOP.
+
+LOOP IF 1 IF 0.
+END LOOP.
+
+LOOP IF !.
+END LOOP.
+
+LOOP.
+END LOOP IF !.
+
+LOOP.
+END LOOP !.
+
+LOOP.
+])
+AT_CHECK([pspp loop.sps], 1, [dnl
+loop.sps:2.1-2.5: error: BREAK: This command cannot appear outside LOOP...END
+LOOP.
+ 2 | BREAK.
+ | ^~~~~
+
+loop.sps:3.1-3.8: error: END LOOP: This command cannot appear outside LOOP...
+END LOOP.
+ 3 | END LOOP.
+ | ^~~~~~~~
+
+loop.sps:5.15: error: LOOP: Only one index clause may be specified.
+ 5 | LOOP A=1 TO 5 B=1 TO 5.
+ | ^
+
+loop.sps:7.6: error: LOOP: Syntax error expecting identifier.
+ 7 | LOOP 5.
+ | ^
+
+loop.sps:9.8: error: LOOP: Syntax error expecting `='.
+ 9 | LOOP B !.
+ | ^
+
+loop.sps:11.8: error: LOOP: Syntax error parsing expression.
+ 11 | LOOP B=!.
+ | ^
+
+loop.sps:13.13: error: LOOP: Syntax error parsing expression.
+ 13 | LOOP A=1 TO !.
+ | ^
+
+loop.sps:15.13: error: LOOP: Syntax error parsing expression.
+ 15 | LOOP A=1 BY !.
+ | ^
+
+loop.sps:17.20-17.21: error: LOOP: Subcommand TO may only be specified once.
+ 17 | LOOP A=1 TO 5 BY 5 TO !.
+ | ^~
+
+loop.sps:19.21-19.22: error: LOOP: Subcommand BY may only be specified once.
+ 19 | LOOP A=1 BY 5 TO 10 BY !.
+ | ^~
+
+loop.sps:21.1-21.9: error: LOOP: Required subcommand TO was not specified.
+ 21 | LOOP A=1.
+ | ^~~~~~~~~
+
+loop.sps:23.6: error: LOOP: Syntax error expecting identifier.
+ 23 | LOOP !.
+ | ^
+
+loop.sps:26.11-26.12: error: LOOP: Subcommand IF may only be specified once.
+ 26 | LOOP IF 1 IF 0.
+ | ^~
+
+loop.sps:29.9: error: LOOP: Syntax error parsing expression.
+ 29 | LOOP IF !.
+ | ^
+
+loop.sps:33.13: error: LOOP: Syntax error parsing expression.
+ 33 | END LOOP IF !.
+ | ^
+
+loop.sps:36.10: error: LOOP: Syntax error expecting end of command.
+ 36 | END LOOP !.
+ | ^
+
+error: LOOP: At end of input: Syntax error expecting END LOOP.
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([MATCH FILES])
+
+m4_define([PREPARE_MATCH_FILES],
+ [AT_DATA([data1.txt], [dnl
+1aB
+8aM
+3aE
+5aG
+0aA
+5aH
+6aI
+7aJ
+2aD
+7aK
+1aC
+7aL
+4aF
+])
+
+ AT_DATA([data2.txt], [dnl
+1bN
+3bO
+4bP
+6bQ
+7bR
+9bS
+])
+
+ AT_DATA([prepare.sps], [dnl
+DATA LIST NOTABLE FILE='data1.txt' /a b c 1-3 (A).
+SAVE OUTFILE='data1.sav'.
+DATA LIST NOTABLE FILE='data2.txt' /a b c 1-3 (A).
+SAVE OUTFILE='data2.sav'.
+])
+ AT_CHECK([pspp -O format=csv prepare.sps])
+ AT_CHECK([test -f data1.sav && test -f data2.sav])])
+
+dnl CHECK_MATCH_FILES(TYPE2, SOURCE1, SOURCE2)
+dnl
+dnl Checks the MATCH FILES procedure with the specified combination of:
+dnl
+dnl - TYPE2: Either "file" or "table" for the type of matching used for
+dnl the second data source. (The first data source is always "file").
+dnl
+dnl - SOURCE1: Either "system" or "active" for the source of data for
+dnl the first data source.
+dnl
+dnl - SOURCE2: Either "system" or "active" for the source of data for
+dnl the second data source. (SOURCE1 and SOURCE2 may not both be
+dnl "active".)
+m4_define([CHECK_MATCH_FILES],
+ [AT_SETUP([MATCH FILES -- $2 file and $3 $1])
+ PREPARE_MATCH_FILES
+ AT_DATA([expout],
+ [m4_if([$1], [file], [dnl
+Table: Data List
+a,b,c,d,ina,inb,first,last
+0,a,A,,1,0,1,1
+1,a,B,N,1,1,1,0
+1,a,C,,1,0,0,1
+2,a,D,,1,0,1,1
+3,a,E,O,1,1,1,1
+4,a,F,P,1,1,1,1
+5,a,G,,1,0,1,0
+5,a,H,,1,0,0,1
+6,a,I,Q,1,1,1,1
+7,a,J,R,1,1,1,0
+7,a,K,,1,0,0,0
+7,a,L,,1,0,0,1
+8,a,M,,1,0,1,1
+9,b,,S,0,1,1,1
+], [dnl
+Table: Data List
+a,b,c,d,ina,inb,first,last
+0,a,A,,1,0,1,1
+1,a,B,N,1,1,1,0
+1,a,C,N,1,1,0,1
+2,a,D,,1,0,1,1
+3,a,E,O,1,1,1,1
+4,a,F,P,1,1,1,1
+5,a,G,,1,0,1,0
+5,a,H,,1,0,0,1
+6,a,I,Q,1,1,1,1
+7,a,J,R,1,1,1,0
+7,a,K,R,1,1,0,0
+7,a,L,R,1,1,0,1
+8,a,M,,1,0,1,1
+])])
+
+ AT_DATA([match-files.sps], [dnl
+m4_if([$2], [active], [GET FILE='data1.sav'.],
+ [$3], [active], [GET FILE='data2.sav'.],
+ [])
+MATCH FILES
+ FILE=m4_if([$2], [active], [*], ['data1.sav']) /IN=ina /SORT
+ $1=m4_if([$3], [active], [*], ['data2.sav']) /in=inb /rename c=d
+ /BY a /FIRST=first /LAST=last.
+LIST.
+])
+ AT_CHECK([pspp -o pspp.csv match-files.sps])
+ AT_CHECK([cat pspp.csv], [0], [expout])
+ AT_CLEANUP])
+
+CHECK_MATCH_FILES([file], [system], [system])
+CHECK_MATCH_FILES([file], [system], [active])
+CHECK_MATCH_FILES([file], [active], [system])
+CHECK_MATCH_FILES([table], [system], [system])
+CHECK_MATCH_FILES([table], [system], [active])
+CHECK_MATCH_FILES([table], [active], [system])
+
+AT_SETUP([MATCH FILES parallel match])
+PREPARE_MATCH_FILES
+AT_DATA([match-files.sps], [dnl
+MATCH FILES FILE='data1.sav' /FILE='data2.sav' /RENAME (a b c=d e f).
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv match-files.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+a,b,c,d,e,f
+1,a,B,1,b,N
+8,a,M,3,b,O
+3,a,E,4,b,P
+5,a,G,6,b,Q
+0,a,A,7,b,R
+5,a,H,9,b,S
+6,a,I,,,
+7,a,J,,,
+2,a,D,,,
+7,a,K,,,
+1,a,C,,,
+7,a,L,,,
+4,a,F,,,
+])
+AT_CLEANUP
+
+dnl Test bug handling TABLE from active dataset found by John Darrington.
+AT_SETUP([MATCH FILES bug with TABLE from active dataset])
+AT_DATA([match-files.sps], [dnl
+DATA LIST LIST NOTABLE /x * y *.
+BEGIN DATA
+3 30
+2 21
+1 22
+END DATA.
+
+SAVE OUTFILE='bar.sav'.
+
+DATA LIST LIST NOTABLE /x * z *.
+BEGIN DATA
+3 8
+2 9
+END DATA.
+
+MATCH FILES TABLE=* /FILE='bar.sav' /BY=x.
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv match-files.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+x,z,y
+3.00,8.00,30.00
+2.00,. ,21.00
+1.00,. ,22.00
+])
+AT_CLEANUP
+
+dnl Tests for a bug which caused MATCH FILES to crash
+dnl when used with scratch variables.
+AT_SETUP([MATCH FILES bug with scratch variables])
+AT_DATA([match-files.sps], [dnl
+DATA LIST LIST /w * x * y * .
+BEGIN DATA
+4 5 6
+1 2 3
+END DATA.
+
+COMPUTE j=0.
+LOOP #k = 1 to 10.
+COMPUTE j=#k + j.
+END LOOP.
+
+MATCH FILES FILE=* /DROP=w.
+LIST.
+FINISH.
+])
+AT_CHECK([pspp -o pspp.csv match-files.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+w,F8.0
+x,F8.0
+y,F8.0
+
+Table: Data List
+x,y,j
+5.00,6.00,55.00
+2.00,3.00,55.00
+])
+AT_CLEANUP
+
+dnl Tests for a bug that caused MATCH FILES to crash
+dnl with incompatible variables, especially but not
+dnl exclusively when one variable came from the active
+dnl file.
+AT_SETUP([MATCH FILES with incompatible variable types])
+AT_DATA([match-files.sps], [dnl
+DATA LIST LIST NOTABLE/name (A6) x.
+BEGIN DATA.
+al,7
+brad,8
+carl,9
+END DATA.
+SAVE OUTFILE='x.sav'.
+
+DATA LIST LIST NOTABLE/name (A7) y.
+BEGIN DATA.
+al,1
+carl,2
+dan,3
+END DATA.
+MATCH FILES/FILE='x.sav'/FILE=*/BY name.
+LIST.
+])
+AT_CHECK([pspp -O format=csv match-files.sps], [1], [dnl
+match-files.sps:15: error: MATCH FILES: Variable name has different type or width in different files.
+
+"match-files.sps:15.13-15.24: note: MATCH FILES: In file `x.sav', name is a string with width 6.
+ 15 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
+ | ^~~~~~~~~~~~"
+
+"match-files.sps:15.26-15.31: note: MATCH FILES: In file *, name is a string with width 7.
+ 15 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
+ | ^~~~~~"
+
+match-files.sps:16: error: Stopping syntax file processing here to avoid a cascade of dependent command failures.
+])
+AT_CLEANUP
+
+AT_SETUP([MATCH FILES syntax errors])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='match-files.sps' ERROR=IGNORE.
+])
+AT_DATA([match-files.sps], [dnl
+MATCH FILES/FILE=*.
+
+DATA LIST LIST NOTABLE/name (A6) x.
+BEGIN DATA.
+al,7
+brad,8
+carl,9
+END DATA.
+SAVE OUTFILE='x.sav'.
+
+TEMPORARY.
+MATCH FILES/FILE=*.
+
+DATA LIST LIST NOTABLE/name (A7) y.
+BEGIN DATA.
+al,1
+carl,2
+dan,3
+END DATA.
+MATCH FILES/FILE='x.sav'/FILE=*/BY name.
+MATCH FILES/FILE='x.sav'/IN=**.
+MATCH FILES/FILE='x.sav'/IN=x/IN=y.
+MATCH FILES/FILE='x.sav'/BY=x/BY=y.
+MATCH FILES/FILE='x.sav'/BY=**.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY y.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY x.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST x/FIRST y.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=**.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST x/LAST y.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=**.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=xyzzy.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=ALL.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/KEEP=xyzzy.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x **.
+MATCH FILES/FILE='x.sav'/TABLE=*/RENAME(name=name2).
+MATCH FILES/FILE='x.sav'/SORT/FILE=*/RENAME(name=name2)/SORT.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=x.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x.
+MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/IN=x.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"match-files.sps:1.18: error: MATCH FILES: Cannot specify the active dataset since none has been defined.
+ 1 | MATCH FILES/FILE=*.
+ | ^"
+
+"match-files.sps:12.18: error: MATCH FILES: This command may not be used after TEMPORARY when the active dataset is an input source. Temporary transformations will be made permanent.
+ 12 | MATCH FILES/FILE=*.
+ | ^"
+
+match-files.sps:20: error: MATCH FILES: Variable name has different type or width in different files.
+
+"match-files.sps:20.13-20.24: note: MATCH FILES: In file `x.sav', name is a string with width 6.
+ 20 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
+ | ^~~~~~~~~~~~"
+
+"match-files.sps:20.26-20.31: note: MATCH FILES: In file *, name is a string with width 7.
+ 20 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
+ | ^~~~~~"
+
+"match-files.sps:21.29-21.30: error: MATCH FILES: Syntax error expecting identifier.
+ 21 | MATCH FILES/FILE='x.sav'/IN=**.
+ | ^~"
+
+"match-files.sps:22.34: error: MATCH FILES: Multiple IN subcommands for a single FILE or TABLE.
+ 22 | MATCH FILES/FILE='x.sav'/IN=x/IN=y.
+ | ^"
+
+"match-files.sps:23.31-23.32: error: MATCH FILES: Subcommand BY may only be specified once.
+ 23 | MATCH FILES/FILE='x.sav'/BY=x/BY=y.
+ | ^~"
+
+"match-files.sps:24.29-24.30: error: MATCH FILES: Syntax error expecting variable name.
+ 24 | MATCH FILES/FILE='x.sav'/BY=**.
+ | ^~"
+
+"match-files.sps:25.13-25.24: error: MATCH FILES: File `x.sav' lacks BY variable y.
+ 25 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY y.
+ | ^~~~~~~~~~~~"
+
+"match-files.sps:26.26-26.31: error: MATCH FILES: File * lacks BY variable x.
+ 26 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY x.
+ | ^~~~~~"
+
+"match-files.sps:27.60-27.64: error: MATCH FILES: Subcommand FIRST may only be specified once.
+ 27 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST x/FIRST y.
+ | ^~~~~"
+
+"match-files.sps:28.58-28.59: error: MATCH FILES: Syntax error expecting identifier.
+ 28 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=**.
+ | ^~"
+
+"match-files.sps:29.59-29.62: error: MATCH FILES: Subcommand LAST may only be specified once.
+ 29 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST x/LAST y.
+ | ^~~~"
+
+"match-files.sps:30.57-30.58: error: MATCH FILES: Syntax error expecting identifier.
+ 30 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=**.
+ | ^~"
+
+"match-files.sps:31.57-31.61: error: MATCH FILES: xyzzy is not a variable name.
+ 31 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=xyzzy.
+ | ^~~~~"
+
+"match-files.sps:32.52-32.59: error: MATCH FILES: Cannot DROP all variables from dictionary.
+ 32 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=ALL.
+ | ^~~~~~~~"
+
+"match-files.sps:33.57-33.61: error: MATCH FILES: xyzzy is not a variable name.
+ 33 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/KEEP=xyzzy.
+ | ^~~~~"
+
+"match-files.sps:34.59-34.60: error: MATCH FILES: Syntax error expecting end of command.
+ 34 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x **.
+ | ^~"
+
+"match-files.sps:35.26-35.32: error: MATCH FILES: BY is required when TABLE is specified.
+ 35 | MATCH FILES/FILE='x.sav'/TABLE=*/RENAME(name=name2).
+ | ^~~~~~~"
+
+"match-files.sps:36.26-36.29: error: MATCH FILES: BY is required when SORT is specified.
+ 36 | MATCH FILES/FILE='x.sav'/SORT/FILE=*/RENAME(name=name2)/SORT.
+ | ^~~~"
+
+"match-files.sps:37.58: error: MATCH FILES: Variable name x specified on FIRST subcommand duplicates an existing variable name.
+ 37 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=x.
+ | ^"
+
+"match-files.sps:38.57: error: MATCH FILES: Variable name x specified on LAST subcommand duplicates an existing variable name.
+ 38 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x.
+ | ^"
+
+"match-files.sps:39.55: error: MATCH FILES: Variable name x specified on IN subcommand duplicates an existing variable name.
+ 39 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/IN=x.
+ | ^"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017, 2020 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([MATRIX DATA])
+
+dnl Keep this test in sync with Example 1 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - LOWER DIAGONAL with ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ var01 TO var08
+ /FILE='matrix-data.txt'.
+FORMATS var01 TO var08(F5.2).
+LIST.
+])
+AT_DATA([matrix-data.txt], [dnl
+MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+N 92 92 92 92 92 92 92 92
+'CORR' 1.00
+CORR .18 1.00
+CORR -.22 -.17 1.00
+"CORR" .36 .31 -.14 1.00
+COR .27 .16 -.12 .22 1.00
+CORR .33 .15 -.17 .24 .21 1.00
+CORR .50 .29 -.20 .32 .12 .38 1.00
+CORR .17 .29 -.05 .20 .27 .20 .04 1.00
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
+MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
+STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
+N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
+CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
+CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
+CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
+CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
+CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
+CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
+CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
+CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - UPPER DIAGONAL with ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var02 var03 var04
+ /format = upper diagonal.
+
+begin data
+mean 34 35 36 37
+sd 22 11 55 66
+n_ve 100 101 102 103
+corr 1 9 8 7
+corr 1 6 5
+corr 1 4
+corr 1
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+MEAN,,34.0000,35.0000,36.0000,37.0000
+STDDEV,,22.0000,11.0000,55.0000,66.0000
+N,,100.0000,101.0000,102.0000,103.0000
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var02,9.0000,1.0000,6.0000,5.0000
+CORR,var03,8.0000,6.0000,1.0000,4.0000
+CORR,var04,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - FULL with ROWTYPE_])
+dnl Just for fun, this one is in a different case.
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = ROWTYPE_ var01 var02 var03 var04
+ /format = full diagonal.
+
+begin data
+MEAN 34 35 36 37
+SD 22 11 55 66
+N 100 101 102 103
+CORR 1 9 8 7
+CORR 9 1 6 5
+CORR 8 6 1 4
+CORR 7 5 4 1
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+MEAN,,34.0000,35.0000,36.0000,37.0000
+STDDEV,,22.0000,11.0000,55.0000,66.0000
+N,,100.0000,101.0000,102.0000,103.0000
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var02,9.0000,1.0000,6.0000,5.0000
+CORR,var03,8.0000,6.0000,1.0000,4.0000
+CORR,var04,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+
+AT_SETUP([MATRIX DATA - UPPER NODIAGONAL with ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var02 var03 var04
+ /format = upper nodiagonal.
+
+begin data
+mean 34 35 36 37
+sd 22 11 55 66
+n 100 101 102 103
+corr 9 8 7
+corr 6 5
+corr 4
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+MEAN,,34.0000,35.0000,36.0000,37.0000
+STDDEV,,22.0000,11.0000,55.0000,66.0000
+N,,100.0000,101.0000,102.0000,103.0000
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var02,9.0000,1.0000,6.0000,5.0000
+CORR,var03,8.0000,6.0000,1.0000,4.0000
+CORR,var04,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 2 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - UPPER NODIAGONAL with ROWTYPE_ - 2])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ var01 TO var08
+ /FORMAT=UPPER NODIAGONAL.
+BEGIN DATA.
+MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+N 92 92 92 92 92 92 92 92
+CORR .17 .50 -.33 .27 .36 -.22 .18
+CORR .29 .29 -.20 .32 .12 .38
+CORR .05 .20 -.15 .16 .21
+CORR .20 .32 -.17 .12
+CORR .27 .12 -.24
+CORR -.20 -.38
+CORR .04
+END DATA.
+FORMATS var01 TO var08(F6.2).
+LIST.
+])
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
+MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
+STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
+N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
+CORR,var01,1.00,.17,.50,-.33,.27,.36,-.22,.18
+CORR,var02,.17,1.00,.29,.29,-.20,.32,.12,.38
+CORR,var03,.50,.29,1.00,.05,.20,-.15,.16,.21
+CORR,var04,-.33,.29,.05,1.00,.20,.32,-.17,.12
+CORR,var05,.27,-.20,.20,.20,1.00,.27,.12,-.24
+CORR,var06,.36,.32,-.15,.32,.27,1.00,-.20,-.38
+CORR,var07,-.22,.12,.16,-.17,.12,-.20,1.00,.04
+CORR,var08,.18,.38,.21,.12,-.24,-.38,.04,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - LOWER NODIAGONAL with ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var02 var03 var04
+ /format = lower nodiagonal
+ /cells = 2.
+
+begin data
+mean 34 35 36 37
+sd 22 11 55 66
+n 100 101 102 103
+corr 9
+corr 8 6
+corr 7 5 4
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+"matrix-data.sps:4.6-4.10: warning: MATRIX DATA: CELLS is ignored when VARIABLES includes ROWTYPE_.
+ 4 | /cells = 2.
+ | ^~~~~"
+
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+MEAN,,34.0000,35.0000,36.0000,37.0000
+STDDEV,,22.0000,11.0000,55.0000,66.0000
+N,,100.0000,101.0000,102.0000,103.0000
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var02,9.0000,1.0000,6.0000,5.0000
+CORR,var03,8.0000,6.0000,1.0000,4.0000
+CORR,var04,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - split data])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = s1 s2 rowtype_ var01 var02 var03
+ /split=s1 s2.
+
+begin data
+8 0 mean 21.4 5.0 72.9
+8 0 sd 6.5 1.6 22.8
+8 0 n 106 106 106
+8 0 corr 1
+8 0 corr .41 1
+8 0 corr -.16 -.22 1
+8 1 mean 11.4 1.0 52.9
+8 1 sd 9.5 8.6 12.8
+8 1 n 10 11 12
+8 1 corr 1
+8 1 corr .51 1
+8 1 corr .36 -.41 1
+end data.
+
+display dictionary.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+s1,1,Nominal,Input,8,Right,F4.0,F4.0
+s2,2,Nominal,Input,8,Right,F4.0,F4.0
+ROWTYPE_,3,Nominal,Input,8,Left,A8,A8
+VARNAME_,4,Nominal,Input,8,Left,A8,A8
+var01,5,Scale,Input,8,Right,F10.4,F10.4
+var02,6,Scale,Input,8,Right,F10.4,F10.4
+var03,7,Scale,Input,8,Right,F10.4,F10.4
+
+Table: Split Values
+Variable,Value
+s1,8
+s2,0
+
+Table: Data List
+s1,s2,ROWTYPE_,VARNAME_,var01,var02,var03
+8,0,MEAN,,21.4000,5.0000,72.9000
+8,0,STDDEV,,6.5000,1.6000,22.8000
+8,0,N,,106.0000,106.0000,106.0000
+8,0,CORR,var01,1.0000,.4100,-.1600
+8,0,CORR,var02,.4100,1.0000,-.2200
+8,0,CORR,var03,-.1600,-.2200,1.0000
+
+Table: Split Values
+Variable,Value
+s1,8
+s2,1
+
+Table: Data List
+s1,s2,ROWTYPE_,VARNAME_,var01,var02,var03
+8,1,MEAN,,11.4000,1.0000,52.9000
+8,1,STDDEV,,9.5000,8.6000,12.8000
+8,1,N,,10.0000,11.0000,12.0000
+8,1,CORR,var01,1.0000,.5100,.3600
+8,1,CORR,var02,.5100,1.0000,-.4100
+8,1,CORR,var03,.3600,-.4100,1.0000
+])
+
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 4 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - split data - 2])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=s1 ROWTYPE_ var01 TO var04
+ /SPLIT=s1
+ /FORMAT=FULL.
+BEGIN DATA.
+0 MEAN 34 35 36 37
+0 SD 22 11 55 66
+0 N 99 98 99 92
+0 CORR 1 .9 .8 .7
+0 CORR .9 1 .6 .5
+0 CORR .8 .6 1 .4
+0 CORR .7 .5 .4 1
+1 MEAN 44 45 34 39
+1 SD 23 15 51 46
+1 N 98 34 87 23
+1 CORR 1 .2 .3 .4
+1 CORR .2 1 .5 .6
+1 CORR .3 .5 1 .7
+1 CORR .4 .6 .7 1
+END DATA.
+FORMATS var01 TO var04(F5.1).
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Split Values
+Variable,Value
+s1,0
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+0,MEAN,,34.0,35.0,36.0,37.0
+0,STDDEV,,22.0,11.0,55.0,66.0
+0,N,,99.0,98.0,99.0,92.0
+0,CORR,var01,1.0,.9,.8,.7
+0,CORR,var02,.9,1.0,.6,.5
+0,CORR,var03,.8,.6,1.0,.4
+0,CORR,var04,.7,.5,.4,1.0
+
+Table: Split Values
+Variable,Value
+s1,1
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+1,MEAN,,44.0,45.0,34.0,39.0
+1,STDDEV,,23.0,15.0,51.0,46.0
+1,N,,98.0,34.0,87.0,23.0
+1,CORR,var01,1.0,.2,.3,.4
+1,CORR,var02,.2,1.0,.5,.6
+1,CORR,var03,.3,.5,1.0,.7
+1,CORR,var04,.4,.6,.7,1.0
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 5 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - factor variables])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ f1 var01 TO var04
+ /FACTOR=f1.
+BEGIN DATA.
+MEAN 0 34 35 36 37
+SD 0 22 11 55 66
+N 0 99 98 99 92
+MEAN 1 44 45 34 39
+SD 1 23 15 51 46
+N 1 98 34 87 23
+CORR . 1
+CORR . .9 1
+CORR . .8 .6 1
+CORR . .7 .5 .4 1
+END DATA.
+FORMATS var01 TO var04(F5.1).
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
+MEAN,0,,34.0,35.0,36.0,37.0
+STDDEV,0,,22.0,11.0,55.0,66.0
+N,0,,99.0,98.0,99.0,92.0
+MEAN,1,,44.0,45.0,34.0,39.0
+STDDEV,1,,23.0,15.0,51.0,46.0
+N,1,,98.0,34.0,87.0,23.0
+CORR,.,var01,1.0,.9,.8,.7
+CORR,.,var02,.9,1.0,.6,.5
+CORR,.,var03,.8,.6,1.0,.4
+CORR,.,var04,.7,.5,.4,1.0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - factors and splits])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = s f rowtype_ var01 var02 var03
+ /split=s
+ /factor=f.
+
+begin data
+8 0 mean 21.4 5.0 72.9
+8 0 sd 6.5 1.6 22.8
+8 0 n 106 106 106
+8 . corr 1
+8 . corr .41 1
+8 . corr -.16 -.22 1
+9 1 mean 11.4 1.0 52.9
+9 1 sd 9.5 8.6 12.8
+9 1 n 10 11 12
+9 . corr 1
+9 . corr .51 1
+9 . corr .36 -.41 1
+end data.
+
+display dictionary.
+
+list.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+s,1,Nominal,Input,8,Right,F4.0,F4.0
+ROWTYPE_,2,Nominal,Input,8,Left,A8,A8
+f,3,Nominal,Input,8,Right,F4.0,F4.0
+VARNAME_,4,Nominal,Input,8,Left,A8,A8
+var01,5,Scale,Input,8,Right,F10.4,F10.4
+var02,6,Scale,Input,8,Right,F10.4,F10.4
+var03,7,Scale,Input,8,Right,F10.4,F10.4
+
+Table: Split Values
+Variable,Value
+s,8
+
+Table: Data List
+s,ROWTYPE_,f,VARNAME_,var01,var02,var03
+8,MEAN,0,,21.4000,5.0000,72.9000
+8,STDDEV,0,,6.5000,1.6000,22.8000
+8,N,0,,106.0000,106.0000,106.0000
+8,CORR,.,var01,1.0000,.4100,-.1600
+8,CORR,.,var02,.4100,1.0000,-.2200
+8,CORR,.,var03,-.1600,-.2200,1.0000
+
+Table: Split Values
+Variable,Value
+s,9
+
+Table: Data List
+s,ROWTYPE_,f,VARNAME_,var01,var02,var03
+9,MEAN,1,,11.4000,1.0000,52.9000
+9,STDDEV,1,,9.5000,8.6000,12.8000
+9,N,1,,10.0000,11.0000,12.0000
+9,CORR,.,var01,1.0000,.5100,.3600
+9,CORR,.,var02,.5100,1.0000,-.4100
+9,CORR,.,var03,.3600,-.4100,1.0000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - bad ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var02 var03 var04
+ /format = upper diagonal.
+
+begin data
+cork 1 9 8 7
+corr 1 6 5
+corr 1 4
+corr 1
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
+"matrix-data.sps:6.1-6.4: error: Unknown row type ""cork""."
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - unexpected ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ f1 var01 var02 var03 var04
+ /content = corr (sd)
+ /factor = f1
+ /format = upper diagonal.
+
+begin data
+corr . 1 9 8 7
+corr . 1 6 5
+corr . 1 4
+corr . 1
+sd . 1 2 3 4
+
+corr 0 1 9 8 7
+corr 0 1 6 5
+corr 0 1 4
+corr 0 1
+sd 0 1 2 3 4
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+matrix-data.sps:12: warning: Data contains pooled row type STDDEV not included in CONTENTS.
+
+matrix-data.sps:14: warning: Data contains with-factors row type CORR not included in CONTENTS.
+
+Table: Data List
+ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
+CORR,.,var01,1.0000,9.0000,8.0000,7.0000
+CORR,.,var02,9.0000,1.0000,6.0000,5.0000
+CORR,.,var03,8.0000,6.0000,1.0000,4.0000
+CORR,.,var04,7.0000,5.0000,4.0000,1.0000
+STDDEV,.,,1.0000,2.0000,3.0000,4.0000
+CORR,0,var01,1.0000,9.0000,8.0000,7.0000
+CORR,0,var02,9.0000,1.0000,6.0000,5.0000
+CORR,0,var03,8.0000,6.0000,1.0000,4.0000
+CORR,0,var04,7.0000,5.0000,4.0000,1.0000
+STDDEV,0,,1.0000,2.0000,3.0000,4.0000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - bad number])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var02 var03 var04
+ /format = upper diagonal.
+
+begin data
+corr 1 9 8 7
+corr 1 x 5
+corr 1 4
+corr 1
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
+matrix-data.sps:7.15: error: Field contents are not numeric.
+
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var02,9.0000,1.0000,. ,5.0000
+CORR,var03,8.0000,. ,1.0000,4.0000
+CORR,var04,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - long variable names])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var_two variable_number_three variableFour
+ /format = upper diagonal.
+
+begin data
+mean 34 35 36 37
+sd 22 11 55 66
+n_vector 100 101 102 103
+corr 1 9 8 7
+corr 1 6 5
+corr 1 4
+corr 1
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var_two,variable_number_three,variableFour
+MEAN,,34.0000,35.0000,36.0000,37.0000
+STDDEV,,22.0000,11.0000,55.0000,66.0000
+N,,100.0000,101.0000,102.0000,103.0000
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var_two,9.0000,1.0000,6.0000,5.0000
+CORR,variable_number_three,8.0000,6.0000,1.0000,4.0000
+CORR,variableFour,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - read integrity])
+dnl Check that matrices presented are read correctly.
+dnl The example below is an unlikely one since all
+dnl covariance/correlation matrices must be symmetrical
+dnl but it serves a purpose for this test.
+AT_DATA([matrix-reader.sps], [dnl
+matrix data
+ variables = rowtype_ var01 to var9
+ /format = full.
+
+begin data
+n 1 2 3 4 5 6 7 8 9
+sd 100 200 300 400 500 600 700 800 900
+corr 11 12 13 14 15 16 17 18 19
+corr 21 22 23 24 25 26 27 28 29
+corr 31 32 33 34 35 36 37 38 39
+corr 41 42 43 44 45 46 47 48 49
+corr 51 52 53 54 55 56 57 58 59
+corr 61 62 63 64 65 66 67 68 69
+corr 71 72 73 74 75 76 77 78 79
+corr 81 82 83 84 85 86 87 88 89
+corr 91 92 93 94 95 96 97 98 99
+end data.
+DEBUG MATRIX READ.
+FORMATS var01 to var09(F3.0).
+list.
+factor /matrix = in (corr = *)
+ /analysis var02 var04 var06
+ /method = correlation
+ /rotation = norotate
+ /print correlation.
+])
+
+AT_CHECK([pspp --testing-mode -O format=csv matrix-reader.sps], [0], [dnl
+Table: Debug Matrix Reader
+,,,var01,var02,var03,var04,var05,var06,var07,var08,var09
+1,Correlation,var01,11.000,12.000,13.000,14.000,15.000,16.000,17.000,18.000,19.000
+,,var02,21.000,22.000,23.000,24.000,25.000,26.000,27.000,28.000,29.000
+,,var03,31.000,32.000,33.000,34.000,35.000,36.000,37.000,38.000,39.000
+,,var04,41.000,42.000,43.000,44.000,45.000,46.000,47.000,48.000,49.000
+,,var05,51.000,52.000,53.000,54.000,55.000,56.000,57.000,58.000,59.000
+,,var06,61.000,62.000,63.000,64.000,65.000,66.000,67.000,68.000,69.000
+,,var07,71.000,72.000,73.000,74.000,75.000,76.000,77.000,78.000,79.000
+,,var08,81.000,82.000,83.000,84.000,85.000,86.000,87.000,88.000,89.000
+,,var09,91.000,92.000,93.000,94.000,95.000,96.000,97.000,98.000,99.000
+,N,Value,1.000,2.000,3.000,4.000,5.000,6.000,7.000,8.000,9.000
+,Standard Deviation,Value,100.000,200.000,300.000,400.000,500.000,600.000,700.000,800.000,900.000
+
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08,var09
+N,,1,2,3,4,5,6,7,8,9
+STDDEV,,100,200,300,400,500,600,700,800,900
+CORR,var01,11,12,13,14,15,16,17,18,19
+CORR,var02,21,22,23,24,25,26,27,28,29
+CORR,var03,31,32,33,34,35,36,37,38,39
+CORR,var04,41,42,43,44,45,46,47,48,49
+CORR,var05,51,52,53,54,55,56,57,58,59
+CORR,var06,61,62,63,64,65,66,67,68,69
+CORR,var07,71,72,73,74,75,76,77,78,79
+CORR,var08,81,82,83,84,85,86,87,88,89
+CORR,var09,91,92,93,94,95,96,97,98,99
+
+Table: Correlation Matrix
+,,var02,var04,var06
+Correlation,var02,22.000,24.000,26.000
+,var04,42.000,44.000,46.000
+,var06,62.000,64.000,66.000
+
+Table: Component Matrix
+,Component,
+,1,2
+var02,6.73,-2.23
+var04,6.95,2.15
+var06,9.22,.01
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - too many rows])
+dnl Test for a crash which occurred when the matrix had more rows declared
+dnl than variables to hold them.
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_
+ var01 var02 var03 var04
+ / format = upper diagonal .
+begin data
+ mean 21.4 5.0 72.9 17.4
+ sd 6.5 1.6 22.8 5.7
+ n 106 106 106 106
+ corr 1.00 .32 .48 .28
+ corr 1.00 .72 .54 .44
+ corr 1.00 .50 .59 .64
+ corr 1.00 .62 .49 -.30
+ corr 1.00 .56 -.38 .52
+ corr 1.00 -.73 .91 .80
+ corr 1.00 -.65 -.60
+ corr 1.00 .70
+ corr 1.00
+end data .
+FORMATS var01 TO var04 (F6.2).
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
+matrix-data.sps:10.29-10.31: error: Extraneous data expecting end of line.
+
+matrix-data.sps:11.24-11.31: error: Extraneous data expecting end of line.
+
+matrix-data.sps:12.19-12.32: error: Extraneous data expecting end of line.
+
+matrix-data.sps:18: error: Matrix CORR had 9 rows but 4 rows were expected.
+
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+MEAN,,21.40,5.00,72.90,17.40
+STDDEV,,6.50,1.60,22.80,5.70
+N,,106.00,106.00,106.00,106.00
+CORR,var01,1.00,.32,.48,.28
+CORR,var02,.32,1.00,.72,.54
+CORR,var03,.48,.72,1.00,.50
+CORR,var04,.28,.54,.50,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - too few rows])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ s1 var01 var02 var03 var04
+ /split s1
+ /format = upper diagonal
+ /file='matrix-data.txt'.
+FORMATS var01 TO var04 (F6.2).
+LIST.
+])
+AT_DATA([matrix-data.txt], [dnl
+mean 1 21.4 5.0 72.9 17.4
+sd 1 6.5 1.6 22.8 5.7
+n 1 106 106 106 106
+corr 1 1.00 .32 .48 .28
+corr 2 1.00 .32 .48 .28
+corr 2 2.00 .72 .54
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
+matrix-data.txt:5: error: Matrix CORR had 1 rows but 4 rows were expected.
+
+matrix-data.txt:6: error: Matrix CORR had 2 rows but 4 rows were expected.
+
+Table: Split Values
+Variable,Value
+s1,1
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+1,MEAN,,21.40,5.00,72.90,17.40
+1,STDDEV,,6.50,1.60,22.80,5.70
+1,N,,106.00,106.00,106.00,106.00
+1,CORR,var01,1.00,.32,.48,.28
+1,CORR,var02,.32,1.00,. ,. @&t@
+1,CORR,var03,.48,. ,1.00,. @&t@
+1,CORR,var04,.28,. ,. ,1.00
+
+Table: Split Values
+Variable,Value
+s1,2
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+2,CORR,var01,1.00,.32,.48,.28
+2,CORR,var02,.32,2.00,.72,.54
+2,CORR,var03,.48,.72,1.00,. @&t@
+2,CORR,var04,.28,.54,. ,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - badly formed])
+AT_DATA([data.sps], [dnl
+data list list NOTABLE /ROWTYPE_ (a8) VARNAME_(a4) v1 v2 v3 v4xxxxxxxxxxxxxxxxxxxxxzzzzzzzzzzzzzxxxxxxxxx.
+begin data
+mean "" 1 2 3 4
+sd "" 5 6 7 8
+n "" 2 3 4 5
+corr v1 11 22 33 44
+corr v2 55 66 77 88
+corr v3 111 222 333 444
+corr v4 4 3 21 1
+end data.
+
+DEBUG MATRIX READ.
+])
+
+AT_CHECK([pspp --testing-mode -O format=csv data.sps], [0], [dnl
+data.sps:12: warning: DEBUG MATRIX READ: CORR matrix has 4 columns but 3 rows named variables to be analyzed (and 1 rows named unknown variables).
+
+Table: Debug Matrix Reader
+,,,v1,v2,v3,v4xxxxxxxxxxxxxxxxxxxxxzzzzzzzzzzzzzxxxxxxxxx
+1,Correlation,v1,11.000,22.000,33.000,44.000
+,,v2,55.000,66.000,77.000,88.000
+,,v3,111.000,222.000,333.000,444.000
+,,v4xxxxxxxxxxxxxxxxxxxxxzzzzzzzzzzzzzxxxxxxxxx,. ,. ,. ,. @&t@
+,N,Value,2.000,3.000,4.000,5.000
+,Mean,Value,1.000,2.000,3.000,4.000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - N subcommand])
+AT_DATA([matrix-data.sps], [dnl
+matrix data
+ variables = rowtype_ var01 var02 var03 var04
+ /n = 99
+ /format = upper nodiagonal.
+begin data
+mean 34 35 36 37
+sd 22 11 55 66
+n_vector 1 2 3 4
+corr 9 8 7
+corr 6 5
+corr 4
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
+matrix-data.sps:8: error: N record is not allowed with N subcommand. Ignoring N record.
+
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04
+N,,99.0000,99.0000,99.0000,99.0000
+MEAN,,34.0000,35.0000,36.0000,37.0000
+STDDEV,,22.0000,11.0000,55.0000,66.0000
+CORR,var01,1.0000,9.0000,8.0000,7.0000
+CORR,var02,9.0000,1.0000,6.0000,5.0000
+CORR,var03,8.0000,6.0000,1.0000,4.0000
+CORR,var04,7.0000,5.0000,4.0000,1.0000
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 3 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - N subcommand - 2])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ var01 TO var08
+ /FORMAT=UPPER NODIAGONAL
+ /N 92.
+BEGIN DATA.
+MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+CORR .17 .50 -.33 .27 .36 -.22 .18
+CORR .29 .29 -.20 .32 .12 .38
+CORR .05 .20 -.15 .16 .21
+CORR .20 .32 -.17 .12
+CORR .27 .12 -.24
+CORR -.20 -.38
+CORR .04
+END DATA.
+FORMATS var01 TO var08(F6.2).
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
+N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
+MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
+STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
+CORR,var01,1.00,.17,.50,-.33,.27,.36,-.22,.18
+CORR,var02,.17,1.00,.29,.29,-.20,.32,.12,.38
+CORR,var03,.50,.29,1.00,.05,.20,-.15,.16,.21
+CORR,var04,-.33,.29,.05,1.00,.20,.32,-.17,.12
+CORR,var05,.27,-.20,.20,.20,1.00,.27,.12,-.24
+CORR,var06,.36,.32,-.15,.32,.27,1.00,-.20,-.38
+CORR,var07,-.22,.12,.16,-.17,.12,-.20,1.00,.04
+CORR,var08,.18,.38,.21,.12,-.24,-.38,.04,1.00
+])
+AT_CLEANUP
+
+dnl A "no-crash" test. This was observed to cause problems.
+dnl See bug #58596
+AT_SETUP([MATRIX DATA - crash])
+
+AT_DATA([matrix-data.sps], [dnl
+begin data
+corr 31
+
+matrix data
+ var1
+begin data
+ corr 1.00
+end data .
+
+matrix data
+ variables = roxtype_ var01
+ /format = upper nodiagonal.
+begin data
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [ignore])
+AT_CLEANUP
+\f
+dnl Keep this test in sync with Example 6 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - LOWER DIAGONAL without ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=var01 TO var08
+ /CONTENTS=MEAN SD N CORR.
+BEGIN DATA.
+24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+ 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+ 92 92 92 92 92 92 92 92
+1.00
+ .18 1.00
+-.22 -.17 1.00
+ .36 .31 -.14 1.00
+ .27 .16 -.12 .22 1.00
+ .33 .15 -.17 .24 .21 1.00
+ .50 .29 -.20 .32 .12 .38 1.00
+ .17 .29 -.05 .20 .27 .20 .04 1.00
+END DATA.
+FORMATS var01 TO var08(F5.2).
+LIST.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
+MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
+STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
+N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
+CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
+CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
+CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
+CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
+CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
+CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
+CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
+CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - extraneous data without ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=var01 TO var08
+ /CONTENTS=MEAN SD N CORR.
+BEGIN DATA.
+24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+ 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+ 92 92 92 92 92 92 92 92
+1.00 .18
+ .18 1.00
+-.22 -.17 1.00
+ .36 .31 -.14 1.00
+ .27 .16 -.12 .22 1.00
+ .33 .15 -.17 .24 .21 1.00
+ .50 .29 -.20 .32 .12 .38 1.00
+ .17 .29 -.05 .20 .27 .20 .04 1.00
+END DATA.
+FORMATS var01 TO var08(F5.2).
+LIST.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [1], [dnl
+matrix-data.sps:8.8-8.10: error: Extraneous data expecting end of line.
+
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
+MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
+STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
+N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
+CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
+CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
+CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
+CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
+CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
+CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
+CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
+CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 7 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - Split variables with explicit values without ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=s1 var01 TO var04
+ /SPLIT=s1
+ /FORMAT=FULL
+ /CONTENTS=MEAN SD N CORR.
+BEGIN DATA.
+0 34 35 36 37
+0 22 11 55 66
+0 99 98 99 92
+0 1 .9 .8 .7
+0 .9 1 .6 .5
+0 .8 .6 1 .4
+0 .7 .5 .4 1
+1 44 45 34 39
+1 23 15 51 46
+1 98 34 87 23
+1 1 .2 .3 .4
+1 .2 1 .5 .6
+1 .3 .5 1 .7
+1 .4 .6 .7 1
+END DATA.
+FORMATS var01 TO var04(F5.2).
+LIST.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
+Table: Split Values
+Variable,Value
+s1,0
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+0,MEAN,,34.00,35.00,36.00,37.00
+0,STDDEV,,22.00,11.00,55.00,66.00
+0,N,,99.00,98.00,99.00,92.00
+0,CORR,var01,1.00,.90,.80,.70
+0,CORR,var02,.90,1.00,.60,.50
+0,CORR,var03,.80,.60,1.00,.40
+0,CORR,var04,.70,.50,.40,1.00
+
+Table: Split Values
+Variable,Value
+s1,1
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+1,MEAN,,44.00,45.00,34.00,39.00
+1,STDDEV,,23.00,15.00,51.00,46.00
+1,N,,98.00,34.00,87.00,23.00
+1,CORR,var01,1.00,.20,.30,.40
+1,CORR,var02,.20,1.00,.50,.60
+1,CORR,var03,.30,.50,1.00,.70
+1,CORR,var04,.40,.60,.70,1.00
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 8 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - Split variable with sequential values without ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=var01 TO var04
+ /SPLIT=s1
+ /FORMAT=FULL
+ /CONTENTS=MEAN SD N CORR.
+BEGIN DATA.
+34 35 36 37
+22 11 55 66
+99 98 99 92
+ 1 .9 .8 .7
+.9 1 .6 .5
+.8 .6 1 .4
+.7 .5 .4 1
+44 45 34 39
+23 15 51 46
+98 34 87 23
+ 1 .2 .3 .4
+.2 1 .5 .6
+.3 .5 1 .7
+.4 .6 .7 1
+END DATA.
+FORMATS var01 TO var04(F5.2).
+LIST.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
+Table: Split Values
+Variable,Value
+s1,1
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+1,MEAN,,34.00,35.00,36.00,37.00
+1,STDDEV,,22.00,11.00,55.00,66.00
+1,N,,99.00,98.00,99.00,92.00
+1,CORR,var01,1.00,.90,.80,.70
+1,CORR,var02,.90,1.00,.60,.50
+1,CORR,var03,.80,.60,1.00,.40
+1,CORR,var04,.70,.50,.40,1.00
+
+Table: Split Values
+Variable,Value
+s1,2
+
+Table: Data List
+s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
+2,MEAN,,44.00,45.00,34.00,39.00
+2,STDDEV,,23.00,15.00,51.00,46.00
+2,N,,98.00,34.00,87.00,23.00
+2,CORR,var01,1.00,.20,.30,.40
+2,CORR,var02,.20,1.00,.50,.60
+2,CORR,var03,.30,.50,1.00,.70
+2,CORR,var04,.40,.60,.70,1.00
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 9 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - Factor variables grouping within-cell records by factor without ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=f1 var01 TO var04
+ /FACTOR=f1
+ /CELLS=2
+ /CONTENTS=(MEAN SD N) CORR.
+BEGIN DATA.
+0 34 35 36 37
+0 22 11 55 66
+0 99 98 99 92
+1 44 45 34 39
+1 23 15 51 46
+1 98 34 87 23
+ 1
+ .9 1
+ .8 .6 1
+ .7 .5 .4 1
+END DATA.
+FORMATS var01 TO var04(F5.1).
+LIST.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
+Table: Data List
+ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
+MEAN,0,,34.0,35.0,36.0,37.0
+STDDEV,0,,22.0,11.0,55.0,66.0
+N,0,,99.0,98.0,99.0,92.0
+MEAN,1,,44.0,45.0,34.0,39.0
+STDDEV,1,,23.0,15.0,51.0,46.0
+N,1,,98.0,34.0,87.0,23.0
+CORR,.,var01,1.0,.9,.8,.7
+CORR,.,var02,.9,1.0,.6,.5
+CORR,.,var03,.8,.6,1.0,.4
+CORR,.,var04,.7,.5,.4,1.0
+])
+AT_CLEANUP
+
+dnl Keep this test in sync with Example 10 in doc/matrices.texi.
+AT_SETUP([MATRIX DATA - Factor variables grouping within-cell records by row type without ROWTYPE_])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=f1 var01 TO var04
+ /FACTOR=f1
+ /CELLS=2
+ /CONTENTS=(MEAN) (SD) (N) CORR.
+BEGIN DATA.
+0 34 35 36 37
+1 44 45 34 39
+0 22 11 55 66
+1 23 15 51 46
+0 99 98 99 92
+1 98 34 87 23
+ 1
+ .9 1
+ .8 .6 1
+ .7 .5 .4 1
+END DATA.
+FORMATS var01 TO var04(F5.1).
+LIST.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
+Table: Data List
+ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
+MEAN,0,,34.0,35.0,36.0,37.0
+MEAN,1,,44.0,45.0,34.0,39.0
+STDDEV,0,,22.0,11.0,55.0,66.0
+STDDEV,1,,23.0,15.0,51.0,46.0
+N,0,,99.0,98.0,99.0,92.0
+N,1,,98.0,34.0,87.0,23.0
+CORR,.,var01,1.0,.9,.8,.7
+CORR,.,var02,.9,1.0,.6,.5
+CORR,.,var03,.8,.6,1.0,.4
+CORR,.,var04,.7,.5,.4,1.0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX DATA - syntax errors])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA VARIABLES=var01 varname_.
+MATRIX DATA VARIABLES=v v v.
+MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/SPLIT=rowtype_.
+MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/FACTORS=rowtype_.
+MATRIX DATA VARIABLES=rowtype_ s1 v1 v2 v3/SPLIT=v1/FACTORS=v1.
+
+MATRIX DATA VARIABLES=v1 v2 v3/FORMAT=FULL NODIAGONAL.
+MATRIX DATA VARIABLES=v1 v2 v3/FACTORS=v1.
+MATRIX DATA VARIABLES=v1 v2 v3.
+BEGIN DATA.
+END DATA.
+MATRIX DATA VARIABLES=v1/FACTORS=v1.
+MATRIX DATA VARIABLES=v1 v2 v3 ROWTYPE_.
+MATRIX DATA VARIABLES=v1 v2 v3/CONTENTS=N/N=5.
+MATRIX DATA VARIABLES=v1/CONTENTS=XYZZY.
+MATRIX DATA VARIABLES=v1/CONTENTS=(.
+MATRIX DATA VARIABLES=v1/CONTENTS=(CORR.
+MATRIX DATA VARIABLES=v1/CONTENTS=).
+MATRIX DATA.
+MATRIX DATA VARIABLES=v*.
+MATRIX DATA VARIABLES=v/N=-1.
+MATRIX DATA VARIABLES=v/FORMAT=XYZZY.
+MATRIX DATA VARIABLES=v/FILE=123.
+MATRIX DATA VARIABLES=v/SPLIT=123.
+MATRIX DATA VARIABLES=v/CELLS=-1.
+MATRIX DATA VARIABLES=v/XYZZY.
+])
+AT_CHECK([pspp matrix-data.sps -O format=csv], [1], [dnl
+"matrix-data.sps:1.23-1.36: error: MATRIX DATA: VARIABLES may not include VARNAME_.
+ 1 | MATRIX DATA VARIABLES=var01 varname_.
+ | ^~~~~~~~~~~~~~"
+
+"matrix-data.sps:2.25: error: MATRIX DATA: Variable v appears twice in variable list.
+ 2 | MATRIX DATA VARIABLES=v v v.
+ | ^"
+
+"matrix-data.sps:3.47-3.54: error: MATRIX DATA: ROWTYPE_ is not allowed on SPLIT or FACTORS.
+ 3 | MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/SPLIT=rowtype_.
+ | ^~~~~~~~"
+
+"matrix-data.sps:4.49-4.56: error: MATRIX DATA: ROWTYPE_ is not allowed on SPLIT or FACTORS.
+ 4 | MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/FACTORS=rowtype_.
+ | ^~~~~~~~"
+
+"matrix-data.sps:5.61-5.62: error: MATRIX DATA: v1 may not appear on both SPLIT and FACTORS.
+ 5 | MATRIX DATA VARIABLES=rowtype_ s1 v1 v2 v3/SPLIT=v1/FACTORS=v1.
+ | ^~"
+
+"matrix-data.sps:7.32-7.53: error: MATRIX DATA: FORMAT=FULL and FORMAT=NODIAGONAL are mutually exclusive.
+ 7 | MATRIX DATA VARIABLES=v1 v2 v3/FORMAT=FULL NODIAGONAL.
+ | ^~~~~~~~~~~~~~~~~~~~~~"
+
+matrix-data.sps:8: error: MATRIX DATA: CELLS is required when factor variables are specified and VARIABLES does not include ROWTYPE_.
+
+matrix-data.sps:9: warning: MATRIX DATA: CONTENTS was not specified and VARIABLES does not include ROWTYPE_. Assuming CONTENTS=CORR.
+
+matrix-data.sps:12: error: MATRIX DATA: CELLS is required when factor variables are specified and VARIABLES does not include ROWTYPE_.
+
+"matrix-data.sps:13.13-13.39: error: MATRIX DATA: VARIABLES includes ROWTYPE_ but the continuous variables are not the last ones on VARIABLES.
+ 13 | MATRIX DATA VARIABLES=v1 v2 v3 ROWTYPE_.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"matrix-data.sps:14.43-14.45: error: MATRIX DATA: Cannot specify N on CONTENTS along with the N subcommand.
+ 14 | MATRIX DATA VARIABLES=v1 v2 v3/CONTENTS=N/N=5.
+ | ^~~"
+
+"matrix-data.sps:15.35-15.39: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
+ 15 | MATRIX DATA VARIABLES=v1/CONTENTS=XYZZY.
+ | ^~~~~"
+
+"matrix-data.sps:16.36: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
+ 16 | MATRIX DATA VARIABLES=v1/CONTENTS=@{:@.
+ | ^"
+
+"matrix-data.sps:17.40: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
+ 17 | MATRIX DATA VARIABLES=v1/CONTENTS=@{:@CORR.
+ | ^"
+
+"matrix-data.sps:18.35: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
+ 18 | MATRIX DATA VARIABLES=v1/CONTENTS=@:}@.
+ | ^"
+
+"matrix-data.sps:19.12: error: MATRIX DATA: Syntax error expecting VARIABLES.
+ 19 | MATRIX DATA.
+ | ^"
+
+"matrix-data.sps:20.24: error: MATRIX DATA: Syntax error expecting `/'.
+ 20 | MATRIX DATA VARIABLES=v*.
+ | ^"
+
+"matrix-data.sps:21.27-21.28: error: MATRIX DATA: Syntax error expecting non-negative integer for N.
+ 21 | MATRIX DATA VARIABLES=v/N=-1.
+ | ^~"
+
+"matrix-data.sps:22.32-22.36: error: MATRIX DATA: Syntax error expecting LIST, FREE, UPPER, LOWER, FULL, DIAGONAL, or NODIAGONAL.
+ 22 | MATRIX DATA VARIABLES=v/FORMAT=XYZZY.
+ | ^~~~~"
+
+"matrix-data.sps:23.30-23.32: error: MATRIX DATA: Syntax error expecting a file name or handle name.
+ 23 | MATRIX DATA VARIABLES=v/FILE=123.
+ | ^~~"
+
+"matrix-data.sps:24.31-24.33: error: MATRIX DATA: Syntax error expecting variable name.
+ 24 | MATRIX DATA VARIABLES=v/SPLIT=123.
+ | ^~~"
+
+"matrix-data.sps:25.31-25.32: error: MATRIX DATA: Syntax error expecting non-negative integer for CELLS.
+ 25 | MATRIX DATA VARIABLES=v/CELLS=-1.
+ | ^~"
+
+"matrix-data.sps:26.25-26.29: error: MATRIX DATA: Syntax error expecting N, FORMAT, FILE, SPLIT, FACTORS, CELLS, or CONTENTS.
+ 26 | MATRIX DATA VARIABLES=v/XYZZY.
+ | ^~~~~"
+])
+AT_CLEANUP
+
+dnl I don't know what lunatic thought this was OK, but we strive to be
+dnl compatible.
+AT_SETUP([MATRIX DATA - plus and minus as delimiters])
+AT_DATA([matrix-data.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ var01 TO var08.
+BEGIN DATA.
+MEAN+24.3+5.4+69.7+20.1+13.4+2.7+27.9+3.7
+SD +5.7+1.5+23.5+5.8+2.8+4.5+5.4+1.5
+N+92+92+92+92+92+92+92+92
+CORR+1.00
+CORR+.18+1.00
+CORR-.22e+0-.17+1.00
+CORR+.36d-0+.31-.14+1.00
+CORR+.27+.16-.12+.22+1.00
+CORR+.33+.15-.17+.24+.21+1.00
+CORR+.50+.29-.20+.32+.12+.38+1.00
+CORR+.17+.29-.05+.20+.27+.20+.04+1.00
+END DATA.
+FORMATS var01 TO var08(F5.2).
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
+MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
+STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
+N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
+CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
+CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
+CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
+CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
+CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
+CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
+CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
+CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
+])
+AT_CLEANUP
--- /dev/null
+AT_BANNER([Matrix reader])
+
+AT_SETUP([Matrix reader - negative tests])
+AT_DATA([matrix-reader.sps], [dnl
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) VARNAME_(a8) f1 (f1) ROWTYPE_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(f8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (f8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTPYE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARANME_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (a1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (a1) VARNAME_ (a8) c1 to c5 (f8.2).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (a8).
+DEBUG MATRIX READ NODATA.
+
+DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8).
+DEBUG MATRIX READ NODATA.
+])
+AT_CHECK([pspp matrix-reader.sps --testing-mode -O format=csv], [1], [dnl
+matrix-reader.sps:5: error: DEBUG MATRIX READ: Variable ROWTYPE_ must precede VARNAME_ in matrix file dictionary.
+
+matrix-reader.sps:8: error: DEBUG MATRIX READ: Matrix dataset variable ROWTYPE_ should be of string type.
+
+matrix-reader.sps:11: error: DEBUG MATRIX READ: Matrix dataset variable VARNAME_ should be of string type.
+
+matrix-reader.sps:14: error: DEBUG MATRIX READ: Matrix dataset lacks a variable called ROWTYPE_.
+
+matrix-reader.sps:17: error: DEBUG MATRIX READ: Matrix dataset lacks a variable called VARNAME_.
+
+matrix-reader.sps:20: error: DEBUG MATRIX READ: Matrix dataset variable s1 should be numeric.
+
+matrix-reader.sps:23: error: DEBUG MATRIX READ: Matrix dataset variable f1 should be numeric.
+
+matrix-reader.sps:26: error: DEBUG MATRIX READ: Matrix dataset variable c1 should be numeric.
+
+matrix-reader.sps:29: error: DEBUG MATRIX READ: Matrix dataset does not have any continuous variables.
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+AT_BANNER([MATRIX])
+
+AT_SETUP([MATRIX - empty matrices])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE a={}.
+PRINT a.
+COMPUTE b={a; 1; 2; 3}.
+PRINT b.
+COMPUTE c={a, 1, 2, 3}.
+PRINT c.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+a
+
+b
+ 1
+ 2
+ 3
+
+c
+ 1 2 3
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - submatrices as rvalues - all columns or all rows])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(1, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1}, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2}, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2, 3}, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1; 3; 2}, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 3, 3}, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(1:2, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(1:3, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({}, :).
+
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1}).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2}).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2, 3}).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1; 3; 2}).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 3, 3}).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:2).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:3).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {}).
+
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, :).
+
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(0, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 0).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(4, :).
+PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 4).
+
+PRINT {}(:,{}).
+PRINT {}({},:).
+PRINT {}({},{}).
+
+PRINT {1, 2, 3, 4}({1, 2; 3, 4}, :).
+PRINT {1, 2, 3, 4}(:, {1, 2; 3, 4}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(1, :)
+ 1 2 3
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}({1}, :)
+ 1 2 3
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2}, :)
+ 1 2 3
+ 4 5 6
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2, 3}, :)
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}({1; 3; 2}, :)
+ 1 2 3
+ 7 8 9
+ 4 5 6
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 3, 3}, :)
+ 1 2 3
+ 7 8 9
+ 7 8 9
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(1:2, :)
+ 1 2 3
+ 4 5 6
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(1:3, :)
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}({}, :)
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1)
+ 1
+ 4
+ 7
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1})
+ 1
+ 4
+ 7
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2})
+ 1 2
+ 4 5
+ 7 8
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2, 3})
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1; 3; 2})
+ 1 3 2
+ 4 6 5
+ 7 9 8
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 3, 3})
+ 1 3 3
+ 4 6 6
+ 7 9 9
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:2)
+ 1 2
+ 4 5
+ 7 8
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:3)
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {})
+
+
+
+{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, :)
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+matrix.sps:24.35: error: MATRIX: 0 is not a valid row index for a 3×3 matrix.
+ 24 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(0, :).
+ | ^
+
+matrix.sps:25.38: error: MATRIX: 0 is not a valid column index for a 3×3
+matrix.
+ 25 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 0).
+ | ^
+
+matrix.sps:26.35: error: MATRIX: 4 is not a valid row index for a 3×3 matrix.
+ 26 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(4, :).
+ | ^
+
+matrix.sps:27.38: error: MATRIX: 4 is not a valid column index for a 3×3
+matrix.
+ 27 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 4).
+ | ^
+
+{}(:,{})
+
+{}({},:)
+
+{}({},{})
+
+matrix.sps:33.20-33.31: error: MATRIX: Matrix row index must be scalar or
+vector, not a 2×2 matrix.
+ 33 | PRINT {1, 2, 3, 4}({1, 2; 3, 4}, :).
+ | ^~~~~~~~~~~~
+
+matrix.sps:34.23-34.34: error: MATRIX: Matrix column index must be scalar or
+vector, not a 2×2 matrix.
+ 34 | PRINT {1, 2, 3, 4}(:, {1, 2; 3, 4}).
+ | ^~~~~~~~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - COMPUTE submatrices as lvalues])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE y={1, 2, 3; 4, 5, 6; 7, 8, 9}.
+
+COMPUTE x1=y.
+COMPUTE x1(1, :) = {11, 12, 13}.
+PRINT x1.
+
+COMPUTE x2=y.
+COMPUTE x2(2, :) = {14, 15, 16}.
+PRINT x2.
+
+COMPUTE x3=y.
+COMPUTE x3(3, :) = {17, 18, 19}.
+PRINT x3.
+
+COMPUTE x4=y.
+COMPUTE x4(:, 1) = {11; 14; 17}.
+PRINT x4.
+
+COMPUTE x5=y.
+COMPUTE x5(:, 2) = {12; 15; 18}.
+PRINT x5.
+
+COMPUTE x6=y.
+COMPUTE x6(:, 3) = {13; 16; 19}.
+PRINT x6.
+
+COMPUTE x7=y.
+COMPUTE x7(1, 1) = 11.
+PRINT x7.
+
+COMPUTE x8=y.
+COMPUTE x8(1:2, 2:3) = {12, 13; 15, 16}.
+PRINT x8.
+
+COMPUTE x9=y.
+COMPUTE x9({3, 1}, {2; 3}) = {18, 19; 12, 13}.
+PRINT x9.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+x1
+ 11 12 13
+ 4 5 6
+ 7 8 9
+
+x2
+ 1 2 3
+ 14 15 16
+ 7 8 9
+
+x3
+ 1 2 3
+ 4 5 6
+ 17 18 19
+
+x4
+ 11 2 3
+ 14 5 6
+ 17 8 9
+
+x5
+ 1 12 3
+ 4 15 6
+ 7 18 9
+
+x6
+ 1 2 13
+ 4 5 16
+ 7 8 19
+
+x7
+ 11 2 3
+ 4 5 6
+ 7 8 9
+
+x8
+ 1 12 13
+ 4 15 16
+ 7 8 9
+
+x9
+ 1 12 13
+ 4 5 6
+ 7 18 19
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - COMPUTE submatrices as lvalues - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE x={1, 2, 3; 4, 5, 6; 7, 8, 9}.
+COMPUTE x(1, :) = {}.
+COMPUTE x(1, :) = 15.
+COMPUTE x(1, :) = {11, 12}.
+COMPUTE x(1, :) = {11, 12, 13, 14}.
+COMPUTE x(:, 1) = {}.
+COMPUTE x(:, 1) = 15.
+COMPUTE x(:, 1) = {11, 12}.
+COMPUTE x(:, 1) = {11, 12, 13, 14}.
+COMPUTE x(:) = 1.
+COMPUTE x(0, 1) = 1.
+COMPUTE x(1, 0) = 1.
+COMPUTE x({1, 0, 2}, 1) = {1; 2; 3}.
+COMPUTE x(4, 3) = 1.
+COMPUTE x(3, 4) = 1.
+COMPUTE x({1, 2; 3, 4}, 5) = 1.
+COMPUTE x(3, {1, 2; 3, 4}) = 1.
+PRINT x.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:3.9-3.15: error: MATRIX: Numbers of indexes for assigning to x
+differ from the size of the source matrix.
+ 3 | COMPUTE x(1, :) = {}.
+ | ^~~~~~~
+
+matrix.sps:3.11: note: MATRIX: There is 1 row index.
+ 3 | COMPUTE x(1, :) = {}.
+ | ^
+
+matrix.sps:3.14: note: MATRIX: Destination matrix x has 3 columns.
+ 3 | COMPUTE x(1, :) = {}.
+ | ^
+
+matrix.sps:3.19-3.20: note: MATRIX: The source matrix is 0×0.
+ 3 | COMPUTE x(1, :) = {}.
+ | ^~
+
+matrix.sps:4.9-4.15: error: MATRIX: Number of column indexes for assigning to x
+differs from number of columns in source matrix.
+ 4 | COMPUTE x(1, :) = 15.
+ | ^~~~~~~
+
+matrix.sps:4.14: note: MATRIX: Destination matrix x has 3 columns.
+ 4 | COMPUTE x(1, :) = 15.
+ | ^
+
+matrix.sps:4.19-4.20: note: MATRIX: The source matrix is 1×1.
+ 4 | COMPUTE x(1, :) = 15.
+ | ^~
+
+matrix.sps:5.9-5.15: error: MATRIX: Number of column indexes for assigning to x
+differs from number of columns in source matrix.
+ 5 | COMPUTE x(1, :) = {11, 12}.
+ | ^~~~~~~
+
+matrix.sps:5.14: note: MATRIX: Destination matrix x has 3 columns.
+ 5 | COMPUTE x(1, :) = {11, 12}.
+ | ^
+
+matrix.sps:5.19-5.26: note: MATRIX: The source matrix is 1×2.
+ 5 | COMPUTE x(1, :) = {11, 12}.
+ | ^~~~~~~~
+
+matrix.sps:6.9-6.15: error: MATRIX: Number of column indexes for assigning to x
+differs from number of columns in source matrix.
+ 6 | COMPUTE x(1, :) = {11, 12, 13, 14}.
+ | ^~~~~~~
+
+matrix.sps:6.14: note: MATRIX: Destination matrix x has 3 columns.
+ 6 | COMPUTE x(1, :) = {11, 12, 13, 14}.
+ | ^
+
+matrix.sps:6.19-6.34: note: MATRIX: The source matrix is 1×4.
+ 6 | COMPUTE x(1, :) = {11, 12, 13, 14}.
+ | ^~~~~~~~~~~~~~~~
+
+matrix.sps:7.9-7.15: error: MATRIX: Numbers of indexes for assigning to x
+differ from the size of the source matrix.
+ 7 | COMPUTE x(:, 1) = {}.
+ | ^~~~~~~
+
+matrix.sps:7.11: note: MATRIX: Destination matrix x has 3 rows.
+ 7 | COMPUTE x(:, 1) = {}.
+ | ^
+
+matrix.sps:7.14: note: MATRIX: There is 1 column index.
+ 7 | COMPUTE x(:, 1) = {}.
+ | ^
+
+matrix.sps:7.19-7.20: note: MATRIX: The source matrix is 0×0.
+ 7 | COMPUTE x(:, 1) = {}.
+ | ^~
+
+matrix.sps:8.9-8.15: error: MATRIX: Number of row indexes for assigning to x
+differs from number of rows in source matrix.
+ 8 | COMPUTE x(:, 1) = 15.
+ | ^~~~~~~
+
+matrix.sps:8.11: note: MATRIX: Destination matrix x has 3 rows.
+ 8 | COMPUTE x(:, 1) = 15.
+ | ^
+
+matrix.sps:8.19-8.20: note: MATRIX: The source matrix is 1×1.
+ 8 | COMPUTE x(:, 1) = 15.
+ | ^~
+
+matrix.sps:9.9-9.15: error: MATRIX: Numbers of indexes for assigning to x
+differ from the size of the source matrix.
+ 9 | COMPUTE x(:, 1) = {11, 12}.
+ | ^~~~~~~
+
+matrix.sps:9.11: note: MATRIX: Destination matrix x has 3 rows.
+ 9 | COMPUTE x(:, 1) = {11, 12}.
+ | ^
+
+matrix.sps:9.14: note: MATRIX: There is 1 column index.
+ 9 | COMPUTE x(:, 1) = {11, 12}.
+ | ^
+
+matrix.sps:9.19-9.26: note: MATRIX: The source matrix is 1×2.
+ 9 | COMPUTE x(:, 1) = {11, 12}.
+ | ^~~~~~~~
+
+matrix.sps:10.9-10.15: error: MATRIX: Numbers of indexes for assigning to x
+differ from the size of the source matrix.
+ 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
+ | ^~~~~~~
+
+matrix.sps:10.11: note: MATRIX: Destination matrix x has 3 rows.
+ 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
+ | ^
+
+matrix.sps:10.14: note: MATRIX: There is 1 column index.
+ 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
+ | ^
+
+matrix.sps:10.19-10.34: note: MATRIX: The source matrix is 1×4.
+ 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
+ | ^~~~~~~~~~~~~~~~
+
+matrix.sps:11.9-11.12: error: MATRIX: Can't use vector indexing on 3×3 matrix
+x.
+ 11 | COMPUTE x(:) = 1.
+ | ^~~~
+
+matrix.sps:12.11: error: MATRIX: 0 is not a valid row index for a 3×3 matrix.
+ 12 | COMPUTE x(0, 1) = 1.
+ | ^
+
+matrix.sps:13.14: error: MATRIX: 0 is not a valid column index for a 3×3
+matrix.
+ 13 | COMPUTE x(1, 0) = 1.
+ | ^
+
+matrix.sps:14.11-14.19: error: MATRIX: 0 is not a valid row index for a 3×3
+matrix.
+ 14 | COMPUTE x({1, 0, 2}, 1) = {1; 2; 3}.
+ | ^~~~~~~~~
+
+matrix.sps:15.11: error: MATRIX: 4 is not a valid row index for a 3×3 matrix.
+ 15 | COMPUTE x(4, 3) = 1.
+ | ^
+
+matrix.sps:16.14: error: MATRIX: 4 is not a valid column index for a 3×3
+matrix.
+ 16 | COMPUTE x(3, 4) = 1.
+ | ^
+
+matrix.sps:17.11-17.22: error: MATRIX: Matrix row index must be scalar or
+vector, not a 2×2 matrix.
+ 17 | COMPUTE x({1, 2; 3, 4}, 5) = 1.
+ | ^~~~~~~~~~~~
+
+matrix.sps:18.14-18.25: error: MATRIX: Matrix column index must be scalar or
+vector, not a 2×2 matrix.
+ 18 | COMPUTE x(3, {1, 2; 3, 4}) = 1.
+ | ^~~~~~~~~~~~
+
+x
+ 1 2 3
+ 4 5 6
+ 7 8 9
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - subvectors as rvalues])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT {10, 20, 30}({}).
+PRINT {10, 20, 30}(2).
+PRINT {10, 20, 30}({2}).
+PRINT {10, 20, 30}({1,3}).
+PRINT {10, 20, 30}({2,3}).
+PRINT {10, 20, 30}({1;3}).
+PRINT {10, 20, 30}({2;3}).
+PRINT {10, 20, 30}(2:3).
+PRINT {10, 20, 30}(:).
+
+PRINT {10; 20; 30}({}).
+PRINT {10; 20; 30}(2).
+PRINT {10; 20; 30}({2}).
+PRINT {10; 20; 30}({1,3}).
+PRINT {10; 20; 30}({2,3}).
+PRINT {10; 20; 30}({1;3}).
+PRINT {10; 20; 30}({2;3}).
+PRINT {10; 20; 30}(2:3).
+PRINT {10; 20; 30}(:).
+
+PRINT {}({}).
+
+PRINT {1, 2; 3, 4}(:).
+PRINT {1, 2, 3, 4}({1, 2; 3, 4}).
+PRINT {1, 2, 3, 4}(0).
+PRINT {1, 2, 3, 4}(5).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+{10, 20, 30}({})
+
+{10, 20, 30}(2)
+ 20
+
+{10, 20, 30}({2})
+ 20
+
+{10, 20, 30}({1,3})
+ 10 30
+
+{10, 20, 30}({2,3})
+ 20 30
+
+{10, 20, 30}({1;3})
+ 10 30
+
+{10, 20, 30}({2;3})
+ 20 30
+
+{10, 20, 30}(2:3)
+ 20 30
+
+{10, 20, 30}(:)
+ 10 20 30
+
+{10; 20; 30}({})
+
+{10; 20; 30}(2)
+ 20
+
+{10; 20; 30}({2})
+ 20
+
+{10; 20; 30}({1,3})
+ 10
+ 30
+
+{10; 20; 30}({2,3})
+ 20
+ 30
+
+{10; 20; 30}({1;3})
+ 10
+ 30
+
+{10; 20; 30}({2;3})
+ 20
+ 30
+
+{10; 20; 30}(2:3)
+ 20
+ 30
+
+{10; 20; 30}(:)
+ 10
+ 20
+ 30
+
+{}({})
+
+matrix.sps:24.7-24.18: error: MATRIX: Vector index operator may not be applied
+to a 2×2 matrix.
+ 24 | PRINT {1, 2; 3, 4}(:).
+ | ^~~~~~~~~~~~
+
+matrix.sps:25.20-25.31: error: MATRIX: Vector index must be scalar or vector,
+not a 2×2 matrix.
+ 25 | PRINT {1, 2, 3, 4}({1, 2; 3, 4}).
+ | ^~~~~~~~~~~~
+
+matrix.sps:26.20: error: MATRIX: Index 0 is out of range for vector with 4
+elements.
+ 26 | PRINT {1, 2, 3, 4}(0).
+ | ^
+
+matrix.sps:27.20: error: MATRIX: Index 5 is out of range for vector with 4
+elements.
+ 27 | PRINT {1, 2, 3, 4}(5).
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - COMPUTE subvectors as lvalues])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE r={1, 2, 3, 4, 5, 6, 7, 8, 9}.
+
+COMPUTE r1=r.
+COMPUTE r1(:) = {11, 12, 13, 14, 15, 16, 17, 18, 19}.
+PRINT r1.
+
+COMPUTE r2=r.
+COMPUTE r2(:) = {11; 12; 13; 14; 15; 16; 17; 18; 19}.
+PRINT r2.
+
+COMPUTE r3=r.
+COMPUTE r3(1) = 11.
+PRINT r3.
+
+COMPUTE r4=r.
+COMPUTE r4(1:2) = {11:12}.
+PRINT r4.
+
+COMPUTE r5=r.
+COMPUTE r5({8;9}) = {18:19}.
+PRINT r5.
+
+COMPUTE c={1, 2, 3, 4, 5, 6, 7, 8, 9}.
+
+COMPUTE c1=c.
+COMPUTE c1(:) = {11, 12, 13, 14, 15, 16, 17, 18, 19}.
+PRINT c1.
+
+COMPUTE c2=c.
+COMPUTE c2(:) = {11; 12; 13; 14; 15; 16; 17; 18; 19}.
+PRINT c2.
+
+COMPUTE c3=c.
+COMPUTE c3(1) = 11.
+PRINT c3.
+
+COMPUTE c4=c.
+COMPUTE c4(1:2) = {11:12}.
+PRINT c4.
+
+COMPUTE c5=c.
+COMPUTE c5(8:9) = {18:19}.
+PRINT c5.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+r1
+ 11 12 13 14 15 16 17 18 19
+
+r2
+ 11 12 13 14 15 16 17 18 19
+
+r3
+ 11 2 3 4 5 6 7 8 9
+
+r4
+ 11 12 3 4 5 6 7 8 9
+
+r5
+ 1 2 3 4 5 6 7 18 19
+
+c1
+ 11 12 13 14 15 16 17 18 19
+
+c2
+ 11 12 13 14 15 16 17 18 19
+
+c3
+ 11 2 3 4 5 6 7 8 9
+
+c4
+ 11 12 3 4 5 6 7 8 9
+
+c5
+ 1 2 3 4 5 6 7 18 19
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - COMPUTE subvectors as lvalues - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE r={1, 2, 3, 4, 5, 6, 7, 8, 9}.
+COMPUTE r(1:3) = {1, 2; 3, 4}.
+COMPUTE r(1:3) = {}.
+COMPUTE r(1:3) = {1}.
+COMPUTE r(1:3) = {1, 2}.
+COMPUTE r(1:3) = {1, 2, 3, 4}.
+COMPUTE r(1:3) = {}.
+COMPUTE r(1:3) = {1}.
+COMPUTE r(1:3) = {1; 2}.
+COMPUTE r(1:3) = {1; 2; 3; 4}.
+COMPUTE r(:) = {1; 2; 3; 4}.
+COMPUTE r(0) = 5.
+COMPUTE r(10) = 5.
+COMPUTE r({1, 2; 3, 4}) = 1.
+
+COMPUTE c={1, 2, 3, 4, 5, 6, 7, 8, 9}.
+COMPUTE c(1:3) = {1, 2; 3, 4}.
+COMPUTE c(1:3) = {}.
+COMPUTE c(1:3) = {1}.
+COMPUTE c(1:3) = {1, 2}.
+COMPUTE c(1:3) = {1, 2, 3, 4}.
+COMPUTE c(1:3) = {}.
+COMPUTE c(1:3) = {1}.
+COMPUTE c(1:3) = {1; 2}.
+COMPUTE c(1:3) = {1; 2; 3; 4}.
+COMPUTE c(:) = {1; 2; 3; 4}.
+COMPUTE c(0) = 5.
+COMPUTE c(10) = 5.
+COMPUTE c({1, 2; 3, 4}) = 1.
+
+COMPUTE m = {1, 2; 3, 4}.
+COMPUTE m(5) = 1.
+COMPUTE m(:) = 1.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:3.9-3.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 3 | COMPUTE r(1:3) = {1, 2; 3, 4}.
+ | ^~~~~~
+
+matrix.sps:3.18-3.29: error: MATRIX: The source is an 2×2 matrix.
+ 3 | COMPUTE r(1:3) = {1, 2; 3, 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:4.9-4.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 4 | COMPUTE r(1:3) = {}.
+ | ^~~~~~
+
+matrix.sps:4.18-4.19: error: MATRIX: The source vector has 0 elements.
+ 4 | COMPUTE r(1:3) = {}.
+ | ^~
+
+matrix.sps:5.9-5.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 5 | COMPUTE r(1:3) = {1}.
+ | ^~~~~~
+
+matrix.sps:5.19: error: MATRIX: The source vector has 1 element.
+ 5 | COMPUTE r(1:3) = {1}.
+ | ^
+
+matrix.sps:6.9-6.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 6 | COMPUTE r(1:3) = {1, 2}.
+ | ^~~~~~
+
+matrix.sps:6.18-6.23: error: MATRIX: The source vector has 2 elements.
+ 6 | COMPUTE r(1:3) = {1, 2}.
+ | ^~~~~~
+
+matrix.sps:7.9-7.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 7 | COMPUTE r(1:3) = {1, 2, 3, 4}.
+ | ^~~~~~
+
+matrix.sps:7.18-7.29: error: MATRIX: The source vector has 4 elements.
+ 7 | COMPUTE r(1:3) = {1, 2, 3, 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:8.9-8.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 8 | COMPUTE r(1:3) = {}.
+ | ^~~~~~
+
+matrix.sps:8.18-8.19: error: MATRIX: The source vector has 0 elements.
+ 8 | COMPUTE r(1:3) = {}.
+ | ^~
+
+matrix.sps:9.9-9.14: error: MATRIX: Only an 3-element vector may be assigned to
+this 3-element subvector of r.
+ 9 | COMPUTE r(1:3) = {1}.
+ | ^~~~~~
+
+matrix.sps:9.19: error: MATRIX: The source vector has 1 element.
+ 9 | COMPUTE r(1:3) = {1}.
+ | ^
+
+matrix.sps:10.9-10.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of r.
+ 10 | COMPUTE r(1:3) = {1; 2}.
+ | ^~~~~~
+
+matrix.sps:10.18-10.23: error: MATRIX: The source vector has 2 elements.
+ 10 | COMPUTE r(1:3) = {1; 2}.
+ | ^~~~~~
+
+matrix.sps:11.9-11.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of r.
+ 11 | COMPUTE r(1:3) = {1; 2; 3; 4}.
+ | ^~~~~~
+
+matrix.sps:11.18-11.29: error: MATRIX: The source vector has 4 elements.
+ 11 | COMPUTE r(1:3) = {1; 2; 3; 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:12.9-12.12: error: MATRIX: Only an 9-element vector may be assigned
+to this 9-element subvector of r.
+ 12 | COMPUTE r(:) = {1; 2; 3; 4}.
+ | ^~~~
+
+matrix.sps:12.16-12.27: error: MATRIX: The source vector has 4 elements.
+ 12 | COMPUTE r(:) = {1; 2; 3; 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:13.11: error: MATRIX: Index 0 is out of range for vector with 9
+elements.
+ 13 | COMPUTE r(0) = 5.
+ | ^
+
+matrix.sps:14.11-14.12: error: MATRIX: Index 10 is out of range for vector with
+9 elements.
+ 14 | COMPUTE r(10) = 5.
+ | ^~
+
+matrix.sps:15.11-15.22: error: MATRIX: Vector index must be scalar or vector,
+not a 2×2 matrix.
+ 15 | COMPUTE r({1, 2; 3, 4}) = 1.
+ | ^~~~~~~~~~~~
+
+matrix.sps:18.9-18.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 18 | COMPUTE c(1:3) = {1, 2; 3, 4}.
+ | ^~~~~~
+
+matrix.sps:18.18-18.29: error: MATRIX: The source is an 2×2 matrix.
+ 18 | COMPUTE c(1:3) = {1, 2; 3, 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:19.9-19.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 19 | COMPUTE c(1:3) = {}.
+ | ^~~~~~
+
+matrix.sps:19.18-19.19: error: MATRIX: The source vector has 0 elements.
+ 19 | COMPUTE c(1:3) = {}.
+ | ^~
+
+matrix.sps:20.9-20.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 20 | COMPUTE c(1:3) = {1}.
+ | ^~~~~~
+
+matrix.sps:20.19: error: MATRIX: The source vector has 1 element.
+ 20 | COMPUTE c(1:3) = {1}.
+ | ^
+
+matrix.sps:21.9-21.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 21 | COMPUTE c(1:3) = {1, 2}.
+ | ^~~~~~
+
+matrix.sps:21.18-21.23: error: MATRIX: The source vector has 2 elements.
+ 21 | COMPUTE c(1:3) = {1, 2}.
+ | ^~~~~~
+
+matrix.sps:22.9-22.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 22 | COMPUTE c(1:3) = {1, 2, 3, 4}.
+ | ^~~~~~
+
+matrix.sps:22.18-22.29: error: MATRIX: The source vector has 4 elements.
+ 22 | COMPUTE c(1:3) = {1, 2, 3, 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:23.9-23.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 23 | COMPUTE c(1:3) = {}.
+ | ^~~~~~
+
+matrix.sps:23.18-23.19: error: MATRIX: The source vector has 0 elements.
+ 23 | COMPUTE c(1:3) = {}.
+ | ^~
+
+matrix.sps:24.9-24.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 24 | COMPUTE c(1:3) = {1}.
+ | ^~~~~~
+
+matrix.sps:24.19: error: MATRIX: The source vector has 1 element.
+ 24 | COMPUTE c(1:3) = {1}.
+ | ^
+
+matrix.sps:25.9-25.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 25 | COMPUTE c(1:3) = {1; 2}.
+ | ^~~~~~
+
+matrix.sps:25.18-25.23: error: MATRIX: The source vector has 2 elements.
+ 25 | COMPUTE c(1:3) = {1; 2}.
+ | ^~~~~~
+
+matrix.sps:26.9-26.14: error: MATRIX: Only an 3-element vector may be assigned
+to this 3-element subvector of c.
+ 26 | COMPUTE c(1:3) = {1; 2; 3; 4}.
+ | ^~~~~~
+
+matrix.sps:26.18-26.29: error: MATRIX: The source vector has 4 elements.
+ 26 | COMPUTE c(1:3) = {1; 2; 3; 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:27.9-27.12: error: MATRIX: Only an 9-element vector may be assigned
+to this 9-element subvector of c.
+ 27 | COMPUTE c(:) = {1; 2; 3; 4}.
+ | ^~~~
+
+matrix.sps:27.16-27.27: error: MATRIX: The source vector has 4 elements.
+ 27 | COMPUTE c(:) = {1; 2; 3; 4}.
+ | ^~~~~~~~~~~~
+
+matrix.sps:28.11: error: MATRIX: Index 0 is out of range for vector with 9
+elements.
+ 28 | COMPUTE c(0) = 5.
+ | ^
+
+matrix.sps:29.11-29.12: error: MATRIX: Index 10 is out of range for vector with
+9 elements.
+ 29 | COMPUTE c(10) = 5.
+ | ^~
+
+matrix.sps:30.11-30.22: error: MATRIX: Vector index must be scalar or vector,
+not a 2×2 matrix.
+ 30 | COMPUTE c({1, 2; 3, 4}) = 1.
+ | ^~~~~~~~~~~~
+
+matrix.sps:33.9-33.12: error: MATRIX: Can't use vector indexing on 2×2 matrix
+m.
+ 33 | COMPUTE m(5) = 1.
+ | ^~~~
+
+matrix.sps:34.9-34.12: error: MATRIX: Can't use vector indexing on 2×2 matrix
+m.
+ 34 | COMPUTE m(:) = 1.
+ | ^~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - COMPUTE - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE x.
+COMPUTE x=.
+COMPUTE x(5)=1.
+COMPUTE y(5)=1.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.10: error: COMPUTE: Syntax error expecting `='.
+ 2 | COMPUTE x.
+ | ^
+
+matrix.sps:3.11: error: COMPUTE: Syntax error expecting matrix expression.
+ 3 | COMPUTE x=.
+ | ^
+
+matrix.sps:4.9: error: MATRIX: Undefined variable x.
+ 4 | COMPUTE x(5)=1.
+ | ^
+
+matrix.sps:5.9: error: COMPUTE: Undefined variable y.
+ 5 | COMPUTE y(5)=1.
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - elementwise arithmetic operators])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT (-(5)).
+PRINT (-{1,2;3,4}).
+
+PRINT ({1,2;3,4} + {5,6;7,8}).
+PRINT ({1,2;3,4} + 5).
+PRINT (5 + {5,6;7,8}).
+PRINT ({1,2;3,4} + {5,6}).
+
+PRINT ({1,2;3,4} - {5,6;7,8}).
+PRINT ({1,2;3,4} - 5).
+PRINT (5 - {5,6;7,8}).
+PRINT ({1,2;3,4} - {5,6}).
+
+PRINT ({1,2;3,4} * 5).
+PRINT (5 * {5,6;7,8}).
+
+PRINT ({2,4;6,8} / 2).
+PRINT (12 / {1,2;3,4}).
+PRINT ({2,8;18,32} / {1,2;3,4}).
+
+PRINT ({1,2;3,4} &* {5,6;7,8}).
+PRINT ({1,2;3,4} &* 5).
+PRINT (5 &* {5,6;7,8}).
+PRINT ({1,2;3,4} &* {5,6}).
+
+PRINT ({2,4;6,8} &/ 2).
+PRINT (12 &/ {1,2;3,4}).
+PRINT ({2,8;18,32} &/ {1,2;3,4}).
+
+PRINT ({1,2;3,4} &** 2).
+PRINT (2 &** {1,2;3,4}).
+PRINT ({1,2;3,4} &** {2,3;4,5}).
+PRINT ({1,2;3,4} &** {5,6}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+(-(5))
+ -5
+
+(-{1,2;3,4})
+ -1 -2
+ -3 -4
+
+({1,2;3,4} + {5,6;7,8})
+ 6 8
+ 10 12
+
+({1,2;3,4} + 5)
+ 6 7
+ 8 9
+
+(5 + {5,6;7,8})
+ 10 11
+ 12 13
+
+matrix.sps:8.8-8.24: error: MATRIX: The operands of + must have the same
+dimensions or one must be a scalar.
+ 8 | PRINT ({1,2;3,4} + {5,6}).
+ | ^~~~~~~~~~~~~~~~~
+
+matrix.sps:8.8-8.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
+ 8 | PRINT ({1,2;3,4} + {5,6}).
+ | ^~~~~~~~~
+
+matrix.sps:8.20-8.24: note: MATRIX: The right-hand operand is a 1×2 matrix.
+ 8 | PRINT ({1,2;3,4} + {5,6}).
+ | ^~~~~
+
+({1,2;3,4} - {5,6;7,8})
+ -4 -4
+ -4 -4
+
+({1,2;3,4} - 5)
+ -4 -3
+ -2 -1
+
+(5 - {5,6;7,8})
+ 0 -1
+ -2 -3
+
+matrix.sps:13.8-13.24: error: MATRIX: The operands of - must have the same
+dimensions or one must be a scalar.
+ 13 | PRINT ({1,2;3,4} - {5,6}).
+ | ^~~~~~~~~~~~~~~~~
+
+matrix.sps:13.8-13.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
+ 13 | PRINT ({1,2;3,4} - {5,6}).
+ | ^~~~~~~~~
+
+matrix.sps:13.20-13.24: note: MATRIX: The right-hand operand is a 1×2 matrix.
+ 13 | PRINT ({1,2;3,4} - {5,6}).
+ | ^~~~~
+
+({1,2;3,4} * 5)
+ 5 10
+ 15 20
+
+(5 * {5,6;7,8})
+ 25 30
+ 35 40
+
+({2,4;6,8} / 2)
+ 1 2
+ 3 4
+
+(12 / {1,2;3,4})
+ 12 6
+ 4 3
+
+({2,8;18,32} / {1,2;3,4})
+ 2 4
+ 6 8
+
+({1,2;3,4} &* {5,6;7,8})
+ 5 12
+ 21 32
+
+({1,2;3,4} &* 5)
+ 5 10
+ 15 20
+
+(5 &* {5,6;7,8})
+ 25 30
+ 35 40
+
+matrix.sps:25.8-25.25: error: MATRIX: The operands of &* must have the same
+dimensions or one must be a scalar.
+ 25 | PRINT ({1,2;3,4} &* {5,6}).
+ | ^~~~~~~~~~~~~~~~~~
+
+matrix.sps:25.8-25.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
+ 25 | PRINT ({1,2;3,4} &* {5,6}).
+ | ^~~~~~~~~
+
+matrix.sps:25.21-25.25: note: MATRIX: The right-hand operand is a 1×2 matrix.
+ 25 | PRINT ({1,2;3,4} &* {5,6}).
+ | ^~~~~
+
+({2,4;6,8} &/ 2)
+ 1 2
+ 3 4
+
+(12 &/ {1,2;3,4})
+ 12 6
+ 4 3
+
+({2,8;18,32} &/ {1,2;3,4})
+ 2 4
+ 6 8
+
+({1,2;3,4} &** 2)
+ 1 4
+ 9 16
+
+(2 &** {1,2;3,4})
+ 2 4
+ 8 16
+
+({1,2;3,4} &** {2,3;4,5})
+ 1 8
+ 81 1024
+
+matrix.sps:34.8-34.26: error: MATRIX: The operands of &** must have the same
+dimensions or one must be a scalar.
+ 34 | PRINT ({1,2;3,4} &** {5,6}).
+ | ^~~~~~~~~~~~~~~~~~~
+
+matrix.sps:34.8-34.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
+ 34 | PRINT ({1,2;3,4} &** {5,6}).
+ | ^~~~~~~~~
+
+matrix.sps:34.22-34.26: note: MATRIX: The right-hand operand is a 1×2 matrix.
+ 34 | PRINT ({1,2;3,4} &** {5,6}).
+ | ^~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - relational operators])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT ({1, 1; 2, 2} > {1, 2; 1, 2}).
+PRINT ({1, 1; 2, 2} > 1).
+PRINT (2 > {1, 2; 1, 2}).
+PRINT ({1, 2} > {1; 2}).
+
+PRINT ({1, 1; 2, 2} < {1, 2; 1, 2}).
+PRINT ({1, 1; 2, 2} < 2).
+PRINT (1 < {1, 2; 1, 2}).
+PRINT ({1, 2} < {1; 2}).
+
+PRINT ({1, 1; 2, 2} <> {1, 2; 1, 2}).
+PRINT ({1, 1; 2, 2} <> 2).
+PRINT (1 <> {1, 2; 1, 2}).
+PRINT ({1, 2} <> {1; 2}).
+
+PRINT ({1, 1; 2, 2} >= {1, 2; 1, 2}).
+PRINT ({1, 1; 2, 2} >= 2).
+PRINT (1 >= {1, 2; 1, 2}).
+PRINT ({1, 2} >= {1; 2}).
+
+PRINT ({1, 1; 2, 2} <= {1, 2; 1, 2}).
+PRINT ({1, 1; 2, 2} <= 2).
+PRINT (1 <= {1, 2; 1, 2}).
+PRINT ({1, 2} <= {1; 2}).
+
+PRINT ({1, 1; 2, 2} = {1, 2; 1, 2}).
+PRINT ({1, 1; 2, 2} = 2).
+PRINT (1 = {1, 2; 1, 2}).
+PRINT ({1, 2} = {1; 2}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+({1, 1; 2, 2} > {1, 2; 1, 2})
+ 0 0
+ 1 0
+
+({1, 1; 2, 2} > 1)
+ 0 0
+ 1 1
+
+(2 > {1, 2; 1, 2})
+ 1 0
+ 1 0
+
+matrix.sps:5.8-5.22: error: MATRIX: The operands of > must have the same
+dimensions or one must be a scalar.
+ 5 | PRINT ({1, 2} > {1; 2}).
+ | ^~~~~~~~~~~~~~~
+
+matrix.sps:5.8-5.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 5 | PRINT ({1, 2} > {1; 2}).
+ | ^~~~~~
+
+matrix.sps:5.17-5.22: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 5 | PRINT ({1, 2} > {1; 2}).
+ | ^~~~~~
+
+({1, 1; 2, 2} < {1, 2; 1, 2})
+ 0 1
+ 0 0
+
+({1, 1; 2, 2} < 2)
+ 1 1
+ 0 0
+
+(1 < {1, 2; 1, 2})
+ 0 1
+ 0 1
+
+matrix.sps:10.8-10.22: error: MATRIX: The operands of < must have the same
+dimensions or one must be a scalar.
+ 10 | PRINT ({1, 2} < {1; 2}).
+ | ^~~~~~~~~~~~~~~
+
+matrix.sps:10.8-10.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 10 | PRINT ({1, 2} < {1; 2}).
+ | ^~~~~~
+
+matrix.sps:10.17-10.22: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 10 | PRINT ({1, 2} < {1; 2}).
+ | ^~~~~~
+
+({1, 1; 2, 2} <> {1, 2; 1, 2})
+ 0 1
+ 1 0
+
+({1, 1; 2, 2} <> 2)
+ 1 1
+ 0 0
+
+(1 <> {1, 2; 1, 2})
+ 0 1
+ 0 1
+
+matrix.sps:15.8-15.23: error: MATRIX: The operands of <> must have the same
+dimensions or one must be a scalar.
+ 15 | PRINT ({1, 2} <> {1; 2}).
+ | ^~~~~~~~~~~~~~~~
+
+matrix.sps:15.8-15.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 15 | PRINT ({1, 2} <> {1; 2}).
+ | ^~~~~~
+
+matrix.sps:15.18-15.23: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 15 | PRINT ({1, 2} <> {1; 2}).
+ | ^~~~~~
+
+({1, 1; 2, 2} >= {1, 2; 1, 2})
+ 1 0
+ 1 1
+
+({1, 1; 2, 2} >= 2)
+ 0 0
+ 1 1
+
+(1 >= {1, 2; 1, 2})
+ 1 0
+ 1 0
+
+matrix.sps:20.8-20.23: error: MATRIX: The operands of >= must have the same
+dimensions or one must be a scalar.
+ 20 | PRINT ({1, 2} >= {1; 2}).
+ | ^~~~~~~~~~~~~~~~
+
+matrix.sps:20.8-20.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 20 | PRINT ({1, 2} >= {1; 2}).
+ | ^~~~~~
+
+matrix.sps:20.18-20.23: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 20 | PRINT ({1, 2} >= {1; 2}).
+ | ^~~~~~
+
+({1, 1; 2, 2} <= {1, 2; 1, 2})
+ 1 1
+ 0 1
+
+({1, 1; 2, 2} <= 2)
+ 1 1
+ 1 1
+
+(1 <= {1, 2; 1, 2})
+ 1 1
+ 1 1
+
+matrix.sps:25.8-25.23: error: MATRIX: The operands of <= must have the same
+dimensions or one must be a scalar.
+ 25 | PRINT ({1, 2} <= {1; 2}).
+ | ^~~~~~~~~~~~~~~~
+
+matrix.sps:25.8-25.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 25 | PRINT ({1, 2} <= {1; 2}).
+ | ^~~~~~
+
+matrix.sps:25.18-25.23: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 25 | PRINT ({1, 2} <= {1; 2}).
+ | ^~~~~~
+
+({1, 1; 2, 2} = {1, 2; 1, 2})
+ 1 0
+ 0 1
+
+({1, 1; 2, 2} = 2)
+ 0 0
+ 1 1
+
+(1 = {1, 2; 1, 2})
+ 1 0
+ 1 0
+
+matrix.sps:30.8-30.22: error: MATRIX: The operands of = must have the same
+dimensions or one must be a scalar.
+ 30 | PRINT ({1, 2} = {1; 2}).
+ | ^~~~~~~~~~~~~~~
+
+matrix.sps:30.8-30.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 30 | PRINT ({1, 2} = {1; 2}).
+ | ^~~~~~
+
+matrix.sps:30.17-30.22: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 30 | PRINT ({1, 2} = {1; 2}).
+ | ^~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - logical operators])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT (NOT {-1, 0, 1}).
+
+PRINT ({-1, 0, 1; -1, 0, 1; -1, 0, 1} AND {-1, -1, -1; 0, 0, 0; 1, 1, 1}).
+PRINT ({-1, 0, 1} AND -1).
+PRINT ({-1, 0, 1} AND 0).
+PRINT ({-1, 0, 1} AND 1).
+PRINT ({-1, 0} AND {2; 3}).
+
+PRINT ({-1, 0, 1; -1, 0, 1; -1, 0, 1} OR {-1, -1, -1; 0, 0, 0; 1, 1, 1}).
+PRINT ({-1, 0, 1} OR -1).
+PRINT ({-1, 0, 1} OR 0).
+PRINT ({-1, 0, 1} OR 1).
+PRINT ({-1, 0} OR {2; 3}).
+
+PRINT ({-1, 0, 1; -1, 0, 1; -1, 0, 1} XOR {-1, -1, -1; 0, 0, 0; 1, 1, 1}).
+PRINT ({-1, 0, 1} XOR -1).
+PRINT ({-1, 0, 1} XOR 0).
+PRINT ({-1, 0, 1} XOR 1).
+PRINT ({-1, 0} XOR {2; 3}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+(NOT {-1, 0, 1})
+ 1 1 0
+
+({-1, 0, 1; -1, 0, 1; -1, 0, 1} AND {-1, -1, -1; 0, 0, 0; 1, 1, 1})
+ 0 0 0
+ 0 0 0
+ 0 0 1
+
+({-1, 0, 1} AND -1)
+ 0 0 0
+
+({-1, 0, 1} AND 0)
+ 0 0 0
+
+({-1, 0, 1} AND 1)
+ 0 0 1
+
+matrix.sps:8.8-8.25: error: MATRIX: The operands of AND must have the same
+dimensions or one must be a scalar.
+ 8 | PRINT ({-1, 0} AND {2; 3}).
+ | ^~~~~~~~~~~~~~~~~~
+
+matrix.sps:8.8-8.14: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 8 | PRINT ({-1, 0} AND {2; 3}).
+ | ^~~~~~~
+
+matrix.sps:8.20-8.25: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 8 | PRINT ({-1, 0} AND {2; 3}).
+ | ^~~~~~
+
+({-1, 0, 1; -1, 0, 1; -1, 0, 1} OR {-1, -1, -1; 0, 0, 0; 1, 1, 1})
+ 0 0 1
+ 0 0 1
+ 1 1 1
+
+({-1, 0, 1} OR -1)
+ 0 0 1
+
+({-1, 0, 1} OR 0)
+ 0 0 1
+
+({-1, 0, 1} OR 1)
+ 1 1 1
+
+matrix.sps:14.8-14.24: error: MATRIX: The operands of OR must have the same
+dimensions or one must be a scalar.
+ 14 | PRINT ({-1, 0} OR {2; 3}).
+ | ^~~~~~~~~~~~~~~~~
+
+matrix.sps:14.8-14.14: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 14 | PRINT ({-1, 0} OR {2; 3}).
+ | ^~~~~~~
+
+matrix.sps:14.19-14.24: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 14 | PRINT ({-1, 0} OR {2; 3}).
+ | ^~~~~~
+
+({-1, 0, 1; -1, 0, 1; -1, 0, 1} XOR {-1, -1, -1; 0, 0, 0; 1, 1, 1})
+ 0 0 1
+ 0 0 1
+ 1 1 0
+
+({-1, 0, 1} XOR -1)
+ 0 0 1
+
+({-1, 0, 1} XOR 0)
+ 0 0 1
+
+({-1, 0, 1} XOR 1)
+ 1 1 0
+
+matrix.sps:20.8-20.25: error: MATRIX: The operands of XOR must have the same
+dimensions or one must be a scalar.
+ 20 | PRINT ({-1, 0} XOR {2; 3}).
+ | ^~~~~~~~~~~~~~~~~~
+
+matrix.sps:20.8-20.14: note: MATRIX: The left-hand operand is a 1×2 matrix.
+ 20 | PRINT ({-1, 0} XOR {2; 3}).
+ | ^~~~~~~
+
+matrix.sps:20.20-20.25: note: MATRIX: The right-hand operand is a 2×1 matrix.
+ 20 | PRINT ({-1, 0} XOR {2; 3}).
+ | ^~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - matrix operators])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT ({0, 1; 0, 0} * {0, 0; 1, 0}).
+PRINT ({0, 0; 1, 0} * {0, 1; 0, 0}).
+PRINT ({1, 2, 3; 4, 5, 6} * {7, 8; 9, 10; 11, 12}).
+PRINT ({3, 4, 2} * {13, 9, 7, 15; 8, 7, 4, 6; 6, 4, 0, 3}).
+COMPUTE m = {0, 1, 0, 0; 1, 0, 1, 0; 0, 1, 0, 1; 0, 0, 1, 0}.
+PRINT m**-2.
+PRINT m**-1.
+PRINT m**0.
+PRINT m**1.
+PRINT m**2.
+PRINT m**3.
+PRINT m**5.
+PRINT {3, 3.5; 3.2, 3.6}**-1/FORMAT F6.2.
+
+PRINT ({1, 2, 3} * {1, 2}).
+PRINT {1, 2, 3}**2.
+PRINT m**{1, 2}.
+PRINT m**1.5.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+({0, 1; 0, 0} * {0, 0; 1, 0})
+ 1 0
+ 0 0
+
+({0, 0; 1, 0} * {0, 1; 0, 0})
+ 0 0
+ 0 1
+
+({1, 2, 3; 4, 5, 6} * {7, 8; 9, 10; 11, 12})
+ 58 64
+ 139 154
+
+({3, 4, 2} * {13, 9, 7, 15; 8, 7, 4, 6; 6, 4, 0, 3})
+ 83 63 37 75
+
+m**-2
+ 2 0 -1 0
+ 0 1 0 -1
+ -1 0 1 0
+ 0 -1 0 2
+
+m**-1
+ 0 1 0 -1
+ 1 0 0 0
+ 0 0 0 1
+ -1 0 1 0
+
+m**0
+ 1 0 0 0
+ 0 1 0 0
+ 0 0 1 0
+ 0 0 0 1
+
+m**1
+ 0 1 0 0
+ 1 0 1 0
+ 0 1 0 1
+ 0 0 1 0
+
+m**2
+ 1 0 1 0
+ 0 2 0 1
+ 1 0 2 0
+ 0 1 0 1
+
+m**3
+ 0 2 0 1
+ 2 0 3 0
+ 0 3 0 2
+ 1 0 2 0
+
+m**5
+ 0 5 0 3
+ 5 0 8 0
+ 0 8 0 5
+ 3 0 5 0
+
+{3, 3.5; 3.2, 3.6}**-1
+ -9.00 8.75
+ 8.00 -7.50
+
+matrix.sps:16.8-16.25: error: MATRIX: Matrices not conformable for
+multiplication.
+ 16 | PRINT ({1, 2, 3} * {1, 2}).
+ | ^~~~~~~~~~~~~~~~~~
+
+matrix.sps:16.8-16.16: note: MATRIX: The left-hand operand is a 1×3 matrix.
+ 16 | PRINT ({1, 2, 3} * {1, 2}).
+ | ^~~~~~~~~
+
+matrix.sps:16.20-16.25: note: MATRIX: The right-hand operand is a 1×2 matrix.
+ 16 | PRINT ({1, 2, 3} * {1, 2}).
+ | ^~~~~~
+
+matrix.sps:17.7-17.15: error: MATRIX: Matrix exponentation with ** requires a
+square matrix on the left-hand size, not one with dimensions 1×3.
+ 17 | PRINT {1, 2, 3}**2.
+ | ^~~~~~~~~
+
+matrix.sps:18.10-18.15: error: MATRIX: Matrix exponentiation with ** requires a
+scalar on the right-hand side, not a matrix with dimensions 1×2.
+ 18 | PRINT m**{1, 2}.
+ | ^~~~~~
+
+matrix.sps:19.10-19.12: error: MATRIX: Exponent 1.5 in matrix exponentiation is
+non-integer or outside the valid range.
+ 19 | PRINT m**1.5.
+ | ^~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - sequences and construction])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT {1:3:-1}.
+PRINT {1:3}.
+PRINT {1:10:2}.
+PRINT {1:11:2}.
+
+PRINT {-1:-3}.
+PRINT {-1:-3:-1}.
+PRINT {-1:-10:-2}.
+PRINT {-1:-11:-2}.
+
+PRINT {1:1}.
+PRINT {1:1:-1}.
+
+PRINT {1:3:0}.
+PRINT {-1:-3:0}.
+
+PRINT {1, 2; 3}.
+PRINT {{2; 5}, 3}.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+{1:3:-1}
+
+{1:3}
+ 1 2 3
+
+{1:10:2}
+ 1 3 5 7 9
+
+{1:11:2}
+ 1 3 5 7 9 11
+
+{-1:-3}
+
+{-1:-3:-1}
+ -1 -2 -3
+
+{-1:-10:-2}
+ -1 -3 -5 -7 -9
+
+{-1:-11:-2}
+ -1 -3 -5 -7 -9 -11
+
+{1:1}
+ 1
+
+{1:1:-1}
+ 1
+
+matrix.sps:15.12: error: MATRIX: The increment operand to : must be nonzero.
+ 15 | PRINT {1:3:0}.
+ | ^
+
+matrix.sps:16.14: error: MATRIX: The increment operand to : must be nonzero.
+ 16 | PRINT {-1:-3:0}.
+ | ^
+
+matrix.sps:18.7-18.15: error: MATRIX: This expression tries to vertically join
+matrices with differing numbers of columns.
+ 18 | PRINT {1, 2; 3}.
+ | ^~~~~~~~~
+
+matrix.sps:18.8-18.11: note: MATRIX: This operand is a 1×2 matrix.
+ 18 | PRINT {1, 2; 3}.
+ | ^~~~
+
+matrix.sps:18.14: note: MATRIX: This operand is a 1×1 matrix.
+ 18 | PRINT {1, 2; 3}.
+ | ^
+
+matrix.sps:19.7-19.17: error: MATRIX: This expression tries to horizontally
+join matrices with differing numbers of rows.
+ 19 | PRINT {{2; 5}, 3}.
+ | ^~~~~~~~~~~
+
+matrix.sps:19.8-19.13: note: MATRIX: This operand is a 2×1 matrix.
+ 19 | PRINT {{2; 5}, 3}.
+ | ^~~~~~
+
+matrix.sps:19.16: note: MATRIX: This operand is a 1×1 matrix.
+ 19 | PRINT {{2; 5}, 3}.
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - comments])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+* Comment one.
+PRINT (1+2).
+COMMENT Comment two.
+PRINT (3+4).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+(1+2)
+ 3
+
+(3+4)
+ 7
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - string matrices])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE m={'This is', 'a string', 'matrix', 'including', 'some', 'long strings'}.
+PRINT m/FORMAT=A8.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+m
+ This is a string matrix includin some long str
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - ABS ALL ANY ARSIN ARTAN])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT ABS({-1, 0, 1}).
+
+PRINT ALL({0, 0, 0}).
+PRINT ALL({-1, 1}).
+PRINT ALL({-1, 0, 1}).
+
+PRINT ANY({0, 0, 0}).
+PRINT ANY({-1, 1}).
+PRINT ANY({-1, 0, 1}).
+
+PRINT ARSIN({-1, 0, 1})/FORMAT=F5.2.
+
+PRINT ARTAN({-5, -1, 0, 1, 5})/FORMAT=F5.2.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+ABS({-1, 0, 1})
+ 1 0 1
+
+ALL({0, 0, 0})
+ 0
+
+ALL({-1, 1})
+ 1
+
+ALL({-1, 0, 1})
+ 0
+
+ANY({0, 0, 0})
+ 0
+
+ANY({-1, 1})
+ 1
+
+ANY({-1, 0, 1})
+ 1
+
+ARSIN({-1, 0, 1})
+ -1.57 .00 1.57
+
+ARTAN({-5, -1, 0, 1, 5})
+ -1.37 -.79 .00 .79 1.37
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - BLOCK CHOL CMAX CMIN COS])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT BLOCK({1, 2; 3, 4}, 5, {7; 8; 9}, {10, 11}).
+
+COMPUTE b=CHOL({4, 12, -16; 12, 37, -43; -16, -43, 98}).
+PRINT b.
+PRINT (T(b)*b).
+
+PRINT CMAX({9, 3, 4; 5, 8, 6; 7, 4, 11}).
+
+PRINT CMIN({9, 3, 4; 5, 8, 6; 7, 4, 11}).
+
+PRINT COS({0.785, 1.57; 3.14, 1.57 + 3.14}) /FORMAT=F5.2.
+
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+BLOCK({1, 2; 3, 4}, 5, {7; 8; 9}, {10, 11})
+ 1 2 0 0 0 0
+ 3 4 0 0 0 0
+ 0 0 5 0 0 0
+ 0 0 0 7 0 0
+ 0 0 0 8 0 0
+ 0 0 0 9 0 0
+ 0 0 0 0 10 11
+
+b
+ 2 6 -8
+ 0 1 5
+ 0 0 3
+
+(T(b)*b)
+ 4 12 -16
+ 12 37 -43
+ -16 -43 98
+
+CMAX({9, 3, 4; 5, 8, 6; 7, 4, 11})
+ 9 8 11
+
+CMIN({9, 3, 4; 5, 8, 6; 7, 4, 11})
+ 5 3 4
+
+COS({0.785, 1.57; 3.14, 1.57 + 3.14})
+ .71 .00
+ -1.00 .00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - CSSQ CSUM DESIGN DET DIAG])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT CSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9}).
+PRINT CSUM({1, 2, 3; 4, 5, 6; 7, 8, 9}).
+PRINT DESIGN({1, 2, 0; 2, 1, 0; 3, 0, 1}).
+PRINT DESIGN({1, 2, 0; 2, 2, 0; 3, 2, 1}).
+PRINT DET({1, 2, 3; 4, 5, 6; 7, 8, 9}) /FORMAT F4.1.
+PRINT DIAG({1, 2, 3, 4; 4, 5, 6, 7; 7, 8, 9, 10}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+CSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9})
+ 66 93 126
+
+CSUM({1, 2, 3; 4, 5, 6; 7, 8, 9})
+ 12 15 18
+
+DESIGN({1, 2, 0; 2, 1, 0; 3, 0, 1})
+ 1 0 0 0 0 1 1 0
+ 0 1 0 0 1 0 1 0
+ 0 0 1 1 0 0 0 1
+
+warning: Column 2 in DESIGN argument has constant value.
+
+DESIGN({1, 2, 0; 2, 2, 0; 3, 2, 1})
+ 1 0 0 1 0
+ 0 1 0 1 0
+ 0 0 1 0 1
+
+DET({1, 2, 3; 4, 5, 6; 7, 8, 9})
+ .0
+
+DIAG({1, 2, 3, 4; 4, 5, 6, 7; 7, 8, 9, 10})
+ 1
+ 5
+ 9
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - EVAL EXP GINV GRADE GSCH])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT EVAL({2, 0, 0; 0, 3, 4; 0, 4, 9})/FORMAT=F5.2.
+
+PRINT EXP({2, 3; 4, 5})/FORMAT F5.2.
+
+PRINT GINV({1, 2})/FORMAT F5.2.
+COMPUTE a={1, 2, 3; 4, 5, 6; 7, 8, 9}.
+COMPUTE g=GINV(a).
+PRINT (a*g*a)/FORMAT F5.2.
+
+PRINT GRADE({1, 0, 3; 3, 1, 2; 3, 0, 5}).
+COMPUTE x={26, 690, 323, 208, 671, 818, 732, 711, 585, 792}.
+COMPUTE asort=x.
+COMPUTE asort(GRADE(asort))=asort.
+PRINT asort.
+COMPUTE dsort=x.
+COMPUTE dsort(GRADE(-dsort))=dsort.
+PRINT dsort.
+
+PRINT (GSCH({3, 2; 1, 2}) * SQRT(10))/FORMAT F5.2.
+PRINT (GSCH({0, 3, 6, 2; 0, 1, 2, 2}) * SQRT(10))/FORMAT F5.2.
+PRINT GSCH({0; 0}).
+PRINT GSCH({0, 0, 0; 0, 0, 0}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+EVAL({2, 0, 0; 0, 3, 4; 0, 4, 9})
+ 11.00
+ 2.00
+ 1.00
+
+EXP({2, 3; 4, 5})
+ 7.39 20.09
+ 54.60 148.4
+
+GINV({1, 2})
+ .20
+ .40
+
+(a*g*a)
+ 1.00 2.00 3.00
+ 4.00 5.00 6.00
+ 7.00 8.00 9.00
+
+GRADE({1, 0, 3; 3, 1, 2; 3, 0, 5})
+ 3 1 6
+ 7 4 5
+ 8 2 9
+
+asort
+ 26 208 323 585 671 690 711 732 792 818
+
+dsort
+ 818 792 732 711 690 671 585 323 208 26
+
+(GSCH({3, 2; 1, 2}) * SQRT(10))
+ 3.00 -1.00
+ 1.00 3.00
+
+(GSCH({0, 3, 6, 2; 0, 1, 2, 2}) * SQRT(10))
+ 3.00 -1.00
+ 1.00 3.00
+
+matrix.sps:22.12-22.17: error: MATRIX: GSCH requires its argument to have at
+least as many columns as rows, but it has dimensions 2×1.
+ 22 | PRINT GSCH({0; 0}).
+ | ^~~~~~
+
+matrix.sps:23.12-23.29: error: MATRIX: 2×3 argument to GSCH contains only 0
+linearly independent columns.
+ 23 | PRINT GSCH({0, 0, 0; 0, 0, 0}).
+ | ^~~~~~~~~~~~~~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - IDENT INV KRONEKER LG10 LN])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT IDENT(1).
+PRINT IDENT(2).
+PRINT IDENT(3,5).
+PRINT IDENT(5,3).
+
+PRINT INV({3, 3.5; 3.2, 3.6})/FORMAT F8.2.
+PRINT INV({4, 7; 2, 6})/FORMAT F8.2.
+PRINT (INV({4, -2, 1; 5, 0, 3; -1, 2, 6})*52)/FORMAT F8.2.
+
+PRINT KRONEKER({1, 2; 3, 4}, {0, 5; 6, 7}).
+
+PRINT LG10({1, 10, 100, 1000}).
+
+PRINT LN({1, 2; 3, 4})/FORMAT F5.2.
+PRINT LN(0).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+IDENT(1)
+ 1
+
+IDENT(2)
+ 1 0
+ 0 1
+
+IDENT(3,5)
+ 1 0 0 0 0
+ 0 1 0 0 0
+ 0 0 1 0 0
+
+IDENT(5,3)
+ 1 0 0
+ 0 1 0
+ 0 0 1
+ 0 0 0
+ 0 0 0
+
+INV({3, 3.5; 3.2, 3.6})
+ -9.00 8.75
+ 8.00 -7.50
+
+INV({4, 7; 2, 6})
+ .60 -.70
+ -.20 .40
+
+(INV({4, -2, 1; 5, 0, 3; -1, 2, 6})*52)
+ -6.00 14.00 -6.00
+ -33.00 25.00 -7.00
+ 10.00 -6.00 10.00
+
+KRONEKER({1, 2; 3, 4}, {0, 5; 6, 7})
+ 0 5 0 10
+ 6 7 12 14
+ 0 15 0 20
+ 18 21 24 28
+
+LG10({1, 10, 100, 1000})
+ 0 1 2 3
+
+LN({1, 2; 3, 4})
+ .00 .69
+ 1.10 1.39
+
+matrix.sps:16.7-16.11: error: MATRIX: Argument 1 to matrix function LN must be
+greater than 0.
+ 16 | PRINT LN(0).
+ | ^~~~~
+
+matrix.sps:16.10: note: MATRIX: Argument 1 is 0.
+ 16 | PRINT LN(0).
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MAGIC])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+
+LOOP n=3 to 10.
+ COMPUTE m=MAGIC(n).
+ COMPUTE total=n*(n**2 + 1) / 2.
+ COMPUTE tb={MSUM(DIAG(T(m))), CSUM(m), MSUM(DIAG(m))} - total.
+ COMPUTE lr=RSUM(m) - total.
+ PRINT {tb; lr, m, lr; tb}/FORMAT F4.0.
+END LOOP.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0
+ 0 8 1 6 0
+ 0 3 5 7 0
+ 0 4 9 2 0
+ 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0
+ 0 1 5 12 16 0
+ 0 15 11 6 2 0
+ 0 14 8 9 3 0
+ 0 4 10 7 13 0
+ 0 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0 0
+ 0 17 24 1 8 15 0
+ 0 23 5 7 14 16 0
+ 0 4 6 13 20 22 0
+ 0 10 12 19 21 3 0
+ 0 11 18 25 2 9 0
+ 0 0 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0 0 0
+ 0 1 5 9 28 32 36 0
+ 0 35 30 27 10 7 2 0
+ 0 24 14 22 18 17 16 0
+ 0 13 23 15 19 20 21 0
+ 0 34 31 26 11 6 3 0
+ 0 4 8 12 25 29 33 0
+ 0 0 0 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0 0 0 0
+ 0 30 39 48 1 10 19 28 0
+ 0 38 47 7 9 18 27 29 0
+ 0 46 6 8 17 26 35 37 0
+ 0 5 14 16 25 34 36 45 0
+ 0 13 15 24 33 42 44 4 0
+ 0 21 23 32 41 43 3 12 0
+ 0 22 31 40 49 2 11 20 0
+ 0 0 0 0 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0 0 0 0 0
+ 0 1 9 17 25 40 48 56 64 0
+ 0 63 55 47 39 26 18 10 2 0
+ 0 3 11 19 27 38 46 54 62 0
+ 0 61 53 45 37 28 20 12 4 0
+ 0 60 52 44 32 33 21 13 5 0
+ 0 6 14 22 30 35 43 51 59 0
+ 0 58 50 42 34 31 23 15 7 0
+ 0 8 16 24 36 29 41 49 57 0
+ 0 0 0 0 0 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0 0 0 0 0 0
+ 0 47 58 69 80 1 12 23 34 45 0
+ 0 57 68 79 9 11 22 33 44 46 0
+ 0 67 78 8 10 21 32 43 54 56 0
+ 0 77 7 18 20 31 42 53 55 66 0
+ 0 6 17 19 30 41 52 63 65 76 0
+ 0 16 27 29 40 51 62 64 75 5 0
+ 0 26 28 39 50 61 72 74 4 15 0
+ 0 36 38 49 60 71 73 3 14 25 0
+ 0 37 48 59 70 81 2 13 24 35 0
+ 0 0 0 0 0 0 0 0 0 0 0
+{tb; lr, m, lr; tb}
+ 0 0 0 0 0 0 0 0 0 0 0 0
+ 0 1 9 17 25 33 68 76 84 92 100 0
+ 0 99 91 83 75 67 34 26 18 10 2 0
+ 0 3 11 19 27 35 66 74 82 90 98 0
+ 0 97 89 81 72 65 36 29 20 12 4 0
+ 0 60 42 58 44 56 50 49 53 47 46 0
+ 0 41 59 43 57 45 51 52 48 54 55 0
+ 0 96 88 80 73 64 37 28 21 13 5 0
+ 0 6 14 22 30 38 63 71 79 87 95 0
+ 0 94 86 78 70 62 39 31 23 15 7 0
+ 0 8 16 24 32 40 61 69 77 85 93 0
+ 0 0 0 0 0 0 0 0 0 0 0 0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MAKE MDIAG MMAX MMIN MOD])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT MAKE(1, 2, 3).
+PRINT MAKE(2, 1, 4).
+PRINT MAKE(2, 3, 5).
+
+PRINT MDIAG({1, 2, 3, 4}).
+PRINT MDIAG({1; 2; 3; 4}).
+PRINT MDIAG({1, 2; 3, 4}).
+
+PRINT MMAX({55, 44; 66, 11}).
+
+PRINT MMIN({55, 44; 66, 11}).
+
+PRINT MOD({5, 4, 3, 2, 1, 0}, 3).
+PRINT MOD({5, 4, 3, 2, 1, 0}, -3).
+PRINT MOD({-5, -4, -3, -2, -1, 0}, 3).
+PRINT MOD({-5, -4, -3, -2, -1, 0}, -3).
+PRINT MOD({5, 4, 3, 2, 1, 0}, 1.5) /FORMAT F5.1.
+PRINT MOD({5, 4, 3, 2, 1, 0}, 0).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+MAKE(1, 2, 3)
+ 3 3
+
+MAKE(2, 1, 4)
+ 4
+ 4
+
+MAKE(2, 3, 5)
+ 5 5 5
+ 5 5 5
+
+MDIAG({1, 2, 3, 4})
+ 1 0 0 0
+ 0 2 0 0
+ 0 0 3 0
+ 0 0 0 4
+
+MDIAG({1; 2; 3; 4})
+ 1 0 0 0
+ 0 2 0 0
+ 0 0 3 0
+ 0 0 0 4
+
+matrix.sps:8.13-8.24: error: MATRIX: Function MDIAG argument 1 must be a
+vector, not a 2×2 matrix.
+ 8 | PRINT MDIAG({1, 2; 3, 4}).
+ | ^~~~~~~~~~~~
+
+MMAX({55, 44; 66, 11})
+ 66
+
+MMIN({55, 44; 66, 11})
+ 11
+
+MOD({5, 4, 3, 2, 1, 0}, 3)
+ 2 1 0 2 1 0
+
+MOD({5, 4, 3, 2, 1, 0}, -3)
+ 2 1 0 2 1 0
+
+MOD({-5, -4, -3, -2, -1, 0}, 3)
+ -2 -1 0 -2 -1 0
+
+MOD({-5, -4, -3, -2, -1, 0}, -3)
+ -2 -1 0 -2 -1 0
+
+MOD({5, 4, 3, 2, 1, 0}, 1.5)
+ .5 1.0 .0 .5 1.0 .0
+
+matrix.sps:19.7-19.32: error: MATRIX: Argument 2 to matrix function MOD must
+not be equal to 0.
+ 19 | PRINT MOD({5, 4, 3, 2, 1, 0}, 0).
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MSSQ MSUM NCOL NROW RANK])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT MSSQ({1, 0, 1; -2, -3, 1; 3, 3, 0}).
+
+PRINT MSUM({1, 0, 1; -2, -3, 1; 3, 3, 0}).
+
+PRINT NCOL({1, 0; -2, -3; 3, 3}).
+
+PRINT NROW({1, 0; -2, -3; 3, 3}).
+
+PRINT RANK({1, 0, 1; -2, -3, 1; 3, 3, 0}).
+PRINT RANK({1, 1, 0, 2; -1, -1, 0, -2}).
+PRINT RANK({1, -1; 1, -1; 0, 0; 2, -2}).
+PRINT RANK({1, 2, 1; -2, -3, 1; 3, 5, 0}).
+PRINT RANK({1, 0, 2; 2, 1, 0; 3, 2, 1}).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+MSSQ({1, 0, 1; -2, -3, 1; 3, 3, 0})
+ 34
+
+MSUM({1, 0, 1; -2, -3, 1; 3, 3, 0})
+ 4
+
+NCOL({1, 0; -2, -3; 3, 3})
+ 2
+
+NROW({1, 0; -2, -3; 3, 3})
+ 3
+
+RANK({1, 0, 1; -2, -3, 1; 3, 3, 0})
+ 2
+
+RANK({1, 1, 0, 2; -1, -1, 0, -2})
+ 1
+
+RANK({1, -1; 1, -1; 0, 0; 2, -2})
+ 1
+
+RANK({1, 2, 1; -2, -3, 1; 3, 5, 0})
+ 2
+
+RANK({1, 0, 2; 2, 1, 0; 3, 2, 1})
+ 3
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - RESHAPE RMAX RMIN RND RNKORDER])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT RESHAPE(1:12, 1, 12).
+PRINT RESHAPE(1:12, 2, 6).
+PRINT RESHAPE(1:12, 3, 4).
+PRINT RESHAPE(1:12, 4, 3).
+PRINT RESHAPE(1:12, 6, 2).
+PRINT RESHAPE(1:12, 12, 1).
+
+PRINT RMAX({1, 0, 1; -2, -3, 1; 3, 3, 0}).
+
+PRINT RMIN({1, 0, 1; -2, -3, 1; 3, 3, 0}).
+
+PRINT RND({-1.6, -1.5, -1.4;
+ -.6, -.5, -.4;
+ .4, .5, .6;
+ 1.4, 1.5, 1.6})/FORMAT F5.1.
+
+PRINT RNKORDER({1, 0, 3; 3, 1, 2; 3, 0, 5}) /FORMAT F5.1.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+RESHAPE(1:12, 1, 12)
+ 1 2 3 4 5 6 7 8 9 10 11 12
+
+RESHAPE(1:12, 2, 6)
+ 1 2 3 4 5 6
+ 7 8 9 10 11 12
+
+RESHAPE(1:12, 3, 4)
+ 1 2 3 4
+ 5 6 7 8
+ 9 10 11 12
+
+RESHAPE(1:12, 4, 3)
+ 1 2 3
+ 4 5 6
+ 7 8 9
+ 10 11 12
+
+RESHAPE(1:12, 6, 2)
+ 1 2
+ 3 4
+ 5 6
+ 7 8
+ 9 10
+ 11 12
+
+RESHAPE(1:12, 12, 1)
+ 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+
+RMAX({1, 0, 1; -2, -3, 1; 3, 3, 0})
+ 1
+ 1
+ 3
+
+RMIN({1, 0, 1; -2, -3, 1; 3, 3, 0})
+ 0
+ -3
+ 0
+
+RND({-1.6, -1.5, -1.4;
+ -.6, -.5, -.4;
+ .4, .5, .6;
+ 1.4, 1.5, 1.6})
+ -2.0 -2.0 -1.0
+ -1.0 .0 .0
+ .0 .0 1.0
+ 1.0 2.0 2.0
+
+RNKORDER({1, 0, 3; 3, 1, 2; 3, 0, 5})
+ 3.5 1.5 7.0
+ 7.0 3.5 5.0
+ 7.0 1.5 9.0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - RSSQ RSUM SIN SOLVE SQRT])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT RSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9}).
+PRINT RSUM({1, 2, 3; 4, 5, 6; 7, 8, 9}).
+
+PRINT SIN({0, .78, 1.57, 2.35, 3.14}) /FORMAT F5.2.
+
+PRINT SOLVE({2, 3; 4, 9}, {6, 2; 15, 5}) /FORMAT=F6.2.
+PRINT SOLVE({1, 3, -2; 3, 5, 6; 2, 4, 3}, {5; 7; 8}) /FORMAT=F6.2.
+PRINT SOLVE({2, 1, -1; -3, -1, 2; -2, 1, 2}, {8; -11; -3}) /FORMAT=F6.2.
+PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
+
+PRINT SQRT({0, 1, 2, 3, 4, 9, 81}) /FORMAT=F5.2.
+PRINT SQRT(-1).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+RSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9})
+ 14
+ 77
+ 194
+
+RSUM({1, 2, 3; 4, 5, 6; 7, 8, 9})
+ 6
+ 15
+ 24
+
+SIN({0, .78, 1.57, 2.35, 3.14})
+ .00 .70 1.00 .71 .00
+
+SOLVE({2, 3; 4, 9}, {6, 2; 15, 5})
+ 1.50 .50
+ 1.00 .33
+
+SOLVE({1, 3, -2; 3, 5, 6; 2, 4, 3}, {5; 7; 8})
+ -15.00
+ 8.00
+ 2.00
+
+SOLVE({2, 1, -1; -3, -1, 2; -2, 1, 2}, {8; -11; -3})
+ 2.00
+ 3.00
+ -1.00
+
+matrix.sps:10.7-10.33: error: MATRIX: SOLVE arguments must have the same number
+of rows.
+ 10 | PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+matrix.sps:10.13-10.24: note: MATRIX: Argument 1 has dimensions 2×2.
+ 10 | PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
+ | ^~~~~~~~~~~~
+
+matrix.sps:10.27-10.32: note: MATRIX: Argument 2 has dimensions 1×2.
+ 10 | PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
+ | ^~~~~~
+
+SQRT({0, 1, 2, 3, 4, 9, 81})
+ .00 1.00 1.41 1.73 2.00 3.00 9.00
+
+matrix.sps:13.7-13.14: error: MATRIX: Argument 1 to matrix function SQRT must
+be greater than or equal to 0.
+ 13 | PRINT SQRT(-1).
+ | ^~~~~~~~
+
+matrix.sps:13.12-13.13: note: MATRIX: Argument 1 is -1.
+ 13 | PRINT SQRT(-1).
+ | ^~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - SSCP SVAL SWEEP TRACE TRANSPOS TRUNC])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE m={1, 2, 3; 4, 5, 6}
+COMPUTE sscp1=SSCP(m).
+COMPUTE sscp2=T(m)*m.
+PRINT sscp1.
+PRINT (sscp1 <> sscp2).
+
+PRINT SVAL({1, 1; 0, 0})/FORMAT F5.2.
+PRINT SVAL({1, 0, 1; 0, 1, 1; 0, 0, 0})/FORMAT F5.2.
+PRINT SVAL({1, 0, 0, 0, 2; 0, 0, 3, 0, 0; 0, 0, 0, 0, 0; 0, 2, 0, 0, 0})
+ /FORMAT F5.2.
+PRINT SVAL({2, 4; 1, 3; 0, 0; 0, 0})/FORMAT F5.2.
+
+COMPUTE s0 = {6, 12, 0, 12; 12, 28, 0, 25; 0, 0, 6, 2; 12, 25, 2, 28}.
+PRINT SWEEP(s0, 1)/FORMAT F5.2.
+PRINT SWEEP(SWEEP(s0, 1), 2)/FORMAT F5.2.
+PRINT SWEEP(SWEEP(SWEEP(s0, 1), 2), 3)/FORMAT F5.2.
+
+COMPUTE s1 = {6, 12, 0, 12; 12, 0, 0, 25; 0, 0, 6, 2; 12, 25, 2, 28}.
+PRINT SWEEP(s1, 2).
+
+COMPUTE s2 = {0, 1, 2; 3, 4, 5; 6, 7, 8}.
+PRINT SWEEP(s2, 1).
+PRINT SWEEP(s2, 2).
+PRINT SWEEP(s2, 3).
+
+PRINT TRACE(s0).
+
+PRINT T(s0).
+PRINT TRANSPOS(s0).
+PRINT ALL(T(T(s0)) = s0).
+
+PRINT TRUNC(SVAL({2, 4; 1, 3; 0, 0; 0, 0})).
+PRINT TRUNC(-SVAL({2, 4; 1, 3; 0, 0; 0, 0})).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+sscp1
+ 17 22 27
+ 22 29 36
+ 27 36 45
+
+(sscp1 <> sscp2)
+ 0 0 0
+ 0 0 0
+ 0 0 0
+
+SVAL({1, 1; 0, 0})
+ 1.41
+ .00
+
+SVAL({1, 0, 1; 0, 1, 1; 0, 0, 0})
+ 1.73
+ 1.00
+ .00
+
+SVAL({1, 0, 0, 0, 2; 0, 0, 3, 0, 0; 0, 0, 0, 0, 0; 0, 2, 0, 0, 0})
+ 3.00
+ 2.24
+ 2.00
+ .00
+
+SVAL({2, 4; 1, 3; 0, 0; 0, 0})
+ 5.46
+ .37
+
+SWEEP(s0, 1)
+ .17 2.00 .00 2.00
+ -2.00 4.00 .00 1.00
+ .00 .00 6.00 2.00
+ -2.00 1.00 2.00 4.00
+
+SWEEP(SWEEP(s0, 1), 2)
+ 1.17 -.50 .00 1.50
+ -.50 .25 .00 .25
+ .00 .00 6.00 2.00
+ -1.50 -.25 2.00 3.75
+
+SWEEP(SWEEP(SWEEP(s0, 1), 2), 3)
+ 1.17 -.50 .00 1.50
+ -.50 .25 .00 .25
+ .00 .00 .17 .33
+ -1.50 -.25 -.33 3.08
+
+SWEEP(s1, 2)
+ 6 0 0 12
+ 0 0 0 0
+ 0 0 6 2
+ 12 0 2 28
+
+SWEEP(s2, 1)
+ 0 0 0
+ 0 4 5
+ 0 7 8
+
+SWEEP(s2, 2)
+ -.7500000000 -.2500000000 .7500000000
+ .7500000000 .2500000000 1.2500000000
+ .7500000000 -1.7500000000 -.7500000000
+
+SWEEP(s2, 3)
+ -1.5000000000 -.7500000000 -.2500000000
+ -.7500000000 -.3750000000 -.6250000000
+ .7500000000 .8750000000 .1250000000
+
+TRACE(s0)
+ 68
+
+T(s0)
+ 6 12 0 12
+ 12 28 0 25
+ 0 0 6 2
+ 12 25 2 28
+
+TRANSPOS(s0)
+ 6 12 0 12
+ 12 28 0 25
+ 0 0 6 2
+ 12 25 2 28
+
+ALL(T(T(s0)) = s0)
+ 1
+
+TRUNC(SVAL({2, 4; 1, 3; 0, 0; 0, 0}))
+ 5
+ 0
+
+TRUNC(-SVAL({2, 4; 1, 3; 0, 0; 0, 0}))
+ -5
+ 0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - UNIFORM])
+AT_DATA([matrix.sps], [dnl
+SET SEED=10.
+MATRIX.
+PRINT (UNIFORM(4, 5)*10)/FORMAT F5.2.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+(UNIFORM(4, 5)*10)
+ 7.71 2.99 .21 4.95 6.34
+ 4.43 7.49 8.32 4.99 5.83
+ 2.25 .25 1.98 7.09 7.61
+ 2.66 1.69 2.64 .88 1.50
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - invalid function arguments])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE x=MOD({1,2,3},{4,5,6}).
+COMPUTE x=MDIAG({1, 2; 3, 4}).
+COMPUTE x=ARSIN(2).
+COMPUTE x=ARSIN({1, 1; -1, 2}).
+COMPUTE x=CDF.UNIFORM(2,1,1).
+COMPUTE x=CDF.UNIFORM(1,2,1).
+COMPUTE x=CDF.UNIFORM({1,2},1,1).
+COMPUTE x=MAGIC(2).
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.23-2.29: error: MATRIX: Function MOD argument 2 must be a scalar,
+not a 1×3 matrix.
+ 2 | COMPUTE x=MOD({1,2,3},{4,5,6}).
+ | ^~~~~~~
+
+matrix.sps:3.17-3.28: error: MATRIX: Function MDIAG argument 1 must be a
+vector, not a 2×2 matrix.
+ 3 | COMPUTE x=MDIAG({1, 2; 3, 4}).
+ | ^~~~~~~~~~~~
+
+matrix.sps:4.17: error: MATRIX: Argument 1 to matrix function ARSIN is 2, which
+is outside the valid range [[-1,1]].
+ 4 | COMPUTE x=ARSIN(2).
+ | ^
+
+matrix.sps:5.17-5.29: error: MATRIX: Row 2, column 2 of argument 1 to matrix
+function ARSIN is 2, which is outside the valid range [[-1,1]].
+ 5 | COMPUTE x=ARSIN({1, 1; -1, 2}).
+ | ^~~~~~~~~~~~~
+
+error: Argument 1 to matrix function CDF.UNIFORM must be less than or equal to
+argument 3.
+
+matrix.sps:6.23: note: MATRIX: Argument 1 is 2.
+ 6 | COMPUTE x=CDF.UNIFORM(2,1,1).
+ | ^
+
+matrix.sps:6.27: note: MATRIX: Argument 3 is 1.
+ 6 | COMPUTE x=CDF.UNIFORM(2,1,1).
+ | ^
+
+error: Argument 2 to matrix function CDF.UNIFORM must be less than or equal to
+argument 3.
+
+matrix.sps:7.25: note: MATRIX: Argument 2 is 2.
+ 7 | COMPUTE x=CDF.UNIFORM(1,2,1).
+ | ^
+
+matrix.sps:7.27: note: MATRIX: Argument 3 is 1.
+ 7 | COMPUTE x=CDF.UNIFORM(1,2,1).
+ | ^
+
+error: Argument 1 to matrix function CDF.UNIFORM must be less than or equal to
+argument 3.
+
+matrix.sps:8.23-8.27: note: MATRIX: Row 1, column 2 of argument 1 is 2.
+ 8 | COMPUTE x=CDF.UNIFORM({1,2},1,1).
+ | ^~~~~
+
+matrix.sps:8.31: note: MATRIX: Argument 3 is 1.
+ 8 | COMPUTE x=CDF.UNIFORM({1,2},1,1).
+ | ^
+
+matrix.sps:9.11-9.18: error: MATRIX: Argument 1 to matrix function MAGIC must
+be greater than or equal to 3.
+ 9 | COMPUTE x=MAGIC(2).
+ | ^~~~~~~~
+
+matrix.sps:9.17: note: MATRIX: Argument 1 is 2.
+ 9 | COMPUTE x=MAGIC(2).
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - invalid number function arguments])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE x=ABS().
+COMPUTE x=ABS(1,2).
+COMPUTE x=KRONEKER(1,2,3).
+COMPUTE x=IDENT().
+COMPUTE x=IDENT(1,2,3).
+COMPUTE x=BLOCK().
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2: error: COMPUTE: Matrix function ABS requires 1 argument.
+
+matrix.sps:3: error: COMPUTE: Matrix function ABS requires 1 argument.
+
+matrix.sps:4: error: COMPUTE: Matrix function KRONEKER requires 2 arguments.
+
+matrix.sps:5: error: COMPUTE: Matrix function IDENT requires 1 or 2 arguments,
+but 0 were provided.
+
+matrix.sps:6: error: COMPUTE: Matrix function IDENT requires 1 or 2 arguments,
+but 3 were provided.
+
+matrix.sps:7: error: COMPUTE: Matrix function BLOCK requires at least one
+argument.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - CALL SETDIAG])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE x={1, 2, 3; 4, 5, 6; 7, 8, 9}.
+
+COMPUTE x1=x.
+CALL SETDIAG(x1, 10).
+PRINT x1.
+
+COMPUTE x2=x.
+CALL SETDIAG(x2, {10, 11}).
+PRINT x2.
+
+COMPUTE x3=x.
+CALL SETDIAG(x3, {10, 11, 12}).
+PRINT x3.
+
+COMPUTE x4=x.
+CALL SETDIAG(x4, {10, 11, 12, 13}).
+PRINT x4.
+
+COMPUTE x5=x.
+CALL SETDIAG(x5, {10, 11; 12, 13}).
+PRINT x5.
+
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+x1
+ 10 2 3
+ 4 10 6
+ 7 8 10
+
+x2
+ 10 2 3
+ 4 11 6
+ 7 8 9
+
+x3
+ 10 2 3
+ 4 11 6
+ 7 8 12
+
+x4
+ 10 2 3
+ 4 11 6
+ 7 8 12
+
+matrix.sps:21.18-21.33: error: MATRIX: SETDIAG argument 2 must be a scalar or a
+vector, not a 2×2 matrix.
+ 21 | CALL SETDIAG(x5, {10, 11; 12, 13}).
+ | ^~~~~~~~~~~~~~~~
+
+x5
+ 1 2 3
+ 4 5 6
+ 7 8 9
+])
+AT_CLEANUP
+
+dnl I have some doubts about the correctness of the results below.
+AT_SETUP([MATRIX - CALL EIGEN])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+CALL EIGEN({1, 0; 0, 1}, evec, eval).
+PRINT evec.
+PRINT eval.
+
+CALL EIGEN({3, 2, 4; 2, 0, 2; 4, 2, 3}, evec2, eval2).
+PRINT evec2.
+PRINT eval2.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+evec
+ 1 0
+ 0 1
+
+eval
+ 1
+ 1
+
+evec2
+ -.6666666667 .0000000000 .7453559925
+ -.3333333333 -.8944271910 -.2981423970
+ -.6666666667 .4472135955 -.5962847940
+
+eval2
+ 8.0000000000
+ -1.0000000000
+ -1.0000000000
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - CALL SVD])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+CALL SVD({3, 2, 2; 2, 3, -2}, u, s, v).
+PRINT (u * s * T(v))/FORMAT F5.1.
+
+CALL SVD({2, 4; 1, 3; 0, 0; 0, 0}, u, s, v).
+PRINT (u*s*T(v))/FORMAT F5.1.
+
+CALL SVD({-3, 1; 6, -2; 6, -2}, u, s, v).
+PRINT (u*s*T(v))/FORMAT F5.1.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+(u * s * T(v))
+ 3.0 2.0 2.0
+ 2.0 3.0 -2.0
+
+(u*s*T(v))
+ 2.0 4.0
+ 1.0 3.0
+ .0 .0
+ .0 .0
+
+(u*s*T(v))
+ -3.0 1.0
+ 6.0 -2.0
+ 6.0 -2.0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - PRINT])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+PRINT/TITLE="title 1".
+PRINT/SPACE=2/TITLE="title 2".
+
+COMPUTE m={1, 2, 3; 3, 4, 5; 6, 7, 8}.
+PRINT m/RLABELS=123, a b c, long name.
+PRINT m/RNAMES={'123', 'a b c', 'long name'}.
+PRINT m/CLABELS=col1, col2, long column name.
+PRINT m/CNAMES={'col1', 'col2', 'long column name'}.
+PRINT m/RLABELS=123, a b c, long name
+ /CLABELS=col1, col2, long column name.
+PRINT m/RNAMES={'123', 'a b c', 'long name'}
+ /CNAMES={'col1', 'col2', 'long column name'}.
+PRINT {123e10, 456e10, 500}.
+END MATRIX.
+])
+
+AT_DATA([matrix-tables.sps], [dnl
+SET MDISPLAY=TABLES.
+INCLUDE 'matrix.sps'.
+])
+
+AT_CHECK([pspp matrix.sps], [0], [dnl
+title 1
+
+
+
+title 2
+
+m
+123 1 2 3
+a b c 3 4 5
+long nam 6 7 8
+
+m
+123 1 2 3
+a b c 3 4 5
+long nam 6 7 8
+
+m
+ col1 col2 long col
+ 1 2 3
+ 3 4 5
+ 6 7 8
+
+m
+ col1 col2 long col
+ 1 2 3
+ 3 4 5
+ 6 7 8
+
+m
+ col1 col2 long col
+123 1 2 3
+a b c 3 4 5
+long nam 6 7 8
+
+m
+ col1 col2 long col
+123 1 2 3
+a b c 3 4 5
+long nam 6 7 8
+
+{123e10, 456e10, 500}
+ 10 ** 12 X
+ 1.2300000000 4.5600000000 .0000000005
+])
+
+AT_CHECK([pspp matrix-tables.sps], [0], [dnl
+title 1
+
+
+
+title 2
+
+ m
++---------+-----+
+|123 |1 2 3|
+|a b c |3 4 5|
+|long name|6 7 8|
++---------+-----+
+
+ m
++--------+-----+
+|123 |1 2 3|
+|a b c |3 4 5|
+|long nam|6 7 8|
++--------+-----+
+
+ m
++----+----+----------------+
+|col1|col2|long column name|
++----+----+----------------+
+| 1| 2| 3|
+| 3| 4| 5|
+| 6| 7| 8|
++----+----+----------------+
+
+ m
++----+----+--------+
+|col1|col2|long col|
++----+----+--------+
+| 1| 2| 3|
+| 3| 4| 5|
+| 6| 7| 8|
++----+----+--------+
+
+ m
++---------+----+----+----------------+
+| |col1|col2|long column name|
++---------+----+----+----------------+
+|123 | 1| 2| 3|
+|a b c | 3| 4| 5|
+|long name| 6| 7| 8|
++---------+----+----+----------------+
+
+ m
++--------+----+----+--------+
+| |col1|col2|long col|
++--------+----+----+--------+
+|123 | 1| 2| 3|
+|a b c | 3| 4| 5|
+|long nam| 6| 7| 8|
++--------+----+----+--------+
+
+ {123e10, 456e10, 500}
++----------------------------------------------+
+|1.2300000000[[a]] 4.5600000000[[a]] .0000000005[[a]]|
++----------------------------------------------+
+a. × 10**12
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - DO IF])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+DO IF 1.
+PRINT/TITLE '1'.
+END IF.
+
+DO IF 0.
+PRINT/TITLE '2'.
+ELSE IF 1.
+PRINT/TITLE '3'.
+END IF.
+
+DO IF -1.
+PRINT/TITLE '4'.
+ELSE IF 0.
+PRINT/TITLE '5'.
+ELSE.
+PRINT/TITLE '6'.
+END IF.
+
+DO IF {1, 2}.
+END IF.
+
+DO IF 0.
+ELSE IF {}.
+END IF.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+1
+
+3
+
+6
+
+matrix.sps:20.7-20.12: error: MATRIX: Expression for DO IF must evaluate to
+scalar, not a 1×2 matrix.
+ 20 | DO IF {1, 2}.
+ | ^~~~~~
+
+matrix.sps:24.9-24.10: error: MATRIX: Expression for ELSE IF must evaluate to
+scalar, not a 0×0 matrix.
+ 24 | ELSE IF {}.
+ | ^~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - unbounded LOOP])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+* Truly unbounded loop.
+COMPUTE x=0.
+COMPUTE y={}.
+LOOP.
+COMPUTE x=x+1.
+COMPUTE y={y, x}.
+END LOOP.
+PRINT x.
+PRINT y.
+
+* Unbounded loop terminates with BREAK.
+COMPUTE x=0.
+COMPUTE y={}.
+LOOP.
+COMPUTE x=x+1.
+COMPUTE y={y, x}.
+DO IF x >= 20.
+ BREAK.
+END IF.
+END LOOP.
+PRINT x.
+PRINT y.
+
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+x
+ 40
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
+40
+
+x
+ 20
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+20
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - indexed or conditional LOOP])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+* Indexed loop terminates based on index.
+COMPUTE y={}.
+LOOP x=1 TO 20.
+COMPUTE y={y, x}.
+END LOOP.
+PRINT x.
+PRINT y.
+
+* Indexed loop terminates based on MXLOOPS.
+COMPUTE y={}.
+LOOP x=1 TO 50.
+COMPUTE y={y, x}.
+END LOOP.
+PRINT x.
+PRINT y.
+
+* Indexed loop terminates with BREAK.
+COMPUTE y={}.
+LOOP x=1 TO 50.
+COMPUTE y={y, x}.
+DO IF x >= 20.
+ BREAK.
+END IF.
+END LOOP.
+PRINT x.
+PRINT y.
+
+* Indexed loop terminates with top IF.
+COMPUTE y={}.
+LOOP x=1 TO 50 IF NCOL(y) < 15.
+COMPUTE y={y, x}.
+END LOOP.
+PRINT x.
+PRINT y.
+
+* Indexed loop terminates with bottom IF.
+COMPUTE y={}.
+LOOP x=1 TO 50.
+COMPUTE y={y, x}.
+END LOOP IF NCOL(y) >= 22.
+PRINT x.
+PRINT y.
+
+* Index behavior.
+COMPUTE indexing={
+ 1, 10, 1;
+ 1, 10, 2;
+ 1, 10, 3;
+ 1, 10, -1;
+ 1, 10, 0;
+ 10, 1, -1;
+ 10, 1, -2;
+ 10, 1, -3;
+ 10, 1, 1;
+ 10, 1, 0
+}.
+LOOP i=1 TO NROW(indexing).
+ COMPUTE y={}.
+ LOOP j=indexing(i, 1) TO indexing(i, 2) BY indexing(i, 3).
+ COMPUTE y={y, j}.
+ END LOOP.
+ PRINT {indexing(i, :), y}.
+END LOOP.
+
+LOOP i={} TO 5.
+END LOOP.
+
+LOOP i=5 TO {}.
+END LOOP.
+
+LOOP i=5 TO 8 BY {}.
+END LOOP.
+
+LOOP IF {}.
+END LOOP.
+
+LOOP.
+END LOOP IF {}.
+
+LOOP i=1e100 to 1e200.
+END LOOP.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+x
+ 20
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+20
+
+x
+ 40
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
+40
+
+x
+ 20
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+20
+
+x
+ 16
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
+
+x
+ 22
+
+y
+ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
+20 21 22
+
+{indexing(i, :), y}
+ 1 10 1 1 2 3 4 5 6 7 8 9 10
+{indexing(i, :), y}
+ 1 10 2 1 3 5 7 9
+{indexing(i, :), y}
+ 1 10 3 1 4 7 10
+{indexing(i, :), y}
+ 1 10 -1
+{indexing(i, :), y}
+ 1 10 0
+{indexing(i, :), y}
+ 10 1 -1 10 9 8 7 6 5 4 3 2 1
+{indexing(i, :), y}
+ 10 1 -2 10 8 6 4 2
+{indexing(i, :), y}
+ 10 1 -3 10 7 4 1
+{indexing(i, :), y}
+ 10 1 1
+{indexing(i, :), y}
+ 10 1 0
+
+matrix.sps:66.8-66.9: error: MATRIX: Expression for LOOP must evaluate to
+scalar, not a 0×0 matrix.
+ 66 | LOOP i={} TO 5.
+ | ^~
+
+matrix.sps:69.13-69.14: error: MATRIX: Expression for TO must evaluate to
+scalar, not a 0×0 matrix.
+ 69 | LOOP i=5 TO {}.
+ | ^~
+
+matrix.sps:72.18-72.19: error: MATRIX: Expression for BY must evaluate to
+scalar, not a 0×0 matrix.
+ 72 | LOOP i=5 TO 8 BY {}.
+ | ^~
+
+matrix.sps:75.9-75.10: error: MATRIX: Expression for LOOP IF must evaluate to
+scalar, not a 0×0 matrix.
+ 75 | LOOP IF {}.
+ | ^~
+
+matrix.sps:79.13-79.14: error: MATRIX: Expression for END LOOP IF must evaluate
+to scalar, not a 0×0 matrix.
+ 79 | END LOOP IF {}.
+ | ^~
+
+matrix.sps:81.8-81.12: error: MATRIX: Expression for LOOP is outside the
+integer range.
+ 81 | LOOP i=1e100 to 1e200.
+ | ^~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - BREAK outside LOOP])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+BREAK.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.1-2.5: error: BREAK: BREAK not inside LOOP.
+ 2 | BREAK.
+ | ^~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - READ])
+AT_DATA([matrix.txt], [dnl
+9
+8
+7
+6
+1 2 3
+4 5 6
+7 8 9
+10 11 12,13
+14, 15 ,16 , 17
+18
+19
+20 21 22 23
+ 12 34
+5 6
+ 78 89
+10 11
+$1 $2 3
+4 $5 6
+$1 $2 $3 4
+ $5$6 $78
+1% 2% 3% 4
+ 56% 7%8
+abcdefghijkl
+ABCDEFGHIJKL
+])
+AT_DATA([matrix2.txt], [dnl
+2, 3, 5, 7
+11, 13, 17, 19
+23, 29, 31, 37
+41, 43, 47, 53
+])
+AT_DATA([matrix3.txt], [dnl
+1 5
+3 1 2 3
+5 6 -1 2 5 1
+2 8 9
+3 1 3 2
+])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+READ x/FILE='matrix.txt'/SIZE=4/FIELD=1 TO 1.
+PRINT x.
+READ x/FILE='matrix.txt'/SIZE={3,3}/FIELD=1 TO 80.
+PRINT x.
+READ x/SIZE={2,4}/FIELD=1 TO 80.
+PRINT x.
+READ x(:,2)/FILE='matrix.txt'/FIELD=1 TO 80.
+PRINT x.
+READ x(1,:)/SIZE={1,4}/FIELD=1 TO 80.
+PRINT x.
+
+READ x/SIZE={2,6}/FIELD=1 TO 20 BY 5.
+PRINT x.
+READ x/SIZE={2,3}/FIELD=1 TO 20/FORMAT=DOLLAR.
+PRINT x.
+READ x/SIZE={2,4}/FIELD=1 TO 20/FORMAT=DOLLAR5.1.
+PRINT x.
+READ x/SIZE={2,4}/FIELD=1 TO 12/FORMAT='4PCT'.
+PRINT x.
+READ x/SIZE={2,4}/FIELD=1 TO 12/FORMAT='4A'.
+PRINT x/FORMAT=A3.
+
+COMPUTE y={}.
+LOOP IF NOT EOF('matrix2.txt').
+READ x/FILE='matrix2.txt'/SIZE={1,4}/FIELD=1 TO 80.
+COMPUTE y={y; x}.
+END LOOP.
+PRINT y.
+
+COMPUTE m = MAKE(5, 5, 0).
+LOOP i = 1 TO 5.
+READ count /FILE='matrix3.txt' /FIELD=1 TO 1 /SIZE=1.
+READ m(i, 1:count) /FIELD=3 TO 100 /REREAD.
+END LOOP.
+PRINT m.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [0], [dnl
+x
+ 9
+ 8
+ 7
+ 6
+
+x
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+x
+ 10 11 12 13
+ 14 15 16 17
+
+x
+ 10 18 12 13
+ 14 19 16 17
+
+x
+ 20 21 22 23
+ 14 19 16 17
+
+x
+ 1 2 3 4 5 6
+ 7 8 8 9 10 11
+
+x
+ 1 2 3
+ 4 5 6
+
+x
+ 1 2 3 4
+ 5 6 7 8
+
+x
+ 1 2 3 4
+ 5 6 7 8
+
+x
+ abc def ghi jkl
+ ABC DEF GHI JKL
+
+y
+ 2 3 5 7
+ 11 13 17 19
+ 23 29 31 37
+ 41 43 47 53
+
+m
+ 5 0 0 0 0
+ 1 2 3 0 0
+ 6 -1 2 5 1
+ 8 9 0 0 0
+ 1 3 2 0 0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - READ - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+READ !.
+READ x/FILE=!.
+READ x/ENCODING=!.
+READ x/FIELD=!.
+READ x/FIELD=1 !.
+READ x/FIELD=1 TO !.
+READ x/FIELD=1 TO 0.
+READ x/FIELD=1 TO 10 BY !.
+READ x/FIELD=1 TO 10 BY 6.
+READ x/SIZE=!.
+READ x/MODE=!.
+READ x/FORMAT=!.
+READ x/FORMAT=F8.2/FORMAT=F8.2.
+READ x/FORMAT='5XYZZY'.
+READ x/FORMAT=XYZZY.
+READ x/!.
+READ x.
+READ x/FIELD=1 TO 10.
+READ x/FIELD=1 TO 10/SIZE={1,2}.
+READ x/FIELD=1 TO 10/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='15F'.
+READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT=F5.
+READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
+READ x/FIELD=1 TO 10/SIZE={1,2;3,4}/FILE='matrix.txt'.
+READ x/FIELD=1 TO 10/SIZE={1,2,3}/FILE='matrix.txt'.
+READ x/FIELD=1 TO 10/SIZE={-1}/FILE='matrix.txt'.
+COMPUTE x={1,2,3}.
+READ x(:,:)/FIELD=1 TO 10/SIZE={2,2}/FILE='matrix.txt'.
+READ x/FIELD=1 TO 10/SIZE={1,3}/FILE='matrix.txt'/MODE=SYMMETRIC.
+READ x/FIELD=1 TO 10/SIZE=2/FILE='matrix.txt'.
+END MATRIX.
+])
+AT_DATA([matrix.txt], [dnl
+xyzzy
+.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.6: error: READ: Syntax error expecting identifier.
+ 2 | READ !.
+ | ^
+
+matrix.sps:3.13: error: READ: Syntax error expecting a file name or handle
+name.
+ 3 | READ x/FILE=!.
+ | ^
+
+matrix.sps:4.17: error: READ: Syntax error expecting string.
+ 4 | READ x/ENCODING=!.
+ | ^
+
+matrix.sps:5.14: error: READ: Syntax error expecting positive integer for
+FIELD.
+ 5 | READ x/FIELD=!.
+ | ^
+
+matrix.sps:6.16: error: READ: Syntax error expecting `TO'.
+ 6 | READ x/FIELD=1 !.
+ | ^
+
+matrix.sps:7.19: error: READ: Syntax error expecting positive integer for TO.
+ 7 | READ x/FIELD=1 TO !.
+ | ^
+
+matrix.sps:8.19: error: READ: Syntax error expecting positive integer for TO.
+ 8 | READ x/FIELD=1 TO 0.
+ | ^
+
+matrix.sps:9.25: error: READ: Syntax error expecting integer between 1 and 10
+for BY.
+ 9 | READ x/FIELD=1 TO 10 BY !.
+ | ^
+
+matrix.sps:10.14-10.25: error: READ: Field width 6 does not evenly divide
+record width 10.
+ 10 | READ x/FIELD=1 TO 10 BY 6.
+ | ^~~~~~~~~~~~
+
+matrix.sps:10.14-10.20: note: READ: This syntax designates the record width.
+ 10 | READ x/FIELD=1 TO 10 BY 6.
+ | ^~~~~~~
+
+matrix.sps:10.25: note: READ: This syntax specifies the field width.
+ 10 | READ x/FIELD=1 TO 10 BY 6.
+ | ^
+
+matrix.sps:11.13: error: READ: Syntax error expecting matrix expression.
+ 11 | READ x/SIZE=!.
+ | ^
+
+matrix.sps:12.13: error: READ: Syntax error expecting RECTANGULAR or SYMMETRIC.
+ 12 | READ x/MODE=!.
+ | ^
+
+matrix.sps:13.15: error: READ: Syntax error expecting identifier.
+ 13 | READ x/FORMAT=!.
+ | ^
+
+matrix.sps:14.20-14.25: error: READ: Subcommand FORMAT may only be specified
+once.
+ 14 | READ x/FORMAT=F8.2/FORMAT=F8.2.
+ | ^~~~~~
+
+matrix.sps:15.15-15.22: error: READ: Unknown format XYZZY.
+ 15 | READ x/FORMAT='5XYZZY'.
+ | ^~~~~~~~
+
+matrix.sps:16.15-16.19: error: READ: Unknown format type `XYZZY'.
+ 16 | READ x/FORMAT=XYZZY.
+ | ^~~~~
+
+matrix.sps:17.8: error: READ: Syntax error expecting FILE, FIELD, MODE, REREAD,
+or FORMAT.
+ 17 | READ x/!.
+ | ^
+
+matrix.sps:18.1-18.7: error: READ: Required subcommand FIELD was not specified.
+ 18 | READ x.
+ | ^~~~~~~
+
+matrix.sps:19: error: READ: SIZE is required for reading data into a full
+matrix (as opposed to a submatrix).
+
+matrix.sps:19.6: note: READ: This expression designates a full matrix.
+ 19 | READ x/FIELD=1 TO 10.
+ | ^
+
+matrix.sps:20.1-20.32: error: READ: Required subcommand FILE was not specified.
+ 20 | READ x/FIELD=1 TO 10/SIZE={1,2}.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+matrix.sps:21: error: READ: 15 repetitions cannot fit in record width 10.
+
+matrix.sps:21.57-21.61: note: READ: This syntax designates the number of
+repetitions.
+ 21 | READ x/FIELD=1 TO 10/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='15F'.
+ | ^~~~~
+
+matrix.sps:21.14-21.20: note: READ: This syntax designates the record width.
+ 21 | READ x/FIELD=1 TO 10/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='15F'.
+ | ^~~~~~~
+
+matrix.sps:22: error: READ: This command specifies two different field widths.
+
+matrix.sps:22.62-22.63: note: READ: This syntax specifies field width 5.
+ 22 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT=F5.
+ | ^~
+
+matrix.sps:22.25: note: READ: This syntax specifies field width 2.
+ 22 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT=F5.
+ | ^
+
+matrix.sps:23: error: READ: This command specifies two different field widths.
+
+matrix.sps:23.62-23.65: note: READ: This syntax specifies 2 repetitions.
+ 23 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
+ | ^~~~
+
+matrix.sps:23.14-23.20: note: READ: This syntax designates record width 10,
+which divided by 2 repetitions implies field width 5.
+ 23 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
+ | ^~~~~~~
+
+matrix.sps:23.25: note: READ: This syntax specifies field width 2.
+ 23 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
+ | ^
+
+matrix.sps:24.27-24.35: error: MATRIX: SIZE must evaluate to a scalar or a 2-
+element vector, not a 2×2 matrix.
+ 24 | READ x/FIELD=1 TO 10/SIZE={1,2;3,4}/FILE='matrix.txt'.
+ | ^~~~~~~~~
+
+matrix.sps:25.27-25.33: error: MATRIX: SIZE must evaluate to a scalar or a 2-
+element vector, not a 1×3 matrix.
+ 25 | READ x/FIELD=1 TO 10/SIZE={1,2,3}/FILE='matrix.txt'.
+ | ^~~~~~~
+
+matrix.sps:26.28-26.29: error: MATRIX: Matrix dimensions -1×1 specified on SIZE
+are outside valid range.
+ 26 | READ x/FIELD=1 TO 10/SIZE={-1}/FILE='matrix.txt'.
+ | ^~
+
+matrix.sps:28: error: MATRIX: Dimensions specified on SIZE differ from
+dimensions of destination submatrix.
+
+matrix.sps:28.32-28.36: note: MATRIX: SIZE specifies dimensions 2×2.
+ 28 | READ x(:,:)/FIELD=1 TO 10/SIZE={2,2}/FILE='matrix.txt'.
+ | ^~~~~
+
+matrix.sps:28.6-28.11: note: MATRIX: Destination submatrix has dimensions 1×3.
+ 28 | READ x(:,:)/FIELD=1 TO 10/SIZE={2,2}/FILE='matrix.txt'.
+ | ^~~~~~
+
+matrix.sps:29: error: MATRIX: Cannot read non-square 1×3 matrix using READ with
+MODE=SYMMETRIC.
+
+matrix.txt:1.1-1.5: warning: Error reading "xyzzy" as format F for matrix row
+1, column 1: Field contents are not numeric.
+
+matrix.txt:2.1: warning: Error reading "." as format F for matrix row 2, column
+1: Matrix data may not contain missing value.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - WRITE])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+WRITE {1.5, 2; 3, 4.12345}/OUTFILE='matrix.txt'/FIELD=1 TO 80.
+WRITE {1.5, 2; 3, 4.12345}/OUTFILE='matrix.txt'/FIELD=1 TO 5.
+WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 80 BY 5.
+WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=F8.2.
+WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=E.
+WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 10 BY 10/FORMAT=E.
+WRITE "abcdefhi"/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=A8.
+WRITE "abcdefhi"/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=A4.
+WRITE "abcdefhi"/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=AHEX12.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps])
+AT_CHECK([cat matrix.txt], [0], [dnl
+ 1.5 2
+ 3 4.12345
+ 1.5 2
+ 3
+ 4.12345
+ 1 2
+ 3 4
+ 1.00 2.00
+ 3.00 4.00
+ 1 2
+ 3 4
+ 1.E+000
+ 2.E+000
+ 3.E+000
+ 4.E+000
+ abcdefhi
+ abcd
+ 616263646566
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - WRITE - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+WRITE !.
+WRITE 1/OUTFILE=!.
+WRITE 1/ENCODING=!.
+WRITE 1/FIELD=!.
+WRITE 1/FIELD=1 !.
+WRITE 1/FIELD=1 TO 0.
+WRITE 1/FIELD=1 TO 10 BY 20.
+WRITE 1/FIELD=1 TO 10 BY 6.
+WRITE 1/MODE=TRAPEZOIDAL.
+WRITE 1/FORMAT=F5/FORMAT=F5.
+WRITE 1/FORMAT='5ASDF'.
+WRITE 1/FORMAT=ASDF5.
+WRITE 1/!.
+WRITE 1.
+WRITE 1/FIELD=1 TO 10.
+WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT='15F'.
+WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
+WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT=E.
+WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT=A9.
+WRITE {1,2}/FIELD=1 TO 10/OUTFILE='matrix.txt'/MODE=TRIANGULAR.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.7: error: WRITE: Syntax error expecting matrix expression.
+ 2 | WRITE !.
+ | ^
+
+matrix.sps:3.17: error: WRITE: Syntax error expecting a file name or handle
+name.
+ 3 | WRITE 1/OUTFILE=!.
+ | ^
+
+matrix.sps:4.18: error: WRITE: Syntax error expecting string.
+ 4 | WRITE 1/ENCODING=!.
+ | ^
+
+matrix.sps:5.15: error: WRITE: Syntax error expecting positive integer for
+FIELD.
+ 5 | WRITE 1/FIELD=!.
+ | ^
+
+matrix.sps:6.17: error: WRITE: Syntax error expecting `TO'.
+ 6 | WRITE 1/FIELD=1 !.
+ | ^
+
+matrix.sps:7.20: error: WRITE: Syntax error expecting positive integer for TO.
+ 7 | WRITE 1/FIELD=1 TO 0.
+ | ^
+
+matrix.sps:8.26-8.27: error: WRITE: Syntax error expecting integer between 1
+and 10 for BY.
+ 8 | WRITE 1/FIELD=1 TO 10 BY 20.
+ | ^~
+
+matrix.sps:9.15-9.26: error: WRITE: Field width 6 does not evenly divide record
+width 10.
+ 9 | WRITE 1/FIELD=1 TO 10 BY 6.
+ | ^~~~~~~~~~~~
+
+matrix.sps:9.15-9.21: note: WRITE: This syntax designates the record width.
+ 9 | WRITE 1/FIELD=1 TO 10 BY 6.
+ | ^~~~~~~
+
+matrix.sps:9.26: note: WRITE: This syntax specifies the field width.
+ 9 | WRITE 1/FIELD=1 TO 10 BY 6.
+ | ^
+
+matrix.sps:10.14-10.24: error: WRITE: Syntax error expecting RECTANGULAR or
+TRIANGULAR.
+ 10 | WRITE 1/MODE=TRAPEZOIDAL.
+ | ^~~~~~~~~~~
+
+matrix.sps:11.19-11.24: error: WRITE: Subcommand FORMAT may only be specified
+once.
+ 11 | WRITE 1/FORMAT=F5/FORMAT=F5.
+ | ^~~~~~
+
+matrix.sps:12.16-12.22: error: WRITE: Unknown format ASDF.
+ 12 | WRITE 1/FORMAT='5ASDF'.
+ | ^~~~~~~
+
+matrix.sps:13.16-13.20: error: WRITE: Unknown format type `ASDF'.
+ 13 | WRITE 1/FORMAT=ASDF5.
+ | ^~~~~
+
+matrix.sps:14.9: error: WRITE: Syntax error expecting OUTFILE, FIELD, MODE,
+HOLD, or FORMAT.
+ 14 | WRITE 1/!.
+ | ^
+
+matrix.sps:15.1-15.8: error: WRITE: Required subcommand FIELD was not
+specified.
+ 15 | WRITE 1.
+ | ^~~~~~~~
+
+matrix.sps:16.1-16.22: error: WRITE: Required subcommand OUTFILE was not
+specified.
+ 16 | WRITE 1/FIELD=1 TO 10.
+ | ^~~~~~~~~~~~~~~~~~~~~~
+
+matrix.sps:17.51-17.55: note: WRITE: This syntax designates the number of
+repetitions.
+ 17 | WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT='15F'.
+ | ^~~~~
+
+matrix.sps:17.15-17.21: note: WRITE: This syntax designates the record width.
+ 17 | WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT='15F'.
+ | ^~~~~~~
+
+matrix.sps:18: error: WRITE: This command specifies two different field widths.
+
+matrix.sps:18.56-18.59: note: WRITE: This syntax specifies 5 repetitions.
+ 18 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
+ | ^~~~
+
+matrix.sps:18.15-18.21: note: WRITE: This syntax designates record width 10,
+which divided by 5 repetitions implies field width 2.
+ 18 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
+ | ^~~~~~~
+
+matrix.sps:18.26: note: WRITE: This syntax specifies field width 5.
+ 18 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
+ | ^
+
+matrix.sps:19: error: WRITE: Output format E5.0 specifies width 5, but E
+requires a width between 6 and 40.
+
+matrix.sps:19.56: note: WRITE: This syntax specifies format E.
+ 19 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT=E.
+ | ^
+
+matrix.sps:19.26: note: WRITE: This syntax specifies field width 5.
+ 19 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT=E.
+ | ^
+
+matrix.sps:20.51-20.52: error: WRITE: Format A9 is too wide for 8-byte matrix
+elements.
+ 20 | WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT=A9.
+ | ^~
+
+matrix.sps:21.7-21.11: error: MATRIX: WRITE with MODE=TRIANGULAR requires a
+square matrix but the matrix to be written has dimensions 1×2.
+ 21 | WRITE {1,2}/FIELD=1 TO 10/OUTFILE='matrix.txt'/MODE=TRIANGULAR.
+ | ^~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - GET])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /a b c.
+MISSING VALUES a(1) b(5).
+BEGIN DATA.
+0 0 0
+1 2 3
+4 5 6
+7 8 .
+END DATA.
+
+MATRIX.
+GET x0 /NAMES=names0.
+PRINT x0.
+PRINT names0/FORMAT=A8.
+END MATRIX.
+
+MATRIX.
+GET x1 /VARIABLES=a b c /NAMES=names1 /MISSING=OMIT.
+PRINT x1.
+PRINT names1/FORMAT=A8.
+END MATRIX.
+
+MATRIX.
+GET x2 /VARIABLES=a b /NAMES=names2 /MISSING=OMIT.
+PRINT x2.
+PRINT names2/FORMAT=A8.
+END MATRIX.
+
+MATRIX.
+GET x3 /FILE=* /VARIABLES=a b c /NAMES=names3 /MISSING=5.
+PRINT x3.
+PRINT names3/FORMAT=A8.
+END MATRIX.
+
+MATRIX.
+GET x4 /FILE=* /VARIABLES=a b /NAMES=names4 /MISSING=5.
+PRINT x4.
+PRINT names4/FORMAT=A8.
+END MATRIX.
+
+SAVE OUTFILE='matrix.sav'.
+NEW FILE.
+
+MATRIX.
+GET x5 /FILE='matrix.sav' /VARIABLES=a b c /NAMES=names5 /MISSING=ACCEPT.
+PRINT x5.
+PRINT names5/FORMAT=A8.
+END MATRIX.
+
+MATRIX.
+GET x6 /FILE='matrix.sav' /VARIABLES=a b c /NAMES=names6 /MISSING=ACCEPT /SYSMIS=9.
+PRINT x6.
+PRINT names6/FORMAT=A8.
+END MATRIX.
+
+MATRIX.
+GET x7 /FILE='matrix.sav' /VARIABLES=a b c /NAMES=names7 /MISSING=ACCEPT /SYSMIS=OMIT.
+PRINT x7.
+PRINT names7/FORMAT=A8.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:11: error: MATRIX: Variable a in case 2 has user-missing value 1.
+
+matrix.sps:12.7-12.8: error: MATRIX: Uninitialized variable x0 used in
+expression.
+ 12 | PRINT x0.
+ | ^~
+
+names0
+ a
+ b
+ c
+
+matrix.sps:17: error: MATRIX: Variable c in case 4 is system-missing.
+
+matrix.sps:18.7-18.8: error: MATRIX: Uninitialized variable x1 used in
+expression.
+ 18 | PRINT x1.
+ | ^~
+
+names1
+ a
+ b
+ c
+
+x2
+ 0 0
+ 7 8
+
+names2
+ a
+ b
+
+matrix.sps:29: error: MATRIX: Variable c in case 4 is system-missing.
+
+matrix.sps:30.7-30.8: error: MATRIX: Uninitialized variable x3 used in
+expression.
+ 30 | PRINT x3.
+ | ^~
+
+names3
+ a
+ b
+ c
+
+x4
+ 0 0
+ 5 2
+ 4 5
+ 7 8
+
+names4
+ a
+ b
+
+matrix.sps:44: error: MATRIX: Variable c in case 4 is system-missing.
+
+matrix.sps:45.7-45.8: error: MATRIX: Uninitialized variable x5 used in
+expression.
+ 45 | PRINT x5.
+ | ^~
+
+names5
+ a
+ b
+ c
+
+x6
+ 0 0 0
+ 1 2 3
+ 4 5 6
+ 7 8 9
+
+names6
+ a
+ b
+ c
+
+x7
+ 0 0 0
+ 1 2 3
+ 4 5 6
+
+names7
+ a
+ b
+ c
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - GET - negative])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /a b c * d(a1).
+MISSING VALUES a(1) b(5).
+BEGIN DATA.
+0 0 0 a
+1 2 3 b
+4 5 6 b
+7 8 . d
+END DATA.
+SAVE OUTFILE='matrix.sav'.
+
+MATRIX.
+GET !.
+GET x/VARIABLES=!.
+GET x/FILE=!.
+GET x/ENCODING=!.
+GET x/NAMES=!.
+GET x/MISSING=!.
+GET x/SYSMIS=!.
+GET x/!.
+GET x/VARIABLES=!.
+GET x/VARIABLES=x TO !.
+GET x/VARIABLES=x.
+GET x/VARIABLES=c TO a.
+GET x/VARIABLES=d.
+GET x.
+END MATRIX.
+
+NEW FILE.
+MATRIX.
+GET x/VARIABLES=a.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:12.5: error: GET: Syntax error expecting identifier.
+ 12 | GET !.
+ | ^
+
+matrix.sps:13.17: error: GET: Syntax error expecting variable name.
+ 13 | GET x/VARIABLES=!.
+ | ^
+
+matrix.sps:14.12: error: GET: Syntax error expecting a file name or handle
+name.
+ 14 | GET x/FILE=!.
+ | ^
+
+matrix.sps:15.16: error: GET: Syntax error expecting string.
+ 15 | GET x/ENCODING=!.
+ | ^
+
+matrix.sps:16.13: error: GET: Syntax error expecting identifier.
+ 16 | GET x/NAMES=!.
+ | ^
+
+matrix.sps:17.15: error: GET: Syntax error expecting ACCEPT or OMIT or a number
+for MISSING.
+ 17 | GET x/MISSING=!.
+ | ^
+
+matrix.sps:18.14: error: GET: Syntax error expecting OMIT or a number for
+SYSMIS.
+ 18 | GET x/SYSMIS=!.
+ | ^
+
+matrix.sps:19.7: error: GET: Syntax error expecting FILE, VARIABLES, NAMES,
+MISSING, or SYSMIS.
+ 19 | GET x/!.
+ | ^
+
+matrix.sps:20.17: error: GET: Syntax error expecting variable name.
+ 20 | GET x/VARIABLES=!.
+ | ^
+
+matrix.sps:21.22: error: GET: Syntax error expecting variable name.
+ 21 | GET x/VARIABLES=x TO !.
+ | ^
+
+matrix.sps:22.17: error: MATRIX: x is not a variable name.
+ 22 | GET x/VARIABLES=x.
+ | ^
+
+matrix.sps:23.17-23.22: error: MATRIX: c TO a is not valid syntax since c
+precedes a in the dictionary.
+ 23 | GET x/VARIABLES=c TO a.
+ | ^~~~~~
+
+matrix.sps:24.17: error: MATRIX: d is not a numeric variable.
+ 24 | GET x/VARIABLES=d.
+ | ^
+
+matrix.sps:25: error: MATRIX: Variable d is not numeric.
+
+matrix.sps:30: error: MATRIX: The GET command cannot read an empty active file.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - SAVE])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+SAVE {1,2,3; 4,5,6}/OUTFILE='matrix.sav'.
+SAVE {7,8,9}/VARIABLES=a b c d.
+
+SAVE {1,2,3}/OUTFILE='matrix2.sav'/VARIABLES=v01 TO v03.
+SAVE {4,5,6}/NAMES={'x', 'y', 'z', 'w'}.
+
+SAVE {1,'abcd',3}/OUTFILE='matrix3.sav'/NAMES={'a', 'b', 'c'}/STRINGS=b.
+SAVE {4,'xyzw',6}/STRINGS=a, b.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps])
+AT_CHECK([pspp-convert matrix.sav matrix.csv && cat matrix.csv], [0], [dnl
+COL1,COL2,COL3
+1,2,3
+4,5,6
+7,8,9
+])
+AT_CHECK([pspp-convert matrix2.sav matrix2.csv && cat matrix2.csv], [0], [dnl
+v01,v02,v03
+1,2,3
+4,5,6
+])
+AT_CHECK([pspp-convert matrix3.sav matrix3.csv && cat matrix3.csv], [0], [dnl
+a,b,c
+1,abcd,3
+4,xyzw,6
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - SAVE - inline])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+SAVE {1,2,3; 4,5,6}/OUTFILE=*.
+SAVE {7,8,9}/VARIABLES=a b c d.
+END MATRIX.
+LIST.
+
+MATRIX.
+SAVE {1,2,3}/OUTFILE=*/VARIABLES=v01 TO v03.
+SAVE {4,5,6}/NAMES={'x', 'y', 'z', 'w'}.
+END MATRIX.
+LIST.
+
+MATRIX.
+SAVE {1,'abcd',3}/OUTFILE=*/NAMES={'a', 'b', 'c'}/STRINGS=b.
+SAVE {4,'xyzw',6}/STRINGS=a, b.
+END MATRIX.
+LIST.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Data List
+COL1,COL2,COL3
+1.00,2.00,3.00
+4.00,5.00,6.00
+7.00,8.00,9.00
+
+Table: Data List
+v01,v02,v03
+1.00,2.00,3.00
+4.00,5.00,6.00
+
+Table: Data List
+a,b,c
+1.00,abcd,3.00
+4.00,xyzw,6.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - SAVE - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+SAVE !.
+SAVE 1/OUTFILE=!.
+SAVE 1/VARIABLES=!.
+SAVE 1/NAMES=!.
+SAVE 1/!.
+SAVE 1.
+SAVE 1/OUTFILE='matrix.sav'/NAMES={'a'}/VARIABLES=a.
+SAVE 1/OUTFILE='matrix2.sav'.
+SAVE {1,2}/OUTFILE='matrix2.sav'.
+SAVE {1,2}/OUTFILE='matrix3.sav'/NAMES={'a', 'a'}.
+SAVE {1,2}/OUTFILE='matrix4.sav'/STRINGS=a.
+SAVE {1,2}/OUTFILE='matrix5.sav'/STRINGS=a, b.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.6: error: SAVE: Syntax error expecting matrix expression.
+ 2 | SAVE !.
+ | ^
+
+matrix.sps:3.16: error: SAVE: Syntax error expecting a file name or handle
+name.
+ 3 | SAVE 1/OUTFILE=!.
+ | ^
+
+matrix.sps:4.18: error: SAVE: Syntax error expecting variable name.
+ 4 | SAVE 1/VARIABLES=!.
+ | ^
+
+matrix.sps:5.14: error: SAVE: Syntax error expecting matrix expression.
+ 5 | SAVE 1/NAMES=!.
+ | ^
+
+matrix.sps:6.8: error: SAVE: Syntax error expecting OUTFILE, VARIABLES, NAMES,
+or STRINGS.
+ 6 | SAVE 1/!.
+ | ^
+
+matrix.sps:7.1-7.7: error: SAVE: Required subcommand OUTFILE was not specified.
+ 7 | SAVE 1.
+ | ^~~~~~~
+
+matrix.sps:8.35-8.39: warning: SAVE: Ignoring NAMES because VARIABLES was also
+specified.
+ 8 | SAVE 1/OUTFILE='matrix.sav'/NAMES={'a'}/VARIABLES=a.
+ | ^~~~~
+
+matrix.sps:10: error: MATRIX: Cannot save 1×2 matrix to `matrix2.sav' because
+the first SAVE to `matrix2.sav' in this matrix program wrote a 1-column matrix.
+
+matrix.sps:9: error: MATRIX: This is the location of the first SAVE to
+`matrix2.sav'.
+
+error: Duplicate variable name a in SAVE statement.
+
+error: The SAVE command STRINGS subcommand specifies an unknown variable a.
+
+error: The SAVE command STRINGS subcommand specifies 2 unknown variables,
+including a.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET])
+AT_DATA([matrix.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ var01 TO var08.
+BEGIN DATA.
+MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+N 92 92 92 92 92 92 92 92
+CORR 1.00
+CORR .18 1.00
+CORR -.22 -.17 1.00
+CORR .36 .31 -.14 1.00
+CORR .27 .16 -.12 .22 1.00
+CORR .33 .15 -.17 .24 .21 1.00
+CORR .50 .29 -.20 .32 .12 .38 1.00
+CORR .17 .29 -.05 .20 .27 .20 .04 1.00
+END DATA.
+
+MATRIX.
+MGET.
+PRINT MN/FORMAT=F5.1.
+PRINT SD/FORMAT=F5.1.
+PRINT NC/FORMAT=F5.0.
+PRINT CR/FORMAT=F5.2.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Matrix Variables Created by MGET
+,Dimensions,
+,Rows,Columns
+MN,1,8
+SD,1,8
+NC,1,8
+CR,8,8
+
+MN
+24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
+
+SD
+5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
+
+NC
+92 92 92 92 92 92 92 92
+
+CR
+1.00 .18 -.22 .36 .27 .33 .50 .17
+.18 1.00 -.17 .31 .16 .15 .29 .29
+-.22 -.17 1.00 -.14 -.12 -.17 -.20 -.05
+.36 .31 -.14 1.00 .22 .24 .32 .20
+.27 .16 -.12 .22 1.00 .21 .12 .27
+.33 .15 -.17 .24 .21 1.00 .38 .20
+.50 .29 -.20 .32 .12 .38 1.00 .04
+.17 .29 -.05 .20 .27 .20 .04 1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET with split variables])
+AT_DATA([matrix.sps], [dnl
+matrix data
+ variables = s1 s2 rowtype_ var01 var02 var03
+ /split=s1 s2.
+
+begin data
+8 0 mean 21.4 5.0 72.9
+8 0 sd 6.5 1.6 22.8
+8 0 n 106 106 106
+8 0 corr 1
+8 0 corr .41 1
+8 0 corr -.16 -.22 1
+8 1 mean 11.4 1.0 52.9
+8 1 sd 9.5 8.6 12.8
+8 1 n 10 11 12
+8 1 corr 1
+8 1 corr .51 1
+8 1 corr .36 -.41 1
+end data.
+
+MATRIX.
+MGET.
+PRINT MNS1/FORMAT=F5.1.
+PRINT SDS1/FORMAT=F5.1.
+PRINT NCS1/FORMAT=F5.0.
+PRINT CRS1/FORMAT=F5.2.
+PRINT MNS2/FORMAT=F5.1.
+PRINT SDS2/FORMAT=F5.1.
+PRINT NCS2/FORMAT=F5.0.
+PRINT CRS2/FORMAT=F5.2.
+END MATRIX.
+])
+AT_CHECK([pspp -O format=csv matrix.sps], [0], [dnl
+Table: Matrix Variables Created by MGET
+,Split Values,,Dimensions,
+,s1,s2,Rows,Columns
+MNS1,8,0,1,3
+SDS1,8,0,1,3
+NCS1,8,0,1,3
+CRS1,8,0,3,3
+MNS2,8,1,1,3
+SDS2,8,1,1,3
+NCS2,8,1,1,3
+CRS2,8,1,3,3
+
+MNS1
+21.4 5.0 72.9
+
+SDS1
+6.5 1.6 22.8
+
+NCS1
+106 106 106
+
+CRS1
+1.00 .41 -.16
+.41 1.00 -.22
+-.16 -.22 1.00
+
+MNS2
+11.4 1.0 52.9
+
+SDS2
+9.5 8.6 12.8
+
+NCS2
+10 11 12
+
+CRS2
+1.00 .51 .36
+.51 1.00 -.41
+.36 -.41 1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET with factor variables])
+AT_DATA([matrix.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ f1 var01 TO var04
+ /FACTOR=f1.
+BEGIN DATA.
+MEAN 0 34 35 36 37
+SD 0 22 11 55 66
+N 0 99 98 99 92
+MEAN 1 44 45 34 39
+SD 1 23 15 51 46
+N 1 98 34 87 23
+CORR . 1
+CORR . .9 1
+CORR . .8 .6 1
+CORR . .7 .5 .4 1
+END DATA.
+FORMATS var01 TO var04(F5.1).
+SAVE OUTFILE='matrix.sav'.
+])
+AT_DATA([matrix2.sps], [dnl
+MATRIX.
+MGET FILE='matrix.sav'.
+PRINT MNF1/FORMAT=F2.0.
+PRINT SDF1/FORMAT=F2.0.
+PRINT NCF1/FORMAT=F2.0.
+PRINT MNF2/FORMAT=F2.0.
+PRINT SDF2/FORMAT=F2.0.
+PRINT NCF2/FORMAT=F2.0.
+PRINT CR/FORMAT=F3.1.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps])
+AT_CHECK([pspp -O format=csv matrix2.sps], [0], [dnl
+Table: Matrix Variables Created by MGET
+,Factors,Dimensions,
+,f1,Rows,Columns
+MNF1,0,1,4
+SDF1,0,1,4
+NCF1,0,1,4
+MNF2,1,1,4
+SDF2,1,1,4
+NCF2,1,1,4
+CR,.,4,4
+
+MNF1
+34 35 36 37
+
+SDF1
+22 11 55 66
+
+NCF1
+99 98 99 92
+
+MNF2
+44 45 34 39
+
+SDF2
+23 15 51 46
+
+NCF2
+98 34 87 23
+
+CR
+1.0 .9 .8 .7
+.9 1.0 .6 .5
+.8 .6 1.0 .4
+.7 .5 .4 1.0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET with factor and split variables])
+AT_DATA([matrix.sps], [dnl
+matrix data
+ variables = s f rowtype_ var01 var02 var03
+ /split=s
+ /factor=f.
+
+begin data
+8 0 mean 21.4 5.0 72.9
+8 0 sd 6.5 1.6 22.8
+8 0 n 106 106 106
+8 . corr 1
+8 . corr .41 1
+8 . corr -.16 -.22 1
+9 1 mean 11.4 1.0 52.9
+9 1 sd 9.5 8.6 12.8
+9 1 n 10 11 12
+9 . corr 1
+9 . corr .51 1
+9 . corr .36 -.41 1
+end data.
+
+MATRIX.
+MGET.
+PRINT MNF1S1/FORMAT=F5.1.
+PRINT SDF1S1/FORMAT=F5.1.
+PRINT NCF1S1/FORMAT=F5.0.
+PRINT CRS1/FORMAT=F5.2.
+PRINT MNF1S2/FORMAT=F5.1.
+PRINT SDF1S2/FORMAT=F5.1.
+PRINT NCF1S2/FORMAT=F5.0.
+PRINT CRS2/FORMAT=F5.2.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Matrix Variables Created by MGET
+,Split Values,Factors,Dimensions,
+,s,f,Rows,Columns
+MNF1S1,8,0,1,3
+SDF1S1,8,0,1,3
+NCF1S1,8,0,1,3
+CRS1,8,.,3,3
+MNF1S2,9,1,1,3
+SDF1S2,9,1,1,3
+NCF1S2,9,1,1,3
+CRS2,9,.,3,3
+
+MNF1S1
+21.4 5.0 72.9
+
+SDF1S1
+6.5 1.6 22.8
+
+NCF1S1
+106 106 106
+
+CRS1
+1.00 .41 -.16
+.41 1.00 -.22
+-.16 -.22 1.00
+
+MNF1S2
+11.4 1.0 52.9
+
+SDF1S2
+9.5 8.6 12.8
+
+NCF1S2
+10 11 12
+
+CRS2
+1.00 .51 .36
+.51 1.00 -.41
+.36 -.41 1.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET with TYPE])
+AT_DATA([matrix.sps], [dnl
+MATRIX DATA
+ VARIABLES=ROWTYPE_ f1 var01 TO var04
+ /FACTOR=f1.
+BEGIN DATA.
+MEAN 0 34 35 36 37
+SD 0 22 11 55 66
+N 0 99 98 99 92
+MEAN 1 44 45 34 39
+SD 1 23 15 51 46
+N 1 98 34 87 23
+CORR . 1
+CORR . .9 1
+CORR . .8 .6 1
+CORR . .7 .5 .4 1
+END DATA.
+FORMATS var01 TO var04(F5.1).
+SAVE OUTFILE='matrix.sav'.
+])
+AT_DATA([matrix2.sps], [dnl
+MATRIX.
+MGET/FILE='matrix.sav'/TYPE=CORR.
+PRINT CR/FORMAT=F3.1.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps])
+AT_CHECK([pspp -O format=csv matrix2.sps], [0], [dnl
+Table: Matrix Variables Created by MGET
+,Factors,Dimensions,
+,f1,Rows,Columns
+CR,.,4,4
+
+CR
+1.0 .9 .8 .7
+.9 1.0 .6 .5
+.8 .6 1.0 .4
+.7 .5 .4 1.0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - negative - parsing])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+MGET !.
+MGET FILE=!.
+MGET ENCODING=!.
+MGET TYPE=!.
+MGET TYPE=CORR !.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.6: error: MGET: Syntax error expecting FILE or TYPE.
+ 2 | MGET !.
+ | ^
+
+matrix.sps:3.11: error: MGET: Syntax error expecting a file name or handle
+name.
+ 3 | MGET FILE=!.
+ | ^
+
+matrix.sps:4.15: error: MGET: Syntax error expecting string.
+ 4 | MGET ENCODING=!.
+ | ^
+
+matrix.sps:5.11: error: MGET: Syntax error expecting COV, CORR, MEAN, STDDEV,
+N, or COUNT.
+ 5 | MGET TYPE=!.
+ | ^
+
+matrix.sps:6.16: error: MGET: Syntax error expecting COV, CORR, MEAN, STDDEV,
+N, or COUNT.
+ 6 | MGET TYPE=CORR !.
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - missing VARNAME_ and ROWTYPE_])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+BEGIN DATA.
+1
+END DATA.
+
+MATRIX.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:7: error: MATRIX: Matrix data file lacks ROWTYPE_ variable.
+
+matrix.sps:7: error: MATRIX: Matrix data file lacks VARNAME_ variable.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - wrong format for VARNAME_ and ROWTYPE_])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /VARNAME_ * ROWTYPE_ (A7).
+BEGIN DATA.
+1 asdf
+END DATA.
+
+MATRIX.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:7: error: MATRIX: ROWTYPE_ variable in matrix data file must be 8-
+byte string, but it has width 7.
+
+matrix.sps:7: error: MATRIX: VARNAME_ variable in matrix data file must be 8-
+byte string, but it has width 0.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - wrong order for VARNAME_ and ROWTYPE_])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /VARNAME_ ROWTYPE_ (A8).
+BEGIN DATA.
+asdf jkl;
+END DATA.
+
+MATRIX.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:7: error: MATRIX: ROWTYPE_ must precede VARNAME_ in matrix data
+file.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - no continuous variables])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ (A8).
+BEGIN DATA.
+asdf jkl;
+END DATA.
+
+MATRIX.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:7: error: MATRIX: Matrix data file contains no continuous variables.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - unexpected string variables])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ cvar1 (A8).
+BEGIN DATA.
+asdf jkl; zxcv
+END DATA.
+
+MATRIX.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:7: error: MATRIX: Matrix data file contains unexpected string
+variable cvar1.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - unknown ROWTYPE_])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ (A8) cvar1.
+BEGIN DATA.
+asdf jkl; 1
+END DATA.
+
+MATRIX.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [1], [dnl
+"matrix.sps:7: error: MATRIX: Matrix data file contains unknown ROWTYPE_ ""asdf""."
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - duplicate matrix variable name])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ (A8) cvar1.
+BEGIN DATA.
+corr jkl; 1
+END DATA.
+
+MATRIX.
+MGET.
+MGET.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Matrix Variables Created by MGET
+,Dimensions,
+,Rows,Columns
+CR,1,1
+
+matrix.sps:8: warning: MATRIX: Matrix data file contains variable with existing name CR.
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MGET - missing values in input])
+AT_DATA([matrix.sps], [dnl
+DATA LIST LIST NOTABLE /s1 * ROWTYPE_ VARNAME_ (A8) cvar1 cvar2.
+BEGIN DATA.
+1 n "" 1 .
+2 n "" . .
+END DATA.
+
+MATRIX.
+MGET.
+PRINT ncs1/FORMAT=F5.
+PRINT ncs2/FORMAT=F5.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [1], [dnl
+"matrix.sps:8: error: MATRIX: Matrix data file variable NCS1 contains a missing value, which was treated as zero."
+
+"matrix.sps:8: error: MATRIX: Matrix data file variable NCS2 contains 2 missing values, which were treated as zero."
+
+Table: Matrix Variables Created by MGET
+,Split Values,Dimensions,
+,s1,Rows,Columns
+NCS1,1.00,1,2
+NCS2,2.00,1,2
+
+ncs1
+1 0
+
+ncs2
+0 0
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MSAVE])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+MSAVE {1, 2; 3, 4}/TYPE=CORR/VARIABLES=X,Y/OUTFILE='matrix.sav'.
+MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV/VARIABLES=X,Y.
+MSAVE {11, 12}/TYPE=MEAN.
+MSAVE {13, 14}/TYPE=STDDEV.
+MSAVE {15, 16}/TYPE=N.
+MSAVE {17, 18}/TYPE=COUNT.
+END MATRIX.
+GET 'matrix.sav'.
+LIST.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Data List
+ROWTYPE_,VARNAME_,X,Y
+CORR,X,1.00,2.00
+CORR,Y,3.00,4.00
+COV,X,5.00,6.00
+COV,Y,7.00,8.00
+COV,,9.00,10.00
+MEAN,,11.00,12.00
+STDDEV,,13.00,14.00
+N,,15.00,16.00
+COUNT,,17.00,18.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MSAVE with factor variables])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+MSAVE {1, 2; 3, 4}/TYPE=CORR/FACTOR={1,1}/FNAMES=X,Y/OUTFILE='matrix.sav'.
+MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV.
+MSAVE {11, 12}/TYPE=MEAN.
+MSAVE {13, 14}/FACTOR={2,1}/TYPE=STDDEV.
+MSAVE {15, 16}/TYPE=N.
+MSAVE {17, 18}/FACTOR={1,2}/TYPE=COUNT.
+END MATRIX.
+GET 'matrix.sav'.
+LIST.
+
+MATRIX.
+MSAVE {1, 2; 3, 4}/TYPE=CORR/FACTOR={5,6,7,8}/OUTFILE='matrix2.sav'.
+END MATRIX.
+GET 'matrix2.sav'.
+LIST.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Data List
+ROWTYPE_,X,Y,VARNAME_,COL1,COL2
+CORR,1.00,1.00,COL1,1.00,2.00
+CORR,1.00,1.00,COL2,3.00,4.00
+COV,1.00,1.00,COL1,5.00,6.00
+COV,1.00,1.00,COL2,7.00,8.00
+COV,1.00,1.00,,9.00,10.00
+MEAN,1.00,1.00,,11.00,12.00
+STDDEV,2.00,1.00,,13.00,14.00
+N,2.00,1.00,,15.00,16.00
+COUNT,1.00,2.00,,17.00,18.00
+
+Table: Data List
+ROWTYPE_,FAC1,FAC2,FAC3,FAC4,VARNAME_,COL1,COL2
+CORR,5.00,6.00,7.00,8.00,COL1,1.00,2.00
+CORR,5.00,6.00,7.00,8.00,COL2,3.00,4.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MSAVE with split variables])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+MSAVE {1, 2; 3, 4}/TYPE=CORR/SPLIT={1,1}/SNAMES=X,Y/OUTFILE='matrix.sav'.
+MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV.
+MSAVE {11, 12}/TYPE=MEAN.
+MSAVE {13, 14}/SPLIT={2,1}/TYPE=STDDEV.
+MSAVE {15, 16}/TYPE=N.
+MSAVE {17, 18}/SPLIT={1,2}/TYPE=COUNT.
+END MATRIX.
+GET 'matrix.sav'.
+LIST.
+
+MATRIX.
+MSAVE {1, 2; 3, 4}/TYPE=CORR/SPLIT={5,6,7,8}/OUTFILE='matrix2.sav'.
+END MATRIX.
+GET 'matrix2.sav'.
+LIST.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Data List
+X,Y,ROWTYPE_,VARNAME_,COL1,COL2
+1.00,1.00,CORR,COL1,1.00,2.00
+1.00,1.00,CORR,COL2,3.00,4.00
+1.00,1.00,COV,COL1,5.00,6.00
+1.00,1.00,COV,COL2,7.00,8.00
+1.00,1.00,COV,,9.00,10.00
+1.00,1.00,MEAN,,11.00,12.00
+2.00,1.00,STDDEV,,13.00,14.00
+2.00,1.00,N,,15.00,16.00
+1.00,2.00,COUNT,,17.00,18.00
+
+Table: Data List
+SPL1,SPL2,SPL3,SPL4,ROWTYPE_,VARNAME_,COL1,COL2
+5.00,6.00,7.00,8.00,CORR,COL1,1.00,2.00
+5.00,6.00,7.00,8.00,CORR,COL2,3.00,4.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MSAVE with factor and split variables])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+MSAVE {1, 2; 3, 4}/TYPE=CORR/SPLIT=1/FACTOR=1/OUTFILE='matrix.sav'.
+MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV.
+MSAVE {11, 12}/FACTOR=2/TYPE=MEAN.
+MSAVE {13, 14}/FACTOR=1/SPLIT=2/TYPE=STDDEV.
+MSAVE {15, 16}/TYPE=N.
+MSAVE {17, 18}/FACTOR=2/TYPE=COUNT.
+END MATRIX.
+GET 'matrix.sav'.
+LIST.
+])
+AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Data List
+SPL1,ROWTYPE_,FAC1,VARNAME_,COL1,COL2
+1.00,CORR,1.00,COL1,1.00,2.00
+1.00,CORR,1.00,COL2,3.00,4.00
+1.00,COV,1.00,COL1,5.00,6.00
+1.00,COV,1.00,COL2,7.00,8.00
+1.00,COV,1.00,,9.00,10.00
+1.00,MEAN,2.00,,11.00,12.00
+2.00,STDDEV,1.00,,13.00,14.00
+2.00,N,1.00,,15.00,16.00
+2.00,COUNT,2.00,,17.00,18.00
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - MSAVE - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+MSAVE !.
+MSAVE 1/TYPE=!.
+MSAVE 1/OUTFILE=!.
+MSAVE 1/VARIABLES=!.
+MSAVE 1/FNAMES=!.
+MSAVE 1/SNAMES=!.
+MSAVE 1/SPLIT=!.
+MSAVE 1/FACTOR=!.
+MSAVE 1/!.
+MSAVE 1.
+MSAVE 1/TYPE=COV/FNAMES=x.
+MSAVE 1/TYPE=COV/SNAMES=x.
+MSAVE 1/TYPE=COV.
+
+MSAVE 1/TYPE=COV/OUTFILE='matrix.sav'
+ /FACTOR=1 /FNAMES=y
+ /SPLIT=2 /SNAMES=z
+ /VARIABLES=w.
+MSAVE 1/TYPE=COV/OUTFILE='matrix2.sav'.
+MSAVE 1/TYPE=COV/VARIABLES=x.
+MSAVE 1/TYPE=COV/FNAMES=x.
+MSAVE 1/TYPE=COV/SNAMES=x.
+END MATRIX.
+
+MATRIX.
+MSAVE 1/TYPE=COV/VARIABLES=x/OUTFILE='matrix3.sav'/FACTOR=1/SPLIT=2.
+MSAVE {1,2}/TYPE=COV/VARIABLES=x/OUTFILE='matrix3.sav'/FACTOR=1/SPLIT=2.
+MSAVE {1,2;3}/TYPE=COV.
+MSAVE 0/TYPE=COV/FACTOR={1,2}.
+MSAVE 0/TYPE=COV/FACTOR=1/SPLIT={1;2}.
+END MATRIX.
+
+MATRIX.
+MSAVE 1/TYPE=COV/OUTFILE='matrix4.sav'/SNAMES=x,x/SPLIT=1.
+END MATRIX.
+
+MATRIX.
+MSAVE 1/TYPE=COV/OUTFILE='matrix5.sav'/SNAMES=x/FNAMES=x/SPLIT=1/FACTOR=1.
+END MATRIX.
+
+MATRIX.
+MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/FNAMES=x/FACTOR=1.
+END MATRIX.
+
+MATRIX.
+MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/SNAMES=x/SPLIT=1.
+END MATRIX.
+
+MATRIX.
+MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=VARNAME_.
+MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=ROWTYPE_.
+MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=VARNAME_.
+MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=ROWTYPE_.
+MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=VARNAME_.
+MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=ROWTYPE_.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.7: error: MSAVE: Syntax error expecting matrix expression.
+ 2 | MSAVE !.
+ | ^
+
+matrix.sps:3.14: error: MSAVE: Syntax error expecting COV, CORR, MEAN, STDDEV,
+N, or COUNT.
+ 3 | MSAVE 1/TYPE=!.
+ | ^
+
+matrix.sps:4.17: error: MSAVE: Syntax error expecting a file name or handle
+name.
+ 4 | MSAVE 1/OUTFILE=!.
+ | ^
+
+matrix.sps:5.19: error: MSAVE: Syntax error expecting variable name.
+ 5 | MSAVE 1/VARIABLES=!.
+ | ^
+
+matrix.sps:6.16: error: MSAVE: Syntax error expecting variable name.
+ 6 | MSAVE 1/FNAMES=!.
+ | ^
+
+matrix.sps:7.16: error: MSAVE: Syntax error expecting variable name.
+ 7 | MSAVE 1/SNAMES=!.
+ | ^
+
+matrix.sps:8.15: error: MSAVE: Syntax error expecting matrix expression.
+ 8 | MSAVE 1/SPLIT=!.
+ | ^
+
+matrix.sps:9.16: error: MSAVE: Syntax error expecting matrix expression.
+ 9 | MSAVE 1/FACTOR=!.
+ | ^
+
+matrix.sps:10.9: error: MSAVE: Syntax error expecting TYPE, OUTFILE, VARIABLES,
+FNAMES, SNAMES, SPLIT, or FACTOR.
+ 10 | MSAVE 1/!.
+ | ^
+
+matrix.sps:11.1-11.8: error: MSAVE: Required subcommand TYPE was not specified.
+ 11 | MSAVE 1.
+ | ^~~~~~~~
+
+matrix.sps:12.25: error: MSAVE: FNAMES requires FACTOR.
+ 12 | MSAVE 1/TYPE=COV/FNAMES=x.
+ | ^
+
+matrix.sps:13.25: error: MSAVE: SNAMES requires SPLIT.
+ 13 | MSAVE 1/TYPE=COV/SNAMES=x.
+ | ^
+
+matrix.sps:14.1-14.17: error: MSAVE: Required subcommand OUTFILE was not
+specified.
+ 14 | MSAVE 1/TYPE=COV.
+ | ^~~~~~~~~~~~~~~~~
+
+matrix.sps:20: error: MSAVE: OUTFILE must name the same file on each MSAVE
+within a single MATRIX command.
+
+matrix.sps:16.26-16.37: note: MSAVE: This is the OUTFILE on the first MSAVE
+command.
+ 16 | MSAVE 1/TYPE=COV/OUTFILE='matrix.sav'
+ | ^~~~~~~~~~~~
+
+matrix.sps:20.26-20.38: note: MSAVE: This is the OUTFILE on a later MSAVE
+command.
+ 20 | MSAVE 1/TYPE=COV/OUTFILE='matrix2.sav'.
+ | ^~~~~~~~~~~~~
+
+matrix.sps:21: error: MSAVE: VARIABLES must specify the same variables on each
+MSAVE within a given MATRIX.
+
+matrix.sps:19.16: error: MSAVE: This is the specification of VARIABLES on the
+first MSAVE.
+ 19 | /VARIABLES=w.
+ | ^
+
+matrix.sps:21.28: error: MSAVE: This is a different specification of VARIABLES
+on a later MSAVE.
+ 21 | MSAVE 1/TYPE=COV/VARIABLES=x.
+ | ^
+
+matrix.sps:22: error: MSAVE: FNAMES must specify the same variables on each
+MSAVE within a given MATRIX.
+
+matrix.sps:17.23: error: MSAVE: This is the specification of FNAMES on the
+first MSAVE.
+ 17 | /FACTOR=1 /FNAMES=y
+ | ^
+
+matrix.sps:22.25: error: MSAVE: This is a different specification of FNAMES on
+a later MSAVE.
+ 22 | MSAVE 1/TYPE=COV/FNAMES=x.
+ | ^
+
+matrix.sps:23: error: MSAVE: SNAMES must specify the same variables on each
+MSAVE within a given MATRIX.
+
+matrix.sps:18.22: error: MSAVE: This is the specification of SNAMES on the
+first MSAVE.
+ 18 | /SPLIT=2 /SNAMES=z
+ | ^
+
+matrix.sps:23.25: error: MSAVE: This is a different specification of SNAMES on
+a later MSAVE.
+ 23 | MSAVE 1/TYPE=COV/SNAMES=x.
+ | ^
+
+matrix.sps:28.7-28.11: error: MATRIX: Matrix on MSAVE has 2 columns but there
+are 1 variables.
+ 28 | MSAVE {1,2}/TYPE=COV/VARIABLES=x/OUTFILE='matrix3.sav'/FACTOR=1/
+SPLIT=2.
+ | ^~~~~
+
+matrix.sps:29.7-29.13: error: MATRIX: This expression tries to vertically join
+matrices with differing numbers of columns.
+ 29 | MSAVE {1,2;3}/TYPE=COV.
+ | ^~~~~~~
+
+matrix.sps:29.8-29.10: note: MATRIX: This operand is a 1×2 matrix.
+ 29 | MSAVE {1,2;3}/TYPE=COV.
+ | ^~~
+
+matrix.sps:29.12: note: MATRIX: This operand is a 1×1 matrix.
+ 29 | MSAVE {1,2;3}/TYPE=COV.
+ | ^
+
+matrix.sps:30.25-30.29: error: MATRIX: There are 1 factor variables, but 2
+factor values were supplied.
+ 30 | MSAVE 0/TYPE=COV/FACTOR={1,2}.
+ | ^~~~~
+
+matrix.sps:31.33-31.37: error: MATRIX: There are 1 split variables, but 2 split
+values were supplied.
+ 31 | MSAVE 0/TYPE=COV/FACTOR=1/SPLIT={1;2}.
+ | ^~~~~
+
+matrix.sps:35.49: error: MSAVE: Variable x appears twice in variable list.
+ 35 | MSAVE 1/TYPE=COV/OUTFILE='matrix4.sav'/SNAMES=x,x/SPLIT=1.
+ | ^
+
+matrix.sps:39.56: error: MATRIX: Duplicate or invalid FACTOR variable name x.
+ 39 | MSAVE 1/TYPE=COV/OUTFILE='matrix5.sav'/SNAMES=x/FNAMES=x/SPLIT=1/
+FACTOR=1.
+ | ^
+
+matrix.sps:43.50: error: MATRIX: Duplicate or invalid variable name x.
+ 43 | MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/FNAMES=x/FACTOR=1.
+ | ^
+
+matrix.sps:47.50: error: MATRIX: Duplicate or invalid variable name x.
+ 47 | MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/SNAMES=x/SPLIT=1.
+ | ^
+
+matrix.sps:51.47-51.54: error: MSAVE: Variable name VARNAME_ is reserved.
+ 51 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=VARNAME_.
+ | ^~~~~~~~
+
+matrix.sps:52.47-52.54: error: MSAVE: Variable name ROWTYPE_ is reserved.
+ 52 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=ROWTYPE_.
+ | ^~~~~~~~
+
+matrix.sps:53.47-53.54: error: MSAVE: Variable name VARNAME_ is reserved.
+ 53 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=VARNAME_.
+ | ^~~~~~~~
+
+matrix.sps:54.47-54.54: error: MSAVE: Variable name ROWTYPE_ is reserved.
+ 54 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=ROWTYPE_.
+ | ^~~~~~~~
+
+matrix.sps:55.50-55.57: error: MSAVE: Variable name VARNAME_ is reserved.
+ 55 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=VARNAME_.
+ | ^~~~~~~~
+
+matrix.sps:56.50-56.57: error: MSAVE: Variable name ROWTYPE_ is reserved.
+ 56 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=ROWTYPE_.
+ | ^~~~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - DISPLAY])
+AT_DATA([matrix-template.sps], [dnl
+MATRIX.
+COMPUTE a={1,2,3}.
+COMPUTE b={1;2;3}.
+COMPUTE c={T(b),a}.
+COMPUTE d={T(a),b}.
+command.
+END MATRIX.
+])
+for command in 'DISPLAY' 'DISPLAY DICTIONARY' 'DISPLAY STATUS'; do
+ sed "s/command/$command/" < matrix-template.sps > matrix.sps
+ AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
+Table: Matrix Variables
+,Dimension,,Size (kB)
+,Rows,Columns,
+a,1,3,0
+b,3,1,0
+c,1,6,0
+d,3,2,0
+])
+done
+AT_CLEANUP
+
+AT_SETUP([MATRIX - DISPLAY - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+DISPLAY !.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.9: error: DISPLAY: Syntax error expecting DICTIONARY or STATUS.
+ 2 | DISPLAY !.
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - RELEASE])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+COMPUTE x=1.
+PRINT x.
+RELEASE X.
+PRINT x.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+x
+ 1
+
+matrix.sps:5.7: error: MATRIX: Uninitialized variable x used in expression.
+ 5 | PRINT x.
+ | ^
+])
+AT_CLEANUP
+
+AT_SETUP([MATRIX - RELEASE - negative])
+AT_DATA([matrix.sps], [dnl
+MATRIX.
+RELEASE !.
+RELEASE x.
+COMPUTE x=1.
+RELEASE x, !.
+COMPUTE x=1.
+RELEASE x y.
+COMPUTE x=1.
+RELEASE x.
+RELEASE x.
+END MATRIX.
+])
+AT_CHECK([pspp matrix.sps], [1], [dnl
+matrix.sps:2.9: error: RELEASE: Syntax error expecting end of command.
+ 2 | RELEASE !.
+ | ^
+
+matrix.sps:3.9: error: RELEASE: Syntax error expecting variable name.
+ 3 | RELEASE x.
+ | ^
+
+matrix.sps:5.12: error: RELEASE: Syntax error expecting end of command.
+ 5 | RELEASE x, !.
+ | ^
+
+matrix.sps:7.11: error: RELEASE: Syntax error expecting end of command.
+ 7 | RELEASE x y.
+ | ^
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2021 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([MCONVERT])
+
+AT_SETUP([MCONVERT])
+AT_DATA([mconvert.sps], [dnl
+MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
+BEGIN DATA.
+0 COV 1.0
+0 COV 1.0 16.0
+0 COV 8.1 18.0 81.0
+1 CORR 1
+1 CORR .25 1
+1 CORR .9 .5 1
+1 STDDEV 1 4 9
+END DATA.
+FORMATS var01 TO var03(F5.2).
+SPLIT FILE OFF.
+MCONVERT.
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
+Table: Data List
+s,ROWTYPE_,VARNAME_,var01,var02,var03
+0,CORR,var01,1.00,.25,.90
+0,CORR,var02,.25,1.00,.50
+0,CORR,var03,.90,.50,1.00
+0,STDDEV,,1.00,4.00,9.00
+1,STDDEV,,1.00,4.00,9.00
+1,COV,var01,1.00,1.00,8.10
+1,COV,var02,1.00,16.00,18.00
+1,COV,var03,8.10,18.00,81.00
+])
+AT_CLEANUP
+
+AT_SETUP([MCONVERT from .sav file])
+AT_DATA([input.sps], [dnl
+MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
+BEGIN DATA.
+0 COV 1.0
+0 COV 1.0 16.0
+0 COV 8.1 18.0 81.0
+1 CORR 1
+1 CORR .25 1
+1 CORR .9 .5 1
+1 STDDEV 1 4 9
+END DATA.
+FORMATS var01 TO var03(F5.2).
+SPLIT FILE OFF.
+SAVE OUTFILE='input.sav'.
+])
+AT_DATA([mconvert.sps], [dnl
+MCONVERT MATRIX=IN('input.sav').
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv input.sps])
+AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
+Table: Data List
+s,ROWTYPE_,VARNAME_,var01,var02,var03
+0,CORR,var01,1.00,.25,.90
+0,CORR,var02,.25,1.00,.50
+0,CORR,var03,.90,.50,1.00
+0,STDDEV,,1.00,4.00,9.00
+1,STDDEV,,1.00,4.00,9.00
+1,COV,var01,1.00,1.00,8.10
+1,COV,var02,1.00,16.00,18.00
+1,COV,var03,8.10,18.00,81.00
+])
+AT_CLEANUP
+
+AT_SETUP([MCONVERT to .sav file])
+AT_DATA([mconvert.sps], [dnl
+MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
+BEGIN DATA.
+0 COV 1.0
+0 COV 1.0 16.0
+0 COV 8.1 18.0 81.0
+1 CORR 1
+1 CORR .25 1
+1 CORR .9 .5 1
+1 STDDEV 1 4 9
+END DATA.
+FORMATS var01 TO var03(F5.2).
+SPLIT FILE OFF.
+MCONVERT/REPLACE/OUT('output.sav').
+LIST.
+])
+AT_DATA([output.sps], [dnl
+GET 'output.sav'.
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
+Table: Data List
+s,ROWTYPE_,VARNAME_,var01,var02,var03
+0,COV,var01,1.00,1.00,8.10
+0,COV,var02,1.00,16.00,18.00
+0,COV,var03,8.10,18.00,81.00
+1,CORR,var01,1.00,.25,.90
+1,CORR,var02,.25,1.00,.50
+1,CORR,var03,.90,.50,1.00
+1,STDDEV,,1.00,4.00,9.00
+])
+AT_CHECK([pspp -O format=csv output.sps], [0], [dnl
+Table: Data List
+s,ROWTYPE_,VARNAME_,var01,var02,var03
+0,CORR,var01,1.00,.25,.90
+0,CORR,var02,.25,1.00,.50
+0,CORR,var03,.90,.50,1.00
+0,STDDEV,,1.00,4.00,9.00
+1,STDDEV,,1.00,4.00,9.00
+1,COV,var01,1.00,1.00,8.10
+1,COV,var02,1.00,16.00,18.00
+1,COV,var03,8.10,18.00,81.00
+])
+AT_CLEANUP
+
+AT_SETUP([MCONVERT from .sav file to .sav file])
+AT_DATA([input.sps], [dnl
+MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
+BEGIN DATA.
+0 COV 1.0
+0 COV 1.0 16.0
+0 COV 8.1 18.0 81.0
+1 CORR 1
+1 CORR .25 1
+1 CORR .9 .5 1
+1 STDDEV 1 4 9
+END DATA.
+FORMATS var01 TO var03(F5.2).
+SPLIT FILE OFF.
+SAVE OUTFILE='input.sav'.
+])
+AT_DATA([mconvert.sps], [dnl
+MCONVERT MATRIX=IN('input.sav') OUT('output.sav')/REPLACE.
+LIST.
+])
+AT_DATA([output.sps], [dnl
+GET 'output.sav'.
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv input.sps])
+AT_CHECK([pspp -O format=csv mconvert.sps], [1], [dnl
+"mconvert.sps:2.1-2.4: error: LIST: LIST is allowed only after the active dataset has been defined.
+ 2 | LIST.
+ | ^~~~"
+])
+AT_CHECK([pspp -O format=csv output.sps], [0], [dnl
+Table: Data List
+s,ROWTYPE_,VARNAME_,var01,var02,var03
+0,CORR,var01,1.00,.25,.90
+0,CORR,var02,.25,1.00,.50
+0,CORR,var03,.90,.50,1.00
+0,STDDEV,,1.00,4.00,9.00
+1,STDDEV,,1.00,4.00,9.00
+1,COV,var01,1.00,1.00,8.10
+1,COV,var02,1.00,16.00,18.00
+1,COV,var03,8.10,18.00,81.00
+])
+AT_CLEANUP
+
+AT_SETUP([MCONVERT with APPEND])
+AT_DATA([mconvert.sps], [dnl
+MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
+BEGIN DATA.
+0 COV 1.0
+0 COV 1.0 16.0
+0 COV 8.1 18.0 81.0
+1 CORR 1
+1 CORR .25 1
+1 CORR .9 .5 1
+1 STDDEV 1 4 9
+END DATA.
+FORMATS var01 TO var03(F5.2).
+SPLIT FILE OFF.
+MCONVERT/APPEND.
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
+Table: Data List
+s,ROWTYPE_,VARNAME_,var01,var02,var03
+0,COV,var01,1.00,1.00,8.10
+0,COV,var02,1.00,16.00,18.00
+0,COV,var03,8.10,18.00,81.00
+0,CORR,var01,1.00,.25,.90
+0,CORR,var02,.25,1.00,.50
+0,CORR,var03,.90,.50,1.00
+0,STDDEV,,1.00,4.00,9.00
+1,CORR,var01,1.00,.25,.90
+1,CORR,var02,.25,1.00,.50
+1,CORR,var03,.90,.50,1.00
+1,STDDEV,,1.00,4.00,9.00
+1,COV,var01,1.00,1.00,8.10
+1,COV,var02,1.00,16.00,18.00
+1,COV,var03,8.10,18.00,81.00
+])
+AT_CLEANUP
+
+AT_SETUP([MCONVERT negative test])
+AT_DATA([mconvert.sps], [MCONVERT.
+])
+AT_CHECK([pspp mconvert.sps], [1], [dnl
+mconvert.sps:1: error: MCONVERT: No active file is defined and no external file
+is specified on MATRIX=IN.
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017, 2019 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([MEANS procedure])
+
+AT_SETUP([MEANS simple])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-simple.sps], [dnl
+data list notable list /hand * score * w *.
+begin data.
+1 17 4
+1 16 5
+2 21 1
+2 22 1
+2 20 8
+end data.
+
+weight by w.
+
+means tables = score by hand
+ /cells = mean count.
+])
+
+AT_CHECK([pspp -O format=csv means-simple.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * hand,19,100.0%,0,.0%,19,100.0%
+
+Table: Report
+hand,Mean,N
+1.00,16.44,9
+2.00,20.30,10
+Total,18.47,19
+])
+
+AT_CLEANUP
+
+AT_SETUP([MEANS very simple])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([very-simple.sps], [dnl
+data list notable list /score *.
+begin data.
+17
+17
+17
+16
+17
+16
+16
+16
+16
+21
+22
+20
+20
+20
+20
+20
+20
+20
+20
+end data.
+
+means tables = score
+ /cells = mean count.
+])
+
+AT_CHECK([pspp -O format=csv very-simple.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score,19,100.0%,0,.0%,19,100.0%
+
+Table: Report
+Mean,N
+18.47,19
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([MEANS empty factor spec])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-bad.sps], [dnl
+data list list /outcome *.
+begin data.
+1
+2
+3
+end data.
+
+MEANS TABLES = outcome
+ BY.
+])
+
+AT_CHECK([pspp -O format=csv means-bad.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([MEANS parser bug])
+AT_KEYWORDS([categorical categoricals])
+
+dnl This bug caused an infinite loop
+AT_DATA([means-bad.sps], [dnl
+DATA LIST notable LIST /a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 fylo *.
+begin data.
+1 2 3 4 5 6 7 8 9 0 11
+end data.
+
+MEANS TABLES = a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a BY fylo.
+])
+
+AT_CHECK([pspp -O format=csv means-bad.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+dnl This example is based upon info from https://libguides.library.kent.edu/SPSS/CompareMeans
+AT_SETUP([MEANS default missing behaviour])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-missing.sps], [dnl
+data list notable list /w * score * a * b *.
+begin data
+ 12 . 0 0
+ 13 . 0 1
+ 11 . 1 0
+ 7 . 1 1
+ 5 1 0 .
+ 91 1 0 0
+ 130 1 0 1
+ 4 1 1 .
+ 90 1 1 0
+ 72 1 1 1
+end data.
+
+weight by w.
+
+MEANS tables=score
+ /cells = count.
+
+MEANS tables=score by a
+ /cells = count.
+
+MEANS tables=score by a by b
+ /cells = count.
+])
+
+AT_CHECK([pspp -O format=csv means-missing.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score,392,90.1%,43,9.9%,435,100.0%
+
+Table: Report
+N
+392
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * a,392,90.1%,43,9.9%,435,100.0%
+
+Table: Report
+a,N
+.00,226
+1.00,166
+Total,392
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * a * b,383,88.0%,52,12.0%,435,100.0%
+
+Table: Report
+a,b,N
+.00,.00,91
+,1.00,130
+,Total,221
+1.00,.00,90
+,1.00,72
+,Total,162
+Total,.00,181
+,1.00,202
+,Total,383
+])
+
+AT_CLEANUP
+
+
+dnl This example from https://www.spss-tutorials.com/spss-means-command/
+AT_SETUP([MEANS two way])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-freelancer.sps], [dnl
+data list notable list /income_2010 * gender sector_2010.
+begin data
+6072.40 0 5
+12706.65 1 4
+14912.82 0 2
+16338.36 1 5
+22606.99 0 .
+23544.95 1 1
+24985.21 0 2
+26586.48 0 1
+29076.24 1 3
+31010.18 0 2
+33190.63 1 1
+35570.67 1 4
+36202.60 1 4
+36205.85 1 2
+36262.56 1 .
+38283.56 0 1
+38569.91 1 5
+39057.56 1 4
+39594.68 1 5
+42087.38 0 1
+42370.92 0 2
+42931.32 1 2
+45907.58 0 4
+45911.32 1 .
+47227.09 1 3
+50440.71 1 5
+57440.17 1 3
+58918.86 0 5
+59430.07 1 2
+61135.95 0 4
+64193.85 0 4
+64857.02 0 3
+65903.42 0 4
+66592.38 1 3
+70986.10 0 3
+71229.94 0 4
+74663.05 1 4
+76676.14 1 4
+79260.80 0 4
+80311.71 0 4
+end data.
+
+means income_2010 by gender by sector_2010
+ /cells count min mean stddev.
+])
+
+AT_CHECK([pspp -O format=csv means-freelancer.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+income_2010 * gender * sector_2010,37,92.5%,3,7.5%,40,100.0%
+
+Table: Report
+gender,sector_2010,N,Minimum,Mean,Std. Deviation
+.00,1.00,3,26586.48,35652.47,8078.46
+,2.00,4,14912.82,28319.78,11482.43
+,3.00,2,64857.02,67921.56,4333.91
+,4.00,7,45907.58,66849.04,11787.11
+,5.00,2,6072.40,32495.63,37368.09
+,Total,18,6072.40,49389.68,22371.48
+1.00,1.00,2,23544.95,28367.79,6820.53
+,2.00,3,36205.85,46189.08,11949.93
+,3.00,4,29076.24,50083.97,16084.44
+,4.00,6,12706.65,45812.78,24995.16
+,5.00,4,16338.36,36235.92,14311.04
+,Total,19,12706.65,42918.90,17851.64
+Total,1.00,5,23544.95,32738.60,7757.62
+,2.00,7,14912.82,35978.05,14309.27
+,3.00,6,29076.24,56029.83,15615.06
+,4.00,13,12706.65,57139.99,21187.85
+,5.00,6,6072.40,34989.15,20146.69
+,Total,37,6072.40,46066.84,20160.12
+])
+
+AT_CLEANUP
+
+
+dnl Check that rows are suppressed and that things generally work ok
+dnl when there are a 2 way instance contains an unbalanced set of
+dnl categorical values.
+AT_SETUP([MEANS unbalanced])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-unbalanced.sps], [dnl
+data list notable list /b c x *.
+begin data.
+4 1 123
+3 1 123
+5 0 246
+4 0 246
+3 0 246
+end data.
+
+* The data above lack a 5 1 case.
+
+means
+ table=x by b by c
+ /cells = mean count
+ .
+])
+
+AT_CHECK([pspp -O format=csv means-unbalanced.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+x * b * c,5,100.0%,0,.0%,5,100.0%
+
+Table: Report
+b,c,Mean,N
+3.00,.00,246.00,1
+,1.00,123.00,1
+,Total,184.50,2
+4.00,.00,246.00,1
+,1.00,123.00,1
+,Total,184.50,2
+5.00,.00,246.00,1
+,Total,246.00,1
+Total,.00,246.00,3
+,1.00,123.00,2
+,Total,196.80,5
+])
+
+AT_CLEANUP
+
+dnl This example kindly provided by Dana Williams
+AT_SETUP([MEANS three way])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-threeway.sps], [dnl
+data list notable list /score a b c.
+begin data.
+3 0 0 0
+4 0 0 1
+41 0 0 2
+5 0 1 0
+6 0 1 1
+7 1 0 0
+8 1 0 1
+9 1 1 0
+10 1 1 1
+end data.
+
+means score by a by b by c.
+])
+
+AT_CHECK([pspp -O format=csv means-threeway.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * a * b * c,9,100.0%,0,.0%,9,100.0%
+
+Table: Report
+a,b,c,Mean,N,Std. Deviation
+.00,.00,.00,3.00,1,NaN
+,,1.00,4.00,1,NaN
+,,2.00,41.00,1,NaN
+,,Total,16.00,3,21.66
+,1.00,.00,5.00,1,NaN
+,,1.00,6.00,1,NaN
+,,Total,5.50,2,.71
+,Total,.00,4.00,2,1.41
+,,1.00,5.00,2,1.41
+,,2.00,41.00,1,NaN
+,,Total,11.80,5,16.36
+1.00,.00,.00,7.00,1,NaN
+,,1.00,8.00,1,NaN
+,,Total,7.50,2,.71
+,1.00,.00,9.00,1,NaN
+,,1.00,10.00,1,NaN
+,,Total,9.50,2,.71
+,Total,.00,8.00,2,1.41
+,,1.00,9.00,2,1.41
+,,Total,8.50,4,1.29
+Total,.00,.00,5.00,2,2.83
+,,1.00,6.00,2,2.83
+,,2.00,41.00,1,NaN
+,,Total,12.60,5,16.01
+,1.00,.00,7.00,2,2.83
+,,1.00,8.00,2,2.83
+,,Total,7.50,4,2.38
+,Total,.00,6.00,4,2.58
+,,1.00,7.00,4,2.58
+,,2.00,41.00,1,NaN
+,,Total,10.33,9,11.73
+])
+
+AT_CLEANUP
+
+dnl The above example again, but with string variables for
+dnl the control vars.
+AT_SETUP([MEANS three way string])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-threeway-string.sps], [dnl
+data list notable list /score (f22.2) a (a24) b (a16) c (a8).
+begin data.
+3 fooberrycrumblexzaQ fosilationwereqd zero
+4 fooberrycrumblexzaQ fosilationwereqd one
+41 fooberrycrumblexzaQ fosilationwereqd two
+5 fooberrycrumblexzaQ onlyonekonboys zero
+6 fooberrycrumblexzaQ onlyonekonboys one
+7 wontledingbatsXASDF fosilationwereqd zero
+8 wontledingbatsXASDF fosilationwereqd one
+9 wontledingbatsXASDF onlyonekonboys zero
+10 wontledingbatsXASDF onlyonekonboys one
+end data.
+
+means score by a by b by c.
+])
+
+AT_CHECK([pspp -O format=csv means-threeway-string.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * a * b * c,9,100.0%,0,.0%,9,100.0%
+
+Table: Report
+a,b,c,Mean,N,Std. Deviation
+fooberrycrumblexzaQ,fosilationwereqd,one,4.00,1,NaN
+,,two,41.00,1,NaN
+,,zero,3.00,1,NaN
+,,Total,16.00,3,21.66
+,onlyonekonboys,one,6.00,1,NaN
+,,zero,5.00,1,NaN
+,,Total,5.50,2,.71
+,Total,one,5.00,2,1.41
+,,two,41.00,1,NaN
+,,zero,4.00,2,1.41
+,,Total,11.80,5,16.36
+wontledingbatsXASDF,fosilationwereqd,one,8.00,1,NaN
+,,zero,7.00,1,NaN
+,,Total,7.50,2,.71
+,onlyonekonboys,one,10.00,1,NaN
+,,zero,9.00,1,NaN
+,,Total,9.50,2,.71
+,Total,one,9.00,2,1.41
+,,zero,8.00,2,1.41
+,,Total,8.50,4,1.29
+Total,fosilationwereqd,one,6.00,2,2.83
+,,two,41.00,1,NaN
+,,zero,5.00,2,2.83
+,,Total,12.60,5,16.01
+,onlyonekonboys,one,8.00,2,2.83
+,,zero,7.00,2,2.83
+,,Total,7.50,4,2.38
+,Total,one,7.00,4,2.58
+,,two,41.00,1,NaN
+,,zero,6.00,4,2.58
+,,Total,10.33,9,11.73
+])
+
+AT_CLEANUP
+
+
+
+dnl An example with multiple tables
+AT_SETUP([MEANS multiple tables])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-multi-table.sps], [dnl
+data list notable list /a * b * c * x * y *.
+begin data.
+6 3 0 123 456
+6 3 1 123 456
+6 4 0 123 456
+6 4 1 123 456
+6 5 0 123 456
+6 5 1 123 456
+7 3 0 123 456
+7 3 1 123 456
+7 4 0 123 456
+7 4 1 123 456
+7 5 0 123 456
+7 5 1 123 456
+8 3 0 123 456
+8 3 1 123 456
+8 4 0 123 456
+8 4 1 123 456
+8 5 0 123 456
+8 5 1 123 456
+9 3 0 123 456
+9 3 1 123 456
+9 4 0 123 456
+9 4 1 123 456
+9 5 0 123 456
+9 5 1 123 456
+end data.
+
+
+means table = x by b by c
+ /x by b
+ /y by a by b
+ cells = min count .
+])
+
+AT_CHECK([pspp -O format=csv means-multi-table.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+x * b * c,24,100.0%,0,.0%,24,100.0%
+
+Table: Report
+b,c,Minimum,N
+3.00,.00,123.00,4
+,1.00,123.00,4
+,Total,123.00,8
+4.00,.00,123.00,4
+,1.00,123.00,4
+,Total,123.00,8
+5.00,.00,123.00,4
+,1.00,123.00,4
+,Total,123.00,8
+Total,.00,123.00,12
+,1.00,123.00,12
+,Total,123.00,24
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+x * b,24,100.0%,0,.0%,24,100.0%
+
+Table: Report
+b,Minimum,N
+3.00,123.00,8
+4.00,123.00,8
+5.00,123.00,8
+Total,123.00,24
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+y * a * b,24,100.0%,0,.0%,24,100.0%
+
+Table: Report
+a,b,Minimum,N
+6.00,3.00,456.00,2
+,4.00,456.00,2
+,5.00,456.00,2
+,Total,456.00,6
+7.00,3.00,456.00,2
+,4.00,456.00,2
+,5.00,456.00,2
+,Total,456.00,6
+8.00,3.00,456.00,2
+,4.00,456.00,2
+,5.00,456.00,2
+,Total,456.00,6
+9.00,3.00,456.00,2
+,4.00,456.00,2
+,5.00,456.00,2
+,Total,456.00,6
+Total,3.00,456.00,8
+,4.00,456.00,8
+,5.00,456.00,8
+,Total,456.00,24
+])
+
+AT_CLEANUP
+
+
+
+dnl An example with more than one dependent variable.
+dnl This case uses a somewhat different table layout.
+AT_SETUP([MEANS multi variable])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-multi-variable.sps], [dnl
+data list notable list /b c x y.
+begin data.
+5 1 123 55
+5 1 123 55
+5 1 123 55
+5 1 123 55
+4 1 456 44
+4 1 456 44
+4 1 456 44
+4 1 456 44
+3 1 789 55
+3 1 789 55
+3 1 789 55
+3 1 789 55
+5 0 246 99
+5 0 246 99
+5 0 246 99
+5 0 246 .
+4 0 987 99
+4 0 987 99
+4 0 987 99
+4 0 987 99
+3 0 654 11
+3 0 654 11
+3 0 654 11
+3 0 654 11
+end data.
+
+means
+ table = x y by b by c
+ .
+])
+
+AT_CHECK([pspp -O format=csv means-multi-variable.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+x * b * c,24,100.0%,0,.0%,24,100.0%
+y * b * c,23,95.8%,1,4.2%,24,100.0%
+
+Table: x * y * b * c
+b,c,,x,y
+3.00,.00,Mean,654.00,11.00
+,,N,4,4
+,,Std. Deviation,.00,.00
+,1.00,Mean,789.00,55.00
+,,N,4,4
+,,Std. Deviation,.00,.00
+,Total,Mean,721.50,33.00
+,,N,8,8
+,,Std. Deviation,72.16,23.52
+4.00,.00,Mean,987.00,99.00
+,,N,4,4
+,,Std. Deviation,.00,.00
+,1.00,Mean,456.00,44.00
+,,N,4,4
+,,Std. Deviation,.00,.00
+,Total,Mean,721.50,71.50
+,,N,8,8
+,,Std. Deviation,283.83,29.40
+5.00,.00,Mean,246.00,99.00
+,,N,4,3
+,,Std. Deviation,.00,.00
+,1.00,Mean,123.00,55.00
+,,N,4,4
+,,Std. Deviation,.00,.00
+,Total,Mean,184.50,73.86
+,,N,8,7
+,,Std. Deviation,65.75,23.52
+Total,.00,Mean,629.00,67.00
+,,N,12,11
+,,Std. Deviation,316.50,44.40
+,1.00,Mean,456.00,51.33
+,,N,12,12
+,,Std. Deviation,283.98,5.42
+,Total,Mean,542.50,58.83
+,,N,24,23
+,,Std. Deviation,307.06,31.22
+])
+
+
+AT_CLEANUP
+
+
+dnl This example is based upon one kindly provided by Dana Williams
+dnl It exercises the most complex case where there are multiple
+dnl dependent variables AND multiple control variables in each layer.
+AT_SETUP([MEANS multi combination])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-multi-combination.sps], [dnl
+data list notable list /one (F22.5) two (F22.5) three four five six.
+begin data
+1 1 1 1 1 1
+2 1 1 1 1 1
+1 2 1 1 1 1
+2 2 1 1 1 1
+1 1 2 1 1 1
+2 1 2 1 1 1
+1 2 2 1 1 1
+2 2 2 1 1 1
+1 1 1 2 1 1
+2 1 1 2 1 1
+1 2 1 2 1 1
+2 2 1 2 1 1
+1 1 2 2 1 1
+2 1 2 2 1 1
+1 2 2 2 1 1
+2 2 2 2 1 1
+1 1 1 1 2 1
+2 1 1 1 2 1
+1 2 1 1 2 1
+2 2 1 1 2 1
+1 1 2 1 2 1
+2 1 2 1 2 1
+1 2 2 1 2 1
+2 2 2 1 2 1
+1 1 1 2 2 1
+2 1 1 2 2 1
+1 2 1 2 2 1
+2 2 1 2 2 1
+1 1 2 2 2 1
+2 1 2 2 2 1
+1 2 2 2 2 1
+2 2 2 2 2 1
+1 1 1 1 1 2
+2 1 1 1 1 2
+1 2 1 1 1 2
+2 2 1 1 1 2
+1 1 2 1 1 2
+2 1 2 1 1 2
+1 2 2 1 1 2
+2 2 2 1 1 2
+1 1 1 2 1 2
+2 1 1 2 1 2
+1 2 1 2 1 2
+2 2 1 2 1 2
+1 1 2 2 1 2
+2 1 2 2 1 2
+1 2 2 2 1 2
+2 2 2 2 1 2
+1 1 1 1 2 2
+2 1 1 1 2 2
+1 2 1 1 2 2
+2 2 1 1 2 2
+1 1 2 1 2 2
+2 1 2 1 2 2
+1 2 2 1 2 2
+2 2 2 1 2 2
+1 1 1 2 2 2
+2 1 1 2 2 2
+1 2 1 2 2 2
+2 2 1 2 2 2
+1 1 2 2 2 2
+2 1 2 2 2 2
+1 2 2 2 2 2
+2 2 2 2 2 2
+end data.
+
+recode six (2 = 62) (1 = 61).
+recode five (2 = 52) (1 = 51).
+recode four (2 = 42) (1 = 41).
+recode three (2 = 32) (1 = 31).
+
+means tables = one two BY three four BY five six.
+])
+
+AT_CHECK([pspp -O format=csv means-multi-combination.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+one * three * five,64,100.0%,0,.0%,64,100.0%
+two * three * five,64,100.0%,0,.0%,64,100.0%
+one * three * six,64,100.0%,0,.0%,64,100.0%
+two * three * six,64,100.0%,0,.0%,64,100.0%
+one * four * five,64,100.0%,0,.0%,64,100.0%
+two * four * five,64,100.0%,0,.0%,64,100.0%
+one * four * six,64,100.0%,0,.0%,64,100.0%
+two * four * six,64,100.0%,0,.0%,64,100.0%
+
+Table: one * two * three * five
+three,five,,one,two
+31.00,51.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,52.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+32.00,51.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,52.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+Total,51.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,52.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,Total,Mean,1.50000,1.50000
+,,N,64,64
+,,Std. Deviation,.50395,.50395
+
+Table: one * two * three * six
+three,six,,one,two
+31.00,61.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,62.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+32.00,61.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,62.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+Total,61.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,62.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,Total,Mean,1.50000,1.50000
+,,N,64,64
+,,Std. Deviation,.50395,.50395
+
+Table: one * two * four * five
+four,five,,one,two
+41.00,51.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,52.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+42.00,51.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,52.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+Total,51.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,52.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,Total,Mean,1.50000,1.50000
+,,N,64,64
+,,Std. Deviation,.50395,.50395
+
+Table: one * two * four * six
+four,six,,one,two
+41.00,61.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,62.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+42.00,61.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,62.00,Mean,1.50000,1.50000
+,,N,16,16
+,,Std. Deviation,.51640,.51640
+,Total,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+Total,61.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,62.00,Mean,1.50000,1.50000
+,,N,32,32
+,,Std. Deviation,.50800,.50800
+,Total,Mean,1.50000,1.50000
+,,N,64,64
+,,Std. Deviation,.50395,.50395
+])
+
+AT_CLEANUP
+
+
+dnl This example was observed to cause a crash in the
+dnl destructor. Found by zzuf.
+AT_SETUP([MEANS clean up])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-bad.sps], [dnl
+data list notable list /one two three four five six.
+begin data
+1 1 1 1 1 1
+2 1 1 1 1 !
+1 2 2 2 2 2
+2 2 2 2 2 2
+end data.
+
+means tables = one two BY thsee four BY five six.
+])
+
+AT_CHECK([pspp -O format=csv means-bad.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+dnl Another example which caused a crash.
+dnl Found by zzuf.
+AT_SETUP([MEANS control all missing])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-bad.sps], [dnl
+data list notable list /a * b * y * uu *.
+begin data.
+6 3 . 5
+6 3 . 5
+6 4 . 5
+end data.
+
+means table = b by a by y by uu
+ .
+])
+
+AT_CHECK([pspp -O format=csv means-bad.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+b * a * y * uu,0,.0%,3,100.0%,3,100.0%
+
+"warning: The table ""a * y * uu"" has no non-empty control variables. No result for this table will be displayed."
+])
+
+AT_CLEANUP
+
+
+dnl Do some tests on the MISSING keyword.
+AT_SETUP([MEANS missing classes])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-missing-classes.sps], [dnl
+data list notable list /hand * score *.
+begin data.
+1 17
+1 17
+1 17
+1 16
+1 17
+1 16
+1 16
+1 .
+1 99
+2 21
+2 22
+2 20
+2 20
+2 20
+2 20
+2 20
+2 20
+2 20
+2 20
+9 55
+end data.
+
+missing values score (99).
+missing values hand (9).
+
+means tables=score by hand
+ /cells = count max
+ /missing = dependent
+ .
+
+means tables=score by hand
+ /cells = count max
+ /missing = include
+ .
+
+means tables=score by hand
+ /cells = count max
+ .
+
+])
+
+AT_CHECK([pspp -O format=csv means-missing-classes.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * hand,18,90.0%,2,10.0%,20,100.0%
+
+Table: Report
+hand,N,Maximum
+1.00,7,17.00
+2.00,10,22.00
+9.00,1,55.00
+Total,18,55.00
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * hand,19,95.0%,1,5.0%,20,100.0%
+
+Table: Report
+hand,N,Maximum
+1.00,8,99.00
+2.00,10,22.00
+9.00,1,55.00
+Total,19,99.00
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+score * hand,17,85.0%,3,15.0%,20,100.0%
+
+Table: Report
+hand,N,Maximum
+1.00,7,17.00
+2.00,10,22.00
+Total,17,22.00
+])
+
+AT_CLEANUP
+
+
+dnl Make sure that behaviour with SPLIT is correct.
+AT_SETUP([MEANS split])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-split.sps], [dnl
+data list notable list /b g *.
+begin data
+2 0
+2 0
+4 0
+4 0
+11 1
+11 1
+end data.
+
+split file by g.
+
+means b /cells = count mean.
+])
+
+AT_CHECK([pspp -O format=csv means-split.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+b,4,100.0%,0,.0%,4,100.0%
+
+Table: Report
+N,Mean
+4,3.00
+
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+b,2,100.0%,0,.0%,2,100.0%
+
+Table: Report
+N,Mean
+2,11.00
+])
+
+AT_CLEANUP
+
+
+dnl Test the output with unusual dependent variable formats
+AT_SETUP([MEANS formats])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([means-formats.sps], [dnl
+data list notable list /hours (TIME11.0) rate (DOLLAR8.2).
+begin data
+12:00 4.09
+14:01 5.23
+end data.
+
+means hours rate
+ /cells = mean count max range.
+])
+
+AT_CHECK([pspp -O format=csv means-formats.sps], [0], [dnl
+Table: Case Processing Summary
+,Cases,,,,,
+,Included,,Excluded,,Total,
+,N,Percent,N,Percent,N,Percent
+hours,2,100.0%,0,.0%,2,100.0%
+rate,2,100.0%,0,.0%,2,100.0%
+
+Table: hours * rate
+,hours,rate
+Mean,13:00:30,$4.66
+N,2,2
+Maximum,14:01:00,$5.23
+Range,02:01:00,$1.14
+])
+
+AT_CLEANUP
+
+AT_SETUP([MEANS syntax errors])
+AT_DATA([means.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+MEANS TABLES **.
+MEANS x BY **.
+MEANS x/MISSING=**.
+MEANS x/CELLS=**.
+MEANS x/ **.
+])
+AT_CHECK([pspp -O format=csv means.sps], [1], [dnl
+"means.sps:2.14-2.15: error: MEANS: Syntax error expecting `='.
+ 2 | MEANS TABLES **.
+ | ^~"
+
+"means.sps:3.12-3.13: error: MEANS: Syntax error expecting variable name.
+ 3 | MEANS x BY **.
+ | ^~"
+
+"means.sps:4.17-4.18: error: MEANS: Syntax error expecting INCLUDE or DEPENDENT.
+ 4 | MEANS x/MISSING=**.
+ | ^~"
+
+"means.sps:5.15-5.16: error: MEANS: Syntax error expecting one of the following: MEAN, COUNT, STDDEV, SEMEAN, SUM, MIN, MAX, RANGE, VARIANCE, KURT, SEKURT, SKEW, SESKEW, FIRST, LAST, HARMONIC, GEOMETRIC.
+ 5 | MEANS x/CELLS=**.
+ | ^~"
+
+"means.sps:6.10-6.11: error: MEANS: Syntax error expecting MISSING or CELLS.
+ 6 | MEANS x/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([MISSING VALUES])
+
+AT_SETUP([MISSING VALUES valid cases])
+AT_DATA([missing-values.sps], [dnl
+DATA LIST NOTABLE/str1 1-5 (A) str2 6-8 (A) date1 9-19 (DATE) num1 20-25
+ longstr 26-36 (A).
+
+* Numeric missing values.
+MISSING VALUES date1 num1 (1).
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES date1 num1 (1, 2).
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES date1 num1 (1, 2, 3).
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES date1 num1 (9999998, 9999984, 3).
+DISPLAY DICTIONARY date1 num1.
+
+* Numeric missing values using the first variable's format.
+MISSING VALUES num1 date1 ('1').
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES num1 date1 ('1', '2').
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES num1 date1 ('1', '2', '3').
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES date1 num1 ('06-AUG-05').
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES date1 num1 ('06-AUG-05', '01-OCT-78').
+DISPLAY DICTIONARY date1 num1.
+MISSING VALUES date1 num1 ('06-AUG-05', '01-OCT-78', '14-FEB-81').
+DISPLAY DICTIONARY date1 num1.
+
+* Ranges of numeric missing values.
+MISSING VALUES num1 (1 THRU 2).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (LO THRU 2).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (LOWEST THRU 2).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (1 THRU HI).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (1 THRU HIGHEST).
+DISPLAY DICTIONARY num1.
+
+* A range of numeric missing values, plus an individual value.
+MISSING VALUES num1 (1 THRU 2, 3).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (LO THRU 2, 3).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (LOWEST THRU 2, 3).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (1 THRU HI, -1).
+DISPLAY DICTIONARY num1.
+MISSING VALUES num1 (1 THRU HIGHEST, -1).
+DISPLAY DICTIONARY num1.
+
+* String missing values.
+MISSING VALUES str1 str2 longstr ('abc ','def').
+DISPLAY DICTIONARY str1 str2 longstr.
+
+* May mix variable types when clearing missing values.
+MISSING VALUES ALL ().
+MISSING VALUES num1 (1).
+DISPLAY DICTIONARY
+])
+AT_CHECK([pspp -o pspp.csv missing-values.sps])
+AT_CHECK([cat pspp.csv | sed '/^Table/d
+/^Name/d
+s/^\([[a-z0-9]]*\),.*,\([[^,]]*\)$/\1: \2/'], [0], [dnl
+date1: 1
+num1: 1
+
+date1: 1; 2
+num1: 1; 2
+
+date1: 1; 2; 3
+num1: 1; 2; 3
+
+date1: 9999998; 9999984; 3
+num1: 9999998; 9999984; 3
+
+date1: 1
+num1: 1
+
+date1: 1; 2
+num1: 1; 2
+
+date1: 1; 2; 3
+num1: 1; 2; 3
+
+date1: 13342665600
+num1: 13342665600
+
+date1: 13342665600; 12495427200
+num1: 13342665600; 12495427200
+
+date1: 13342665600; 12495427200; 12570336000
+num1: 13342665600; 12495427200; 12570336000
+
+num1: 1 THRU 2
+
+num1: LOWEST THRU 2
+
+num1: LOWEST THRU 2
+
+num1: 1 THRU HIGHEST
+
+num1: 1 THRU HIGHEST
+
+num1: 1 THRU 2; 3
+
+num1: LOWEST THRU 2; 3
+
+num1: LOWEST THRU 2; 3
+
+num1: 1 THRU HIGHEST; -1
+
+num1: 1 THRU HIGHEST; -1
+
+str1: """abc ""; ""def """
+str2: """abc""; ""def"""
+longstr: """abc ""; ""def """
+
+str1: @&t@
+str2: @&t@
+date1: @&t@
+num1: 1
+longstr: @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([MISSING VALUES invalid cases])
+AT_DATA([missing-values.sps], [dnl
+DATA LIST NOTABLE/str1 1-5 (A) str2 6-8 (A) date1 9-19 (DATE) num1 20-25
+ longstr 26-36 (A).
+
+* Too long for str2.
+MISSING VALUES str1 str2 longstr ('abcde').
+
+* Long string missing value longer than 8 bytes.
+MISSING VALUES longstr ('abcdefghijk').
+
+* No string ranges.
+MISSING VALUES str1 ('a' THRU 'z').
+
+* Mixing string and numeric variables.
+MISSING VALUES str1 num1 ('123').
+
+* Too many values.
+MISSING VALUES num1 (1, 2, 3, 4).
+MISSING VALUES num1 (1 THRU 2, 3 THRU 4).
+MISSING VALUES num1 (1, 2 THRU 3, 4).
+MISSING VALUES str1 ('abc', 'def', 'ghi', 'jkl').
+
+* Bad range.
+MISSING VALUES num1 (2 THRU 1).
+])
+AT_CHECK([pspp -O format=csv missing-values.sps], [1], [dnl
+"missing-values.sps:5.35-5.41: error: MISSING VALUES: Missing values are too long to assign to variable str2 with width 3.
+ 5 | MISSING VALUES str1 str2 longstr ('abcde').
+ | ^~~~~~~"
+
+"missing-values.sps:8.25-8.37: error: MISSING VALUES: Truncating missing value to maximum acceptable length (8 bytes).
+ 8 | MISSING VALUES longstr ('abcdefghijk').
+ | ^~~~~~~~~~~~~"
+
+"missing-values.sps:11.26-11.29: error: MISSING VALUES: Syntax error expecting string.
+ 11 | MISSING VALUES str1 ('a' THRU 'z').
+ | ^~~~"
+
+"missing-values.sps:14.27-14.31: error: MISSING VALUES: Cannot assign string missing values to numeric variable num1.
+ 14 | MISSING VALUES str1 num1 ('123').
+ | ^~~~~"
+
+"missing-values.sps:17.22-17.31: error: MISSING VALUES: Too many numeric missing values. At most three individual values or one value and one range are allowed.
+ 17 | MISSING VALUES num1 (1, 2, 3, 4).
+ | ^~~~~~~~~~"
+
+"missing-values.sps:18.22-18.39: error: MISSING VALUES: Too many numeric missing values. At most three individual values or one value and one range are allowed.
+ 18 | MISSING VALUES num1 (1 THRU 2, 3 THRU 4).
+ | ^~~~~~~~~~~~~~~~~~"
+
+"missing-values.sps:19.22-19.35: error: MISSING VALUES: Too many numeric missing values. At most three individual values or one value and one range are allowed.
+ 19 | MISSING VALUES num1 (1, 2 THRU 3, 4).
+ | ^~~~~~~~~~~~~~"
+
+"missing-values.sps:20.22-20.47: error: MISSING VALUES: Too many string missing values. At most three individual values are allowed.
+ 20 | MISSING VALUES str1 ('abc', 'def', 'ghi', 'jkl').
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"missing-values.sps:23.22-23.29: warning: MISSING VALUES: The high end of the range (1) is below the low end (2). The range will be treated as if reversed.
+ 23 | MISSING VALUES num1 (2 THRU 1).
+ | ^~~~~~~~"
+])
+AT_CLEANUP
+
+AT_SETUP([MISSING VALUES syntax errors])
+AT_DATA([missing-values.sps], [dnl
+DATA LIST LIST NOTABLE/n1 to n10 (F8.2) s1 to s10 (A8).
+MISSING VALUES **.
+MISSING VALUES n1 **.
+MISSING VALUES s1 (1).
+MISSING VALUES n1 (1**).
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='missing-values.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"missing-values.sps:2.16-2.17: error: MISSING VALUES: Syntax error expecting variable name.
+ 2 | MISSING VALUES **.
+ | ^~"
+
+"missing-values.sps:3.19-3.20: error: MISSING VALUES: Syntax error expecting `@{:@'.
+ 3 | MISSING VALUES n1 **.
+ | ^~"
+
+"missing-values.sps:4.20: error: MISSING VALUES: Syntax error expecting string.
+ 4 | MISSING VALUES s1 (1).
+ | ^"
+
+"missing-values.sps:5.21-5.22: error: MISSING VALUES: Syntax error expecting number.
+ 5 | MISSING VALUES n1 (1**).
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([MRSETS])
+
+m4_define([DEFINE_MRSETS_DATA],
+ [DATA LIST NOTABLE /w x y z 1-4 a b c d 5-8 (a).
+BEGIN DATA.
+1234acbd
+5678efgh
+END DATA.])
+
+m4_define([DEFINE_MRSETS],
+ [DEFINE_MRSETS_DATA
+
+[VARIABLE LABEL
+ w 'duplicate variable label'
+ x 'Variable x'
+ z 'Duplicate variable label'.
+VALUE LABELS
+ /w 1 'w value 1'
+ /y 1 'duplicate Value label'
+ /z 1 'duplicate value Label'
+ /a b c d 'a' 'burger' 'b' 'fries' 'c' 'shake' 'd' 'taco'.
+ADD VALUE LABELS
+ /b 'b' 'Fries'
+ /c 'b' 'XXX'.
+MRSETS
+ /MDGROUP NAME=$a
+ LABEL='First multiple dichotomy group'
+ CATEGORYLABELS=VARLABELS
+ VARIABLES=w x y z
+ VALUE=5
+ /MDGROUP NAME=$b
+ CATEGORYLABELS=COUNTEDVALUES
+ VARIABLES=z y
+ VALUE=123
+ /MDGROUP NAME=$c
+ LABELSOURCE=VARLABEL
+ CATEGORYLABELS=COUNTEDVALUES
+ VARIABLES=w x y z
+ VALUE=1
+ /MDGROUP NAME=$d
+ LABELSOURCE=VARLABEL
+ VARIABLES=a b c d
+ VALUE='c'
+ /MCGROUP NAME=$e
+ LABEL='First multiple category group'
+ VARIABLES=w x y z
+ /MCGROUP NAME=$f
+ VARIABLES=a b c d.
+]])
+
+m4_define([DEFINE_MRSETS_OUTPUT], [dnl
+"mrsets.sps:23.16-23.22: warning: MRSETS: Variables w and z specified as part of multiple dichotomy group $a have the same variable label. Categories represented by these variables will not be distinguishable in output.
+ 23 | VARIABLES=w x y z
+ | ^~~~~~~"
+
+"mrsets.sps:27.16-27.18: warning: MRSETS: Variable z specified as part of multiple dichotomy group $b (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
+ 27 | VARIABLES=z y
+ | ^~~"
+
+"mrsets.sps:27.16-27.18: warning: MRSETS: Variable y specified as part of multiple dichotomy group $b (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
+ 27 | VARIABLES=z y
+ | ^~~"
+
+"mrsets.sps:32.16-32.22: warning: MRSETS: Variable x specified as part of multiple dichotomy group $c (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
+ 32 | VARIABLES=w x y z
+ | ^~~~~~~"
+
+"mrsets.sps:32.16-32.22: warning: MRSETS: Variables y and z specified as part of multiple dichotomy group $c (which has CATEGORYLABELS=COUNTEDVALUES) have the same value label for the group's counted value. These categories will not be distinguishable in output.
+ 32 | VARIABLES=w x y z
+ | ^~~~~~~"
+
+"mrsets.sps:35.6-35.25: warning: MRSETS: MDGROUP subcommand for group $d specifies LABELSOURCE=VARLABEL but not CATEGORYLABELS=COUNTEDVALUES. Ignoring LABELSOURCE.
+ 35 | LABELSOURCE=VARLABEL
+ | ^~~~~~~~~~~~~~~~~~~~"
+
+"mrsets.sps:40.16-40.22: warning: MRSETS: Variables specified on MCGROUP should have the same categories, but w and y (and possibly others) in multiple category group $e have different value labels for value 1.
+ 40 | VARIABLES=w x y z
+ | ^~~~~~~"
+
+"mrsets.sps:42.16-42.22: warning: MRSETS: Variables specified on MCGROUP should have the same categories, but a and c (and possibly others) in multiple category group $f have different value labels for value b.
+ 42 | VARIABLES=a b c d.
+ | ^~~~~~~"
+])
+
+m4_define([MRSETS_DISPLAY_OUTPUT], [dnl
+Table: Multiple Response Sets
+Name,Label,Encoding,Counted Value,Member Variables
+$a,First multiple dichotomy group,Dichotomies,5,"w
+x
+y
+z"
+$b,,Dichotomies,123,"z
+y"
+$c,duplicate variable label,Dichotomies,1,"w
+x
+y
+z"
+$d,,Dichotomies,c,"a
+b
+c
+d"
+$e,First multiple category group,Categories,,"w
+x
+y
+z"
+$f,,Categories,,"a
+b
+c
+d"
+])
+
+AT_SETUP([MRSETS add, display, delete])
+AT_DATA([mrsets.sps],
+ [DEFINE_MRSETS
+[MRSETS
+ /DISPLAY NAME=[$a]
+ /DISPLAY NAME=ALL
+ /DELETE NAME=[$c]
+ /DISPLAY NAME=ALL
+ /DELETE NAME=ALL
+ /DISPLAY NAME=ALL.
+]])
+AT_CHECK([pspp -o - -O format=csv -o mrsets.csv -o mrsets.txt mrsets.sps], [0],
+ [DEFINE_MRSETS_OUTPUT
+Table: Multiple Response Sets
+Name,Label,Encoding,Counted Value,Member Variables
+$a,First multiple dichotomy group,Dichotomies,5,"w
+x
+y
+z"
+
+Table: Multiple Response Sets
+Name,Label,Encoding,Counted Value,Member Variables
+$a,First multiple dichotomy group,Dichotomies,5,"w
+x
+y
+z"
+$b,,Dichotomies,123,"z
+y"
+$c,duplicate variable label,Dichotomies,1,"w
+x
+y
+z"
+$d,,Dichotomies,c,"a
+b
+c
+d"
+$e,First multiple category group,Categories,,"w
+x
+y
+z"
+$f,,Categories,,"a
+b
+c
+d"
+
+Table: Multiple Response Sets
+Name,Label,Encoding,Counted Value,Member Variables
+$a,First multiple dichotomy group,Dichotomies,5,"w
+x
+y
+z"
+$b,,Dichotomies,123,"z
+y"
+$d,,Dichotomies,c,"a
+b
+c
+d"
+$e,First multiple category group,Categories,,"w
+x
+y
+z"
+$f,,Categories,,"a
+b
+c
+d"
+
+"mrsets.sps:50.19-50.21: note: MRSETS: The active dataset dictionary does not contain any multiple response sets.
+ 50 | /DISPLAY NAME=ALL.
+ | ^~~"
+])
+AT_CLEANUP
+
+AT_SETUP([MRSETS read and write])
+AT_DATA([mrsets.sps],
+ [DEFINE_MRSETS
+SAVE OUTFILE='mrsets.sav'.
+])
+AT_CHECK([pspp -O format=csv mrsets.sps], [0], [DEFINE_MRSETS_OUTPUT])
+AT_DATA([mrsets2.sps],
+ [GET FILE='mrsets.sav'.
+MRSETS /DISPLAY NAME=ALL.
+])
+AT_CHECK([pspp -O format=csv mrsets2.sps], [0], [MRSETS_DISPLAY_OUTPUT],
+ [], [hd mrsets.sav])
+AT_CLEANUP
+
+AT_SETUP([MRSETS syntax errors])
+AT_DATA([mrsets.sps], [dnl
+DATA LIST NOTABLE /w x y z 1-4 a b c d 5-8 (a).
+BEGIN DATA.
+1234acbd
+5678efgh
+END DATA.
+VARIABLE LABEL
+ w 'duplicate variable label'
+ x 'Variable x'
+ z 'Duplicate variable label'.
+VALUE LABELS
+ /w 1 'w value 1'
+ /y 1 'duplicate Value label'
+ /z 1 'duplicate value Label'
+ /a b c d 'a' 'burger' 'b' 'fries' 'c' 'shake' 'd' 'taco'.
+ADD VALUE LABELS
+ /b 'b' 'Fries'
+ /c 'b' 'XXX'.
+
+MRSETS /MDGROUP NAME **.
+MRSETS /MDGROUP NAME=**.
+MRSETS /MDGROUP NAME=x.
+MRSETS /MDGROUP VARIABLES **.
+MRSETS /MDGROUP VARIABLES=**.
+MRSETS /MDGROUP VARIABLES=a.
+MRSETS /MDGROUP LABEL **.
+MRSETS /MDGROUP LABEL=**.
+MRSETS /MDGROUP LABELSOURCE=**.
+MRSETS /MDGROUP VALUE **.
+MRSETS /MDGROUP VALUE=1.5.
+MRSETS /MDGROUP VALUE=**.
+MRSETS /MDGROUP CATEGORYLABELS **.
+MRSETS /MDGROUP CATEGORYLABELS=**.
+MRSETS /MDGROUP **.
+MRSETS /MCGROUP **.
+MRSETS /MDGROUP.
+MRSETS /MDGROUP NAME=[$x].
+MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE=1.
+MRSETS /MDGROUP NAME=[$x] VARIABLES=x y VALUE='a'.
+MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE='xyzzy'.
+MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE='y' LABELSOURCE=VARLABEL.
+MRSETS /MDGROUP NAME=[$x] VARIABLES=w z VALUE=1 CATEGORYLABELS=VARLABELS.
+MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE='y'
+ LABELSOURCE=VARLABEL CATEGORYLABELS=COUNTEDVALUES
+ LABEL='foo'.
+MRSETS /MDGROUP NAME=[$x] VARIABLES=y z VALUE=1
+ LABELSOURCE=VARLABEL CATEGORYLABELS=COUNTEDVALUES.
+MRSETS /MCGROUP NAME=[$x] VARIABLES=w x y z.
+
+MRSETS /DELETE **.
+MRSETS /DELETE NAME**.
+MRSETS /DELETE NAME=[[**]].
+MRSETS /DELETE NAME=[[$x]].
+MRSETS /DELETE NAME=**.
+
+MRSETS /DISPLAY NAME=ALL.
+])
+AT_CHECK([pspp -O format=csv mrsets.sps], [1], [dnl
+"mrsets.sps:19.22-19.23: error: MRSETS: Syntax error expecting `='.
+ 19 | MRSETS /MDGROUP NAME **.
+ | ^~"
+
+"mrsets.sps:20.22-20.23: error: MRSETS: Syntax error expecting identifier.
+ 20 | MRSETS /MDGROUP NAME=**.
+ | ^~"
+
+"mrsets.sps:21.22: error: MRSETS: x is not a valid name for a multiple response set. Multiple response set names must begin with `$'.
+ 21 | MRSETS /MDGROUP NAME=x.
+ | ^"
+
+"mrsets.sps:22.27-22.28: error: MRSETS: Syntax error expecting `='.
+ 22 | MRSETS /MDGROUP VARIABLES **.
+ | ^~"
+
+"mrsets.sps:23.27-23.28: error: MRSETS: Syntax error expecting variable name.
+ 23 | MRSETS /MDGROUP VARIABLES=**.
+ | ^~"
+
+"mrsets.sps:24.27: error: MRSETS: At least two variables are required.
+ 24 | MRSETS /MDGROUP VARIABLES=a.
+ | ^"
+
+"mrsets.sps:25.23-25.24: error: MRSETS: Syntax error expecting `='.
+ 25 | MRSETS /MDGROUP LABEL **.
+ | ^~"
+
+"mrsets.sps:26.23-26.24: error: MRSETS: Syntax error expecting string.
+ 26 | MRSETS /MDGROUP LABEL=**.
+ | ^~"
+
+"mrsets.sps:27.28-27.30: error: MRSETS: Syntax error expecting `=VARLABEL'.
+ 27 | MRSETS /MDGROUP LABELSOURCE=**.
+ | ^~~"
+
+"mrsets.sps:28.23-28.24: error: MRSETS: Syntax error expecting `='.
+ 28 | MRSETS /MDGROUP VALUE **.
+ | ^~"
+
+"mrsets.sps:29.23-29.25: error: MRSETS: Numeric VALUE must be an integer.
+ 29 | MRSETS /MDGROUP VALUE=1.5.
+ | ^~~"
+
+"mrsets.sps:30.23-30.24: error: MRSETS: Syntax error expecting integer or string.
+ 30 | MRSETS /MDGROUP VALUE=**.
+ | ^~"
+
+"mrsets.sps:31.32-31.33: error: MRSETS: Syntax error expecting `='.
+ 31 | MRSETS /MDGROUP CATEGORYLABELS **.
+ | ^~"
+
+"mrsets.sps:32.32-32.33: error: MRSETS: Syntax error expecting VARLABELS or COUNTEDVALUES.
+ 32 | MRSETS /MDGROUP CATEGORYLABELS=**.
+ | ^~"
+
+"mrsets.sps:33.17-33.18: error: MRSETS: Syntax error expecting NAME, VARIABLES, LABEL, LABELSOURCE, VALUE, or CATEGORYLABELS.
+ 33 | MRSETS /MDGROUP **.
+ | ^~"
+
+"mrsets.sps:34.17-34.18: error: MRSETS: Syntax error expecting NAME, VARIABLES, or LABEL.
+ 34 | MRSETS /MCGROUP **.
+ | ^~"
+
+"mrsets.sps:35.16: error: MRSETS: Required NAME specification missing from MDGROUP subcommand.
+ 35 | MRSETS /MDGROUP.
+ | ^"
+
+"mrsets.sps:36.24: error: MRSETS: Required VARIABLES specification missing from MDGROUP subcommand.
+ 36 | MRSETS /MDGROUP NAME=$x.
+ | ^"
+
+mrsets.sps:37: error: MRSETS: VARIABLES and VALUE must have the same type.
+
+"mrsets.sps:37.35-37.37: note: MRSETS: These are string variables.
+ 37 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE=1.
+ | ^~~"
+
+"mrsets.sps:37.45: note: MRSETS: This is a numeric value.
+ 37 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE=1.
+ | ^"
+
+mrsets.sps:38: error: MRSETS: VARIABLES and VALUE must have the same type.
+
+"mrsets.sps:38.35-38.37: note: MRSETS: These are numeric variables.
+ 38 | MRSETS /MDGROUP NAME=$x VARIABLES=x y VALUE='a'.
+ | ^~~"
+
+"mrsets.sps:38.45-38.47: note: MRSETS: This is a string value.
+ 38 | MRSETS /MDGROUP NAME=$x VARIABLES=x y VALUE='a'.
+ | ^~~"
+
+mrsets.sps:39: error: MRSETS: The VALUE string must be no longer than the narrowest variable in the group.
+
+"mrsets.sps:39.45-39.51: note: MRSETS: The VALUE string is 5 bytes long.
+ 39 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='xyzzy'.
+ | ^~~~~~~"
+
+"mrsets.sps:39.35-39.37: note: MRSETS: Variable a has a width of 1 bytes.
+ 39 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='xyzzy'.
+ | ^~~"
+
+"mrsets.sps:40.49-40.68: warning: MRSETS: MDGROUP subcommand for group $x specifies LABELSOURCE=VARLABEL but not CATEGORYLABELS=COUNTEDVALUES. Ignoring LABELSOURCE.
+ 40 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='y' LABELSOURCE=VARLABEL.
+ | ^~~~~~~~~~~~~~~~~~~~"
+
+"mrsets.sps:41.35-41.37: warning: MRSETS: Variables w and z specified as part of multiple dichotomy group $x have the same variable label. Categories represented by these variables will not be distinguishable in output.
+ 41 | MRSETS /MDGROUP NAME=$x VARIABLES=w z VALUE=1 CATEGORYLABELS=VARLABELS.
+ | ^~~"
+
+"mrsets.sps:44: warning: MRSETS: MDGROUP subcommand for group $x specifies both LABEL and LABELSOURCE, but only one of these subcommands may be used at a time. Ignoring LABELSOURCE."
+
+"mrsets.sps:44.3-44.13: note: MRSETS: Here is the LABEL setting.
+ 44 | LABEL='foo'.
+ | ^~~~~~~~~~~"
+
+"mrsets.sps:43.3-43.22: note: MRSETS: Here is the LABELSOURCE setting.
+ 43 | LABELSOURCE=VARLABEL CATEGORYLABELS=COUNTEDVALUES
+ | ^~~~~~~~~~~~~~~~~~~~"
+
+"mrsets.sps:42.35-42.37: warning: MRSETS: Variable a specified as part of multiple dichotomy group $x (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
+ 42 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='y'
+ | ^~~"
+
+"mrsets.sps:42.35-42.37: warning: MRSETS: Variable b specified as part of multiple dichotomy group $x (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
+ 42 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='y'
+ | ^~~"
+
+"mrsets.sps:45.35-45.37: warning: MRSETS: Variables y and z specified as part of multiple dichotomy group $x (which has CATEGORYLABELS=COUNTEDVALUES) have the same value label for the group's counted value. These categories will not be distinguishable in output.
+ 45 | MRSETS /MDGROUP NAME=$x VARIABLES=y z VALUE=1
+ | ^~~"
+
+"mrsets.sps:47.35-47.41: warning: MRSETS: Variables specified on MCGROUP should have the same categories, but w and y (and possibly others) in multiple category group $x have different value labels for value 1.
+ 47 | MRSETS /MCGROUP NAME=$x VARIABLES=w x y z.
+ | ^~~~~~~"
+
+"mrsets.sps:49.16-49.17: error: MRSETS: Syntax error expecting `NAME='.
+ 49 | MRSETS /DELETE **.
+ | ^~"
+
+"mrsets.sps:50.16-50.21: error: MRSETS: Syntax error expecting `NAME='.
+ 50 | MRSETS /DELETE NAME**.
+ | ^~~~~~"
+
+"mrsets.sps:51.22-51.23: error: MRSETS: Syntax error expecting identifier.
+ 51 | MRSETS /DELETE NAME=[[**]].
+ | ^~"
+
+"mrsets.sps:53.21-53.22: error: MRSETS: Syntax error expecting `@<:@' or ALL.
+ 53 | MRSETS /DELETE NAME=**.
+ | ^~"
+
+"mrsets.sps:55.22-55.24: note: MRSETS: The active dataset dictionary does not contain any multiple response sets.
+ 55 | MRSETS /DISPLAY NAME=ALL.
+ | ^~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017, 2022 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([NPAR TESTS])
+
+AT_SETUP([NPAR TESTS BINOMIAL P < 0.5; N1/N2 < 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x * w *.
+BEGIN DATA.
+1 6
+2 15
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.3) = x
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
+x,Group 1,1.000,6.000,.286,.300,.551
+,Group 2,2.000,15.000,.714,,
+,Total,,21.000,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P < 0.5; N1/N2 > 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 7
+2 6
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.4) = x
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
+x,Group 1,1,7,.538,.400,.229
+,Group 2,2,6,.462,,
+,Total,,13,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P < 0.5; N1/N2 = 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 8
+2 8
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.4) = x
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
+x,Group 1,1,8,.500,.400,.284
+,Group 2,2,8,.500,,
+,Total,,16,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P > 0.5; N1/N2 < 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 11
+2 12
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.6) = x
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
+x,Group 1,1,11,.478,.600,.164
+,Group 2,2,12,.522,,
+,Total,,23,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P > 0.5; N1/N2 > 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 11
+2 9
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.6) = x.
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
+x,Group 1,1,11,.550,.600,.404
+,Group 2,2,9,.450,,
+,Total,,20,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P > 0.5; N1/N2 = 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 11
+2 11
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.6) = x.
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
+x,Group 1,1,11,.500,.600,.228
+,Group 2,2,11,.500,,
+,Total,,22,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 < 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 8
+2 15
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL = x
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
+x,Group 1,1,8,.348,.500,.210
+,Group 2,2,15,.652,,
+,Total,,23,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 > 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 12
+2 6
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.5) = x.
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
+x,Group 1,1,12,.667,.500,.238
+,Group 2,2,6,.333,,
+,Total,,18,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 = 1])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
+BEGIN DATA.
+1 10
+2 10
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.5) = x
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
+x,Group 1,1,10,.500,.500,1.000
+,Group 2,2,10,.500,,
+,Total,,20,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 = 1 Cutpoint])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x * w *.
+BEGIN DATA.
+9 3
+10 7
+11 16
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.5) = x (10)
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
+x,Group 1,<= 10,10.000,.385,.500,.327
+,Group 2,,16.000,.615,,
+,Total,,26.000,1.000,,
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 = 1 Named values])
+AT_DATA([npar.sps], [dnl
+SET FORMAT F8.3.
+
+DATA LIST LIST NOTABLE /x * w *.
+BEGIN DATA.
+10 10
+15 45
+20 13
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ /BINOMIAL(0.5) = x (10, 20)
+ .
+])
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Binomial Test
+,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
+x,Group 1,10.000,10.000,.435,.500,.678
+,Group 2,20.000,13.000,.565,,
+,Total,,23.000,1.000,,
+])
+AT_CLEANUP
+
+
+
+dnl Test for a bug which caused binomial to crash.
+AT_SETUP([NPAR TESTS BINOMIAL - crash])
+AT_DATA([nparX.sps], [dnl
+data list list /range *.
+begin data.
+0
+1
+end data.
+
+* This is invalid syntax
+NPAR TEST
+ /BINOMIAL(0.5) = Range().
+
+])
+AT_CHECK([pspp -O format=csv nparX.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS CHISQUARE])
+AT_DATA([npar.sps], [dnl
+DATA LIST NOTABLE LIST /x * y * w *.
+BEGIN DATA.
+1 2 1
+2 1 3
+3.1 1 4
+3.2 2 1
+4 2 2
+5 3 1
+1 4 2
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ CHISQUARE=x y
+ .
+
+NPAR TESTS
+ CHISQUARE=y
+ /EXPECTED=3 4 5 4
+ .
+
+NPAR TESTS
+ CHISQUARE=x y(2, 4)
+ /EXPECTED = 6 10 3
+ .
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: x
+Value,Observed N,Expected N,Residual
+1.00,3.00,2.33,.67
+2.00,3.00,2.33,.67
+3.10,4.00,2.33,1.67
+3.20,1.00,2.33,-1.33
+4.00,2.00,2.33,-.33
+5.00,1.00,2.33,-1.33
+Total,14.00,,
+
+Table: y
+Value,Observed N,Expected N,Residual
+1.00,7.00,3.50,3.50
+2.00,4.00,3.50,.50
+3.00,1.00,3.50,-2.50
+4.00,2.00,3.50,-1.50
+Total,14.00,,
+
+Table: Test Statistics
+,Chi-square,df,Asymp. Sig.
+x,3.14,5,.678
+y,6.00,3,.112
+
+Table: y
+Value,Observed N,Expected N,Residual
+1.00,7.00,2.63,4.38
+2.00,4.00,3.50,.50
+3.00,1.00,4.38,-3.38
+4.00,2.00,3.50,-1.50
+Total,14.00,,
+
+Table: Test Statistics
+,Chi-square,df,Asymp. Sig.
+y,10.61,3,.014
+
+Table: Frequencies
+,x,,,,y,,,
+,Category,Observed N,Expected N,Residual,Category,Observed N,Expected N,Residual
+1,2.00,3.00,3.16,-.16,2.00,4.00,2.21,1.79
+2,3.00,5.00,5.26,-.26,3.00,1.00,3.68,-2.68
+3,4.00,2.00,1.58,.42,4.00,2.00,1.11,.89
+Total,,10.00,,,,7.00,,
+
+Table: Test Statistics
+,Chi-square,df,Asymp. Sig.
+x,.13,2,.936
+y,4.13,2,.127
+])
+
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS CHISQUARE expected values missing])
+AT_DATA([npar.sps], [dnl
+DATA LIST NOTABLE LIST /x * y * w *.
+BEGIN DATA.
+1 2 1
+2 1 3
+3.1 1 4
+3.2 2 1
+4 2 2
+5 3 1
+1 4 2
+END DATA.
+
+WEIGHT BY w.
+
+NPAR TESTS
+ CHISQUARE=y
+ /EXPECTED = 3 4 5 4 3 1
+ .
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [1], [dnl
+"error: CHISQUARE test specified 6 expected values, but variable y has 4 distinct values."
+
+Table: Test Statistics
+,Chi-square,df,Asymp. Sig.
+y,.00,0,1.000
+])
+
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS CHISQUARE with DESCRIPTIVES])
+AT_DATA([npar.sps], [dnl
+DATA LIST NOTABLE LIST /x * y * w * .
+BEGIN DATA.
+1 2 1
+2 1 3
+3.1 1 4
+3.2 2 1
+4 2 2
+5 3 1
+1 4 2
+. 5 1
+END DATA.
+
+WEIGHT BY w.
+
+MISSING VALUES x (4).
+
+NPAR TESTS
+ CHISQUARE=x y(-2,5)
+ /MISSING=ANALYSIS
+ /STATISTICS=DESCRIPTIVES
+ .
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Frequencies
+,x,,,,y,,,
+,Category,Observed N,Expected N,Residual,Category,Observed N,Expected N,Residual
+1,-2.00,.00,1.50,-1.50,-2.00,.00,1.88,-1.88
+2,-1.00,.00,1.50,-1.50,-1.00,.00,1.88,-1.88
+3,.00,.00,1.50,-1.50,.00,.00,1.88,-1.88
+4,1.00,3.00,1.50,1.50,1.00,7.00,1.88,5.13
+5,2.00,3.00,1.50,1.50,2.00,4.00,1.88,2.13
+6,3.00,5.00,1.50,3.50,3.00,1.00,1.88,-.88
+7,4.00,.00,1.50,-1.50,4.00,2.00,1.88,.13
+8,5.00,1.00,1.50,-.50,5.00,1.00,1.88,-.88
+Total,,12.00,,,,15.00,,
+
+Table: Test Statistics
+,Chi-square,df,Asymp. Sig.
+x,17.33,7,.015
+y,22.87,7,.002
+
+Table: Descriptive Statistics
+,N,Mean,Std. Deviation,Minimum,Maximum
+x,12.00,2.47,1.19,1.00,5.00
+y,15.00,2.07,1.33,1.00,5.00
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS CHISQUARE, listwise missing])
+AT_DATA([npar.sps], [dnl
+DATA LIST NOTABLE LIST /x * y * w * .
+BEGIN DATA.
+1 2 1
+2 1 3
+3.1 1 4
+3.2 2 1
+4 2 2
+5 3 1
+1 4 2
+. 5 1
+END DATA.
+
+WEIGHT BY w.
+
+* MISSING VALUES x (4).
+
+NPAR TESTS
+ CHISQUARE=x y(-2,5)
+ /MISSING=LISTWISE
+ /STATISTICS=DESCRIPTIVES
+ .
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Frequencies
+,x,,,,y,,,
+,Category,Observed N,Expected N,Residual,Category,Observed N,Expected N,Residual
+1,-2.00,.00,1.75,-1.75,-2.00,.00,1.75,-1.75
+2,-1.00,.00,1.75,-1.75,-1.00,.00,1.75,-1.75
+3,.00,.00,1.75,-1.75,.00,.00,1.75,-1.75
+4,1.00,3.00,1.75,1.25,1.00,7.00,1.75,5.25
+5,2.00,3.00,1.75,1.25,2.00,4.00,1.75,2.25
+6,3.00,5.00,1.75,3.25,3.00,1.00,1.75,-.75
+7,4.00,2.00,1.75,.25,4.00,2.00,1.75,.25
+8,5.00,1.00,1.75,-.75,5.00,.00,1.75,-1.75
+Total,,14.00,,,,14.00,,
+
+Table: Test Statistics
+,Chi-square,df,Asymp. Sig.
+x,13.43,7,.062
+y,26.00,7,.001
+
+Table: Descriptive Statistics
+,N,Mean,Std. Deviation,Minimum,Maximum
+x,14.00,2.69,1.23,1.00,5.00
+y,14.00,1.86,1.10,1.00,4.00
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS WILCOXON])
+AT_DATA([npar.sps], [dnl
+data list notable list /foo * bar * w (f8.0).
+begin data.
+1.00 1.00 1
+1.00 2.00 1
+2.00 1.00 1
+1.00 4.00 1
+2.00 5.00 1
+1.00 19.00 1
+2.00 7.00 1
+4.00 5.00 1
+1.00 12.00 1
+2.00 13.00 1
+2.00 2.00 1
+12.00 .00 2
+12.00 1.00 1
+13.00 1.00 1
+end data
+
+variable labels foo "first" bar "second".
+
+weight by w.
+
+npar test
+ /wilcoxon=foo with bar (paired)
+ /missing analysis
+ /method=exact.
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Ranks
+,,N,Mean Rank,Sum of Ranks
+first - second,Negative Ranks,8,6.00,48.00
+,Positive Ranks,5,8.60,43.00
+,Ties,2,,
+,Total,15,,
+
+Table: Test Statistics
+,first - second
+Z,-.18
+Asymp. Sig. (2-tailed),.861
+Exact Sig. (2-tailed),.893
+Exact Sig. (1-tailed),.446
+])
+
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS WILCOXON with missing values])
+AT_DATA([npar.sps], [dnl
+data list notable list /foo * bar * dummy *.
+begin data.
+1.00 1.00 1
+1.00 2.00 1
+2.00 1.00 1
+1.00 4.00 .
+2.00 5.00 .
+1.00 19.00 .
+2.00 7.00 1
+4.00 5.00 1
+1.00 12.00 1
+2.00 13.00 1
+2.00 2.00 1
+12.00 .00 1
+12.00 .00 1
+34.2 . 1
+12.00 1.00 1
+13.00 1.00 1
+end data
+
+variable labels foo "first" bar "second".
+
+npar test
+ /wilcoxon=foo with bar (paired)
+ /missing analysis
+ /method=exact.
+])
+
+dnl This is the same output as the previous test.
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Ranks
+,,N,Mean Rank,Sum of Ranks
+first - second,Negative Ranks,8,6.00,48.00
+,Positive Ranks,5,8.60,43.00
+,Ties,2,,
+,Total,15,,
+
+Table: Test Statistics
+,first - second
+Z,-.18
+Asymp. Sig. (2-tailed),.861
+Exact Sig. (2-tailed),.893
+Exact Sig. (1-tailed),.446
+])
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS SIGN])
+AT_DATA([npar.sps], [dnl
+set format = F9.3.
+
+data list notable list /age * height rank *.
+begin data.
+10 12 11
+12 13 13
+13 14 12
+12 12 10
+9 9 10
+10.3 10.2 12
+end data.
+
+npar tests
+ /sign=age height WITH height rank (PAIRED)
+ /MISSING ANALYSIS
+ /METHOD=EXACT
+ .
+])
+AT_CHECK([pspp -o pspp.csv npar.sps])
+dnl Some machines return .313 instead of .312
+dnl (see bug #31611).
+AT_CHECK([sed -e 's/\.313$/.312/' -e 's/^Exact Sig\. (1-tailed),\.313/Exact Sig. (1-tailed),.312/' pspp.csv], [0], [dnl
+Table: Frequencies
+,,N
+age - height,Negative Differences,3
+,Positive Differences,1
+,Ties,2
+,Total,6
+height - rank,Negative Differences,2
+,Positive Differences,3
+,Ties,1
+,Total,6
+
+Table: Test Statistics
+,age - height,height - rank
+Exact Sig. (2-tailed),.625,1.000
+Exact Sig. (1-tailed),.312,.500
+Point Probability,.250,.312
+])
+AT_CLEANUP
+
+
+AT_SETUP([NPAR Kruskal-Wallis test])
+
+dnl Simple case
+AT_DATA([kw-simple.sps], [dnl
+set format = F9.3.
+
+data list notable list /gv * xscore *.
+begin data
+1 96
+1 128
+1 83
+2 132
+2 135
+2 109
+3 115
+1 61
+1 101
+2 82
+2 124
+3 149
+3 166
+3 147
+end data.
+
+value label /gv
+ 1 "timed out"
+ 2 "hit wicket"
+ 3 "handled the ball".
+
+npar tests
+ /kruskal-wallis xscore by gv (1, 3)
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt kw-simple.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Ranks
+,,N,Mean Rank
+xscore,timed out,5,4.400
+,hit wicket,5,7.400
+,handled the ball,4,11.500
+,Total,14,
+
+Table: Test Statistics
+,xscore
+Chi-Square,6.406
+df,2
+Asymp. Sig.,.041
+])
+
+
+dnl Now try a missing value in the group variable
+AT_DATA([kw-missing-group.sps], [dnl
+set format = F9.3.
+
+data list notable list /gv * xscore *.
+begin data
+1 96
+1 128
+1 83
+1 61
+1 101
+2 82
+2 124
+2 132
+2 135
+2 109
+3 115
+3 149
+3 166
+3 147
+2.5 344
+end data.
+
+missing values gv (2.5).
+
+value label /gv
+ 1 "timed out"
+ 2 "hit wicket"
+ 3 "handled the ball".
+
+npar tests
+ /kruskal-wallis xscore by gv (1, 3)
+ /missing=exclude
+ .
+])
+
+AT_CHECK([pspp -o pspp2.csv kw-missing-group.sps])
+
+dnl The result should be the same as before
+AT_CHECK([diff pspp.csv pspp2.csv], [0])
+
+dnl Reverse the order of the group values
+AT_DATA([kw-reverse-group.sps], [dnl
+set format = F9.3.
+
+data list notable list /gv * xscore *.
+begin data
+1 96
+1 128
+1 83
+1 61
+1 101
+2 82
+2 124
+2 132
+2 135
+2 109
+3 115
+3 149
+3 166
+3 147
+end data.
+
+value label /gv
+ 1 "timed out"
+ 2 "hit wicket"
+ 3 "handled the ball".
+
+npar tests
+ /kruskal-wallis xscore by gv (3, 1)
+ /missing=exclude
+ .
+])
+
+AT_CHECK([pspp -o pspp2.csv kw-reverse-group.sps])
+
+dnl The result should be the same as before
+AT_CHECK([diff pspp.csv pspp2.csv], [0])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR Kruskal-Wallis multiple-variables])
+
+AT_DATA([kw-multi.sps], [dnl
+set format = F9.3.
+
+data list notable list /gv * xscore * yscore.
+begin data
+1 96 .
+1 128 .
+1 83 .
+2 132 132
+2 135 135
+2 109 109
+3 115 115
+1 61 .
+1 101 .
+2 82 82
+2 124 124
+3 149 149
+3 166 166
+3 147 147
+4 . 96
+4 . 128
+4 . 83
+4 . 61
+4 . 101
+end data.
+
+value label /gv
+ 1 "timed out"
+ 2 "hit wicket"
+ 3 "handled the ball"
+ 4 "bowled"
+ 5 "lbw"
+ .
+
+npar tests
+ /k-w xscore yscore by gv (1, 5)
+ .
+
+])
+
+
+AT_CHECK([pspp -o pspp.csv kw-multi.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Ranks
+,,N,Mean Rank
+xscore,timed out,5,4.400
+,hit wicket,5,7.400
+,handled the ball,4,11.500
+,Total,14,
+yscore,hit wicket,5,7.400
+,handled the ball,4,11.500
+,bowled,5,4.400
+,Total,14,
+
+Table: Test Statistics
+,xscore,yscore
+Chi-Square,6.406,6.406
+df,2,2
+Asymp. Sig.,.041,.041
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS Runs])
+AT_DATA([npar-runs.sps], [dnl
+set format F11.4.
+data list notable list /score * w *.
+begin data
+4 6
+. 4
+4 3
+3 20
+2 29
+1 42
+6 18
+5 7
+6 78
+5 10
+6 46
+5 5
+6 17
+5 1
+6 11
+4 2
+3 7
+2 6
+1 10
+4 13
+3 22
+3 11
+2 24
+1 18
+4 4
+3 12
+2 10
+1 25
+4 4
+3 7
+2 3
+1 4
+4 2
+3 3
+2 2
+1 4
+end data.
+
+weight by w.
+
+npar tests
+ /runs (MEDIAN) = score
+ /runs (MEAN) = score
+ /runs (MODE) = score
+ .
+])
+
+AT_CHECK([pspp -O format=csv npar-runs.sps], [0],
+[Table: Runs Test
+,score
+Test Value (median),3.0000
+Cases < Test Value,177.0000
+Cases ≥ Test Value,309.0000
+Total Cases,486.0000
+Number of Runs,12
+Z,-20.9931
+Asymp. Sig. (2-tailed),.000
+
+Table: Runs Test
+,score
+Test Value (mean),3.6379
+Cases < Test Value,259.0000
+Cases ≥ Test Value,227.0000
+Total Cases,486.0000
+Number of Runs,12
+Z,-21.0650
+Asymp. Sig. (2-tailed),.000
+
+Table: Runs Test
+,score
+Test Value (mode),6.0000
+Cases < Test Value,316.0000
+Cases ≥ Test Value,170.0000
+Total Cases,486.0000
+Number of Runs,11
+Z,-21.0742
+Asymp. Sig. (2-tailed),.000
+])
+
+AT_CLEANUP
+
+
+dnl Thanks to Douglas Bonett for providing this test case.
+AT_SETUP([NPAR TESTS Runs (2)])
+AT_DATA([npar-runs.sps], [dnl
+data list notable free /y.
+begin data
+1 1 2 1 2 1 1 2 1 1 1 2 1 2
+end data.
+NPAR TEST /RUNS(1.5) = y.
+])
+
+AT_CHECK([pspp -O format=csv npar-runs.sps], [0], [dnl
+Table: Runs Test
+,y
+Test Value,1.50
+Cases < Test Value,9
+Cases ≥ Test Value,5
+Total Cases,14
+Number of Runs,10
+Z,1.26
+Asymp. Sig. (2-tailed),.206
+])
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Friedman])
+AT_DATA([npar-friedman.sps], [dnl
+set format F15.4.
+data list notable list /x * y * z.
+begin data
+9.5 6.5 8.1
+8.0 6.0 6.0
+7.0 6.5 4.2
+9.5 5.0 7.3
+9.0 7.0 6.2
+8.5 6.9 6.5
+7.5 8.0 6.5
+6.0 8.0 3.1
+5.0 6.0 4.9
+7.5 7.5 6.2
+end data.
+
+npar tests
+ /friedman = x y z.
+])
+
+AT_CHECK([pspp -O format=csv npar-friedman.sps], [0], [dnl
+Table: Ranks
+,Mean Rank
+x,2.6500
+y,2.1000
+z,1.2500
+
+Table: Test Statistics
+N,10
+Chi-Square,10.4737
+df,2
+Asymp. Sig.,.005
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS Mann-Whitney])
+AT_DATA([npar-mann-whitney.sps], [dnl
+SET FORMAT = F11.4
+
+data list notable list /height * sex (f1.0).
+begin data.
+201 1
+84 1
+83 1
+94 1
+88 0
+99 0
+55 0
+69 0
+86 1
+79 1
+91 0
+201 0
+88 1
+85 1
+82 1
+88 0
+75 0
+99 0
+81 0
+72 1
+89 1
+92 1
+80 0
+82 0
+76 0
+65 0
+85 0
+76 1
+145 1
+24 1
+1 4
+-4 5
+34 5
+21 4
+end data.
+
+NPAR TESTS
+ /M-W = height BY sex (0,1).
+])
+
+AT_CHECK([pspp -O format=csv npar-mann-whitney.sps], [0], [dnl
+Table: Ranks
+,,N,Mean Rank,Sum of Ranks
+height,0,15,14.5333,218.0000
+,1,15,16.4667,247.0000
+,Total,30,,
+
+Table: Test Statistics
+,Mann-Whitney U,Wilcoxon W,Z,Asymp. Sig. (2-tailed)
+height,98.0000,218.0000,-.6020,.547
+])
+
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Mann-Whitney Multiple])
+dnl Check for a bug where the ranks were inappropriately allocated, when
+dnl multiple variables were tested and MISSING=ANALYSIS chosen.
+
+cp "$abs_srcdir/language/mann-whitney.txt" .
+
+AT_DATA([npar-mann-whitney.sps], [dnl
+SET FORMAT = F11.3
+
+DATA LIST NOTABLE FILE='mann-whitney.txt'
+ LIST /I002_01 I002_02 I002_03 I002_04 sum_HL *.
+
+VARIABLE LABELS
+ I002_01 'IOS: Familie'
+ I002_02 'IOS: Freunde'
+ I002_03 'IOS: Partner*in'
+ I002_04 'IOS: Bekannte'.
+
+MISSING VALUES I002_01 I002_02 I002_03 I002_04 (-9 -1).
+
+NPAR TESTS
+ /MISSING=ANALYSIS
+ /M-W=I002_01 I002_02 I002_03 I002_04 BY sum_HL (0 1).
+])
+
+AT_CHECK([pspp -O format=csv npar-mann-whitney.sps], [0], [dnl
+Table: Ranks
+,,N,Mean Rank,Sum of Ranks
+IOS: Familie,.000,114,110.018,12542.000
+,1.000,115,119.939,13793.000
+,Total,229,,
+IOS: Freunde,.000,115,108.339,12459.000
+,1.000,115,122.661,14106.000
+,Total,230,,
+IOS: Partner*in,.000,97,95.351,9249.000
+,1.000,91,93.593,8517.000
+,Total,188,,
+IOS: Bekannte,.000,115,111.065,12772.500
+,1.000,115,119.935,13792.500
+,Total,230,,
+
+Table: Test Statistics
+,Mann-Whitney U,Wilcoxon W,Z,Asymp. Sig. (2-tailed)
+IOS: Familie,5987.000,12542.000,-1.167,.243
+IOS: Freunde,5789.000,12459.000,-1.674,.094
+IOS: Partner*in,4331.000,8517.000,-.245,.807
+IOS: Bekannte,6102.500,12772.500,-1.046,.296
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS Cochran])
+AT_DATA([npar-cochran.sps], [dnl
+set format f11.3.
+
+data list notable list /v1 * v2 * v3 * v4 * v5 * v6 * v7 *.
+begin data.
+2 1 1 2 1 1 2
+2 2 2 2 1 1 1
+1 1 2 2 1 1 2
+2 2 2 2 1 1 2
+2 1 2 1 1 2 1
+1 2 2 1 1 1 1
+1 2 2 2 2 2 2
+2 2 1 2 1 1 1
+1 2 1 2 1 1 2
+end data.
+
+npar tests
+ /cochran = v1 to v7 .
+
+])
+
+AT_CHECK([pspp -o pspp.csv npar-cochran.sps])
+
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Frequencies
+,Value,
+,Success (2),Failure (1)
+v1,5,4
+v2,6,3
+v3,6,3
+v4,7,2
+v5,1,8
+v6,2,7
+v7,5,4
+
+Table: Test Statistics
+,Value
+N,9
+Cochran's Q,12.735
+df,6
+Asymp. Sig.,.047
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS Kendall])
+AT_DATA([npar-kendall.sps], [dnl
+SET FORMAT F14.3.
+
+data list notable list /v1 * v2 * v3
+begin data.
+ 7 7 2
+ 5 6 5
+ 8 6 4
+ 5 7 4
+ 5 4 4
+ 8 6 5
+ 6 3 5
+ 7 6 5
+ 8 5 5
+ . 2 2
+ 5 4 5
+ 3 4 4
+ 5 1 2
+ 5 2 1
+ 7 6 5
+ 6 3 4
+ 6 6 6
+ 5 4 5
+ 4 3 4
+ 9 1 1
+ 6 2 1
+ 3 7 8
+ 6 3 4
+ 4 4 4
+ 5 4 3
+ 6 5 2
+ 4 4 8
+ 4 6 4
+ 6 5 5
+ 7 8 6
+ 5 3 5
+end data.
+
+npar tests
+ /kendall = all
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv npar-kendall.sps])
+
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Ranks
+,Mean Rank
+v1,2.500
+v2,1.817
+v3,1.683
+
+Table: Test Statistics
+N,30
+Kendall's W,.233
+Chi-Square,13.960
+df,2
+Asymp. Sig.,.001
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS McNemar])
+
+AT_DATA([mcnemar.sps], [dnl
+set format = F12.3.
+data list notable list /v1 * v2 * junk *.
+begin data.
+0 0 0
+0 0 0
+0 0 0
+0 0 0
+0 1 0
+0 1 0
+0 1 0
+0 1 0
+0 1 1
+0 1 1
+0 1 1
+0 1 1
+0 1 1
+1 0 1
+1 0 1
+1 1 1
+1 1 1
+1 1 0
+1 1 0
+1 1 1
+end data.
+
+npar tests
+ /mcnemar = v1 WITH v2 junk.
+])
+
+AT_CHECK([pspp -O format=csv mcnemar.sps], [0], [dnl
+Table: v1 & v2
+v1,v2,
+,.000,1.000
+.000,4,9
+1.000,2,5
+
+Table: v1 & junk
+v1,junk,
+,.000,1.000
+.000,8,5
+1.000,2,5
+
+Table: Test Statistics
+,N,Exact Sig. (2-tailed),Exact Sig. (1-tailed),Point Probability
+v1 & v2,20,.065,.033,.027
+v1 & junk,20,.453,.227,.164
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS McNemar Symetricity])
+
+AT_DATA([mcnemar.sps], [dnl
+data list notable list /var1 var2 w (F2.0).
+begin data
+0 0 9
+0 1 8
+1 0 1
+1 1 5
+end data.
+
+weight by w.
+
+NPAR TEST
+ /MCNEMAR var1 WITH var2 (PAIRED).
+
+NPAR TEST
+ /MCNEMAR var2 WITH var1 (PAIRED).
+])
+
+AT_CHECK([pspp -O format=csv mcnemar.sps], [0], [dnl
+Table: var1 & var2
+var1,var2,
+,0,1
+0,9,8
+1,1,5
+
+Table: Test Statistics
+,N,Exact Sig. (2-tailed),Exact Sig. (1-tailed),Point Probability
+var1 & var2,23,.039,.020,.02
+
+Table: var2 & var1
+var2,var1,
+,0,1
+0,9,1
+1,8,5
+
+Table: Test Statistics
+,N,Exact Sig. (2-tailed),Exact Sig. (1-tailed),Point Probability
+var2 & var1,23,.039,.020,.02
+])
+
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS Kolmogorov-Smirnov Uniform parameters given])
+
+AT_DATA([ks-uniform.sps], [dnl
+set format F12.3.
+data list notable list /x *.
+begin data
+.554
+.382
+.329
+.480
+.711
+.503
+.203
+.477
+.621
+.581
+end data.
+
+npar tests k-s (uniform 0 1) = x.
+])
+
+AT_CHECK([pspp -O format=csv ks-uniform.sps], [0], [dnl
+Table: One-Sample Kolmogorov-Smirnov Test
+,,x
+N,,10
+Uniform Parameters,Minimum,.000
+,Maximum,1.000
+Most Extreme Differences,Absolute,.289
+,Positive,.289
+,Negative,-.229
+Kolmogorov-Smirnov Z,,.914
+Asymp. Sig. (2-tailed),,.374
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Kolmogorov-Smirnov Normal parameters imputed])
+
+AT_DATA([ks-normal.sps], [dnl
+set format = F12.3.
+
+data list notable list /foo * bar *.
+begin data.
+65 12.5
+59 14.2
+43 12.6
+57
+68
+79
+51
+62
+57
+73
+58
+58
+68
+75
+47
+70
+59
+71
+52
+48 13.0
+58 14.1
+37 15.0
+39 13.1
+58 13.2
+43 14.5
+58 13.5
+86 14.0
+63 12.5
+80 12.8
+70
+63
+53
+53
+48
+49
+51
+47
+81
+66
+78
+65
+69
+70 12.1
+63 12.5
+64 12.4
+39 13.8
+51 13.2
+68 14.0
+76 12.6
+53 12.1
+71 13.5
+47 13.8
+87 14.1
+72 12.9
+48 12.1
+75 12.8
+51 13.4
+63 13.9
+61 12.5
+61 12.4
+66 12.8
+82 12.9
+81 13.6
+46
+52
+71
+73
+58
+57
+46
+58
+52 13.5
+71 13.2
+57 12.8
+78 14.1
+73 12.1
+50 12.6
+71
+51
+51
+68
+84
+64
+66
+65
+52
+56
+70
+68
+66
+78
+65
+71
+53
+81
+53
+57
+64
+61
+43
+56
+37
+74
+66
+81
+67
+80
+68
+76
+70
+80
+42
+74
+80
+70
+60
+39
+72
+69
+63
+72
+63
+49
+53 13.2
+43 13.8
+51 12.5
+63 12.6
+64 12.9
+65 13.0
+64 12.5
+66 12.0
+55
+62
+58
+48
+67
+46
+36
+61
+55
+77
+74
+60
+70
+69
+57
+49
+63
+69
+63
+76
+53
+54
+42
+64
+66
+61
+62
+73
+73
+60
+79
+40
+48
+76
+60
+76
+54
+69
+65
+69
+51
+54
+82
+end data.
+
+npar tests
+ /k-s (normal) = foo bar.
+])
+
+AT_CHECK([pspp -O format=csv ks-normal.sps], [0], [dnl
+Table: One-Sample Kolmogorov-Smirnov Test
+,,foo,bar
+N,,174,48
+Normal Parameters,Mean,62.109,13.108
+,Std. Deviation,11.548,.718
+Most Extreme Differences,Absolute,.059,.115
+,Positive,.055,.115
+,Negative,-.059,-.082
+Kolmogorov-Smirnov Z,,.785,.795
+Asymp. Sig. (2-tailed),,.569,.552
+])
+
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Median Test (median imputed)])
+
+AT_DATA([median1.sps], [dnl
+set format F12.3.
+data list notable list /ignore * animal * years * w *.
+begin data
+99 1 10 1
+99 4 1 1
+99 5 11 1
+99 5 10 1
+99 3 7 1
+99 6 10 1
+99 0 7 1
+99 3 14 1
+99 2 3 1
+99 1 1 1
+99 4 7 1
+99 5 12 1
+99 3 6 1
+99 4 1 1
+99 3 5 1
+99 5 7 1
+99 4 6 1
+99 3 14 1
+99 4 8 1
+99 5 13 1
+99 2 0 1
+99 4 7 1
+99 4 7 1
+99 1 0 1
+99 2 8 1
+99 4 10 1
+99 2 3 1
+99 2 0 1
+99 4 8 1
+99 1 8 1
+end data.
+
+
+variable label years 'Years expected'.
+variable label animal 'Animal Genus'.
+
+add value labels animal 1 'Animal 1' 2 'Animal 2' 3 'Animal 3' 4 'Animal 4' 5 'Animal 5'.
+
+npar tests
+ /median = years by animal (1, 5)
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv median1.sps], [0], [dnl
+Table: Frequencies
+,,Animal Genus,,,,
+,,Animal 1,Animal 2,Animal 3,Animal 4,Animal 5
+Years expected,> Median,2,1,2,3,4
+,≤ Median,2,4,3,6,1
+
+Table: Test Statistics
+,N,Median,Chi-Square,df,Asymp. Sig.
+Years expected,28,7.000,4.317,4,.365
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Median Test (median given)])
+
+AT_DATA([median2.sps], [dnl
+set format F12.3.
+data list notable list /ignore * animal * years * w *.
+begin data
+99 1 10 1
+99 4 1 1
+99 5 11 1
+99 5 10 1
+99 3 7 1
+99 3 14 1
+99 2 3 1
+99 1 1 1
+99 4 7 1
+99 5 12 1
+99 3 6 1
+99 4 1 1
+99 3 5 1
+99 5 7 1
+99 4 6 1
+99 3 14 1
+99 4 8 1
+99 5 13 1
+99 2 0 1
+99 4 7 1
+99 4 7 1
+99 1 0 1
+99 2 8 1
+99 4 10 1
+99 2 3 1
+99 2 0 1
+99 4 8 1
+99 1 8 1
+end data.
+
+
+variable label years 'Years expected'.
+variable label animal 'Animal Genus'.
+
+add value labels animal 1 'Animal 1' 2 'Animal 2' 3 'Animal 3' 4 'Animal 4' 5 'Animal 5'.
+
+npar tests
+ /median (7) = years by animal (1, 5)
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv median2.sps], [0], [dnl
+Table: Frequencies
+,,Animal Genus,,,,
+,,Animal 1,Animal 2,Animal 3,Animal 4,Animal 5
+Years expected,> Median,2,1,2,3,4
+,≤ Median,2,4,3,6,1
+
+Table: Test Statistics
+,N,Median,Chi-Square,df,Asymp. Sig.
+Years expected,28,7.000,4.317,4,.365
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Median Test (two sample)])
+
+AT_DATA([median3.sps], [dnl
+set format F12.3.
+data list notable list /xx * animal * years * w *.
+begin data
+99 1 10 1
+99 4 1 1
+99 5 11 1
+99 5 10 1
+99 3 7 1
+99 3 14 1
+99 2 3 1
+99 1 1 1
+99 4 7 1
+99 5 12 1
+99 3 6 1
+99 4 1 1
+99 3 5 1
+99 5 7 1
+99 4 6 1
+99 3 14 1
+99 4 8 1
+99 5 13 1
+99 2 0 1
+99 4 7 1
+99 4 7 1
+99 1 0 1
+99 2 8 1
+99 4 10 1
+99 2 3 1
+99 2 0 1
+99 4 8 1
+99 1 8 1
+end data.
+
+
+variable label years 'Years expected'.
+variable label animal 'Animal Genus'.
+
+add value labels animal 1 'Animal 1' 2 'Animal 2' 3 'Animal 3' 4 'Animal 4' 5 'Animal 5'.
+
+npar tests
+ /median (7) = xx years by animal (5, 1)
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv median3.sps], [0], [dnl
+Table: Frequencies
+,,Animal Genus,
+,,Animal 1,Animal 5
+xx,> Median,4,5
+,≤ Median,0,0
+Years expected,> Median,2,4
+,≤ Median,2,1
+
+Table: Test Statistics
+,N,Median,Chi-Square,df,Asymp. Sig.
+xx,9,7.000,NaN,1,NaN
+Years expected,9,7.000,.900,1,.343
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS Jonckheere-Terpstra])
+
+AT_DATA([jt.sps], [dnl
+set format = F12.3.
+data list notable list /x * g * w *.
+begin data.
+52 2 2
+58 2 1
+60 2 1
+62 2 1
+58 0 1
+44 2 1
+46 2 1
+14 3 1
+32 2 1
+16 3 1
+56 2 1
+26 3 1
+40 3 2
+50 4 1
+6 5 1
+34 2 3
+36 2 2
+40 2 2
+50 2 1
+end data.
+
+weight by w.
+
+npar test /jonckheere-terpstra = x by g (5, 2).
+])
+
+
+AT_CHECK([pspp -O format=csv jt.sps], [0], [dnl
+Table: Jonckheere-Terpstra Test
+,Number of levels in g,N,Observed J-T Statistic,Mean J-T Statistic,Std. Deviation of J-T Statistic,Std. J-T Statistic,Asymp. Sig. (2-tailed)
+x,4,24.000,29.500,65.000,15.902,-2.232,.026
+])
+
+AT_CLEANUP
+
+dnl Checks that (PAIRED) can have lists where the same
+dnl variable appears more than once.
+AT_SETUP([NPAR TESTS (PAIRED)])
+AT_DATA([npar.sps], [dnl
+set format = F12.3.
+data list notable list /a * b * c *.
+begin data.
+1 2 4
+4 5 3
+1 2 2
+4 5 1
+end data.
+
+npar tests /wilcoxon a b with c c (paired).
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
+Table: Ranks
+,,N,Mean Rank,Sum of Ranks
+a - c,Negative Ranks,2,2.500,5.000
+,Positive Ranks,2,2.500,5.000
+,Ties,0,,
+,Total,4,,
+b - c,Negative Ranks,1,1.500,1.500
+,Positive Ranks,2,2.250,4.500
+,Ties,1,,
+,Total,4,,
+
+Table: Test Statistics
+,a - c,b - c
+Z,.000,-.816
+Asymp. Sig. (2-tailed),1.000,.414
+])
+
+
+AT_CLEANUP
+
+
+
+AT_SETUP([NPAR TESTS CHISQUARE crash])
+dnl This syntax had been observed to crash pspp
+
+AT_DATA([npar.sps], [dnl
+data list list /x *.
+begin data.
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+end data.
+
+* This happens to be invalid syntax. But should not crash.
+NPAR TEST
+ /CHISQUARE= x(0.098, 99.098)
+ /EXPECTED = 1.2.
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([NPAR TESTS - crash on invalid syntax])
+
+AT_DATA([npar.sps], [dnl
+data list notable list /ev * xscore *.
+begin data.
+2 109
+3 115
+1 61
+1 101
+3 147
+end data.
+
+
+npar tests
+ /kruskal-wallis xscore by(gv (1, 3).
+])
+
+AT_CHECK([pspp -O format=csv npar.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+
+AT_SETUP([NPAR TESTS - crash on unterminated string])
+
+AT_DATA([npar.sps], [dnl
+DATA LIST NOTABLE LIST /x * y * w * .
+BEGIN DATA.
+3.1 1 4
+3.2 2 1
+4 2 6
+END DATA.
+
+
+NPAR TESTS
+" CHISQUARE=x y(-2,5)
+ /STATISTICS=DESCRIPTIVES
+ .
+]) dnl "
+
+AT_CHECK([pspp -O format=csv npar.sps], [1], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([NPAR TESTS - syntax errors])
+AT_DATA([npar.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+NPAR TESTS COCHRAN **.
+NPAR TESTS FRIEDMAN **.
+NPAR TESTS KENDALL **.
+NPAR TESTS RUNS **.
+NPAR TESTS RUNS (**).
+NPAR TESTS RUNS (MEAN **).
+NPAR TESTS RUNS (MEAN)=**.
+NPAR TESTS CHISQUARE **.
+NPAR TESTS CHISQUARE x **.
+NPAR TESTS CHISQUARE x (**).
+NPAR TESTS CHISQUARE x (1 **).
+NPAR TESTS CHISQUARE x (1, -1).
+NPAR TESTS CHISQUARE x (1, 2 **).
+NPAR TESTS CHISQUARE x /EXPECTED **.
+NPAR TESTS CHISQUARE x /EXPECTED=1* **.
+NPAR TESTS CHISQUARE x (1,5)/EXPECTED=2.
+NPAR TESTS BINOMIAL (**).
+NPAR TESTS BINOMIAL (1 **).
+NPAR TESTS BINOMIAL (1)**.
+NPAR TESTS BINOMIAL x(**).
+NPAR TESTS BINOMIAL x(1,**).
+NPAR TESTS BINOMIAL x(1,2**).
+NPAR TESTS BINOMIAL x(1**).
+NPAR TESTS K-S **.
+NPAR TESTS K-S (**).
+NPAR TESTS K-S (NORMAL **).
+NPAR TESTS K-S (NORMAL)=**.
+NPAR TESTS J-T **.
+NPAR TESTS J-T x **.
+NPAR TESTS J-T x BY **.
+NPAR TESTS J-T x BY y **.
+NPAR TESTS J-T x BY y (**).
+NPAR TESTS J-T x BY y (1, **).
+NPAR TESTS J-T x BY y (1, 2 **).
+NPAR TESTS MCNEMAR **.
+NPAR TESTS MCNEMAR x **.
+NPAR TESTS MCNEMAR x WITH **.
+NPAR TESTS MCNEMAR x WITH y (**).
+NPAR TESTS MCNEMAR x WITH y (PAIRED **).
+NPAR TESTS MCNEMAR x WITH y z (PAIRED).
+NPAR TESTS MEDIAN (**).
+NPAR TESTS MEDIAN (1 **).
+NPAR TESTS MISSING/MISSING.
+NPAR TESTS MISSING **.
+NPAR TESTS METHOD/METHOD.
+NPAR TESTS METHOD EXACT TIMER(**).
+NPAR TESTS METHOD EXACT TIMER(5 **).
+NPAR TESTS STATISTICS **.
+NPAR TESTS ALGORITHM **.
+NPAR TESTS **.
+])
+AT_CHECK([pspp -O format=csv npar.sps], [1], [dnl
+"npar.sps:2.20-2.21: error: NPAR TESTS: Syntax error expecting variable name.
+ 2 | NPAR TESTS COCHRAN **.
+ | ^~"
+
+"npar.sps:3.21-3.22: error: NPAR TESTS: Syntax error expecting variable name.
+ 3 | NPAR TESTS FRIEDMAN **.
+ | ^~"
+
+"npar.sps:4.20-4.21: error: NPAR TESTS: Syntax error expecting variable name.
+ 4 | NPAR TESTS KENDALL **.
+ | ^~"
+
+"npar.sps:5.17-5.18: error: NPAR TESTS: Syntax error expecting `@{:@'.
+ 5 | NPAR TESTS RUNS **.
+ | ^~"
+
+"npar.sps:6.18-6.19: error: NPAR TESTS: Syntax error expecting MEAN, MEDIAN, MODE or a number.
+ 6 | NPAR TESTS RUNS (**).
+ | ^~"
+
+"npar.sps:7.23-7.24: error: NPAR TESTS: Syntax error expecting `@:}@='.
+ 7 | NPAR TESTS RUNS (MEAN **).
+ | ^~"
+
+"npar.sps:8.24-8.25: error: NPAR TESTS: Syntax error expecting variable name.
+ 8 | NPAR TESTS RUNS (MEAN)=**.
+ | ^~"
+
+"npar.sps:9.22-9.23: error: NPAR TESTS: Syntax error expecting variable name.
+ 9 | NPAR TESTS CHISQUARE **.
+ | ^~"
+
+"npar.sps:10.24-10.25: error: NPAR TESTS: Syntax error expecting `BEGIN DATA'.
+ 10 | NPAR TESTS CHISQUARE x **.
+ | ^~"
+
+"npar.sps:10.24-10.25: error: NPAR TESTS: Syntax error expecting end of command.
+ 10 | NPAR TESTS CHISQUARE x **.
+ | ^~"
+
+"npar.sps:11.25-11.26: error: NPAR TESTS: Syntax error expecting number.
+ 11 | NPAR TESTS CHISQUARE x (**).
+ | ^~"
+
+"npar.sps:12.27-12.28: error: NPAR TESTS: Syntax error expecting `,'.
+ 12 | NPAR TESTS CHISQUARE x (1 **).
+ | ^~"
+
+"npar.sps:13.28-13.29: error: NPAR TESTS: Syntax error expecting number greater than 1 for HI.
+ 13 | NPAR TESTS CHISQUARE x (1, -1).
+ | ^~"
+
+"npar.sps:14.30-14.31: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 14 | NPAR TESTS CHISQUARE x (1, 2 **).
+ | ^~"
+
+"npar.sps:15.34-15.35: error: NPAR TESTS: Syntax error expecting `='.
+ 15 | NPAR TESTS CHISQUARE x /EXPECTED **.
+ | ^~"
+
+"npar.sps:16.37-16.38: error: NPAR TESTS: Syntax error expecting number.
+ 16 | NPAR TESTS CHISQUARE x /EXPECTED=1* **.
+ | ^~"
+
+"npar.sps:17.39: error: NPAR TESTS: 1 expected values were given, but the specified range (1-5) requires exactly 5 values.
+ 17 | NPAR TESTS CHISQUARE x (1,5)/EXPECTED=2.
+ | ^"
+
+"npar.sps:18.22-18.23: error: NPAR TESTS: Syntax error expecting number.
+ 18 | NPAR TESTS BINOMIAL (**).
+ | ^~"
+
+"npar.sps:19.24-19.25: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 19 | NPAR TESTS BINOMIAL (1 **).
+ | ^~"
+
+"npar.sps:20.24-20.25: error: NPAR TESTS: Syntax error expecting `='.
+ 20 | NPAR TESTS BINOMIAL (1)**.
+ | ^~"
+
+"npar.sps:21.23-21.24: error: NPAR TESTS: Syntax error expecting number.
+ 21 | NPAR TESTS BINOMIAL x(**).
+ | ^~"
+
+"npar.sps:22.25-22.26: error: NPAR TESTS: Syntax error expecting number.
+ 22 | NPAR TESTS BINOMIAL x(1,**).
+ | ^~"
+
+"npar.sps:23.26-23.27: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 23 | NPAR TESTS BINOMIAL x(1,2**).
+ | ^~"
+
+"npar.sps:24.24-24.25: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 24 | NPAR TESTS BINOMIAL x(1**).
+ | ^~"
+
+"npar.sps:25.16-25.17: error: NPAR TESTS: Syntax error expecting `@{:@'.
+ 25 | NPAR TESTS K-S **.
+ | ^~"
+
+"npar.sps:26.17-26.18: error: NPAR TESTS: Syntax error expecting NORMAL, POISSON, UNIFORM, or EXPONENTIAL.
+ 26 | NPAR TESTS K-S (**).
+ | ^~"
+
+"npar.sps:27.24-27.25: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 27 | NPAR TESTS K-S (NORMAL **).
+ | ^~"
+
+"npar.sps:28.25-28.26: error: NPAR TESTS: Syntax error expecting variable name.
+ 28 | NPAR TESTS K-S (NORMAL)=**.
+ | ^~"
+
+"npar.sps:29.16-29.17: error: NPAR TESTS: Syntax error expecting variable name.
+ 29 | NPAR TESTS J-T **.
+ | ^~"
+
+"npar.sps:30.18-30.19: error: NPAR TESTS: Syntax error expecting `BY'.
+ 30 | NPAR TESTS J-T x **.
+ | ^~"
+
+"npar.sps:31.21-31.22: error: NPAR TESTS: Syntax error expecting variable name.
+ 31 | NPAR TESTS J-T x BY **.
+ | ^~"
+
+"npar.sps:32.23-32.24: error: NPAR TESTS: Syntax error expecting `@{:@'.
+ 32 | NPAR TESTS J-T x BY y **.
+ | ^~"
+
+"npar.sps:33.24-33.25: error: NPAR TESTS: Syntax error expecting number.
+ 33 | NPAR TESTS J-T x BY y (**).
+ | ^~"
+
+"npar.sps:34.27-34.28: error: NPAR TESTS: Syntax error expecting number.
+ 34 | NPAR TESTS J-T x BY y (1, **).
+ | ^~"
+
+"npar.sps:35.29-35.30: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 35 | NPAR TESTS J-T x BY y (1, 2 **).
+ | ^~"
+
+"npar.sps:36.20-36.21: error: NPAR TESTS: Syntax error expecting variable name.
+ 36 | NPAR TESTS MCNEMAR **.
+ | ^~"
+
+"npar.sps:37.22-37.23: error: NPAR TESTS: Syntax error expecting end of command.
+ 37 | NPAR TESTS MCNEMAR x **.
+ | ^~"
+
+"npar.sps:38.27-38.28: error: NPAR TESTS: Syntax error expecting variable name.
+ 38 | NPAR TESTS MCNEMAR x WITH **.
+ | ^~"
+
+"npar.sps:39.30-39.31: error: NPAR TESTS: Syntax error expecting `PAIRED@:}@'.
+ 39 | NPAR TESTS MCNEMAR x WITH y (**).
+ | ^~"
+
+"npar.sps:40.30-40.38: error: NPAR TESTS: Syntax error expecting `PAIRED)'.
+ 40 | NPAR TESTS MCNEMAR x WITH y (PAIRED **).
+ | ^~~~~~~~~"
+
+"npar.sps:41.20-41.29: error: NPAR TESTS: PAIRED was specified, but the number of variables preceding WITH (1) does not match the number following (2).
+ 41 | NPAR TESTS MCNEMAR x WITH y z (PAIRED).
+ | ^~~~~~~~~~"
+
+"npar.sps:42.20-42.21: error: NPAR TESTS: Syntax error expecting number.
+ 42 | NPAR TESTS MEDIAN (**).
+ | ^~"
+
+"npar.sps:43.22-43.23: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 43 | NPAR TESTS MEDIAN (1 **).
+ | ^~"
+
+"npar.sps:44.20-44.26: error: NPAR TESTS: Subcommand MISSING may only be specified once.
+ 44 | NPAR TESTS MISSING/MISSING.
+ | ^~~~~~~"
+
+"npar.sps:45.20-45.21: error: NPAR TESTS: Syntax error expecting ANALYSIS, LISTWISE, INCLUDE, or EXCLUDE.
+ 45 | NPAR TESTS MISSING **.
+ | ^~"
+
+"npar.sps:46.19-46.24: error: NPAR TESTS: Subcommand METHOD may only be specified once.
+ 46 | NPAR TESTS METHOD/METHOD.
+ | ^~~~~~"
+
+"npar.sps:47.31-47.32: error: NPAR TESTS: Syntax error expecting number.
+ 47 | NPAR TESTS METHOD EXACT TIMER(**).
+ | ^~"
+
+"npar.sps:48.33-48.34: error: NPAR TESTS: Syntax error expecting `@:}@'.
+ 48 | NPAR TESTS METHOD EXACT TIMER(5 **).
+ | ^~"
+
+"npar.sps:49.23-49.24: error: NPAR TESTS: Syntax error expecting DESCRIPTIVES, QUARTILES, or ALL.
+ 49 | NPAR TESTS STATISTICS **.
+ | ^~"
+
+"npar.sps:50.22-50.23: error: NPAR TESTS: Syntax error expecting COMPATIBLE or ENHANCED.
+ 50 | NPAR TESTS ALGORITHM **.
+ | ^~"
+
+"npar.sps:51.12-51.13: error: NPAR TESTS: Syntax error expecting one of the following: COCHRAN, FRIEDMAN, KENDALL, RUNS, CHISQUARE, BINOMIAL, K-S, J-T, K-W, MCNEMAR, M-W, MEDIAN, WILCOXON, SIGN, MISSING, METHOD, STATISTICS, ALGORITHM.
+ 51 | NPAR TESTS **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+AT_BANNER([NUMERIC])
+
+AT_SETUP([NUMERIC])
+AT_DATA([numeric.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+NUMERIC n/k(F5).
+DISPLAY DICTIONARY.
+])
+AT_CHECK([pspp -O format=csv numeric.sps], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+x,1,Unknown,Input,8,Right,F8.2,F8.2
+y,2,Unknown,Input,8,Right,F8.2,F8.2
+z,3,Unknown,Input,8,Right,F8.2,F8.2
+n,4,Unknown,Input,8,Right,F8.2,F8.2
+k,5,Unknown,Input,8,Right,F5.0,F5.0
+])
+AT_CLEANUP
+
+AT_SETUP([NUMERIC syntax errors])
+AT_DATA([numeric.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+NUMERIC **.
+NUMERIC n **.
+NUMERIC x.
+NUMERIC n (**).
+NUMERIC n (F50).
+NUMERIC n (A8).
+NUMERIC n (F8.0 **).
+])
+AT_CHECK([pspp -O format=csv numeric.sps], [1], [dnl
+"numeric.sps:2.9-2.10: error: NUMERIC: Syntax error expecting variable name.
+ 2 | NUMERIC **.
+ | ^~"
+
+"numeric.sps:3.11-3.12: error: NUMERIC: Syntax error expecting end of command.
+ 3 | NUMERIC n **.
+ | ^~"
+
+"numeric.sps:4.9: error: NUMERIC: There is already a variable named x.
+ 4 | NUMERIC x.
+ | ^"
+
+"numeric.sps:5.12-5.13: error: NUMERIC: Syntax error expecting valid format specifier.
+ 5 | NUMERIC n (**).
+ | ^~"
+
+"numeric.sps:6.12-6.14: error: NUMERIC: Output format F50.0 specifies width 50, but F requires a width between 1 and 40.
+ 6 | NUMERIC n (F50).
+ | ^~~"
+
+"numeric.sps:7.12-7.13: error: NUMERIC: Format type A8 may not be used with a numeric variable.
+ 7 | NUMERIC n (A8).
+ | ^~"
+
+"numeric.sps:8.17-8.18: error: NUMERIC: Syntax error expecting `@:}@'.
+ 8 | NUMERIC n (F8.0 **).
+ | ^~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017, 2020 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([ONEWAY procedure])
+
+AT_SETUP([ONEWAY basic operation])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([oneway.sps],
+ [DATA LIST NOTABLE LIST /QUALITY * BRAND * .
+BEGIN DATA
+7 3
+4 3
+3 1
+2 1
+1 1
+4 2
+2 2
+3 2
+5 3
+1 1
+4 1
+5 2
+2 2
+3 3
+6 3
+END DATA
+
+VARIABLE LABELS brand 'Manufacturer'.
+VARIABLE LABELS quality 'Breaking Strain'.
+
+VALUE LABELS /brand 1 'Aspeger' 2 'Bloggs' 3 'Charlies'.
+
+ONEWAY
+ quality BY brand
+ /STATISTICS descriptives homogeneity
+ /CONTRAST = -2 1 1
+ /CONTRAST = 0 -1 1
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Descriptives
+,Manufacturer,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
+,,,,,,Lower Bound,Upper Bound,,
+Breaking Strain,Aspeger,5,2.20,1.30,.58,.58,3.82,1.00,4.00
+,Bloggs,5,3.20,1.30,.58,1.58,4.82,2.00,5.00
+,Charlies,5,5.00,1.58,.71,3.04,6.96,3.00,7.00
+,Total,15,3.47,1.77,.46,2.49,4.45,1.00,7.00
+
+Table: Test of Homogeneity of Variances
+,Levene Statistic,df1,df2,Sig.
+Breaking Strain,.09,2,12,.913
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+Breaking Strain,Between Groups,20.13,2,10.07,5.12,.025
+,Within Groups,23.60,12,1.97,,
+,Total,43.73,14,,,
+
+Table: Contrast Coefficients
+Contrast,Manufacturer,,
+,Aspeger,Bloggs,Charlies
+1,-2,1,1
+2,0,-1,1
+
+Table: Contrast Tests
+,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
+Breaking Strain,Assume equal variances,1,3.80,1.54,2.47,12.00,.029
+,,2,1.80,.89,2.03,12.00,.065
+,Does not assume equal variances,1,3.80,1.48,2.56,8.74,.031
+,,2,1.80,.92,1.96,7.72,.086
+])
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY with splits])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([oneway-splits.sps],
+[DATA LIST NOTABLE LIST /QUALITY * BRAND * S *.
+BEGIN DATA
+3 1 1
+2 1 1
+1 1 1
+1 1 1
+4 1 1
+5 2 1
+2 2 1
+4 2 2
+2 2 2
+3 2 2
+7 3 2
+4 3 2
+5 3 2
+3 3 2
+6 3 2
+END DATA
+
+VARIABLE LABELS brand 'Manufacturer'.
+VARIABLE LABELS quality 'Breaking Strain'.
+
+VALUE LABELS /brand 1 'Aspeger' 2 'Bloggs' 3 'Charlies'.
+
+SPLIT FILE by s.
+
+ONEWAY
+ quality BY brand
+ /STATISTICS descriptives homogeneity
+ /CONTRAST = -2 2
+ /CONTRAST = -1 1
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-splits.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Split Values
+Variable,Value
+S,1.00
+
+Table: Descriptives
+,Manufacturer,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
+,,,,,,Lower Bound,Upper Bound,,
+Breaking Strain,Aspeger,5,2.20,1.30,.58,.58,3.82,1.00,4.00
+,Bloggs,2,3.50,2.12,1.50,-15.56,22.56,2.00,5.00
+,Total,7,2.57,1.51,.57,1.17,3.97,1.00,5.00
+
+Table: Test of Homogeneity of Variances
+,Levene Statistic,df1,df2,Sig.
+Breaking Strain,1.09,1,5,.345
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+Breaking Strain,Between Groups,2.41,1,2.41,1.07,.349
+,Within Groups,11.30,5,2.26,,
+,Total,13.71,6,,,
+
+Table: Contrast Coefficients
+Contrast,Manufacturer,
+,Aspeger,Bloggs
+1,-2,2
+2,-1,1
+
+Table: Contrast Tests
+,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
+Breaking Strain,Assume equal variances,1,2.60,2.52,1.03,5.00,.349
+,,2,1.30,1.26,1.03,5.00,.349
+,Does not assume equal variances,1,2.60,3.22,.81,1.32,.539
+,,2,1.30,1.61,.81,1.32,.539
+
+Table: Split Values
+Variable,Value
+S,2.00
+
+Table: Descriptives
+,Manufacturer,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
+,,,,,,Lower Bound,Upper Bound,,
+Breaking Strain,Bloggs,3,3.00,1.00,.58,.52,5.48,2.00,4.00
+,Charlies,5,5.00,1.58,.71,3.04,6.96,3.00,7.00
+,Total,8,4.25,1.67,.59,2.85,5.65,2.00,7.00
+
+Table: Test of Homogeneity of Variances
+,Levene Statistic,df1,df2,Sig.
+Breaking Strain,.92,1,6,.374
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+Breaking Strain,Between Groups,7.50,1,7.50,3.75,.101
+,Within Groups,12.00,6,2.00,,
+,Total,19.50,7,,,
+
+Table: Contrast Coefficients
+Contrast,Manufacturer,
+,Bloggs,Charlies
+1,-2,2
+2,-1,1
+
+Table: Contrast Tests
+,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
+Breaking Strain,Assume equal variances,1,4.00,2.07,1.94,6.00,.101
+,,2,2.00,1.03,1.94,6.00,.101
+,Does not assume equal variances,1,4.00,1.83,2.19,5.88,.072
+,,2,2.00,.91,2.19,5.88,.072
+])
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY with missing values])
+AT_KEYWORDS([categorical categoricals])
+dnl Check that missing are treated properly
+AT_DATA([oneway-missing1.sps],
+[DATA LIST NOTABLE LIST /v1 * v2 * dep * vn *.
+BEGIN DATA
+. . 1 4
+3 3 1 2
+2 2 1 2
+1 1 1 2
+1 1 1 2
+4 4 1 2
+5 5 2 2
+2 2 2 2
+4 4 2 2
+2 2 2 2
+3 3 2 2
+7 7 3 2
+4 4 3 2
+5 5 3 2
+3 3 3 2
+6 6 3 2
+END DATA
+
+ONEWAY
+ v1 v2 BY dep
+ /STATISTICS descriptives homogeneity
+ /MISSING ANALYSIS
+ .
+])
+
+AT_DATA([oneway-missing2.sps],
+[DATA LIST NOTABLE LIST /v1 * v2 * dep * vn * .
+BEGIN DATA
+4 . 1 2
+3 3 1 2
+2 2 1 2
+1 1 1 2
+1 1 1 2
+4 4 1 2
+5 5 2 2
+2 2 2 2
+4 4 2 2
+2 2 2 2
+3 3 2 2
+7 7 3 2
+4 4 3 2
+5 5 3 2
+3 3 3 2
+6 6 3 2
+END DATA
+
+ONEWAY
+ v1 v2 BY dep
+ /STATISTICS descriptives homogeneity
+ /MISSING LISTWISE
+ .
+])
+
+
+
+AT_CHECK([pspp -O format=csv oneway-missing1.sps > first.out], [0])
+
+AT_CHECK([pspp -O format=csv oneway-missing2.sps > second.out], [0])
+
+AT_CHECK([diff first.out second.out], [0], [])
+
+dnl Now a test with missing values in the independent variable
+AT_DATA([oneway-missing3.sps],
+[DATA LIST NOTABLE LIST /v1 * v2 * dep * vn * .
+BEGIN DATA
+4 2 . 2
+3 3 1 2
+2 2 1 2
+1 1 1 2
+1 1 1 2
+4 4 1 2
+5 5 2 2
+2 2 2 2
+4 4 2 2
+2 2 2 2
+3 3 2 2
+7 7 3 2
+4 4 3 2
+5 5 3 4
+3 3 3 2
+6 6 3 2
+END DATA
+
+ONEWAY
+ v1 v2 BY dep
+ /STATISTICS descriptives homogeneity
+ /MISSING ANALYSIS
+ .
+])
+
+AT_CHECK([pspp -O format=csv oneway-missing3.sps > third.out], [0])
+
+AT_CHECK([diff first.out third.out], [0], [])
+
+AT_CLEANUP
+
+
+
+
+
+AT_SETUP([ONEWAY descriptives subcommand])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([oneway-descriptives.sps],
+ [DATA LIST NOTABLE LIST /QUALITY * BRAND * .
+BEGIN DATA
+13 11
+12 11
+11 11
+11 11
+14 11
+15 25
+12 25
+14 25
+12 25
+13 25
+17 301
+14 301
+15 301
+13 301
+16 301
+END DATA
+
+
+ONEWAY
+ quality BY brand
+ /STATISTICS descriptives
+ .
+])
+
+AT_CHECK([pspp -O format=csv oneway-descriptives.sps], [0],
+[Table: Descriptives
+,BRAND,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
+,,,,,,Lower Bound,Upper Bound,,
+QUALITY,11.00,5,12.20,1.30,.58,10.58,13.82,11.00,14.00
+,25.00,5,13.20,1.30,.58,11.58,14.82,12.00,15.00
+,301.00,5,15.00,1.58,.71,13.04,16.96,13.00,17.00
+,Total,15,13.47,1.77,.46,12.49,14.45,11.00,17.00
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+QUALITY,Between Groups,20.13,2,10.07,5.12,.025
+,Within Groups,23.60,12,1.97,,
+,Total,43.73,14,,,
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([ONEWAY homogeneity subcommand])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([oneway-homogeneity.sps],
+ [DATA LIST NOTABLE LIST /QUALITY * BRAND * .
+BEGIN DATA
+13 11
+12 11
+11 11
+11 11
+14 11
+15 25
+12 25
+14 25
+12 25
+13 25
+17 301
+14 301
+15 301
+13 301
+16 301
+END DATA
+
+
+ONEWAY
+ quality BY brand
+ /STATISTICS homogeneity
+ .
+])
+
+AT_CHECK([pspp -O format=csv oneway-homogeneity.sps], [0],
+[Table: Test of Homogeneity of Variances
+,Levene Statistic,df1,df2,Sig.
+QUALITY,.09,2,12,.913
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+QUALITY,Between Groups,20.13,2,10.07,5.12,.025
+,Within Groups,23.60,12,1.97,,
+,Total,43.73,14,,,
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([ONEWAY multiple variables])
+AT_KEYWORDS([categorical categoricals])
+dnl check that everything works ok when several different dependent variables are specified.
+dnl This of course does not mean that we're doing a multivariate analysis. It's just like
+dnl running several tests at once.
+AT_DATA([multivar.sps],
+[DATA LIST notable LIST /x * y * z * g *.
+begin data.
+1 1 0 10
+1 1 9 10
+9 1 2 10
+1 1 3 20
+1 1 8 20
+1 1 1 20
+1 1 2 20
+0 1 3 20
+1 1 4 30
+0 1 5 30
+1 1 6 30
+0 1 7 30
+1 2 8 30
+2 2 9 30
+1 2 1 30
+1 2 0 30
+1 2 2 40
+8 2 3 40
+1 2 4 40
+1 2 9 40
+9 2 8 40
+7 3 7 40
+2 3 6 40
+3 3 5 40
+end data.
+
+ONEWAY x y z by g
+ /STATISTICS = DESCRIPTIVES HOMOGENEITY
+ /CONTRAST 3 2 0 -5
+ /CONTRAST 2 -9 7 0
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt multivar.sps])
+
+dnl Some machines return 3.88 instead of 3.87 below (see bug #31611).
+AT_CHECK([sed -e 's/^,Within Groups,3.88/,Within Groups,3.87/' pspp.csv], [0],
+ [Table: Descriptives
+,g,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
+,,,,,,Lower Bound,Upper Bound,,
+x,10.00,3,3.67,4.62,2.67,-7.81,15.14,1.00,9.00
+,20.00,5,.80,.45,.20,.24,1.36,.00,1.00
+,30.00,8,.88,.64,.23,.34,1.41,.00,2.00
+,40.00,8,4.00,3.42,1.21,1.14,6.86,1.00,9.00
+,Total,24,2.25,2.83,.58,1.05,3.45,.00,9.00
+y,10.00,3,1.00,.00,.00,1.00,1.00,1.00,1.00
+,20.00,5,1.00,.00,.00,1.00,1.00,1.00,1.00
+,30.00,8,1.50,.53,.19,1.05,1.95,1.00,2.00
+,40.00,8,2.38,.52,.18,1.94,2.81,2.00,3.00
+,Total,24,1.63,.71,.15,1.32,1.93,1.00,3.00
+z,10.00,3,3.67,4.73,2.73,-8.07,15.41,.00,9.00
+,20.00,5,3.40,2.70,1.21,.05,6.75,1.00,8.00
+,30.00,8,5.00,3.21,1.13,2.32,7.68,.00,9.00
+,40.00,8,5.50,2.45,.87,3.45,7.55,2.00,9.00
+,Total,24,4.67,2.99,.61,3.40,5.93,.00,9.00
+
+Table: Test of Homogeneity of Variances
+,Levene Statistic,df1,df2,Sig.
+x,18.76,3,20,.000
+y,71.41,3,20,.000
+z,.89,3,20,.463
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+x,Between Groups,56.16,3,18.72,2.92,.059
+,Within Groups,128.34,20,6.42,,
+,Total,184.50,23,,,
+y,Between Groups,7.75,3,2.58,13.33,.000
+,Within Groups,3.87,20,.19,,
+,Total,11.63,23,,,
+z,Between Groups,17.47,3,5.82,.62,.610
+,Within Groups,187.87,20,9.39,,
+,Total,205.33,23,,,
+
+Table: Contrast Coefficients
+Contrast,g,,,
+,10.00,20.00,30.00,40.00
+1,3,2,0,-5
+2,2,-9,7,0
+
+Table: Contrast Tests
+,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
+x,Assume equal variances,1,-7.40,6.67,-1.11,20.00,.280
+,,2,6.26,12.32,.51,20.00,.617
+,Does not assume equal variances,1,-7.40,10.04,-.74,4.53,.497
+,,2,6.26,5.85,1.07,2.87,.366
+y,Assume equal variances,1,-6.88,1.16,-5.94,20.00,.000
+,,2,3.50,2.14,1.63,20.00,.118
+,Does not assume equal variances,1,-6.88,.91,-7.51,7.00,.000
+,,2,3.50,1.32,2.65,7.00,.033
+z,Assume equal variances,1,-9.70,8.07,-1.20,20.00,.243
+,,2,11.73,14.91,.79,20.00,.440
+,Does not assume equal variances,1,-9.70,9.57,-1.01,3.64,.373
+,,2,11.73,14.53,.81,9.88,.438
+])
+
+AT_CLEANUP
+
+
+
+dnl Tests that everything treats weights properly
+AT_SETUP([ONEWAY vs. weights])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([oneway-unweighted.sps],
+[DATA LIST NOTABLE LIST /QUALITY * BRAND * W *.
+BEGIN DATA
+3 1 1
+2 1 1
+1 1 1
+1 1 1
+4 1 1
+5 2 1
+2 2 1
+4 2 1
+4 2 1
+4 2 1
+2 2 1
+2 2 1
+3 2 1
+7 3 1
+4 3 1
+5 3 1
+5 3 1
+3 3 1
+6 3 1
+END DATA.
+
+WEIGHT BY W.
+
+ONEWAY
+ quality BY brand
+ /STATISTICS descriptives homogeneity
+ .
+])
+
+AT_CHECK([pspp -o pspp-unweighted.csv oneway-unweighted.sps], [0], [ignore], [ignore])
+
+AT_DATA([oneway-weighted.sps],
+[DATA LIST NOTABLE LIST /QUALITY * BRAND * W *.
+BEGIN DATA
+3 1 1
+2 1 1
+1 1 2
+4 1 1
+5 2 1
+2 2 1
+4 2 3
+2 2 2
+3 2 1
+7 3 1
+4 3 1
+5 3 2
+3 3 1
+6 3 1
+END DATA.
+
+WEIGHT BY W.
+
+ONEWAY
+ quality BY brand
+ /STATISTICS descriptives homogeneity
+ .
+])
+
+AT_CHECK([pspp -o pspp-weighted.csv oneway-weighted.sps], [0], [ignore], [ignore])
+
+AT_CHECK([diff pspp-weighted.csv pspp-unweighted.csv], [0])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([ONEWAY posthoc LSD and BONFERRONI])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([oneway-pig.sps],[dnl
+SET FORMAT F12.3.
+data list notable list /pigmentation * family *.
+begin data.
+36 1
+39 1
+43 1
+38 1
+37 1
+46 2
+47 2
+47 2
+47 2
+43 2
+40 3
+50 3
+44 3
+48 3
+50 3
+45 4
+53 4
+56 4
+52 4
+56 4
+end data.
+
+
+oneway pigmentation by family
+ /statistics = descriptives
+ /posthoc = lsd bonferroni alpha (0.05)
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-pig.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Descriptives
+,family,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
+,,,,,,Lower Bound,Upper Bound,,
+pigmentation,1.000,5,38.600,2.702,1.208,35.245,41.955,36.000,43.000
+,2.000,5,46.000,1.732,.775,43.849,48.151,43.000,47.000
+,3.000,5,46.400,4.336,1.939,41.016,51.784,40.000,50.000
+,4.000,5,52.400,4.506,2.015,46.806,57.994,45.000,56.000
+,Total,20,45.850,5.967,1.334,43.057,48.643,36.000,56.000
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+pigmentation,Between Groups,478.950,3,159.650,12.927,.000
+,Within Groups,197.600,16,12.350,,
+,Total,676.550,19,,,
+
+Table: Multiple Comparisons (pigmentation)
+,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
+,,,,,,Lower Bound,Upper Bound
+LSD,1.000,2.000,-7.400,2.223,.004,-12.112,-2.688
+,,3.000,-7.800,2.223,.003,-12.512,-3.088
+,,4.000,-13.800,2.223,.000,-18.512,-9.088
+,2.000,1.000,7.400,2.223,.004,2.688,12.112
+,,3.000,-.400,2.223,.859,-5.112,4.312
+,,4.000,-6.400,2.223,.011,-11.112,-1.688
+,3.000,1.000,7.800,2.223,.003,3.088,12.512
+,,2.000,.400,2.223,.859,-4.312,5.112
+,,4.000,-6.000,2.223,.016,-10.712,-1.288
+,4.000,1.000,13.800,2.223,.000,9.088,18.512
+,,2.000,6.400,2.223,.011,1.688,11.112
+,,3.000,6.000,2.223,.016,1.288,10.712
+Bonferroni,1.000,2.000,-7.400,2.223,.025,-14.086,-.714
+,,3.000,-7.800,2.223,.017,-14.486,-1.114
+,,4.000,-13.800,2.223,.000,-20.486,-7.114
+,2.000,1.000,7.400,2.223,.025,.714,14.086
+,,3.000,-.400,2.223,1.000,-7.086,6.286
+,,4.000,-6.400,2.223,.065,-13.086,.286
+,3.000,1.000,7.800,2.223,.017,1.114,14.486
+,,2.000,.400,2.223,1.000,-6.286,7.086
+,,4.000,-6.000,2.223,.095,-12.686,.686
+,4.000,1.000,13.800,2.223,.000,7.114,20.486
+,,2.000,6.400,2.223,.065,-.286,13.086
+,,3.000,6.000,2.223,.095,-.686,12.686
+])
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY posthoc Tukey HSD and Games-Howell])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([oneway-tukey.sps],[dnl
+set format = f11.3.
+data list notable list /libido * dose *.
+begin data.
+3 0
+2 0
+1 0
+1 0
+4 0
+5 1
+2 1
+4 1
+2 1
+3 1
+7 2
+4 2
+5 2
+3 2
+6 2
+end data.
+
+variable label dose 'Dose of Viagra'.
+
+add value labels dose 0 'Placebo' 1 '1 Dose' 2 '2 Doses'.
+
+oneway libido by dose
+ /posthoc tukey gh.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-tukey.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+libido,Between Groups,20.133,2,10.067,5.119,.025
+,Within Groups,23.600,12,1.967,,
+,Total,43.733,14,,,
+
+Table: Multiple Comparisons (libido)
+,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
+,,,,,,Lower Bound,Upper Bound
+Tukey HSD,Placebo,1 Dose,-1.000,.887,.516,-3.366,1.366
+,,2 Doses,-2.800,.887,.021,-5.166,-.434
+,1 Dose,Placebo,1.000,.887,.516,-1.366,3.366
+,,2 Doses,-1.800,.887,.147,-4.166,.566
+,2 Doses,Placebo,2.800,.887,.021,.434,5.166
+,,1 Dose,1.800,.887,.147,-.566,4.166
+Games-Howell,Placebo,1 Dose,-1.000,.887,.479,-3.356,1.356
+,,2 Doses,-2.800,.887,.039,-5.439,-.161
+,1 Dose,Placebo,1.000,.887,.479,-1.356,3.356
+,,2 Doses,-1.800,.887,.185,-4.439,.839
+,2 Doses,Placebo,2.800,.887,.039,.161,5.439
+,,1 Dose,1.800,.887,.185,-.839,4.439
+])
+
+AT_CLEANUP
+
+AT_SETUP([ONEWAY posthoc Sidak])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([oneway-sidak.sps],[dnl
+SET FORMAT F20.4.
+
+DATA LIST notable LIST /program score.
+BEGIN DATA.
+1 9
+1 12
+1 14
+1 11
+1 13
+2 10
+2 6
+2 9
+2 9
+2 10
+3 12
+3 14
+3 11
+3 13
+3 11
+4 9
+4 8
+4 11
+4 7
+4 8
+END DATA.
+
+ONEWAY
+ score BY program
+ /MISSING ANALYSIS
+ /POSTHOC = SIDAK.
+])
+
+AT_CHECK([pspp -O format=csv oneway-sidak.sps], [0],
+[Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+score,Between Groups,54.9500,3,18.3167,7.0449,.003
+,Within Groups,41.6000,16,2.6000,,
+,Total,96.5500,19,,,
+
+Table: Multiple Comparisons (score)
+,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
+,,,,,,Lower Bound,Upper Bound
+Šidák,1.0000,2.0000,3.0000,1.0198,.056,-.0575,6.0575
+,,3.0000,-.4000,1.0198,.999,-3.4575,2.6575
+,,4.0000,3.2000,1.0198,.038,.1425,6.2575
+,2.0000,1.0000,-3.0000,1.0198,.056,-6.0575,.0575
+,,3.0000,-3.4000,1.0198,.025,-6.4575,-.3425
+,,4.0000,.2000,1.0198,1.000,-2.8575,3.2575
+,3.0000,1.0000,.4000,1.0198,.999,-2.6575,3.4575
+,,2.0000,3.4000,1.0198,.025,.3425,6.4575
+,,4.0000,3.6000,1.0198,.017,.5425,6.6575
+,4.0000,1.0000,-3.2000,1.0198,.038,-6.2575,-.1425
+,,2.0000,-.2000,1.0198,1.000,-3.2575,2.8575
+,,3.0000,-3.6000,1.0198,.017,-6.6575,-.5425
+])
+
+AT_CLEANUP
+
+AT_SETUP([ONEWAY posthoc Scheffe])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([oneway-scheffe.sps],[dnl
+set format = f11.3.
+data list notable list /usage * group *.
+begin data.
+21.00 1
+19.00 1
+18.00 1
+25.00 1
+14.00 1
+13.00 1
+24.00 1
+19.00 1
+20.00 1
+21.00 1
+15.00 2
+10.00 2
+13.00 2
+16.00 2
+14.00 2
+24.00 2
+16.00 2
+14.00 2
+18.00 2
+16.00 2
+10.00 3
+ 7.00 3
+13.00 3
+20.00 3
+ .00 3
+ 8.00 3
+ 6.00 3
+ 1.00 3
+12.00 3
+14.00 3
+18.00 4
+15.00 4
+ 3.00 4
+27.00 4
+ 6.00 4
+14.00 4
+13.00 4
+11.00 4
+ 9.00 4
+18.00 4
+end data.
+
+variable label usage 'Days of Use'.
+
+add value labels group 0 'none' 1 'one' 2 'two' 3 'three' 4 'four'.
+
+oneway usage by group
+ /posthoc scheffe.
+])
+
+AT_CHECK([pspp -O format=csv oneway-scheffe.sps], [0],
+[Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+Days of Use,Between Groups,555.275,3,185.092,6.663,.001
+,Within Groups,1000.100,36,27.781,,
+,Total,1555.375,39,,,
+
+Table: Multiple Comparisons (Days of Use)
+,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
+,,,,,,Lower Bound,Upper Bound
+Scheffé,one,two,3.800,2.357,.467,-3.112,10.712
+,,three,10.300,2.357,.001,3.388,17.212
+,,four,6.000,2.357,.110,-.912,12.912
+,two,one,-3.800,2.357,.467,-10.712,3.112
+,,three,6.500,2.357,.072,-.412,13.412
+,,four,2.200,2.357,.832,-4.712,9.112
+,three,one,-10.300,2.357,.001,-17.212,-3.388
+,,two,-6.500,2.357,.072,-13.412,.412
+,,four,-4.300,2.357,.358,-11.212,2.612
+,four,one,-6.000,2.357,.110,-12.912,.912
+,,two,-2.200,2.357,.832,-9.112,4.712
+,,three,4.300,2.357,.358,-2.612,11.212
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY bad contrast count])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([oneway-bad-contrast.sps],[dnl
+DATA LIST NOTABLE LIST /height * weight * temperature * sex *.
+BEGIN DATA.
+1884 88.6 39.97 0
+1801 90.9 39.03 0
+1801 91.7 38.98 0
+1607 56.3 36.26 1
+1608 46.3 46.26 1
+1607 55.9 37.84 1
+1604 56.6 36.81 1
+1606 56.1 34.56 1
+END DATA.
+
+ONEWAY /VARIABLES= height weight temperature BY sex
+ /CONTRAST = -1 1
+ /CONTRAST = -3 3
+ /CONTRAST = 2 -2 1
+ /CONTRAST = -9 9
+ .
+])
+
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-bad-contrast.sps], [0], [dnl
+oneway-bad-contrast.sps:18: warning: ONEWAY: In contrast list 3, the number of coefficients (3) does not equal the number of groups (2). This contrast list will be ignored.
+])
+AT_CHECK([cat pspp.csv], [0], [dnl
+"oneway-bad-contrast.sps:18: warning: ONEWAY: In contrast list 3, the number of coefficients (3) does not equal the number of groups (2). This contrast list will be ignored."
+
+Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+height,Between Groups,92629.63,1,92629.63,120.77,.000
+,Within Groups,4601.87,6,766.98,,
+,Total,97231.50,7,,,
+weight,Between Groups,2451.65,1,2451.65,174.59,.000
+,Within Groups,84.25,6,14.04,,
+,Total,2535.90,7,,,
+temperature,Between Groups,1.80,1,1.80,.13,.733
+,Within Groups,84.55,6,14.09,,
+,Total,86.36,7,,,
+
+Table: Contrast Coefficients
+Contrast,sex,
+,.00,1.00
+1,-1,1
+2,-3,3
+3,-9,9
+
+Table: Contrast Tests
+,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
+height,Assume equal variances,1,-222.27,20.23,-10.99,6.00,.000
+,,2,-666.80,60.68,-10.99,6.00,.000
+,,3,-2000.40,182.03,-10.99,6.00,.000
+,Does not assume equal variances,1,-222.27,27.67,-8.03,2.00,.015
+,,2,-666.80,83.02,-8.03,2.00,.015
+,,3,-2000.40,249.07,-8.03,2.00,.015
+weight,Assume equal variances,1,-36.16,2.74,-13.21,6.00,.000
+,,2,-108.48,8.21,-13.21,6.00,.000
+,,3,-325.44,24.63,-13.21,6.00,.000
+,Does not assume equal variances,1,-36.16,2.19,-16.48,5.42,.000
+,,2,-108.48,6.58,-16.48,5.42,.000
+,,3,-325.44,19.75,-16.48,5.42,.000
+temperature,Assume equal variances,1,-.98,2.74,-.36,6.00,.733
+,,2,-2.94,8.22,-.36,6.00,.733
+,,3,-8.83,24.67,-.36,6.00,.733
+,Does not assume equal variances,1,-.98,2.07,-.47,4.19,.660
+,,2,-2.94,6.22,-.47,4.19,.660
+,,3,-8.83,18.66,-.47,4.19,.660
+])
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY crash on single category independent variable])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([crash.sps],[
+input program.
+loop #i = 1 to 10.
+compute test = #i.
+end case.
+end loop.
+end file.
+end input program.
+
+compute x = 1.
+
+oneway test by x.
+])
+
+AT_CHECK([pspp -O format=csv crash.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([ONEWAY crash on missing dependent variable])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([crash2.sps],[dnl
+data list notable list /dv1 * dv2 * y * .
+begin data.
+2 . 2
+1 . 2
+1 . 1
+2 . 4
+3 . 4
+4 . 4
+5 . 4
+end data.
+
+ONEWAY
+ /VARIABLES= dv1 dv2 BY y
+ /STATISTICS = DESCRIPTIVES
+ /POSTHOC = BONFERRONI LSD SCHEFFE SIDAK TUKEY
+ /MISSING = ANALYSIS
+ .
+])
+
+AT_CHECK([pspp -O format=csv crash2.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+
+
+AT_SETUP([ONEWAY Games-Howell test with few cases])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([crash3.sps],[dnl
+data list notable list /dv * y * .
+begin data.
+2 2
+1 2
+1 1
+2 4
+3 4
+end data.
+
+ONEWAY
+ /VARIABLES= dv BY y
+ /POSTHOC = GH
+ .
+])
+
+AT_CHECK([pspp -O format=csv crash3.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY Crash on empty data])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([crash4.sps],[dnl
+DATA LIST NOTABLE LIST /height * weight * temperature * sex *.
+BEGIN DATA.
+1801 . . 0
+1606 . . 1
+END DATA.
+
+ONEWAY /VARIABLES= height weight temperature BY sex
+ /CONTRAST = -1 1
+ /CONTRAST = -3 3
+ /CONTRAST = 2 -2 1
+ /CONTRAST = -9 9
+ .
+])
+
+AT_CHECK([pspp -O format=csv crash4.sps], [0], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([ONEWAY Crash on invalid dependent variable])
+AT_KEYWORDS([categorical categoricals])
+AT_DATA([crash5.sps],[dnl
+data list notable list /a * b *.
+begin data.
+3 0
+2 0
+6 2
+end data.
+
+oneway a"by b.
+
+])
+
+AT_CHECK([pspp -O format=csv crash5.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+
+AT_SETUP([ONEWAY Crash on unterminated string])
+AT_KEYWORDS([categorical categoricals])
+
+AT_DATA([crash6.sps], [dnl
+DATA LIST NOTABLE LIST /height * weight * temperature * sex *.
+BEGIN DATA.
+1801 . . 0
+1606 . 0 . 1
+END DATA.
+
+ONEWAY /VARIABLES= height weight temperature BY sex
+ /CONTRAST =" 2 -2 1
+ .
+])
+
+AT_CHECK([pspp -O format=csv crash6.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([ONEWAY contrast bug])
+
+AT_KEYWORDS([categorical categoricals])
+
+
+
+dnl this example comes from: https://case.truman.edu/files/2015/06/SPSS-One-Way-ANOVA.pdf
+AT_DATA([contrasts.sps],
+[
+SET FORMAT=F10.3.
+
+DATA LIST notable LIST /relieftime drugs *.
+begin data.
+12 0
+15 0
+18 0
+16 0
+20 0
+20 1
+21 1
+22 1
+19 1
+20 1
+17 2
+16 2
+19 2
+15 2
+19 2
+14 3
+13 3
+12 3
+14 3
+11 3
+end data.
+
+ONEWAY relieftime by drugs
+ /CONTRAST 3 -1 -1 -1
+ /CONTRAST 0 2 -1 -1
+ /CONTRAST 0 0 1 -1
+ .
+])
+
+AT_CHECK([pspp -O format=csv contrasts.sps], [0], [Table: ANOVA
+,,Sum of Squares,df,Mean Square,F,Sig.
+relieftime,Between Groups,146.950,3,48.983,12.723,.000
+,Within Groups,61.600,16,3.850,,
+,Total,208.550,19,,,
+
+Table: Contrast Coefficients
+Contrast,drugs,,,
+,.000,1.000,2.000,3.000
+1,3,-1,-1,-1
+2,0,2,-1,-1
+3,0,0,1,-1
+
+Table: Contrast Tests
+,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
+relieftime,Assume equal variances,1,-1.800,3.040,-.592,16.000,.562
+,,2,10.800,2.149,5.025,16.000,.000
+,,3,4.400,1.241,3.546,16.000,.003
+,Does not assume equal variances,1,-1.800,4.219,-.427,4.611,.689
+,,2,10.800,1.421,7.599,10.158,.000
+,,3,4.400,.990,4.445,7.315,.003
+])
+AT_CLEANUP
+
+AT_SETUP([ONEWAY syntax errors])
+AT_DATA([oneway.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+ONEWAY/ **.
+ONEWAY **.
+ONEWAY x **.
+ONEWAY x BY **.
+ONEWAY x BY y/STATISTICS=**.
+ONEWAY x BY y/POSTHOC=ALPHA **.
+ONEWAY x BY y/POSTHOC=ALPHA(**).
+ONEWAY x BY y/POSTHOC=ALPHA(123 **).
+ONEWAY x BY y/POSTHOC=**.
+ONEWAY x BY y/CONTRAST=**.
+ONEWAY x BY y/MISSING=**.
+ONEWAY x BY y/ **.
+])
+AT_CHECK([pspp -O format=csv oneway.sps], [1], [dnl
+"oneway.sps:2.9-2.10: error: ONEWAY: Syntax error expecting VARIABLES.
+ 2 | ONEWAY/ **.
+ | ^~"
+
+"oneway.sps:3.8-3.9: error: ONEWAY: Syntax error expecting variable name.
+ 3 | ONEWAY **.
+ | ^~"
+
+"oneway.sps:4.10-4.11: error: ONEWAY: Syntax error expecting `BY'.
+ 4 | ONEWAY x **.
+ | ^~"
+
+"oneway.sps:5.13-5.14: error: ONEWAY: Syntax error expecting variable name.
+ 5 | ONEWAY x BY **.
+ | ^~"
+
+"oneway.sps:6.26-6.27: error: ONEWAY: Syntax error expecting DESCRIPTIVES or HOMOGENEITY.
+ 6 | ONEWAY x BY y/STATISTICS=**.
+ | ^~"
+
+"oneway.sps:7.29-7.30: error: ONEWAY: Syntax error expecting `('.
+ 7 | ONEWAY x BY y/POSTHOC=ALPHA **.
+ | ^~"
+
+"oneway.sps:8.29-8.30: error: ONEWAY: Syntax error expecting number.
+ 8 | ONEWAY x BY y/POSTHOC=ALPHA(**).
+ | ^~"
+
+"oneway.sps:9.33-9.34: error: ONEWAY: Syntax error expecting `)'.
+ 9 | ONEWAY x BY y/POSTHOC=ALPHA(123 **).
+ | ^~"
+
+"oneway.sps:10.23-10.24: error: ONEWAY: Unknown post hoc analysis method.
+ 10 | ONEWAY x BY y/POSTHOC=**.
+ | ^~"
+
+"oneway.sps:11.24-11.25: error: ONEWAY: Syntax error expecting number.
+ 11 | ONEWAY x BY y/CONTRAST=**.
+ | ^~"
+
+"oneway.sps:12.23-12.24: error: ONEWAY: Syntax error expecting INCLUDE, EXCLUDE, LISTWISE, or ANALYSIS.
+ 12 | ONEWAY x BY y/MISSING=**.
+ | ^~"
+
+"oneway.sps:13.16-13.17: error: ONEWAY: Syntax error expecting STATISTICS, POSTHOC, CONTRAST, or MISSING.
+ 13 | ONEWAY x BY y/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+AT_BANNER([OUTPUT MODIFY])
+
+AT_SETUP([OUTPUT MODIFY syntax errors])
+AT_DATA([output.sps], [dnl
+OUTPUT MODIFY/SELECT **.
+OUTPUT MODIFY/TABLECELLS SELECT **.
+OUTPUT MODIFY/TABLECELLS SELECT=**.
+OUTPUT MODIFY/TABLECELLS SELECT=[[**]].
+OUTPUT MODIFY/TABLECELLS FORMAT **.
+OUTPUT MODIFY/TABLECELLS FORMAT=ASDF5.
+OUTPUT MODIFY/TABLECELLS **.
+OUTPUT MODIFY/TABLECELLS SELECT="xyzzy" FORMAT=F8.2.
+])
+AT_CHECK([pspp -O format=csv output.sps], [1], [dnl
+"output.sps:1.22-1.23: error: OUTPUT MODIFY: Syntax error expecting TABLES.
+ 1 | OUTPUT MODIFY/SELECT **.
+ | ^~"
+
+"output.sps:2.33-2.34: error: OUTPUT MODIFY: Syntax error expecting `='.
+ 2 | OUTPUT MODIFY/TABLECELLS SELECT **.
+ | ^~"
+
+"output.sps:3.33-3.34: error: OUTPUT MODIFY: Syntax error expecting `[['.
+ 3 | OUTPUT MODIFY/TABLECELLS SELECT=**.
+ | ^~"
+
+"output.sps:4.34-4.35: error: OUTPUT MODIFY: Syntax error expecting `]]'.
+ 4 | OUTPUT MODIFY/TABLECELLS SELECT=[[**]].
+ | ^~"
+
+"output.sps:5.33-5.34: error: OUTPUT MODIFY: Syntax error expecting `='.
+ 5 | OUTPUT MODIFY/TABLECELLS FORMAT **.
+ | ^~"
+
+"output.sps:6.38: error: OUTPUT MODIFY: Unknown format type `ASDF'.
+ 6 | OUTPUT MODIFY/TABLECELLS FORMAT=ASDF5.
+ | ^"
+
+"output.sps:7.26-7.27: error: OUTPUT MODIFY: Syntax error expecting SELECT or FORMAT.
+ 7 | OUTPUT MODIFY/TABLECELLS **.
+ | ^~"
+
+"output.sps:8.33-8.39: error: OUTPUT MODIFY: Syntax error expecting `@<:@'.
+ 8 | OUTPUT MODIFY/TABLECELLS SELECT=""xyzzy"" FORMAT=F8.2.
+ | ^~~~~~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([PERMISSIONS])
+
+AT_SETUP([PERMISSIONS])
+AT_DATA([foobar], [Hello
+])
+
+chmod 666 foobar
+AT_CHECK([ls -l foobar], [0], [stdout])
+AT_CHECK([sed 's/^\(..........\).*/\1/' stdout], [0], [-rw-rw-rw-
+])
+
+AT_DATA([permissions.sps], [PERMISSIONS /FILE='foobar' PERMISSIONS=READONLY.
+])
+AT_CHECK([pspp -O format=csv permissions.sps])
+AT_CHECK([ls -l foobar], [0], [stdout])
+AT_CHECK([sed 's/^\(..........\).*/\1/' stdout], [0], [-r--r--r--
+])
+
+AT_DATA([permissions.sps], [PERMISSIONS /FILE='foobar' PERMISSIONS=WRITEABLE.
+])
+AT_CHECK([pspp -O format=csv permissions.sps])
+AT_CHECK([ls -l foobar], [0], [stdout])
+AT_CHECK([sed 's/^\(..........\).*/\1/' stdout], [0], [-rw-r--r--
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([PERMISSIONS - bad syntax])
+AT_DATA([pe.sps], [[PERMI|SIONS /FILE='foobar' PERMISSIONS=WRITEABLE.
+]])
+
+AT_CHECK([pspp -O format=csv pe.sps], [1], [dnl
+"pe.sps:1.6: error: PERMISSIONS: Syntax error expecting STRING.
+ 1 | PERMI|SIONS /FILE='foobar' PERMISSIONS=WRITEABLE.
+ | ^"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([PRINT SPACE])
+
+AT_SETUP([PRINT SPACE without arguments])
+AT_DATA([print-space.sps], [dnl
+DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+1
+2
+END DATA.
+PRINT/x.
+PRINT SPACE.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print-space.sps], [0], [dnl
+1 @&t@
+
+2 @&t@
+
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT SPACE with number])
+AT_DATA([print-space.sps], [dnl
+DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+1
+2
+END DATA.
+PRINT/x.
+PRINT SPACE 2.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print-space.sps], [0], [dnl
+1 @&t@
+
+
+2 @&t@
+
+
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT SPACE to file])
+AT_CAPTURE_FILE([output.txt])
+AT_DATA([print-space.sps], [dnl
+DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+1
+2
+END DATA.
+PRINT OUTFILE='output.txt'/x.
+PRINT SPACE OUTFILE='output.txt'.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print-space.sps])
+AT_CHECK([cat output.txt], [0], [dnl
+ 1 @&t@
+ @&t@
+ 2 @&t@
+ @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT SPACE to file with number])
+AT_CAPTURE_FILE([output.txt])
+AT_DATA([print-space.sps], [dnl
+DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+1
+2
+END DATA.
+PRINT OUTFILE='output.txt'/x.
+PRINT SPACE OUTFILE='output.txt' 2.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print-space.sps])
+AT_CHECK([cat output.txt], [0], [dnl
+ 1 @&t@
+ @&t@
+ @&t@
+ 2 @&t@
+ @&t@
+ @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT SPACE syntax errors])
+AT_DATA([print-space.sps], [dnl
+DATA LIST NOTABLE /x 1.
+PRINT SPACE OUTFILE=**.
+PRINT SPACE OUTFILE='out.txt' ENCODING=**.
+PRINT SPACE **.
+PRINT SPACE 1 'xyzzy'.
+])
+AT_CHECK([pspp -O format=csv print-space.sps], [1], [dnl
+"print-space.sps:2.21-2.22: error: PRINT SPACE: Syntax error expecting a file name or handle name.
+ 2 | PRINT SPACE OUTFILE=**.
+ | ^~"
+
+"print-space.sps:3.40-3.41: error: PRINT SPACE: Syntax error expecting string.
+ 3 | PRINT SPACE OUTFILE='out.txt' ENCODING=**.
+ | ^~"
+
+"print-space.sps:4.13-4.14: error: PRINT SPACE: Syntax error parsing expression.
+ 4 | PRINT SPACE **.
+ | ^~"
+
+"print-space.sps:5.15-5.21: error: PRINT SPACE: Syntax error expecting end of command.
+ 5 | PRINT SPACE 1 'xyzzy'.
+ | ^~~~~~~"
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT SPACE evaluation errors])
+AT_DATA([print-space.sps], [dnl
+DATA LIST NOTABLE /x 1.
+BEGIN DATA.
+1
+END DATA.
+PRINT SPACE $SYSMIS.
+PRINT SPACE -1.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print-space.sps], [0], [dnl
+"print-space.sps:5.13-5.19: warning: EXECUTE: The expression on PRINT SPACE evaluated to the system-missing value.
+ 5 | PRINT SPACE $SYSMIS.
+ | ^~~~~~~"
+
+
+
+"print-space.sps:6.13-6.14: warning: EXECUTE: The expression on PRINT SPACE evaluated to -1.
+ 6 | PRINT SPACE -1.
+ | ^~"
+
+
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([PRINT and WRITE])
+
+dnl These tests check unusual aspects of the PRINT and WRITE
+dnl transformations:
+dnl
+dnl - PRINT puts spaces between variables, unless a format
+dnl is specified explicitly.
+dnl
+dnl - WRITE doesn't put space between variables.
+dnl
+dnl - PRINT to an external file prefixes each line with a space.
+dnl
+dnl - PRINT EJECT to an external file indicates a formfeed by a "1"
+dnl in the first column.
+dnl
+dnl - WRITE writes out spaces for system-missing values, not a period.
+dnl
+dnl - When no output is specified, an empty record is output.
+
+AT_SETUP([PRINT numeric variables])
+AT_DATA([print.sps], [dnl
+data list notable /x y 1-2.
+begin data.
+12
+34
+ 6
+7
+90
+end data.
+
+print /x y.
+print eject /x y 1-2.
+print /x '-' y.
+print.
+
+execute.
+])
+AT_CHECK([pspp -O format=csv print.sps], [0], [dnl
+1 2 @&t@
+
+
+
+12
+1 -2 @&t@
+
+3 4 @&t@
+
+
+
+34
+3 -4 @&t@
+
+. 6 @&t@
+
+
+
+.6
+. -6 @&t@
+
+7 . @&t@
+
+
+
+7.
+7 -. @&t@
+
+9 0 @&t@
+
+
+
+90
+9 -0 @&t@
+
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT string variables])
+AT_DATA([print.sps], [dnl
+DATA LIST FREE /s8 (a8) s10 (a10) vl255 (a255) vl258 (a258).
+BEGIN DATA.
+12345678
+AaaaaaaaaZ
+AbbbbMaryHadALittleLambbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbZ
+AccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccZ
+87654321
+AnnnnnnnnZ
+AmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmZ
+AoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooWhyIsItSoZ
+END DATA.
+
+print
+ outfile='print.txt'
+ /s10 * vl255 * vl258 *.
+
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print.sps])
+AT_CHECK([cat print.txt], [0], [dnl
+ AaaaaaaaaZ AbbbbMaryHadALittleLambbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbZ AccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccZ @&t@
+ AnnnnnnnnZ AmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmZ AoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooWhyIsItSoZ @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT to file])
+AT_DATA([print.sps], [dnl
+data list notable /x y 1-2.
+begin data.
+12
+34
+ 6
+7
+90
+end data.
+
+print outfile='print.out' /x y.
+print eject outfile='print.out' /x y (f1,f1).
+print outfile='print.out' /x '-' y.
+print outfile='print.out'.
+
+execute.
+])
+AT_CHECK([pspp -O format=csv print.sps])
+AT_CHECK([cat print.out], [0], [dnl
+ 1 2 @&t@
+112
+ 1 -2 @&t@
+ @&t@
+ 3 4 @&t@
+134
+ 3 -4 @&t@
+ @&t@
+ . 6 @&t@
+1.6
+ . -6 @&t@
+ @&t@
+ 7 . @&t@
+17.
+ 7 -. @&t@
+ @&t@
+ 9 0 @&t@
+190
+ 9 -0 @&t@
+ @&t@
+])
+AT_CLEANUP
+
+dnl Tests for a bug which crashed when deallocating after a bad PRINT
+dnl command.
+AT_SETUP([PRINT crash bug])
+AT_DATA([print.sps], [dnl
+DATA LIST LIST NOTABLE /a * b *.
+BEGIN DATA.
+1 2
+3 4
+END DATA.
+
+PRINT F8.2
+LIST.
+])
+AT_CHECK([pspp -O format=csv print.sps], [1], [dnl
+"print.sps:7.7-7.10: error: PRINT: Syntax error expecting OUTFILE, ENCODING, RECORDS, TABLE, or NOTABLE.
+ 7 | PRINT F8.2
+ | ^~~~"
+
+Table: Data List
+a,b
+1.00,2.00
+3.00,4.00
+])
+AT_CLEANUP
+
+AT_SETUP([WRITE to file])
+AT_DATA([write.sps], [dnl
+data list notable /x y 1-2.
+begin data.
+12
+34
+ 6
+7
+90
+end data.
+
+write outfile='write.out' /x y.
+write outfile='write.out' /x y (2(f1)).
+write outfile='write.out' /x '-' y.
+write outfile='write.out'.
+
+execute.
+])
+AT_CHECK([pspp -O format=csv write.sps])
+AT_CHECK([cat write.out], [0], [dnl
+12
+12
+1-2
+
+34
+34
+3-4
+
+ 6
+ 6
+ -6
+
+7 @&t@
+7 @&t@
+7- @&t@
+
+90
+90
+9-0
+
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT overwrites file])
+AT_DATA([output.txt], [abcdef
+])
+AT_DATA([print.sps], [dnl
+DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+5
+END DATA.
+PRINT OUTFILE='output.txt'/x.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print.sps])
+AT_CHECK([cat output.txt], [0], [ 5 @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT overwrites file atomically])
+AT_DATA([output.txt], [abcdef
+])
+AT_DATA([print.sps],
+[[DATA LIST NOTABLE/x 1.
+BEGIN DATA.
+5
+END DATA.
+PRINT OUTFILE='output.txt'/x.
+HOST COMMAND=['kill -TERM $PPID'].
+]])
+AT_CHECK([pspp -O format=csv print.sps], [143], [], [ignore])
+AT_CHECK([cat output.txt], [0], [abcdef
+])
+AT_CHECK(
+ [for file in *.tmp*; do if test -e $file; then echo $file; exit 1; fi; done])
+AT_CLEANUP
+
+AT_SETUP([PRINT to same file being read])
+AT_DATA([data.txt], [5
+])
+AT_DATA([print.sps], [dnl
+DATA LIST FILE='data.txt' NOTABLE/x 1.
+COMPUTE y = x + 1.
+PRINT OUTFILE='data.txt'/y.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print.sps])
+AT_CHECK([cat data.txt], [0], [ 6.00 @&t@
+])
+AT_CHECK(
+ [for file in *.tmp*; do if test -e $file; then echo $file; exit 1; fi; done])
+AT_CLEANUP
+
+AT_SETUP([PRINT to special file])
+AT_SKIP_IF([test ! -c /dev/null])
+AT_CHECK([ln -s /dev/null foo.out || exit 77])
+AT_SKIP_IF([test ! -c foo.out])
+AT_DATA([print.sps], [dnl
+DATA LIST NOTABLE /x 1.
+BEGIN DATA.
+1
+2
+3
+4
+5
+END DATA.
+PRINT OUTFILE='foo.out'/x.
+PRINT OUTFILE='foo2.out'/x.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print.sps])
+AT_CHECK([cat foo2.out], [0], [dnl
+ 1 @&t@
+ 2 @&t@
+ 3 @&t@
+ 4 @&t@
+ 5 @&t@
+])
+ls -l foo.out foo2.out
+AT_CHECK([test -c foo.out])
+AT_CLEANUP
+
+AT_SETUP([PRINT with special line ends])
+AT_DATA([print.sps], [dnl
+FILE HANDLE lf /NAME='lf.txt' /ENDS=LF.
+FILE HANDLE crlf /NAME='crlf.txt' /ENDS=CRLF.
+DATA LIST NOTABLE /x 1.
+BEGIN DATA.
+1
+2
+3
+4
+5
+END DATA.
+PRINT OUTFILE=lf/x.
+PRINT OUTFILE=crlf/x.
+EXECUTE.
+])
+AT_CHECK([pspp -O format=csv print.sps])
+AT_CHECK([cat lf.txt], [0], [dnl
+ 1 @&t@
+ 2 @&t@
+ 3 @&t@
+ 4 @&t@
+ 5 @&t@
+])
+AT_CHECK([tr '\r' R < crlf.txt], [0], [dnl
+ 1 R
+ 2 R
+ 3 R
+ 4 R
+ 5 R
+])
+AT_CLEANUP
+
+AT_SETUP([PRINT syntax errors])
+AT_DATA([print.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+PRINT OUTFILE=**.
+PRINT ENCODING=**.
+PRINT RECORDS=-1.
+PRINT **.
+PRINT/ **.
+PRINT/'string' 0.
+PRINT/'string' 5-3.
+PRINT/y.
+PRINT/x 0.
+PRINT/x (A8).
+])
+AT_CHECK([pspp -O format=csv print.sps], [1], [dnl
+"print.sps:2.15-2.16: error: PRINT: Syntax error expecting a file name or handle name.
+ 2 | PRINT OUTFILE=**.
+ | ^~"
+
+"print.sps:3.16-3.17: error: PRINT: Syntax error expecting string.
+ 3 | PRINT ENCODING=**.
+ | ^~"
+
+"print.sps:4.15-4.16: error: PRINT: Syntax error expecting non-negative integer for RECORDS.
+ 4 | PRINT RECORDS=-1.
+ | ^~"
+
+"print.sps:5.7-5.8: error: PRINT: Syntax error expecting OUTFILE, ENCODING, RECORDS, TABLE, or NOTABLE.
+ 5 | PRINT **.
+ | ^~"
+
+"print.sps:6.8-6.9: error: PRINT: Syntax error expecting variable name.
+ 6 | PRINT/ **.
+ | ^~"
+
+"print.sps:7.16: error: PRINT: Column positions for fields must be positive.
+ 7 | PRINT/'string' 0.
+ | ^"
+
+"print.sps:8.16-8.18: error: PRINT: The ending column for a field must be greater than the starting column.
+ 8 | PRINT/'string' 5-3.
+ | ^~~"
+
+"print.sps:9.7: error: PRINT: y is not a variable name.
+ 9 | PRINT/y.
+ | ^"
+
+"print.sps:10.9: error: PRINT: Column positions for fields must be positive.
+ 10 | PRINT/x 0.
+ | ^"
+
+"print.sps:11.9-11.12: error: PRINT: Numeric variable x is not compatible with string format A8.
+ 11 | PRINT/x (A8).
+ | ^~~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([QUICK CLUSTER])
+
+AT_SETUP([QUICK CLUSTER with small data set])
+AT_DATA([quick-cluster.sps], [dnl
+DATA LIST LIST /x y z.
+BEGIN DATA.
+22,2930,4099
+17,3350,4749
+22,2640,3799
+20, 3250,4816
+15,4080,7827
+4,5,4
+5,6,5
+6,7,6
+7,8,7
+8,9,8
+9,10,9
+END DATA.
+QUICK CLUSTER x y z
+ /CRITERIA=CLUSTER(2) MXITER(20).
+])
+AT_CHECK([pspp -o pspp.csv quick-cluster.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+y,F8.0
+z,F8.0
+
+Table: Final Cluster Centers
+,Cluster,
+,1,2
+x,6.50,19.20
+y,7.50,3250.00
+z,6.50,5058.00
+
+Table: Number of Cases in each Cluster
+,,Count
+Cluster,1,6
+,2,5
+Valid,,11
+])
+AT_CLEANUP
+
+AT_SETUP([QUICK CLUSTER with large data set])
+AT_KEYWORDS([slow])
+AT_DATA([quick-cluster.sps], [dnl
+input program.
+loop #i = 1 to 50000.
+compute x = 3.
+end case.
+end loop.
+end file.
+end input program.
+QUICK CLUSTER x /CRITERIA = CLUSTER(4) NOINITIAL.
+])
+AT_CHECK([pspp -o pspp.csv quick-cluster.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Final Cluster Centers
+,Cluster,,,
+,1,2,3,4
+x,NaN,NaN,NaN,3.00
+
+Table: Number of Cases in each Cluster
+,,Count
+Cluster,1,0
+,2,0
+,3,0
+,4,50000
+Valid,,50000
+])
+AT_CLEANUP
+
+
+AT_SETUP([QUICK CLUSTER with weights])
+AT_DATA([qc-weighted.sps], [dnl
+input program.
+loop #i = 1 to 400.
+ compute x = mod (#i, 4).
+ compute w = 5.
+ end case.
+end loop.
+loop #i = 1 to 400.
+ compute x = mod (#i, 4).
+ compute w = 3.
+ end case.
+end loop.
+end file.
+end input program.
+
+weight by w.
+
+QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
+])
+
+AT_CHECK([pspp -o pspp-w.csv qc-weighted.sps])
+
+
+AT_DATA([qc-unweighted.sps], [dnl
+input program.
+loop #i = 1 to 3200.
+ compute x = mod (#i, 4).
+ end case.
+end loop.
+end file.
+end input program.
+
+QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
+])
+
+AT_CHECK([pspp -o pspp-unw.csv qc-unweighted.sps])
+
+AT_CHECK([diff pspp-w.csv pspp-unw.csv], [0])
+
+AT_CLEANUP
+
+AT_SETUP([QUICK CLUSTER with listwise missing])
+AT_DATA([quick-miss.sps], [dnl
+data list notable list /x *.
+begin data.
+1
+1
+2
+3
+4
+.
+2
+end data.
+
+QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
+])
+
+AT_CHECK([pspp -o pspp-m.csv quick-miss.sps])
+
+AT_DATA([quick-nmiss.sps], [dnl
+data list notable list /x *.
+begin data.
+1
+1
+2
+3
+4
+2
+end data.
+
+QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
+])
+
+AT_CHECK([pspp -o pspp-nm.csv quick-nmiss.sps])
+
+AT_CHECK([diff pspp-m.csv pspp-nm.csv], [0])
+
+AT_CLEANUP
+
+
+AT_SETUP([QUICK CLUSTER with pairwise missing])
+
+dnl This test runs two programs, which are identical except that one
+dnl has an extra case with one missing value. Because the syntax uses
+dnl NOINITIAL and NOUPDATE, the results should be identical except for
+dnl the final classification.
+
+AT_DATA([quick-s.sps], [dnl
+data list notable list /x * y *.
+begin data.
+1 2
+1 2.2
+1.1 1.9
+1 9
+1 10
+1.3 9.5
+0.9 8.9
+3.5 2
+3.4 3
+3.5 2.5
+3.1 2.0
+end data.
+
+QUICK CLUSTER x y
+ /PRINT = INITIAL
+ /CRITERIA = CLUSTER(3) NOINITIAL NOUPDATE
+ .
+])
+
+AT_CHECK([pspp -O format=csv quick-s.sps > pspp-s.csv])
+
+AT_DATA([quick-pw.sps], [dnl
+data list notable list /x * y *.
+begin data.
+1 2
+1 2.2
+1.1 1.9
+1 9
+1 10
+1.3 9.5
+0.9 8.9
+3.5 2
+3.4 3
+3.5 2.5
+3.1 2.0
+. 2.3
+end data.
+
+QUICK CLUSTER x y
+ /CRITERIA = CLUSTER(3) NOINITIAL NOUPDATE
+ /PRINT = INITIAL
+ /MISSING = PAIRWISE
+ .
+])
+
+AT_CHECK([pspp -O format=csv quick-pw.sps > pspp-pw.csv])
+
+AT_CHECK([head -n 13 pspp-s.csv > top-s.csv])
+AT_CHECK([head -n 13 pspp-pw.csv > top-pw.csv])
+AT_CHECK([diff top-s.csv top-pw.csv])
+
+
+AT_CHECK([grep Valid pspp-s.csv], [0], [Valid,,11
+])
+
+AT_CHECK([grep Valid pspp-pw.csv], [0], [Valid,,12
+])
+
+
+AT_CLEANUP
+
+
+
+AT_SETUP([QUICK CLUSTER crash on bad cluster quantity])
+AT_DATA([badn.sps], [dnl
+data list notable list /x * y *.
+begin data.
+1 2
+1 2.2
+end data.
+
+QUICK CLUSTER x y
+ /CRITERIA = CLUSTER(0)
+ .
+])
+
+AT_CHECK([pspp -O format=csv badn.sps], [1], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([QUICK CLUSTER infinite loop on bad command name])
+AT_DATA([quick-cluster.sps], [dnl
+data list notable list /x y.
+begin data.
+1 2
+1 2.2
+end data.
+
+QUICK CLUSTER x y /UNSUPPORTED.
+])
+AT_CHECK([pspp -O format=csv quick-cluster.sps], [1], [dnl
+"quick-cluster.sps:7.20-7.30: error: QUICK CLUSTER: Syntax error expecting MISSING, PRINT, SAVE, or CRITERIA.
+ 7 | QUICK CLUSTER x y /UNSUPPORTED.
+ | ^~~~~~~~~~~"
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([QUICK CLUSTER /PRINT subcommand])
+AT_DATA([quick-cluster.sps], [dnl
+data list notable list /cluster (A8) x y (F8.0).
+begin data.
+A 10.45 9.38
+A 10.67 9.17
+A 10.86 9.63
+A 8.77 8.45
+A 8.04 11.77
+A 10.34 9.83
+A 10.37 10.54
+A 11.49 8.18
+A 10.17 11.10
+A 11.37 9.16
+A 10.25 8.83
+A 8.69 9.92
+A 10.36 10.39
+A 10.89 10.51
+A 9.9 11.39
+A 11.1 10.91
+A 11.77 8.47
+A 9.5 10.46
+B -11.01 -9.21
+B -10.8 -11.76
+B -10.03 -10.29
+B -9.54 -9.17
+B -10.16 -9.82
+B -10.01 -8.63
+B -9.6 -10.22
+B -11.36 -10.93
+B -10.63 -10.97
+B -9.53 -10.78
+B -9.40 -10.26
+B -10.76 -9.76
+B -9.9 -10.11
+B -10.16 -9.75
+B -8.65 -11.31
+B -10.10 -10.90
+B -11.67 -9.89
+B -11.11 -9.23
+B -8.7 -8.43
+B -11.35 -8.68
+C -10.20 9.00
+C -10.12 9.92
+C -10.41 10.16
+C -9.86 10.12
+C -10.31 10.12
+C -9.57 10.16
+C -9.69 9.93
+C -9.14 10.84
+C -9.8 10.19
+C -9.97 10.22
+C -11.65 10.81
+C -9.80 11.39
+C -10.31 10.74
+C -10.26 10.38
+C -11.57 10.02
+C -10.50 9.75
+C -9.06 9.63
+C -10.17 10.82
+C -10.22 9.99
+end data.
+
+QUICK CLUSTER x y
+ /CRITERIA=CLUSTERS(3)
+ /PRINT=INITIAL CLUSTER.
+])
+
+AT_CHECK([pspp -O format=csv quick-cluster.sps], [0], [dnl
+Table: Initial Cluster Centers
+,Cluster,,
+,1,2,3
+x,-11,-12,11
+y,-12,11,11
+
+Table: Final Cluster Centers
+,Cluster,,
+,1,2,3
+x,-10,-10,10
+y,-10,10,10
+
+Table: Number of Cases in each Cluster
+,,Count
+Cluster,1,20
+,2,19
+,3,18
+Valid,,57
+
+Table: Cluster Membership
+Case Number,Cluster
+1,3
+2,3
+3,3
+4,3
+5,3
+6,3
+7,3
+8,3
+9,3
+10,3
+11,3
+12,3
+13,3
+14,3
+15,3
+16,3
+17,3
+18,3
+19,1
+20,1
+21,1
+22,1
+23,1
+24,1
+25,1
+26,1
+27,1
+28,1
+29,1
+30,1
+31,1
+32,1
+33,1
+34,1
+35,1
+36,1
+37,1
+38,1
+39,2
+40,2
+41,2
+42,2
+43,2
+44,2
+45,2
+46,2
+47,2
+48,2
+49,2
+50,2
+51,2
+52,2
+53,2
+54,2
+55,2
+56,2
+57,2
+])
+
+AT_CLEANUP
+
+
+dnl Test for a crash which happened on bad input syntax
+AT_SETUP([QUICK CLUSTER -- Empty Parentheses])
+
+AT_DATA([empty-parens.sps], [dnl
+data list notable list /x * y *.
+begin data.
+1 2
+1 2.2
+end data.
+
+QUICK CLUSTER x y
+ /CRITERIA = CONVERGE()
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv empty-parens.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([QUICK CLUSTER with save])
+AT_DATA([quick-cluster.sps], [dnl
+DATA LIST notable LIST /x y z.
+BEGIN DATA.
+22,2930,4099
+17,3350,4749
+22,2640,3799
+20, 3250,4816
+15,4080,7827
+4,5,4
+5,6,5
+6,7,6
+7,8,7
+8,9,8
+9,10,9
+END DATA.
+QUICK CLUSTER x y z
+ /CRITERIA=CLUSTER(2) MXITER(20)
+ /SAVE = CLUSTER (cluster) DISTANCE (distance).
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv quick-cluster.sps], [0], [dnl
+Table: Final Cluster Centers
+,Cluster,
+,1,2
+x,6.50,19.20
+y,7.50,3250.00
+z,6.50,5058.00
+
+Table: Number of Cases in each Cluster
+,,Count
+Cluster,1,6
+,2,5
+Valid,,11
+
+Table: Data List
+x,y,z,cluster,distance
+22.00,2930.00,4099.00,2.00,1010.98
+17.00,3350.00,4749.00,2.00,324.79
+22.00,2640.00,3799.00,2.00,1399.00
+20.00,3250.00,4816.00,2.00,242.00
+15.00,4080.00,7827.00,2.00,2890.72
+4.00,5.00,4.00,1.00,4.33
+5.00,6.00,5.00,1.00,2.60
+6.00,7.00,6.00,1.00,.87
+7.00,8.00,7.00,1.00,.87
+8.00,9.00,8.00,1.00,2.60
+9.00,10.00,9.00,1.00,4.33
+])
+AT_CLEANUP
+
+
+AT_SETUP([QUICK CLUSTER with single save])
+AT_DATA([quick-cluster.sps], [dnl
+DATA LIST notable LIST /x y z.
+BEGIN DATA.
+22,2930,4099
+17,3350,4749
+22,2640,3799
+20, 3250,4816
+15,4080,7827
+4,5,4
+5,6,5
+6,7,6
+7,8,7
+8,9,8
+9,10,9
+END DATA.
+QUICK CLUSTER x y z
+ /CRITERIA=CLUSTER(2) MXITER(20)
+ /SAVE = DISTANCE.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv quick-cluster.sps], [0], [dnl
+Table: Final Cluster Centers
+,Cluster,
+,1,2
+x,6.50,19.20
+y,7.50,3250.00
+z,6.50,5058.00
+
+Table: Number of Cases in each Cluster
+,,Count
+Cluster,1,6
+,2,5
+Valid,,11
+
+Table: Data List
+x,y,z,QCL_0
+22.00,2930.00,4099.00,1010.98
+17.00,3350.00,4749.00,324.79
+22.00,2640.00,3799.00,1399.00
+20.00,3250.00,4816.00,242.00
+15.00,4080.00,7827.00,2890.72
+4.00,5.00,4.00,4.33
+5.00,6.00,5.00,2.60
+6.00,7.00,6.00,.87
+7.00,8.00,7.00,.87
+8.00,9.00,8.00,2.60
+9.00,10.00,9.00,4.33
+])
+AT_CLEANUP
+
+
+dnl This one was noticed to crash at one point.
+AT_SETUP([QUICK CLUSTER crash on bizarre input])
+AT_DATA([badn.sps], [dnl
+data list notable list /x.
+begin da\a*
+22
+17
+22
+22
+15
+4,
+5,
+6,
+7,j8,
+9,
+end data.
+
+quick cluster x
+" /criteria=cluster(2) mxiter(20)
+ /save = distance
+ .
+
+list.
+]) dnl "
+
+AT_CHECK([pspp -O format=csv badn.sps], [1], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([QUICK CLUSTER syntax errors])
+AT_DATA([quick-cluster.sps], [dnl
+DATA LIST LIST NOTABLE /x y.
+QUICK CLUSTER **.
+QUICK CLUSTER x/MISSING=**.
+QUICK CLUSTER x/PRINT=**.
+QUICK CLUSTER x/SAVE=CLUSTER(**).
+QUICK CLUSTER x/SAVE=CLUSTER(x).
+QUICK CLUSTER x/SAVE=CLUSTER(c **).
+QUICK CLUSTER x/SAVE=DISTANCE(**).
+QUICK CLUSTER x/SAVE=DISTANCE(x).
+QUICK CLUSTER x/SAVE=DISTANCE(d **).
+QUICK CLUSTER x/SAVE=**.
+QUICK CLUSTER x/CRITERIA=CLUSTERS **.
+QUICK CLUSTER x/CRITERIA=CLUSTERS(**).
+QUICK CLUSTER x/CRITERIA=CLUSTERS(5 **).
+QUICK CLUSTER x/CRITERIA=CONVERGE **.
+QUICK CLUSTER x/CRITERIA=CONVERGE(**).
+QUICK CLUSTER x/CRITERIA=CONVERGE(5 **).
+QUICK CLUSTER x/CRITERIA=**.
+QUICK CLUSTER x/ **.
+])
+AT_CHECK([pspp -O format=csv quick-cluster.sps], [1], [dnl
+"quick-cluster.sps:2.15-2.16: error: QUICK CLUSTER: Syntax error expecting variable name.
+ 2 | QUICK CLUSTER **.
+ | ^~"
+
+"quick-cluster.sps:3.25-3.26: error: QUICK CLUSTER: Syntax error expecting LISTWISE, DEFAULT, PAIRWISE, INCLUDE, or EXCLUDE.
+ 3 | QUICK CLUSTER x/MISSING=**.
+ | ^~"
+
+"quick-cluster.sps:4.23-4.24: error: QUICK CLUSTER: Syntax error expecting CLUSTER or INITIAL.
+ 4 | QUICK CLUSTER x/PRINT=**.
+ | ^~"
+
+"quick-cluster.sps:5.30-5.31: error: QUICK CLUSTER: Syntax error expecting identifier.
+ 5 | QUICK CLUSTER x/SAVE=CLUSTER(**).
+ | ^~"
+
+"quick-cluster.sps:6.30: error: QUICK CLUSTER: A variable called `x' already exists.
+ 6 | QUICK CLUSTER x/SAVE=CLUSTER(x).
+ | ^"
+
+"quick-cluster.sps:7.32-7.33: error: QUICK CLUSTER: Syntax error expecting `@:}@'.
+ 7 | QUICK CLUSTER x/SAVE=CLUSTER(c **).
+ | ^~"
+
+"quick-cluster.sps:8.31-8.32: error: QUICK CLUSTER: Syntax error expecting identifier.
+ 8 | QUICK CLUSTER x/SAVE=DISTANCE(**).
+ | ^~"
+
+"quick-cluster.sps:9.31: error: QUICK CLUSTER: A variable called `x' already exists.
+ 9 | QUICK CLUSTER x/SAVE=DISTANCE(x).
+ | ^"
+
+"quick-cluster.sps:10.33-10.34: error: QUICK CLUSTER: Syntax error expecting `@:}@'.
+ 10 | QUICK CLUSTER x/SAVE=DISTANCE(d **).
+ | ^~"
+
+"quick-cluster.sps:11.22-11.23: error: QUICK CLUSTER: Syntax error expecting CLUSTER or DISTANCE.
+ 11 | QUICK CLUSTER x/SAVE=**.
+ | ^~"
+
+"quick-cluster.sps:12.35-12.36: error: QUICK CLUSTER: Syntax error expecting `('.
+ 12 | QUICK CLUSTER x/CRITERIA=CLUSTERS **.
+ | ^~"
+
+"quick-cluster.sps:13.35-13.36: error: QUICK CLUSTER: Syntax error expecting positive integer for CLUSTERS.
+ 13 | QUICK CLUSTER x/CRITERIA=CLUSTERS(**).
+ | ^~"
+
+"quick-cluster.sps:14.37-14.38: error: QUICK CLUSTER: Syntax error expecting `)'.
+ 14 | QUICK CLUSTER x/CRITERIA=CLUSTERS(5 **).
+ | ^~"
+
+"quick-cluster.sps:15.35-15.36: error: QUICK CLUSTER: Syntax error expecting `('.
+ 15 | QUICK CLUSTER x/CRITERIA=CONVERGE **.
+ | ^~"
+
+"quick-cluster.sps:16.35-16.36: error: QUICK CLUSTER: Syntax error expecting positive number for CONVERGE.
+ 16 | QUICK CLUSTER x/CRITERIA=CONVERGE(**).
+ | ^~"
+
+"quick-cluster.sps:17.37-17.38: error: QUICK CLUSTER: Syntax error expecting `)'.
+ 17 | QUICK CLUSTER x/CRITERIA=CONVERGE(5 **).
+ | ^~"
+
+"quick-cluster.sps:18.26-18.27: error: QUICK CLUSTER: Syntax error expecting CLUSTERS, CONVERGE, MXITER, NOINITIAL, or NOUPDATE.
+ 18 | QUICK CLUSTER x/CRITERIA=**.
+ | ^~"
+
+"quick-cluster.sps:19.18-19.19: error: QUICK CLUSTER: Syntax error expecting MISSING, PRINT, SAVE, or CRITERIA.
+ 19 | QUICK CLUSTER x/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([RANK])
+
+AT_SETUP([RANK simple case with defaults])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /x (f8).
+BEGIN DATA.
+-1
+0
+1
+2
+2
+4
+5
+END DATA.
+
+RANK x.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,Rx,RANK
+
+Table: Data List
+x,Rx
+-1,1.000
+0,2.000
+1,3.000
+2,4.500
+2,4.500
+4,6.000
+5,7.000
+])
+AT_CLEANUP
+
+# This checks for regression against a crash reported as bug #38482
+# that occurred when multiple variables were specified without any
+# rank specifications.
+AT_SETUP([RANK multiple variables with defaults])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /x * y * z *.
+BEGIN DATA.
+ 1.00 2.00 3.00
+ 4.00 5.00 6.00
+END DATA.
+
+RANK ALL.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,Rx,RANK
+y,Ry,RANK
+z,Rz,RANK
+
+Table: Data List
+x,y,z,Rx,Ry,Rz
+1.00,2.00,3.00,1.000,1.000,1.000
+4.00,5.00,6.00,2.000,2.000,2.000
+])
+AT_CLEANUP
+
+AT_SETUP([RANK with RANK, RFRACTION, N])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /a * b *.
+BEGIN DATA.
+0 24
+1 32
+2 31
+2 32
+4 30
+5 29
+6 1
+7 43
+8 .
+9 45
+END DATA.
+
+RANK a b (D)
+ /PRINT=YES
+ /RANK
+ /TIES=HIGH
+ /RFRACTION
+ /N INTO count
+ .
+
+DISPLAY DICTIONARY.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+a,Ra,RANK
+b,Rb,RANK
+a,RFR001,RFRACTION
+b,RFR002,RFRACTION
+a,count,N
+b,Nb,N
+
+Table: Variables
+Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+a,1,,Nominal,Input,8,Right,F8.2,F8.2
+b,2,,Nominal,Input,8,Right,F8.2,F8.2
+Ra,3,RANK of a,Ordinal,Input,8,Right,F9.3,F9.3
+RFR001,4,RFRACTION of a,Ordinal,Input,8,Right,F6.4,F6.4
+count,5,N of a,Scale,Input,8,Right,F6.0,F6.0
+Rb,6,RANK of b,Ordinal,Input,8,Right,F9.3,F9.3
+RFR002,7,RFRACTION of b,Ordinal,Input,8,Right,F6.4,F6.4
+Nb,8,N of b,Scale,Input,8,Right,F6.0,F6.0
+
+Table: Data List
+a,b,Ra,RFR001,count,Rb,RFR002,Nb
+.00,24.00,10.000,1.0000,10,8.000,.8889,9
+1.00,32.00,9.000,.9000,10,4.000,.4444,9
+2.00,31.00,8.000,.8000,10,5.000,.5556,9
+2.00,32.00,8.000,.8000,10,4.000,.4444,9
+4.00,30.00,6.000,.6000,10,6.000,.6667,9
+5.00,29.00,5.000,.5000,10,7.000,.7778,9
+6.00,1.00,4.000,.4000,10,9.000,1.0000,9
+7.00,43.00,3.000,.3000,10,2.000,.2222,9
+8.00,. ,2.000,.2000,10,. ,. ,.
+9.00,45.00,1.000,.1000,10,1.000,.1111,9
+])
+AT_CLEANUP
+
+AT_SETUP([RANK with SAVAGE, PERCENT, PROPORTION, NTILES])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /a * b *.
+BEGIN DATA.
+0 24
+1 32
+2 31
+2 32
+4 30
+5 29
+6 1
+7 43
+8 8
+9 45
+END DATA.
+
+RANK a
+ /PRINT=YES
+ /TIES=CONDENSE
+ /SAVAGE
+ /PERCENT
+ /PROPORTION
+ /NTILES(4)
+ /NORMAL
+.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function,Fraction
+a,Sa,SAVAGE,
+a,Pa,PERCENT,
+a,PRO001,PROPORTION,BLOM
+a,Na,NTILES,
+a,NOR001,NORMAL,BLOM
+
+Table: Data List
+a,b,Sa,Pa,PRO001,Na,NOR001
+.00,24.00,-.9000,10.00,.0610,1,-1.547
+1.00,32.00,-.7889,20.00,.1585,1,-1.000
+2.00,31.00,-.5925,30.00,.2561,2,-.6554
+2.00,32.00,-.5925,30.00,.2561,2,-.6554
+4.00,30.00,-.3544,40.00,.3537,2,-.3755
+5.00,29.00,-.1544,50.00,.4512,2,-.1226
+6.00,1.00,.0956,60.00,.5488,3,.1226
+7.00,43.00,.4290,70.00,.6463,3,.3755
+8.00,8.00,.9290,80.00,.7439,3,.6554
+9.00,45.00,1.9290,90.00,.8415,4,1.0005
+])
+AT_CLEANUP
+
+AT_SETUP([RANK with SPLIT FILE])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /a * g1 g2 *.
+BEGIN DATA.
+2 1 2
+2 1 2
+3 1 2
+4 1 2
+5 1 2
+1 0 2
+2 0 2
+3 0 2
+4 0 2
+5 0 2
+6 0 2
+7 0 2
+8 0 2
+6 1 2
+7 1 2
+7 1 2
+8 1 2
+9 1 1
+END DATA.
+
+RANK a (D) BY g2 g1
+ /PRINT=YES
+ /TIES=LOW
+ /MISSING=INCLUDE
+ /FRACTION=RANKIT
+ /RANK
+ /NORMAL
+ .
+
+SPLIT FILE BY g1.
+
+RANK a (D) BY g2
+ /PRINT=YES
+ /TIES=LOW
+ /MISSING=INCLUDE
+ /FRACTION=RANKIT
+ /RANK
+ /NORMAL
+ .
+
+SPLIT FILE OFF.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function,Fraction,Grouping Variables
+a,Ra,RANK,,g2 g1
+a,Na,NORMAL,RANKIT,g2 g1
+
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function,Fraction,Grouping Variables
+a,RAN001,RANK,,g2
+a,NOR001,NORMAL,RANKIT,g2
+
+Table: Data List
+a,g1,g2,Ra,Na,RAN001,NOR001
+2.00,1.00,2.00,8.000,.9674,4.000,.5244
+2.00,1.00,2.00,8.000,.9674,4.000,.5244
+3.00,1.00,2.00,7.000,.5895,3.000,.0000
+4.00,1.00,2.00,6.000,.2822,2.000,-.5244
+5.00,1.00,2.00,5.000,.0000,1.000,-1.282
+1.00,.00,2.00,8.000,1.5341,8.000,1.5341
+2.00,.00,2.00,7.000,.8871,7.000,.8871
+3.00,.00,2.00,6.000,.4888,6.000,.4888
+4.00,.00,2.00,5.000,.1573,5.000,.1573
+5.00,.00,2.00,4.000,-.1573,4.000,-.1573
+6.00,.00,2.00,3.000,-.4888,3.000,-.4888
+7.00,.00,2.00,2.000,-.8871,2.000,-.8871
+8.00,.00,2.00,1.000,-1.534,1.000,-1.534
+6.00,1.00,2.00,4.000,-.2822,4.000,1.1503
+7.00,1.00,2.00,2.000,-.9674,2.000,-.3186
+7.00,1.00,2.00,2.000,-.9674,2.000,-.3186
+8.00,1.00,2.00,1.000,-1.593,1.000,-1.150
+9.00,1.00,1.00,1.000,.0000,1.000,.0000
+])
+AT_CLEANUP
+
+# Also tests small ranks for special case of SAVAGE ranks.
+AT_SETUP([RANK with fractional ranks])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /a * w * .
+BEGIN DATA.
+1 1.5
+2 0.2
+3 0.1
+4 1
+5 1
+6 1
+7 1
+8 1
+END DATA.
+
+WEIGHT BY w.
+
+RANK a
+ /FRACTION=TUKEY
+ /PROPORTION
+ /SAVAGE
+.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function,Fraction
+a,Pa,PROPORTION,TUKEY
+a,Sa,SAVAGE,
+
+Table: Data List
+a,w,Pa,Sa
+1.00,1.50,.1285,-.8016
+2.00,.20,.1776,-.6905
+3.00,.10,.1986,-.6905
+4.00,1.00,.3458,-.5305
+5.00,1.00,.4860,-.2905
+6.00,1.00,.6262,.0262
+7.00,1.00,.7664,.4929
+8.00,1.00,.9065,1.3929
+])
+AT_CLEANUP
+
+AT_SETUP([RANK all-ties due to tiny weights])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /x * w *.
+BEGIN DATA.
+1 0.1
+2 0.1
+3 0.1
+4 0.2
+5 0.1
+6 0.1
+7 0.1
+8 0.1
+END DATA.
+
+WEIGHT BY w.
+
+RANK x
+ /TIES=low
+ /RANK into xl.
+
+
+RANK x
+ /TIES=high
+ /RANK into xh.
+
+RANK x
+ /TIES=condense
+ /RANK into xc.
+
+
+* Test VW fraction
+
+RANK x
+ /FRACTION=VW
+ /NORMAL.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,xl,RANK
+
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,xh,RANK
+
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,xc,RANK
+
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function,Fraction
+x,Nx,NORMAL,VW
+
+Table: Data List
+x,w,xl,xh,xc,Nx
+1.00,.10,.000,.100,1.000,-1.938
+2.00,.10,.100,.200,2.000,-1.412
+3.00,.10,.200,.300,3.000,-1.119
+4.00,.20,.300,.500,4.000,-.8046
+5.00,.10,.500,.600,5.000,-.5549
+6.00,.10,.600,.700,6.000,-.4067
+7.00,.10,.700,.800,7.000,-.2670
+8.00,.10,.800,.900,8.000,-.1323
+])
+AT_CLEANUP
+
+AT_SETUP([RANK and TEMPORARY])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /age (f2) gender (a1).
+BEGIN DATA.
+44 m
+32 f
+43 m
+49 m
+33 f
+35 f
+29 f
+50 m
+42 m
+33 f
+48 m
+END DATA.
+
+TEMPORARY.
+SELECT IF gender = 'm'.
+RANK age /RANK INTO Rm.
+
+TEMPORARY.
+SELECT IF gender = 'f'.
+RANK age /RANK INTO Rf.
+
+LIST.
+])
+AT_CHECK([pspp -O format=csv rank.sps], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+age,Rm,RANK
+
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+age,Rf,RANK
+
+Table: Data List
+age,gender,Rm,Rf
+44,m,3.000,. @&t@
+32,f,. ,2.000
+43,m,2.000,. @&t@
+49,m,5.000,. @&t@
+33,f,. ,3.500
+35,f,. ,5.000
+29,f,. ,1.000
+50,m,6.000,. @&t@
+42,m,1.000,. @&t@
+33,f,. ,3.500
+48,m,4.000,. @&t@
+])
+AT_CLEANUP
+
+AT_SETUP([RANK variable name fallback])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /foo * rfoo * ran003 *.
+BEGIN DATA.
+0 3 2
+1 3 2
+2 3 2
+2 3 2
+4 3 2
+5 3 2
+6 3 2
+7 3 2
+8 3 2
+9 3 2
+END DATA.
+
+RANK foo.
+
+
+DISPLAY DICTIONARY.
+])
+AT_CHECK([pspp -o pspp.csv rank.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+foo,RAN001,RANK
+
+Table: Variables
+Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+foo,1,,Nominal,Input,8,Right,F8.2,F8.2
+rfoo,2,,Nominal,Input,8,Right,F8.2,F8.2
+ran003,3,,Nominal,Input,8,Right,F8.2,F8.2
+RAN001,4,RANK of foo,Ordinal,Input,8,Right,F9.3,F9.3
+])
+AT_CLEANUP
+
+AT_SETUP([RANK robust variable name creation])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST notable /x * rx * ran001 TO ran999.
+BEGIN DATA.
+1
+2
+3
+4
+5
+6
+7
+END DATA.
+
+RANK x.
+
+DELETE VAR ran001 TO ran999.
+
+LIST.
+])
+AT_CHECK([pspp -O format=csv rank.sps], [0], [dnl
+"rank.sps:3: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"rank.sps:4: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"rank.sps:5: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"rank.sps:6: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"rank.sps:7: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"rank.sps:8: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+"rank.sps:9: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
+
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,RNKRA01,RANK
+
+Table: Data List
+x,rx,RNKRA01
+1.00,. ,1.000
+2.00,. ,2.000
+3.00,. ,3.000
+4.00,. ,4.000
+5.00,. ,5.000
+6.00,. ,6.000
+7.00,. ,7.000
+])
+AT_CLEANUP
+
+dnl Test for proper behaviour in the face of invalid input.
+AT_SETUP([RANK handling of invalid input])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /x * a (a2).
+BEGIN DATA.
+-1 s
+0 s
+1 s
+2 s
+2 s
+4 s
+5 s
+END DATA.
+
+DEBUG XFORM FAIL.
+
+RANK x.
+])
+AT_CHECK([pspp -O format=csv --testing-mode rank.sps], [1], [dnl
+Table: Variables Created by RANK
+Existing Variable,New Variable,Function
+x,Rx,RANK
+
+rank.sps:14: error: RANK: DEBUG XFORM FAIL transformation executed
+])
+AT_CLEANUP
+
+AT_SETUP([RANK syntax errors])
+AT_DATA([rank.sps], [dnl
+DATA LIST LIST NOTABLE /x y z * a b c (a2).
+RANK VARIABLES **.
+RANK **.
+RANK x BY **.
+RANK x/TIES **.
+RANK x/TIES=**.
+RANK x/FRACTION **.
+RANK x/FRACTIO=**.
+RANK x/PRINT **.
+RANK x/PRINT=**.
+RANK x/MISSING **.
+RANK x/MISSING=**.
+RANK x/NTILES **.
+RANK x/NTILES(**).
+RANK x/NTILES(5 **).
+RANK x/ **.
+RANK x/N INTO v w.
+RANK x/N INTO y.
+RANK x y/N INTO v v.
+])
+AT_CHECK([pspp -O format=csv rank.sps], [1], [dnl
+"rank.sps:2.16-2.17: error: RANK: Syntax error expecting `='.
+ 2 | RANK VARIABLES **.
+ | ^~"
+
+"rank.sps:3.6-3.7: error: RANK: Syntax error expecting variable name.
+ 3 | RANK **.
+ | ^~"
+
+"rank.sps:4.11-4.12: error: RANK: Syntax error expecting variable name.
+ 4 | RANK x BY **.
+ | ^~"
+
+"rank.sps:5.13-5.14: error: RANK: Syntax error expecting `='.
+ 5 | RANK x/TIES **.
+ | ^~"
+
+"rank.sps:6.13-6.14: error: RANK: Syntax error expecting MEAN, LOW, HIGH, or CONDENSE.
+ 6 | RANK x/TIES=**.
+ | ^~"
+
+"rank.sps:7.17-7.18: error: RANK: Syntax error expecting `='.
+ 7 | RANK x/FRACTION **.
+ | ^~"
+
+"rank.sps:8.16-8.17: error: RANK: Syntax error expecting BLOM, TUKEY, VW, or RANKIT.
+ 8 | RANK x/FRACTIO=**.
+ | ^~"
+
+"rank.sps:9.14-9.15: error: RANK: Syntax error expecting `='.
+ 9 | RANK x/PRINT **.
+ | ^~"
+
+"rank.sps:10.14-10.15: error: RANK: Syntax error expecting YES or NO.
+ 10 | RANK x/PRINT=**.
+ | ^~"
+
+"rank.sps:11.16-11.17: error: RANK: Syntax error expecting `='.
+ 11 | RANK x/MISSING **.
+ | ^~"
+
+"rank.sps:12.16-12.17: error: RANK: Syntax error expecting INCLUDE or EXCLUDE.
+ 12 | RANK x/MISSING=**.
+ | ^~"
+
+"rank.sps:13.15-13.16: error: RANK: Syntax error expecting `('.
+ 13 | RANK x/NTILES **.
+ | ^~"
+
+"rank.sps:14.15-14.16: error: RANK: Syntax error expecting positive integer for NTILES.
+ 14 | RANK x/NTILES(**).
+ | ^~"
+
+"rank.sps:15.17-15.18: error: RANK: Syntax error expecting `)'.
+ 15 | RANK x/NTILES(5 **).
+ | ^~"
+
+"rank.sps:16.9-16.10: error: RANK: Syntax error expecting RANK, NORMAL, RFRACTION, N, SAVAGE, PERCENT, PROPORTION, or NTILES.
+ 16 | RANK x/ **.
+ | ^~"
+
+"rank.sps:17.15-17.17: error: RANK: Too many variables in INTO clause.
+ 17 | RANK x/N INTO v w.
+ | ^~~"
+
+"rank.sps:18.15: error: RANK: Variable y already exists.
+ 18 | RANK x/N INTO y.
+ | ^"
+
+"rank.sps:19.19: error: RANK: Duplicate variable name v.
+ 19 | RANK x y/N INTO v v.
+ | ^"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([RECODE transformation])
+
+m4_define([RECODE_SAMPLE_DATA],
+ [DATA LIST LIST NOTABLE/x (f1) s (a4) t (a10).
+MISSING VALUES x(9)/s('xxx').
+BEGIN DATA.
+0, '', ''
+1, a, a
+2, ab, ab
+3, abc, abc
+4, abcd, abcd
+5, 123, 123
+6, ' 123', ' 123'
+7, +1, +1
+8, 1x, 1x
+9, abcd, abcdefghi
+, xxx, abcdefghij
+END DATA.
+])
+
+AT_SETUP([RECODE numeric to numeric, without INTO])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+NUMERIC x0 TO x8 (F3).
+MISSING VALUES x0 to x8 (9).
+COMPUTE x0=value(x).
+RECODE x0 (1=9).
+COMPUTE x1=value(x).
+RECODE x1 (1=9)(3=8)(5=7).
+COMPUTE x2=value(x).
+RECODE x2 (1=8)(2,3,4,5,6,8=9)(9=1).
+COMPUTE x3=value(x).
+RECODE x3 (1 THRU 9=10)(MISSING=11).
+COMPUTE x4=value(x).
+RECODE x4 (MISSING=11)(1 THRU 9=10).
+COMPUTE x5=value(x).
+RECODE x5 (LOWEST THRU 5=1).
+COMPUTE x6=value(x).
+RECODE x6 (4 THRU HIGHEST=2).
+COMPUTE x7=value(x).
+RECODE x7 (LO THRU HI=3).
+COMPUTE x8=value(x).
+RECODE x8 (SYSMIS=4).
+LIST x x0 TO x8.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+x,x0,x1,x2,x3,x4,x5,x6,x7,x8
+0,0,0,0,0,0,1,0,3,0
+1,9,9,8,10,10,1,1,3,1
+2,2,2,9,10,10,1,2,3,2
+3,3,8,9,10,10,1,3,3,3
+4,4,4,9,10,10,1,2,3,4
+5,5,7,9,10,10,1,2,3,5
+6,6,6,9,10,10,6,2,3,6
+7,7,7,7,10,10,7,2,3,7
+8,8,8,9,10,10,8,2,3,8
+9,9,9,1,10,11,9,2,3,9
+.,.,.,.,11,11,.,.,.,4
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE numeric to numeric, with INTO, without COPY])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+NUMERIC ix0 TO ix8 (F3).
+RECODE x (1=9) INTO ix0.
+RECODE x (1=9)(3=8)(5=7) INTO ix1.
+RECODE x (1=8)(2,3,4,5,6,8=9)(9=1) INTO ix2.
+RECODE x (1 THRU 9=10)(MISSING=11) INTO ix3.
+RECODE x (MISSING=11)(1 THRU 9=10) INTO ix4.
+RECODE x (LOWEST THRU 5=1) INTO ix5.
+RECODE x (4 THRU HIGHEST=2) INTO ix6.
+RECODE x (LO THRU HI=3) INTO ix7.
+RECODE x (SYSMIS=4) INTO ix8.
+LIST x ix0 TO ix8.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+x,ix0,ix1,ix2,ix3,ix4,ix5,ix6,ix7,ix8
+0,.,.,.,.,.,1,.,3,.
+1,9,9,8,10,10,1,.,3,.
+2,.,.,9,10,10,1,.,3,.
+3,.,8,9,10,10,1,.,3,.
+4,.,.,9,10,10,1,2,3,.
+5,.,7,9,10,10,1,2,3,.
+6,.,.,9,10,10,.,2,3,.
+7,.,.,.,10,10,.,2,3,.
+8,.,.,9,10,10,.,2,3,.
+9,.,.,1,10,11,.,2,3,.
+.,.,.,.,11,11,.,.,.,4
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE numeric to numeric, with INTO, with COPY])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+NUMERIC cx0 TO cx8 (F3).
+RECODE x (1=9)(ELSE=COPY) INTO cx0.
+RECODE x (1=9)(3=8)(5=7)(ELSE=COPY) INTO cx1.
+RECODE x (1=8)(2,3,4,5,6,8=9)(9=1)(ELSE=COPY) INTO cx2.
+RECODE x (1 THRU 9=10)(MISSING=11)(ELSE=COPY) INTO cx3.
+RECODE x (MISSING=11)(1 THRU 9=10)(ELSE=COPY) INTO cx4.
+RECODE x (LOWEST THRU 5=1)(ELSE=COPY) INTO cx5.
+RECODE x (4 THRU HIGHEST=2)(ELSE=COPY) INTO cx6.
+RECODE x (LO THRU HI=3)(ELSE=COPY) INTO cx7.
+RECODE x (SYSMIS=4)(ELSE=COPY) INTO cx8.
+RECODE x (5=COPY)(ELSE=22) INTO cx9.
+LIST x cx0 TO cx9.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+x,cx0,cx1,cx2,cx3,cx4,cx5,cx6,cx7,cx8,cx9
+0,0,0,0,0,0,1,0,3,0,22.00
+1,9,9,8,10,10,1,1,3,1,22.00
+2,2,2,9,10,10,1,2,3,2,22.00
+3,3,8,9,10,10,1,3,3,3,22.00
+4,4,4,9,10,10,1,2,3,4,22.00
+5,5,7,9,10,10,1,2,3,5,5.00
+6,6,6,9,10,10,6,2,3,6,22.00
+7,7,7,7,10,10,7,2,3,7,22.00
+8,8,8,9,10,10,8,2,3,8,22.00
+9,9,9,1,10,11,9,2,3,9,22.00
+.,.,.,.,11,11,.,.,.,4,22.00
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE string to string, with INTO, without COPY])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+STRING s0 TO s3 (A4)/t0 TO t3 (A10).
+RECODE s t ('a'='b')('ab'='bc') INTO s0 t0.
+RECODE s t ('abcd'='xyzw') INTO s1 t1.
+RECODE s t ('abc'='def')(ELSE='xyz') INTO s2 t2.
+RECODE t ('a'='b')('abcdefghi'='xyz')('abcdefghij'='jklmnopqr') INTO t3.
+RECODE s (MISSING='gone') INTO s3.
+LIST s t s0 TO s3 t0 TO t3.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+s,t,s0,s1,s2,s3,t0,t1,t2,t3
+,,,,xyz,,,,xyz,
+a,a,b,,xyz,,b,,xyz,b
+ab,ab,bc,,xyz,,bc,,xyz,
+abc,abc,,,def,,,,def,
+abcd,abcd,,xyzw,xyz,,,xyzw,xyz,
+123,123,,,xyz,,,,xyz,
+123,123,,,xyz,,,,xyz,
++1,+1,,,xyz,,,,xyz,
+1x,1x,,,xyz,,,,xyz,
+abcd,abcdefghi,,xyzw,xyz,,,,xyz,xyz
+xxx,abcdefghij,,,xyz,gone,,,xyz,jklmnopqr
+])
+AT_CLEANUP
+
+AT_SETUP(RECODE string to string, with INTO, with COPY])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+STRING cs0 TO cs2 (A4)/ct0 TO ct3 (A10).
+RECODE s t ('a'='b')('ab'='bc')(ELSE=COPY) INTO cs0 ct0.
+RECODE s t ('abcd'='xyzw')(ELSE=COPY) INTO cs1 ct1.
+RECODE s t ('abc'='def')(ELSE='xyz')(ELSE=COPY) INTO cs2 ct2.
+RECODE t ('a'='b')('abcdefghi'='xyz')('abcdefghij'='jklmnopqr')(ELSE=COPY)
+ INTO ct3.
+LIST s t cs0 TO cs2 ct0 TO ct3.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+s,t,cs0,cs1,cs2,ct0,ct1,ct2,ct3
+,,,,xyz,,,xyz,
+a,a,b,a,xyz,b,a,xyz,b
+ab,ab,bc,ab,xyz,bc,ab,xyz,ab
+abc,abc,abc,abc,def,abc,abc,def,abc
+abcd,abcd,abcd,xyzw,xyz,abcd,xyzw,xyz,abcd
+123,123,123,123,xyz,123,123,xyz,123
+123,123,123,123,xyz,123,123,xyz,123
++1,+1,+1,+1,xyz,+1,+1,xyz,+1
+1x,1x,1x,1x,xyz,1x,1x,xyz,1x
+abcd,abcdefghi,abcd,xyzw,xyz,abcdefghi,abcdefghi,xyz,xyz
+xxx,abcdefghij,xxx,xxx,xyz,abcdefghij,abcdefghij,xyz,jklmnopqr
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE string to numeric])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+NUMERIC ns0 TO ns2 (F3)/nt0 TO nt2 (F3).
+RECODE s t (CONVERT)(' '=0)('abcd'=1) INTO ns0 nt0.
+RECODE s t (' '=0)(CONVERT)('abcd'=1) INTO ns1 nt1.
+RECODE s t ('1x'=1)('abcd'=2)(ELSE=3) INTO ns2 nt2.
+LIST s t ns0 TO ns2 nt0 TO nt2.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+s,t,ns0,ns1,ns2,nt0,nt1,nt2
+,,.,0,3,.,0,3
+a,a,.,.,3,.,.,3
+ab,ab,.,.,3,.,.,3
+abc,abc,.,.,3,.,.,3
+abcd,abcd,1,1,2,1,1,2
+123,123,123,123,3,123,123,3
+123,123,123,123,3,123,123,3
++1,+1,1,1,3,1,1,3
+1x,1x,.,.,1,.,.,1
+abcd,abcdefghi,1,1,2,.,.,3
+xxx,abcdefghij,.,.,3,.,.,3
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE numeric to string])
+AT_DATA([recode.sps],
+ [RECODE_SAMPLE_DATA
+STRING sx0 TO sx2 (a10).
+RECODE x (1 THRU 9='abcdefghij') INTO sx0.
+RECODE x (0,1,3,5,7,MISSING='xxx') INTO sx1.
+RECODE x (2 THRU 6,SYSMIS='xyz')(ELSE='foobar') INTO sx2.
+LIST x sx0 TO sx2.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Data List
+x,sx0,sx1,sx2
+0,,xxx,foobar
+1,abcdefghij,xxx,foobar
+2,abcdefghij,,xyz
+3,abcdefghij,xxx,xyz
+4,abcdefghij,,xyz
+5,abcdefghij,xxx,xyz
+6,abcdefghij,,xyz
+7,abcdefghij,xxx,foobar
+8,abcdefghij,,foobar
+9,abcdefghij,xxx,foobar
+.,,xxx,xyz
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE bug in COPY])
+AT_DATA([recode.sps],
+ [DATA LIST LIST
+ /A (A1)
+ B (A1).
+
+BEGIN DATA
+1 2
+2 3
+3 4
+END DATA.
+
+** Clearly, the else=copy is superfluous here
+RECODE A ("1"="3") ("3"="1") (ELSE=COPY).
+EXECUTE.
+LIST.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Reading free-form data from INLINE.
+Variable,Format
+A,A1
+B,A1
+
+Table: Data List
+A,B
+3,2
+2,3
+1,4
+])
+AT_CLEANUP
+
+AT_SETUP([RECODE bug in COPY with INTO])
+AT_DATA([recode.sps],
+ [DATA LIST LIST
+ /A (A1)
+ B (A1).
+
+BEGIN DATA
+1 2
+2 3
+3 4
+END DATA.
+
+STRING A1 (A1).
+RECODE A ("1"="3") ("3"="1") (ELSE=COPY) INTO a1.
+EXECUTE.
+LIST.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [0],
+ [Table: Reading free-form data from INLINE.
+Variable,Format
+A,A1
+B,A1
+
+Table: Data List
+A,B,A1
+1,2,3
+2,3,2
+3,4,1
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([RECODE increased string widths])
+
+AT_DATA([recode.sps],[dnl
+data list notable list /x (a1) y (a8) z *.
+begin data.
+a a 2
+a two 2
+b three 2
+c b 2
+end data.
+
+recode x y ("a" = "first") .
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv recode.sps], [1], [dnl
+recode.sps:9: error: RECODE: At least one target variable is too narrow for the output values.
+
+"recode.sps:9.19-9.25: note: RECODE: This recoding output value has width 5.
+ 9 | recode x y (""a"" = ""first"") .
+ | ^~~~~~~"
+
+"recode.sps:9.8-9.10: note: RECODE: Target variable x only has width 1.
+ 9 | recode x y (""a"" = ""first"") .
+ | ^~~"
+
+Table: Data List
+x,y,z
+a,a,2.00
+a,two,2.00
+b,three,2.00
+c,b,2.00
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([RECODE crash on invalid dest variable])
+
+AT_DATA([recode.sps],[dnl
+DATA LIST LIST NOTABLE/x (f1) s (a4) t (a10).
+MISSING VALUES x(9)/s('xxx').
+BEGIN DATA.
+0, '', ''
+1, a, a
+2, ab, ab
+3, abc, abc
+END DATA.
+
+RECODE x (1=9) INTO ".
+EXECUTE.
+dnl "
+])
+
+AT_CHECK([pspp -O format=csv recode.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([RECODE syntax errors])
+AT_DATA([recode.sps], [dnl
+DATA LIST LIST NOTABLE/n1 to n4 (F8.0) s1 (A1) s2 (A2) s3 (A3) s4 (A4).
+RECODE **.
+RECODE n1 **.
+RECODE n1(**).
+RECODE s1(**).
+RECODE s1('x' THRU 'y').
+RECODE n1(1=**).
+RECODE n1(CONVERT).
+RECODE n1(1=2)(1='x').
+RECODE n1(1='x')(1=2).
+RECODE s1(CONVERT)('1'='1').
+RECODE n1(1=2 **).
+RECODE n1(CONVERT).
+RECODE s1(CONVERT) INTO n1 n2.
+RECODE n1(1='1') INTO xyzzy.
+RECODE n1(1='1').
+RECODE s1('1'=1).
+RECODE n1(1='1') INTO n2.
+RECODE s1(CONVERT) INTO s2.
+RECODE n1 TO n4(1='123456') INTO s1 TO s4.
+])
+AT_CHECK([pspp -O format=csv recode.sps], [1], [dnl
+"recode.sps:2.8-2.9: error: RECODE: Syntax error expecting variable name.
+ 2 | RECODE **.
+ | ^~"
+
+"recode.sps:3.11-3.12: error: RECODE: Syntax error expecting `('.
+ 3 | RECODE n1 **.
+ | ^~"
+
+"recode.sps:4.11-4.12: error: RECODE: Syntax error expecting number.
+ 4 | RECODE n1(**).
+ | ^~"
+
+"recode.sps:5.11-5.12: error: RECODE: Syntax error expecting string.
+ 5 | RECODE s1(**).
+ | ^~"
+
+"recode.sps:6.15-6.18: error: RECODE: THRU is not allowed with string variables.
+ 6 | RECODE s1('x' THRU 'y').
+ | ^~~~"
+
+"recode.sps:7.13-7.14: error: RECODE: Syntax error expecting output value.
+ 7 | RECODE n1(1=**).
+ | ^~"
+
+"recode.sps:8.11-8.17: error: RECODE: CONVERT requires string input values.
+ 8 | RECODE n1(CONVERT).
+ | ^~~~~~~"
+
+recode.sps:9: error: RECODE: Output values must be all numeric or all string.
+
+"recode.sps:9.13: note: RECODE: This output value is numeric.
+ 9 | RECODE n1(1=2)(1='x').
+ | ^"
+
+"recode.sps:9.18-9.20: note: RECODE: This output value is string.
+ 9 | RECODE n1(1=2)(1='x').
+ | ^~~"
+
+recode.sps:10: error: RECODE: Output values must be all numeric or all string.
+
+"recode.sps:10.20: note: RECODE: This output value is numeric.
+ 10 | RECODE n1(1='x')(1=2).
+ | ^"
+
+"recode.sps:10.13-10.15: note: RECODE: This output value is string.
+ 10 | RECODE n1(1='x')(1=2).
+ | ^~~"
+
+recode.sps:11: error: RECODE: Output values must be all numeric or all string.
+
+"recode.sps:11.11-11.17: note: RECODE: This output value is numeric.
+ 11 | RECODE s1(CONVERT)('1'='1').
+ | ^~~~~~~"
+
+"recode.sps:11.24-11.26: note: RECODE: This output value is string.
+ 11 | RECODE s1(CONVERT)('1'='1').
+ | ^~~"
+
+"recode.sps:12.15-12.16: error: RECODE: Syntax error expecting `)'.
+ 12 | RECODE n1(1=2 **).
+ | ^~"
+
+"recode.sps:13.11-13.17: error: RECODE: CONVERT requires string input values.
+ 13 | RECODE n1(CONVERT).
+ | ^~~~~~~"
+
+recode.sps:14: error: RECODE: Source and target variable counts must match.
+
+"recode.sps:14.8-14.9: note: RECODE: There is 1 source variable.
+ 14 | RECODE s1(CONVERT) INTO n1 n2.
+ | ^~"
+
+"recode.sps:14.25-14.29: note: RECODE: There are 2 target variables.
+ 14 | RECODE s1(CONVERT) INTO n1 n2.
+ | ^~~~~"
+
+recode.sps:15: error: RECODE: All string variables specified on INTO must already exist. (Use the STRING command to create a string variable.)
+
+"recode.sps:15.23-15.27: note: RECODE: There is no variable named xyzzy.
+ 15 | RECODE n1(1='1') INTO xyzzy.
+ | ^~~~~"
+
+"recode.sps:16.10-16.16: error: RECODE: INTO is required with numeric input values and string output values.
+ 16 | RECODE n1(1='1').
+ | ^~~~~~~"
+
+"recode.sps:17.10-17.16: error: RECODE: INTO is required with string input values and numeric output values.
+ 17 | RECODE s1('1'=1).
+ | ^~~~~~~"
+
+"recode.sps:18.23-18.24: error: RECODE: Type mismatch: cannot store string data in numeric variable n2.
+ 18 | RECODE n1(1='1') INTO n2.
+ | ^~"
+
+"recode.sps:19.25-19.26: error: RECODE: Type mismatch: cannot store numeric data in string variable s2.
+ 19 | RECODE s1(CONVERT) INTO s2.
+ | ^~"
+
+recode.sps:20: error: RECODE: At least one target variable is too narrow for the output values.
+
+"recode.sps:20.19-20.26: note: RECODE: This recoding output value has width 6.
+ 20 | RECODE n1 TO n4(1='123456') INTO s1 TO s4.
+ | ^~~~~~~~"
+
+"recode.sps:20.29-20.41: note: RECODE: Target variable s1 only has width 1.
+ 20 | RECODE n1 TO n4(1='123456') INTO s1 TO s4.
+ | ^~~~~~~~~~~~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([LINEAR REGRESSION])
+
+AT_SETUP([LINEAR REGRESSION - basic])
+AT_DATA([regression.sps], [dnl
+set format = F22.3.
+data list notable list / v0 to v2.
+filter by v0.
+begin data
+ 0.65377128 7.735648 -23.97588
+-0.13087553 6.142625 -19.63854
+ 0.34880368 7.651430 -25.26557
+ 0.69249021 6.125125 -16.57090
+-0.07368178 8.245789 -25.80001
+-0.34404919 6.031540 -17.56743
+ 0.75981559 9.832291 -28.35977
+-0.46958313 5.343832 -16.79548
+-0.06108490 8.838262 -29.25689
+ 0.56154863 6.200189 -18.58219
+end data
+regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=pred resid.
+list.
+])
+
+AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
+"regression.sps:16.82-16.96: warning: REGRESSION: REGRESSION with SAVE ignores FILTER. All cases will be processed.
+ 16 | regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=pred resid.
+ | ^~~~~~~~~~~~~~~"
+
+Table: Model Summary (v2)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.971,.942,.925,1.337
+
+Table: ANOVA (v2)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,202.753,2,101.376,56.754,.000
+Residual,12.504,7,1.786,,
+Total,215.256,9,,,
+
+Table: Coefficients (v2)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),2.191,2.357,.000,.930,.380
+v0,1.813,1.053,.171,1.722,.129
+v1,-3.427,.332,-1.026,-10.334,.000
+
+Table: Data List
+v0,v1,v2,RES1,PRED1
+.654,7.736,-23.976,-.84,-23.13
+-.131,6.143,-19.639,-.54,-19.10
+.349,7.651,-25.266,-1.87,-23.40
+.692,6.125,-16.571,.97,-17.54
+-.074,8.246,-25.800,.40,-26.20
+-.344,6.032,-17.567,1.53,-19.10
+.760,9.832,-28.360,1.77,-30.13
+-.470,5.344,-16.795,.18,-16.97
+-.061,8.838,-29.257,-1.05,-28.21
+.562,6.200,-18.582,-.54,-18.04
+])
+AT_CLEANUP
+
+
+AT_SETUP([LINEAR REGRESSION - one save])
+AT_DATA([regression.sps], [dnl
+set format = F22.3.
+data list notable list / v0 to v2.
+begin data
+ 0.65377128 7.735648 -23.97588
+-0.13087553 6.142625 -19.63854
+ 0.34880368 7.651430 -25.26557
+ 0.69249021 6.125125 -16.57090
+-0.07368178 8.245789 -25.80001
+-0.34404919 6.031540 -17.56743
+ 0.75981559 9.832291 -28.35977
+-0.46958313 5.343832 -16.79548
+-0.06108490 8.838262 -29.25689
+ 0.56154863 6.200189 -18.58219
+end data
+regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=resid.
+regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=pred.
+list.
+])
+
+AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
+Table: Model Summary (v2)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.971,.942,.925,1.337
+
+Table: ANOVA (v2)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,202.753,2,101.376,56.754,.000
+Residual,12.504,7,1.786,,
+Total,215.256,9,,,
+
+Table: Coefficients (v2)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),2.191,2.357,.000,.930,.380
+v0,1.813,1.053,.171,1.722,.129
+v1,-3.427,.332,-1.026,-10.334,.000
+
+Table: Model Summary (v2)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.971,.942,.925,1.337
+
+Table: ANOVA (v2)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,202.753,2,101.376,56.754,.000
+Residual,12.504,7,1.786,,
+Total,215.256,9,,,
+
+Table: Coefficients (v2)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),2.191,2.357,.000,.930,.380
+v0,1.813,1.053,.171,1.722,.129
+v1,-3.427,.332,-1.026,-10.334,.000
+
+Table: Data List
+v0,v1,v2,RES1,PRED1
+.654,7.736,-23.976,-.84,-23.13
+-.131,6.143,-19.639,-.54,-19.10
+.349,7.651,-25.266,-1.87,-23.40
+.692,6.125,-16.571,.97,-17.54
+-.074,8.246,-25.800,.40,-26.20
+-.344,6.032,-17.567,1.53,-19.10
+.760,9.832,-28.360,1.77,-30.13
+-.470,5.344,-16.795,.18,-16.97
+-.061,8.838,-29.257,-1.05,-28.21
+.562,6.200,-18.582,-.54,-18.04
+])
+AT_CLEANUP
+
+
+# Test to ensure that the /SAVE subcommand works properly when SPLIT is active
+AT_SETUP([LINEAR REGRESSION - SAVE vs SPLITS])
+
+# Generate some test data based on a linear model
+AT_DATA([gen-data.sps], [dnl
+set seed = 1.
+input program.
+loop #c = 1 to 20.
+ compute x0 = rv.normal (0,1).
+ compute x1 = rv.normal (0,2).
+ compute err = rv.normal (0,0.1).
+ compute y = 4 - 2 * x0 + 3 * x1 + err.
+ compute g = (#c > 10).
+ end case.
+end loop.
+end file.
+end input program.
+
+print outfile='regdata.txt' /g x0 x1 y err *.
+execute.
+])
+
+AT_CHECK([pspp -O format=csv gen-data.sps], [0], [ignore])
+
+# Use our test data to create a predictor and a residual variable
+# for G == 0
+AT_DATA([regression0.sps], [dnl
+data list notable file='regdata.txt' list /g x0 x1 y err *.
+
+select if (g = 0).
+
+regression
+ /variables = x0 x1
+ /dependent = y
+ /statistics = all
+ /save = pred resid.
+ .
+
+print outfile='outdata-g0.txt' /g x0 x1 y err res1 pred1 *.
+execute.
+])
+
+
+AT_CHECK([pspp -O format=csv regression0.sps], [0], [ignore])
+
+# Use our test data to create a predictor and a residual variable
+# for G == 1
+AT_DATA([regression1.sps], [dnl
+data list notable file='regdata.txt' list /g x0 x1 y err *.
+
+select if (g = 1).
+
+regression
+ /variables = x0 x1
+ /dependent = y
+ /statistics = all
+ /save = pred resid.
+ .
+
+print outfile='outdata-g1.txt' /g x0 x1 y err res1 pred1 *.
+execute.
+])
+
+
+AT_CHECK([pspp -O format=csv regression1.sps], [0], [ignore])
+
+# Use our test data to create a predictor and a residual variable
+# The data is split on G
+AT_DATA([regression-split.sps], [dnl
+data list notable file='regdata.txt' list /g x0 x1 y err *.
+
+split file by g.
+
+regression
+ /variables = x0 x1
+ /dependent = y
+ /statistics = all
+ /save = pred resid.
+ .
+
+print outfile='outdata-split.txt' /g x0 x1 y err res1 pred1 *.
+execute.
+])
+
+AT_CHECK([pspp -O format=csv regression-split.sps], [0], [ignore])
+
+# The concatenation of G==0 and G==1 should be identical to the SPLIT data
+AT_CHECK([cat outdata-g0.txt outdata-g1.txt | diff outdata-split.txt - ], [0], [])
+
+AT_CLEANUP
+
+
+# Test that the procedure behaves sensibly when presented with
+# multiple dependent variables
+AT_SETUP([LINEAR REGRESSION multiple dependent variables])
+AT_DATA([regression.sps], [dnl
+set seed = 2.
+input program.
+loop #c = 1 to 200.
+ compute x0 = rv.normal (0, 1).
+ compute x1 = rv.normal (0, 2).
+ compute err = rv.normal (0, 0.8).
+ compute y = 2 - 1.5 * x0 + 8.4 * x1 + err.
+ compute ycopy = y.
+ end case.
+end loop.
+end file.
+end input program.
+
+regression
+ /variables = x0 x1
+ /dependent = y ycopy
+ /statistics = default.
+])
+
+AT_CHECK([pspp -O format=csv regression.sps > output], [0], [ignore])
+
+AT_CHECK([head -16 output > first], [0], [])
+AT_CHECK([tail -16 output > second], [0], [])
+
+AT_CHECK([sed -e 's/ycopy/y/g' second | diff first -], [0], [])
+
+
+AT_CLEANUP
+
+# Tests the QR decomposition used by the REGRESSION command.
+AT_SETUP([LINEAR REGRESSION test of QR decomposition])
+AT_DATA([regression.sps], [dnl
+data list list / v0 to v1.
+begin data
+-12.84099361 0.873270778
+ 16.64932538 0.371315664
+ -1.88061907 0.505503722
+ -6.20952354 0.734698282
+ 0.33272576 0.891224610
+ -5.54912717 0.052318165
+ 6.11832417 0.448853404
+ 11.78124974 0.470447593
+ 0.75960353 0.565082303
+ 6.06432768 0.149316743
+ -2.64919436 0.752532411
+-10.32250712 0.798263603
+ 2.06355038 0.469129797
+ -9.71851742 0.927162270
+ 4.65582553 0.250629262
+ 9.54574474 0.847032310
+ 7.35544368 0.197028541
+ -2.09609740 0.400584261
+ 10.30101161 0.671546480
+ -5.24501039 0.929962876
+ 1.73412473 0.758161354
+ -3.12732732 0.569785505
+ 12.66261501 0.630640223
+ -2.90956805 0.576067804
+ 4.89649177 0.624483995
+ 13.64613114 0.591089881
+ 14.03198397 0.544587572
+ 2.23566810 0.967898139
+ 5.37367760 0.916246929
+ 9.01346888 0.451702743
+ 0.75378683 0.235544137
+ -3.47470624 0.742668194
+ -1.02063266 0.860311687
+ -2.67132813 0.082460702
+ 23.67661680 0.932553932
+ 7.95061359 0.430161125
+ 2.05300558 0.066331375
+ -2.01332644 0.163705417
+ 20.00663784 0.587292630
+ 3.06099417 0.161411889
+ -3.46115358 0.216684625
+ -6.85287183 0.548714855
+ -4.27923809 0.630997663
+ -0.94863395 0.880612945
+ 4.47481747 0.359885215
+-12.80962955 0.886070341
+ 9.35753086 0.187176558
+ 2.81002235 0.063035095
+ 0.01532424 0.964327101
+ 0.29867732 0.866408063
+ -2.89035649 0.812135868
+ 4.17352811 0.608884061
+ 18.15502183 0.920568258
+ -2.92662792 0.550792959
+ -6.08090449 0.965036595
+ -1.09135397 0.862548019
+ 7.02816784 0.042277017
+-21.20245068 0.430673493
+ -8.83397584 0.724976162
+ -0.89055843 0.017934904
+ 7.03871587 0.308829557
+ 3.84286316 0.685105924
+ 4.50280692 0.447635420
+ 11.39207346 0.875177896
+ 10.86673874 0.518530912
+ 7.09853081 0.588367569
+-12.82864915 0.184667098
+ 13.74888760 0.610891139
+ 0.37379146 0.557720134
+ -9.79020267 0.942839981
+ 0.71574466 0.564570338
+-17.56040637 0.182061777
+ 2.52620466 0.306875011
+ 5.37718673 0.366807049
+ -1.83964300 0.465772898
+ 6.04848363 0.644501799
+ 4.57402403 0.121419591
+ 8.55606848 0.373011464
+ -8.46827907 0.491176571
+ -1.77989798 0.734722847
+ -0.68661121 0.540984182
+ 1.55798880 0.822587656
+ 5.22810831 0.333747878
+ 9.50280477 0.068100934
+ -3.74521465 0.248537644
+ 1.36045068 0.851827791
+ 4.41604088 0.197207162
+ -3.72568327 0.726916693
+ -5.36123334 0.906513529
+ 3.61594583 0.414340595
+-10.01952852 0.140372658
+ 25.48681482 0.354309660
+ -3.34529093 0.090075388
+-18.00437582 0.461438059
+ -5.29782460 0.004362856
+ 2.79608522 0.861294398
+ -1.64076209 0.345775481
+ 6.82802334 0.137933862
+ -0.45416818 0.404379208
+ -1.66868582 0.797685201
+-10.02820292 0.075876582
+ 5.68232031 0.404815042
+ 8.25113850 0.769173748
+ -2.83544237 0.076583474
+ 0.87659945 0.092751009
+ 6.60270870 0.530444351
+-12.63924989 0.362099960
+ -6.24451253 0.641993458
+ 3.53339015 0.461991892
+ -0.74012232 0.437409755
+ 15.37311996 0.974913038
+ -8.09464797 0.543308711
+ -9.61320222 0.221564578
+ 0.21843662 0.856512540
+ -1.56958954 0.610709221
+ 6.44977372 0.200382138
+-13.29136274 0.093222309
+ 6.46257214 0.024135196
+ -3.82727990 0.601335801
+ 0.43081953 0.268230667
+ 19.06654416 0.219972815
+ 17.02906651 0.996849502
+-10.18073139 0.012543080
+ 12.72088788 0.910600764
+ 10.45328185 0.331285901
+ 7.14370922 0.896312020
+ -2.81754334 0.048741266
+ 6.40217095 0.075796756
+ -3.18030478 0.666325307
+ 8.64585957 0.120549153
+ 1.37952764 0.899991932
+-11.81143886 0.601949630
+ 0.03899706 0.363808260
+-10.63828243 0.031092967
+ -6.66940972 0.246204205
+ -5.07374962 0.951272057
+ 4.82281566 0.063928187
+-21.93693564 0.050972680
+ -4.54569883 0.225839693
+ -0.92422779 0.437796785
+ -1.11683029 0.740215139
+ 16.77765554 0.851072372
+ 9.73614597 0.388180586
+ 14.05345168 0.063760129
+ 1.20512012 0.665964184
+ 8.00307080 0.102447114
+ 8.01252623 0.580929209
+-13.54924183 0.438420739
+ 9.87164361 0.970859344
+ 17.63437095 0.250501797
+ -3.42503574 0.873290220
+ -2.45873197 0.847756049
+ 17.29212092 0.411683187
+ 1.15496098 0.530658504
+ -2.14438907 0.592255367
+ -1.79942021 0.517773009
+ -1.30677990 0.830860762
+ 1.70233874 0.291826660
+ -3.05532536 0.801767829
+ -4.06732625 0.092294501
+ 6.34665476 0.270426235
+ 9.46946411 0.196915311
+ 14.50919907 0.480357167
+ 8.93767237 0.778228613
+ 1.90298854 0.903146151
+ 18.50500507 0.598561307
+ 4.45123027 0.555898218
+ 11.37344114 0.616557707
+-12.14693218 0.409187285
+ 18.27198688 0.141619222
+ -5.75939569 0.056989619
+ -4.05515382 0.369281201
+ 16.69882098 0.946885257
+ 6.39050536 0.679704228
+ 4.04213339 0.662792380
+ 6.89608366 0.419877433
+ 1.56496633 0.358227958
+ 5.16679947 0.095144366
+ -3.06280456 0.883265975
+ 2.76279175 0.866571973
+ 1.84969249 0.264869828
+ 21.79840498 0.702650979
+ 1.42450528 0.719308635
+ 0.96797046 0.111937435
+ 18.26840323 0.075621738
+ 13.38288377 0.573399086
+ 2.41101500 0.766238677
+ 3.83866337 0.499888953
+ -1.56577367 0.695244089
+ -0.90342790 0.671654151
+ 10.83775583 0.026041124
+ -9.89767935 0.745297991
+ 11.74840150 0.309144074
+ 1.73069359 0.814063985
+ -5.27966183 0.591005828
+ 3.33030043 0.559401806
+ 1.31427975 0.520950237
+-10.04588558 0.507008362
+ 10.41228345 0.425867272
+ 1.71961097 0.595783108
+-17.54904427 0.328788939
+ -2.23545419 0.223377350
+ -8.68774333 0.980964240
+ -3.48048220 0.008877675
+ -3.69635326 0.090236718
+ 9.76114237 0.769375983
+-10.25662038 0.508137553
+ 0.11155446 0.468504431
+ -8.06824580 0.414098962
+ 3.10031660 0.327130207
+ -3.33393146 0.756896774
+ -3.96276749 0.530956360
+ 14.53610268 0.846474699
+ 1.70505918 0.754662464
+ -1.93495001 0.656650411
+ 5.01974522 0.745337633
+ 13.41249973 0.489362476
+ 11.49288744 0.335924476
+ 12.59019763 0.155560469
+-10.17947298 0.677318449
+ 0.05556115 0.655090105
+ 3.82092860 0.051838719
+ 8.23041456 0.918272190
+ -0.50314649 0.772015826
+ 20.05162157 0.880265258
+ 8.98816884 0.666646668
+ -6.28312120 0.138534416
+ 3.68589909 0.274559458
+ 0.59699510 0.253180863
+ -2.74783135 0.983525221
+ 0.32515065 0.839969577
+ -3.60606166 0.330646732
+ -0.82037740 0.129591173
+ 6.12444860 0.098536516
+ 10.95671074 0.033546728
+ -2.84911174 0.720288722
+ 6.04597572 0.577061422
+ -0.60147150 0.674096868
+ -5.30458364 0.291468008
+ 2.68044943 0.379853840
+ 0.85986585 0.984214339
+-12.77906359 0.882390290
+ 7.21420144 0.550884826
+ 2.31817022 0.231021556
+ 11.60161950 0.888496654
+ -0.19346228 0.242609713
+ 5.07478120 0.759161318
+ 14.54155003 0.040387654
+ 3.81039636 0.874572741
+ 2.23233049 0.448317248
+ 0.19481869 0.201906051
+ 2.81530451 0.132131690
+ 12.39893259 0.674693704
+ 0.47054642 0.632959494
+ 2.16152913 0.734480632
+ 0.33398836 0.315024718
+ 7.35509037 0.304570986
+ -2.92336559 0.539062343
+ 5.79622573 0.392393310
+ -2.37607425 0.403380474
+ 0.04498550 0.756875541
+ -1.63674414 0.613789514
+ 11.80310547 0.832651469
+ 6.30630243 0.850689403
+ 1.48394652 0.096243229
+ 4.03361865 0.799660045
+ 3.54707273 0.408520520
+ 2.00327040 0.702944912
+ 17.30761707 0.380542812
+ 5.72738968 0.105447516
+-13.64604891 0.328506659
+ 8.35976334 0.702173924
+ -7.41197443 0.134396488
+-15.95683040 0.618526462
+ 8.76889573 0.950243069
+ -1.13482624 0.113477080
+ -0.60311407 0.090444247
+ 4.95508365 0.612511543
+ 5.36934491 0.979213258
+ -0.03554882 0.807185690
+-11.58131144 0.183341373
+ 4.46809041 0.796330582
+ 12.49741067 0.346860912
+ 8.63824488 0.073684997
+ 0.49990913 0.732519306
+ 12.82688360 0.109400213
+ 13.20375065 0.850369092
+ -8.41110869 0.177717087
+ 16.31959963 0.727704840
+ 17.59203613 0.235311681
+ 0.32148420 0.842195936
+ 5.43148331 0.670904647
+ 7.14649727 0.028190029
+ 0.25410683 0.421535783
+-12.41047826 0.086404379
+-10.64180909 0.229659236
+ -6.40185653 0.876365242
+ 15.63063324 0.667672536
+ 1.94280423 0.799266628
+ -5.76507450 0.367344192
+ 8.60895533 0.154109357
+ 9.38306751 0.788742770
+ 3.43573528 0.284535277
+ 4.81848966 0.872283177
+ 11.65839314 0.234109111
+ -5.57884822 0.030363060
+ -3.94238060 0.325320686
+ 9.38133340 0.201141788
+ -7.65003459 0.647734396
+ 11.23091019 0.084927159
+ -6.07705432 0.037273791
+ 7.46380750 0.506897136
+ 7.42034855 0.869351148
+ -4.43031973 0.231191152
+ -1.07351537 0.480234836
+ -1.40653281 0.690620421
+ -3.82710168 0.990191328
+ 5.04583490 0.543427375
+-11.54265099 0.270542185
+ 0.49059479 0.991447248
+ -1.40871469 0.555998766
+ 3.64241437 0.743840673
+-18.30031589 0.357478210
+ 4.27487959 0.770619738
+ 1.28805821 0.654787106
+ -3.19542768 0.218110139
+ 12.53375654 0.011857644
+ 11.78889419 0.054127726
+ -5.38392310 0.839309080
+ 16.38024181 0.228801038
+ -0.59622631 0.134381782
+ -0.74107258 0.258146632
+-12.31429450 0.020524447
+ -0.79785028 0.968028764
+ 6.39899711 0.038162566
+ 7.42024044 0.716163692
+ -3.62470664 0.018201813
+ -2.55049724 0.162446610
+-10.79888854 0.683070478
+ 10.18490144 0.546461234
+ -2.76979044 0.198830067
+ 4.85164813 0.094100357
+ 0.96477200 0.381801756
+ 8.13344336 0.639730450
+ 9.04684412 0.786084368
+ 10.41746272 0.828304181
+ 0.94334368 0.798419831
+ 10.13116556 0.191715972
+ -4.12728628 0.575178239
+ -9.59222379 0.876405375
+ 1.64680258 0.391003085
+ -4.58897613 0.039176486
+ 0.38394379 0.511577564
+ -4.80428215 0.222785463
+ 0.35363661 0.681658725
+ -9.63685708 0.183035382
+ 3.54363414 0.766127414
+ 6.89610808 0.967514568
+ -2.03781105 0.464416752
+ 8.67956196 0.421424078
+ -1.09959038 0.061231448
+ 7.12587456 0.028601318
+ -6.93064672 0.402561175
+ 8.57989199 0.925089270
+ -9.55071810 0.454993099
+ -8.11914736 0.509644286
+ -5.41909698 0.077813151
+-17.03336572 0.875713545
+ -1.27438609 0.602163625
+ 3.09834374 0.105599007
+ -1.59865741 0.439939102
+ 11.82272089 0.754984309
+ 4.30969696 0.483834579
+-10.76886192 0.222486992
+ 7.05419803 0.903020271
+ 7.36096847 0.440357053
+ -2.05864869 0.581170147
+ -9.08366913 0.318677911
+ 8.57119930 0.605668919
+ 7.87702340 0.570206991
+ 5.22035786 0.542344385
+ 2.37238850 0.595969470
+ -4.29809941 0.634313781
+ 4.51647479 0.796663089
+ -0.62478780 0.562099444
+ 8.50866078 0.490014249
+ 3.46694991 0.122890089
+ -7.31956453 0.885170890
+ 2.20259268 0.167180856
+ -1.81003626 0.702563515
+ 8.44526939 0.973495019
+ 8.19767069 0.881261264
+ -5.92422578 0.686557351
+ -0.11826129 0.712798344
+ 5.66132869 0.922826429
+ -5.40845018 0.642183516
+ 6.67839036 0.680978989
+ 11.88962825 0.487904896
+ 3.32266332 0.931709581
+ 0.24234019 0.405641313
+-12.79023339 0.361005489
+-13.57875491 0.266289733
+ 1.81304596 0.775093821
+ 0.36755600 0.400225605
+ -9.15574205 0.518040748
+ -3.90436548 0.396869908
+ 9.24764042 0.669374848
+ 0.74869385 0.609881390
+ -3.62958907 0.928867495
+ -0.02527232 0.557679930
+ 0.04433418 0.152565816
+ 11.76152632 0.865663501
+ -4.62181124 0.007000650
+ 5.82271403 0.389678502
+ 0.33289002 0.532940826
+ -7.65647076 0.681574524
+ 11.81023732 0.107165912
+ 11.42121999 0.989580324
+ -5.47120641 0.762285550
+ 3.82311561 0.388755074
+ 16.91059711 0.461236022
+ 4.14012105 0.802420151
+ -1.35278659 0.036646959
+-12.81733350 0.179096148
+ -0.94395134 0.450959878
+ -5.39002376 0.264783062
+ 4.16227017 0.780743762
+ 7.26179625 0.821382454
+ 15.10062276 0.469253936
+ 1.45877225 0.685434405
+ -9.87966760 0.767201511
+ 7.03156071 0.195142483
+-11.71327419 0.774014869
+ -4.55518706 0.973103604
+ -1.75221406 0.175172193
+ 10.35631400 0.080670414
+ 4.97650495 0.597478189
+ 2.25703939 0.585949751
+ 10.72500409 0.339720931
+ -5.02901029 0.997874377
+ 6.24398637 0.655067479
+ -5.83880059 0.184948259
+ 2.17256077 0.746741866
+ -5.59809380 0.277523381
+ 8.19384177 0.334565607
+ 3.35250431 0.952057263
+ 16.20874892 0.901400446
+ 1.63205839 0.235388475
+ -1.07921163 0.608376332
+ 0.24315118 0.862639830
+ 15.61923078 0.050955422
+ 1.99639207 0.358905687
+ 8.14825538 0.190069662
+ 4.55210835 0.784025901
+ 13.51582298 0.973572910
+ 15.42415796 0.969080992
+ 2.23978124 0.551857514
+ 1.00858991 0.919566804
+ -2.77293574 0.906998180
+ 7.10750420 0.934792213
+ -8.01377290 0.682306063
+ 9.67873875 0.239576806
+ 7.54867950 0.065860266
+ 13.85701962 0.733823443
+ 8.48212853 0.285731085
+ 3.55278843 0.998255904
+ 21.94592206 0.205463912
+ -2.07957143 0.948665109
+ 1.54169997 0.200109744
+-11.36934275 0.447122472
+ 3.07094572 0.815147945
+ 6.45818709 0.007849948
+ 1.85594578 0.818796540
+ -2.43799564 0.962013689
+-17.96539549 0.654190963
+ -0.93433746 0.454930236
+-11.06904368 0.898560975
+ 14.89733742 0.479152492
+ -5.72390675 0.136197255
+ 9.46781102 0.669006610
+ 5.35954546 0.259381138
+ 3.78388994 0.933778797
+ 1.95373423 0.517555994
+ 10.96772341 0.666138826
+ 9.40585102 0.779906833
+ 0.75347502 0.142656741
+ 7.64803672 0.734297119
+ -0.40051164 0.362230232
+ 10.00747057 0.660820381
+-12.86024975 0.072988046
+ 1.43515528 0.229672223
+ -6.75981709 0.658534537
+ -5.61355474 0.795897133
+ -4.40596595 0.038787666
+ -1.37033650 0.371835229
+ 6.66666573 0.560963737
+ 8.18430044 0.284787698
+ -0.55742330 0.622783662
+ -0.39757686 0.673551753
+-12.68628005 0.373038561
+ 4.06416215 0.760546238
+ 4.65859855 0.516761886
+ 3.55304076 0.266856843
+ -7.35294817 0.615783196
+ 1.01222898 0.158266779
+ 9.91052610 0.285619547
+ -6.42966726 0.573689954
+ 10.97425098 0.985095061
+ 5.79394599 0.404333309
+ 10.09106608 0.441037857
+ -1.47295537 0.577661077
+ -2.07959719 0.547176133
+ -8.76910940 0.498979558
+-11.15658312 0.135862745
+ -0.88456783 0.326480064
+ 9.71607440 0.998076370
+ -8.76072622 0.386244511
+-19.26823092 0.461833959
+ -0.11280313 0.064155908
+ 0.64625887 0.172078148
+ -5.35323428 0.331153163
+ -1.71034509 0.330955888
+ 4.27104744 0.590544244
+ 7.33843789 0.263171531
+ -5.38121637 0.539675802
+ -6.87566548 0.127313096
+ -2.50161298 0.269417630
+ 10.99076986 0.097362729
+ 6.34017269 0.318528587
+ -4.63672382 0.451038055
+-11.55122495 0.987073278
+ 4.78618612 0.297342215
+ 2.97547390 0.197312152
+ -5.54495280 0.499701114
+ 17.67606173 0.810316588
+ 16.01578815 0.643667608
+ -0.16258467 0.228284761
+ 7.92123340 0.784289369
+ -2.26303900 0.270764770
+ 5.84136933 0.437763291
+ 4.96955217 0.389720490
+ -8.09516710 0.829068548
+ 14.59207207 0.513593803
+ 2.80954688 0.650799867
+ 4.53653552 0.672326278
+ 4.49116737 0.807447691
+ 18.87549709 0.647303378
+ 9.80118464 0.932576117
+-13.02124969 0.038651904
+ -6.15189291 0.697593318
+ 15.81920283 0.249825051
+ 10.81503188 0.152372300
+-23.58738366 0.593560367
+ 8.15716338 0.411680007
+ 3.45349379 0.351061414
+ -6.39345334 0.374926213
+ 8.72924585 0.165759028
+ 22.17948804 0.003736780
+ -4.73053410 0.582425257
+ 16.88289626 0.484899167
+ -1.78826142 0.663273340
+ -0.78106025 0.337039969
+ -2.92461669 0.810174719
+-13.89224399 0.177428986
+ 4.56809819 0.025010350
+ -1.07452825 0.649632933
+ 0.58148751 0.829606422
+ 12.13329525 0.354819526
+ 17.35605568 0.284862590
+-12.43678107 0.827661083
+ -1.89492796 0.574929572
+ 2.18520382 0.846299917
+ 18.11449649 0.603173531
+ 4.34508582 0.484049042
+ 17.49394569 0.094811656
+ 10.67752350 0.166176400
+ 17.13374502 0.547208197
+ 4.42138123 0.768691494
+ 5.38445574 0.788597361
+ 0.79946671 0.851883720
+ -4.67547904 0.995621191
+ -5.61496422 0.523793593
+-20.52093126 0.881207308
+ -8.95996814 0.851078124
+ -7.63483710 0.739657373
+ 11.02131097 0.678060014
+-10.56228517 0.202393048
+ 6.48841788 0.143946271
+ 3.44853632 0.913249620
+ -0.02080024 0.070765134
+ 2.08654297 0.032468089
+ 8.13415912 0.439470874
+ 11.19028936 0.944954026
+ 0.26670866 0.492724593
+ -9.33692734 0.982611921
+ 17.23967092 0.313428994
+ 0.36906670 0.660669528
+ 7.89735684 0.977628886
+ -4.00171487 0.379327632
+ 5.01615432 0.735627296
+ 0.42214214 0.092461754
+ 13.60634772 0.218359635
+ 6.57431413 0.067653525
+ -1.77668341 0.717799276
+ 5.16227422 0.325502093
+-15.29091550 0.332815338
+ 3.33602480 0.594168551
+ 13.80131443 0.817724470
+ 5.92111679 0.947854666
+ 3.59747624 0.330860216
+ -6.79722403 0.093518715
+ -1.86606213 0.824179728
+ 17.05226458 0.466573672
+ 10.39712467 0.409067778
+ -4.78536386 0.891470739
+ 11.92963128 0.719633060
+ -1.44230992 0.232628002
+-12.31860616 0.834134222
+ 2.93439660 0.957842480
+ 14.27963295 0.546264646
+ 2.17488820 0.701170328
+ 10.78772417 0.612332448
+ -0.47049341 0.378564293
+ -0.35140634 0.034337429
+ 5.04887868 0.211697132
+ -3.51562580 0.663243607
+ -0.82013387 0.602497174
+ 2.78954743 0.325294790
+ 8.67905777 0.820296625
+-12.70343389 0.315467361
+ -2.59373236 0.015571904
+ -4.60369241 0.293737716
+ 1.58669084 0.671091860
+-10.44245103 0.501340276
+ 4.85215578 0.141572007
+ 10.46303284 0.801814632
+ 1.27898298 0.236929983
+ -1.72225479 0.608500539
+ 20.18685735 0.827124630
+ 3.27308817 0.542065179
+ 1.01596956 0.254672115
+ -8.88872881 0.460876757
+-11.31397349 0.636168639
+ 0.85294367 0.816417328
+ 3.54262337 0.944147626
+-10.53603202 0.675775741
+ 4.34832198 0.121988381
+ 11.56451662 0.283063133
+ -7.36454369 0.500596540
+ -8.23701113 0.379483261
+ -8.36081323 0.219730782
+ -6.39158860 0.739171315
+ -1.40518544 0.478709398
+ -4.01314821 0.460476388
+ -7.34814047 0.406242873
+ -7.80836711 0.730648091
+ -0.57729135 0.152336258
+ 4.98352832 0.026424939
+ -3.78181635 0.453598432
+ 20.16821827 0.845273124
+ 5.20758271 0.573569671
+ -3.05534245 0.286828574
+ -5.31306254 0.961990401
+ 1.09307567 0.006478724
+ -3.75412572 0.598277695
+ -2.38444245 0.777900122
+ 2.46837742 0.280363751
+ 9.72195519 0.041094463
+ 3.96271247 0.604775284
+ 2.14105354 0.400315328
+ 7.88645912 0.404573389
+ -4.03565076 0.798377309
+ 10.80180959 0.932152434
+-10.89359212 0.446813857
+ 1.43144578 0.310194540
+ 4.79825196 0.504826858
+ 10.73201365 0.384306369
+ -4.07526187 0.893893643
+ -2.84330198 0.390202663
+ 5.81825057 0.830581384
+ -2.77842745 0.382966910
+ -7.70333673 0.157692966
+ -3.25753058 0.726303603
+ 8.50032387 0.556524444
+ 2.35027236 0.857076526
+ -1.70740565 0.194760923
+ -3.40693880 0.696420946
+ -8.03983352 0.514393263
+ -1.85105344 0.609459979
+ -9.01148029 0.526019631
+ 18.37344635 0.690793045
+ 16.46079416 0.811535334
+ 4.10224315 0.043403618
+ 7.06657672 0.831274577
+ 15.31421824 0.434558881
+-12.36760970 0.004215634
+ 1.95473415 0.277788662
+ -0.93207006 0.368433415
+ 15.39919341 0.843189783
+ 5.23452387 0.626226925
+ 11.40805770 0.002417288
+ -1.30282837 0.072493756
+ 3.92130690 0.675355182
+ 2.53148399 0.027222295
+ 4.92705318 0.934429364
+ 5.54978818 0.042268708
+ -2.19608977 0.246743834
+ -0.62565550 0.858214200
+ -8.98329365 0.646827226
+ 12.78468146 0.533966352
+ 2.01061290 0.418710227
+ 1.03689579 0.019241741
+ 8.01166696 0.992268130
+ -4.49786437 0.694127903
+ -8.15387184 0.066275002
+ 2.22256207 0.083301613
+-12.27145086 0.535369809
+ 9.95709112 0.227692557
+ 14.58198717 0.667298058
+ 5.98046083 0.922503625
+ 1.25640725 0.632933575
+ 9.77623752 0.136171032
+ 5.57068426 0.374916651
+-10.07048336 0.470411379
+ 3.69267954 0.897278365
+ 2.22185354 0.212539549
+ 7.96155623 0.720525208
+ -6.21272358 0.771491819
+ 2.63054735 0.474989115
+ -2.81488890 0.675381020
+ 4.52747191 0.118615879
+ -3.22975936 0.783991133
+ 11.42834761 0.423344824
+ 0.26512464 0.617515445
+ -5.84322807 0.210915613
+ 9.61073028 0.988117333
+ -6.11878012 0.492318959
+ 5.30581443 0.339379499
+ -6.40132703 0.903540026
+ 1.22921808 0.122161655
+ 8.08547837 0.197296588
+ -0.77943801 0.935963718
+ 11.43194858 0.828270943
+ -5.41689395 0.556863468
+ 15.14667847 0.565186375
+ -5.15327419 0.542802437
+ -3.95903082 0.643379366
+ 5.78847793 0.391369361
+ 11.54430873 0.158789330
+ 1.90340148 0.841316129
+ 14.69680285 0.532022770
+ 0.68552840 0.367073827
+ -8.72287967 0.250127491
+ 9.35401445 0.836083158
+ 5.32139524 0.996712598
+-14.53387897 0.825434481
+ -2.93925146 0.513153861
+ 12.54386493 0.713306793
+ 2.04842442 0.993893406
+ 2.87461954 0.049843312
+ 4.89765230 0.376710062
+ -6.23945314 0.321108142
+ -3.45840168 0.854710947
+ 9.05807160 0.199992188
+ 3.33815006 0.787302467
+ 4.22244242 0.351841910
+ 15.75879160 0.268699469
+ 2.78549859 0.920299974
+ -4.46643118 0.727283862
+ 0.48021298 0.428672083
+ 2.55814938 0.130915212
+ 5.00692968 0.062266047
+ 4.78801127 0.325124688
+ 6.39524485 0.693406744
+-10.46792584 0.458128441
+ 10.14111908 0.353412759
+-10.56424183 0.821588957
+ 7.60967746 0.267669137
+ -2.34956688 0.434855697
+ 23.82269027 0.802311880
+ 8.37170447 0.445185000
+ 10.05024769 0.778687843
+ -9.15753018 0.957292819
+ 12.17438228 0.774769426
+ 1.57960028 0.783591989
+ 0.06719501 0.849073924
+ 16.21114558 0.243444943
+ -3.79808298 0.842994720
+ 8.98927715 0.020537113
+ 7.72362992 0.984168340
+ 11.25158442 0.152385348
+-21.23936903 0.909204114
+ 7.34995949 0.987249305
+ -7.99435203 0.335456401
+ -2.78218185 0.768517548
+ 11.59547596 0.466617637
+ 15.90870706 0.071892573
+ 5.58160897 0.554485703
+ 16.05253351 0.815206562
+ -3.23103465 0.280495460
+ -4.61108636 0.035757819
+ 5.41596511 0.746146856
+ 2.92445613 0.136743821
+ 11.23628254 0.681316365
+-12.93714705 0.838791576
+ 9.94668264 0.084457395
+ -4.56061529 0.983605894
+ -4.24795688 0.601732731
+ -2.83740044 0.375102341
+ -0.43078317 0.403870303
+ 15.19689584 0.114826374
+-10.29920266 0.731582141
+ 6.01686515 0.641655876
+ 6.69431335 0.496723697
+ 4.62223602 0.328118236
+ -1.74309026 0.072604771
+ 14.31971261 0.827101483
+ -1.86629155 0.613346722
+ 8.30971428 0.274948560
+ 8.50080711 0.059822908
+ -7.94061422 0.121069240
+ -2.72096492 0.710791774
+ 3.33259421 0.398621625
+ 1.73248470 0.488581205
+ 9.56008489 0.011104565
+ 12.71499762 0.038568985
+ 4.11512127 0.219846314
+ -0.96707584 0.822646857
+ 4.98621667 0.633779997
+ 4.69384821 0.295708955
+ 10.16008645 0.778287787
+ -7.72973800 0.097096969
+ 2.87264210 0.796538177
+ 4.56095440 0.862952770
+ 5.02621658 0.934628629
+ 3.18138681 0.805600816
+ -1.02245780 0.317640678
+ 18.16001652 0.992503640
+ 4.13729026 0.941910149
+ 1.61211303 0.377271914
+ 1.71520009 0.735196094
+ 3.26325421 0.514432564
+ 12.94663819 0.591190711
+ 10.53239931 0.005877708
+ 8.06705056 0.340779884
+ -5.09007267 0.332516161
+ 12.31973355 0.323119296
+ -2.69957650 0.633232996
+ 12.51207803 0.377641090
+ 8.02081444 0.859293157
+ -0.13098726 0.099370804
+ -0.97757546 0.852873609
+ 16.73605399 0.595854575
+ 3.63219184 0.329310613
+ -4.79105630 0.247760146
+ -4.77209495 0.708235587
+ 0.92107647 0.924567254
+ 12.12724271 0.433550712
+ -5.07731478 0.200109463
+ 9.16019579 0.897456586
+ 18.33260560 0.649877409
+ 1.93596773 0.584401505
+ 8.51254631 0.283154523
+ 11.41092928 0.698703314
+ 10.85035748 0.351078210
+ 12.62749979 0.570101319
+ -2.32028296 0.313842122
+ -2.45778301 0.007943144
+ 6.93102526 0.108737491
+ -0.67304654 0.245399613
+ 9.27294774 0.204010286
+ 14.29292826 0.396294626
+ 14.05843185 0.864613328
+ -3.73515954 0.305862948
+ 0.36606339 0.116802407
+ -5.79235478 0.457308058
+ 8.70346900 0.858244380
+ -8.91321043 0.077001581
+ 0.58499566 0.503209780
+ 0.39160153 0.324883353
+ 7.46715326 0.343451039
+ 12.36256009 0.679483638
+ 8.84283689 0.687359177
+ -6.39396909 0.113065562
+ -3.67844896 0.667335667
+ 9.36904962 0.009815419
+ -3.25244888 0.213105120
+ 19.09389976 0.593130536
+ 7.28826611 0.829483570
+ -5.44565944 0.956490203
+ 7.96993416 0.770961635
+ 0.20683778 0.006497153
+ -3.73273760 0.037042812
+-10.64745846 0.813594448
+ 5.70578906 0.157678242
+ 4.05282218 0.224663656
+ 14.77711159 0.577586777
+ 0.89685942 0.297213941
+ 3.92600687 0.672347849
+-12.29347477 0.367072171
+ -9.33603480 0.456544225
+ -0.86683190 0.088696811
+ 4.65685745 0.779783359
+ 1.24438030 0.712958633
+ 11.43533814 0.920345548
+-10.18380242 0.044456697
+ -1.20684029 0.992051648
+ -9.78059038 0.611477837
+ 3.05588762 0.581933667
+ 3.47419279 0.769325101
+ 0.87528245 0.455214184
+ -3.13185655 0.805887381
+ -0.82283965 0.707668384
+ -1.86717272 0.984060013
+ 16.56357048 0.217369677
+ -2.11052646 0.474156371
+ -1.39795364 0.958554209
+ 4.87468692 0.328779186
+ 2.69163553 0.401633221
+ 6.08640626 0.599963560
+ 7.41420081 0.240202007
+ 5.73729928 0.696034193
+ 7.31747120 0.569520861
+ -6.20465547 0.214005920
+ 17.52477873 0.667125450
+ 12.97855692 0.796977778
+ -3.35883428 0.379721403
+ -2.90306972 0.552454626
+ 5.31617371 0.401625473
+ -3.86414389 0.830986352
+-14.94107832 0.702705123
+ -5.74060402 0.833328045
+ -8.10116203 0.078855027
+ 23.48247017 0.568666620
+ 20.22005082 0.357069809
+ -2.53387193 0.637455425
+ 15.72048831 0.845354124
+ -4.41494567 0.934471473
+ -8.02254420 0.378467959
+ -0.13398716 0.489382793
+ 0.95967155 0.813667919
+ 0.14835664 0.215786848
+ 14.31875579 0.675145039
+ -6.36589196 0.822037848
+ 8.25942906 0.156787526
+ -7.33597529 0.051076292
+ 12.58936771 0.666507807
+ 2.34653798 0.626196518
+ -0.69351398 0.050664564
+ 7.08738260 0.808776877
+ 5.19653521 0.779008623
+ 3.20900427 0.197212774
+ 7.81171331 0.744975548
+ 6.49008186 0.991318119
+ 7.27471854 0.839642650
+ -7.68367290 0.880500743
+ 12.04846713 0.797754890
+ 14.93435279 0.190527791
+ -3.83641079 0.075995951
+ 2.15090497 0.426560973
+ -3.61166623 0.777188818
+ 8.49333248 0.891445999
+ -7.46936100 0.148607446
+ 13.85406193 0.983656455
+ 12.20477754 0.499345090
+ 10.09415710 0.638127733
+ 5.37134772 0.110929011
+ 8.17660840 0.879411588
+ -4.38804367 0.608933700
+-11.78145902 0.265134740
+ 6.18940186 0.970982743
+-16.24831477 0.844983635
+ 9.52790402 0.578152651
+ 16.44372225 0.264144422
+ -2.48286428 0.893865621
+ 5.33297280 0.512990215
+ -2.68912507 0.851636020
+ 9.94607707 0.644483197
+ -1.93526852 0.550759844
+ 2.34310539 0.787853650
+ 11.79131608 0.983668283
+ 3.16689104 0.605394987
+ 10.47759320 0.919442774
+ 2.86973133 0.557835916
+ 10.30674302 0.442504870
+ 10.92820575 0.976183635
+ 7.98050212 0.139334994
+ -0.64719705 0.981199028
+ -2.63625596 0.341524563
+ 11.38799583 0.858987904
+ 1.37321916 0.202373294
+ 12.66698520 0.142127091
+ 5.83599540 0.864497670
+ 4.88659560 0.472598564
+ 13.00108599 0.961629827
+ 5.79514791 0.408377170
+ -1.47807631 0.536772872
+ -3.38142805 0.288956265
+ 6.25154986 0.828695103
+ 2.40919373 0.478123848
+ 3.72990486 0.056539500
+ -9.90915815 0.603356617
+ 0.21737084 0.737251896
+ 5.36929388 0.026920178
+ -1.05027354 0.034992509
+ -4.97887624 0.506301429
+ -6.40058435 0.014061876
+ -0.14610837 0.619699963
+ 3.78483619 0.653952701
+ 3.84143365 0.162122572
+ 2.66030676 0.196542503
+-10.56809462 0.386200215
+ -5.01140125 0.711703654
+ -3.09809005 0.118120179
+ -2.76110171 0.118809515
+ 2.85825107 0.129646974
+ 2.75993661 0.171779333
+ 11.55931169 0.372165133
+ 9.21211486 0.969079819
+ 6.02207148 0.498965865
+ -3.52883224 0.954619249
+ -2.60190803 0.069405278
+ 1.34183694 0.569402487
+-11.35155228 0.766344735
+ 1.04661568 0.023673810
+ -1.90461932 0.179728300
+ 13.72465582 0.467775796
+ 19.14882438 0.476924297
+ -1.07480326 0.944407858
+ -8.44289331 0.059804028
+ 1.89732882 0.743225795
+ -7.87832463 0.672539050
+-12.24163608 0.916803014
+-12.77212790 0.648129714
+ 6.39197262 0.622954436
+ 5.26261666 0.494421400
+-10.65239640 0.695527931
+ 4.63841458 0.499519163
+ -2.94276544 0.429201572
+ 4.68788953 0.639613685
+ -1.03031400 0.349342009
+ -2.69946354 0.221796918
+-15.32237714 0.631289988
+ -8.31962698 0.925363812
+ -5.80897714 0.833536878
+ 7.16070989 0.832098478
+-10.99679727 0.794048223
+ 0.84514458 0.748014415
+ 2.23308495 0.111176288
+ 3.56351018 0.599805508
+ 0.88336430 0.746908710
+-14.63461670 0.314391808
+ 4.39039715 0.079604833
+ -7.07001439 0.633705345
+ 2.11252583 0.461468123
+ 7.60219364 0.497389476
+ -4.87713428 0.039952736
+ 2.17515292 0.421830084
+ 0.64302362 0.267982804
+-22.29371533 0.646257366
+ 0.31652779 0.060548371
+ 7.93445046 0.343570449
+ 0.28292029 0.651909785
+ 2.77775640 0.637679287
+ 6.22941586 0.291132945
+ 23.68567532 0.708513840
+ 9.49503014 0.645200206
+ 0.87405420 0.063154289
+ -4.04931224 0.110797498
+ 8.91607239 0.732917195
+ -5.77728018 0.635435595
+-16.37296319 0.343727613
+ 9.87409940 0.774177478
+ -8.11360210 0.377765616
+ 14.54242540 0.204343527
+ 0.36239636 0.115528352
+ 19.51009176 0.181365423
+ 1.23592729 0.011676577
+-15.81877035 0.767155028
+ -0.05911251 0.737944231
+ -6.55395965 0.214062137
+ -7.85591487 0.539865054
+ -9.73010882 0.730924287
+ 11.79433862 0.267116856
+ -8.84308360 0.088069165
+ -5.56689174 0.405987947
+ 7.59010135 0.655631611
+ 10.07629305 0.031106157
+ -6.19331485 0.052350502
+ -4.58626710 0.326901540
+ -5.19431549 0.125740555
+ -2.08129025 0.034657174
+-10.48798034 0.153632237
+ 13.04657686 0.317295703
+ 1.94142067 0.731437668
+ -1.62470735 0.701070475
+-12.27046912 0.505781742
+ -2.96095228 0.122808075
+ -2.91847765 0.372668438
+-14.83230131 0.100749725
+ 16.57350659 0.707854947
+ 10.05473238 0.244046174
+ 5.50858969 0.070691273
+ 7.65309196 0.245393047
+ 7.16359996 0.056261015
+ 4.33026356 0.318855549
+ -4.65721575 0.271249938
+ 2.85909691 0.309377566
+ 3.02736080 0.553944209
+ 6.22796768 0.763945813
+ -4.47036396 0.197721195
+ 2.78901176 0.441166128
+ -9.94574794 0.964660659
+ 1.86451969 0.704635530
+-10.38926659 0.772304221
+ -5.36565800 0.029527218
+ 1.99230152 0.578448308
+ 13.65547415 0.936050102
+ -2.05229879 0.851521142
+ 0.99504588 0.974891334
+ -1.46027404 0.320227281
+ -8.45614275 0.727910071
+ -0.95201934 0.199101032
+ -2.46642929 0.462252060
+ -6.44060430 0.703637604
+ -2.58115910 0.084948525
+ 0.76248197 0.125769097
+ 12.00603845 0.675927328
+ 1.97538215 0.782502470
+ 2.23331320 0.870228155
+ -3.10226060 0.485056198
+ 12.59337170 0.584729095
+ -2.42247402 0.387588168
+ 9.41981063 0.374604221
+ 6.26806243 0.727453335
+ -5.30630356 0.427294265
+ 13.81542647 0.394246994
+ 1.05647858 0.646684666
+-12.25005208 0.010531726
+ -4.58162076 0.077133994
+ 0.58094190 0.400275636
+ 5.79443858 0.641731247
+ 8.87635216 0.913593476
+ 9.71048520 0.955285711
+ 13.10563373 0.908471848
+ 4.99194220 0.967014095
+ 15.88178853 0.041518216
+ 9.35962068 0.864770023
+ -7.53095731 0.300106124
+ 12.18427585 0.248876997
+ 9.22034502 0.450149366
+ -1.02861237 0.684246939
+ -2.98140404 0.326901490
+ -4.64316598 0.425381055
+ 15.35233259 0.630774937
+ -1.85655250 0.226889991
+ 15.43748330 0.584219351
+ 10.39060893 0.387854461
+ 2.80705696 0.564024865
+ 3.48201221 0.787103673
+ 7.03787977 0.112019552
+ 8.41853061 0.798376796
+ 15.63925527 0.873984550
+ -4.05742183 0.699131238
+ 6.56954685 0.720018710
+ 2.44007265 0.232697343
+ 3.75597926 0.975133449
+ 2.92362149 0.290975435
+ -4.74372257 0.003738451
+ 1.28365940 0.987536495
+ 15.65288265 0.179629701
+-11.76385004 0.850614822
+ 1.56331228 0.592017435
+ -9.64774741 0.024951969
+ -9.44879860 0.993960270
+ 29.33340056 0.913358233
+ 7.97233120 0.021820585
+-12.10837953 0.401535846
+ -1.20729618 0.984977268
+ 3.63219301 0.491142613
+ 2.79853507 0.663823888
+ 3.19584583 0.612511282
+ -0.81790885 0.908769330
+ -1.67795944 0.611690031
+ 2.55137163 0.109447998
+ 4.36572889 0.382049700
+ -6.35667866 0.162787163
+ -0.76239101 0.892383562
+ -3.99558996 0.466572017
+ -0.47513018 0.457760464
+-10.69568261 0.544872910
+ 4.30943512 0.982456072
+ 2.91825703 0.823403368
+ 0.10753188 0.945676881
+ -8.38623073 0.923085521
+ 4.95690232 0.188128654
+ 5.39956649 0.331692462
+ -1.47421789 0.327711090
+ -1.81689665 0.713285385
+ 5.15137860 0.414906436
+ 5.68897151 0.110799415
+ 0.78825159 0.396824099
+ -1.78376652 0.929264595
+ 0.76991060 0.950124414
+ 15.81469073 0.951245195
+ -4.33820920 0.009896093
+ 1.67174323 0.821983745
+ 0.38997945 0.928857784
+ 1.97848484 0.175680230
+ -5.81067801 0.772580245
+-10.45208478 0.418845035
+ 13.34024524 0.905645046
+ -8.79585122 0.906516178
+ 2.89093397 0.113010960
+ 2.22324289 0.799940482
+ 8.95497981 0.984663669
+ 0.93288527 0.277914575
+-17.35306978 0.455587022
+ -3.26914604 0.406757639
+ 8.75871227 0.067059659
+ 1.79914932 0.784879863
+ -0.67305388 0.006393497
+ 1.66805704 0.039614073
+ 9.03868439 0.601066847
+ 4.29458670 0.015772820
+ -8.15564320 0.939633197
+ 12.50538902 0.766347628
+ -0.45547258 0.464314122
+ -9.47180656 0.640114882
+-13.25567198 0.125841930
+ 2.87660101 0.381931128
+ 7.37834152 0.648958712
+ -0.45874073 0.303139498
+ 4.87941450 0.500090729
+ 4.50344891 0.311329309
+ -6.14257896 0.061368838
+ 5.98243271 0.873804882
+ -2.64694079 0.080493398
+-17.79727796 0.188420116
+-13.52552336 0.798403568
+ 2.29086373 0.518700767
+ 5.21493652 0.788828533
+ -8.09641615 0.775041349
+ 5.87005782 0.079447757
+ 10.74795720 0.955691540
+ -8.01115709 0.004508053
+ -7.53735064 0.054195934
+ -6.79130165 0.877193354
+ -1.26419539 0.837772170
+ 8.31082852 0.967509866
+ 21.83090247 0.261529880
+ 11.20453234 0.913858875
+ 7.19128396 0.541942489
+ -2.93623595 0.860095891
+ -3.61446403 0.022418065
+ -6.59997709 0.532998307
+ 3.71486934 0.522669434
+ 18.03420874 0.295064126
+ -8.75452291 0.390175021
+ -7.83680812 0.263760724
+ -1.10263921 0.501819826
+ 2.05633484 0.338684642
+ 5.25636848 0.558667384
+ -7.33260497 0.457327559
+ 3.86721296 0.612182242
+-15.94331373 0.478329365
+ 3.71501899 0.264241588
+ 18.26175822 0.023212661
+ -5.21093378 0.184378036
+ -2.44074986 0.297114134
+-11.88339919 0.875956945
+ 7.52127093 0.927322099
+ 5.31597834 0.416344968
+ 11.42012314 0.952491078
+ 13.64950955 0.794183413
+ 12.50861255 0.390723282
+ -3.48142207 0.538708662
+ 14.32910902 0.085221990
+ 5.76196699 0.313860477
+ 2.63751452 0.917424732
+ 2.99975231 0.208662214
+ 7.09852825 0.798246964
+ -1.95742636 0.352166210
+ 7.80534904 0.386523123
+ -4.47229047 0.290188493
+ -4.35535158 0.761527294
+ 1.47083860 0.447897289
+ 3.09504296 0.048513534
+ 3.50446804 0.925072429
+ 12.00487617 0.574499971
+ 10.35171466 0.934193962
+ -5.63256003 0.968833982
+ 7.15625220 0.467160468
+ -7.81378393 0.790220187
+ 4.52101003 0.014459969
+ 12.90773453 0.990835752
+ 7.70737851 0.785329264
+ -3.37196794 0.066025357
+ -5.12793918 0.347459322
+ -7.96083724 0.216608294
+ 12.81247279 0.287880308
+ 4.63872463 0.426881173
+ -3.85439309 0.336532356
+ 4.55633320 0.108001536
+ -2.40824634 0.135247519
+ 1.65932541 0.005108006
+ 3.26129578 0.093163961
+ -3.52114597 0.544041275
+ -9.08479260 0.111212700
+ 7.75150456 0.942726234
+ 7.44829768 0.396996218
+ 14.44430576 0.525470762
+ -2.13457508 0.207577358
+ 9.74871681 0.537177845
+ -4.53338693 0.625854028
+ 16.15962824 0.947933141
+ -0.17711664 0.480940902
+ -7.21470818 0.006952612
+ -6.27644212 0.737909602
+ -0.81648165 0.230003567
+ -2.10429152 0.209671398
+ 7.69291241 0.987903443
+ -0.32284504 0.183904658
+ -0.90833921 0.782169770
+ 10.35542238 0.201758865
+ 10.40788689 0.540802365
+ 10.80011849 0.298263948
+ 7.39943598 0.785716539
+-12.71674257 0.154135834
+ 16.67139866 0.116794235
+ 12.47832985 0.998179468
+ 3.24041348 0.653080096
+ -5.50381593 0.995396942
+-10.41952811 0.576472768
+ 4.33514092 0.146434686
+ 4.41294276 0.507165968
+ -0.14746982 0.698144836
+ -1.33323051 0.466481571
+ -7.01201350 0.797150114
+ 6.58669848 0.942287809
+ 7.19444974 0.053569397
+ -5.66046997 0.435728340
+ -5.07828702 0.497727572
+ 13.72045272 0.324222944
+ 9.99111984 0.355713969
+ -4.42363728 0.071790181
+ -2.34300923 0.618434528
+ 4.98594041 0.605667438
+ -2.45307721 0.894546647
+ -3.52276424 0.760086779
+ -3.69489441 0.960758209
+ 13.04792817 0.511273320
+ 21.61433486 0.236270637
+ -9.57303968 0.964235539
+ 11.70400744 0.391045695
+ 4.25170422 0.411577090
+ 7.87516537 0.952858161
+ 9.89202673 0.971106687
+ 7.51554467 0.505791978
+ 2.17944879 0.893835908
+ 0.82420351 0.213912155
+ 2.47121932 0.688019842
+-14.88503628 0.640950883
+ 8.16032283 0.277742858
+ -4.65776244 0.129415853
+ 7.48274838 0.074213153
+ 7.70537066 0.476778957
+ 5.88202944 0.351838898
+ 1.95618325 0.106331699
+ 7.22064623 0.511434587
+-18.64632081 0.009314188
+ -6.16794611 0.526204245
+ 2.14042033 0.675800465
+ -1.89535048 0.916845690
+ -7.77156605 0.069742819
+ -6.84078801 0.865082345
+ 8.17539904 0.095895629
+ 10.75170309 0.821383078
+ 14.31498805 0.117893208
+ -2.82264467 0.809086411
+ 0.11117380 0.400587471
+ 6.43898314 0.333163663
+ 9.48110784 0.465173316
+ 5.39395511 0.695273081
+ -2.05636570 0.508060862
+ 0.68666117 0.647109494
+ -6.41880322 0.267530762
+ -0.12096589 0.986901165
+ 6.46062643 0.588580914
+ -4.20926136 0.550783675
+ 4.07354300 0.907963701
+ 10.84045143 0.900920521
+ 2.64504664 0.767700269
+ 10.34578229 0.197810342
+ -0.19222273 0.281932395
+ 3.47400952 0.977555902
+ 11.04549389 0.694010579
+ 6.79729856 0.056652433
+ 9.28300628 0.598930531
+ -3.53453282 0.183412212
+ 8.04028248 0.250746943
+ 5.75964045 0.424692336
+ -4.98252741 0.867446071
+ 12.00352175 0.289615080
+ 7.53497791 0.350915526
+ 2.54579776 0.655113837
+ -9.29572208 0.900136667
+ 6.41659249 0.100570650
+ 7.37095646 0.907179211
+-12.78417775 0.262214556
+ 0.87962107 0.624657444
+ -5.96939907 0.296725805
+ -2.56857339 0.604065931
+ 4.27131811 0.962952479
+ 21.72603838 0.442485270
+ 6.10056565 0.418383130
+ 10.48099521 0.333593221
+ 19.28363092 0.382408442
+ 2.12080726 0.601206970
+ -6.82450704 0.740158518
+ 11.32395692 0.627015570
+ 5.00040701 0.476274658
+-11.64750733 0.105099095
+ 5.77442654 0.576560214
+ 0.31340364 0.516479036
+ -2.09881449 0.146089191
+ 5.12411327 0.368130477
+ 1.70530391 0.621828438
+-12.95649749 0.355726301
+ 8.43735652 0.275383759
+-15.56161079 0.413160084
+ 5.28942694 0.069125495
+ 5.96040877 0.438716686
+ -2.59318107 0.571116303
+ 6.95988992 0.650760909
+ 14.00074797 0.623645969
+ 1.66101456 0.558763985
+ -2.57968349 0.648185379
+ -5.47584253 0.716901151
+ 6.37222581 0.060563130
+ 2.83664864 0.842419730
+ 1.48926558 0.620280308
+ 0.33471689 0.170312461
+ 5.21648412 0.317639631
+ 0.51733642 0.843867329
+ 9.86005834 0.306036746
+ -5.81145791 0.975655452
+ -5.43219061 0.303385368
+ 5.87157118 0.677369776
+ 2.08889926 0.310200439
+ -2.53433085 0.194730908
+ 7.01359575 0.674259533
+ -2.00936260 0.682056466
+ -2.98240739 0.787899917
+ -7.43289210 0.357483044
+-12.58905988 0.981387385
+ 5.78095517 0.533526274
+ -1.23065889 0.687266774
+ -6.82309960 0.293249774
+ 8.47000829 0.842056399
+ -5.81624772 0.303700280
+-14.83571031 0.311387926
+ 4.66808472 0.091222946
+ -2.90144463 0.438301785
+ 10.62458662 0.828335698
+ 7.88002491 0.990156110
+ 10.27680283 0.251087079
+ -9.42498970 0.292462244
+ 6.73027640 0.213065205
+ 1.28169895 0.353152789
+-14.29203733 0.264563048
+ 20.35772711 0.265208837
+ 3.55095071 0.242905653
+-17.97067670 0.373951756
+ 10.53141139 0.247520698
+ 0.05293205 0.579940423
+ 12.79674707 0.288031751
+ -5.44235185 0.075899079
+ 14.29464811 0.960707538
+ -1.36753291 0.124265178
+ -4.25946974 0.521720352
+-12.46519252 0.385503339
+ -6.65343143 0.540942219
+ 5.55949184 0.143194404
+ -1.20480594 0.515905644
+ -4.13839908 0.164461445
+ -2.21345425 0.812969725
+ 3.94223380 0.229238952
+-10.78661097 0.395049514
+ 3.06997341 0.791234255
+ 24.82205477 0.110859039
+ 6.28791249 0.867125744
+ -2.80296119 0.703583849
+ 13.24274039 0.425951975
+ -0.19577471 0.361568727
+ -2.34894781 0.954814545
+ 19.76339577 0.635462177
+ -1.87591480 0.149121567
+ -7.70962391 0.711708342
+ -2.46291902 0.390902746
+end data
+regression /variables=v0 v1 /statistics defaults /dependent=v0 /method=enter.
+])
+
+AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+v0,F8.0
+v1,F8.0
+
+Table: Model Summary (v0)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.05,.00,.00,8.11
+
+Table: ANOVA (v0)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,235.23,1,235.23,3.58,.059
+Residual,98438.40,1498,65.71,,
+Total,98673.63,1499,,,
+
+Table: Coefficients (v0)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),1.24,.42,.00,2.95,.003
+v1,1.37,.72,.05,1.89,.059
+])
+
+AT_CLEANUP
+
+AT_SETUP([LINEAR REGRESSION no crash on all missing])
+AT_DATA([regcrash.sps], [dnl
+data list list /x * y.
+begin data.
+ . .
+ . .
+ . .
+ . .
+ . .
+ . .
+ . .
+ . .
+ . .
+ . .
+end data.
+
+
+regression /variables=x y /dependent=y.
+])
+
+AT_CHECK([pspp -o pspp.csv regcrash.sps], [1], [ignore], [ignore])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([LINEAR REGRESSION missing dependent variable])
+
+dnl Test for a bug where missing values in the dependent variable were not being
+dnl ignored like they should have been.
+AT_DATA([reg-mdv-ref.sps], [dnl
+data list notable list / v0 to v2.
+begin data
+ 0.65377128 7.735648 -23.97588
+-0.13087553 6.142625 -19.63854
+ 0.34880368 7.651430 -25.26557
+ 0.69249021 6.125125 -16.57090
+-0.07368178 8.245789 -25.80001
+-0.34404919 6.031540 -17.56743
+ 0.75981559 9.832291 -28.35977
+-0.46958313 5.343832 -16.79548
+-0.06108490 8.838262 -29.25689
+ 0.56154863 6.200189 -18.58219
+end data
+regression /variables=v0 v1
+ /statistics defaults
+ /dependent=v2
+ /method=enter.
+])
+
+AT_CHECK([pspp -o pspp-ref.csv reg-mdv-ref.sps])
+
+AT_DATA([reg-mdv.sps], [dnl
+data list notable list / v0 to v2.
+begin data
+ 0.65377128 7.735648 -23.97588
+-0.13087553 6.142625 -19.63854
+ 0.34880368 7.651430 -25.26557
+ 0.69249021 6.125125 -16.57090
+-0.07368178 8.245789 -25.80001
+-0.34404919 6.031540 -17.56743
+ 0.75981559 9.832291 -28.35977
+-0.46958313 5.343832 -16.79548
+-0.06108490 8.838262 -29.25689
+ 0.56154863 6.200189 -18.58219
+ 0.5 8 9
+end data
+
+missing values v2 (9).
+
+regression /variables=v0 v1
+ /statistics defaults
+ /dependent=v2
+ /method=enter.
+])
+
+AT_CHECK([pspp -o pspp.csv reg-mdv.sps])
+
+AT_CHECK([diff pspp.csv pspp-ref.csv])
+
+
+AT_CLEANUP
+
+AT_SETUP([LINEAR REGRESSION with invalid syntax (and empty dataset)])
+
+AT_DATA([ss.sps], [dnl
+data list notable list / v0 to v2.
+begin data
+end data.
+
+regression /variables=v0 v1
+ /statistics r coeff anova
+ /dependent=v2
+ /method=enter v2.
+])
+
+AT_CHECK([pspp ss.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+dnl The following example comes from
+dnl http://www.ats.ucla.edu/stat/spss/output/reg_spss%28long%29.htm
+AT_SETUP([LINEAR REGRESSION coefficient confidence interval])
+
+AT_DATA([conf.sps], [dnl
+set format = F22.3.
+
+data list notable list /math female socst read science *
+begin data.
+ 41.00 .00 57.00 57.00 47.00
+ 53.00 1.00 61.00 68.00 63.00
+ 54.00 .00 31.00 44.00 58.00
+ 47.00 .00 56.00 63.00 53.00
+ 57.00 .00 61.00 47.00 53.00
+ 51.00 .00 61.00 44.00 63.00
+ 42.00 .00 61.00 50.00 53.00
+ 45.00 .00 36.00 34.00 39.00
+ 54.00 .00 51.00 63.00 58.00
+ 52.00 .00 51.00 57.00 50.00
+ 51.00 .00 61.00 60.00 53.00
+ 51.00 .00 61.00 57.00 63.00
+ 71.00 .00 71.00 73.00 61.00
+ 57.00 .00 46.00 54.00 55.00
+ 50.00 .00 56.00 45.00 31.00
+ 43.00 .00 56.00 42.00 50.00
+ 51.00 .00 56.00 47.00 50.00
+ 60.00 .00 56.00 57.00 58.00
+ 62.00 .00 61.00 68.00 55.00
+ 57.00 .00 46.00 55.00 53.00
+ 35.00 .00 41.00 63.00 66.00
+ 75.00 .00 66.00 63.00 72.00
+ 45.00 .00 56.00 50.00 55.00
+ 57.00 .00 61.00 60.00 61.00
+ 45.00 .00 46.00 37.00 39.00
+ 46.00 .00 31.00 34.00 39.00
+ 66.00 .00 66.00 65.00 61.00
+ 57.00 .00 46.00 47.00 58.00
+ 49.00 .00 46.00 44.00 39.00
+ 49.00 .00 41.00 52.00 55.00
+ 57.00 .00 51.00 42.00 47.00
+ 64.00 .00 61.00 76.00 64.00
+ 63.00 .00 71.00 65.00 66.00
+ 57.00 .00 31.00 42.00 72.00
+ 50.00 .00 61.00 52.00 61.00
+ 58.00 .00 66.00 60.00 61.00
+ 75.00 .00 66.00 68.00 66.00
+ 68.00 .00 66.00 65.00 66.00
+ 44.00 .00 36.00 47.00 36.00
+ 40.00 .00 51.00 39.00 39.00
+ 41.00 .00 51.00 47.00 42.00
+ 62.00 .00 51.00 55.00 58.00
+ 57.00 .00 51.00 52.00 55.00
+ 43.00 .00 41.00 42.00 50.00
+ 48.00 .00 66.00 65.00 63.00
+ 63.00 .00 46.00 55.00 69.00
+ 39.00 .00 47.00 50.00 49.00
+ 70.00 .00 51.00 65.00 63.00
+ 63.00 .00 46.00 47.00 53.00
+ 59.00 .00 51.00 57.00 47.00
+ 61.00 .00 56.00 53.00 57.00
+ 38.00 .00 41.00 39.00 47.00
+ 61.00 .00 46.00 44.00 50.00
+ 49.00 .00 71.00 63.00 55.00
+ 73.00 .00 66.00 73.00 69.00
+ 44.00 .00 42.00 39.00 26.00
+ 42.00 .00 32.00 37.00 33.00
+ 39.00 .00 46.00 42.00 56.00
+ 55.00 .00 41.00 63.00 58.00
+ 52.00 .00 51.00 48.00 44.00
+ 45.00 .00 61.00 50.00 58.00
+ 61.00 .00 66.00 47.00 69.00
+ 39.00 .00 46.00 44.00 34.00
+ 41.00 .00 36.00 34.00 36.00
+ 50.00 .00 61.00 50.00 36.00
+ 40.00 .00 26.00 44.00 50.00
+ 60.00 .00 66.00 60.00 55.00
+ 47.00 .00 26.00 47.00 42.00
+ 59.00 .00 44.00 63.00 65.00
+ 49.00 .00 36.00 50.00 44.00
+ 46.00 .00 51.00 44.00 39.00
+ 58.00 .00 61.00 60.00 58.00
+ 71.00 .00 66.00 73.00 63.00
+ 58.00 .00 66.00 68.00 74.00
+ 46.00 .00 51.00 55.00 58.00
+ 43.00 .00 31.00 47.00 45.00
+ 54.00 .00 61.00 55.00 49.00
+ 56.00 .00 66.00 68.00 63.00
+ 46.00 .00 46.00 31.00 39.00
+ 54.00 .00 56.00 47.00 42.00
+ 57.00 .00 56.00 63.00 55.00
+ 54.00 .00 36.00 36.00 61.00
+ 71.00 .00 56.00 68.00 66.00
+ 48.00 .00 56.00 63.00 63.00
+ 40.00 .00 41.00 55.00 44.00
+ 64.00 .00 66.00 55.00 63.00
+ 51.00 .00 56.00 52.00 53.00
+ 39.00 .00 56.00 34.00 42.00
+ 40.00 .00 31.00 50.00 34.00
+ 61.00 .00 56.00 55.00 61.00
+ 66.00 .00 46.00 52.00 47.00
+ 49.00 .00 46.00 63.00 66.00
+ 65.00 1.00 61.00 68.00 69.00
+ 52.00 1.00 48.00 39.00 44.00
+ 46.00 1.00 51.00 44.00 47.00
+ 61.00 1.00 51.00 50.00 63.00
+ 72.00 1.00 56.00 71.00 66.00
+ 71.00 1.00 71.00 63.00 69.00
+ 40.00 1.00 41.00 34.00 39.00
+ 69.00 1.00 61.00 63.00 61.00
+ 64.00 1.00 66.00 68.00 69.00
+ 56.00 1.00 61.00 47.00 66.00
+ 49.00 1.00 41.00 47.00 33.00
+ 54.00 1.00 51.00 63.00 50.00
+ 53.00 1.00 51.00 52.00 61.00
+ 66.00 1.00 56.00 55.00 42.00
+ 67.00 1.00 56.00 60.00 50.00
+ 40.00 1.00 33.00 35.00 51.00
+ 46.00 1.00 56.00 47.00 50.00
+ 69.00 1.00 71.00 71.00 58.00
+ 40.00 1.00 56.00 57.00 61.00
+ 41.00 1.00 51.00 44.00 39.00
+ 57.00 1.00 66.00 65.00 46.00
+ 58.00 1.00 56.00 68.00 59.00
+ 57.00 1.00 66.00 73.00 55.00
+ 37.00 1.00 41.00 36.00 42.00
+ 55.00 1.00 46.00 43.00 55.00
+ 62.00 1.00 66.00 73.00 58.00
+ 64.00 1.00 56.00 52.00 58.00
+ 40.00 1.00 51.00 41.00 39.00
+ 50.00 1.00 51.00 60.00 50.00
+ 46.00 1.00 56.00 50.00 50.00
+ 53.00 1.00 56.00 50.00 39.00
+ 52.00 1.00 46.00 47.00 48.00
+ 45.00 1.00 46.00 47.00 34.00
+ 56.00 1.00 61.00 55.00 58.00
+ 45.00 1.00 56.00 50.00 44.00
+ 54.00 1.00 41.00 39.00 50.00
+ 56.00 1.00 46.00 50.00 47.00
+ 41.00 1.00 26.00 34.00 29.00
+ 54.00 1.00 56.00 57.00 50.00
+ 72.00 1.00 56.00 57.00 54.00
+ 56.00 1.00 51.00 68.00 50.00
+ 47.00 1.00 46.00 42.00 47.00
+ 49.00 1.00 66.00 61.00 44.00
+ 60.00 1.00 66.00 76.00 67.00
+ 54.00 1.00 46.00 47.00 58.00
+ 55.00 1.00 56.00 46.00 44.00
+ 33.00 1.00 41.00 39.00 42.00
+ 49.00 1.00 61.00 52.00 44.00
+ 43.00 1.00 51.00 28.00 44.00
+ 50.00 1.00 52.00 42.00 50.00
+ 52.00 1.00 51.00 47.00 39.00
+ 48.00 1.00 41.00 47.00 44.00
+ 58.00 1.00 66.00 52.00 53.00
+ 43.00 1.00 61.00 47.00 48.00
+ 41.00 1.00 31.00 50.00 55.00
+ 43.00 1.00 51.00 44.00 44.00
+ 46.00 1.00 41.00 47.00 40.00
+ 44.00 1.00 41.00 45.00 34.00
+ 43.00 1.00 46.00 47.00 42.00
+ 61.00 1.00 56.00 65.00 58.00
+ 40.00 1.00 51.00 43.00 50.00
+ 49.00 1.00 61.00 47.00 53.00
+ 56.00 1.00 66.00 57.00 58.00
+ 61.00 1.00 71.00 68.00 55.00
+ 50.00 1.00 61.00 52.00 54.00
+ 51.00 1.00 61.00 42.00 47.00
+ 42.00 1.00 41.00 42.00 42.00
+ 67.00 1.00 66.00 66.00 61.00
+ 53.00 1.00 61.00 47.00 53.00
+ 50.00 1.00 58.00 57.00 51.00
+ 51.00 1.00 31.00 47.00 63.00
+ 72.00 1.00 61.00 57.00 61.00
+ 48.00 1.00 61.00 52.00 55.00
+ 40.00 1.00 31.00 44.00 40.00
+ 53.00 1.00 61.00 50.00 61.00
+ 39.00 1.00 36.00 39.00 47.00
+ 63.00 1.00 41.00 57.00 55.00
+ 51.00 1.00 37.00 57.00 53.00
+ 45.00 1.00 43.00 42.00 50.00
+ 39.00 1.00 61.00 47.00 47.00
+ 42.00 1.00 39.00 42.00 31.00
+ 62.00 1.00 51.00 60.00 61.00
+ 44.00 1.00 51.00 44.00 35.00
+ 65.00 1.00 66.00 63.00 54.00
+ 63.00 1.00 71.00 65.00 55.00
+ 54.00 1.00 41.00 39.00 53.00
+ 45.00 1.00 36.00 50.00 58.00
+ 60.00 1.00 51.00 52.00 56.00
+ 49.00 1.00 51.00 60.00 50.00
+ 48.00 1.00 51.00 44.00 39.00
+ 57.00 1.00 61.00 52.00 63.00
+ 55.00 1.00 61.00 55.00 50.00
+ 66.00 1.00 56.00 50.00 66.00
+ 64.00 1.00 71.00 65.00 58.00
+ 55.00 1.00 51.00 52.00 53.00
+ 42.00 1.00 36.00 47.00 42.00
+ 56.00 1.00 61.00 63.00 55.00
+ 53.00 1.00 66.00 50.00 53.00
+ 41.00 1.00 41.00 42.00 42.00
+ 42.00 1.00 41.00 36.00 50.00
+ 53.00 1.00 56.00 50.00 55.00
+ 42.00 1.00 51.00 41.00 34.00
+ 60.00 1.00 56.00 47.00 50.00
+ 52.00 1.00 56.00 55.00 42.00
+ 38.00 1.00 46.00 42.00 36.00
+ 57.00 1.00 52.00 57.00 55.00
+ 58.00 1.00 61.00 55.00 58.00
+ 65.00 1.00 61.00 63.00 53.00
+end data.
+
+regression
+ /variables = math female socst read
+ /statistics = coeff r anova ci (95)
+ /dependent = science
+ /method = enter
+])
+
+AT_CHECK([pspp -O format=csv conf.sps], [0], [dnl
+Table: Model Summary (science)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.699,.489,.479,7.148
+
+Table: ANOVA (science)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,9543.721,4,2385.930,46.695,.000
+Residual,9963.779,195,51.096,,
+Total,19507.500,199,,,
+
+Table: Coefficients (science)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.,95% Confidence Interval for B,
+,B,Std. Error,Beta,,,Lower Bound,Upper Bound
+(Constant),12.325,3.194,.000,3.859,.000,6.027,18.624
+math,.389,.074,.368,5.252,.000,.243,.535
+female,-2.010,1.023,-.101,-1.965,.051,-4.027,.007
+socst,.050,.062,.054,.801,.424,-.073,.173
+read,.335,.073,.347,4.607,.000,.192,.479
+])
+
+
+AT_CLEANUP
+
+
+dnl Checks for regression against bug #44877.
+AT_SETUP([LINEAR REGRESSION crash with long string variables])
+AT_DATA([regression.sps], [dnl
+SET DECIMAL=DOT.
+
+DATA LIST notable LIST /text (A24) Y * X1 *
+BEGIN DATA.
+V00276601 0.00 90.00
+V00292909 10.00 30.00
+V00291204 20.00 20.00
+V00300070 0.00 90.00
+END DATA.
+
+REGRESSION
+/VARIABLES= Y
+/DEPENDENT= X1
+/METHOD=ENTER
+/STATISTICS=COEFF R ANOVA
+/SAVE= RESID.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv regression.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Model Summary (X1)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.95,.89,.84,15.08
+
+Table: ANOVA (X1)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,3820.45,1,3820.45,16.81,.055
+Residual,454.55,2,227.27,,
+Total,4275.00,3,,,
+
+Table: Coefficients (X1)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),85.45,10.16,.00,8.41,.004
+Y,-3.73,.91,-.95,-4.10,.055
+
+Table: Data List
+text,Y,X1,RES1
+V00276601,.00,90.00,4.55
+V00292909,10.00,30.00,-18.18
+V00291204,20.00,20.00,9.09
+V00300070,.00,90.00,4.55
+])
+AT_CLEANUP
+
+
+dnl Test for a crash which happened on bad input syntax
+AT_SETUP([LINEAR REGRESSION -- Empty Parentheses])
+
+AT_DATA([empty-parens.sps], [dnl
+set format = F22.3.
+
+data list notable list /math female socst read science *
+begin data.
+ 58.00 1.00 61.00 55.00 58.00
+ 65.00 1.00 61.00 63.00 53.00
+end data.
+
+regression
+ /variables = math female socst read
+ /statistics = coeff r anova ci ()
+ /dependent = science
+ /method = enter
+])
+
+AT_CHECK([pspp -o pspp.csv empty-parens.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+
+
+AT_SETUP([LINEAR REGRESSION varibles on ENTER subcommand])
+AT_DATA([regression.sps], [dnl
+SET FORMAT=F10.3.
+
+DATA LIST notable LIST /number * value *.
+BEGIN DATA
+ 16 7.25
+ 0 .00
+ 1 .10
+ 9 27.9
+ 0 .00
+ 7 3.65
+ 14 16.8
+ 24 9.15
+ 0 .00
+ 24 19.0
+ 7 4.05
+ 12 7.90
+ 6 .75
+ 11 1.40
+ 0 .00
+ 3 2.30
+ 12 7.60
+ 11 6.80
+ 16 8.65
+END DATA.
+
+REGRESSION
+ /STATISTICS COEFF R ANOVA
+ /DEPENDENT value
+ /METHOD=ENTER number.
+])
+
+
+AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
+Table: Model Summary (value)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.612,.374,.338,6.176
+
+Table: ANOVA (value)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,388.065,1,388.065,10.173,.005
+Residual,648.498,17,38.147,,
+Total,1036.563,18,,,
+
+Table: Coefficients (value)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),.927,2.247,.000,.413,.685
+number,.611,.192,.612,3.189,.005
+])
+
+AT_CLEANUP
+
+
+
+AT_SETUP([LINEAR REGRESSION /ORIGIN])
+AT_DATA([regression-origin.sps], [dnl
+SET FORMAT=F10.3.
+
+DATA LIST notable LIST /number * value *.
+BEGIN DATA
+ 16 7.25
+ 0 .00
+ 1 .10
+ 9 27.9
+ 0 .00
+ 7 3.65
+ 14 16.8
+ 24 9.15
+ 0 .00
+ 24 19.0
+ 7 4.05
+ 12 7.90
+ 6 .75
+ 11 1.40
+ 0 .00
+ 3 2.30
+ 12 7.60
+ 11 6.80
+ 16 8.65
+END DATA.
+
+REGRESSION
+ /STATISTICS COEFF R ANOVA
+ /DEPENDENT value
+ /ORIGIN
+ /METHOD=ENTER number.
+])
+
+
+AT_CHECK([pspp -O format=csv regression-origin.sps], [0], [dnl
+Table: Model Summary (value)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.802,.643,.622,6.032
+
+Table: ANOVA (value)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,1181.726,1,1181.726,32.475,.000
+Residual,654.989,18,36.388,,
+Total,1836.715,19,,,
+
+Table: Coefficients (value)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+number,.672,.118,.802,5.699,.000
+])
+
+AT_CLEANUP
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([REGRESSION tutorial example])
+cp $top_srcdir/examples/repairs.sav .
+AT_DATA([regression.sps], [dnl
+GET FILE='repairs.sav'.
+REGRESSION /VARIABLES=mtbf duty_cycle /DEPENDENT=mttr.
+REGRESSION /VARIABLES=mtbf /DEPENDENT=mttr.
+])
+
+AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
+Table: Model Summary (Mean time to repair (hours) )
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.94,.89,.88,6.54
+
+Table: ANOVA (Mean time to repair (hours) )
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,9576.26,2,4788.13,111.94,.000
+Residual,1154.94,27,42.78,,
+Total,10731.20,29,,,
+
+Table: Coefficients (Mean time to repair (hours) )
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),10.59,3.11,.00,3.40,.002
+Mean time between failures (months) ,3.02,.20,.95,14.88,.000
+Ratio of working to non-working time,-1.12,3.69,-.02,-.30,.763
+
+Table: Model Summary (Mean time to repair (hours) )
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.94,.89,.89,6.43
+
+Table: ANOVA (Mean time to repair (hours) )
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,9572.30,1,9572.30,231.28,.000
+Residual,1158.90,28,41.39,,
+Total,10731.20,29,,,
+
+Table: Coefficients (Mean time to repair (hours) )
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
+,B,Std. Error,Beta,,
+(Constant),9.90,2.10,.00,4.71,.000
+Mean time between failures (months) ,3.01,.20,.94,15.21,.000
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([LINEAR REGRESSION vif])
+AT_DATA([regression-vif.sps], [dnl
+SET FORMAT=F10.3.
+
+data list notable list /competence_x1 motivation_x2 performance_y.
+begin data
+32 34 36
+35 37 39
+38 45 49
+31 41 41
+36 40 38
+32 38 36
+33 39 37
+31 40 41
+30 37 40
+35 37 43
+31 34 36
+34 32 35
+31 42 34
+25 36 40
+35 42 40
+36 41 44
+30 38 32
+34 41 41
+34 41 44
+22 27 26
+27 26 33
+30 30 35
+30 35 37
+37 39 44
+29 35 36
+31 35 29
+31 45 41
+29 30 32
+29 35 36
+31 37 37
+36 45 42
+32 44 39
+27 26 31
+33 39 35
+20 25 28
+30 36 39
+27 37 39
+25 39 36
+32 38 38
+32 38 35
+end data.
+
+regression /variables=competence_x1 motivation_x2
+ /statistics=defaults tol
+ /dependent=performance_y
+ .
+])
+
+
+AT_CHECK([pspp -O format=csv regression-vif.sps], [0], [dnl
+Table: Model Summary (performance_y)
+R,R Square,Adjusted R Square,Std. Error of the Estimate
+.785,.616,.595,2.980
+
+Table: ANOVA (performance_y)
+,Sum of Squares,df,Mean Square,F,Sig.
+Regression,526.494,2,263.247,29.641,.000
+Residual,328.606,37,8.881,,
+Total,855.100,39,,,
+
+Table: Coefficients (performance_y)
+,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.,Collinearity Statistics,
+,B,Std. Error,Beta,,,Tolerance,VIF
+(Constant),7.220,4.020,.000,1.796,.080,,
+competence_x1,.432,.166,.358,2.609,.013,.552,1.812
+motivation_x2,.453,.125,.499,3.636,.001,.552,1.812
+])
+
+AT_CLEANUP
+
+AT_SETUP([REGRESSION syntax errors])
+AT_DATA([regression.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+REGRESSION VARIABLES=**.
+REGRESSION METHOD=ENTER x/VARIABLES.
+REGRESSION DEPENDENT=x/VARIABLES.
+REGRESSION DEPENDENT=**.
+REGRESSION METHOD=**.
+REGRESSION METHOD=ENTER **.
+REGRESSION STATISTICS=**.
+REGRESSION STATISTICS=CI(**).
+REGRESSION STATISTICS=CI(1 **).
+REGRESSION SAVE=**.
+REGRESSION **.
+])
+AT_CHECK([pspp -O format=csv regression.sps], [1], [dnl
+"regression.sps:2.22-2.23: error: REGRESSION: Syntax error expecting variable name.
+ 2 | REGRESSION VARIABLES=**.
+ | ^~"
+
+"regression.sps:3.27-3.35: error: REGRESSION: VARIABLES may not appear after METHOD.
+ 3 | REGRESSION METHOD=ENTER x/VARIABLES.
+ | ^~~~~~~~~"
+
+"regression.sps:4.24-4.32: error: REGRESSION: VARIABLES may not appear after DEPENDENT.
+ 4 | REGRESSION DEPENDENT=x/VARIABLES.
+ | ^~~~~~~~~"
+
+"regression.sps:5.22-5.23: error: REGRESSION: Syntax error expecting variable name.
+ 5 | REGRESSION DEPENDENT=**.
+ | ^~"
+
+"regression.sps:6.19-6.20: error: REGRESSION: Syntax error expecting ENTER.
+ 6 | REGRESSION METHOD=**.
+ | ^~"
+
+"regression.sps:7.25-7.26: error: REGRESSION: Syntax error expecting variable name.
+ 7 | REGRESSION METHOD=ENTER **.
+ | ^~"
+
+"regression.sps:8.23-8.24: error: REGRESSION: Syntax error expecting ALL, DEFAULTS, R, COEFF, ANOVA, BCOV, TOL, or CI.
+ 8 | REGRESSION STATISTICS=**.
+ | ^~"
+
+"regression.sps:9.26-9.27: error: REGRESSION: Syntax error expecting number.
+ 9 | REGRESSION STATISTICS=CI(**).
+ | ^~"
+
+"regression.sps:10.28-10.29: error: REGRESSION: Syntax error expecting `@:}@'.
+ 10 | REGRESSION STATISTICS=CI(1 **).
+ | ^~"
+
+"regression.sps:11.17-11.18: error: REGRESSION: Syntax error expecting PRED or RESID.
+ 11 | REGRESSION SAVE=**.
+ | ^~"
+
+"regression.sps:12.12-12.13: error: REGRESSION: Syntax error expecting VARIABLES, DEPENDENT, ORIGIN, NOORIGIN, METHOD, STATISTICS, or SAVE.
+ 12 | REGRESSION **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([RELIABILITY])
+
+AT_SETUP([RELIABILITY])
+AT_DATA([reliability.sps], [dnl
+data list notable list /var1 *
+ var2 *
+ var6 *
+ var7 *
+ var8 *
+ var9 *
+ var11 *
+ var12 *
+ var15 *
+ var16 *
+ var17 *
+ var19 *
+ .
+
+begin data.
+6 7 7 5 7 7 7 7 7 7 6 6
+6 7 7 6 7 6 7 5 6 5 7 7
+6 6 7 6 5 3 6 4 5 6 4 5
+4 6 5 6 6 5 4 3 5 6 5 6
+5 6 5 5 6 5 4 4 6 6 5 5
+6 6 7 6 6 5 6 5 6 6 5 6
+5 6 6 5 6 5 5 4 6 5 5 5
+5 7 7 7 7 7 6 5 7 7 7 7
+6 6 6 5 5 7 6 5 6 6 5 6
+. . . . . . . . . . . .
+6 6 5 5 5 6 6 4 6 5 5 5
+7 7 7 6 7 6 7 6 6 6 7 6
+4 7 6 6 6 5 5 4 4 5 5 6
+5 6 3 5 4 1 4 6 2 3 3 2
+3 6 6 5 6 2 4 2 2 4 4 5
+6 6 7 5 6 5 7 6 5 6 6 5
+6 5 6 6 5 6 6 6 6 4 5 5
+5 7 7 . 6 6 6 5 6 6 6 6
+5 7 5 5 4 6 7 6 5 4 6 5
+7 7 7 6 7 7 7 6 7 7 7 6
+3 6 5 6 5 7 7 3 4 7 5 7
+6 7 7 6 5 6 5 5 6 6 6 6
+5 5 6 5 5 5 5 4 5 5 5 6
+6 6 7 4 5 6 6 6 6 5 5 6
+6 5 6 6 4 4 5 4 5 6 4 5
+5 6 7 6 6 7 7 5 6 6 6 5
+5 6 5 7 4 6 6 5 7 7 5 6
+. . . . . . . . . . . .
+7 6 6 5 6 6 7 6 6 5 5 6
+6 6 7 7 7 7 7 6 7 6 6 7
+7 5 5 . 5 . 7 3 5 4 5 3
+7 6 7 5 4 5 7 5 7 5 5 6
+6 5 6 6 6 5 5 5 5 6 5 6
+7 7 7 7 7 7 7 7 5 6 7 7
+. . . . . . . . . . . .
+5 5 6 7 5 6 6 4 6 6 6 5
+6 6 5 7 5 6 7 5 6 5 4 6
+7 6 7 6 7 5 6 7 7 6 6 6
+5 6 5 6 5 6 7 2 5 7 3 7
+6 6 5 6 5 6 6 6 6 6 5 6
+7 6 7 6 6 6 6 6 6 7 6 7
+7 7 6 5 6 6 7 7 7 4 6 5
+3 7 7 6 6 7 7 7 6 6 6 4
+3 5 3 4 3 3 3 3 3 3 3 5
+5 7 7 7 5 7 6 2 6 7 6 7
+7 7 7 7 7 7 7 6 7 7 7 6
+6 5 7 4 4 4 5 6 5 5 4 5
+4 7 7 4 4 3 6 3 5 3 4 5
+7 7 7 7 7 7 7 7 7 7 7 5
+3 6 5 5 4 5 4 4 5 5 3 5
+6 7 6 6 6 7 7 6 6 6 7 6
+2 5 4 6 3 2 2 2 2 7 2 2
+4 6 6 5 5 5 6 5 5 6 6 5
+5 7 4 5 6 6 6 5 6 6 5 6
+5 7 7 5 6 5 6 5 5 4 5 4
+4 5 6 5 6 4 5 5 5 4 5 5
+7 6 6 5 5 6 7 5 6 5 7 6
+5 6 6 5 4 5 5 3 4 5 5 5
+5 7 6 4 4 5 6 5 6 4 4 6
+6 6 6 6 5 7 7 6 5 5 6 6
+6 6 7 6 7 6 6 5 6 7 6 5
+7 6 7 6 7 6 7 7 5 5 6 6
+5 6 6 5 5 5 6 5 6 7 7 5
+5 6 6 5 6 5 6 6 6 6 6 6
+5 5 5 5 6 4 5 3 4 7 6 5
+5 7 7 6 6 6 6 5 6 7 6 7
+6 6 7 7 7 5 6 5 5 5 5 4
+2 7 5 4 6 5 5 2 5 6 4 6
+6 7 7 5 6 6 7 6 6 7 5 7
+5 6 7 6 6 3 5 7 6 6 5 6
+6 6 6 3 5 5 5 6 6 6 4 5
+4 7 7 4 7 4 5 5 5 7 4 4
+. . . . . . . . . . . .
+6 6 7 6 7 6 7 7 6 7 7 6
+. . . . . . . . . . . .
+5 6 5 7 6 5 6 6 5 6 4 6
+5 5 5 5 4 5 5 5 7 5 5 5
+6 6 6 4 5 4 6 6 6 4 5 4
+6 5 7 4 6 4 6 5 6 6 6 3
+5 7 6 5 5 5 5 5 6 7 6 6
+5 5 7 7 5 5 6 6 5 5 5 7
+5 6 7 6 7 5 6 4 6 7 6 7
+4 5 5 5 6 5 6 5 6 6 5 6
+6 5 5 5 6 3 4 5 5 4 5 3
+6 6 6 5 5 5 4 3 4 5 5 5
+6 7 7 6 2 3 6 6 6 5 7 7
+6 7 5 5 6 6 6 5 6 6 6 6
+6 7 7 6 7 7 7 5 5 6 6 6
+6 6 6 6 7 6 6 7 6 6 6 6
+5 6 6 6 3 5 6 6 5 5 4 6
+4 6 5 6 6 5 6 5 6 6 5 5
+6 4 6 5 4 6 7 4 5 6 5 5
+6 7 6 4 6 5 7 6 7 7 6 5
+6 7 7 6 7 6 7 7 7 6 6 6
+6 6 6 4 5 6 7 7 5 6 4 4
+3 3 5 3 3 1 5 6 3 2 3 3
+7 7 5 6 6 7 7 6 7 7 7 7
+5 6 6 6 7 5 4 5 4 7 6 7
+3 6 5 4 3 3 3 5 5 6 3 4
+5 7 6 4 6 5 5 6 6 7 5 6
+5 7 6 6 6 6 6 5 6 7 7 6
+7 7 5 6 7 7 7 7 6 5 7 7
+6 7 6 6 5 6 7 7 6 5 6 6
+6 7 7 7 7 6 6 7 6 7 7 7
+4 6 4 7 3 6 5 5 4 3 5 6
+5 5 7 5 4 6 7 5 4 6 6 5
+5 5 6 4 6 5 7 6 5 5 5 6
+. . . . . . . . . . . .
+. . . . . . . . . . . .
+5 7 7 5 6 6 7 7 6 6 6 7
+6 7 7 1 2 1 7 7 5 5 5 2
+. . . . . . . . . . . .
+3 7 4 6 4 7 4 6 4 7 4 7
+5 7 3 5 5 6 7 5 4 7 7 4
+4 7 7 5 4 6 7 7 6 5 4 4
+6 6 2 2 6 4 6 5 5 1 5 2
+5 5 6 4 5 4 6 5 5 6 5 5
+. . . . . . . . . . . .
+5 7 6 6 6 6 6 6 5 6 6 6
+6 6 6 5 6 6 6 6 7 5 6 7
+3 6 3 3 5 3 3 5 3 5 7 4
+4 4 6 3 3 3 4 3 4 2 3 6
+5 7 7 6 5 4 7 5 7 7 3 7
+4 5 4 4 4 4 3 3 3 4 3 3
+6 7 7 5 6 6 7 5 4 5 5 5
+3 5 3 3 1 3 4 3 4 7 6 7
+4 5 4 4 4 3 4 5 6 6 4 5
+5 6 3 4 5 3 5 3 4 5 6 4
+5 5 5 6 6 6 6 4 5 6 6 5
+6 7 7 2 2 6 7 7 7 7 5 7
+5 7 7 4 6 5 7 5 5 5 6 6
+6 6 7 7 5 5 5 7 6 7 7 7
+6 5 7 3 6 5 6 5 5 6 5 4
+5 7 6 5 6 6 6 5 6 5 5 6
+4 5 5 5 6 3 5 3 3 6 5 5
+. . . . . . . . . . . .
+5 6 6 4 4 4 5 3 5 5 2 6
+5 6 7 5 5 6 6 5 5 6 6 6
+6 7 7 6 4 7 7 6 7 5 6 7
+6 6 5 4 5 2 7 6 6 5 6 6
+2 2 2 2 2 2 3 2 3 1 1 2
+end data.
+
+RELIABILITY
+ /VARIABLES=var2 var8 var15 var17 var6
+ /SCALE('Everything') var6 var8 var15 var17
+ /MODEL=ALPHA.
+
+RELIABILITY
+ /VARIABLES=var6 var8 var15 var17
+ /SCALE('Nothing') ALL
+ /MODEL=SPLIT(2)
+ .
+
+RELIABILITY
+ /VARIABLES=var2 var6 var8 var15 var17 var19
+ /SCALE('Totals') var6 var8 var15 var17
+ /SUMMARY = total
+ /STATISTICS = DESCRIPTIVES COVARIANCES
+ .
+
+
+RELIABILITY
+ /VARIABLES=var6 var8 var15 var17
+ .
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt reliability.sps], [0], [dnl
+reliability.sps:174.4-174.40: warning: RELIABILITY: The STATISTICS subcommand is not yet implemented. No statistics will be produced.
+ 174 | /STATISTICS = DESCRIPTIVES COVARIANCES
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Scale: Everything
+
+Table: Case Processing Summary
+Cases,N,Percent
+Valid,131,92.9%
+Excluded,10,7.1%
+Total,141,100.0%
+
+Table: Reliability Statistics
+Cronbach's Alpha,N of Items
+.75,4
+
+Scale: Nothing
+
+Table: Case Processing Summary
+Cases,N,Percent
+Valid,131,92.9%
+Excluded,10,7.1%
+Total,141,100.0%
+
+Table: Reliability Statistics
+Cronbach's Alpha,Part 1,Value,.55
+,,N of Items,2
+,Part 2,Value,.63
+,,N of Items,2
+,Total N of Items,,4
+Correlation Between Forms,,,.61
+Spearman-Brown Coefficient,Equal Length,,.75
+,Unequal Length,,.75
+Guttman Split-Half Coefficient,,,.75
+
+"reliability.sps:174.4-174.40: warning: RELIABILITY: The STATISTICS subcommand is not yet implemented. No statistics will be produced.
+ 174 | /STATISTICS = DESCRIPTIVES COVARIANCES
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+Scale: Totals
+
+Table: Case Processing Summary
+Cases,N,Percent
+Valid,131,92.9%
+Excluded,10,7.1%
+Total,141,100.0%
+
+Table: Reliability Statistics
+Cronbach's Alpha,N of Items
+.75,4
+
+Table: Item-Total Statistics
+,Scale Mean if Item Deleted,Scale Variance if Item Deleted,Corrected Item-Total Correlation,Cronbach's Alpha if Item Deleted
+var6,15.97,8.43,.51,.71
+var8,16.56,7.86,.53,.70
+var15,16.47,8.45,.56,.68
+var17,16.60,8.00,.57,.67
+
+Scale: ANY
+
+Table: Case Processing Summary
+Cases,N,Percent
+Valid,131,92.9%
+Excluded,10,7.1%
+Total,141,100.0%
+
+Table: Reliability Statistics
+Cronbach's Alpha,N of Items
+.75,4
+])
+AT_CLEANUP
+
+
+dnl This was causing a AT.
+AT_SETUP([RELIABILITY bad syntax])
+AT_DATA([bad-syntax.sps], [dnl
+data list notable list /x * y *.
+begin data.
+1 10
+2 20
+3 30
+4 50
+5 50
+end data.
+
+* This syntax is incorrect
+reliability x y.
+])
+
+AT_CHECK([pspp -O format=csv bad-syntax.sps], [1], [ignore])
+
+AT_CLEANUP
+
+dnl Checks for a crash when bad syntax followed scale specification.
+AT_SETUP([RELIABILITY bad syntax 2])
+AT_DATA([bad-syntax.sps], [dnl
+new file.
+data list notable list /f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 *.
+begin data.
+end data.
+
+* This syntax is incorrect
+reliability variables=f01 to f10/asdfj.
+])
+AT_CHECK([pspp -O format=csv bad-syntax.sps], [1], [ignore])
+AT_CLEANUP
+
+
+dnl Checks for a crash when the active file was empty. Bug #38660.
+AT_SETUP([RELIABILITY crash with no data])
+AT_DATA([reliability.sps], [dnl
+new file.
+data list notable list /f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 *.
+begin data.
+end data.
+
+reliability variables=f01 to f10.
+])
+AT_CHECK([pspp -O format=csv reliability.sps], [0], [])
+AT_CLEANUP
+
+
+
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([RELIABILITY tutorial example])
+AT_DATA([tut-example.sps], [dnl
+get file='hotel.sav'.
+
+compute v3 = 6 - v3.
+compute v5 = 6 - v5.
+
+reliability variables = v1 v3 v4.
+])
+
+AT_CHECK([ln -s $top_srcdir/examples/hotel.sav .], [0])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt tut-example.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Scale: ANY
+
+Table: Case Processing Summary
+Cases,N,Percent
+Valid,17,100.0%
+Excluded,0,.0%
+Total,17,100.0%
+
+Table: Reliability Statistics
+Cronbach's Alpha,N of Items
+.81,3
+])
+AT_CLEANUP
+
+AT_SETUP([RELIABILITY syntax errors])
+AT_DATA([reliability.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+RELIABILITY **.
+RELIABILITY VARIABLES=**.
+RELIABILITY VARIABLES=x/ **.
+RELIABILITY VARIABLES=x y/SCALE **.
+RELIABILITY VARIABLES=x y/SCALE(**).
+RELIABILITY VARIABLES=x y/SCALE('a' **).
+RELIABILITY VARIABLES=x y/SCALE('a')=**.
+RELIABILITY VARIABLES=x y/MODEL SPLIT(**).
+RELIABILITY VARIABLES=x y/MODEL SPLIT(1 **).
+RELIABILITY VARIABLES=x y/MODEL **.
+RELIABILITY VARIABLES=x y/SUMMARY **.
+RELIABILITY VARIABLES=x y/MISSING=**.
+RELIABILITY VARIABLES=x y/STATISTICS=x y zz y/ **.
+RELIABILITY VARIABLES=x y/MODEL SPLIT(5).
+])
+AT_CHECK([pspp -O format=csv reliability.sps], [1], [dnl
+"reliability.sps:2.13-2.14: error: RELIABILITY: Syntax error expecting VARIABLES.
+ 2 | RELIABILITY **.
+ | ^~"
+
+"reliability.sps:3.23-3.24: error: RELIABILITY: Syntax error expecting variable name.
+ 3 | RELIABILITY VARIABLES=**.
+ | ^~"
+
+"reliability.sps:4.23: warning: RELIABILITY: Reliability on a single variable is not useful.
+ 4 | RELIABILITY VARIABLES=x/ **.
+ | ^"
+
+"reliability.sps:4.26-4.27: error: RELIABILITY: Syntax error expecting SCALE, MODEL, SUMMARY, MISSING, or STATISTICS.
+ 4 | RELIABILITY VARIABLES=x/ **.
+ | ^~"
+
+"reliability.sps:5.33-5.34: error: RELIABILITY: Syntax error expecting `('.
+ 5 | RELIABILITY VARIABLES=x y/SCALE **.
+ | ^~"
+
+"reliability.sps:6.33-6.34: error: RELIABILITY: Syntax error expecting string.
+ 6 | RELIABILITY VARIABLES=x y/SCALE(**).
+ | ^~"
+
+"reliability.sps:7.37-7.38: error: RELIABILITY: Syntax error expecting `)'.
+ 7 | RELIABILITY VARIABLES=x y/SCALE('a' **).
+ | ^~"
+
+"reliability.sps:8.38-8.39: error: RELIABILITY: Syntax error expecting variable name.
+ 8 | RELIABILITY VARIABLES=x y/SCALE('a')=**.
+ | ^~"
+
+"reliability.sps:9.39-9.40: error: RELIABILITY: Syntax error expecting number.
+ 9 | RELIABILITY VARIABLES=x y/MODEL SPLIT(**).
+ | ^~"
+
+"reliability.sps:10.41-10.42: error: RELIABILITY: Syntax error expecting `@:}@'.
+ 10 | RELIABILITY VARIABLES=x y/MODEL SPLIT(1 **).
+ | ^~"
+
+"reliability.sps:11.33-11.34: error: RELIABILITY: Syntax error expecting ALPHA or SPLIT.
+ 11 | RELIABILITY VARIABLES=x y/MODEL **.
+ | ^~"
+
+"reliability.sps:12.35-12.36: error: RELIABILITY: Syntax error expecting TOTAL or ALL.
+ 12 | RELIABILITY VARIABLES=x y/SUMMARY **.
+ | ^~"
+
+"reliability.sps:13.35-13.36: error: RELIABILITY: Syntax error expecting INCLUDE or EXCLUDE.
+ 13 | RELIABILITY VARIABLES=x y/MISSING=**.
+ | ^~"
+
+"reliability.sps:14.27-14.45: warning: RELIABILITY: The STATISTICS subcommand is not yet implemented. No statistics will be produced.
+ 14 | RELIABILITY VARIABLES=x y/STATISTICS=x y zz y/ **.
+ | ^~~~~~~~~~~~~~~~~~~"
+
+"reliability.sps:14.48-14.49: error: RELIABILITY: Syntax error expecting SCALE, MODEL, SUMMARY, MISSING, or STATISTICS.
+ 14 | RELIABILITY VARIABLES=x y/STATISTICS=x y zz y/ **.
+ | ^~"
+
+"reliability.sps:15.39: error: RELIABILITY: The split point must be less than the number of variables.
+ 15 | RELIABILITY VARIABLES=x y/MODEL SPLIT(5).
+ | ^"
+
+"reliability.sps:15.23-15.25: note: RELIABILITY: There are 2 variables.
+ 15 | RELIABILITY VARIABLES=x y/MODEL SPLIT(5).
+ | ^~~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([RENAME VARIABLES])
+
+AT_SETUP([RENAME VARIABLES])
+AT_DATA([rename-variables.sps], [dnl
+DATA LIST LIST /brakeFluid y auxiliary warp (F2.0).
+BEGIN DATA.
+1 3 5 9
+2 3 6 10
+3 3 7 11
+4 3 8 11
+END DATA.
+
+LIST.
+
+RENAME VARIABLES brakeFluid=applecarts y=bananamobiles.
+RENAME VARIABLES (warp auxiliary=foobar xyzzy).
+
+LIST.
+
+SAVE /OUTFILE='rename.sav'.
+])
+AT_CHECK([pspp -O format=csv rename-variables.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+brakeFluid,F2.0
+y,F2.0
+auxiliary,F2.0
+warp,F2.0
+
+Table: Data List
+brakeFluid,y,auxiliary,warp
+1,3,5,9
+2,3,6,10
+3,3,7,11
+4,3,8,11
+
+Table: Data List
+applecarts,bananamobiles,xyzzy,foobar
+1,3,5,9
+2,3,6,10
+3,3,7,11
+4,3,8,11
+])
+AT_CHECK([grep '[bB][rR][aA][kK][eE]' rename.sav], [1], [ignore-nolog])
+AT_CLEANUP
+
+
+AT_SETUP([RENAME VARIABLES -- multiple sets])
+AT_DATA([rename-variables.sps], [dnl
+data list list /a b c d e *.
+begin data.
+1 2 3 4 5
+end data.
+
+rename variables (a b=x y) (c d e=z zz zzz).
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv rename-variables.sps], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+a,F8.0
+b,F8.0
+c,F8.0
+d,F8.0
+e,F8.0
+
+Table: Data List
+x,y,z,zz,zzz
+1.00,2.00,3.00,4.00,5.00
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([RENAME VARIABLES -- invalid syntax 1])
+
+AT_DATA([rename-variables.sps], [dnl
+DATA LIST LIST /brakeFluid y auxiliary warp (F2.0).
+RENAME VARIABLES warp auxiliary=foobar xyzzy.
+])
+
+AT_CHECK([pspp -o pspp.csv rename-variables.sps], [1], [dnl
+rename-variables.sps:2.23-2.31: error: RENAME VARIABLES: Syntax error expecting `='.
+ 2 | RENAME VARIABLES warp auxiliary=foobar xyzzy.
+ | ^~~~~~~~~
+])
+AT_CLEANUP
+
+AT_SETUP([RENAME VARIABLES -- invalid syntax 2])
+AT_DATA([rename-variables.sps], [dnl
+DATA LIST LIST /brakeFluid y auxiliary warp (F2.0).
+RENAME VARIABLES (brakeFluid=applecarts y=bananamobiles).
+])
+
+AT_CHECK([pspp -o pspp.csv rename-variables.sps], [1], [dnl
+rename-variables.sps:2.19-2.41: error: RENAME VARIABLES: Differing number of variables in old name list (1) and in new name list (2).
+ 2 | RENAME VARIABLES (brakeFluid=applecarts y=bananamobiles).
+ | ^~~~~~~~~~~~~~~~~~~~~~~
+])
+AT_CLEANUP
+
+
+
+
+AT_SETUP([RENAME VARIABLES -- invalid syntax 3])
+AT_DATA([rename-variables.sps], [dnl
+DATA LIST NOTABLE LIST /z y p q (F2.0).
+BEGIN DATA.
+4 3 8 11
+END DATA.
+
+RENAME VARIABLES z=a y}bqnanamobiles.
+
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv rename-variables.sps], [1], [ignore])
+
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([ROC])
+
+AT_SETUP([ROC free distribution])
+AT_DATA([roc.sps], [dnl
+set format F10.3.
+data list notable list /x * y * w * a *.
+begin data.
+1 1 2 1
+1 2 28 0
+2 3 4 1
+2 4 14 0
+3 5 10 1
+. . 1 0
+3 1 5 0
+4 2 14 1
+4 3 2 0
+5 4 20 1
+5 4 20 .
+5 5 1 0
+end data.
+
+weight by w.
+
+roc x by a (1)
+ /plot = none
+ /print = se coordinates
+ /criteria = testpos(large) distribution(free) ci(99)
+ /missing = exclude .
+])
+AT_CHECK([pspp -o pspp.csv roc.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Case Summary
+a,Valid N (listwise),
+,Unweighted,Weighted
+Positive,5,50.000
+Negative,5,50.000
+
+Table: Area Under the Curve
+Variable under test,Area,Std. Error,Asymptotic Sig.,Asymp. 99% Confidence Interval,
+,,,,Lower Bound,Upper Bound
+x,.910,.030,.000,.839,.981
+
+Table: Coordinates of the Curve
+Test variable,Positive if greater than or equal to,Sensitivity,1 - Specificity
+x,.000,1.000,1.000
+,1.500,.960,.440
+,2.500,.880,.160
+,3.500,.680,.060
+,4.500,.400,.020
+,6.000,.000,.000
+])
+AT_CLEANUP
+
+AT_SETUP([ROC negative exponential distribution])
+AT_DATA([roc.sps], [dnl
+set format F10.3.
+data list notable list /x * y * w * a *.
+begin data.
+1 1 2 1
+1 2 28 0
+2 3 4 1
+2 4 14 0
+3 5 10 1
+. . 1 0
+3 1 5 0
+4 2 14 1
+4 3 2 0
+5 4 20 1
+5 4 20 .
+5 5 1 0
+end data.
+
+weight by w.
+
+roc x y by a (1)
+ /plot = curve(reference)
+ /print = se coordinates
+ /criteria = testpos(large) distribution(negexpo) ci(95)
+ /missing = exclude .
+])
+AT_CHECK([pspp -o pspp.csv roc.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Case Summary
+a,Valid N (listwise),
+,Unweighted,Weighted
+Positive,5,50.000
+Negative,5,50.000
+
+Table: Area Under the Curve
+Variable under test,Area,Std. Error,Asymptotic Sig.,Asymp. 95% Confidence Interval,
+,,,,Lower Bound,Upper Bound
+x,.910,.030,.000,.860,.960
+y,.697,.052,.001,.611,.783
+
+Table: Coordinates of the Curve
+Test variable,Positive if greater than or equal to,Sensitivity,1 - Specificity
+x,.000,1.000,1.000
+,1.500,.960,.440
+,2.500,.880,.160
+,3.500,.680,.060
+,4.500,.400,.020
+,6.000,.000,.000
+y,.000,1.000,1.000
+,1.500,.960,.900
+,2.500,.680,.340
+,3.000,.600,.340
+,3.500,.600,.300
+,4.500,.200,.020
+,6.000,.000,.000
+])
+AT_CLEANUP
+
+AT_SETUP([ROC with anomaly])
+AT_DATA([roc.sps], [dnl
+set format F10.3.
+data list notable list /x * a * comment (a20).
+begin data.
+0 1 ""
+0 0 ""
+1 1 ""
+1 0 ""
+2 1 ""
+2 0 ""
+5 1 ""
+5 0 ""
+10 1 ""
+10 0 ""
+15 1 ""
+15 0 ""
+20 1 ""
+20 1 ""
+22 0 "here and"
+22 0 "here is the anomaly"
+25 1 ""
+25 0 ""
+30 1 ""
+30 0 ""
+35 1 ""
+35 0 ""
+38 1 ""
+38 0 ""
+39 1 ""
+39 0 ""
+40 1 ""
+40 0 ""
+end data.
+
+roc x by a (1)
+ /plot = none
+ print = se
+ .
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt roc.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Case Summary
+a,Valid N (listwise),
+,Unweighted,Weighted
+Positive,14,14.000
+Negative,14,14.000
+
+Table: Area Under the Curve
+Variable under test,Area,Std. Error,Asymptotic Sig.,Asymp. 95% Confidence Interval,
+,,,,Lower Bound,Upper Bound
+x,.490,.111,.927,.307,.673
+])
+AT_CLEANUP
+
+
+
+
+AT_SETUP([ROC crash on no state variable])
+AT_DATA([roc.sps], [dnl
+data list notable list /x * y * w * a *.
+begin data.
+5 5 1 0
+end data.
+
+
+roc x y By(a (1)
+ .
+])
+
+AT_CHECK([pspp -o pspp.csv roc.sps], [1], [ignore])
+
+AT_CLEANUP
+
+
+AT_SETUP([ROC crash on invalid syntax])
+AT_DATA([roc.sps], [dnl
+data list notable list /x * y * a *.
+bggin data.
+1 1 2
+1 2 28
+end data.
+
+
+roc x y by a (1)
+ /criteria = ci(y5)
+])
+
+AT_CHECK([pspp -O format=csv roc.sps], [1], [ignore])
+
+AT_CLEANUP
+
+AT_SETUP([ROC syntax errors])
+AT_DATA([roc.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+ROC **.
+ROC x **.
+ROC x BY **.
+ROC x BY y **.
+ROC x BY y(**).
+ROC x BY y(5 **).
+ROC x BY y(5)/MISSING=**.
+ROC x BY y(5)/PLOT=CURVE(**).
+ROC x BY y(5)/PLOT=CURVE(REFERENCE **).
+ROC x BY y(5)/PLOT=**.
+ROC x BY y(5)/PRINT=**.
+ROC x BY y(5)/CRITERIA=CUTOFF **.
+ROC x BY y(5)/CRITERIA=CUTOFF(**).
+ROC x BY y(5)/CRITERIA=CUTOFF(INCLUDE **).
+ROC x BY y(5)/CRITERIA=TESTPOS **.
+ROC x BY y(5)/CRITERIA=TESTPOS(**).
+ROC x BY y(5)/CRITERIA=TESTPOS(LARGE **).
+ROC x BY y(5)/CRITERIA=CI **.
+ROC x BY y(5)/CRITERIA=CI(**).
+ROC x BY y(5)/CRITERIA=CI(5 **).
+ROC x BY y(5)/CRITERIA=DISTRIBUTION **.
+ROC x BY y(5)/CRITERIA=DISTRIBUTION(**).
+ROC x BY y(5)/CRITERIA=DISTRIBUTION(FREE **).
+ROC x BY y(5)/CRITERIA=**.
+ROC x BY y(5)/ **.
+])
+AT_CHECK([pspp -O format=csv roc.sps], [1], [dnl
+"roc.sps:2.5-2.6: error: ROC: Syntax error expecting variable name.
+ 2 | ROC **.
+ | ^~"
+
+"roc.sps:3.7-3.8: error: ROC: Syntax error expecting `BY'.
+ 3 | ROC x **.
+ | ^~"
+
+"roc.sps:4.10-4.11: error: ROC: Syntax error expecting variable name.
+ 4 | ROC x BY **.
+ | ^~"
+
+"roc.sps:5.12-5.13: error: ROC: Syntax error expecting `('.
+ 5 | ROC x BY y **.
+ | ^~"
+
+"roc.sps:6.12-6.13: error: ROC: Syntax error expecting number.
+ 6 | ROC x BY y(**).
+ | ^~"
+
+"roc.sps:7.14-7.15: error: ROC: Syntax error expecting `)'.
+ 7 | ROC x BY y(5 **).
+ | ^~"
+
+"roc.sps:8.23-8.24: error: ROC: Syntax error expecting INCLUDE or EXCLUDE.
+ 8 | ROC x BY y(5)/MISSING=**.
+ | ^~"
+
+"roc.sps:9.26-9.27: error: ROC: Syntax error expecting REFERENCE.
+ 9 | ROC x BY y(5)/PLOT=CURVE(**).
+ | ^~"
+
+"roc.sps:10.36-10.37: error: ROC: Syntax error expecting `@:}@'.
+ 10 | ROC x BY y(5)/PLOT=CURVE(REFERENCE **).
+ | ^~"
+
+"roc.sps:11.20-11.21: error: ROC: Syntax error expecting CURVE or NONE.
+ 11 | ROC x BY y(5)/PLOT=**.
+ | ^~"
+
+"roc.sps:12.21-12.22: error: ROC: Syntax error expecting SE or COORDINATES.
+ 12 | ROC x BY y(5)/PRINT=**.
+ | ^~"
+
+"roc.sps:13.31-13.32: error: ROC: Syntax error expecting `('.
+ 13 | ROC x BY y(5)/CRITERIA=CUTOFF **.
+ | ^~"
+
+"roc.sps:14.31-14.32: error: ROC: Syntax error expecting INCLUDE or EXCLUDE.
+ 14 | ROC x BY y(5)/CRITERIA=CUTOFF(**).
+ | ^~"
+
+"roc.sps:15.39-15.40: error: ROC: Syntax error expecting `)'.
+ 15 | ROC x BY y(5)/CRITERIA=CUTOFF(INCLUDE **).
+ | ^~"
+
+"roc.sps:16.32-16.33: error: ROC: Syntax error expecting `('.
+ 16 | ROC x BY y(5)/CRITERIA=TESTPOS **.
+ | ^~"
+
+"roc.sps:17.32-17.33: error: ROC: Syntax error expecting LARGE or SMALL.
+ 17 | ROC x BY y(5)/CRITERIA=TESTPOS(**).
+ | ^~"
+
+"roc.sps:18.38-18.39: error: ROC: Syntax error expecting `)'.
+ 18 | ROC x BY y(5)/CRITERIA=TESTPOS(LARGE **).
+ | ^~"
+
+"roc.sps:19.27-19.28: error: ROC: Syntax error expecting `('.
+ 19 | ROC x BY y(5)/CRITERIA=CI **.
+ | ^~"
+
+"roc.sps:20.27-20.28: error: ROC: Syntax error expecting number.
+ 20 | ROC x BY y(5)/CRITERIA=CI(**).
+ | ^~"
+
+"roc.sps:21.29-21.30: error: ROC: Syntax error expecting `)'.
+ 21 | ROC x BY y(5)/CRITERIA=CI(5 **).
+ | ^~"
+
+"roc.sps:22.37-22.38: error: ROC: Syntax error expecting `('.
+ 22 | ROC x BY y(5)/CRITERIA=DISTRIBUTION **.
+ | ^~"
+
+"roc.sps:23.37-23.38: error: ROC: Syntax error expecting FREE or NEGEXPO.
+ 23 | ROC x BY y(5)/CRITERIA=DISTRIBUTION(**).
+ | ^~"
+
+"roc.sps:24.42-24.43: error: ROC: Syntax error expecting `)'.
+ 24 | ROC x BY y(5)/CRITERIA=DISTRIBUTION(FREE **).
+ | ^~"
+
+"roc.sps:25.24-25.25: error: ROC: Syntax error expecting CUTOFF, TESTPOS, CI, or DISTRIBUTION.
+ 25 | ROC x BY y(5)/CRITERIA=**.
+ | ^~"
+
+"roc.sps:26.16-26.17: error: ROC: Syntax error expecting MISSING, PLOT, PRINT, or CRITERIA.
+ 26 | ROC x BY y(5)/ **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SAMPLE])
+
+AT_SETUP([SAMPLE])
+AT_DATA([sample.sps], [dnl
+set seed=3
+
+data list notable /A 1-2.
+begin data.
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+end data.
+sample .5.
+list.
+])
+AT_CHECK([pspp -o pspp.csv sample.sps])
+AT_CAPTURE_FILE([pspp.csv])
+AT_CHECK(
+ [n=0
+ while read line; do
+ n=`expr $n + 1`
+ line=$(echo "$line" | tr -d '\r')
+ case $line in # (
+ "Table: Data List" | A | [[0-9]] | 10) ;; # (
+ *) echo $line; exit 1;
+ esac
+ done < pspp.csv
+ if test $n -ge 11; then exit 1; fi
+ ])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SAVE TRANSLATE /TYPE=CSV])
+
+m4_define([PREPARE_SAVE_TRANSLATE_CSV], [dnl
+AT_KEYWORDS([SAVE TRANSLATE])
+AT_DATA([data.txt], [dnl
+0 '1 9:30:05' 1/2/2003 "25/8/1995 15:30:00" "'a,b,c'",0
+, '-0 5:17' 10/31/2010 "9/4/2008 9:29:00" " xxx ",1
+1.625,'0 12:00',,,xyzzy,1
+])
+AT_DATA([save-translate.pspp], [dnl
+SET DECIMAL=DOT.
+DATA LIST LIST NOTABLE FILE="data.txt"
+ /number(F8.3) time(DTIME10) date(ADATE10) datetime(DATETIME20) string(A8)
+ filter(F1.0).
+MISSING VALUES number(0) time('0 12:00') string('xyzzy').
+FILTER BY filter.
+SAVE TRANSLATE /OUTFILE="data.csv" /TYPE=m4_if([$2], [], [CSV], [$2])
+ $1.
+])
+AT_CHECK([pspp -O format=csv save-translate.pspp], [0])
+])
+
+AT_SETUP([CSV output -- defaults])
+PREPARE_SAVE_TRANSLATE_CSV
+AT_CHECK([cat data.csv], [0], [dnl
+0,33:30:05,01/02/2003,08/25/1995 15:30:00,"'a,b,c'",0
+ ,-05:17:00,10/31/2010,04/09/2008 09:29:00, xxx,1
+1.625,12:00:00, , ,xyzzy,1
+])
+AT_CLEANUP
+
+AT_SETUP([CSV output -- recode missing, delete unselected])
+PREPARE_SAVE_TRANSLATE_CSV([/MISSING=RECODE /UNSELECTED=DELETE])
+AT_CHECK([cat data.csv], [0], [dnl
+ ,-05:17:00,10/31/2010,04/09/2008 09:29:00, xxx,1
+1.625, , , ,,1
+])
+AT_CLEANUP
+
+AT_SETUP([CSV output -- var names, formats])
+PREPARE_SAVE_TRANSLATE_CSV(
+ [/FIELDNAMES /TEXTOPTIONS FORMAT=VARIABLE /UNSELECTED=RETAIN])
+AT_CHECK([cat data.csv], [0], [dnl
+number,time,date,datetime,string,filter
+.000,1 09:30:05,01/02/2003,25-AUG-1995 15:30:00,"'a,b,c'",0
+ ,-0 05:17,10/31/2010,09-APR-2008 09:29:00, xxx,1
+1.625,0 12:00:00, , ,xyzzy,1
+])
+AT_CLEANUP
+
+AT_SETUP([CSV output -- comma as decimal point])
+PREPARE_SAVE_TRANSLATE_CSV([/FIELDNAMES /TEXTOPTIONS DECIMAL=COMMA])
+AT_CHECK([cat data.csv], [0], [dnl
+number;time;date;datetime;string;filter
+0;33:30:05;01/02/2003;08/25/1995 15:30:00;'a,b,c';0
+ ;-05:17:00;10/31/2010;04/09/2008 09:29:00; xxx;1
+1,625;12:00:00; ; ;xyzzy;1
+])
+AT_CLEANUP
+
+AT_SETUP([CSV output -- custom delimiter, qualifier])
+PREPARE_SAVE_TRANSLATE_CSV(
+ [/FIELDNAMES /TEXTOPTIONS DELIMITER=':' QUALIFIER="'"])
+AT_CHECK([cat data.csv], [0], [dnl
+number:time:date:datetime:string:filter
+0:'33:30:05':01/02/2003:'08/25/1995 15:30:00':'''a,b,c''':0
+ :'-05:17:00':10/31/2010:'04/09/2008 09:29:00': xxx:1
+1.625:'12:00:00': : :xyzzy:1
+])
+AT_CLEANUP
+
+AT_SETUP([CSV output -- KEEP, RENAME quoted])
+PREPARE_SAVE_TRANSLATE_CSV(
+ [/FIELDNAMES /KEEP=time string /RENAME string='long name with spaces' /UNSELECTED=DELETE])
+AT_CHECK([cat data.csv], [0], [dnl
+time,long name with spaces
+-05:17:00, xxx
+12:00:00,xyzzy
+])
+AT_CLEANUP
+
+
+AT_SETUP([CSV output -- KEEP, RENAME multi quoted])
+PREPARE_SAVE_TRANSLATE_CSV(
+ [/FIELDNAMES
+ /RENAME =
+ number = "this one"
+ time = "that one"
+ date = "which one?"
+ datetime = "another variable replacement"
+ string="long name with spaces"
+ /UNSELECTED=DELETE])
+AT_CHECK([cat data.csv], [0], [dnl
+this one,that one,which one?,another variable replacement,long name with spaces,filter
+ ,-05:17:00,10/31/2010,04/09/2008 09:29:00, xxx,1
+1.625,12:00:00, , ,xyzzy,1
+])
+AT_CLEANUP
+
+
+AT_SETUP([CSV output -- KEEP, RENAME bad name ])
+AT_KEYWORDS([SAVE TRANSLATE])
+AT_DATA([bad.sps], [
+data list notable list /Var1 Var2 Var3 Var4 Var5 *.
+begin data
+1 2 3 4 5
+end data.
+
+SAVE TRANSLATE
+/OUTFILE="foo.csv"
+ /TYPE=CSV
+ /MAP
+ /REPLACE
+ /FIELDNAMES
+ /Unselected=DELETE
+ /RENAME =
+ Var4 = Var5
+ (Var1 Var2 = one Var3 )
+ (Var3 = "The second")
+ /CELLS=VALUES
+.
+])
+
+AT_CHECK([pspp -O format=csv bad.sps], [1], [dnl
+"bad.sps:15.9-17.29: error: SAVE TRANSLATE: Requested renaming duplicates variable name Var5.
+ 15 | Var4 = Var5
+ | ^~~~~~~~~~~
+ 16 | (Var1 Var2 = one Var3 )
+ | -------------------------------
+ 17 | (Var3 = ""The second"")
+ | -----------------------------"
+])
+
+
+AT_CLEANUP
+
+
+
+AT_BANNER([SAVE TRANSLATE /TYPE=TAB])
+
+AT_SETUP([TAB output])
+PREPARE_SAVE_TRANSLATE_CSV([/FIELDNAMES], [TAB])
+AT_CHECK([cat data.csv], [0], [dnl
+number time date datetime string filter
+0 33:30:05 01/02/2003 08/25/1995 15:30:00 'a,b,c' 0
+ -05:17:00 10/31/2010 04/09/2008 09:29:00 xxx 1
+1.625 12:00:00 xyzzy 1
+])
+AT_CLEANUP
+
+AT_SETUP([SAVE TRANSLATE syntax errors])
+: > xyzzy.csv
+AT_DATA([save-translate.sps], [dnl
+DATA LIST LIST NOTABLE /v1 to v10.
+SAVE TRANSLATE **.
+SAVE TRANSLATE/OUTFILE=**.
+SAVE TRANSLATE/OUTFILE='xyzzy.txt'/OUTFILE='xyzzy.txt'.
+SAVE TRANSLATE/TYPE=CSV/TYPE=**.
+SAVE TRANSLATE/TYPE=**.
+SAVE TRANSLATE/MISSING=**.
+SAVE TRANSLATE/CELLS=**.
+SAVE TRANSLATE/TEXTOPTIONS DELIMITER=**.
+SAVE TRANSLATE/TEXTOPTIONS DELIMITER='ab'.
+SAVE TRANSLATE/TEXTOPTIONS QUALIFIER=**.
+SAVE TRANSLATE/TEXTOPTIONS QUALIFIER='ab'.
+SAVE TRANSLATE/TEXTOPTIONS DECIMAL=**.
+SAVE TRANSLATE/UNSELECTED=**.
+SAVE TRANSLATE/ **.
+SAVE TRANSLATE/OUTFILE='xyzzy.csv'.
+SAVE TRANSLATE/TYPE=CSV.
+SAVE TRANSLATE/OUTFILE='xyzzy.csv'/TYPE=CSV.
+SAVE TRANSLATE/RENAME **.
+SAVE TRANSLATE/RENAME v1**.
+SAVE TRANSLATE/RENAME(v1**).
+SAVE TRANSLATE/RENAME v1=.
+SAVE TRANSLATE/RENAME v1=**.
+SAVE TRANSLATE/RENAME v1 to v5=v6.
+SAVE TRANSLATE/RENAME (v1=v2 v3).
+SAVE TRANSLATE/RENAME (v1 v2=v3).
+SAVE TRANSLATE/RENAME (v1=v3**.
+SAVE TRANSLATE/RENAME v1=v5.
+SAVE TRANSLATE/RENAME v1 v5=v5 v1.
+SAVE TRANSLATE/RENAME(v1 v5=v5 v1).
+SAVE TRANSLATE/RENAME(v1 to v10=v01 to v10).
+SAVE TRANSLATE/RENAME=v1=v1.
+SAVE TRANSLATE/DROP=ALL.
+SAVE TRANSLATE/DROP=**.
+SAVE TRANSLATE/KEEP=**.
+])
+AT_CHECK([pspp -O format=csv save-translate.sps], [1], [dnl
+"save-translate.sps:2.16-2.17: error: SAVE TRANSLATE: Syntax error expecting `/'.
+ 2 | SAVE TRANSLATE **.
+ | ^~"
+
+"save-translate.sps:3.24-3.25: error: SAVE TRANSLATE: Syntax error expecting a file name or handle name.
+ 3 | SAVE TRANSLATE/OUTFILE=**.
+ | ^~"
+
+"save-translate.sps:4.36-4.42: error: SAVE TRANSLATE: Subcommand OUTFILE may only be specified once.
+ 4 | SAVE TRANSLATE/OUTFILE='xyzzy.txt'/OUTFILE='xyzzy.txt'.
+ | ^~~~~~~"
+
+"save-translate.sps:5.25-5.28: error: SAVE TRANSLATE: Subcommand TYPE may only be specified once.
+ 5 | SAVE TRANSLATE/TYPE=CSV/TYPE=**.
+ | ^~~~"
+
+"save-translate.sps:6.21-6.22: error: SAVE TRANSLATE: Syntax error expecting CSV or TAB.
+ 6 | SAVE TRANSLATE/TYPE=**.
+ | ^~"
+
+"save-translate.sps:7.24-7.25: error: SAVE TRANSLATE: Syntax error expecting IGNORE or RECODE.
+ 7 | SAVE TRANSLATE/MISSING=**.
+ | ^~"
+
+"save-translate.sps:8.22-8.23: error: SAVE TRANSLATE: Syntax error expecting VALUES or LABELS.
+ 8 | SAVE TRANSLATE/CELLS=**.
+ | ^~"
+
+"save-translate.sps:9.38-9.39: error: SAVE TRANSLATE: Syntax error expecting string.
+ 9 | SAVE TRANSLATE/TEXTOPTIONS DELIMITER=**.
+ | ^~"
+
+"save-translate.sps:10.38-10.41: error: SAVE TRANSLATE: The DELIMITER string must contain exactly one character.
+ 10 | SAVE TRANSLATE/TEXTOPTIONS DELIMITER='ab'.
+ | ^~~~"
+
+"save-translate.sps:11.38-11.39: error: SAVE TRANSLATE: Syntax error expecting string.
+ 11 | SAVE TRANSLATE/TEXTOPTIONS QUALIFIER=**.
+ | ^~"
+
+"save-translate.sps:12.38-12.41: error: SAVE TRANSLATE: The QUALIFIER string must contain exactly one character.
+ 12 | SAVE TRANSLATE/TEXTOPTIONS QUALIFIER='ab'.
+ | ^~~~"
+
+"save-translate.sps:13.36-13.37: error: SAVE TRANSLATE: Syntax error expecting DOT or COMMA.
+ 13 | SAVE TRANSLATE/TEXTOPTIONS DECIMAL=**.
+ | ^~"
+
+"save-translate.sps:14.27-14.28: error: SAVE TRANSLATE: Syntax error expecting RETAIN or DELETE.
+ 14 | SAVE TRANSLATE/UNSELECTED=**.
+ | ^~"
+
+"save-translate.sps:15.17-15.18: error: SAVE TRANSLATE: Syntax error expecting MAP, DROP, KEEP, or RENAME.
+ 15 | SAVE TRANSLATE/ **.
+ | ^~"
+
+"save-translate.sps:16.1-16.35: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
+ 16 | SAVE TRANSLATE/OUTFILE='xyzzy.csv'.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"save-translate.sps:17.1-17.24: error: SAVE TRANSLATE: Required subcommand OUTFILE was not specified.
+ 17 | SAVE TRANSLATE/TYPE=CSV.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~"
+
+"save-translate.sps:18.16-18.34: error: SAVE TRANSLATE: Output file `xyzzy.csv' exists but REPLACE was not specified.
+ 18 | SAVE TRANSLATE/OUTFILE='xyzzy.csv'/TYPE=CSV.
+ | ^~~~~~~~~~~~~~~~~~~"
+
+"save-translate.sps:19.23-19.24: error: SAVE TRANSLATE: Syntax error expecting variable name.
+ 19 | SAVE TRANSLATE/RENAME **.
+ | ^~"
+
+"save-translate.sps:20.25-20.26: error: SAVE TRANSLATE: Syntax error expecting `='.
+ 20 | SAVE TRANSLATE/RENAME v1**.
+ | ^~"
+
+"save-translate.sps:21.25-21.26: error: SAVE TRANSLATE: Syntax error expecting `='.
+ 21 | SAVE TRANSLATE/RENAME(v1**).
+ | ^~"
+
+"save-translate.sps:22.26: error: SAVE TRANSLATE: Syntax error expecting variable name.
+ 22 | SAVE TRANSLATE/RENAME v1=.
+ | ^"
+
+"save-translate.sps:23.26-23.27: error: SAVE TRANSLATE: Syntax error expecting variable name.
+ 23 | SAVE TRANSLATE/RENAME v1=**.
+ | ^~"
+
+save-translate.sps:24: error: SAVE TRANSLATE: Old and new variable counts do not match.
+
+"save-translate.sps:24.23-24.30: note: SAVE TRANSLATE: There are 5 old variables.
+ 24 | SAVE TRANSLATE/RENAME v1 to v5=v6.
+ | ^~~~~~~~"
+
+"save-translate.sps:24.32-24.33: note: SAVE TRANSLATE: There is 1 new variable name.
+ 24 | SAVE TRANSLATE/RENAME v1 to v5=v6.
+ | ^~"
+
+save-translate.sps:25: error: SAVE TRANSLATE: Old and new variable counts do not match.
+
+"save-translate.sps:25.24-25.25: note: SAVE TRANSLATE: There is 1 old variable.
+ 25 | SAVE TRANSLATE/RENAME (v1=v2 v3).
+ | ^~"
+
+"save-translate.sps:25.27-25.31: note: SAVE TRANSLATE: There are 2 new variable names.
+ 25 | SAVE TRANSLATE/RENAME (v1=v2 v3).
+ | ^~~~~"
+
+save-translate.sps:26: error: SAVE TRANSLATE: Old and new variable counts do not match.
+
+"save-translate.sps:26.24-26.28: note: SAVE TRANSLATE: There are 2 old variables.
+ 26 | SAVE TRANSLATE/RENAME (v1 v2=v3).
+ | ^~~~~"
+
+"save-translate.sps:26.30-26.31: note: SAVE TRANSLATE: There is 1 new variable name.
+ 26 | SAVE TRANSLATE/RENAME (v1 v2=v3).
+ | ^~"
+
+"save-translate.sps:27.29-27.30: error: SAVE TRANSLATE: Syntax error expecting `)'.
+ 27 | SAVE TRANSLATE/RENAME (v1=v3**.
+ | ^~"
+
+"save-translate.sps:28.23-28.27: error: SAVE TRANSLATE: Requested renaming duplicates variable name v5.
+ 28 | SAVE TRANSLATE/RENAME v1=v5.
+ | ^~~~~"
+
+"save-translate.sps:29.26-29.27: error: SAVE TRANSLATE: Syntax error expecting `='.
+ 29 | SAVE TRANSLATE/RENAME v1 v5=v5 v1.
+ | ^~"
+
+"save-translate.sps:30.1-30.35: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
+ 30 | SAVE TRANSLATE/RENAME(v1 v5=v5 v1).
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"save-translate.sps:31.1-31.44: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
+ 31 | SAVE TRANSLATE/RENAME(v1 to v10=v01 to v10).
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"save-translate.sps:32.1-32.28: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
+ 32 | SAVE TRANSLATE/RENAME=v1=v1.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"save-translate.sps:33.16-33.23: error: SAVE TRANSLATE: Cannot DROP all variables from dictionary.
+ 33 | SAVE TRANSLATE/DROP=ALL.
+ | ^~~~~~~~"
+
+"save-translate.sps:34.21-34.22: error: SAVE TRANSLATE: Syntax error expecting variable name.
+ 34 | SAVE TRANSLATE/DROP=**.
+ | ^~"
+
+"save-translate.sps:35.21-35.22: error: SAVE TRANSLATE: Syntax error expecting variable name.
+ 35 | SAVE TRANSLATE/KEEP=**.
+ | ^~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([IMPORT and EXPORT])
+
+AT_SETUP([IMPORT and EXPORT])
+AT_DATA([import-export.sps], [dnl
+DATA LIST LIST NOTABLE /X Y.
+BEGIN DATA.
+1 2
+3 .
+5 6
+END DATA.
+
+EXPORT /OUTFILE='wiz.por'.
+IMPORT /FILE='wiz.por'.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv import-export.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+X,Y
+1.00,2.00
+3.00,. @&t@
+5.00,6.00
+])
+AT_CLEANUP
+
+AT_BANNER([SAVE])
+
+# UNSELECTED=DELETE used to cause a crash if there was actually a
+# filter variable.
+AT_SETUP([SAVE -- delete unselected])
+AT_DATA([data.txt], [dnl
+0 '1 9:30:05' 1/2/2003 "25/8/1995 15:30:00" "'a,b,c'",0
+, '-0 5:17' 10/31/2010 "9/4/2008 9:29:00" " xxx ",1
+1.625,'0 12:00',,,xyzzy,1
+])
+AT_DATA([save.pspp], [dnl
+SET DECIMAL=DOT.
+DATA LIST LIST NOTABLE FILE="data.txt"
+ /number(F8.3) time(DTIME10) date(ADATE10) datetime(DATETIME20) string(A8)
+ filter(F1.0).
+MISSING VALUES number(0) time('0 12:00') string('xyzzy').
+FILTER BY filter.
+SAVE /OUTFILE="data.sav" /UNSELECTED=DELETE.
+])
+AT_DATA([get.pspp], [dnl
+GET FILE='data.sav'.
+LIST.
+])
+AT_CHECK([pspp -O format=csv save.pspp])
+AT_CHECK([pspp -O format=csv get.pspp], [0], [dnl
+Table: Data List
+number,time,date,datetime,string,filter
+. ,-0 05:17,10/31/2010,09-APR-2008 09:29:00,xxx,1
+1.625,0 12:00:00,.,.,xyzzy,1
+])
+AT_CLEANUP
+
+AT_SETUP([SAVE RENAME with TO])
+AT_DATA([save-rename-to.sps], [dnl
+data list notable list /a b c fxo9*.
+begin data
+1 2 3 8
+end data.
+
+SAVE OUTFILE = "renamed.sav"
+ /RENAME=(A B C = fdo9 TO fdo11).
+
+
+NEW FILE.
+GET FILE = "renamed.sav".
+LIST.
+])
+
+AT_CHECK([pspp -O format=csv save-rename-to.sps], [0], [dnl
+Table: Data List
+fdo9,fdo10,fdo11,fxo9
+1.00,2.00,3.00,8.00
+])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([FILTER])
+
+AT_SETUP([FILTER])
+AT_DATA([filter.sps], [dnl
+data list notable /X 1-2.
+begin data.
+1
+2
+3
+4
+5
+6
+7
+8
+9
+10
+end data.
+compute FILTER_$ = mod(x,2).
+
+filter by filter_$.
+list.
+filter off.
+list.
+compute filter_$ = 1 - filter_$.
+filter by filter_$.
+list.
+])
+AT_CHECK([pspp -o pspp.csv filter.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+X,FILTER_$
+1,1.00
+3,1.00
+5,1.00
+7,1.00
+9,1.00
+
+Table: Data List
+X,FILTER_$
+1,1.00
+2,.00
+3,1.00
+4,.00
+5,1.00
+6,.00
+7,1.00
+8,.00
+9,1.00
+10,.00
+
+Table: Data List
+X,FILTER_$
+2,1.00
+4,1.00
+6,1.00
+8,1.00
+10,1.00
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SET])
+
+# This crashed older versions of PSPP (bug #30682).
+AT_SETUP([SET FORMAT to invalid output format])
+AT_DATA([set.pspp], [dnl
+DATA LIST LIST NOTABLE /x.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+SET FORMAT F41.
+DESCRIPTIVES /x.
+])
+AT_CHECK([pspp -O format=csv set.pspp], [1], [dnl
+"set.pspp:7.12-7.14: error: SET: Output format F41.0 specifies width 41, but F requires a width between 1 and 40.
+ 7 | SET FORMAT F41.
+ | ^~~"
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+x,3,2.00,1.00,1.00,3.00
+Valid N (listwise),3,,,,
+Missing N (listwise),0,,,,
+])
+AT_CLEANUP
+
+
+dnl This scenario was observed to erroneously free things twice
+AT_SETUP([SET crash on invalid cc])
+AT_DATA([set.pspp], [dnl
+SET CCA='xxxx'.SHGW CCA.
+])
+
+AT_CHECK([pspp -O format=csv set.pspp], [1], [dnl
+"set.pspp:1.9-1.14: error: SET: Custom currency string `CCA' for xxxx does not contain exactly three periods or commas (or it contains both).
+ 1 | SET CCA='xxxx'.SHGW CCA.
+ | ^~~~~~"
+])
+AT_CLEANUP
+
+
+
+AT_SETUP([SET MXWARNS])
+dnl Make sure that syntax processing stops and that
+dnl a warning is issued when the MXWARNS figure is
+dnl exceeded.
+AT_DATA([set.pspp], [dnl
+set mxwarns=2.
+data list notable list /x (f8.2) y (f8.2).
+begin data
+1 2
+3 r
+5 x
+q 8
+9 9
+3 x
+w w
+end data.
+
+comment The following line should not be executed.
+list.
+])
+
+AT_CHECK([pspp -O format=csv set.pspp], [0], [dnl
+set.pspp:5.3: warning: Data for variable y is not valid as format F: Field contents are not numeric.
+
+set.pspp:6.3: warning: Data for variable y is not valid as format F: Field contents are not numeric.
+
+set.pspp:7.1: warning: Data for variable x is not valid as format F: Field contents are not numeric.
+
+note: Warnings (3) exceed limit (2). Syntax processing will be halted.
+])
+
+AT_CLEANUP
+
+
+
+
+AT_SETUP([SET MXWARNS special case zero])
+dnl Make sure that MXWARNS interprets zero as infinity.
+AT_DATA([mxwarns.pspp], [dnl
+set mxwarns=0.
+data list notable list /x (f8.2) y (f8.2) z *.
+begin data
+1 2 3
+3 r 3
+5 x 3
+q 8 4
+9 9 4
+3 x 4
+w w 4
+end data.
+
+list.
+])
+
+AT_CHECK([pspp -O format=csv mxwarns.pspp], [0],
+[warning: MXWARNS set to zero. No further warnings will be given even when potentially problematic situations are encountered.
+
+Table: Data List
+x,y,z
+1.00,2.00,3.00
+3.00,. ,3.00
+5.00,. ,3.00
+. ,8.00,4.00
+9.00,9.00,4.00
+3.00,. ,4.00
+. ,. ,4.00
+])
+
+AT_CLEANUP
+
+AT_SETUP([SET macro - MEXPAND MPRINT MITERATE MNEST])
+AT_DATA([set-macro.sps], [dnl
+show mexpand mprint miterate mnest.
+preserve.
+set mexpand=off mprint=on miterate=10 mnest=11.
+show mexpand mprint miterate mnest.
+restore.
+show mexpand mprint miterate mnest.
+])
+AT_CHECK([pspp -O format=csv set-macro.sps], [0], [dnl
+Table: Settings
+MEXPAND,ON
+MPRINT,OFF
+MITERATE,1000
+MNEST,50
+
+Table: Settings
+MEXPAND,OFF
+MPRINT,ON
+MITERATE,10
+MNEST,11
+
+Table: Settings
+MEXPAND,ON
+MPRINT,OFF
+MITERATE,1000
+MNEST,50
+])
+AT_CLEANUP
+
+AT_SETUP([SET syntax errors])
+AT_DATA([set.sps], [dnl
+SET **.
+SET BASETEXTDIRECTION=**.
+SET BLANKS=**.
+SET BOX=**.
+SET CACHE=**.
+SET CCA=**.
+SET CELLSBREAK=**.
+SET CMPTRANS=**.
+SET COMPRESSION=**.
+SET CTEMPLATE=**.
+SET DECIMAL=**.
+SET EPOCH=**.
+SET EPOCH=1234.
+SET ERRORS=**.
+SET FORMAT=**.
+SET FORMAT=A8.
+SET FORMAT=F1.2.
+SET FUZZBITS=40.
+SET HEADER=**.
+SET INCLUDE=**.
+SET JOURNAL=**.
+SET LEADZERO=**.
+SET LENGTH=**.
+SET LOCALE='Neverland'.
+SET LOCALE=**.
+SET MDISPLAY=**.
+SET MESSAGES=**.
+SET MEXPAND=**.
+SET MITERATE=0.
+SET MNEST=0.
+SET MPRINT=**.
+SET MXERRS=0.
+SET MXLOOPS=0.
+SET MXWARNS=-1.
+SET PRINTBACK=**.
+SET RESULTS=**.
+SET RIB=**.
+SET RRB=**.
+SET SAFER=**.
+SET SCOMPRESSION=**.
+SET SEED=**.
+SET SMALL=**.
+SET SUBTITLE=**.
+SET TNUMBERS=**.
+SET TVARS=**.
+SET TLOOK='nonexistent.xml'.
+SET UNDEFINED=**.
+SET WIB=**.
+SET WRB=**.
+SET WIDTH=**.
+SET WORKSPACE=**.
+])
+AT_CHECK([pspp -O format=csv set.sps], [1], [dnl
+"set.sps:1.5-1.6: error: SET: Syntax error expecting the name of a setting.
+ 1 | SET **.
+ | ^~"
+
+"set.sps:2.5-2.24: warning: SET: BASETEXTDIRECTION is not yet implemented.
+ 2 | SET BASETEXTDIRECTION=**.
+ | ^~~~~~~~~~~~~~~~~~~~"
+
+"set.sps:3.12-3.13: error: SET: Syntax error expecting number.
+ 3 | SET BLANKS=**.
+ | ^~"
+
+"set.sps:4.5-4.10: warning: SET: BOX is not yet implemented.
+ 4 | SET BOX=**.
+ | ^~~~~~"
+
+"set.sps:5.5-5.12: warning: SET: CACHE is not yet implemented.
+ 5 | SET CACHE=**.
+ | ^~~~~~~~"
+
+"set.sps:6.9-6.10: error: SET: Syntax error expecting string.
+ 6 | SET CCA=**.
+ | ^~"
+
+"set.sps:7.5-7.17: warning: SET: CELLSBREAK is not yet implemented.
+ 7 | SET CELLSBREAK=**.
+ | ^~~~~~~~~~~~~"
+
+"set.sps:8.5-8.15: warning: SET: CMPTRANS is not yet implemented.
+ 8 | SET CMPTRANS=**.
+ | ^~~~~~~~~~~"
+
+"set.sps:9.5-9.18: warning: SET: COMPRESSION is not yet implemented.
+ 9 | SET COMPRESSION=**.
+ | ^~~~~~~~~~~~~~"
+
+"set.sps:10.5-10.16: warning: SET: CTEMPLATE is not yet implemented.
+ 10 | SET CTEMPLATE=**.
+ | ^~~~~~~~~~~~"
+
+"set.sps:11.13-11.14: error: SET: Syntax error expecting DOT or COMMA.
+ 11 | SET DECIMAL=**.
+ | ^~"
+
+"set.sps:12.11-12.12: error: SET: Syntax error expecting AUTOMATIC or year.
+ 12 | SET EPOCH=**.
+ | ^~"
+
+"set.sps:13.11-13.14: error: SET: Syntax error expecting integer 1500 or greater for EPOCH.
+ 13 | SET EPOCH=1234.
+ | ^~~~"
+
+"set.sps:14.12-14.13: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
+ 14 | SET ERRORS=**.
+ | ^~"
+
+"set.sps:15.12-15.13: error: SET: Syntax error expecting valid format specifier.
+ 15 | SET FORMAT=**.
+ | ^~"
+
+"set.sps:16.5-16.13: error: SET: FORMAT requires numeric output format as an argument. Specified format A8 is of type string.
+ 16 | SET FORMAT=A8.
+ | ^~~~~~~~~"
+
+"set.sps:17.12-17.15: error: SET: Output format F1.2 specifies 2 decimal places, but width 1 does not allow for any decimals.
+ 17 | SET FORMAT=F1.2.
+ | ^~~~"
+
+"set.sps:18.14-18.15: error: SET: Syntax error expecting integer between 0 and 20 for FUZZBITS.
+ 18 | SET FUZZBITS=40.
+ | ^~"
+
+"set.sps:19.5-19.13: warning: SET: HEADER is not yet implemented.
+ 19 | SET HEADER=**.
+ | ^~~~~~~~~"
+
+"set.sps:20.13-20.14: error: SET: Syntax error expecting ON, YES, OFF, or NO.
+ 20 | SET INCLUDE=**.
+ | ^~"
+
+"set.sps:21.13-21.14: error: SET: Syntax error expecting ON or OFF or a file name.
+ 21 | SET JOURNAL=**.
+ | ^~"
+
+"set.sps:22.14-22.15: error: SET: Syntax error expecting ON, YES, OFF, or NO.
+ 22 | SET LEADZERO=**.
+ | ^~"
+
+"set.sps:23.12-23.13: error: SET: Syntax error expecting positive integer for LENGTH.
+ 23 | SET LENGTH=**.
+ | ^~"
+
+"set.sps:24.12-24.22: error: SET: Neverland is not a recognized encoding or locale name.
+ 24 | SET LOCALE='Neverland'.
+ | ^~~~~~~~~~~"
+
+"set.sps:25.12-25.13: error: SET: Syntax error expecting string.
+ 25 | SET LOCALE=**.
+ | ^~"
+
+"set.sps:26.14-26.15: error: SET: Syntax error expecting TEXT or TABLES.
+ 26 | SET MDISPLAY=**.
+ | ^~"
+
+"set.sps:27.14-27.15: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
+ 27 | SET MESSAGES=**.
+ | ^~"
+
+"set.sps:28.13-28.14: error: SET: Syntax error expecting ON, YES, OFF, or NO.
+ 28 | SET MEXPAND=**.
+ | ^~"
+
+"set.sps:29.14: error: SET: Syntax error expecting positive integer for MITERATE.
+ 29 | SET MITERATE=0.
+ | ^"
+
+"set.sps:30.11: error: SET: Syntax error expecting positive integer for MNEST.
+ 30 | SET MNEST=0.
+ | ^"
+
+"set.sps:31.12-31.13: error: SET: Syntax error expecting ON, YES, OFF, or NO.
+ 31 | SET MPRINT=**.
+ | ^~"
+
+"set.sps:32.12: error: SET: Syntax error expecting positive integer for MXERRS.
+ 32 | SET MXERRS=0.
+ | ^"
+
+"set.sps:33.13: error: SET: Syntax error expecting positive integer for MXLOOPS.
+ 33 | SET MXLOOPS=0.
+ | ^"
+
+"set.sps:34.13-34.14: error: SET: Syntax error expecting non-negative integer for MXWARNS.
+ 34 | SET MXWARNS=-1.
+ | ^~"
+
+"set.sps:35.15-35.16: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
+ 35 | SET PRINTBACK=**.
+ | ^~"
+
+"set.sps:36.13-36.14: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
+ 36 | SET RESULTS=**.
+ | ^~"
+
+"set.sps:37.9-37.10: error: SET: Syntax error expecting MSBFIRST, LSBFIRST, VAX, or NATIVE.
+ 37 | SET RIB=**.
+ | ^~"
+
+"set.sps:38.9-38.10: error: SET: Syntax error expecting one of the following: NATIVE, ISL, ISB, IDL, IDB, VF, VD, VG, ZS.
+ 38 | SET RRB=**.
+ | ^~"
+
+"set.sps:39.11-39.12: error: SET: Syntax error expecting ON or YES.
+ 39 | SET SAFER=**.
+ | ^~"
+
+"set.sps:40.18-40.19: error: SET: Syntax error expecting ON, YES, OFF, or NO.
+ 40 | SET SCOMPRESSION=**.
+ | ^~"
+
+"set.sps:41.10-41.11: error: SET: Syntax error expecting number.
+ 41 | SET SEED=**.
+ | ^~"
+
+"set.sps:42.11-42.12: error: SET: Syntax error expecting number.
+ 42 | SET SMALL=**.
+ | ^~"
+
+"set.sps:43.5-43.12: error: SET: Syntax error expecting the name of a setting.
+ 43 | SET SUBTITLE=**.
+ | ^~~~~~~~"
+
+"set.sps:44.14-44.15: error: SET: Syntax error expecting LABELS, VALUES, or BOTH.
+ 44 | SET TNUMBERS=**.
+ | ^~"
+
+"set.sps:45.11-45.12: error: SET: Syntax error expecting LABELS, NAMES, or BOTH.
+ 45 | SET TVARS=**.
+ | ^~"
+
+set.sps:46: error: SET: nonexistent.xml: not found
+
+"set.sps:47.15-47.16: error: SET: Syntax error expecting WARN or NOWARN.
+ 47 | SET UNDEFINED=**.
+ | ^~"
+
+"set.sps:48.9-48.10: error: SET: Syntax error expecting MSBFIRST, LSBFIRST, VAX, or NATIVE.
+ 48 | SET WIB=**.
+ | ^~"
+
+"set.sps:49.9-49.10: error: SET: Syntax error expecting one of the following: NATIVE, ISL, ISB, IDL, IDB, VF, VD, VG, ZS.
+ 49 | SET WRB=**.
+ | ^~"
+
+"set.sps:50.11-50.12: error: SET: Syntax error expecting integer 40 or greater for WIDTH.
+ 50 | SET WIDTH=**.
+ | ^~"
+
+"set.sps:51.15-51.16: error: SET: Syntax error expecting integer 1024 or greater for WORKSPACE.
+ 51 | SET WORKSPACE=**.
+ | ^~"
+])
+AT_CLEANUP
+\f
+AT_BANNER([PRESERVE and RESTORE])
+
+AT_SETUP([PRESERVE of SET FORMAT])
+AT_DATA([set.pspp], [dnl
+SHOW FORMAT.
+PRESERVE.
+SET FORMAT F10.0.
+SHOW FORMAT
+RESTORE.
+SHOW FORMAT.
+])
+AT_CHECK([pspp -O format=csv set.pspp], [0], [dnl
+Table: Settings
+FORMAT,F8.2
+
+Table: Settings
+FORMAT,F10.0
+
+Table: Settings
+FORMAT,F8.2
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SHOW])
+
+AT_SETUP([SHOW N])
+
+AT_DATA([show.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+
+SHOW N.
+])
+
+AT_CHECK([pspp -O format=csv show.sps], [0], [dnl
+Table: Settings
+N,3
+])
+
+AT_CLEANUP
+
+
+AT_SETUP([SHOW N empty])
+
+AT_DATA([shown-empty.sps], [dnl
+SHOW N.
+])
+
+AT_CHECK([pspp -O format=csv shown-empty.sps], [0], [dnl
+Table: Settings
+N,Unknown
+])
+
+AT_CLEANUP
+
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SORT CASES])
+
+m4_divert_push([PREPARE_TESTS])
+[sort_cases_gen_data () {
+ cat > gen-data.py <<'EOF'
+#! /usr/bin/python3
+
+import random
+import sys
+
+data = []
+for i in range(int(sys.argv[1])):
+ data += [i] * int(sys.argv[2])
+random.shuffle(data)
+
+data_txt = open('data.txt', 'w')
+for i, item in enumerate(data):
+ data_txt.write('%s %s\n' % (item, i))
+data_txt.close()
+
+shuffled = ((item, i) for i, item in enumerate(data))
+expout = open('expout', 'w')
+for item, i in sorted(shuffled):
+ expout.write(' %8d %8d \n' % (item, i))
+expout.close()
+EOF
+ $PYTHON3 gen-data.py "$@"]
+}
+m4_divert_pop([PREPARE_TESTS])
+
+m4_define([SORT_CASES_TEST],
+ [AT_SETUP([sort m4_eval([$1 * $2]) cases[]m4_if([$2], [1], [], [ ($1 unique)])[]m4_if([$3], [], [], [ with $3 buffers])])
+ AT_KEYWORDS([SORT CASES $4])
+ AT_CHECK([sort_cases_gen_data $1 $2 $3])
+ AT_CAPTURE_FILE([data.txt])
+ AT_CAPTURE_FILE([output.txt])
+ AT_CAPTURE_FILE([sort-cases.sps])
+ AT_DATA([sort-cases.sps], [dnl
+DATA LIST LIST NOTABLE FILE='data.txt'/x y (F8).
+SORT CASES BY x[]m4_if([$3], [], [], [/BUFFERS=$3]).
+PRINT OUTFILE='output.txt'/x y.
+EXECUTE.
+])
+ AT_CHECK([pspp --testing-mode -o pspp.csv sort-cases.sps])
+ AT_CHECK([cat output.txt], [0], [expout])
+ AT_CLEANUP])
+
+SORT_CASES_TEST(100, 5, 2)
+SORT_CASES_TEST(100, 5, 3)
+SORT_CASES_TEST(100, 5, 4)
+SORT_CASES_TEST(100, 5, 5)
+SORT_CASES_TEST(100, 5, 10)
+SORT_CASES_TEST(100, 5, 50)
+SORT_CASES_TEST(100, 5, 100)
+SORT_CASES_TEST(100, 5)
+
+SORT_CASES_TEST(100, 10, 2)
+SORT_CASES_TEST(100, 10, 3)
+SORT_CASES_TEST(100, 10, 5)
+SORT_CASES_TEST(100, 10)
+
+SORT_CASES_TEST(1000, 5, 5, slow)
+SORT_CASES_TEST(1000, 5, 50, slow)
+SORT_CASES_TEST(1000, 5, [], slow)
+
+SORT_CASES_TEST(100, 100, 3, slow)
+SORT_CASES_TEST(100, 100, 5, slow)
+SORT_CASES_TEST(100, 100, [], slow)
+
+SORT_CASES_TEST(10000, 5, 500, slow)
+
+SORT_CASES_TEST(50000, 1, [], slow)
+
+dnl Bug #33089 caused SORT CASES to delete filtered cases permanently.
+AT_SETUP([SORT CASES preserves filtered cases])
+AT_DATA([sort-cases.sps], [dnl
+DATA LIST FREE /x.
+BEGIN DATA.
+5 4 3 2 1 0
+END DATA.
+COMPUTE mod2 = MOD(x, 2).
+LIST.
+FILTER BY mod2.
+LIST.
+SORT CASES BY x.
+LIST.
+FILTER OFF.
+LIST.
+])
+AT_CHECK([pspp -O format=csv sort-cases.sps], [0], [dnl
+Table: Data List
+x,mod2
+5.00,1.00
+4.00,.00
+3.00,1.00
+2.00,.00
+1.00,1.00
+.00,.00
+
+Table: Data List
+x,mod2
+5.00,1.00
+3.00,1.00
+1.00,1.00
+
+Table: Data List
+x,mod2
+1.00,1.00
+3.00,1.00
+5.00,1.00
+
+Table: Data List
+x,mod2
+.00,.00
+1.00,1.00
+2.00,.00
+3.00,1.00
+4.00,.00
+5.00,1.00
+])
+AT_CLEANUP
+
+AT_SETUP([SORT CASES syntax errors])
+AT_DATA([sort-cases.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+SORT CASES BY **.
+SORT CASES BY x(**).
+SORT CASES BY x(D**).
+SORT CASES BY x(A) x(D) y(**).
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='sort-cases.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"sort-cases.sps:2.15-2.16: error: SORT CASES: Syntax error expecting variable name.
+ 2 | SORT CASES BY **.
+ | ^~"
+
+"sort-cases.sps:3.17-3.18: error: SORT CASES: Syntax error expecting A or D.
+ 3 | SORT CASES BY x(**).
+ | ^~"
+
+"sort-cases.sps:4.18-4.19: error: SORT CASES: Syntax error expecting `)'.
+ 4 | SORT CASES BY x(D**).
+ | ^~"
+
+"sort-cases.sps:5.15-5.23: warning: SORT CASES: Variable x specified twice in sort criteria.
+ 5 | SORT CASES BY x(A) x(D) y(**).
+ | ^~~~~~~~~"
+
+"sort-cases.sps:5.27-5.28: error: SORT CASES: Syntax error expecting A or D.
+ 5 | SORT CASES BY x(A) x(D) y(**).
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SORT VARIABLES])
+
+AT_SETUP([SORT VARIABLES])
+# This reverses the order of its input lines.
+# From the GNU sed manual.
+tac () {
+ sed -n '1! G
+$ p
+h'
+}
+
+test_sort () {
+ cat > sort-variables.sps <<EOF
+DATA LIST FREE/$1.
+$2
+SORT VARIABLES $3.
+DISPLAY NAMES.
+SORT VARIABLES $3(D).
+DISPLAY NAMES.
+EOF
+ AT_CHECK_UNQUOTED([pspp -O format=csv sort-variables.sps], [0],
+[Table: Variables
+Name
+`for var in $4; do echo $var; done`
+
+Table: Variables
+Name
+`for var in $4; do echo $var; done | tac`
+])
+}
+
+test_sort 'x100 c b x99 a y400 y5' '' NAME 'a b c x99 x100 y5 y400'
+test_sort 'c(a10) a(a5) b' '' TYPE 'b a c'
+test_sort 'a (datetime) b (f) c (a5) d (a2) e (a1)' '' FORMAT 'e d c b a'
+test_sort 'a b c' \
+ 'VARIABLE LABEL a "hi there".' \
+ LABEL 'b c a'
+test_sort 'a b c' \
+ 'VALUE LABELS a 123 "xyzzy".' \
+ VALUES 'b c a'
+test_sort 'a b c' \
+ 'MISSING VALUES a (123).' \
+ MISSING 'b c a'
+test_sort 'a b c' \
+ 'VARIABLE LEVEL a (SCALE) b (ORDINAL) c (NOMINAL).' \
+ MEASURE 'c b a'
+test_sort 'b n i s t p' \
+ 'VARIABLE ROLE /INPUT i /TARGET t /BOTH b /NONE n /PARTITION p /SPLIT s.' \
+ ROLE 'i t b n p s'
+test_sort 'c10 c5 c15 c9' \
+ 'VARIABLE WIDTH c10(10) c5(5) c15(15) c9(9).' \
+ COLUMNS 'c5 c9 c10 c15'
+test_sort 'c l r' \
+ 'VARIABLE ALIGNMENT c (CENTER) l (LEFT) r (RIGHT).' \
+ ALIGNMENT 'l r c'
+test_sort 'az ax ay ab' \
+ 'VARIABLE ATTRIBUTE VARIABLES=az ATTRIBUTE=key("z").
+ VARIABLE ATTRIBUTE VARIABLES=ax ATTRIBUTE=key("x").
+ VARIABLE ATTRIBUTE VARIABLES=ay ATTRIBUTE=key("y").
+ VARIABLE ATTRIBUTE VARIABLES=ab ATTRIBUTE=key("b").' \
+ 'ATTRIBUTE key' 'ab ax ay az'
+AT_CLEANUP
+
+AT_SETUP([SORT VARIABLES syntax errors])
+AT_DATA([sort-variables.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+SORT VARIABLES BY **.
+SORT VARIABLES BY ATTRIBUTE **.
+SORT VARIABLES BY NAME (**).
+SORT VARIABLES BY NAME (A **).
+])
+AT_CHECK([pspp -O format=csv sort-variables.sps], [1], [dnl
+"sort-variables.sps:2.19-2.20: error: SORT VARIABLES: Syntax error expecting one of the following: NAME, TYPE, FORMAT, LABEL, VALUES, MISSING, MEASURE, ROLE, COLUMNS, ALIGNMENT, ATTRIBUTE.
+ 2 | SORT VARIABLES BY **.
+ | ^~"
+
+"sort-variables.sps:3.29-3.30: error: SORT VARIABLES: Syntax error expecting identifier.
+ 3 | SORT VARIABLES BY ATTRIBUTE **.
+ | ^~"
+
+"sort-variables.sps:4.25-4.26: error: SORT VARIABLES: Syntax error expecting A or D.
+ 4 | SORT VARIABLES BY NAME (**).
+ | ^~"
+
+"sort-variables.sps:5.27-5.28: error: SORT VARIABLES: Syntax error expecting `)'.
+ 5 | SORT VARIABLES BY NAME (A **).
+ | ^~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SPLIT FILE])
+
+AT_SETUP([SPLIT FILE - basic test])
+AT_DATA([split-file.sps], [dnl
+title 'Test SPLIT FILE utility'.
+
+data list notable /X 1 Y 2.
+begin data.
+12
+16
+17
+19
+15
+14
+27
+20
+26
+25
+28
+29
+24
+end data.
+split file by x.
+list.
+])
+AT_CHECK([pspp -o pspp.csv split-file.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Split Values
+Variable,Value
+X,1
+
+Table: Data List
+X,Y
+1,2
+1,6
+1,7
+1,9
+1,5
+1,4
+
+Table: Split Values
+Variable,Value
+X,2
+
+Table: Data List
+X,Y
+2,7
+2,0
+2,6
+2,5
+2,8
+2,9
+2,4
+])
+AT_CLEANUP
+
+AT_SETUP([SPLIT FILE - vs procedures])
+AT_DATA([split-file.sps], [dnl
+
+* This test is a compendium of those procedures which might
+* have problems when run in conjunction with SPLITS.
+
+data list list /a b c q g *.
+begin data.
+1 2 3 1 0
+4 5 6 0 0
+7 8 9 1 0
+10 11 12 0 1
+13 14 15 1 1
+end data.
+
+split file by g.
+
+
+* The commented out lines are ones which currently fail.
+
+AGGREGATE outfile='foo' /break=c /X=sum(a).
+AUTORECODE variables = c into d .
+COUNT e = c (1 2 3 4 5 6 7).
+CROSSTABS a by b.
+CORRELATIONS /VARIABLES = a b.
+DELETE VARIABLES a.
+DESCRIPTIVES e .
+EXAMINE c by b.
+EXPORT outfile='xxx'.
+FACTOR /VARIABLES = b c d.
+FILTER BY c.
+FREQUENCIES b.
+GLM c BY b.
+GRAPH /HISTOGRAM = b .
+GRAPH /SCATTERPLOT(BIVARIATE) = b with c by e .
+*GRAPH /BAR (GROUPED) = MEAN(b) by c by e.
+GRAPH /BAR = COUNT BY b.
+LIST.
+LOGISTIC REGRESSION q WITH b.
+MEANS c b.
+NPAR TESTS /MCNEMAR q.
+ONEWAY c BY b.
+QUICK CLUSTER b c.
+RANK b c.
+REGRESSION /VARIABLES = c /DEPENDENT = q.
+RELIABILITY /VARIABLES = c b d.
+RENAME VARIABLES (b = bb).
+ROC bb by q(1).
+SAMPLE 0.9 .
+SAVE outfile='xx.sav'.
+SORT CASES by bb.
+T-TEST /GROUP=q(0,1) /VARIABLES=bb.
+USE ALL.
+FLIP /VARIABLES = bb, c .
+
+execute.
+finish.
+])
+
+AT_CHECK([pspp -O format=csv split-file.sps], [0],[ignore])
+
+AT_CLEANUP
+
+AT_SETUP([SPLIT FILE - split variable limit])
+AT_DATA([split-file.sps], [dnl
+DATA LIST LIST NOTABLE /V1 TO V9.
+SPLIT FILE BY V1 TO V9.
+])
+AT_CHECK([pspp split-file.sps], [1], [dnl
+split-file.sps:2.15-2.22: error: SPLIT FILE: At most 8 split variables may be
+specified.
+ 2 | SPLIT FILE BY V1 TO V9.
+ | ^~~~~~~~
+])
+AT_CLEANUP
--- /dev/null
+AT_BANNER([STRING])
+
+AT_SETUP([STRING])
+AT_DATA([string.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+STRING s1 (A8)/s2 (A1).
+DISPLAY DICTIONARY.
+])
+AT_CHECK([pspp -O format=csv string.sps], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+x,1,Unknown,Input,8,Right,F8.2,F8.2
+y,2,Unknown,Input,8,Right,F8.2,F8.2
+z,3,Unknown,Input,8,Right,F8.2,F8.2
+s1,4,Nominal,Input,8,Left,A8,A8
+s2,5,Nominal,Input,1,Left,A1,A1
+])
+AT_CLEANUP
+
+AT_SETUP([STRING syntax errors])
+AT_DATA([string.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+STRING **.
+STRING s **.
+STRING s (**).
+STRING s (F8).
+STRING s (AHEX1).
+STRING s (A8 **).
+STRING x (A8).
+])
+AT_CHECK([pspp -O format=csv string.sps], [1], [dnl
+"string.sps:2.8-2.9: error: STRING: Syntax error expecting variable name.
+ 2 | STRING **.
+ | ^~"
+
+"string.sps:3.10-3.11: error: STRING: Syntax error expecting `('.
+ 3 | STRING s **.
+ | ^~"
+
+"string.sps:4.11-4.12: error: STRING: Syntax error expecting valid format specifier.
+ 4 | STRING s (**).
+ | ^~"
+
+"string.sps:5.11-5.12: error: STRING: String variables are not compatible with numeric format F8.0.
+ 5 | STRING s (F8).
+ | ^~"
+
+"string.sps:6.11-6.15: error: STRING: Output format AHEX1 specifies width 1, but AHEX requires an even width.
+ 6 | STRING s (AHEX1).
+ | ^~~~~"
+
+"string.sps:7.14-7.15: error: STRING: Syntax error expecting `)'.
+ 7 | STRING s (A8 **).
+ | ^~"
+
+"string.sps:8.8: error: STRING: There is already a variable named x.
+ 8 | STRING x (A8).
+ | ^"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([SYSFILE INFO])
+
+AT_SETUP([SYSFILE INFO])
+AT_DATA([sysfile-info.sps], [dnl
+DATA LIST LIST /x * name (a10) .
+BEGIN DATA
+1 one
+2 two
+3 three
+END DATA.
+DOCUMENT A document.
+SAVE OUTFILE='pro.sav'.
+
+sysfile info file='pro.sav'.
+])
+AT_CHECK([pspp -o pspp.csv sysfile-info.sps])
+AT_CHECK(
+ [sed -e '/^Created,/d' \
+ -e '/^Endian,/d' \
+ -e '/^Integer Format,/d' \
+ -e '/^Real Format,/d' \
+ -e '/^Encoding,/d' \
+ -e 's/(Entered.*)/(Entered <date>)/' pspp.csv],
+ [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+name,A10
+
+Table: File Information
+File,pro.sav
+Variables,2
+Cases,3
+Type,SPSS System File
+Weight,Not weighted
+Compression,SAV
+Documents,"DOCUMENT A document.
+ (Entered <date>)"
+
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+x,1,Nominal,Input,8,Right,F8.2,F8.2
+name,2,Nominal,Input,10,Left,A10,A10
+])
+AT_CLEANUP
+
+AT_BANNER([DISPLAY])
+
+dnl DISPLAY DOCUMENTS is tested with commands for documents.
+
+AT_SETUP([DISPLAY FILE LABEL])
+AT_DATA([display.sps], [dnl
+DATA LIST LIST NOTABLE /x * name (a10) .
+
+DISPLAY FILE LABEL.
+
+FILE LABEL 'foo bar baz quux'.
+DISPLAY FILE LABEL.
+])
+AT_CHECK([pspp -O format=csv display.sps], [0], [dnl
+Table: File Label
+Label,(none)
+
+Table: File Label
+Label,foo bar baz quux
+])
+AT_CLEANUP
+
+dnl DISPLAY VECTORS is tested with commands for vectors.
+
+dnl DISPLAY ATTRIBUTES and @ATTRIBUTES are tested with commands for attributes.
+
+AT_SETUP([DISPLAY SCRATCH])
+AT_DATA([sysfile-info.sps], [dnl
+DATA LIST LIST NOTABLE /x * name (a10) .
+DISPLAY SCRATCH.
+COMPUTE #x=0.
+DISPLAY SCRATCH.
+])
+AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
+sysfile-info.sps:2: note: DISPLAY: No variables to display.
+
+Table: Variables
+Name
+#x
+])
+AT_CLEANUP
+
+AT_SETUP([DISPLAY INDEX])
+AT_DATA([sysfile-info.sps], [dnl
+DATA LIST LIST NOTABLE /x * name (a10) .
+DISPLAY INDEX.
+])
+AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
+Table: Variables
+Name,Position
+x,1
+name,2
+])
+AT_CLEANUP
+
+AT_SETUP([DISPLAY NAMES])
+AT_DATA([sysfile-info.sps], [dnl
+DATA LIST LIST NOTABLE /x * name (a10) .
+DISPLAY NAMES.
+])
+AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
+Table: Variables
+Name
+x
+name
+])
+AT_CLEANUP
+
+AT_SETUP([DISPLAY LABELS])
+AT_DATA([sysfile-info.sps], [dnl
+DATA LIST LIST NOTABLE /x * name (a10) .
+VARIABLE LABEL x 'variable one' name 'variable two'.
+VALUE LABEL x 1 'asdf' 2 'jkl;'.
+DISPLAY LABELS.
+])
+AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
+Table: Variables
+Name,Position,Label
+x,1,variable one
+name,2,variable two
+])
+AT_CLEANUP
+
+dnl DISPLAY VARIABLES Is tested in multiple places.
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([T-TEST])
+
+AT_SETUP([T-TEST /PAIRS])
+AT_DATA([t-test.sps], [dnl
+data list list /ID * A * B *.
+begin data.
+1 2.0 3.0
+2 1.0 2.0
+3 2.0 4.5
+4 2.0 4.5
+5 3.0 6.0
+end data.
+
+t-test /PAIRS a with b (PAIRED).
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+ID,F8.0
+A,F8.0
+B,F8.0
+
+Table: Paired Sample Statistics
+,,N,Mean,Std. Deviation,S.E. Mean
+Pair 1,A,5,2.00,.71,.32
+,B,5,4.00,1.54,.69
+
+Table: Paired Samples Correlations
+,,N,Correlation,Sig.
+Pair 1,A & B,5,.918,.028
+
+Table: Paired Samples Test
+,,Paired Differences,,,,,t,df,Sig. (2-tailed)
+,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
+,,,,,Lower,Upper,,,
+Pair 1,A - B,-2.00,.94,.42,-3.16,-.84,-4.78,4,.009
+])
+AT_CLEANUP
+
+
+AT_SETUP([T-TEST /PAIRS with per-analysis missing values])
+
+AT_DATA([ref.sps], [dnl
+data list list /id * a * b * c * d *.
+begin data.
+1 2.0 3.0 4.0 4.0
+2 1.0 2.0 5.1 3.9
+3 2.0 4.5 5.2 3.8
+4 2.0 4.5 5.3 3.7
+56 3.0 6.0 5.9 3.6
+end data.
+
+t-test /PAIRS a c with b d (PAIRED).
+])
+
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+id,F8.0
+a,F8.0
+b,F8.0
+c,F8.0
+d,F8.0
+
+Table: Paired Sample Statistics
+,,N,Mean,Std. Deviation,S.E. Mean
+Pair 1,a,5,2.00,.71,.32
+,b,5,4.00,1.54,.69
+Pair 2,c,5,5.10,.69,.31
+,d,5,3.80,.16,.07
+
+Table: Paired Samples Correlations
+,,N,Correlation,Sig.
+Pair 1,a & b,5,.918,.028
+Pair 2,c & d,5,-.918,.028
+
+Table: Paired Samples Test
+,,Paired Differences,,,,,t,df,Sig. (2-tailed)
+,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
+,,,,,Lower,Upper,,,
+Pair 1,a - b,-2.00,.94,.42,-3.16,-.84,-4.78,4,.009
+Pair 2,c - d,1.30,.84,.37,.26,2.34,3.47,4,.025
+])
+
+AT_CHECK([pspp -o ref.csv ref.sps])
+AT_CHECK([cat ref.csv], [0], [expout])
+AT_DATA([missing.sps], [dnl
+data list list /id * a * b * c * d *.
+begin data.
+1 2.0 3.0 4.0 4.0
+2 1.0 2.0 5.1 3.9
+3 2.0 4.5 5.2 3.8
+4 2.0 4.5 5.3 3.7
+5 3.0 6.0 5.9 .
+6 3.0 . 5.9 3.6
+end data.
+
+
+t-test /MISSING=analysis /PAIRS a c with b d (PAIRED) /CRITERIA=CI(0.95).
+])
+
+AT_CHECK([pspp -o missing.csv missing.sps])
+AT_CHECK([cat missing.csv], [0], [expout])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /PAIRS with listwise missing values])
+AT_DATA([ref.sps], [dnl
+data list list /id * a * b * c * d *.
+begin data.
+1 2.0 3.0 4.0 4.0
+2 1.0 2.0 5.1 3.9
+3 2.0 4.5 5.2 3.8
+4 2.0 4.5 5.3 3.7
+5 3.0 6.0 5.9 3.6
+end data.
+
+t-test /PAIRS a b with c d (PAIRED).
+])
+
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+id,F8.0
+a,F8.0
+b,F8.0
+c,F8.0
+d,F8.0
+
+Table: Paired Sample Statistics
+,,N,Mean,Std. Deviation,S.E. Mean
+Pair 1,a,5,2.00,.71,.32
+,c,5,5.10,.69,.31
+Pair 2,b,5,4.00,1.54,.69
+,d,5,3.80,.16,.07
+
+Table: Paired Samples Correlations
+,,N,Correlation,Sig.
+Pair 1,a & c,5,.410,.493
+Pair 2,b & d,5,-.872,.054
+
+Table: Paired Samples Test
+,,Paired Differences,,,,,t,df,Sig. (2-tailed)
+,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
+,,,,,Lower,Upper,,,
+Pair 1,a - c,-3.10,.76,.34,-4.04,-2.16,-9.14,4,.001
+Pair 2,b - d,.20,1.68,.75,-1.89,2.29,.27,4,.803
+])
+
+AT_CHECK([pspp -o ref.csv ref.sps])
+
+AT_CHECK([cat ref.csv], [0], [expout])
+
+AT_DATA([missing.sps], [dnl
+data list list /id * a * b * c * d *.
+begin data.
+1 2.0 3.0 4.0 4.0
+2 1.0 2.0 5.1 3.9
+3 2.0 4.5 5.2 3.8
+4 2.0 4.5 5.3 3.7
+5 3.0 6.0 5.9 3.6
+6 3.0 6.0 5.9 .
+end data.
+
+
+t-test /MISSING=listwise /PAIRS a b with c d (PAIRED).
+])
+AT_CHECK([pspp -o missing.csv missing.sps])
+AT_CHECK([cat missing.csv], [0], [expout])
+AT_CLEANUP
+
+
+dnl Tests for a bug in the paired samples T test when weighted
+dnl Thanks to Douglas Bonett for reporting this.
+AT_SETUP([T-TEST weighted paired bug])
+AT_DATA([t-test.sps], [dnl
+DATA LIST notable LIST /x y w *.
+BEGIN DATA.
+1 1 255
+1 2 43
+1 3 216
+2 1 3
+2 2 1
+2 3 12
+END DATA.
+
+WEIGHT BY w.
+
+T-TEST
+ PAIRS = y WITH x (PAIRED)
+ /MISSING=ANALYSIS
+ /CRITERIA=CI(0.95).
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Paired Sample Statistics
+,,N,Mean,Std. Deviation,S.E. Mean
+Pair 1,y,530.00,1.94,.96,.04
+,x,530.00,1.03,.17,.01
+
+Table: Paired Samples Correlations
+,,N,Correlation,Sig.
+Pair 1,y & x,530.00,.114,.008
+
+Table: Paired Samples Test
+,,Paired Differences,,,,,t,df,Sig. (2-tailed)
+,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
+,,,,,Lower,Upper,,,
+Pair 1,y - x,.91,.95,.04,.83,.99,22.07,529.00,.000
+])
+AT_CLEANUP
+
+
+dnl Tests for a bug in the paired samples T test.
+dnl Thanks to Mike Griffiths for reporting this problem.
+AT_SETUP([T-TEST /PAIRS bug])
+AT_DATA([t-test.sps], [dnl
+set format f8.3.
+data list list /A * B *.
+begin data.
+11 2
+1 1
+1 1
+end data.
+
+t-test pairs = a with b (paired).
+])
+AT_CHECK([pspp -o pspp.csv t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+A,F8.0
+B,F8.0
+
+Table: Paired Sample Statistics
+,,N,Mean,Std. Deviation,S.E. Mean
+Pair 1,A,3,4.333,5.774,3.333
+,B,3,1.333,.577,.333
+
+Table: Paired Samples Correlations
+,,N,Correlation,Sig.
+Pair 1,A & B,3,1.000,.000
+
+Table: Paired Samples Test
+,,Paired Differences,,,,,t,df,Sig. (2-tailed)
+,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
+,,,,,Lower,Upper,,,
+Pair 1,A - B,3.000,5.196,3.000,-9.908,15.908,1.000,2,.423
+])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /GROUPS])
+AT_DATA([t-test.sps], [dnl
+data list list /ID * INDEP * DEP1 * DEP2 *.
+begin data.
+1 1.1 1 3
+2 1.1 2 4
+3 1.1 2 4
+4 1.1 2 4
+5 1.1 3 5
+6 2.1 3 1
+7 2.1 4 2
+8 2.1 4 2
+9 2.1 4 2
+10 2.1 5 3
+11 3.1 2 2
+end data.
+
+* Note that the last case should be IGNORED since it doesn't have a
+ dependent variable of either 1.1 or 2.1.
+
+t-test /GROUPS=indep(1.1,2.1) /var=dep1 dep2.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+ID,F8.0
+INDEP,F8.0
+DEP1,F8.0
+DEP2,F8.0
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+DEP1,1.10,5,2.00,.71,.32
+,2.10,5,4.00,.71,.32
+DEP2,1.10,5,4.00,.71,.32
+,2.10,5,2.00,.71,.32
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+DEP1,Equal variances assumed,.00,1.000,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
+,Equal variances not assumed,,,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
+DEP2,Equal variances assumed,.00,1.000,4.47,8.00,.002,2.00,.45,.97,3.03
+,Equal variances not assumed,,,4.47,8.00,.002,2.00,.45,.97,3.03
+])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /GROUPS with one value for independent variable])
+AT_DATA([t-test.sps], [dnl
+data list list /INDEP * DEP *.
+begin data.
+ 1 6
+ 1 6
+ 1 7
+ 1 6
+ 1 13
+ 1 4
+ 1 7
+ 1 9
+ 1 7
+ 1 12
+ 1 11
+ 2 11
+ 2 9
+ 2 8
+ 2 4
+ 2 16
+ 2 9
+ 2 9
+ 2 5
+ 2 4
+ 2 10
+ 2 14
+end data.
+t-test /groups=indep(1.514) /var=dep.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+INDEP,F8.0
+DEP,F8.0
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+DEP,≥ 1.51,11,9.00,3.82,1.15
+,< 1.51,11,8.00,2.86,.86
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+DEP,Equal variances assumed,.17,.683,.69,20.00,.495,1.00,1.44,-2.00,4.00
+,Equal variances not assumed,,,.69,18.54,.496,1.00,1.44,-2.02,4.02
+])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /GROUPS with per-analysis missing values])
+AT_DATA([ref.sps], [dnl
+data list list /id * indep * dep1 * dep2 *.
+begin data.
+1 1.0 3.5 6
+2 1.0 2.0 5
+3 1.0 2.0 4
+4 2.0 3.5 3
+56 2.0 3.0 1
+end data.
+
+t-test /group=indep /var=dep1, dep2.
+])
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+id,F8.0
+indep,F8.0
+dep1,F8.0
+dep2,F8.0
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+dep1,1.00,3,2.50,.87,.50
+,2.00,2,3.25,.35,.25
+dep2,1.00,3,5.00,1.00,.58
+,2.00,2,2.00,1.41,1.00
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+dep1,Equal variances assumed,3.75,.148,-1.12,3.00,.346,-.75,.67,-2.89,1.39
+,Equal variances not assumed,,,-1.34,2.78,.279,-.75,.56,-2.61,1.11
+dep2,Equal variances assumed,.60,.495,2.85,3.00,.065,3.00,1.05,-.35,6.35
+,Equal variances not assumed,,,2.60,1.68,.144,3.00,1.15,-2.98,8.98
+])
+AT_CHECK([pspp -o ref.csv ref.sps])
+AT_CHECK([cat ref.csv], [0], [expout])
+AT_DATA([missing.sps], [dnl
+data list list /id * indep * dep1 * dep2.
+begin data.
+1 1.0 3.5 6
+2 1.0 2.0 5
+3 1.0 2.0 4
+4 2.0 3.5 3
+5 2.0 3.0 .
+6 2.0 . 1
+7 . 3.1 5
+end data.
+
+* Note that if the independent variable is missing, then it's implicitly
+* listwise missing.
+
+t-test /missing=analysis /group=indep /var=dep1 dep2.
+])
+AT_CHECK([pspp -o missing.csv missing.sps])
+AT_CHECK([cat missing.csv], [0], [expout])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /GROUPS with listwise missing values])
+AT_DATA([ref.sps], [dnl
+data list list /id * indep * dep1 * dep2.
+begin data.
+1 1.0 3.5 6
+2 1.0 2.0 5
+3 1.0 2.0 4
+4 2.0 3.5 3
+5 2.0 3.0 2
+6 2.0 4.0 1
+end data.
+
+t-test /group=indep /var=dep1 dep2.
+])
+
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+id,F8.0
+indep,F8.0
+dep1,F8.0
+dep2,F8.0
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+dep1,1.00,3,2.50,.87,.50
+,2.00,3,3.50,.50,.29
+dep2,1.00,3,5.00,1.00,.58
+,2.00,3,2.00,1.00,.58
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+dep1,Equal variances assumed,2.00,.230,-1.73,4.00,.158,-1.00,.58,-2.60,.60
+,Equal variances not assumed,,,-1.73,3.20,.176,-1.00,.58,-2.77,.77
+dep2,Equal variances assumed,.00,1.000,3.67,4.00,.021,3.00,.82,.73,5.27
+,Equal variances not assumed,,,3.67,4.00,.021,3.00,.82,.73,5.27
+])
+
+AT_CHECK([pspp -o ref.csv ref.sps])
+AT_CHECK([cat ref.csv], [0], [expout])
+AT_DATA([missing.sps], [dnl
+data list list /id * indep * dep1 * dep2 *.
+begin data.
+1 1.0 3.5 6
+2 1.0 2.0 5
+3 1.0 2.0 4
+4 2.0 3.5 3
+5 2.0 3.0 2
+6 2.0 4.0 1
+7 2.0 . 0
+end data.
+
+t-test /missing=listwise,exclude /group=indep /var=dep1, dep2.
+])
+AT_CHECK([pspp -o missing.csv missing.sps])
+AT_CHECK([cat missing.csv], [0], [expout])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /TESTVAL])
+AT_DATA([t-test.sps], [dnl
+data list list /ID * ABC *.
+begin data.
+1 3.5
+2 2.0
+3 2.0
+4 3.5
+5 3.0
+6 4.0
+end data.
+
+t-test /testval=2.0 /var=abc.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+ID,F8.0
+ABC,F8.0
+
+Table: One-Sample Statistics
+,N,Mean,Std. Deviation,S.E. Mean
+ABC,6,3.00,.84,.34
+
+Table: One-Sample Test
+,Test Value = 2,,,,,
+,t,df,Sig. (2-tailed),Mean Difference,95% Confidence Interval of the Difference,
+,,,,,Lower,Upper
+ABC,2.93,5,.033,1.00,.12,1.88
+])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /TESTVAL with per-analysis missing values])
+AT_DATA([ref.sps], [dnl
+data list list /id * x1 * x2.
+begin data.
+1 3.5 34
+2 2.0 10
+3 2.0 23
+4 3.5 98
+5 3.0 23
+67 4.0 8
+end data.
+
+t-test /testval=3.0 /var=x1 x2.
+])
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+id,F8.0
+x1,F8.0
+x2,F8.0
+
+Table: One-Sample Statistics
+,N,Mean,Std. Deviation,S.E. Mean
+x1,6,3.00,.84,.34
+x2,6,32.67,33.40,13.64
+
+Table: One-Sample Test
+,Test Value = 3,,,,,
+,t,df,Sig. (2-tailed),Mean Difference,95% Confidence Interval of the Difference,
+,,,,,Lower,Upper
+x1,.00,5,1.000,.00,-.88,.88
+x2,2.18,5,.082,29.67,-5.39,64.72
+])
+AT_CHECK([pspp -o ref.csv ref.sps])
+AT_CHECK([cat ref.csv], [0], [expout])
+AT_DATA([missing.sps], [dnl
+data list list /id * x1 * x2.
+begin data.
+1 3.5 34
+2 2.0 10
+3 2.0 23
+4 3.5 98
+5 3.0 23
+6 4.0 .
+7 . 8
+end data.
+
+t-test /missing=analysis /testval=3.0 /var=x1 x2.
+])
+AT_CHECK([pspp -o missing.csv missing.sps])
+AT_CHECK([cat missing.csv], [0], [expout])
+AT_CLEANUP
+
+AT_SETUP([T-TEST /TESTVAL with listwise missing values])
+AT_DATA([ref.sps], [dnl
+data list list /id * x1 * x2.
+begin data.
+1 3.5 34
+2 2.0 10
+3 2.0 23
+4 3.5 98
+5 3.0 23
+end data.
+
+t-test /testval=3.0 /var=x1 x2.
+])
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+id,F8.0
+x1,F8.0
+x2,F8.0
+
+Table: One-Sample Statistics
+,N,Mean,Std. Deviation,S.E. Mean
+x1,5,2.80,.76,.34
+x2,5,37.60,34.82,15.57
+
+Table: One-Sample Test
+,Test Value = 3,,,,,
+,t,df,Sig. (2-tailed),Mean Difference,95% Confidence Interval of the Difference,
+,,,,,Lower,Upper
+x1,-.59,4,.587,-.20,-1.14,.74
+x2,2.22,4,.090,34.60,-8.63,77.83
+])
+AT_CHECK([pspp -o ref.csv ref.sps])
+AT_CHECK([cat ref.csv], [0], [expout])
+AT_DATA([missing.sps], [dnl
+data list list /id * x1 * x2.
+begin data.
+1 3.5 34
+2 2.0 10
+3 2.0 23
+4 3.5 98
+5 3.0 23
+6 4.0 99
+end data.
+
+MISSING VALUES x2(99).
+
+t-test /missing=listwise /testval=3.0 /var=x1 x2.
+])
+AT_CHECK([pspp -o missing.csv missing.sps])
+AT_CHECK([cat missing.csv], [0], [expout])
+AT_CLEANUP
+
+AT_SETUP([T-TEST wih TEMPORARY transformation])
+AT_DATA([ref.sps], [dnl
+data list list /ind * x * .
+begin data.
+1 3.5
+1 2.0
+1 2.0
+2 3.5
+2 3.0
+2 4.0
+end data.
+
+t-test /groups=ind(1,2) /var x.
+])
+AT_DATA([expout], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+ind,F8.0
+x,F8.0
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+x,1.00,3,2.50,.87,.50
+,2.00,3,3.50,.50,.29
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+x,Equal variances assumed,2.00,.230,-1.73,4.00,.158,-1.00,.58,-2.60,.60
+,Equal variances not assumed,,,-1.73,3.20,.176,-1.00,.58,-2.77,.77
+])
+AT_CHECK([pspp -o ref.csv ref.sps])
+AT_CHECK([cat ref.csv], [0], [expout])
+AT_DATA([temporary.sps], [dnl
+data list list /ind * x * .
+begin data.
+1 3.5
+1 2.0
+1 2.0
+2 3.5
+2 3.0
+2 4.0
+2 9.0
+end data.
+
+TEMPORARY.
+SELECT IF x < 7.
+
+t-test /groups=ind(1 2) /var x.
+])
+AT_CHECK([pspp -o temporary.csv temporary.sps])
+AT_CHECK([cat temporary.csv], [0], [expout])
+AT_CLEANUP
+
+dnl This is an example from doc/tutorial.texi
+dnl So if the results of this have to be changed in any way,
+dnl make sure to update that file.
+AT_SETUP([T-TEST tutorial example])
+cp $top_srcdir/examples/physiology.sav .
+AT_DATA([t-test.sps], [dnl
+GET FILE='physiology.sav'.
+RECODE height (179 = SYSMIS).
+T-TEST GROUP=sex(0,1) /VARIABLES=height temperature.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+Height in millimeters ,Male,22,1796.49,49.71,10.60
+,Female,17,1610.77,25.43,6.17
+Internal body temperature in degrees Celcius,Male,22,36.68,1.95,.42
+,Female,18,37.43,1.61,.38
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+Height in millimeters ,Equal variances assumed,.97,.331,14.02,37.00,.000,185.72,13.24,158.88,212.55
+,Equal variances not assumed,,,15.15,32.71,.000,185.72,12.26,160.76,210.67
+Internal body temperature in degrees Celcius,Equal variances assumed,.31,.581,-1.31,38.00,.198,-.75,.57,-1.91,.41
+,Equal variances not assumed,,,-1.33,37.99,.190,-.75,.56,-1.89,.39
+])
+AT_CLEANUP
+
+dnl Tests for a bug which caused T-TEST to crash when given invalid syntax.
+AT_SETUP([T-TEST invalid syntax])
+AT_DATA([t-test.sps], [dnl
+DATA LIST LIST NOTABLE /id * a * .
+BEGIN DATA.
+1 3.5
+2 2.0
+3 2.0
+4 3.5
+5 3.0
+6 4.0
+END DATA.
+
+T-TEST /testval=2.0 .
+T-TEST /groups=id(3) .
+])
+AT_CHECK([pspp -O format=csv t-test.sps], [1], [dnl
+"t-test.sps:11.1-11.21: error: T-TEST: Required subcommand VARIABLES was not specified.
+ 11 | T-TEST /testval=2.0 .
+ | ^~~~~~~~~~~~~~~~~~~~~"
+
+"t-test.sps:12.1-12.22: error: T-TEST: Required subcommand VARIABLES was not specified.
+ 12 | T-TEST /groups=id(3) .
+ | ^~~~~~~~~~~~~~~~~~~~~~"
+])
+AT_CLEANUP
+
+dnl Tests for bug #11227, exhibited when the independent variable is a string.
+AT_SETUP([T-TEST string variable])
+AT_DATA([t-test.sps], [dnl
+data list list /ID * INDEP (a1) DEP1 * DEP2 *.
+begin data.
+1 'a' 1 3
+2 'a' 2 4
+3 'a' 2 4
+4 'a' 2 4
+5 'a' 3 5
+6 'b' 3 1
+7 'b' 4 2
+8 'b' 4 2
+9 'b' 4 2
+10 'b' 5 3
+11 'c' 2 2
+end data.
+
+
+t-test /GROUPS=indep('a','b') /var=dep1 dep2.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+ID,F8.0
+INDEP,A1
+DEP1,F8.0
+DEP2,F8.0
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+DEP1,a,5,2.00,.71,.32
+,b,5,4.00,.71,.32
+DEP2,a,5,4.00,.71,.32
+,b,5,2.00,.71,.32
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+DEP1,Equal variances assumed,.00,1.000,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
+,Equal variances not assumed,,,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
+DEP2,Equal variances assumed,.00,1.000,4.47,8.00,.002,2.00,.45,.97,3.03
+,Equal variances not assumed,,,4.47,8.00,.002,2.00,.45,.97,3.03
+])
+AT_CLEANUP
+
+AT_SETUP([T-TEST string variable, only one value])
+AT_DATA([t-test.sps], [dnl
+data list list notable /id * indep (a1) dep1 * dep2 *.
+begin data.
+1 'a' 1 3
+2 'a' 2 4
+3 'a' 2 4
+4 'a' 2 4
+5 'a' 3 5
+6 'b' 3 1
+7 'b' 4 2
+8 'b' 4 2
+9 'b' 4 2
+10 'b' 5 3
+11 'c' 2 2
+end data.
+
+
+t-test /GROUPS=indep('a') /var=dep1 dep2.
+])
+AT_CHECK([pspp -O format=csv t-test.sps], [1], [dnl
+"t-test.sps:17.16-17.25: error: T-TEST: When applying GROUPS to a string variable, two values must be specified.
+ 17 | t-test /GROUPS=indep('a') /var=dep1 dep2.
+ | ^~~~~~~~~~"
+])
+AT_CLEANUP
+
+dnl Tests for a bug which didn't properly compare string values.
+AT_SETUP([T-TEST string variable comparison bug])
+AT_DATA([t-test.sps], [dnl
+data list list /x * gv (a8).
+begin data.
+3 One
+2 One
+3 One
+2 One
+3 One
+4 Two
+3.5 Two
+3.0 Two
+end data.
+
+t-test group=gv('One', 'Two')
+ /variables = x.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading free-form data from INLINE.
+Variable,Format
+x,F8.0
+gv,A8
+
+Table: Group Statistics
+,Group,N,Mean,Std. Deviation,S.E. Mean
+x,One,5,2.60,.55,.24
+,Two,3,3.50,.50,.29
+
+Table: Independent Samples Test
+,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
+,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
+,,,,,,,,,Lower,Upper
+x,Equal variances assumed,1.13,.329,-2.32,6.00,.060,-.90,.39,-1.85,.05
+,Equal variances not assumed,,,-2.38,4.70,.067,-.90,.38,-1.89,.09
+])
+AT_CLEANUP
+
+
+
+dnl Tests for a bug assert failed when the group variables were not of either class
+AT_SETUP([T-TEST wrong group])
+AT_DATA([t-test-crs.sps], [dnl
+data list list /x * g *.
+begin data.
+1 2
+2 2
+3 2
+4 2
+5 2
+end data.
+
+t-test /variables = x group=g(1,3).
+])
+
+AT_CHECK([pspp t-test-crs.sps], [0],[ignore], [ignore])
+
+AT_CLEANUP
+
+
+
+dnl Tests for a bug assert failed when a non-number was passes as the p value
+AT_SETUP([T-TEST non number p value])
+AT_DATA([t.sps], [dnl
+data list list /age d_frage_1 weight height *.
+begin data.
+1 2 3 1
+4 5 6 2
+end data.
+
+T-TEST /VARIABLES=age weight height
+ /GROUPS=d_frage_1(1,0) /MISSING=ANALYSIS /CRITERIA=CIN(p.95).
+])
+
+AT_CHECK([pspp t.sps], [1],[ignore], [ignore])
+
+AT_CLEANUP
+
+
+
+dnl Another crash on invalid input
+AT_SETUP([T-TEST unterminated string - paired])
+AT_DATA([t.sps], [dnl
+data list list /id * a * b * c * d *.
+begin data.
+5 2.0 3.0 4.0 4.0
+3 1.0 2.0 5.1 3.9
+3 2.0 4.5 5.2(3.8
+4 2.0 4.5 5n3 3.7
+5 3.0 6.0 5.9 3.6
+6 3.4 6.0 5.9 .
+end data.
+
+
+t-test /MISSING=listwise /PAIRS a"b with c d (PA RED).
+]) dnl "
+
+AT_CHECK([pspp t.sps],[1],[ignore],[ignore])
+
+AT_CLEANUP
+
+AT_SETUP([T-TEST syntax errors])
+AT_DATA([t-test.sps], [dnl
+DATA LIST LIST NOTABLE/x y z * s(a10).
+T-TEST TESTVAL=**.
+T-TEST GROUPS=**.
+T-TEST GROUPS=x(**).
+T-TEST GROUPS=x(1,**).
+T-TEST GROUPS=x(1,2 **).
+T-TEST GROUPS=s('a').
+T-TEST VARIABLES=x y PAIRS.
+T-TEST PAIRS=**.
+T-TEST PAIRS=x WITH **.
+T-TEST PAIRS=x WITH y z (PAIRED).
+T-TEST PAIRS=x WITH y/VARIABLES.
+T-TEST VARIABLES=**.
+T-TEST MISSING=**.
+T-TEST CRITERIA=**.
+T-TEST CRITERIA=CIN**.
+T-TEST CRITERIA=CIN(**).
+T-TEST CRITERIA=CIN(1 **).
+T-TEST **.
+T-TEST MISSING=INCLUDE.
+T-TEST MISSING=INCLUDE/TESTVAL=1.
+])
+AT_CHECK([pspp -O format=csv t-test.sps], [1], [dnl
+"t-test.sps:2.16-2.17: error: T-TEST: Syntax error expecting number.
+ 2 | T-TEST TESTVAL=**.
+ | ^~"
+
+"t-test.sps:3.15-3.16: error: T-TEST: Syntax error expecting variable name.
+ 3 | T-TEST GROUPS=**.
+ | ^~"
+
+"t-test.sps:4.17-4.18: error: T-TEST: Syntax error expecting number.
+ 4 | T-TEST GROUPS=x(**).
+ | ^~"
+
+"t-test.sps:5.19-5.20: error: T-TEST: Syntax error expecting number.
+ 5 | T-TEST GROUPS=x(1,**).
+ | ^~"
+
+"t-test.sps:6.21-6.22: error: T-TEST: Syntax error expecting `)'.
+ 6 | T-TEST GROUPS=x(1,2 **).
+ | ^~"
+
+"t-test.sps:7.15-7.20: error: T-TEST: When applying GROUPS to a string variable, two values must be specified.
+ 7 | T-TEST GROUPS=s('a').
+ | ^~~~~~"
+
+"t-test.sps:8.22-8.26: error: T-TEST: VARIABLES subcommand may not be used with PAIRS.
+ 8 | T-TEST VARIABLES=x y PAIRS.
+ | ^~~~~"
+
+"t-test.sps:9.14-9.15: error: T-TEST: Syntax error expecting variable name.
+ 9 | T-TEST PAIRS=**.
+ | ^~"
+
+"t-test.sps:10.21-10.22: error: T-TEST: Syntax error expecting variable name.
+ 10 | T-TEST PAIRS=x WITH **.
+ | ^~"
+
+"t-test.sps:11.14-11.23: error: T-TEST: PAIRED was specified, but the number of variables preceding WITH (1) does not match the number following (2).
+ 11 | T-TEST PAIRS=x WITH y z (PAIRED).
+ | ^~~~~~~~~~"
+
+"t-test.sps:12.23-12.31: error: T-TEST: VARIABLES subcommand may not be used with PAIRS.
+ 12 | T-TEST PAIRS=x WITH y/VARIABLES.
+ | ^~~~~~~~~"
+
+"t-test.sps:13.18-13.19: error: T-TEST: Syntax error expecting variable name.
+ 13 | T-TEST VARIABLES=**.
+ | ^~"
+
+"t-test.sps:14.16-14.17: error: T-TEST: Syntax error expecting INCLUDE, EXCLUDE, LISTWISE, or ANALYSIS.
+ 14 | T-TEST MISSING=**.
+ | ^~"
+
+"t-test.sps:15.17-15.18: error: T-TEST: Syntax error expecting CIN or CI.
+ 15 | T-TEST CRITERIA=**.
+ | ^~"
+
+"t-test.sps:16.20-16.21: error: T-TEST: Syntax error expecting `('.
+ 16 | T-TEST CRITERIA=CIN**.
+ | ^~"
+
+"t-test.sps:17.21-17.22: error: T-TEST: Syntax error expecting number.
+ 17 | T-TEST CRITERIA=CIN(**).
+ | ^~"
+
+"t-test.sps:18.23-18.24: error: T-TEST: Syntax error expecting `)'.
+ 18 | T-TEST CRITERIA=CIN(1 **).
+ | ^~"
+
+"t-test.sps:19.8-19.9: error: T-TEST: Syntax error expecting TESTVAL, GROUPS, PAIRS, VARIABLES, MISSING, or CRITERIA.
+ 19 | T-TEST **.
+ | ^~"
+
+"t-test.sps:20: error: T-TEST: Exactly one of TESTVAL, GROUPS and PAIRS subcommands must be specified."
+
+"t-test.sps:21.1-21.33: error: T-TEST: Required subcommand VARIABLES was not specified.
+ 21 | T-TEST MISSING=INCLUDE/TESTVAL=1.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([TEMPORARY])
+
+dnl Tests for a bug that manifested when all transformations are temporary.
+AT_SETUP([TEMPORARY as first transformation])
+AT_DATA([temporary.sps], [dnl
+DATA LIST LIST NOTABLE /X *.
+BEGIN DATA.
+1
+2
+3
+4
+5
+6
+7
+8
+9
+END DATA.
+
+TEMPORARY.
+SELECT IF x > 5 .
+LIST.
+
+LIST.
+])
+AT_CHECK([pspp -o pspp.csv temporary.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Data List
+X
+6.00
+7.00
+8.00
+9.00
+
+Table: Data List
+X
+1.00
+2.00
+3.00
+4.00
+5.00
+6.00
+7.00
+8.00
+9.00
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([TITLE and related commands])
+
+AT_SETUP([FILE LABEL and (ADD) DOCUMENT])
+AT_DATA([file-label.sps], [dnl
+/* Set up a dummy active dataset in memory.
+data list /X 1 Y 2.
+begin data.
+16
+27
+38
+49
+50
+end data.
+
+/* Add value labels for some further testing of value labels.
+value labels x y 1 'first label' 2 'second label' 3 'third label'.
+add value labels x 1 'first label mark two'.
+
+/* Add a file label and a few documents.
+file label This is a test file label.
+document First line of a document
+Second line of a document
+The last line should end with a period: .
+
+
+/* Display the documents.
+display documents.
+display file label.
+
+ADD DOCUMENT 'Line one' 'Line two'.
+
+/* Save the active dataset then get it and display the documents again.
+save /OUTFILE='foo.save'.
+get /FILE='foo.save'.
+display documents.
+display file label.
+
+/* There is an interesting interaction that occurs if the 'execute'
+/* command below. What happens is that an error message is output
+/* at the next 'save' command that 'foo.save' is already open for
+/* input. This is because the 'get' hasn't been executed yet and
+/* therefore PSPP would be reading from and writing to the same
+/* file at once, which is obviously a Bad Thing. But 'execute'
+/* here clears up that potential problem.
+execute.
+
+/* Add another (shorter) document and try again.
+document There should be another document now.
+display documents.
+
+/* Save and get.
+save /OUTFILE='foo.save'.
+get /FILE='foo.save'.
+display documents.
+display file label.
+
+/* Done.
+])
+AT_CHECK([pspp -o pspp.csv file-label.sps])
+dnl Filter out the dates/times
+AT_CHECK([[sed 's/(Entered [^)]*)/(Entered <date>)/' pspp.csv]], [0], [dnl
+Table: Reading 1 record from INLINE.
+Variable,Record,Columns,Format
+X,1,1-1,F1.0
+Y,1,2-2,F1.0
+
+Table: Documents
+"document First line of a document
+Second line of a document
+The last line should end with a period: .
+ (Entered <date>)"
+
+Table: File Label
+Label,This is a test file label
+
+Table: Documents
+"document First line of a document
+Second line of a document
+The last line should end with a period: .
+ (Entered <date>)
+Line one
+Line two
+ (Entered <date>)"
+
+Table: File Label
+Label,This is a test file label
+
+Table: Documents
+"document First line of a document
+Second line of a document
+The last line should end with a period: .
+ (Entered <date>)
+Line one
+Line two
+ (Entered <date>)
+document There should be another document now.
+ (Entered <date>)"
+
+Table: Documents
+"document First line of a document
+Second line of a document
+The last line should end with a period: .
+ (Entered <date>)
+Line one
+Line two
+ (Entered <date>)
+document There should be another document now.
+ (Entered <date>)"
+
+Table: File Label
+Label,This is a test file label
+])
+AT_CLEANUP
+
+AT_SETUP([TITLE and SUBTITLE])
+for command in TITLE SUBTITLE; do
+ cat >title.sps <<EOF
+$command foo bar.
+SHOW $command.
+
+$command 'foo bar baz quux'.
+SHOW $command.
+EOF
+ cat >expout <<EOF
+Table: Settings
+$command,foo bar
+
+Table: Settings
+$command,foo bar baz quux
+EOF
+ AT_CHECK([pspp -O format=csv title.sps], [0], [expout])
+done
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017, 2020 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+m4_define([CHECK_UPDATE],
+ [AT_SETUP([UPDATE $1 with $2])
+ AT_DATA([a.data], [dnl
+1aB
+8aM
+3aE
+5aG
+0aA
+5aH
+6aI
+7aJ
+2aD
+7aK
+1aC
+7aL
+4aF
+])
+ AT_DATA([b.data], [dnl
+1bN
+3 O
+4bP
+6bQ
+7bR
+9bS
+])
+ m4_if([$1], [sav],
+ [AT_DATA([save-a.sps], [dnl
+DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).
+SAVE OUTFILE='a.sav'.
+])
+ AT_CHECK([pspp -O format=csv save-a.sps])])
+ m4_if([$2], [sav],
+ [AT_DATA([save-b.sps], [dnl
+DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).
+SAVE OUTFILE='b.sav'.
+])
+ AT_CHECK([pspp -O format=csv save-b.sps])])
+ AT_DATA([update.sps], [dnl
+m4_if([$1], [sav], [], [DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).])
+m4_if([$2], [sav], [], [DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).])
+UPDATE
+ m4_if([$1], [sav], [FILE='a.sav'], [FILE=*]) /IN=InA /SORT
+ m4_if([$2], [sav], [FILE='b.sav'], [FILE=*]) /IN=InB /RENAME c=d
+ /BY a.
+LIST.
+])
+ cat update.sps
+ AT_CHECK([pspp -O format=csv update.sps], [0], [dnl
+update.sps:6: warning: UPDATE: Encountered 3 sets of duplicate cases in the master file.
+
+Table: Data List
+a,b,c,d,InA,InB
+0,a,A,,1,0
+1,b,B,N,1,1
+1,a,C,,1,0
+2,a,D,,1,0
+3,a,E,O,1,1
+4,b,F,P,1,1
+5,a,G,,1,0
+5,a,H,,1,0
+6,b,I,Q,1,1
+7,b,J,R,1,1
+7,a,K,,1,0
+7,a,L,,1,0
+8,a,M,,1,0
+9,b,,S,0,1
+])
+AT_CLEANUP
+])
+
+AT_BANNER([UPDATE])
+
+CHECK_UPDATE([sav], [sav])
+CHECK_UPDATE([sav], [inline])
+CHECK_UPDATE([inline], [sav])
+
+dnl Far more syntax errors are possible, but the rest are all covered
+dnl by the MATCH FILES tests.
+AT_SETUP([UPDATE syntax errors])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='update.sps' ERROR=IGNORE.
+])
+AT_DATA([update.sps], [dnl
+DATA LIST LIST NOTABLE/name (A6) x.
+BEGIN DATA.
+al,7
+brad,8
+carl,9
+END DATA.
+SAVE OUTFILE='x.sav'.
+
+DATA LIST LIST NOTABLE/name (A7) y.
+BEGIN DATA.
+al,1
+carl,2
+dan,3
+END DATA.
+UPDATE/FILE='x.sav'/FILE=*/RENAME(name=name2).
+UPDATE/xyzzy.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"update.sps:15.1-15.46: error: UPDATE: Required subcommand BY was not specified.
+ 15 | UPDATE/FILE='x.sav'/FILE=*/RENAME(name=name2).
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"update.sps:16.8-16.12: error: UPDATE: Syntax error expecting BY, MAP, DROP, or KEEP.
+ 16 | UPDATE/xyzzy.
+ | ^~~~~"
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([VALUE LABELS])
+
+AT_SETUP([VALUE LABELS date formats])
+AT_DATA([value-labels.sps], [dnl
+DATA LIST LIST NOTABLE /ad (adate10) dt (datetime20).
+VALUE LABELS ad 'july 10, 1982' 'label 1'
+ '1-2-93' 'label 2'
+ '5-4-2003' 'label 3'
+ /dt '12 Apr 2011 06:09:56' 'label 4'.
+DISPLAY DICTIONARY.
+])
+AT_CHECK([pspp -O format=csv value-labels.sps], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+ad,1,Unknown,Input,8,Right,ADATE10,ADATE10
+dt,2,Unknown,Input,8,Right,DATETIME20.0,DATETIME20.0
+
+Table: Value Labels
+Variable Value,,Label
+ad,07/10/1982,label 1
+,01/02/1993,label 2
+,05/04/2003,label 3
+dt,12-APR-2011 06:09:56,label 4
+])
+AT_CLEANUP
+
+AT_SETUP([VALUE LABELS with new-line])
+AT_DATA([value-labels.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+VALUE LABELS x 1 'one' 2 'first line\nsecond line' 3 'three'.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+DISPLAY DICTIONARY.
+FREQUENCIES x/STAT=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt value-labels.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+x,1,Nominal,Input,8,Right,F8.2,F8.2
+
+Table: Value Labels
+Variable Value,,Label
+x,1.00,one
+,2.00,first line\nsecond line
+,3.00,three
+
+Table: x
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,one,1,33.3%,33.3%,33.3%
+,"first line
+second line",1,33.3%,33.3%,66.7%
+,three,1,33.3%,33.3%,100.0%
+Total,,3,100.0%,,
+])
+AT_CLEANUP
+
+AT_SETUP([VALUE LABELS with new-line in system file])
+AT_DATA([save.sps], [dnl
+DATA LIST LIST NOTABLE /x.
+VALUE LABELS x 1 'one' 2 'first line\nsecond line' 3 'three'.
+BEGIN DATA.
+1
+2
+3
+END DATA.
+SAVE OUTFILE='value-labels.sav'.
+])
+AT_CHECK([pspp -O format=csv save.sps])
+AT_DATA([get.sps], [dnl
+GET FILE='value-labels.sav'.
+DISPLAY DICTIONARY.
+FREQUENCIES x/STAT=NONE.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt get.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+x,1,Nominal,Input,8,Right,F8.2,F8.2
+
+Table: Value Labels
+Variable Value,,Label
+x,1.00,one
+,2.00,first line\nsecond line
+,3.00,three
+
+Table: x
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,one,1,33.3%,33.3%,33.3%
+,"first line
+second line",1,33.3%,33.3%,66.7%
+,three,1,33.3%,33.3%,100.0%
+Total,,3,100.0%,,
+])
+AT_CLEANUP
+
+dnl Tests for a bug which caused VALUE LABELS to
+dnl crash when given invalid syntax.
+AT_SETUP([VALUE LABELS invalid syntax bug])
+AT_DATA([value-labels.sps], [dnl
+DATA LIST LIST NOTABLE /a * pref * .
+BEGIN DATA.
+ 1.00 1.00
+ 1.00 2.00
+ 2.00 1.00
+ 2.00 2.00
+END DATA.
+
+VALUE LABELS /var=a 'label for a'.
+])
+AT_CHECK([pspp -O format=csv value-labels.sps], [1], [dnl
+"value-labels.sps:9.15-9.17: error: VALUE LABELS: var is not a variable name.
+ 9 | VALUE LABELS /var=a 'label for a'.
+ | ^~~"
+])
+AT_CLEANUP
+
+# Tests for a bug which caused a crash if VALUE LABELS had a trailing /.
+AT_SETUP([VALUE LABELS trailing `/' bug])
+AT_DATA([value-labels.sps], [dnl
+DATA LIST LIST NOTABLE /X * .
+BEGIN DATA.
+1
+2
+3
+4
+END DATA.
+
+
+VALUE LABELS X 1 'one' 2 'two' 3 'three'/
+
+
+LIST VARIABLES=X.
+])
+AT_CHECK([pspp -O format=csv value-labels.sps], [0], [dnl
+Table: Data List
+X
+1.00
+2.00
+3.00
+4.00
+])
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([variable display attributes])
+
+AT_SETUP([variable display attribute commands])
+AT_KEYWORDS([VARIABLE ALIGNMENT])
+AT_KEYWORDS([VARIABLE WIDTH])
+AT_KEYWORDS([VARIABLE LEVEL])
+AT_KEYWORDS([VARIABLE ROLE])
+AT_DATA([var-display.sps], [dnl
+DATA LIST FREE /x y z.
+VARIABLE ALIGNMENT x (LEFT)/y (RIGHT)/z (CENTER).
+VARIABLE WIDTH x (10)/y (12)/z (14).
+VARIABLE LEVEL x (SCALE)/y (ORDINAL)/z (NOMINAL).
+VARIABLE ROLE /TARGET x /BOTH y /NONE z.
+DISPLAY DICTIONARY.
+])
+AT_CHECK([pspp -o pspp.csv var-display.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+x,1,Scale,Output,10,Left,F8.2,F8.2
+y,2,Ordinal,Both,12,Right,F8.2,F8.2
+z,3,Nominal,None,14,Center,F8.2,F8.2
+])
+AT_CLEANUP
+
+AT_SETUP([variable display attribute syntax errors])
+AT_KEYWORDS([VARIABLE ALIGNMENT])
+AT_KEYWORDS([VARIABLE WIDTH])
+AT_KEYWORDS([VARIABLE LEVEL])
+AT_KEYWORDS([VARIABLE ROLE])
+AT_DATA([var-display.sps], [dnl
+DATA LIST LIST NOTABLE /x y z.
+VARIABLE ALIGNMENT **.
+VARIABLE ALIGNMENT x **.
+VARIABLE ALIGNMENT x (**).
+VARIABLE ALIGNMENT x (LEFT **).
+VARIABLE WIDTH **.
+VARIABLE WIDTH x **.
+VARIABLE WIDTH x (**).
+VARIABLE WIDTH x (10 **).
+VARIABLE LEVEL **.
+VARIABLE LEVEL x **.
+VARIABLE LEVEL x (**).
+VARIABLE LEVEL x (SCALE **).
+VARIABLE ROLE **.
+VARIABLE ROLE / **.
+VARIABLE ROLE /INPUT **.
+VARIABLE ROLE /INPUT x **.
+])
+AT_CHECK([pspp -O format=csv var-display.sps], [1], [dnl
+"var-display.sps:2.20-2.21: error: VARIABLE ALIGNMENT: Syntax error expecting variable name.
+ 2 | VARIABLE ALIGNMENT **.
+ | ^~"
+
+"var-display.sps:3.22-3.23: error: VARIABLE ALIGNMENT: Syntax error expecting `('.
+ 3 | VARIABLE ALIGNMENT x **.
+ | ^~"
+
+"var-display.sps:4.23-4.24: error: VARIABLE ALIGNMENT: Syntax error expecting LEFT, RIGHT, or CENTER.
+ 4 | VARIABLE ALIGNMENT x (**).
+ | ^~"
+
+"var-display.sps:5.28-5.29: error: VARIABLE ALIGNMENT: Syntax error expecting `)'.
+ 5 | VARIABLE ALIGNMENT x (LEFT **).
+ | ^~"
+
+"var-display.sps:6.16-6.17: error: VARIABLE WIDTH: Syntax error expecting variable name.
+ 6 | VARIABLE WIDTH **.
+ | ^~"
+
+"var-display.sps:7.18-7.19: error: VARIABLE WIDTH: Syntax error expecting `('.
+ 7 | VARIABLE WIDTH x **.
+ | ^~"
+
+"var-display.sps:8.19-8.20: error: VARIABLE WIDTH: Syntax error expecting positive integer.
+ 8 | VARIABLE WIDTH x (**).
+ | ^~"
+
+"var-display.sps:9.22-9.23: error: VARIABLE WIDTH: Syntax error expecting `)'.
+ 9 | VARIABLE WIDTH x (10 **).
+ | ^~"
+
+"var-display.sps:10.16-10.17: error: VARIABLE LEVEL: Syntax error expecting variable name.
+ 10 | VARIABLE LEVEL **.
+ | ^~"
+
+"var-display.sps:11.18-11.19: error: VARIABLE LEVEL: Syntax error expecting `('.
+ 11 | VARIABLE LEVEL x **.
+ | ^~"
+
+"var-display.sps:12.19-12.20: error: VARIABLE LEVEL: Syntax error expecting SCALE, ORDINAL, or NOMINAL.
+ 12 | VARIABLE LEVEL x (**).
+ | ^~"
+
+"var-display.sps:13.25-13.26: error: VARIABLE LEVEL: Syntax error expecting `)'.
+ 13 | VARIABLE LEVEL x (SCALE **).
+ | ^~"
+
+"var-display.sps:14.15-14.16: error: VARIABLE ROLE: Syntax error expecting `/'.
+ 14 | VARIABLE ROLE **.
+ | ^~"
+
+"var-display.sps:15.17-15.18: error: VARIABLE ROLE: Syntax error expecting INPUT, TARGET, BOTH, NONE, PARTITION, or SPLIT.
+ 15 | VARIABLE ROLE / **.
+ | ^~"
+
+"var-display.sps:16.22-16.23: error: VARIABLE ROLE: Syntax error expecting variable name.
+ 16 | VARIABLE ROLE /INPUT **.
+ | ^~"
+
+"var-display.sps:17.24-17.25: error: VARIABLE ROLE: Syntax error expecting `/'.
+ 17 | VARIABLE ROLE /INPUT x **.
+ | ^~"
+])
+AT_CLEANUP
+
+AT_SETUP([variable level inference and SCALEMIN])
+AT_DATA([var-level.sps], [dnl
+DATA LIST LIST NOTABLE /n1 to n3 s1 to s5.
+
+* Nominal formats (copied from data that will default to scale).
+COMPUTE n4=s1.
+COMPUTE n5=s1.
+FORMATS n4(WKDAY5) n5(MONTH5).
+
+* Scale formats (copied from data that will default to nominal).
+COMPUTE s6=n1.
+COMPUTE s7=n1.
+COMPUTE s8=n1.
+FORMATS s6(DOLLAR6.2) s7(CCA8.2) s8(DATETIME17).
+
+STRING string(A8).
+DISPLAY DICTIONARY.
+EXECUTE.
+
+* n1 has 10 unique small values -> nominal.
+* n2 has 23 unique small values -> nominal.
+* n3 is all missing -> nominal.
+* s1 has 24 unique small values -> scale.
+* s2 has one negative value -> scale.
+* s3 has one non-integer value -> scale.
+* s4 has no valid values less than 10 -> scale.
+* s5 has no valid values less than 10,000 -> scale.
+BEGIN DATA.
+1 1 . 1 1 1 10 10001
+2 2 . 2 2 2 11 10002
+3 3 . 3 3 3 12 10003
+4 4 . 4 4 4 13 10004
+5 5 . 5 5 5 14 10005
+6 6 . 6 6 6 15 10006
+7 7 . 7 7 7 16 10007
+8 8 . 8 8 8 17 10008
+9 9 . 9 9 9 18 10009
+10 10 . 10 10 10.5 19 110000
+1 11 . 11 -1 1 11 10001
+2 12 . 12 2 2 12 10002
+3 13 . 13 3 3 13 10003
+4 14 . 14 4 4 14 10004
+5 15 . 15 5 5 15 10005
+6 16 . 16 6 6 16 10006
+7 17 . 17 7 7 17 10007
+8 18 . 18 8 8 18 10008
+9 19 . 19 9 9 19 10009
+1 20 . 20 1 1 20 10001
+2 21 . 21 2 2 21 10002
+3 22 . 22 3 3 22 10003
+4 23 . 23 4 4 23 10004
+5 23 . 24 5 5 24 10005
+6 23 . 24 6 6 25 10006
+END DATA.
+DISPLAY DICTIONARY.
+])
+AT_CHECK([pspp -o pspp.csv var-level.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+n1,1,Unknown,Input,8,Right,F8.2,F8.2
+n2,2,Unknown,Input,8,Right,F8.2,F8.2
+n3,3,Unknown,Input,8,Right,F8.2,F8.2
+s1,4,Unknown,Input,8,Right,F8.2,F8.2
+s2,5,Unknown,Input,8,Right,F8.2,F8.2
+s3,6,Unknown,Input,8,Right,F8.2,F8.2
+s4,7,Unknown,Input,8,Right,F8.2,F8.2
+s5,8,Unknown,Input,8,Right,F8.2,F8.2
+n4,9,Unknown,Input,8,Right,WKDAY5,WKDAY5
+n5,10,Unknown,Input,8,Right,MONTH5,MONTH5
+s6,11,Unknown,Input,8,Right,DOLLAR6.2,DOLLAR6.2
+s7,12,Unknown,Input,8,Right,CCA8.2,CCA8.2
+s8,13,Unknown,Input,8,Right,DATETIME17.0,DATETIME17.0
+string,14,Nominal,Input,8,Left,A8,A8
+
+Table: Variables
+Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
+n1,1,Nominal,Input,8,Right,F8.2,F8.2
+n2,2,Nominal,Input,8,Right,F8.2,F8.2
+n3,3,Nominal,Input,8,Right,F8.2,F8.2
+s1,4,Scale,Input,8,Right,F8.2,F8.2
+s2,5,Scale,Input,8,Right,F8.2,F8.2
+s3,6,Scale,Input,8,Right,F8.2,F8.2
+s4,7,Scale,Input,8,Right,F8.2,F8.2
+s5,8,Scale,Input,8,Right,F8.2,F8.2
+n4,9,Nominal,Input,8,Right,WKDAY5,WKDAY5
+n5,10,Nominal,Input,8,Right,MONTH5,MONTH5
+s6,11,Scale,Input,8,Right,DOLLAR6.2,DOLLAR6.2
+s7,12,Scale,Input,8,Right,CCA8.2,CCA8.2
+s8,13,Scale,Input,8,Right,DATETIME17.0,DATETIME17.0
+string,14,Nominal,Input,8,Left,A8,A8
+])
+AT_CLEANUP
+
+AT_BANNER([VARIABLE LABELS])
+
+AT_SETUP([variable labels])
+
+dnl The following test is to make sure the TVARS command works and that
+dnl variables are displayed accordingly.
+AT_DATA([var-labels.sps], [dnl
+DATA LIST LIST NOTABLE /x * y *.
+BEGIN DATA.
+1 100
+2 200
+3 300
+4 400
+END DATA.
+
+* While no labels have been set, the TVARS is irrelevant.
+SET TVARS=NAMES.
+DESCRIPTIVES ALL.
+
+SET TVARS=LABELS.
+DESCRIPTIVES ALL.
+
+SET TVARS=BOTH.
+DESCRIPTIVES ALL.
+
+VARIABLE LABEL x 'foo' y 'bar'.
+
+* Now, the TVARS setting should have effect
+
+SET TVARS=NAMES.
+DESCRIPTIVES ALL.
+
+SET TVARS=LABELS.
+DESCRIPTIVES ALL.
+
+SET TVARS=BOTH.
+DESCRIPTIVES ALL.
+])
+
+AT_CHECK([pspp -o pspp.csv -o pspp.txt var-labels.sps])
+AT_CHECK([cat pspp.csv], [0],[dnl
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+x,4,2.50,1.29,1.00,4.00
+y,4,250.00,129.10,100.00,400.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+x,4,2.50,1.29,1.00,4.00
+y,4,250.00,129.10,100.00,400.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+x,4,2.50,1.29,1.00,4.00
+y,4,250.00,129.10,100.00,400.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+x,4,2.50,1.29,1.00,4.00
+y,4,250.00,129.10,100.00,400.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+foo,4,2.50,1.29,1.00,4.00
+bar,4,250.00,129.10,100.00,400.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+
+Table: Descriptive Statistics
+,N,Mean,Std Dev,Minimum,Maximum
+x foo,4,2.50,1.29,1.00,4.00
+y bar,4,250.00,129.10,100.00,400.00
+Valid N (listwise),4,,,,
+Missing N (listwise),0,,,,
+])
+
+AT_CLEANUP
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([VECTOR])
+
+AT_SETUP([VECTOR short form])
+AT_DATA([vector.sps], [dnl
+data list notable/x 1.
+vector v(4).
+display vector.
+])
+AT_CHECK([pspp -o pspp.csv vector.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Vectors
+Vector and Position,,Variable,Print Format
+v,1,v1,F8.2
+,2,v2,F8.2
+,3,v3,F8.2
+,4,v4,F8.2
+])
+AT_CLEANUP
+
+AT_SETUP([VECTOR short form with format specification])
+AT_DATA([vector.sps], [dnl
+data list notable/x 1.
+vector #vec(4, comma10.2)
+ /#svec(3, a8).
+display vector.
+])
+AT_CHECK([pspp -o pspp.csv vector.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Vectors
+Vector and Position,,Variable,Print Format
+#vec,1,#vec1,COMMA10.2
+,2,#vec2,COMMA10.2
+,3,#vec3,COMMA10.2
+,4,#vec4,COMMA10.2
+#svec,1,#svec1,A8
+,2,#svec2,A8
+,3,#svec3,A8
+])
+AT_CLEANUP
+
+AT_SETUP([VECTOR short form in INPUT PROGRAM])
+AT_DATA([vector.sps], [dnl
+input program.
+vector x(5).
+data list notable/x5 x2 x3 x1 x4 1-5.
+end input program.
+display vector.
+])
+AT_CHECK([pspp -o pspp.csv vector.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Vectors
+Vector and Position,,Variable,Print Format
+x,1,x1,F8.2
+,2,x2,F8.2
+,3,x3,F8.2
+,4,x4,F8.2
+,5,x5,F8.2
+])
+AT_CLEANUP
+
+AT_SETUP([VECTOR long form])
+AT_DATA([vector.sps], [dnl
+data list notable/u w x y z 1-5.
+vector a=u to y.
+vector b=x to z.
+vector c=all.
+display vector.
+])
+AT_CHECK([pspp -o pspp.csv vector.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Vectors
+Vector and Position,,Variable,Print Format
+a,1,u,F1.0
+,2,w,F1.0
+,3,x,F1.0
+,4,y,F1.0
+b,1,x,F1.0
+,2,y,F1.0
+,3,z,F1.0
+c,1,u,F1.0
+,2,w,F1.0
+,3,x,F1.0
+,4,y,F1.0
+,5,z,F1.0
+])
+AT_CLEANUP
+
+AT_SETUP([VECTOR syntax errors])
+AT_DATA([vector.sps], [dnl
+DATA LIST LIST NOTABLE/x y z.
+VECTOR **.
+VECTOR aslkdfjaklsdjfklasdjfklasjdfklasjdfkajsdlkfajsdkfjaksdjfaklsdkasdjfklasdjfklasjdfkldkl.
+VECTOR dup=x y z.
+VECTOR dup.
+VECTOR v v.
+VECTOR u v=x y z.
+VECTOR v(1, 2).
+VECTOR v(0).
+VECTOR v(F8.2, F8.2).
+VECTOR v(asdf).
+VECTOR v(**).
+VECTOR v(F8.2).
+VECTOR xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(5).
+VECTOR v **.
+])
+AT_DATA([insert.sps], [dnl
+INSERT FILE='vector.sps' ERROR=IGNORE.
+])
+AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
+"vector.sps:2.8-2.9: error: VECTOR: Syntax error expecting identifier.
+ 2 | VECTOR **.
+ | ^~"
+
+"vector.sps:3.8-3.93: error: VECTOR: Identifier `aslkdfjaklsdjfklasdjfklasjdfklasjdfkajsdlkfajsdkfjaksdjfaklsdkasdjfklasdjfklasjdfkldkl' exceeds 64-byte limit.
+ 3 | VECTOR aslkdfjaklsdjfklasdjfklasjdfklasjdfkajsdlkfajsdkfjaksdjfaklsdkasdjfklasdjfklasjdfkldkl.
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"vector.sps:5.8-5.10: error: VECTOR: A vector named dup already exists.
+ 5 | VECTOR dup.
+ | ^~~"
+
+"vector.sps:6.8-6.10: error: VECTOR: Vector name v is given twice.
+ 6 | VECTOR v v.
+ | ^~~"
+
+"vector.sps:7.8-7.11: error: VECTOR: Only a single vector name may be specified when a list of variables is given.
+ 7 | VECTOR u v=x y z.
+ | ^~~~"
+
+"vector.sps:8.9-8.13: error: VECTOR: Vector length may only be specified once.
+ 8 | VECTOR v(1, 2).
+ | ^~~~~"
+
+"vector.sps:9.10: error: VECTOR: Syntax error expecting positive integer.
+ 9 | VECTOR v(0).
+ | ^"
+
+"vector.sps:10.9-10.19: error: VECTOR: Only one format may be specified.
+ 10 | VECTOR v(F8.2, F8.2).
+ | ^~~~~~~~~~~"
+
+"vector.sps:11.10-11.13: error: VECTOR: Unknown format type `asdf'.
+ 11 | VECTOR v(asdf).
+ | ^~~~"
+
+"vector.sps:12.10-12.11: error: VECTOR: Syntax error expecting vector length or format.
+ 12 | VECTOR v(**).
+ | ^~"
+
+"vector.sps:13.9-13.14: error: VECTOR: Vector length is required.
+ 13 | VECTOR v(F8.2).
+ | ^~~~~~"
+
+"vector.sps:14.8-14.74: error: VECTOR: Identifier `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1' exceeds 64-byte limit.
+ 14 | VECTOR xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(5).
+ | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
+
+"vector.sps:15.10-15.11: error: VECTOR: Syntax error expecting `=' or `@{:@'.
+ 15 | VECTOR v **.
+ | ^~"
+])
+AT_CLEANUP
\ No newline at end of file
--- /dev/null
+dnl PSPP - a program for statistical analysis.
+dnl Copyright (C) 2017 Free Software Foundation, Inc.
+dnl
+dnl This program is free software: you can redistribute it and/or modify
+dnl it under the terms of the GNU General Public License as published by
+dnl the Free Software Foundation, either version 3 of the License, or
+dnl (at your option) any later version.
+dnl
+dnl This program is distributed in the hope that it will be useful,
+dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
+dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+dnl GNU General Public License for more details.
+dnl
+dnl You should have received a copy of the GNU General Public License
+dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
+dnl
+AT_BANNER([WEIGHT])
+
+AT_SETUP([WEIGHT])
+AT_DATA([weight.txt], [dnl
+ 18 1
+ 19 7
+ 20 26
+ 21 76
+ 22 57
+ 23 58
+ 24 38
+ 25 38
+ 26 30
+ 27 21
+ 28 23
+ 29 24
+ 30 23
+ 31 14
+ 32 21
+ 33 21
+ 34 14
+ 35 14
+ 36 17
+ 37 11
+ 38 16
+ 39 14
+ 40 15
+ 41 14
+ 42 14
+ 43 8
+ 44 15
+ 45 10
+ 46 12
+ 47 13
+ 48 13
+ 49 5
+ 50 5
+ 51 3
+ 52 7
+ 53 6
+ 54 2
+ 55 2
+ 56 2
+ 57 3
+ 58 1
+ 59 3
+ 61 1
+ 62 3
+ 63 1
+ 64 1
+ 65 2
+ 70 1
+ 78 1
+ 79 1
+ 80 1
+ 94 1
+])
+AT_DATA([weight.sps], [dnl
+SET FORMAT F8.3.
+data list file='weight.txt'/AVAR 1-5 BVAR 6-10.
+weight by BVAR.
+
+descriptives AVAR /statistics all /format serial.
+frequencies AVAR /statistics all.
+])
+AT_CHECK([pspp -o pspp.csv -o pspp.txt weight.sps])
+AT_CHECK([cat pspp.csv], [0], [dnl
+Table: Reading 1 record from `weight.txt'.
+Variable,Record,Columns,Format
+AVAR,1,1-5,F5.0
+BVAR,1,6-10,F5.0
+
+Table: Descriptive Statistics
+,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
+AVAR,730,31.515,.405,10.937,119.608,2.411,.181,1.345,.090,76.000,18,94,23006.00
+Valid N (listwise),730,,,,,,,,,,,,
+Missing N (listwise),0,,,,,,,,,,,,
+
+Table: Statistics
+,,AVAR
+N,Valid,730
+,Missing,0
+Mean,,31.515
+S.E. Mean,,.405
+Median,,28.000
+Mode,,21
+Std Dev,,10.937
+Variance,,119.608
+Kurtosis,,2.411
+S.E. Kurt,,.181
+Skewness,,1.345
+S.E. Skew,,.090
+Range,,76.000
+Minimum,,18
+Maximum,,94
+Sum,,23006.00
+
+Table: AVAR
+,,Frequency,Percent,Valid Percent,Cumulative Percent
+Valid,18,1,.1%,.1%,.1%
+,19,7,1.0%,1.0%,1.1%
+,20,26,3.6%,3.6%,4.7%
+,21,76,10.4%,10.4%,15.1%
+,22,57,7.8%,7.8%,22.9%
+,23,58,7.9%,7.9%,30.8%
+,24,38,5.2%,5.2%,36.0%
+,25,38,5.2%,5.2%,41.2%
+,26,30,4.1%,4.1%,45.3%
+,27,21,2.9%,2.9%,48.2%
+,28,23,3.2%,3.2%,51.4%
+,29,24,3.3%,3.3%,54.7%
+,30,23,3.2%,3.2%,57.8%
+,31,14,1.9%,1.9%,59.7%
+,32,21,2.9%,2.9%,62.6%
+,33,21,2.9%,2.9%,65.5%
+,34,14,1.9%,1.9%,67.4%
+,35,14,1.9%,1.9%,69.3%
+,36,17,2.3%,2.3%,71.6%
+,37,11,1.5%,1.5%,73.2%
+,38,16,2.2%,2.2%,75.3%
+,39,14,1.9%,1.9%,77.3%
+,40,15,2.1%,2.1%,79.3%
+,41,14,1.9%,1.9%,81.2%
+,42,14,1.9%,1.9%,83.2%
+,43,8,1.1%,1.1%,84.2%
+,44,15,2.1%,2.1%,86.3%
+,45,10,1.4%,1.4%,87.7%
+,46,12,1.6%,1.6%,89.3%
+,47,13,1.8%,1.8%,91.1%
+,48,13,1.8%,1.8%,92.9%
+,49,5,.7%,.7%,93.6%
+,50,5,.7%,.7%,94.2%
+,51,3,.4%,.4%,94.7%
+,52,7,1.0%,1.0%,95.6%
+,53,6,.8%,.8%,96.4%
+,54,2,.3%,.3%,96.7%
+,55,2,.3%,.3%,97.0%
+,56,2,.3%,.3%,97.3%
+,57,3,.4%,.4%,97.7%
+,58,1,.1%,.1%,97.8%
+,59,3,.4%,.4%,98.2%
+,61,1,.1%,.1%,98.4%
+,62,3,.4%,.4%,98.8%
+,63,1,.1%,.1%,98.9%
+,64,1,.1%,.1%,99.0%
+,65,2,.3%,.3%,99.3%
+,70,1,.1%,.1%,99.5%
+,78,1,.1%,.1%,99.6%
+,79,1,.1%,.1%,99.7%
+,80,1,.1%,.1%,99.9%
+,94,1,.1%,.1%,100.0%
+Total,,730,100.0%,,
+])
+AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU nGeneral Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DEFINE])
-
-AT_SETUP([simple macro expansion])
-AT_DATA([define.sps], [dnl
-DEFINE !macro()
-a b c d
-e f g h.
-i j k l
-1,2,3,4.
-5+6+7.
-m(n,o).
-"a" "b" "c" 'a' 'b' 'c'.
-"x "" y".
-!ENDDEFINE.
-DEBUG EXPAND.
-!macro
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-a b c d e f g h.
-i j k l 1, 2, 3, 4.
-5 + 6 + 7.
-m(n, o).
-"a" "b" "c" 'a' 'b' 'c'.
-"x "" y".
-])
-AT_CLEANUP
-
-AT_SETUP([redefining a macro])
-AT_DATA([define.sps], [dnl
-DEFINE !macro() 0 !ENDDEFINE.
-DEFINE !macro() 1 !ENDDEFINE.
-DEBUG EXPAND.
-!macro.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-1
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - one !TOKENS(1) positional argument])
-AT_KEYWORDS([TOKENS])
-AT_DATA([define.sps], [dnl
-DEFINE !t1(!positional=!tokens(1)) t1 (!1) !ENDDEFINE.
-DEBUG EXPAND.
-!t1 a.
-!t1 b.
-!t1 a b.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-t1(a)
-
-t1(b)
-
-t1(a)
-
-note: unexpanded token "b"
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion with positional arguments])
-AT_DATA([define.sps], [dnl
-DEFINE !title(!positional !tokens(1)) !1 !ENDDEFINE.
-DEFINE !t1(!positional !tokens(1)) t1 (!1) !ENDDEFINE.
-DEFINE !t2(!positional !tokens(2)) t2 (!1) !ENDDEFINE.
-
-DEFINE !ce(!positional=!charend('/')) ce (!1) !ENDDEFINE.
-DEFINE !ce2(!positional=!charend('(')
- /!positional !charend(')'))
-ce2 (!1, !2)
-!ENDDEFINE.
-
-DEFINE !e(!positional !enclose('{','}')) e (!1) !ENDDEFINE.
-
-DEFINE !cmd(!positional !cmdend) cmd(!1) !ENDDEFINE.
-DEFINE !cmd2(!positional !cmdend
- /!positional !tokens(1))
-cmd2(!1, !2)
-!ENDDEFINE.
-
-DEFINE !p(!positional !tokens(1)
- /!positional !tokens(1)
- /!positional !tokens(1))
-p(!1, !2, !3)(!*)
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!title "!TOKENS(1) argument."
-!t1 a.
-!t1 b.
-!t1 a b.
-
-!title "!TOKENS(2) argument."
-!t2 a b.
-!t2 b c d.
-
-!title "!CHAREND argument."
-!ce/.
-!ce x/.
-!ce x y/.
-!ce x y z/.
-
-!title "Two !CHAREND arguments."
-!ce2 x(y).
-!ce2 1 2 3 4().
-
-!title "!ENCLOSE argument."
-!e {}.
-!e {a}.
-!e {a b}.
-
-!title "!CMDEND argument."
-!cmd 1 2 3 4.
-!cmd2 5 6.
-7.
-
-!title "Three !TOKENS(1) arguments."
-!p a b c.
-!p 1 -2 -3.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-"!TOKENS(1) argument."
-
-t1(a)
-
-t1(b)
-
-t1(a)
-
-note: unexpanded token "b"
-
-"!TOKENS(2) argument."
-
-t2(a b)
-
-t2(b c)
-
-note: unexpanded token "d"
-
-"!CHAREND argument."
-
-ce( )
-
-ce(x)
-
-ce(x y)
-
-ce(x y z)
-
-"Two !CHAREND arguments."
-
-ce2(x, y)
-
-ce2(1 2 3 4, )
-
-"!ENCLOSE argument."
-
-e( )
-
-e(a)
-
-e(a b)
-
-"!CMDEND argument."
-
-cmd(1 2 3 4)
-
-cmd2(5 6, )
-
-note: unexpanded token "7"
-
-"Three !TOKENS(1) arguments."
-
-p(a, b, c) (a b c)
-
-p(1, -2, -3) (1 -2 -3)
-])
-AT_CLEANUP
-
-AT_SETUP([macro call missing positional !TOKENS arguments])
-AT_KEYWORDS([TOKENS])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !tokens(1) !default(x)
- /!positional !tokens(1) !default(y)
- /!positional !tokens(1) !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p a b c.
-!p a b.
-!p a.
-!p.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-(a, b, c)
-
-(a, b, z)
-
-(a, y, z)
-
-(x, y, z)
-])
-AT_CLEANUP
-
-AT_SETUP([macro call incomplete positional !TOKENS arguments])
-AT_KEYWORDS([TOKENS])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !tokens(2) !default(x)
- /!positional !tokens(2) !default(y)
- /!positional !tokens(2) !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p a1 a2 b1 b2 c1 c2.
-!p a1 a2 b1 b2 c1.
-!p a1 a2 b1 b2.
-!p a1 a2 b1.
-!p a1 a2.
-!p a1.
-!p.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-(a1 a2, b1 b2, c1 c2)
-
-define.sps:8.18: error: DEBUG EXPAND: Reached end of command expecting 1 more
-token in argument !3 to macro !p.
- 8 | !p a1 a2 b1 b2 c1.
- | ^
-
-(a1 a2, b1 b2, c1)
-
-(a1 a2, b1 b2, z)
-
-define.sps:10.12: error: DEBUG EXPAND: Reached end of command expecting 1 more
-token in argument !2 to macro !p.
- 10 | !p a1 a2 b1.
- | ^
-
-(a1 a2, b1, z)
-
-(a1 a2, y, z)
-
-define.sps:12.6: error: DEBUG EXPAND: Reached end of command expecting 1 more
-token in argument !1 to macro !p.
- 12 | !p a1.
- | ^
-
-(a1, y, z)
-
-(x, y, z)
-])
-AT_CLEANUP
-
-AT_SETUP([macro call empty positional !CHAREND arguments])
-AT_KEYWORDS([CHAREND])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !charend(',') !default(x)
- /!positional !charend(';') !default(y)
- /!positional !charend(':') !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p a,b;c:.
-!p a,b;:.
-!p a,;c:.
-!p a,;:.
-!p,b;c:.
-!p,b;:.
-!p,;c:.
-!p,;:.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-(a, b, c)
-
-(a, b, )
-
-(a, , c)
-
-(a, , )
-
-(, b, c)
-
-(, b, )
-
-(, , c)
-
-(, , )
-])
-AT_CLEANUP
-
-AT_SETUP([macro call missing positional !CHAREND arguments])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !charend(',') !default(x)
- /!positional !charend(';') !default(y)
- /!positional !charend(':') !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p a,b;c:.
-!p a,b;.
-!p a,;.
-!p ,b;.
-!p ,;.
-
-!p a,.
-!p ,.
-
-!p.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-(a, b, c)
-
-(a, b, z)
-
-(a, , z)
-
-(, b, z)
-
-(, , z)
-
-(a, y, z)
-
-(, y, z)
-
-(x, y, z)
-])
-AT_CLEANUP
-
-AT_SETUP([macro call incomplete positional !CHAREND arguments])
-AT_KEYWORDS([CHAREND])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !charend(',') !default(x)
- /!positional !charend(';') !default(y)
- /!positional !charend(':') !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p a,b;c:.
-!p a,b;c.
-!p a,b;.
-!p a,b.
-!p a,.
-!p a.
-!p.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-(a, b, c)
-
-define.sps:8.9: error: DEBUG EXPAND: Reached end of command expecting ":" in
-argument !3 to macro !p.
- 8 | !p a,b;c.
- | ^
-
-(a, b, c)
-
-(a, b, z)
-
-define.sps:10.7: error: DEBUG EXPAND: Reached end of command expecting ";" in
-argument !2 to macro !p.
- 10 | !p a,b.
- | ^
-
-(a, b, z)
-
-(a, y, z)
-
-define.sps:12.5: error: DEBUG EXPAND: Reached end of command expecting "," in
-argument !1 to macro !p.
- 12 | !p a.
- | ^
-
-(a, y, z)
-
-(x, y, z)
-])
-AT_CLEANUP
-
-AT_SETUP([macro call missing positional !ENCLOSE arguments])
-AT_KEYWORDS([ENCLOSE])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !enclose('(',')') !default(x)
- /!positional !enclose('<','>') !default(y)
- /!positional !enclose('{','}') !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p (a)<b>{c}.
-!p (a)<b>.
-!p (a).
-!p.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-(a, b, c)
-
-(a, b, z)
-
-(a, y, z)
-
-(x, y, z)
-])
-AT_CLEANUP
-
-AT_SETUP([macro call incomplete positional !ENCLOSE arguments])
-AT_KEYWORDS([ENCLOSE])
-AT_DATA([define.sps], [dnl
-DEFINE !p(!positional !enclose('(',')') !default(x)
- /!positional !enclose('<','>') !default(y)
- /!positional !enclose('{','}') !default(z))
-(!1, !2, !3)
-!ENDDEFINE.
-DEBUG EXPAND.
-!p (a)<b>{c}.
-!p (a)<b>{c.
-!p (a)<b>{.
-!p (a)<b>.
-!p (a)<b.
-!p (a)<.
-!p (a).
-!p (a.
-!p (.
-!p.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-(a, b, c)
-
-define.sps:8.12: error: DEBUG EXPAND: Reached end of command expecting "}" in
-argument !3 to macro !p.
- 8 | !p (a)<b>{c.
- | ^
-
-(a, b, c)
-
-define.sps:9.11: error: DEBUG EXPAND: Reached end of command expecting "}" in
-argument !3 to macro !p.
- 9 | !p (a)<b>{.
- | ^
-
-(a, b, )
-
-(a, b, z)
-
-define.sps:11.9: error: DEBUG EXPAND: Reached end of command expecting ">" in
-argument !2 to macro !p.
- 11 | !p (a)<b.
- | ^
-
-(a, b, z)
-
-define.sps:12.8: error: DEBUG EXPAND: Reached end of command expecting ">" in
-argument !2 to macro !p.
- 12 | !p (a)<.
- | ^
-
-(a, , z)
-
-(a, y, z)
-
-define.sps:14.6: error: DEBUG EXPAND: Reached end of command expecting ")" in
-argument !1 to macro !p.
- 14 | !p (a.
- | ^
-
-(a, y, z)
-
-define.sps:15.5: error: DEBUG EXPAND: Reached end of command expecting ")" in
-argument !1 to macro !p.
- 15 | !p (.
- | ^
-
-(, y, z)
-
-(x, y, z)
-])
-AT_CLEANUP
-
-AT_SETUP([keyword macro argument name with ! prefix])
-AT_DATA([define.sps], [dnl
-DEFINE !macro(!x !TOKENS(1).
-])
-AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
-"define.sps:1.15-1.16: error: DEFINE: Keyword macro parameter must be named in definition without ""!"" prefix.
- 1 | DEFINE !macro@{:@!x !TOKENS(1).
- | ^~"
-])
-AT_CLEANUP
-
-AT_SETUP([reserved macro keyword argument name])
-AT_DATA([define.sps], [dnl
-DEFINE !macro(if=!TOKENS(1).
-])
-AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
-"define.sps:1.15-1.16: error: DEFINE: Cannot use macro keyword ""if"" as an argument name.
- 1 | DEFINE !macro@{:@if=!TOKENS(1).
- | ^~"
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - one !TOKENS(1) keyword argument])
-AT_KEYWORDS([TOKENS])
-AT_DATA([define.sps], [dnl
-DEFINE !k(arg1 = !TOKENS(1)) k(!arg1) !ENDDEFINE.
-DEBUG EXPAND.
-!k arg1=x.
-!k arg1=x y.
-!k.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-k(x)
-
-k(x)
-
-note: unexpanded token "y"
-
-k( )
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - one !TOKENS(1) keyword argument - negative])
-AT_KEYWORDS([TOKENS])
-AT_DATA([define.sps], [dnl
-DEFINE !k(arg1 !TOKENS(1)) k(!arg1) !ENDDEFINE.
-DEBUG EXPAND.
-!k arg1.
-!k arg1=.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:3.8: error: DEBUG EXPAND: Found `.' while expecting `=' reading
-argument !arg1 to macro !k.
- 3 | !k arg1.
- | ^
-
-k( )
-
-define.sps:4.9: error: DEBUG EXPAND: Reached end of command expecting 1 more
-token in argument !arg1 to macro !k.
- 4 | !k arg1=.
- | ^
-
-k( )
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - !CHAREND keyword arguments])
-AT_KEYWORDS([CHAREND])
-AT_DATA([define.sps], [dnl
-DEFINE !k(arg1 = !CHAREND('/')
- /arg2 = !CHAREND('/'))
-k(!arg1, !arg2)
-!ENDDEFINE.
-DEBUG EXPAND.
-!k arg1=x/ arg2=y/.
-!k arg1=x/.
-!k arg2=y/.
-!k.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-k(x, y)
-
-k(x, )
-
-k(, y)
-
-k(, )
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - !CHAREND keyword arguments - negative])
-AT_KEYWORDS([CHAREND])
-AT_DATA([define.sps], [dnl
-DEFINE !k(arg1 = !CHAREND('/')
- /arg2 = !CHAREND('/'))
-k(!arg1, !arg2)
-!ENDDEFINE.
-DEBUG EXPAND.
-!k arg1.
-!k arg1=.
-!k arg1=x.
-!k arg1=x/ arg2=y.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:6.8: error: DEBUG EXPAND: Found `.' while expecting `=' reading
-argument !arg1 to macro !k.
- 6 | !k arg1.
- | ^
-
-k(, )
-
-define.sps:7.9: error: DEBUG EXPAND: Reached end of command expecting "/" in
-argument !arg1 to macro !k.
- 7 | !k arg1=.
- | ^
-
-k(, )
-
-define.sps:8.10: error: DEBUG EXPAND: Reached end of command expecting "/" in
-argument !arg1 to macro !k.
- 8 | !k arg1=x.
- | ^
-
-k(x, )
-
-define.sps:9.18: error: DEBUG EXPAND: Reached end of command expecting "/" in
-argument !arg2 to macro !k.
- 9 | !k arg1=x/ arg2=y.
- | ^
-
-k(x, y)
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - !ENCLOSE keyword arguments])
-AT_KEYWORDS([ENCLOSE])
-AT_DATA([define.sps], [dnl
-DEFINE !k(arg1 = !ENCLOSE('(',')')
- /arg2 = !ENCLOSE('{','}'))
-k(!arg1, !arg2)
-!ENDDEFINE.
-DEBUG EXPAND.
-!k arg1=(x) arg2={y}.
-!k arg1=(x).
-!k arg2={y}.
-!k.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-k(x, y)
-
-k(x, )
-
-k(, y)
-
-k(, )
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion - !ENCLOSE keyword arguments - negative])
-AT_KEYWORDS([ENCLOSE])
-AT_DATA([define.sps], [dnl
-DEFINE !k(arg1 = !ENCLOSE('(',')')
- /arg2 = !ENCLOSE('{','}'))
-k(!arg1, !arg2)
-!ENDDEFINE.
-DEBUG EXPAND.
-!k arg1.
-!k arg1=.
-!k arg1=x.
-!k arg1=(x.
-!k arg1=(x) arg2.
-!k arg1=(x) arg2=.
-!k arg1=(x) arg2=y.
-!k arg1=(x) arg2=(y.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:6.8: error: DEBUG EXPAND: Found `.' while expecting `=' reading
-argument !arg1 to macro !k.
- 6 | !k arg1.
- | ^
-
-k(, )
-
-define.sps:7.9: error: DEBUG EXPAND: Found `.' while expecting `@{:@' reading
-argument !arg1 to macro !k.
- 7 | !k arg1=.
- | ^
-
-k(, )
-
-define.sps:8.9: error: DEBUG EXPAND: Found `x' while expecting `@{:@' reading
-argument !arg1 to macro !k.
- 8 | !k arg1=x.
- | ^
-
-k(, )
-
-note: unexpanded token "x"
-
-define.sps:9.11: error: DEBUG EXPAND: Reached end of command expecting "@:}@" in
-argument !arg1 to macro !k.
- 9 | !k arg1=@{:@x.
- | ^
-
-k(x, )
-
-define.sps:10.17: error: DEBUG EXPAND: Found `.' while expecting `=' reading
-argument !arg2 to macro !k.
- 10 | !k arg1=(x) arg2.
- | ^
-
-k(x, )
-
-define.sps:11.18: error: DEBUG EXPAND: Found `.' while expecting `{' reading
-argument !arg2 to macro !k.
- 11 | !k arg1=(x) arg2=.
- | ^
-
-k(x, )
-
-define.sps:12.18: error: DEBUG EXPAND: Found `y' while expecting `{' reading
-argument !arg2 to macro !k.
- 12 | !k arg1=(x) arg2=y.
- | ^
-
-k(x, )
-
-note: unexpanded token "y"
-
-define.sps:13.18: error: DEBUG EXPAND: Found `@{:@' while expecting `{' reading
-argument !arg2 to macro !k.
- 13 | !k arg1=(x) arg2=@{:@y.
- | ^
-
-k(x, )
-
-note: unexpanded token "@{:@"
-
-note: unexpanded token "y"
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !BLANKS in the manual.
-AT_SETUP([macro expansion - !BLANKS])
-AT_KEYWORDS([BLANKS])
-AT_DATA([define.sps], [dnl
-DEFINE !b()
-!BLANKS(0).
-!QUOTE(!BLANKS(0)).
-!BLANKS(1).
-!QUOTE(!BLANKS(1)).
-!BLANKS(2).
-!QUOTE(!BLANKS(2)).
-!BLANKS(5).
-!QUOTE(!BLANKS(5)).
-!ENDDEFINE.
-DEBUG EXPAND.
-!b.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-.
-''.
-.
-' '.
-.
-' '.
-.
-' '.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !CONCAT in the manual.
-AT_SETUP([macro expansion - !CONCAT])
-AT_KEYWORDS([CONCAT])
-AT_DATA([define.sps], [dnl
-DEFINE !c()
-!CONCAT(x, y).
-!CONCAT('x', 'y').
-!CONCAT(12, 34).
-!CONCAT(!NULL, 123).
-!CONCAT(x, 0).
-!CONCAT(x, 0, y).
-!CONCAT(0, x).
-!CONCAT(0, x, y).
-!ENDDEFINE.
-DEBUG EXPAND.
-!c
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-xy.
-xy.
-1234.
-123.
-x0.
-x0y.
-0 x.
-0 xy.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !EVAL in the manual.
-AT_SETUP([macro expansion - !EVAL])
-AT_KEYWORDS([EVAL])
-AT_DATA([define.sps], [dnl
-DEFINE !vars() a b c !ENDDEFINE.
-
-DEFINE !e()
-!vars.
-!QUOTE(!vars).
-!EVAL(!vars).
-!QUOTE(!EVAL(!vars)).
-!ENDDEFINE
-
-DEFINE !e2(!positional !enclose('(',')'))
-!1.
-!QUOTE(!1).
-!EVAL(!1).
-!QUOTE(!EVAL(!1)).
-!ENDDEFINE.
-DEBUG EXPAND.
-!e.
-!e2(!vars).
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-a b c.
-'!vars'.
-a b c.
-'a b c'.
-
-a b c.
-'!vars'.
-a b c.
-'a b c'.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !HEAD in the manual.
-AT_SETUP([macro expansion - !HEAD])
-AT_KEYWORDS([HEAD])
-AT_DATA([define.sps], [dnl
-DEFINE !h()
-!HEAD('a b c').
-!HEAD('a').
-!HEAD(!NULL).
-!HEAD('').
-!ENDDEFINE.
-DEBUG EXPAND.
-!h.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-a.
-a.
-.
-.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !TAIL in the manual.
-AT_SETUP([macro expansion - !TAIL])
-AT_KEYWORDS([TAIL])
-AT_DATA([define.sps], [dnl
-DEFINE !t()
-!TAIL('a b c').
-!TAIL('a').
-!TAIL(!NULL).
-!TAIL('').
-!ENDDEFINE.
-DEBUG EXPAND.
-!t.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-b c.
-.
-.
-.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !INDEX in the manual.
-AT_SETUP([macro expansion - !INDEX])
-AT_KEYWORDS([INDEX])
-AT_DATA([define.sps], [dnl
-DEFINE !i()
-!INDEX(banana, an).
-!INDEX(banana, nan).
-!INDEX(banana, apple).
-!INDEX("banana", nan).
-!INDEX("banana", "nan").
-!INDEX(!UNQUOTE("banana"), !UNQUOTE("nan")).
-!ENDDEFINE.
-DEBUG EXPAND.
-!i.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-2.
-3.
-0.
-4.
-0.
-3.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !LENGTH in the manual.
-AT_SETUP([macro expansion - !LENGTH])
-AT_KEYWORDS([LENGTH])
-AT_DATA([define.sps], [dnl
-DEFINE !l()
-!LENGTH(123).
-!LENGTH(123.00).
-!LENGTH( 123 ).
-!LENGTH("123").
-!LENGTH(xyzzy).
-!LENGTH("xyzzy").
-!LENGTH("xy""zzy").
-!LENGTH(!UNQUOTE("xyzzy")).
-!LENGTH(!UNQUOTE("xy""zzy")).
-!LENGTH(!NULL).
-!ENDDEFINE.
-DEFINE !la(!positional !enclose('(',')'))
-!LENGTH(!1).
-!ENDDEFINE.
-DEBUG EXPAND.
-!l.
-!la(a b c).
-!la().
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-3.
-6.
-3.
-5.
-5.
-7.
-9.
-5.
-6.
-0.
-
-5.
-
-0.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !NULL in the manual.
-AT_SETUP([macro expansion - !NULL])
-AT_KEYWORDS([NULL])
-AT_DATA([define.sps], [dnl
-DEFINE !n()
-!NULL.
-!QUOTE(!NULL).
-!ENDDEFINE.
-DEBUG EXPAND.
-!n.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-.
-''.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !QUOTE and !UNQUOTE in the manual.
-AT_SETUP([macro expansion - !QUOTE and !UNQUOTE])
-AT_KEYWORDS([QUOTE UNQUOTE])
-AT_DATA([define.sps], [dnl
-DEFINE !q(!POS !CMDEND)
-!QUOTE(123.0).
-!QUOTE( 123 ).
-!QUOTE('a b c').
-!QUOTE("a b c").
-!QUOTE(!1).
-
-!UNQUOTE(123.0).
-!UNQUOTE( 123 ).
-!UNQUOTE('a b c').
-!UNQUOTE("a b c").
-!UNQUOTE(!1).
-
-!QUOTE(!UNQUOTE(123.0)).
-!QUOTE(!UNQUOTE( 123 )).
-!QUOTE(!UNQUOTE('a b c')).
-!QUOTE(!UNQUOTE("a b c")).
-!QUOTE(!UNQUOTE(!1)).
-!ENDDEFINE.
-DEBUG EXPAND.
-!q a 'b' c.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-'123.0'.
-'123'.
-'a b c'.
-"a b c".
-'a ''b'' c'.
-
-123.0.
-123.
-a b c.
-a b c.
-a 'b' c.
-
-'123.0'.
-'123'.
-'a b c'.
-'a b c'.
-'a ''b'' c'.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !SUBSTR in the manual.
-AT_SETUP([macro expansion - !SUBSTR])
-AT_KEYWORDS([SUBSTR])
-AT_DATA([define.sps], [dnl
-DEFINE !s()
-!SUBSTR(banana, 3).
-!SUBSTR(banana, 3, 3).
-!SUBSTR("banana", 1, 3).
-!SUBSTR(!UNQUOTE("banana"), 3).
-!SUBSTR("banana", 3, 3).
-!SUBSTR(banana, 3, 0).
-!SUBSTR(banana, 3, 10).
-!SUBSTR(banana, 10, 3).
-!ENDDEFINE.
-DEBUG EXPAND.
-!s.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1-10: At `"ba' in the expansion of `!s',dnl "
-
-define.sps:12.1-12.2: error: DEBUG EXPAND: Unterminated string constant.
- 12 | !s.
- | ^~
-
-nana.
-nan.
-.
-nana.
-ana.
-.
-nana.
-.
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with the examples for !UPCASE in the manual.
-AT_SETUP([macro expansion - !UPCASE])
-AT_KEYWORDS([UPCASE])
-AT_DATA([define.sps], [dnl
-DEFINE !u()
-!UPCASE(freckle).
-!UPCASE('freckle').
-!UPCASE('a b c').
-!UPCASE('A B C').
-!ENDDEFINE.
-DEBUG EXPAND.
-!u.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-FRECKLE.
-FRECKLE.
-A B C.
-A B C.
-])
-AT_CLEANUP
-
-dnl !* is implemented separately inside and outside function arguments
-dnl so this test makes sure to include both.
-AT_SETUP([macro expansion - !*])
-AT_DATA([define.sps], [dnl
-DEFINE !m(!POSITIONAL !TOKENS(1)
- /!POSITIONAL !TOKENS(1))
-!*/
-!LENGTH(!*)/
-!SUBSTR(!*, 3)/
-!QUOTE(!*).
-!ENDDEFINE.
-DEBUG EXPAND.
-!m 123 b
-!m 2 3
-!m '' 'b'.
-])
-AT_CAPTURE_FILE([define.sps])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-123 b / 5 / 3 b / '123 b'.
-
-2 3 / 3 / 3 / '2 3'.
-
-'' 'b' / 6 / 'b' / ''''' ''b'''.
-])
-AT_CLEANUP
-
-AT_SETUP([macro maximum nesting level (MNEST)])
-AT_KEYWORDS([MNEST])
-AT_DATA([define.sps], [dnl
-DEFINE !macro()
-!macro
-!ENDDEFINE.
-!macro.
-])
-AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
-"define.sps:1-3: In the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:1-3: inside the expansion of `!macro',
-define.sps:4.1-4.6: error: DEFINE: Maximum nesting level 50 exceeded. (Use SET MNEST to change the limit.)
- 4 | !macro.
- | ^~~~~~"
-
-"define.sps:4.1-4.6: error: In syntax expanded from `!macro': Syntax error expecting command name.
- 4 | !macro.
- | ^~~~~~"
-])
-AT_CLEANUP
-
-AT_SETUP([macro !IF condition])
-AT_KEYWORDS([if])
-for operators in \
- '!eq !ne !lt !gt !le !ge' \
- ' = <> < > <= >='
-do
- set $operators
- AS_BOX([$operators])
- cat > define.sps <<EOF
-DEFINE !test(!positional !tokens(1))
-!if (!1 $1 1) !then true !else false !ifend
-!if (!1 $2 1) !then true !else false !ifend
-!if (!1 $3 1) !then true !else false !ifend
-!if (!1 $4 1) !then true !else false !ifend
-!if (!1 $5 1) !then true !else false !ifend
-!if (!1 $6 1) !then true !else false !ifend.
-!ENDDEFINE.
-DEBUG EXPAND.
-!test 0
-!test 1
-!test 2
-!test '1'
-!test 1.0
-EOF
- AT_CAPTURE_FILE([define.sps])
- AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-false true true false true false.
-
-true false false false true true.
-
-false true false true false true.
-
-true false false false true true.
-
-false true false true false true.
-])
-done
-AT_CLEANUP
-
-AT_SETUP([macro !IF condition -- case sensitivity])
-AT_KEYWORDS([if])
-for operators in \
- '!eq !ne !lt !gt !le !ge' \
- ' = <> < > <= >='
-do
- set $operators
- AS_BOX([$operators])
- cat > define.sps <<EOF
-DEFINE !test(!positional !tokens(1))
-!if (!1 $1 a) !then true !else false !ifend
-!if (!1 $1 A) !then true !else false !ifend
-!if (!1 $2 a) !then true !else false !ifend
-!if (!1 $2 A) !then true !else false !ifend
-!if (!1 $3 a) !then true !else false !ifend
-!if (!1 $3 A) !then true !else false !ifend
-!if (!1 $4 a) !then true !else false !ifend
-!if (!1 $4 A) !then true !else false !ifend
-!if (!1 $5 a) !then true !else false !ifend
-!if (!1 $5 A) !then true !else false !ifend
-!if (!1 $6 a) !then true !else false !ifend
-!if (!1 $6 A) !then true !else false !ifend
-!if (!1 $1 !null) !then true !else false !ifend
-!if (!1 $2 !null) !then true !else false !ifend.
-!ENDDEFINE.
-DEBUG EXPAND.
-!test a
-!test A
-!test b
-!test B
-EOF
- AT_CAPTURE_FILE([define.sps])
- AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-true false false true false false false true true false true true false true.
-
-false true true false true false false false true true false true false true.
-
-false false true true false false true true false false true true false true.
-
-false false true true true false false true true false false true false true.
-])
-done
-AT_CLEANUP
-
-AT_SETUP([macro !IF condition -- logical operators])
-AT_KEYWORDS([if])
-for operators in \
- '!and !or !not' \
- ' & | ~'
-do
- set $operators
- AS_BOX([$operators])
- cat > define.sps <<EOF
-DEFINE !test_binary(!positional !tokens(1)/!positional !tokens(1))
-!if !1 $1 !2 !then true !else false !ifend
-!if !1 $2 !2 !then true !else false !ifend.
-!ENDDEFINE.
-
-DEFINE !test_unary(!positional !tokens(1))
-!if $3 !1 !then true !else false !ifend.
-!ENDDEFINE.
-
-* These are:
- ((not A) and B) or C
- not (A and B) or C
- not A and (B or C)
-DEFINE !test_prec(!pos !tokens(1)/!pos !tokens(1)/!pos !tokens(1))
-!if $3 !1 $1 !2 $2 !3 !then true !else false !ifend
-!if $3 (!1 $1 !2) $2 !3 !then true !else false !ifend
-!if $3 !1 $1 (!2 $2 !3) !then true !else false !ifend
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!test_binary 0 0
-!test_binary 0 1
-!test_binary 1 0
-!test_binary 1 1
-!test_unary 0
-!test_unary 1
-!test_prec 0 0 0 !test_prec 0 0 1 !test_prec 0 1 0 !test_prec 0 1 1.
-!test_prec 1 0 0 !test_prec 1 0 1 !test_prec 1 1 0 !test_prec 1 1 1.
-EOF
- AT_CAPTURE_FILE([define.sps])
- AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-false false.
-
-false true.
-
-false true.
-
-true true.
-
-true.
-
-false.
-
-false true false
-true true true
-true true true
-true true true
-
-false true false
-true true false
-false false false
-true true false
-])
-done
-AT_CLEANUP
-
-AT_SETUP([macro !LET])
-AT_KEYWORDS([let])
-AT_DATA([define.sps], [dnl
-DEFINE !macro(!POS !CMDEND)
-!LET !v1 = !CONCAT('x',!1,'y')
-!LET !v2 = !QUOTE(!v1)
-!LET !v3 = (!LENGTH(!1) = 1)
-!LET !v4 = (!SUBSTR(!1, 3) = !NULL)
-v1=!v1.
-v2=!v2.
-v3=!v3.
-v4=!v4.
-!ENDDEFINE.
-DEBUG EXPAND.
-!macro 0.
-!macro.
-!macro xyzzy.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-v1 = x0y.
-v2 = x0y.
-v3 = 1.
-v4 = 1.
-
-v1 = xy.
-v2 = xy.
-v3 = 0.
-v4 = 1.
-
-v1 = xxyzzyy.
-v2 = xxyzzyy.
-v3 = 0.
-v4 = 0.
-])
-AT_CLEANUP
-
-AT_SETUP([macro indexed !DO])
-AT_KEYWORDS([index do])
-AT_DATA([define.sps], [dnl
-DEFINE !title(!POS !TOKENS(1)) !1. !ENDDEFINE.
-
-DEFINE !for(!POS !TOKENS(1) / !POS !TOKENS(1))
-!DO !var = !1 !TO !2 !var !DOEND.
-!ENDDEFINE.
-
-DEFINE !forby(!POS !TOKENS(1) / !POS !TOKENS(1) / !POS !TOKENS(1))
-!DO !var = !1 !TO !2 !BY !3 !var !DOEND.
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!title "increasing".
-!for 1 5.
-!forby 1 5 1.
-!forby 1 5 2.
-!forby 1 5 2.5.
-!forby 1 5 -1.
-
-!title "decreasing".
-!for 5 1.
-!forby 5 1 1.
-!forby 5 1 -1.
-!forby 5 1 -2.
-!forby 5 1 -3.
-
-!title "non-integer".
-!for 1.5 3.5.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-"increasing".
-
-1 2 3 4 5.
-
-1 2 3 4 5.
-
-1 3 5.
-
-1 3.5.
-
-.
-
-"decreasing".
-
-.
-
-.
-
-5 4 3 2 1.
-
-5 3 1.
-
-5 2.
-
-"non-integer".
-
-1.5 2.5 3.5.
-])
-AT_CLEANUP
-
-AT_SETUP([macro !DO invalid variable names])
-AT_KEYWORDS([index do])
-AT_DATA([define.sps], [dnl
-DEFINE !for(x=!TOKENS(1) / y=!TOKENS(1))
-!DO !x = !x !TO !y !var !DOEND.
-!ENDDEFINE.
-
-DEFINE !for2(x=!TOKENS(1) / y=!TOKENS(1))
-!DO !noexpand = !x !TO !y !var !DOEND.
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!for x=1 y=5.
-!for2 x=1 y=5.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1-3: At `!x' in the expansion of `!for',
-define.sps:10.1-10.12: error: DEBUG EXPAND: Cannot use argument name or macro
-keyword as !DO variable.
- 10 | !for x=1 y=5.
- | ^~~~~~~~~~~~
-
-!DO 1 = 1 !TO 5 !var !DOEND.
-
-define.sps:5-7: At `!noexpand' in the expansion of `!for2',
-define.sps:11.1-11.13: error: DEBUG EXPAND: Cannot use argument name or macro
-keyword as !DO variable.
- 11 | !for2 x=1 y=5.
- | ^~~~~~~~~~~~~
-
-!DO !noexpand = 1 !TO 5 !var !DOEND.
-])
-AT_CLEANUP
-
-AT_SETUP([macro indexed !DO reaches MITERATE])
-AT_KEYWORDS([index do])
-AT_DATA([define.sps], [dnl
-DEFINE !title(!POS !TOKENS(1)) !1. !ENDDEFINE.
-
-DEFINE !for(!POS !TOKENS(1) / !POS !TOKENS(1))
-!DO !var = !1 !TO !2 !var !DOEND.
-!ENDDEFINE.
-
-DEFINE !forby(!POS !TOKENS(1) / !POS !TOKENS(1) / !POS !TOKENS(1))
-!DO !var = !1 !TO !2 !BY !3 !var !DOEND.
-!ENDDEFINE.
-
-SET MITERATE=3.
-DEBUG EXPAND.
-!title "increasing".
-!for 1 5.
-!forby 1 5 1.
-!forby 1 5 2.
-!forby 1 5 2.5.
-!forby 1 5 -1.
-
-!title "decreasing".
-!for 5 1.
-!forby 5 1 1.
-!forby 5 1 -1.
-!forby 5 1 -2.
-!forby 5 1 -3.
-
-!title "non-integer".
-!for 1.5 3.5.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-"increasing".
-
-In the expansion of `!DO',
-define.sps:3-5: inside the expansion of `!for',
-define.sps:14.1-14.8: error: DEBUG EXPAND: Numerical !DO loop exceeded maximum
-number of iterations 3. (Use SET MITERATE to change the limit.)
- 14 | !for 1 5.
- | ^~~~~~~~
-
-1 2 3 4.
-
-In the expansion of `!DO',
-define.sps:7-9: inside the expansion of `!forby',
-define.sps:15.1-15.12: error: DEBUG EXPAND: Numerical !DO loop exceeded maximum
-number of iterations 3. (Use SET MITERATE to change the limit.)
- 15 | !forby 1 5 1.
- | ^~~~~~~~~~~~
-
-1 2 3 4.
-
-1 3 5.
-
-1 3.5.
-
-.
-
-"decreasing".
-
-.
-
-.
-
-In the expansion of `!DO',
-define.sps:7-9: inside the expansion of `!forby',
-define.sps:23.1-23.13: error: DEBUG EXPAND: Numerical !DO loop exceeded maximum
-number of iterations 3. (Use SET MITERATE to change the limit.)
- 23 | !forby 5 1 -1.
- | ^~~~~~~~~~~~~
-
-5 4 3 2.
-
-5 3 1.
-
-5 2.
-
-"non-integer".
-
-1.5 2.5 3.5.
-])
-AT_CLEANUP
-
-AT_SETUP([!BREAK with macro indexed !DO])
-AT_KEYWORDS([index do break])
-AT_DATA([define.sps], [dnl
-DEFINE !title(!POS !TOKENS(1)) !1. !ENDDEFINE.
-
-DEFINE !for(!POS !TOKENS(1) / !POS !TOKENS(1) / !POS !TOKENS(1))
-!DO !var = !1 !TO !2
- !var
- !IF 1 !THEN
- !IF !var = !3 !THEN
- x
- !BREAK
- y
- !IFEND
- ,
- !IFEND
-!DOEND.
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!for 1 5 4.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-1, 2, 3, 4 x.
-])
-AT_CLEANUP
-
-AT_SETUP([macro list !DO])
-AT_KEYWORDS([index do])
-AT_DATA([define.sps], [dnl
-DEFINE !for(!POS !CMDEND)
-(!DO !i !IN (!1) (!i) !DOEND).
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!for a b c.
-!for 'foo bar baz quux'.
-!for.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-( (a) (b) (c) ).
-
-( (foo) (bar) (baz) (quux) ).
-
-( ).
-])
-AT_CLEANUP
-
-AT_SETUP([macro list !DO reaches MITERATE])
-AT_KEYWORDS([index do])
-AT_DATA([define.sps], [dnl
-DEFINE !for(!POS !CMDEND)
-(!DO !i !IN (!1) (!i) !DOEND).
-!ENDDEFINE.
-
-SET MITERATE=2.
-DEBUG EXPAND.
-!for a b c.
-!for 'foo bar baz quux'.
-!for.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-In the expansion of `!DO',
-define.sps:1-3: inside the expansion of `!for',
-define.sps:7.1-7.10: error: DEBUG EXPAND: !DO loop over list exceeded maximum
-number of iterations 2. (Use SET MITERATE to change the limit.)
- 7 | !for a b c.
- | ^~~~~~~~~~
-
-( (a) (b) ).
-
-In the expansion of `!DO',
-define.sps:1-3: inside the expansion of `!for',
-define.sps:8.1-8.23: error: DEBUG EXPAND: !DO loop over list exceeded maximum
-number of iterations 2. (Use SET MITERATE to change the limit.)
- 8 | !for 'foo bar baz quux'.
- | ^~~~~~~~~~~~~~~~~~~~~~~
-
-( (foo) (bar) ).
-
-( ).
-])
-AT_CLEANUP
-
-AT_SETUP([!BREAK with macro list !DO])
-AT_KEYWORDS([index break do])
-AT_DATA([define.sps], [dnl
-DEFINE !for(!POS !TOKENS(1) / !POS !CMDEND)
-(!DO !i !IN (!2)
- (!i)
- !IF 1 !THEN
- !IF !i = !1 !THEN
- x
- !BREAK
- y
- !IFEND
- ,
- !IFEND
-!DOEND).
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!for d a b c.
-!for baz 'foo bar baz quux'.
-!for e.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-( (a), (b), (c), ).
-
-( (foo), (bar), (baz)x).
-
-( ).
-])
-AT_CLEANUP
-
-AT_SETUP([macro !LET])
-AT_DATA([define.sps], [dnl
-DEFINE !macro(!pos !enclose('(',')'))
-!LET !x=!1
-!LET !y=!QUOTE(!1)
-!LET !z=(!y="abc")
-!y !z
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!macro(1+2).
-!macro(abc).
-])
-AT_CHECK([pspp --testing-mode define.sps -O format=csv], [0], [dnl
-1 + 2 0
-
-abc 1
-])
-AT_CLEANUP
-
-AT_SETUP([macro !LET invalid variable names])
-AT_DATA([define.sps], [dnl
-DEFINE !macro(x=!tokens(1))
-!LET !x=!x
-!ENDDEFINE.
-
-DEFINE !macro2()
-!LET !do=x
-!ENDDEFINE.
-
-DEBUG EXPAND.
-!macro x=1.
-!macro2.
-])
-AT_CHECK([pspp --testing-mode define.sps -O format=csv], [1], [dnl
-"define.sps:1-3: At `!x' in the expansion of `!macro',
-define.sps:10.1-10.10: error: DEBUG EXPAND: Cannot use argument name or macro keyword ""!x"" as !LET variable.
- 10 | !macro x=1.
- | ^~~~~~~~~~"
-
-!LET 1 = 1
-
-"define.sps:5-7: At `!do' in the expansion of `!macro2',
-define.sps:11.1-11.7: error: DEBUG EXPAND: Cannot use argument name or macro keyword ""!do"" as !LET variable.
- 11 | !macro2.
- | ^~~~~~~"
-
-"define.sps:5-7: At `=' in the expansion of `!macro2',
-define.sps:11.1-11.7: error: DEBUG EXPAND: Expected macro variable name following !DO.
- 11 | !macro2.
- | ^~~~~~~"
-
-!LET !do = x
-])
-AT_CLEANUP
-
-AT_SETUP([BEGIN DATA inside a macro])
-AT_DATA([define.sps], [dnl
-DEFINE !macro()
-DATA LIST NOTABLE /x 1.
-BEGIN DATA
-1
-2
-3
-END DATA.
-LIST.
-!ENDDEFINE.
-
-!macro
-])
-AT_CHECK([pspp define.sps -O format=csv], [0], [dnl
-Table: Data List
-x
-1
-2
-3
-])
-AT_CLEANUP
-
-AT_SETUP([TITLE and SUBTITLE with macros])
-AT_KEYWORDS([macro])
-for command in TITLE SUBTITLE; do
- cat >title.sps <<EOF
-DEFINE !paste(!POS !TOKENS(1) / !POS !TOKENS(1))
-!CONCAT(!1,!2)
-!ENDDEFINE.
-$command prefix !paste foo bar suffix.
-SHOW $command.
-EOF
- cat >expout <<EOF
-Table: Settings
-$command,prefix foobar suffix
-EOF
- AT_CHECK([pspp -O format=csv title.sps], [0], [expout])
-done
-AT_CLEANUP
-
-AT_SETUP([error message within macro expansion])
-AT_DATA([define.sps], [dnl
-DEFINE !vars(!POS !TOKENS(1)) a b C !ENDDEFINE.
-DATA LIST NOTABLE /a b 1-2.
-COMPUTE x = !vars x.
-])
-AT_CHECK([pspp -O format=csv define.sps], [1], [dnl
-"define.sps:3.13-3.19: error: COMPUTE: In syntax expanded from `!vars x': Syntax error expecting end of command.
- 3 | COMPUTE x = !vars x.
- | ^~~~~~~"
-])
-AT_CLEANUP
-
-dnl A macro with keyword arguments needs a token of lookahead
-dnl to find out whether another keyword is present. Test that
-dnl this special case works OK.
-AT_SETUP([macro calls in each others' lookahead])
-AT_DATA([define.sps], [dnl
-DEFINE !k(x=!DEFAULT(0) !TOKENS(1)/y=!DEFAULT(0) !TOKENS(1))
-(x=!x)(y=!y)
-!ENDDEFINE.
-DEBUG EXPAND.
-!k
-!k x=1
-!k y=2
-!k y=2 x=1
-!k x=1 y=2.
-])
-AT_CHECK([pspp -O format=csv define.sps --testing-mode], [0], [dnl
-(x = 0) (y = 0)
-
-(x = 1) (y = 0)
-
-(x = 0) (y = 2)
-(x = 1) (y = 2)
-
-(x = 1) (y = 2)
-])
-AT_CLEANUP
-
-AT_SETUP([bad token in macro body])
-AT_DATA([define.sps], [dnl
-DEFINE !x()
-x'123'
-!ENDDEFINE.
-])
-AT_CHECK([pspp define.sps], [1], [dnl
-define.sps:2.1-2.6: error: DEFINE: String of hex digits has 3 characters, which
-is not a multiple of 2.
- 2 | x'123'
- | ^~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([macro name overlaps with macro function name])
-dnl !len is short for macro function !LENGTH. PSPP used to
-dnl reject the following with "`(' expected following !LENGTH".
-dnl Now PSPP only consider macro functions when the name is
-dnl followed by '('.
-AT_DATA([define.sps], [dnl
-DEFINE !len() 5 !ENDDEFINE.
-DEFINE !x() !eval(!len) !ENDDEFINE.
-DEBUG EXPAND.
-!x
-])
-AT_CHECK([pspp -O format=csv define.sps --testing-mode], [0], [dnl
-5
-])
-AT_CLEANUP
-
-AT_SETUP([generic macro function syntax errors])
-AT_DATA([define.sps], [dnl
-
-
-DEFINE !c() !SUBSTR(1x) !ENDDEFINE.
-DEFINE !d() !SUBSTR(1 !ENDDEFINE.
-DEFINE !narg_blanks() !BLANKS() !ENDDEFINE.
-DEFINE !narg_concat() !CONCAT() !ENDDEFINE.
-DEFINE !narg_eval() !EVAL() !ENDDEFINE.
-DEFINE !narg_head() !HEAD() !ENDDEFINE.
-DEFINE !narg_index() !INDEX() !ENDDEFINE.
-DEFINE !narg_length() !LENGTH() !ENDDEFINE.
-DEFINE !narg_null() !NULL() !ENDDEFINE.
-DEFINE !narg_quote() !QUOTE() !ENDDEFINE.
-DEFINE !narg_substr() !SUBSTR() !ENDDEFINE.
-DEFINE !narg_tail() !TAIL() !ENDDEFINE.
-DEFINE !narg_unquote() !UNQUOTE() !ENDDEFINE.
-DEFINE !narg_upcase() !UPCASE() !ENDDEFINE.
-dnl )
-DEBUG EXPAND.
-
-
-!c.
-!d.
-!narg_blanks.
-!narg_concat.
-!narg_eval.
-!narg_head.
-!narg_index.
-!narg_length.
-!narg_null.
-!narg_quote.
-!narg_substr.
-!narg_tail.
-!narg_unquote.
-!narg_upcase.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:3: At `x' in the expansion of `!c',
-define.sps:20.1-20.2: error: DEBUG EXPAND: `,' or `@:}@' expected in call to macro
-function !SUBSTR.
- 20 | !c.
- | ^~
-
-!SUBSTR(1 x)
-
-define.sps:4: In the expansion of `!d',
-define.sps:21.1-21.2: error: DEBUG EXPAND: Missing `@:}@' in call to macro
-function !SUBSTR.
- 21 | !d.
- | ^~
-
-!SUBSTR@{:@1
-
-define.sps:5: In the expansion of `!narg_blanks',
-define.sps:22.1-22.12: error: DEBUG EXPAND: Macro function !BLANKS takes one
-argument (not 0).
- 22 | !narg_blanks.
- | ^~~~~~~~~~~~
-
-!BLANKS( )
-
-define.sps:6: In the expansion of `!narg_concat',
-define.sps:23.1-23.12: error: DEBUG EXPAND: Macro function !CONCAT needs at
-least one argument.
- 23 | !narg_concat.
- | ^~~~~~~~~~~~
-
-!CONCAT( )
-
-define.sps:7: In the expansion of `!narg_eval',
-define.sps:24.1-24.10: error: DEBUG EXPAND: Macro function !EVAL takes one
-argument (not 0).
- 24 | !narg_eval.
- | ^~~~~~~~~~
-
-!EVAL( )
-
-define.sps:8: In the expansion of `!narg_head',
-define.sps:25.1-25.10: error: DEBUG EXPAND: Macro function !HEAD takes one
-argument (not 0).
- 25 | !narg_head.
- | ^~~~~~~~~~
-
-!HEAD( )
-
-define.sps:9: In the expansion of `!narg_index',
-define.sps:26.1-26.11: error: DEBUG EXPAND: Macro function !INDEX takes two
-arguments (not 0).
- 26 | !narg_index.
- | ^~~~~~~~~~~
-
-!INDEX( )
-
-define.sps:10: In the expansion of `!narg_length',
-define.sps:27.1-27.12: error: DEBUG EXPAND: Macro function !LENGTH takes one
-argument (not 0).
- 27 | !narg_length.
- | ^~~~~~~~~~~~
-
-!LENGTH( )
-
-( )
-
-define.sps:12: In the expansion of `!narg_quote',
-define.sps:29.1-29.11: error: DEBUG EXPAND: Macro function !QUOTE takes one
-argument (not 0).
- 29 | !narg_quote.
- | ^~~~~~~~~~~
-
-!QUOTE( )
-
-define.sps:13: In the expansion of `!narg_substr',
-define.sps:30.1-30.12: error: DEBUG EXPAND: Macro function !SUBSTR takes two or
-three arguments (not 0).
- 30 | !narg_substr.
- | ^~~~~~~~~~~~
-
-!SUBSTR( )
-
-define.sps:14: In the expansion of `!narg_tail',
-define.sps:31.1-31.10: error: DEBUG EXPAND: Macro function !TAIL takes one
-argument (not 0).
- 31 | !narg_tail.
- | ^~~~~~~~~~
-
-!TAIL( )
-
-define.sps:15: In the expansion of `!narg_unquote',
-define.sps:32.1-32.13: error: DEBUG EXPAND: Macro function !UNQUOTE takes one
-argument (not 0).
- 32 | !narg_unquote.
- | ^~~~~~~~~~~~~
-
-!UNQUOTE( )
-
-define.sps:16: In the expansion of `!narg_upcase',
-define.sps:33.1-33.12: error: DEBUG EXPAND: Macro function !UPCASE takes one
-argument (not 0).
- 33 | !narg_upcase.
- | ^~~~~~~~~~~~
-
-!UPCASE( )
-])
-AT_CLEANUP
-
-AT_SETUP([specific macro function syntax errors])
-AT_DATA([define.sps], [dnl
-DEFINE !a() !BLANKS(x). !ENDDEFINE.
-DEFINE !b() !SUBSTR(x, y). !ENDDEFINE.
-DEFINE !c() !SUBSTR(x, 1, z). !ENDDEFINE.
-DEBUG EXPAND.
-!a.
-!b.
-!c.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1: In the expansion of `!a',
-define.sps:5.1-5.2: error: DEBUG EXPAND: Argument to !BLANKS must be non-
-negative integer (not "x").
- 5 | !a.
- | ^~
-
-!BLANKS(x).
-
-define.sps:2: In the expansion of `!b',
-define.sps:6.1-6.2: error: DEBUG EXPAND: Second argument of !SUBSTR must be
-positive integer (not "y").
- 6 | !b.
- | ^~
-
-!SUBSTR(x, y).
-
-define.sps:3: In the expansion of `!c',
-define.sps:7.1-7.2: error: DEBUG EXPAND: Third argument of !SUBSTR must be non-
-negative integer (not "z").
- 7 | !c.
- | ^~
-
-!SUBSTR(x, 1, z).
-])
-AT_CLEANUP
-
-AT_SETUP([macro expression errors])
-AT_DATA([define.sps], [dnl
-DEFINE !a() !LET !x = (1. !ENDDEFINE dnl )
-
-DEFINE !b() !DO !x = x. !ENDDEFINE.
-DEFINE !c() !LET !x = (). !ENDDEFINE.
-DEBUG EXPAND.
-!a.
-!b.
-!c.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1-2: At `.' in the expansion of `!a',
-define.sps:5.1-5.2: error: DEBUG EXPAND: Expecting ')' in macro expression.
- 5 | !a.
- | ^~
-
-!LET !x = (1.
-
-At `x' in the expansion of `!DO',
-define.sps:2: inside the expansion of `!b',
-define.sps:6.1-6.2: error: DEBUG EXPAND: Macro expression must evaluate to a
-number (not "x").
- 6 | !b.
- | ^~
-
-!DO !x = x.
-
-define.sps:3: At `)' in the expansion of `!c',
-define.sps:7.1-7.2: error: DEBUG EXPAND: Expecting literal or function
-invocation in macro expression.
- 7 | !c.
- | ^~
-
-!LET !x = ( ).
-])
-AT_CLEANUP
-
-AT_SETUP([macro !IF errors])
-AT_KEYWORDS([IF])
-AT_DATA([define.sps], [dnl
-DEFINE !a() !IF 1 !ENDDEFINE.
-DEFINE !b() !IF 1 !THEN !ENDDEFINE.
-DEFINE !c() !IF 1 !THEN !ELSE !ENDDEFINE.
-DEBUG EXPAND.
-!a.
-!b.
-!c.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1: In the expansion of `!a',
-define.sps:5.1-5.2: error: DEBUG EXPAND: !THEN expected in macro !IF construct.
- 5 | !a.
- | ^~
-
-!IF 1
-
-define.sps:2: In the expansion of `!b',
-define.sps:6.1-6.2: error: DEBUG EXPAND: !ELSE or !IFEND expected in macro !IF
-construct.
- 6 | !b.
- | ^~
-
-!IF 1 !THEN
-
-define.sps:3: In the expansion of `!c',
-define.sps:7.1-7.2: error: DEBUG EXPAND: !IFEND expected in macro !IF
-construct.
- 7 | !c.
- | ^~
-
-!IF 1 !THEN !ELSE
-])
-AT_CLEANUP
-
-AT_SETUP([macro !LET errors])
-AT_KEYWORDS([LET])
-AT_DATA([define.sps], [dnl
-DEFINE !a() !LET !ENDDEFINE.
-DEFINE !b() !LET 0 !ENDDEFINE.
-DEFINE !c() !LET !x !ENDDEFINE.
-DEFINE !d() !LET !x y !ENDDEFINE.
-DEBUG EXPAND.
-!a.
-!b.
-!c.
-!d.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1: In the expansion of `!a',
-define.sps:6.1-6.2: error: DEBUG EXPAND: Expected macro variable name following
-!LET.
- 6 | !a.
- | ^~
-
-!LET
-
-define.sps:2: At `0' in the expansion of `!b',
-define.sps:7.1-7.2: error: DEBUG EXPAND: Expected macro variable name following
-!LET.
- 7 | !b.
- | ^~
-
-!LET 0
-
-define.sps:3: In the expansion of `!c',
-define.sps:8.1-8.2: error: DEBUG EXPAND: Expected `=' following !LET.
- 8 | !c.
- | ^~
-
-!LET !x
-
-define.sps:4: At `y' in the expansion of `!d',
-define.sps:9.1-9.2: error: DEBUG EXPAND: Expected `=' following !LET.
- 9 | !d.
- | ^~
-
-!LET !x y
-])
-AT_CLEANUP
-
-AT_SETUP([macro !DO errors])
-AT_KEYWORDS([DO])
-AT_DATA([define.sps], [dnl
-DEFINE !a() !DO !ENDDEFINE.
-DEFINE !b() !DO 0 !ENDDEFINE.
-DEFINE !c() !DO !x !ENDDEFINE.
-DEFINE !d() !DO !x !in (x) !ENDDEFINE.
-DEFINE !e() !DO !x = x. !ENDDEFINE.
-DEFINE !f() !DO !x = 5 x !ENDDEFINE.
-DEFINE !g() !DO !x = 5 !TO 6 !BY 0 !ENDDEFINE.
-DEFINE !h() !DO !x !ENDDEFINE.
-DEFINE !i() !DO !x 0 !ENDDEFINE.
-DEFINE !j() !BREAK !ENDDEFINE.
-DEBUG EXPAND.
-!a.
-!b.
-!c.
-!d.
-!e.
-!f.
-!g.
-!h.
-!i.
-!j.
-])
-AT_CHECK([pspp --testing-mode define.sps], [1], [dnl
-define.sps:1: In the expansion of `!a',
-define.sps:12.1-12.2: error: DEBUG EXPAND: Expected macro variable name
-following !DO.
- 12 | !a.
- | ^~
-
-!DO
-
-define.sps:2: At `0' in the expansion of `!b',
-define.sps:13.1-13.2: error: DEBUG EXPAND: Expected macro variable name
-following !DO.
- 13 | !b.
- | ^~
-
-!DO 0
-
-define.sps:3: In the expansion of `!c',
-define.sps:14.1-14.2: error: DEBUG EXPAND: Expected `=' or !IN in !DO loop.
- 14 | !c.
- | ^~
-
-!DO !x
-
-In the expansion of `!DO',
-define.sps:4: inside the expansion of `!d',
-define.sps:15.1-15.2: error: DEBUG EXPAND: Missing !DOEND.
- 15 | !d.
- | ^~
-
-!DO !x !in(x)
-
-At `x' in the expansion of `!DO',
-define.sps:5: inside the expansion of `!e',
-define.sps:16.1-16.2: error: DEBUG EXPAND: Macro expression must evaluate to a
-number (not "x").
- 16 | !e.
- | ^~
-
-!DO !x = x.
-
-At `x' in the expansion of `!DO',
-define.sps:6: inside the expansion of `!f',
-define.sps:17.1-17.2: error: DEBUG EXPAND: Expected !TO in numerical !DO loop.
- 17 | !f.
- | ^~
-
-!DO !x = 5 x
-
-In the expansion of `!DO',
-define.sps:7: inside the expansion of `!g',
-define.sps:18.1-18.2: error: DEBUG EXPAND: !BY value cannot be zero.
- 18 | !g.
- | ^~
-
-!DO !x = 5 !TO 6 !BY 0
-
-define.sps:8: In the expansion of `!h',
-define.sps:19.1-19.2: error: DEBUG EXPAND: Expected `=' or !IN in !DO loop.
- 19 | !h.
- | ^~
-
-!DO !x
-
-define.sps:9: At `0' in the expansion of `!i',
-define.sps:20.1-20.2: error: DEBUG EXPAND: Expected `=' or !IN in !DO loop.
- 20 | !i.
- | ^~
-
-!DO !x 0
-
-define.sps:10: At `!BREAK' in the expansion of `!j',
-define.sps:21.1-21.2: error: DEBUG EXPAND: !BREAK outside !DO.
- 21 | !j.
- | ^~
-
-])
-AT_CLEANUP
-
-AT_SETUP([macros in comments])
-AT_KEYWORDS([macro])
-AT_DATA([define.sps], [dnl
-DEFINE !macro() x y z !ENDDEFINE.
-/* !macro.
-*!macro.
-DEBUG EXPAND.
-!macro.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-x y z
-])
-AT_CLEANUP
-
-AT_SETUP([DEFINE syntax errors])
-AT_KEYWORDS([macro])
-AT_DATA([define.sps], [dnl
-DEFINE !macro(!POSITIONAL !CHAREND('x y')) !ENDDEFINE.
-DEFINE !macro(a=!TOKENS(1)/!POSITIONAL !TOKENS(1)) !ENDDEFINE.
-DEFINE !macro(!a=!TOKENS(1)) !ENDDEFINE.
-DEFINE !macro(do=!TOKENS(1)) !ENDDEFINE.
-DEFINE 0() !ENDDEFINE.
-DEFINE x y () !ENDDEFINE.
-DEFINE !macro(1) !ENDDEFINE.
-DEFINE !macro(x 2) !ENDDEFINE.
-DEFINE !macro(x=!DEFAULT 3) !ENDDEFINE.
-DEFINE !macro(x=!TOKENS 4) !ENDDEFINE.
-DEFINE !macro(x=!TOKENS(x)) !ENDDEFINE.
-DEFINE !macro(x=!TOKENS(1 5)) !ENDDEFINE.
-DEFINE !macro(x=!ENCLOSE 6) !ENDDEFINE.
-DEFINE !macro(x=!ENCLOSE('x' y)) !ENDDEFINE.
-DEFINE !macro(x=!ENCLOSE('x',y)) !ENDDEFINE.
-DEFINE !macro(x=!ENCLOSE('x','y' z)) !ENDDEFINE.
-DEFINE !macro(x=!CHAREND 7) !ENDDEFINE.
-DEFINE !macro(x=!CHAREND(8)) !ENDDEFINE.
-DEFINE !macro(x=!CHAREND('x' 9)) !ENDDEFINE.
-DEFINE !macro(x=!WTF) !ENDDEFINE.
-DEFINE !macro(x=!TOKENS(1) x) !ENDDEFINE.
-DEFINE !macro(x=!DEFAULT() !DEFAULT()) !ENDDEFINE.
-DEFINE !macro(x=!TOKENS(1) !CMDEND) !ENDDEFINE.
-DEFINE !macro()
-])
-AT_CHECK([pspp define.sps], [1], [dnl
-define.sps:1.36-1.40: error: DEFINE: String must contain exactly one token.
- 1 | DEFINE !macro(!POSITIONAL !CHAREND('x y')) !ENDDEFINE.
- | ^~~~~
-
-define.sps:2.28-2.38: error: DEFINE: Positional parameters must precede keyword
-parameters.
- 2 | DEFINE !macro(a=!TOKENS(1)/!POSITIONAL !TOKENS(1)) !ENDDEFINE.
- | ^~~~~~~~~~~
-
-define.sps:2.15: note: DEFINE: Here is a previous keyword parameter.
- 2 | DEFINE !macro(a=!TOKENS(1)/!POSITIONAL !TOKENS(1)) !ENDDEFINE.
- | ^
-
-define.sps:3.15-3.16: error: DEFINE: Keyword macro parameter must be named in
-definition without "!" prefix.
- 3 | DEFINE !macro(!a=!TOKENS(1)) !ENDDEFINE.
- | ^~
-
-define.sps:4.15-4.16: error: DEFINE: Cannot use macro keyword "do" as an
-argument name.
- 4 | DEFINE !macro(do=!TOKENS(1)) !ENDDEFINE.
- | ^~
-
-define.sps:5.8: error: DEFINE: Syntax error expecting identifier.
- 5 | DEFINE 0() !ENDDEFINE.
- | ^
-
-define.sps:6.10: error: DEFINE: Syntax error expecting `@{:@'.
- 6 | DEFINE x y () !ENDDEFINE.
- | ^
-
-define.sps:7.15: error: DEFINE: Syntax error expecting identifier.
- 7 | DEFINE !macro(1) !ENDDEFINE.
- | ^
-
-define.sps:8.17: error: DEFINE: Syntax error expecting !TOKENS, !CHAREND, !
-ENCLOSE, or !CMDEND.
- 8 | DEFINE !macro(x 2) !ENDDEFINE.
- | ^
-
-define.sps:9.26: error: DEFINE: Syntax error expecting `@{:@'.
- 9 | DEFINE !macro(x=!DEFAULT 3) !ENDDEFINE.
- | ^
-
-define.sps:10.25: error: DEFINE: Syntax error expecting `@{:@'.
- 10 | DEFINE !macro(x=!TOKENS 4) !ENDDEFINE.
- | ^
-
-define.sps:11.25: error: DEFINE: Syntax error expecting positive integer for !
-TOKENS.
- 11 | DEFINE !macro(x=!TOKENS(x)) !ENDDEFINE.
- | ^
-
-define.sps:12.27: error: DEFINE: Syntax error expecting `@:}@'.
- 12 | DEFINE !macro(x=!TOKENS(1 5)) !ENDDEFINE.
- | ^
-
-define.sps:13.26: error: DEFINE: Syntax error expecting `@{:@'.
- 13 | DEFINE !macro(x=!ENCLOSE 6) !ENDDEFINE.
- | ^
-
-define.sps:14.30: error: DEFINE: Syntax error expecting `,'.
- 14 | DEFINE !macro(x=!ENCLOSE('x' y)) !ENDDEFINE.
- | ^
-
-define.sps:15.30: error: DEFINE: Syntax error expecting string.
- 15 | DEFINE !macro(x=!ENCLOSE('x',y)) !ENDDEFINE.
- | ^
-
-define.sps:16.34: error: DEFINE: Syntax error expecting `@:}@'.
- 16 | DEFINE !macro(x=!ENCLOSE('x','y' z)) !ENDDEFINE.
- | ^
-
-define.sps:17.26: error: DEFINE: Syntax error expecting `@{:@'.
- 17 | DEFINE !macro(x=!CHAREND 7) !ENDDEFINE.
- | ^
-
-define.sps:18.26: error: DEFINE: Syntax error expecting string.
- 18 | DEFINE !macro(x=!CHAREND(8)) !ENDDEFINE.
- | ^
-
-define.sps:19.30: error: DEFINE: Syntax error expecting `@:}@'.
- 19 | DEFINE !macro(x=!CHAREND('x' 9)) !ENDDEFINE.
- | ^
-
-define.sps:20.17-20.20: error: DEFINE: Syntax error expecting !TOKENS, !
-CHAREND, !ENCLOSE, or !CMDEND.
- 20 | DEFINE !macro(x=!WTF) !ENDDEFINE.
- | ^~~~
-
-define.sps:21.28: error: DEFINE: Syntax error expecting `/'.
- 21 | DEFINE !macro(x=!TOKENS(1) x) !ENDDEFINE.
- | ^
-
-define.sps:22.28-22.35: error: DEFINE: !DEFAULT is allowed only once per
-argument.
- 22 | DEFINE !macro(x=!DEFAULT() !DEFAULT()) !ENDDEFINE.
- | ^~~~~~~~
-
-define.sps:23.28-23.34: error: DEFINE: Only one of !TOKENS, !CHAREND, !ENCLOSE,
-or !CMDEND is allowed.
- 23 | DEFINE !macro(x=!TOKENS(1) !CMDEND) !ENDDEFINE.
- | ^~~~~~~
-
-define.sps:25.1: error: DEFINE: Syntax error expecting macro body or !
-ENDDEFINE.
- 25 |
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([macro expansion with token merging])
-AT_DATA([define.sps], [dnl
-DEFINE !foo() "foo" !ENDDEFINE.
-DEFINE !bar() "bar" !ENDDEFINE.
-DEFINE !plus() + !ENDDEFINE.
-DEFINE !minus() - !ENDDEFINE.
-DEFINE !one() 1 !ENDDEFINE.
-ECHO "foo" + "bar".
-ECHO !foo.
-ECHO !bar.
-ECHO !foo + "quux".
-ECHO "baz" + !bar.
-ECHO !foo + !bar.
-ECHO !foo !plus !bar.
-ECHO "two" "strings".
-N OF CASES -/**/1.
-N OF CASES !minus 1.
-N OF CASES - !one.
-N OF CASES !minus !one.
-])
-AT_CHECK([pspp define.sps], [1], [dnl
-foobar
-
-foo
-
-bar
-
-fooquux
-
-bazbar
-
-foobar
-
-foobar
-
-two
-
-define.sps:13.12-13.20: error: ECHO: Syntax error expecting end of command.
- 13 | ECHO "two" "strings".
- | ^~~~~~~~~
-
-define.sps:14.12-14.17: error: N OF CASES: Syntax error expecting positive
-integer for N OF CASES.
- 14 | N OF CASES -/**/1.
- | ^~~~~~
-
-define.sps:15.12-15.19: error: N OF CASES: Syntax error expecting positive
-integer for N OF CASES.
- 15 | N OF CASES !minus 1.
- | ^~~~~~~~
-
-define.sps:16.12-16.17: error: N OF CASES: Syntax error expecting positive
-integer for N OF CASES.
- 16 | N OF CASES - !one.
- | ^~~~~~
-
-define.sps:17.12-17.22: error: N OF CASES: Syntax error expecting positive
-integer for N OF CASES.
- 17 | N OF CASES !minus !one.
- | ^~~~~~~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([one macro calls another])
-AT_DATA([define.sps], [dnl
-DEFINE !a(!pos !enclose('(',')')) [[!1]] !ENDDEFINE.
-DEFINE !b(!pos !enclose('{','}')) !a(x !1 z) !ENDDEFINE.
-DEFINE !c(!pos !enclose('{','}')) !let !tmp=!quote(!concat('<',!1,'>')) !a(!tmp) !ENDDEFINE.
-DEBUG EXPAND.
-!b{y}.
-!c{y}.
-])
-AT_CHECK([pspp --testing-mode define.sps], [0], [dnl
-[[x y z]]
-
-[[ < y > ]]
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DO IF])
-
-AT_SETUP([DO IF])
-(for a in 0 1 ' '; do
- for b in 0 1 ' '; do
- for c in 0 1 ' '; do
- for d in 0 1 ' '; do
- abcd=$a$b$c$d
- echo "$abcd" 1>&3
- if test "$a" = "1"; then
- echo " $abcd A"
- elif test "$a" = " "; then
- :
- elif test "$b" = "1"; then
- echo " $abcd B"
- elif test "$b" = " "; then
- :
- elif test "$c" = "1"; then
- echo " $abcd C"
- elif test "$c" = " "; then
- :
- elif test "$d" = "1"; then
- echo " $abcd D"
- elif test "$d" = " "; then
- :
- else
- echo " $abcd E"
- fi
- done
- done
- done
-done) >expout 3>do-if.txt || exit 99
-AT_DATA([do-if.sps], [dnl
-DATA LIST FILE="do-if.txt"/A B C D 1-4 ABCD 1-4 (A).
-DO IF A.
-PRINT OUTFILE="do-if.out"/ABCD 'A'.
-ELSE IF B.
-PRINT OUTFILE="do-if.out"/ABCD 'B'.
-ELSE IF C.
-PRINT OUTFILE="do-if.out"/ABCD 'C'.
-ELSE IF D.
-PRINT OUTFILE="do-if.out"/ABCD 'D'.
-ELSE.
-PRINT OUTFILE="do-if.out"/ABCD 'E'.
-END IF.
-EXECUTE.
-])
-AT_CHECK([pspp do-if.sps], [0], [ignore])
-AT_CHECK([cat do-if.out], [0], [expout])
-AT_CLEANUP
-
-AT_SETUP([DO IF - negative])
-AT_DATA([do-if.sps], [dnl
-DATA LIST LIST NOTABLE/a b c.
-BEGIN DATA.
-1 2 3
-END DATA.
-
-END IF.
-ELSE.
-ELSE IF 1.
-
-DO IF 0.
-ELSE.
-ELSE.
-END IF.
-
-DO IF 0.
-ELSE.
-ELSE IF 0.
-END IF.
-
-DO IF !.
-END IF.
-
-DO IF 0.
-])
-AT_CHECK([pspp -O format=csv do-if.sps], [1], [dnl
-"do-if.sps:6.1-6.6: error: END IF: This command cannot appear outside DO IF...END IF.
- 6 | END IF.
- | ^~~~~~"
-
-"do-if.sps:7.1-7.4: error: ELSE: This command cannot appear outside DO IF...END IF.
- 7 | ELSE.
- | ^~~~"
-
-"do-if.sps:8.1-8.7: error: ELSE IF: This command cannot appear outside DO IF...END IF.
- 8 | ELSE IF 1.
- | ^~~~~~~"
-
-"do-if.sps:12.1-12.4: error: DO IF: Only one ELSE is allowed within DO IF...END IF.
- 12 | ELSE.
- | ^~~~"
-
-"do-if.sps:11.1-11.5: note: DO IF: This is the location of the previous ELSE clause.
- 11 | ELSE.
- | ^~~~~"
-
-"do-if.sps:10.1-10.8: note: DO IF: This is the location of the DO IF command.
- 10 | DO IF 0.
- | ^~~~~~~~"
-
-"do-if.sps:17.1-17.7: error: DO IF: ELSE IF is not allowed following ELSE within DO IF...END IF.
- 17 | ELSE IF 0.
- | ^~~~~~~"
-
-"do-if.sps:16.1-16.5: note: DO IF: This is the location of the previous ELSE clause.
- 16 | ELSE.
- | ^~~~~"
-
-"do-if.sps:15.1-15.8: note: DO IF: This is the location of the DO IF command.
- 15 | DO IF 0.
- | ^~~~~~~~"
-
-"do-if.sps:20.7: error: DO IF: Syntax error parsing expression.
- 20 | DO IF !.
- | ^"
-
-error: DO IF: At end of input: Syntax error expecting END IF.
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DO REPEAT])
-
-AT_SETUP([DO REPEAT -- simple])
-AT_DATA([do-repeat.sps], [dnl
-INPUT PROGRAM.
-STRING y(A1).
-DO REPEAT xval = 1 2 3 / yval = 'a' 'b' 'c' / var = a b c.
-COMPUTE x=xval.
-COMPUTE y=yval.
-COMPUTE var=xval.
-END CASE.
-END REPEAT PRINT.
-END FILE.
-END INPUT PROGRAM.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv do-repeat.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-COMPUTE x=3.
-COMPUTE y='c'.
-COMPUTE c=3.
-END CASE.
-
-COMPUTE x=2.
-COMPUTE y='b'.
-COMPUTE b=2.
-END CASE.
-
-COMPUTE x=1.
-COMPUTE y='a'.
-COMPUTE a=1.
-END CASE.
-
-Table: Data List
-y,x,a,b,c
-a,1.00,1.00,. ,. @&t@
-b,2.00,. ,2.00,. @&t@
-c,3.00,. ,. ,3.00
-])
-AT_CLEANUP
-
-AT_SETUP([DO REPEAT -- containing BEGIN DATA])
-AT_DATA([do-repeat.sps], [dnl
-DO REPEAT offset = 1 2 3.
-DATA LIST NOTABLE /x 1-2.
-BEGIN DATA.
-10
-20
-30
-END DATA.
-COMPUTE x = x + offset.
-LIST.
-END REPEAT.
-])
-AT_CHECK([pspp -o pspp.csv do-repeat.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-x
-11
-21
-31
-
-Table: Data List
-x
-12
-22
-32
-
-Table: Data List
-x
-13
-23
-33
-])
-AT_CLEANUP
-
-AT_SETUP([DO REPEAT -- dummy vars not expanded in include files])
-AT_DATA([include.sps], [dnl
-COMPUTE y = y + x + 10.
-])
-AT_DATA([do-repeat.sps], [dnl
-INPUT PROGRAM.
-COMPUTE x = 0.
-COMPUTE y = 0.
-END CASE.
-END FILE.
-END INPUT PROGRAM.
-
-DO REPEAT x = 1 2 3.
-INCLUDE include.sps.
-END REPEAT.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv do-repeat.sps], [0], [dnl
-do-repeat.sps:8.11: warning: DO REPEAT: Dummy variable name `x' hides dictionary variable `x'.
- 8 | DO REPEAT x = 1 2 3.
- | ^
-])
-AT_CHECK([cat pspp.csv], [0], [dnl
-"do-repeat.sps:8.11: warning: DO REPEAT: Dummy variable name `x' hides dictionary variable `x'.
- 8 | DO REPEAT x = 1 2 3.
- | ^"
-
-Table: Data List
-x,y
-.00,30.00
-])
-AT_CLEANUP
-
-AT_SETUP([DO REPEAT -- nested])
-AT_DATA([do-repeat.sps], [dnl
-DATA LIST NOTABLE /a 1.
-BEGIN DATA.
-0
-END DATA.
-
-DO REPEAT h = h0 TO h3 / x = 0 TO 3 / y = 8, 7.5, 6, 5.
- COMPUTE h = x + y.
-END REPEAT.
-
-VECTOR v(6).
-COMPUTE #idx = 0.
-DO REPEAT i = 1 TO 2.
- DO REPEAT j = 3 TO 5.
- COMPUTE #x = i + j.
- COMPUTE #idx = #idx + 1.
- COMPUTE v(#idx) = #x.
- END REPEAT.
-END REPEAT.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv do-repeat.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-a,h0,h1,h2,h3,v1,v2,v3,v4,v5,v6
-0,8.00,8.50,8.00,8.00,4.00,5.00,6.00,5.00,6.00,7.00
-])
-AT_CLEANUP
-
-dnl This program tests for a bug that crashed PSPP given an empty DO
-dnl REPEAT...END REPEAT block. See bug #18407.
-AT_SETUP([DO REPEAT -- empty])
-AT_DATA([do-repeat.sps], [dnl
-DATA LIST NOTABLE /a 1.
-BEGIN DATA.
-0
-END DATA.
-
-DO REPEAT h = a.
-END REPEAT.
-])
-AT_CHECK([pspp -o pspp.csv do-repeat.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-])
-AT_CLEANUP
-
-dnl This program tests for a bug that crashed PSPP when END REPEAT
-dnl was missing. See bug #31016.
-AT_SETUP([DO REPEAT -- missing END REPEAT])
-AT_DATA([do-repeat.sps], [dnl
-DATA LIST NOTABLE /x 1.
-DO REPEAT y = 1 TO 10.
-])
-AT_CHECK([pspp -O format=csv do-repeat.sps], [1], [dnl
-error: DO REPEAT: At end of input: Syntax error expecting END REPEAT.
-])
-AT_CLEANUP
-
-AT_SETUP([DO REPEAT -- syntax errors])
-AT_DATA([do-repeat.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-DO REPEAT **.
-END REPEAT.
-DO REPEAT x **.
-END REPEAT.
-DO REPEAT y=1/y=2.
-END REPEAT.
-DO REPEAT y=a b c **.
-END REPEAT.
-DO REPEAT y=1 2 **.
-END REPEAT.
-DO REPEAT y='a' 'b' **.
-END REPEAT.
-DO REPEAT y=**.
-END REPEAT.
-DO REPEAT y=1 2 3/z=4.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='do-repeat.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"do-repeat.sps:2.11-2.12: error: DO REPEAT: Syntax error expecting identifier.
- 2 | DO REPEAT **.
- | ^~"
-
-"do-repeat.sps:4.11: warning: DO REPEAT: Dummy variable name `x' hides dictionary variable `x'.
- 4 | DO REPEAT x **.
- | ^"
-
-"do-repeat.sps:4.13-4.14: error: DO REPEAT: Syntax error expecting `='.
- 4 | DO REPEAT x **.
- | ^~"
-
-"do-repeat.sps:6.15: error: DO REPEAT: Dummy variable name `y' is given twice.
- 6 | DO REPEAT y=1/y=2.
- | ^"
-
-"do-repeat.sps:8.19-8.20: error: DO REPEAT: Syntax error expecting `/' or end of command.
- 8 | DO REPEAT y=a b c **.
- | ^~"
-
-"do-repeat.sps:10.17-10.18: error: DO REPEAT: Syntax error expecting number.
- 10 | DO REPEAT y=1 2 **.
- | ^~"
-
-"do-repeat.sps:12.21-12.22: error: DO REPEAT: Syntax error expecting string.
- 12 | DO REPEAT y='a' 'b' **.
- | ^~"
-
-"do-repeat.sps:14.13-14.14: error: DO REPEAT: Syntax error expecting substitution values.
- 14 | DO REPEAT y=**.
- | ^~"
-
-do-repeat.sps:16: error: DO REPEAT: Each dummy variable must have the same number of substitutions.
-
-"do-repeat.sps:16.11-16.17: note: DO REPEAT: Dummy variable y had 3 substitutions.
- 16 | DO REPEAT y=1 2 3/z=4.
- | ^~~~~~~"
-
-"do-repeat.sps:16.19-16.21: note: DO REPEAT: Dummy variable z had 1 substitution.
- 16 | DO REPEAT y=1 2 3/z=4.
- | ^~~"
-
-error: DO REPEAT: At end of input: Syntax error expecting END REPEAT.
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([LOOP])
-
-m4_define([LOOP_DATA], [dnl
-data list notable /x 1 y 2 z 3.
-begin data.
-121
-252
-393
-404
-end data.
-])
-
-AT_SETUP([LOOP with index])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-loop #i=x to y by z.
-print /#i.
-end loop.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1.00 @&t@
-2.00 @&t@
---------
-2.00 @&t@
-4.00 @&t@
---------
-3.00 @&t@
-6.00 @&t@
-9.00 @&t@
---------
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with IF condition])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-compute #j=x.
-loop if #j <= y.
-print /#j.
-compute #j = #j + z.
-end loop.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1.00 @&t@
-2.00 @&t@
---------
-2.00 @&t@
-4.00 @&t@
---------
-3.00 @&t@
-6.00 @&t@
-9.00 @&t@
---------
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with END IF condition])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-compute #k=x.
-loop.
-print /#k.
-compute #k = #k + z.
-end loop if #k > y.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1.00 @&t@
-2.00 @&t@
---------
-2.00 @&t@
-4.00 @&t@
---------
-3.00 @&t@
-6.00 @&t@
-9.00 @&t@
---------
-4.00 @&t@
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with index and IF based on index])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-loop #m=x to y by z if #m < 4.
-print /#m.
-end loop.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1.00 @&t@
-2.00 @&t@
---------
-2.00 @&t@
---------
-3.00 @&t@
---------
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with index and END IF based on index])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-loop #n=x to y by z.
-print /#n.
-end loop if #n >= 4.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1.00 @&t@
-2.00 @&t@
---------
-2.00 @&t@
-4.00 @&t@
---------
-3.00 @&t@
-6.00 @&t@
---------
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with index and IF and END IF based on index])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-loop #o=x to y by z if mod(#o,2) = 0.
-print /#o.
-end loop if #o >= 4.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
---------
-2.00 @&t@
-4.00 @&t@
---------
---------
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with no conditions containing BREAK])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-compute #p = x.
-loop.
-print /#p.
-compute #p = #p + z.
-do if #p >= y.
-break.
-end if.
-end loop.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1.00 @&t@
---------
-2.00 @&t@
-4.00 @&t@
---------
-3.00 @&t@
-6.00 @&t@
---------
-4.00 @&t@
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with no conditions that ends due to MXLOOPS])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-set mxloops=2.
-loop.
-compute #p = #p + 1.
-print /x #p.
-end loop.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1 1.00 @&t@
-1 2.00 @&t@
---------
-2 3.00 @&t@
-2 4.00 @&t@
---------
-3 5.00 @&t@
-3 6.00 @&t@
---------
-4 7.00 @&t@
-4 8.00 @&t@
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP with IF condition that ends due to MXLOOPS])
-AT_DATA([loop.sps], [dnl
-LOOP_DATA
-set mxloops=3.
-compute #p = x.
-loop.
-print /x #p.
-compute #p = #p + 1.
-end loop if #p >= 6.
-print/'--------'.
-execute.
-])
-AT_CHECK([pspp -o pspp.csv loop.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-1 1.00 @&t@
-1 2.00 @&t@
-1 3.00 @&t@
---------
-2 2.00 @&t@
-2 3.00 @&t@
-2 4.00 @&t@
---------
-3 3.00 @&t@
-3 4.00 @&t@
-3 5.00 @&t@
---------
-4 4.00 @&t@
-4 5.00 @&t@
---------
-])
-AT_CLEANUP
-
-AT_SETUP([LOOP negative])
-AT_DATA([loop.sps], [dnl
-DATA LIST NOTABLE /x 1.
-BREAK.
-END LOOP.
-
-LOOP A=1 TO 5 B=1 TO 5.
-END LOOP.
-LOOP 5.
-END LOOP.
-LOOP B !.
-END LOOP.
-LOOP B=!.
-END LOOP.
-LOOP A=1 TO !.
-END LOOP.
-LOOP A=1 BY !.
-END LOOP.
-LOOP A=1 TO 5 BY 5 TO !.
-END LOOP.
-LOOP A=1 BY 5 TO 10 BY !.
-END LOOP.
-LOOP A=1.
-END LOOP.
-LOOP !.
-END LOOP.
-
-LOOP IF 1 IF 0.
-END LOOP.
-
-LOOP IF !.
-END LOOP.
-
-LOOP.
-END LOOP IF !.
-
-LOOP.
-END LOOP !.
-
-LOOP.
-])
-AT_CHECK([pspp loop.sps], 1, [dnl
-loop.sps:2.1-2.5: error: BREAK: This command cannot appear outside LOOP...END
-LOOP.
- 2 | BREAK.
- | ^~~~~
-
-loop.sps:3.1-3.8: error: END LOOP: This command cannot appear outside LOOP...
-END LOOP.
- 3 | END LOOP.
- | ^~~~~~~~
-
-loop.sps:5.15: error: LOOP: Only one index clause may be specified.
- 5 | LOOP A=1 TO 5 B=1 TO 5.
- | ^
-
-loop.sps:7.6: error: LOOP: Syntax error expecting identifier.
- 7 | LOOP 5.
- | ^
-
-loop.sps:9.8: error: LOOP: Syntax error expecting `='.
- 9 | LOOP B !.
- | ^
-
-loop.sps:11.8: error: LOOP: Syntax error parsing expression.
- 11 | LOOP B=!.
- | ^
-
-loop.sps:13.13: error: LOOP: Syntax error parsing expression.
- 13 | LOOP A=1 TO !.
- | ^
-
-loop.sps:15.13: error: LOOP: Syntax error parsing expression.
- 15 | LOOP A=1 BY !.
- | ^
-
-loop.sps:17.20-17.21: error: LOOP: Subcommand TO may only be specified once.
- 17 | LOOP A=1 TO 5 BY 5 TO !.
- | ^~
-
-loop.sps:19.21-19.22: error: LOOP: Subcommand BY may only be specified once.
- 19 | LOOP A=1 BY 5 TO 10 BY !.
- | ^~
-
-loop.sps:21.1-21.9: error: LOOP: Required subcommand TO was not specified.
- 21 | LOOP A=1.
- | ^~~~~~~~~
-
-loop.sps:23.6: error: LOOP: Syntax error expecting identifier.
- 23 | LOOP !.
- | ^
-
-loop.sps:26.11-26.12: error: LOOP: Subcommand IF may only be specified once.
- 26 | LOOP IF 1 IF 0.
- | ^~
-
-loop.sps:29.9: error: LOOP: Syntax error parsing expression.
- 29 | LOOP IF !.
- | ^
-
-loop.sps:33.13: error: LOOP: Syntax error parsing expression.
- 33 | END LOOP IF !.
- | ^
-
-loop.sps:36.10: error: LOOP: Syntax error expecting end of command.
- 36 | END LOOP !.
- | ^
-
-error: LOOP: At end of input: Syntax error expecting END LOOP.
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([TEMPORARY])
-
-dnl Tests for a bug that manifested when all transformations are temporary.
-AT_SETUP([TEMPORARY as first transformation])
-AT_DATA([temporary.sps], [dnl
-DATA LIST LIST NOTABLE /X *.
-BEGIN DATA.
-1
-2
-3
-4
-5
-6
-7
-8
-9
-END DATA.
-
-TEMPORARY.
-SELECT IF x > 5 .
-LIST.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv temporary.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-X
-6.00
-7.00
-8.00
-9.00
-
-Table: Data List
-X
-1.00
-2.00
-3.00
-4.00
-5.00
-6.00
-7.00
-8.00
-9.00
-])
-AT_CLEANUP
+++ /dev/null
-<?xml version="1.0" encoding="UTF-8"?>
-<gnm:Workbook xmlns:gnm="http://www.gnumeric.org/v10.dtd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gnumeric.org/v8.xsd">
- <gnm:Version Epoch="1" Major="6" Minor="3" Full="1.6.3"/>
- <gnm:Attributes>
- <gnm:Attribute>
- <gnm:type>4</gnm:type>
- <gnm:name>WorkbookView::show_horizontal_scrollbar</gnm:name>
- <gnm:value>TRUE</gnm:value>
- </gnm:Attribute>
- <gnm:Attribute>
- <gnm:type>4</gnm:type>
- <gnm:name>WorkbookView::show_vertical_scrollbar</gnm:name>
- <gnm:value>TRUE</gnm:value>
- </gnm:Attribute>
- <gnm:Attribute>
- <gnm:type>4</gnm:type>
- <gnm:name>WorkbookView::show_notebook_tabs</gnm:name>
- <gnm:value>TRUE</gnm:value>
- </gnm:Attribute>
- <gnm:Attribute>
- <gnm:type>4</gnm:type>
- <gnm:name>WorkbookView::do_auto_completion</gnm:name>
- <gnm:value>TRUE</gnm:value>
- </gnm:Attribute>
- <gnm:Attribute>
- <gnm:type>4</gnm:type>
- <gnm:name>WorkbookView::is_protected</gnm:name>
- <gnm:value>FALSE</gnm:value>
- </gnm:Attribute>
- </gnm:Attributes>
- <gnm:Summary>
- <gnm:Item>
- <gnm:name>application</gnm:name>
- <gnm:val-string>gnumeric</gnm:val-string>
- </gnm:Item>
- <gnm:Item>
- <gnm:name>author</gnm:name>
- <gnm:val-string>John Darrington</gnm:val-string>
- </gnm:Item>
- </gnm:Summary>
- <gnm:SheetNameIndex>
- <gnm:SheetName>This</gnm:SheetName>
- <gnm:SheetName>vars</gnm:SheetName>
- <gnm:SheetName>That</gnm:SheetName>
- <gnm:SheetName>Empty</gnm:SheetName>
- <gnm:SheetName>Blank</gnm:SheetName>
- </gnm:SheetNameIndex>
- <gnm:Geometry Width="1278" Height="633"/>
- <gnm:Sheets>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
- <gnm:Name>This</gnm:Name>
- <gnm:MaxCol>9</gnm:MaxCol>
- <gnm:MaxRow>17</gnm:MaxRow>
- <gnm:Zoom>1</gnm:Zoom>
- <gnm:PrintInformation>
- <gnm:Margins>
- <gnm:top Points="120" PrefUnit="cm"/>
- <gnm:bottom Points="120" PrefUnit="cm"/>
- </gnm:Margins>
- <gnm:Scale type="percentage" percentage="100"/>
- <gnm:vcenter value="0"/>
- <gnm:hcenter value="0"/>
- <gnm:grid value="0"/>
- <gnm:even_if_only_styles value="0"/>
- <gnm:monochrome value="0"/>
- <gnm:draft value="0"/>
- <gnm:titles value="0"/>
- <gnm:order>d_then_r</gnm:order>
- <gnm:orientation>portrait</gnm:orientation>
- <gnm:Header Left="" Middle="&[TAB]" Right=""/>
- <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
- </gnm:PrintInformation>
- <gnm:Styles>
- <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- </gnm:Styles>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="10"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="15"/>
- <gnm:RowInfo No="15" Unit="9.75" MarginA="0" MarginB="0" HardSize="1"/>
- <gnm:RowInfo No="17" Unit="12.75" MarginA="0" MarginB="0"/>
- </gnm:Rows>
- <gnm:Selections CursorCol="7" CursorRow="7">
- <gnm:Selection startCol="7" startRow="7" endCol="7" endRow="7"/>
- </gnm:Selections>
- <gnm:Cells>
- <gnm:Cell Col="0" Row="0" ValueType="60">numeral</gnm:Cell>
- <gnm:Cell Col="1" Row="0" ValueType="60">eng_name</gnm:Cell>
- <gnm:Cell Col="2" Row="0" ValueType="60">xxx</gnm:Cell>
- <gnm:Cell Col="0" Row="1" ValueType="30">1</gnm:Cell>
- <gnm:Cell Col="1" Row="1" ValueType="60">One</gnm:Cell>
- <gnm:Cell Col="2" Row="1" ValueType="60">Eins</gnm:Cell>
- <gnm:Cell Col="0" Row="2" ValueType="30">2</gnm:Cell>
- <gnm:Cell Col="1" Row="2" ValueType="60">Two</gnm:Cell>
- <gnm:Cell Col="2" Row="2" ValueType="60">Zwei</gnm:Cell>
- <gnm:Cell Col="0" Row="3" ValueType="30">3</gnm:Cell>
- <gnm:Cell Col="1" Row="3" ValueType="60">Three</gnm:Cell>
- <gnm:Cell Col="2" Row="3" ValueType="60">Drei</gnm:Cell>
- <gnm:Cell Col="2" Row="4" ValueType="60">Vier</gnm:Cell>
- <gnm:Cell Col="5" Row="6" ValueType="60">XY</gnm:Cell>
- <gnm:Cell Col="6" Row="6" ValueType="60">xxx</gnm:Cell>
- <gnm:Cell Col="7" Row="6" ValueType="60">xxxx</gnm:Cell>
- <gnm:Cell Col="8" Row="6" ValueType="60">xxxx</gnm:Cell>
- <gnm:Cell Col="5" Row="7" ValueType="60">yyy</gnm:Cell>
- <gnm:Cell Col="6" Row="7" ValueType="60">V1</gnm:Cell>
- <gnm:Cell Col="7" Row="7" ValueType="60">V2</gnm:Cell>
- <gnm:Cell Col="5" Row="8" ValueType="60">yyy</gnm:Cell>
- <gnm:Cell Col="6" Row="8" ValueType="30">0</gnm:Cell>
- <gnm:Cell Col="7" Row="8" ValueType="60">fred</gnm:Cell>
- <gnm:Cell Col="8" Row="8" ValueType="30">20</gnm:Cell>
- <gnm:Cell Col="9" Row="8" ValueType="60">$$$$</gnm:Cell>
- <gnm:Cell Col="5" Row="9" ValueType="60">yyy</gnm:Cell>
- <gnm:Cell Col="6" Row="9" ValueType="30">1</gnm:Cell>
- <gnm:Cell Col="7" Row="9" ValueType="30">11</gnm:Cell>
- <gnm:Cell Col="8" Row="9" ValueType="30">21</gnm:Cell>
- <gnm:Cell Col="9" Row="9" ValueType="60">$$$$</gnm:Cell>
- <gnm:Cell Col="5" Row="10" ValueType="60">yyyy</gnm:Cell>
- <gnm:Cell Col="6" Row="10" ValueType="30">2</gnm:Cell>
- <gnm:Cell Col="7" Row="10" ValueType="60">twelve</gnm:Cell>
- <gnm:Cell Col="8" Row="10" ValueType="30">22</gnm:Cell>
- <gnm:Cell Col="9" Row="10" ValueType="60">$$$$</gnm:Cell>
- <gnm:Cell Col="5" Row="11" ValueType="60">yyyy</gnm:Cell>
- <gnm:Cell Col="6" Row="11" ValueType="30">3</gnm:Cell>
- <gnm:Cell Col="7" Row="11" ValueType="30">13</gnm:Cell>
- <gnm:Cell Col="8" Row="11" ValueType="30">23</gnm:Cell>
- <gnm:Cell Col="9" Row="11" ValueType="60">$$$$</gnm:Cell>
- <gnm:Cell Col="1" Row="12" ValueType="60">Eleven</gnm:Cell>
- <gnm:Cell Col="5" Row="12" ValueType="60">yyyy</gnm:Cell>
- <gnm:Cell Col="6" Row="12" ValueType="30">4</gnm:Cell>
- <gnm:Cell Col="7" Row="12" ValueType="30">14</gnm:Cell>
- <gnm:Cell Col="8" Row="12" ValueType="30">24</gnm:Cell>
- <gnm:Cell Col="9" Row="12" ValueType="60">$$$$</gnm:Cell>
- <gnm:Cell Col="5" Row="13" ValueType="60">zzz</gnm:Cell>
- <gnm:Cell Col="6" Row="13" ValueType="60">zzz</gnm:Cell>
- <gnm:Cell Col="7" Row="13" ValueType="60">zzz</gnm:Cell>
- <gnm:Cell Col="8" Row="13" ValueType="60">zzz</gnm:Cell>
- <gnm:Cell Col="9" Row="13" ValueType="60">zzz</gnm:Cell>
- <gnm:Cell Col="1" Row="17" ValueType="60">Seventeen</gnm:Cell>
- </gnm:Cells>
- <gnm:SheetLayout TopLeft="A1"/>
- <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
- </gnm:Sheet>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
- <gnm:Name>vars</gnm:Name>
- <gnm:MaxCol>4</gnm:MaxCol>
- <gnm:MaxRow>8</gnm:MaxRow>
- <gnm:Zoom>1</gnm:Zoom>
- <gnm:PrintInformation>
- <gnm:Margins>
- <gnm:top Points="120" PrefUnit="cm"/>
- <gnm:bottom Points="120" PrefUnit="cm"/>
- </gnm:Margins>
- <gnm:Scale type="percentage" percentage="100"/>
- <gnm:vcenter value="0"/>
- <gnm:hcenter value="0"/>
- <gnm:grid value="0"/>
- <gnm:even_if_only_styles value="0"/>
- <gnm:monochrome value="0"/>
- <gnm:draft value="0"/>
- <gnm:titles value="0"/>
- <gnm:order>d_then_r</gnm:order>
- <gnm:orientation>portrait</gnm:orientation>
- <gnm:Header Left="" Middle="&[TAB]" Right=""/>
- <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
- </gnm:PrintInformation>
- <gnm:Styles>
- <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- </gnm:Styles>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="5"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="5"/>
- </gnm:Rows>
- <gnm:Selections CursorCol="0" CursorRow="0">
- <gnm:Selection startCol="0" startRow="0" endCol="0" endRow="0"/>
- </gnm:Selections>
- <gnm:Cells>
- <gnm:Cell Col="0" Row="0" ValueType="60">1v12</gnm:Cell>
- <gnm:Cell Col="1" Row="0" ValueType="60">var&x@</gnm:Cell>
- <gnm:Cell Col="2" Row="0" ValueType="60">a(43)</gnm:Cell>
- <gnm:Cell Col="3" Row="0" ValueType="60">varx</gnm:Cell>
- <gnm:Cell Col="4" Row="0" ValueType="60">varx</gnm:Cell>
- <gnm:Cell Col="0" Row="1" ValueType="30">1</gnm:Cell>
- <gnm:Cell Col="1" Row="1" ValueType="30">2</gnm:Cell>
- <gnm:Cell Col="2" Row="1" ValueType="30">23</gnm:Cell>
- <gnm:Cell Col="3" Row="1" ValueType="30">2</gnm:Cell>
- <gnm:Cell Col="4" Row="1" ValueType="30">4</gnm:Cell>
- <gnm:Cell Col="0" Row="2" ValueType="30">3</gnm:Cell>
- <gnm:Cell Col="1" Row="2" ValueType="30">4</gnm:Cell>
- <gnm:Cell Col="2" Row="2" ValueType="30">23</gnm:Cell>
- <gnm:Cell Col="3" Row="2" ValueType="30">3</gnm:Cell>
- <gnm:Cell Col="4" Row="2" ValueType="30">4</gnm:Cell>
- <gnm:Cell Col="1" Row="8" ValueType="10"></gnm:Cell>
- </gnm:Cells>
- <gnm:SheetLayout TopLeft="A1"/>
- <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
- </gnm:Sheet>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
- <gnm:Name>That</gnm:Name>
- <gnm:MaxCol>3</gnm:MaxCol>
- <gnm:MaxRow>4</gnm:MaxRow>
- <gnm:Zoom>1</gnm:Zoom>
- <gnm:PrintInformation>
- <gnm:Margins>
- <gnm:top Points="120" PrefUnit="cm"/>
- <gnm:bottom Points="120" PrefUnit="cm"/>
- </gnm:Margins>
- <gnm:Scale type="percentage" percentage="100"/>
- <gnm:vcenter value="0"/>
- <gnm:hcenter value="0"/>
- <gnm:grid value="0"/>
- <gnm:even_if_only_styles value="0"/>
- <gnm:monochrome value="0"/>
- <gnm:draft value="0"/>
- <gnm:titles value="0"/>
- <gnm:order>d_then_r</gnm:order>
- <gnm:orientation>portrait</gnm:orientation>
- <gnm:Header Left="" Middle="&[TAB]" Right=""/>
- <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
- </gnm:PrintInformation>
- <gnm:Styles>
- <gnm:StyleRegion startCol="0" startRow="4096" endCol="63" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="4" startRow="0" endCol="15" endRow="255">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="0" startRow="16" endCol="3" endRow="255">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="16" startRow="0" endCol="63" endRow="4095">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="64" startRow="0" endCol="255" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="3" startRow="0" endCol="3" endRow="15">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="2" startRow="5" endCol="2" endRow="15">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="2" startRow="1" endCol="2" endRow="4">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="0.00">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="0" startRow="256" endCol="15" endRow="4095">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="0" startRow="0" endCol="1" endRow="15">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- <gnm:StyleRegion startCol="2" startRow="0" endCol="2" endRow="0">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- </gnm:Styles>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="2"/>
- <gnm:ColInfo No="2" Unit="90.75" MarginA="2" MarginB="2" HardSize="1"/>
- <gnm:ColInfo No="3" Unit="81" MarginA="2" MarginB="2" HardSize="1"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="5"/>
- </gnm:Rows>
- <gnm:Selections CursorCol="0" CursorRow="5">
- <gnm:Selection startCol="0" startRow="5" endCol="0" endRow="5"/>
- </gnm:Selections>
- <gnm:Cells>
- <gnm:Cell Col="0" Row="0" ValueType="60">name</gnm:Cell>
- <gnm:Cell Col="1" Row="0" ValueType="60">id</gnm:Cell>
- <gnm:Cell Col="2" Row="0" ValueType="60">height</gnm:Cell>
- <gnm:Cell Col="0" Row="1" ValueType="60">fred</gnm:Cell>
- <gnm:Cell Col="1" Row="1" ValueType="30">0</gnm:Cell>
- <gnm:Cell Col="2" Row="1" ValueType="40">23.4</gnm:Cell>
- <gnm:Cell Col="0" Row="2" ValueType="60">bert </gnm:Cell>
- <gnm:Cell Col="1" Row="2" ValueType="30">1</gnm:Cell>
- <gnm:Cell Col="2" Row="2" ValueType="40">0.56</gnm:Cell>
- <gnm:Cell Col="0" Row="3" ValueType="60">charlie</gnm:Cell>
- <gnm:Cell Col="1" Row="3" ValueType="30">2</gnm:Cell>
- <gnm:Cell Col="2" Row="3" ValueType="60">n/a</gnm:Cell>
- <gnm:Cell Col="0" Row="4" ValueType="60">dick</gnm:Cell>
- <gnm:Cell Col="1" Row="4" ValueType="30">3</gnm:Cell>
- <gnm:Cell Col="2" Row="4" ValueType="40" ValueFormat="0.00">-34.09</gnm:Cell>
- </gnm:Cells>
- <gnm:SheetLayout TopLeft="A1"/>
- <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
- </gnm:Sheet>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
- <gnm:Name>Empty</gnm:Name>
- <gnm:MaxCol>-1</gnm:MaxCol>
- <gnm:MaxRow>-1</gnm:MaxRow>
- <gnm:Zoom>1</gnm:Zoom>
- <gnm:PrintInformation>
- <gnm:Margins>
- <gnm:top Points="120" PrefUnit="cm"/>
- <gnm:bottom Points="120" PrefUnit="cm"/>
- </gnm:Margins>
- <gnm:Scale type="percentage" percentage="100"/>
- <gnm:vcenter value="0"/>
- <gnm:hcenter value="0"/>
- <gnm:grid value="0"/>
- <gnm:even_if_only_styles value="0"/>
- <gnm:monochrome value="0"/>
- <gnm:draft value="0"/>
- <gnm:titles value="0"/>
- <gnm:order>d_then_r</gnm:order>
- <gnm:orientation>portrait</gnm:orientation>
- <gnm:Header Left="" Middle="&[TAB]" Right=""/>
- <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
- </gnm:PrintInformation>
- <gnm:Styles>
- <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- </gnm:Styles>
- <gnm:Cols DefaultSizePts="48"/>
- <gnm:Rows DefaultSizePts="12.75"/>
- <gnm:Selections CursorCol="0" CursorRow="0">
- <gnm:Selection startCol="0" startRow="0" endCol="0" endRow="0"/>
- </gnm:Selections>
- <gnm:Cells/>
- <gnm:SheetLayout TopLeft="A1"/>
- <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
- </gnm:Sheet>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE">
- <gnm:Name>Blank</gnm:Name>
- <gnm:MaxCol>3</gnm:MaxCol>
- <gnm:MaxRow>2</gnm:MaxRow>
- <gnm:Zoom>1</gnm:Zoom>
- <gnm:PrintInformation>
- <gnm:Margins>
- <gnm:top Points="120" PrefUnit="cm"/>
- <gnm:bottom Points="120" PrefUnit="cm"/>
- </gnm:Margins>
- <gnm:Scale type="percentage" percentage="100"/>
- <gnm:vcenter value="0"/>
- <gnm:hcenter value="0"/>
- <gnm:grid value="0"/>
- <gnm:even_if_only_styles value="0"/>
- <gnm:monochrome value="0"/>
- <gnm:draft value="0"/>
- <gnm:titles value="0"/>
- <gnm:order>d_then_r</gnm:order>
- <gnm:orientation>portrait</gnm:orientation>
- <gnm:Header Left="" Middle="&[TAB]" Right=""/>
- <gnm:Footer Left="" Middle="Page &[PAGE]" Right=""/>
- </gnm:PrintInformation>
- <gnm:Styles>
- <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- <gnm:Font Unit="10" Bold="0" Italic="0" Underline="0" StrikeThrough="0" Script="0">Sans</gnm:Font>
- <gnm:StyleBorder>
- <gnm:Top Style="0"/>
- <gnm:Bottom Style="0"/>
- <gnm:Left Style="0"/>
- <gnm:Right Style="0"/>
- <gnm:Diagonal Style="0"/>
- <gnm:Rev-Diagonal Style="0"/>
- </gnm:StyleBorder>
- </gnm:Style>
- </gnm:StyleRegion>
- </gnm:Styles>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="48" MarginA="2" MarginB="2" Count="4"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="12.75" MarginA="0" MarginB="0" Count="3"/>
- </gnm:Rows>
- <gnm:Selections CursorCol="3" CursorRow="1">
- <gnm:Selection startCol="3" startRow="1" endCol="3" endRow="1"/>
- </gnm:Selections>
- <gnm:Cells>
- <gnm:Cell Col="0" Row="0" ValueType="60">vone</gnm:Cell>
- <gnm:Cell Col="1" Row="0" ValueType="60">vtwo</gnm:Cell>
- <gnm:Cell Col="2" Row="0" ValueType="60">vthree</gnm:Cell>
- <gnm:Cell Col="3" Row="0" ValueType="60">v4</gnm:Cell>
- <gnm:Cell Col="0" Row="1" ValueType="30">1</gnm:Cell>
- <gnm:Cell Col="1" Row="1" ValueType="30">3</gnm:Cell>
- <gnm:Cell Col="3" Row="1" ValueType="30">5</gnm:Cell>
- <gnm:Cell Col="0" Row="2" ValueType="30">2</gnm:Cell>
- <gnm:Cell Col="1" Row="2" ValueType="30">4</gnm:Cell>
- <gnm:Cell Col="3" Row="2" ValueType="30">6</gnm:Cell>
- </gnm:Cells>
- <gnm:SheetLayout TopLeft="A1"/>
- <gnm:Solver ProblemType="1" Inputs="" MaxTime="0" MaxIter="0" NonNeg="1" Discr="0" AutoScale="0" ShowIter="0" AnswerR="0" SensitivityR="0" LimitsR="0" PerformR="0" ProgramR="0"/>
- </gnm:Sheet>
- </gnm:Sheets>
- <gnm:UIData SelectedTab="4"/>
- <gnm:Calculation ManualRecalc="0" EnableIteration="1" MaxIterations="100" IterationTolerance="0.001"/>
-</gnm:Workbook>
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-m4_define([CHECK_ADD_FILES],
- [AT_SETUP([ADD FILES -- $1 $2 with $3])
- AT_DATA([a.data], [dnl
-1aB
-8aM
-3aE
-5aG
-0aA
-5aH
-6aI
-7aJ
-2aD
-7aK
-1aC
-7aL
-4aF
-])
- AT_DATA([b.data], [dnl
-1bN
-3bO
-4bP
-6bQ
-7bR
-9bS
-])
- m4_if([$2], [sav],
- [AT_DATA([save-a.sps], [dnl
-DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).
-SAVE OUTFILE='a.sav'.
-])
- AT_CHECK([pspp -O format=csv save-a.sps])])
- m4_if([$3], [sav],
- [AT_DATA([save-b.sps], [dnl
-DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).
-SAVE OUTFILE='b.sav'.
-])
- AT_CHECK([pspp -O format=csv save-b.sps])])
- m4_if([$1], [interleave],
- [m4_pushdef([BY], [[/BY a /FIRST=first /LAST=last]])
- m4_pushdef([SORT], [[/SORT]])],
- [m4_pushdef([BY], [])
- m4_pushdef([SORT], [])])
- AT_DATA([add-files.sps], [dnl
-m4_if([$2], [sav], [], [DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).])
-m4_if([$3], [sav], [], [DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).])
-ADD FILES
- m4_if([$2], [sav], [FILE='a.sav'], [FILE=*]) /IN=InA SORT
- m4_if([$3], [sav], [FILE='b.sav'], [FILE=*]) /IN=InB /RENAME c=d
- BY[].
-LIST.
-])
- m4_popdef([BY])
- m4_popdef([SORT])
- AT_CHECK([pspp -O format=csv add-files.sps], [0],
-[m4_if([$1], [interleave], [dnl
-Table: Data List
-a,b,c,d,InA,InB,first,last
-0,a,A,,1,0,1,1
-1,a,B,,1,0,1,0
-1,a,C,,1,0,0,0
-1,b,,N,0,1,0,1
-2,a,D,,1,0,1,1
-3,a,E,,1,0,1,0
-3,b,,O,0,1,0,1
-4,a,F,,1,0,1,0
-4,b,,P,0,1,0,1
-5,a,G,,1,0,1,0
-5,a,H,,1,0,0,1
-6,a,I,,1,0,1,0
-6,b,,Q,0,1,0,1
-7,a,J,,1,0,1,0
-7,a,K,,1,0,0,0
-7,a,L,,1,0,0,0
-7,b,,R,0,1,0,1
-8,a,M,,1,0,1,1
-9,b,,S,0,1,1,1
-], [dnl
-Table: Data List
-a,b,c,d,InA,InB
-1,a,B,,1,0
-8,a,M,,1,0
-3,a,E,,1,0
-5,a,G,,1,0
-0,a,A,,1,0
-5,a,H,,1,0
-6,a,I,,1,0
-7,a,J,,1,0
-2,a,D,,1,0
-7,a,K,,1,0
-1,a,C,,1,0
-7,a,L,,1,0
-4,a,F,,1,0
-1,b,,N,0,1
-3,b,,O,0,1
-4,b,,P,0,1
-6,b,,Q,0,1
-7,b,,R,0,1
-9,b,,S,0,1
-])])
-AT_CLEANUP
-])
-
-AT_BANNER([ADD FILES])
-
-CHECK_ADD_FILES([interleave], [sav], [sav])
-CHECK_ADD_FILES([interleave], [sav], [inline])
-CHECK_ADD_FILES([interleave], [inline], [sav])
-CHECK_ADD_FILES([concatenate], [sav], [sav])
-CHECK_ADD_FILES([concatenate], [sav], [inline])
-CHECK_ADD_FILES([concatenate], [inline], [sav])
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DATA LIST])
-
-AT_SETUP([DATA LIST LIST with empty fields])
-AT_DATA([data-list.pspp], [dnl
-DATA LIST LIST NOTABLE /A B C (F1.0).
-BEGIN DATA.
-,,
-,,3
-,2,
-,2,3
-1,,
-1,,3
-1,2,
-1,2,3
-END DATA.
-
-LIST.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-Table: Data List
-A,B,C
-.,.,.
-.,.,3
-.,2,.
-.,2,3
-1,.,.
-1,.,3
-1,2,.
-1,2,3
-])
-AT_CLEANUP
-
-
-AT_SETUP([DATA LIST LIST with explicit delimiters])
-AT_DATA([data-list.pspp], [dnl
-data list list ('|','X') /A B C D.
-begin data.
-1|23X45|2.03x
-2X22|34|23|
-3|34|34X34
-end data.
-
-list.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-A,F8.0
-B,F8.0
-C,F8.0
-D,F8.0
-
-data-list.pspp:3.9-3.13: warning: Data for variable D is not valid as format F: Number followed by garbage.
-
-Table: Data List
-A,B,C,D
-1.00,23.00,45.00,. @&t@
-2.00,22.00,34.00,23.00
-3.00,34.00,34.00,34.00
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST FREE with SKIP])
-AT_DATA([data-list.pspp], [dnl
-data list free skip=1/A B C D.
-begin data.
-# This record is ignored.
-,1,2,x
-,4,,5
-6
-7,
-8 9
-0,1 ,,,
-,,,,
-2
-
-3
-4
-5
-end data.
-list.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-data-list.pspp:4.6: warning: Data for variable D is not valid as format F: Field contents are not numeric.
-
-Table: Data List
-A,B,C,D
-. ,1.00,2.00,. @&t@
-. ,4.00,. ,5.00
-6.00,7.00,8.00,9.00
-.00,1.00,. ,. @&t@
-. ,. ,. ,. @&t@
-2.00,3.00,4.00,5.00
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST LIST with SKIP and tab delimiter])
-AT_DATA([data-list.pspp], [dnl
-data list list (tab) notable skip=2/A B C D.
-begin data.
-# These records
-# are skipped.
-1 2 3 4
-1 2 3 @&t@
-1 2 4
-1 2 @&t@
-1 3 4
-1 3 @&t@
-1 4
-1 @&t@
- 2 3 4
- 2 3 @&t@
- 2 4
- 2 @&t@
- 3 4
- 3 @&t@
- 4
- @&t@
-end data.
-list.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-Table: Data List
-A,B,C,D
-1.00,2.00,3.00,4.00
-1.00,2.00,3.00,. @&t@
-1.00,2.00,. ,4.00
-1.00,2.00,. ,. @&t@
-1.00,. ,3.00,4.00
-1.00,. ,3.00,. @&t@
-1.00,. ,. ,4.00
-1.00,. ,. ,. @&t@
-. ,2.00,3.00,4.00
-. ,2.00,3.00,. @&t@
-. ,2.00,. ,4.00
-. ,2.00,. ,. @&t@
-. ,. ,3.00,4.00
-. ,. ,3.00,. @&t@
-. ,. ,. ,4.00
-. ,. ,. ,. @&t@
-])
-AT_CLEANUP
-
-dnl Results of this test were confirmed with SPSS 21:
-dnl http://lists.gnu.org/archive/html/pspp-dev/2013-09/msg00003.html
-AT_SETUP([DATA LIST FREE with explicit delimiter at end of line])
-AT_DATA([data-list.pspp], [dnl
-DATA LIST FREE(',')/x y z.
-BEGIN DATA.
-1,2,3
-4,5,6
-7,8,9
-END DATA.
-LIST.
-
-DATA LIST FREE(',')/x y z.
-BEGIN DATA.
-11,12,13,
-14,15,16,
-17,18,19,
-END DATA.
-LIST.
-
-DATA LIST FREE(TAB)/x y z.
-BEGIN DATA.
-21 22 23
-24 25 26
-27 28 29
-END DATA.
-LIST.
-
-DATA LIST FREE(TAB)/x y z.
-BEGIN DATA.
-31 32 33 @&t@
-34 35 36 @&t@
-37 38 39 @&t@
-END DATA.
-LIST.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-Table: Data List
-x,y,z
-1.00,2.00,3.00
-4.00,5.00,6.00
-7.00,8.00,9.00
-
-Table: Data List
-x,y,z
-11.00,12.00,13.00
-14.00,15.00,16.00
-17.00,18.00,19.00
-
-Table: Data List
-x,y,z
-21.00,22.00,23.00
-24.00,25.00,26.00
-27.00,28.00,29.00
-
-Table: Data List
-x,y,z
-31.00,32.00,33.00
-34.00,35.00,36.00
-37.00,38.00,39.00
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST FIXED with multiple records per case])
-AT_DATA([data-list.pspp], [dnl
-data list fixed notable
- /1 start 1-20 (adate)
- /2 end 1-20 (adate)
- /3 count 1-3.
-begin data.
-07-22-2007
-10-06-2007
-x
-07-14-1789
-08-26-1789
-xy
-01-01-1972
-12-31-1999
-682
-end data.
-list.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-data-list.pspp:8.1-8.3: warning: Data for variable count is not valid as format F: Field contents are not numeric.
-
-data-list.pspp:11.1-11.3: warning: Data for variable count is not valid as format F: Field contents are not numeric.
-
-Table: Data List
-start,end,count
-07/22/2007,10/06/2007,.
-07/14/1789,08/26/1789,.
-01/01/1972,12/31/1999,682
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST FIXED with empty trailing record])
-AT_DATA([data-list.pspp], [dnl
-data list fixed notable records=2/x 1 y 2.
-begin data.
-12
-
-34
-
-56
-
-78
-
-90
-
-end data.
-list.
-])
-AT_CHECK([pspp -O format=csv data-list.pspp], [0], [dnl
-Table: Data List
-x,y
-1,2
-3,4
-5,6
-7,8
-9,0
-])
-AT_CLEANUP
-
-dnl Test that PSPP accepts LF and CR LF as line ends, but
-dnl treats isolated CR as linear whitespace.
-AT_SETUP([DATA LIST with various line-ends])
-AT_DATA([data-list.sps], [dnl
-data list list notable file='input.txt'/a b c.
-list.
-])
-printf '1 2 3\n4 5 6\r\n7\r8\r9\r\n10 11 12\n13 14 15 \r\n16\r\r17\r18\n' > input.txt
-dnl Make sure that input.txt actually received the data that we expect.
-dnl It might not have, if we're running on a system that translates \n
-dnl into some other sequence.
-AT_CHECK([cksum input.txt], [0], [1732021750 50 input.txt
-])
-AT_CHECK([pspp -o pspp.csv data-list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-a,b,c
-1.00,2.00,3.00
-4.00,5.00,6.00
-7.00,8.00,9.00
-10.00,11.00,12.00
-13.00,14.00,15.00
-16.00,17.00,18.00
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST properly expands tabs in input])
-AT_DATA([data-list.sps], [dnl
-data list notable /X 1-50 (a).
-begin data.
- 1 12 123 1234 12345 .
-end data.
-print /x.
-print outfile='print.txt' /x.
-write outfile='write.txt' /x.
-execute.
-])
-AT_CHECK([sed -n '/12345/l' data-list.sps], [0], [dnl
-\t1\t12\t123\t1234\t12345 .$
-])
-AT_CHECK([pspp -o pspp.csv data-list.sps])
-dnl The CSV driver drops leading spaces so they don't appear here:
-AT_CHECK([cat pspp.csv], [0], [dnl
-1 12 123 1234 12345 . @&t@
-])
-dnl But they do appear in print.txt. The PRINT command also puts a space
-dnl at the beginning of the line and after the variable:
-AT_CHECK([cat print.txt], [0], [dnl
- 1 12 123 1234 12345 . @&t@
-])
-dnl WRITE doesn't add spaces at the beginning or end of lines:
-AT_CHECK([cat write.txt], [0], [dnl
- 1 12 123 1234 12345 .
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST FREE and LIST report missing delimiters])
-AT_DATA([data-list.sps], [dnl
-DATA LIST FREE NOTABLE/s (a10).
-LIST.
-BEGIN DATA.
-'y'z
-END DATA.
-])
-AT_CHECK([pspp -O format=csv data-list.sps], [0], [dnl
-data-list.sps:4: warning: Missing delimiter following quoted string.
-
-Table: Data List
-s
-y
-z
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST FREE and LIST assume a width if omitted])
-AT_DATA([data-list.sps], [dnl
-DATA LIST FREE TABLE/s (a) d (datetime) f (f).
-])
-AT_CHECK([pspp -O format=csv data-list.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-s,A1
-d,DATETIME17.0
-f,F1.0
-])
-AT_CLEANUP
-
-AT_SETUP([DATA LIST Decimal comma])
-AT_DATA([data-list.sps], [dnl
-SET DECIMAL=COMMA.
-
-DATA LIST NOTABLE LIST /A *.
-BEGIN DATA
-1
-2
-3
-3,5
-4
-4,5
-5
-6
-END DATA
-
-LIST /FORMAT=NUMBERED.
-])
-
-AT_CHECK([pspp -O format=csv data-list.sps], [0], [dnl
-Table: Data List
-Case Number,A
-1,"1,00"
-2,"2,00"
-3,"3,00"
-4,"3,50"
-5,"4,00"
-6,"4,50"
-7,"5,00"
-8,"6,00"
-])
-
-AT_CLEANUP
-
-AT_SETUP([DATA LIST syntax errors])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='data-list.sps' ERROR=IGNORE.
-])
-AT_DATA([data-list.sps], [dnl
-DATA LIST FILE=**.
-DATA LIST ENCODING=**.
-DATA LIST RECORDS=1 RECORDS=2.
-DATA LIST RECORDS=0.
-DATA LIST SKIP=-1.
-DATA LIST END=**.
-INPUT PROGRAM.
-DATA LIST END=xyzzy END=xyzzy.
-END INPUT PROGRAM.
-INPUT PROGRAM.
-DATA LIST END=**.
-END INPUT PROGRAM.
-DATA LIST XYZZY.
-DATA LIST FREE LIST.
-DATA LIST LIST (**).
-DATA LIST **.
-DATA LIST ENCODING='xyzzy'/x.
-INPUT PROGRAM.
-DATA LIST LIST END=xyzzy/x.
-END INPUT PROGRAM.
-DATA LIST FIXED/0.
-DATA LIST FIXED/ **.
-DATA LIST FIXED/x 1.5.
-DATA LIST FIXED/x -1.
-DATA LIST FIXED/x 5-3.
-DATA LIST FIXED/x y 1-3.
-DATA LIST FIXED/x 1-5 (xyzzy).
-DATA LIST FIXED/x 1-5 (**).
-DATA LIST FIXED/x 1 (F,5).
-DATA LIST FIXED/x (2F8.0).
-DATA LIST FIXED/x **.
-DATA LIST FIXED/x 1 x 2.
-INPUT PROGRAM.
-DATA LIST FIXED/x 1.
-DATA LIST FIXED/x 1 (a).
-END INPUT PROGRAM.
-INPUT PROGRAM.
-DATA LIST FIXED/y 2 (a).
-DATA LIST FIXED/y 3-4 (a).
-END INPUT PROGRAM.
-DATA LIST FIXED RECORDS=1/2 x 1-2.
-])
-
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"data-list.sps:1.16-1.17: error: DATA LIST: Syntax error expecting a file name or handle name.
- 1 | DATA LIST FILE=**.
- | ^~"
-
-"data-list.sps:2.20-2.21: error: DATA LIST: Syntax error expecting string.
- 2 | DATA LIST ENCODING=**.
- | ^~"
-
-"data-list.sps:3.21-3.27: error: DATA LIST: Subcommand RECORDS may only be specified once.
- 3 | DATA LIST RECORDS=1 RECORDS=2.
- | ^~~~~~~"
-
-"data-list.sps:4.20: error: DATA LIST: Syntax error expecting one of the following: FILE, ENCODING, RECORDS, SKIP, END, NOTABLE, TABLE, FIXED, FREE, LIST.
- 4 | DATA LIST RECORDS=0.
- | ^"
-
-"data-list.sps:5.16-5.17: error: DATA LIST: Syntax error expecting non-negative integer for SKIP.
- 5 | DATA LIST SKIP=-1.
- | ^~"
-
-"data-list.sps:6.11-6.13: error: DATA LIST: The END subcommand may only be used within INPUT PROGRAM.
- 6 | DATA LIST END=**.
- | ^~~"
-
-"data-list.sps:8.21-8.23: error: DATA LIST: Subcommand END may only be specified once.
- 8 | DATA LIST END=xyzzy END=xyzzy.
- | ^~~"
-
-"data-list.sps:11.15-11.16: error: DATA LIST: Syntax error expecting identifier.
- 11 | DATA LIST END=**.
- | ^~"
-
-"data-list.sps:13.11-13.15: error: DATA LIST: Syntax error expecting one of the following: FILE, ENCODING, RECORDS, SKIP, END, NOTABLE, TABLE, FIXED, FREE, LIST.
- 13 | DATA LIST XYZZY.
- | ^~~~~"
-
-"data-list.sps:14.16-14.19: error: DATA LIST: Only one of FIXED, FREE, or LIST may be specified.
- 14 | DATA LIST FREE LIST.
- | ^~~~"
-
-"data-list.sps:15.17-15.18: error: DATA LIST: Syntax error expecting TAB or delimiter string.
- 15 | DATA LIST LIST (**).
- | ^~"
-
-"data-list.sps:16.11-16.12: error: DATA LIST: Syntax error expecting one of the following: FILE, ENCODING, RECORDS, SKIP, END, NOTABLE, TABLE, FIXED, FREE, LIST.
- 16 | DATA LIST **.
- | ^~"
-
-"data-list.sps:17.11-17.26: warning: DATA LIST: Encoding should not be specified for inline data. It will be ignored.
- 17 | DATA LIST ENCODING='xyzzy'/x.
- | ^~~~~~~~~~~~~~~~"
-
-"data-list.sps:17.29: error: DATA LIST: SPSS-like or Fortran-like format specification expected after variable names.
- 17 | DATA LIST ENCODING='xyzzy'/x.
- | ^"
-
-"data-list.sps:19.16-19.24: error: DATA LIST: The END subcommand may be used only with DATA LIST FIXED.
- 19 | DATA LIST LIST END=xyzzy/x.
- | ^~~~~~~~~"
-
-"data-list.sps:21.17: error: DATA LIST: Syntax error expecting positive integer.
- 21 | DATA LIST FIXED/0.
- | ^"
-
-"data-list.sps:22.18-22.19: error: DATA LIST: Syntax error expecting variable name.
- 22 | DATA LIST FIXED/ **.
- | ^~"
-
-"data-list.sps:23.19-23.21: error: DATA LIST: Syntax error expecting integer.
- 23 | DATA LIST FIXED/x 1.5.
- | ^~~"
-
-"data-list.sps:24.19-24.20: error: DATA LIST: Column positions for fields must be positive.
- 24 | DATA LIST FIXED/x -1.
- | ^~"
-
-"data-list.sps:25.19-25.21: error: DATA LIST: The ending column for a field must be greater than the starting column.
- 25 | DATA LIST FIXED/x 5-3.
- | ^~~"
-
-"data-list.sps:26.21-26.23: error: DATA LIST: The 3 columns 1-3 can't be evenly divided into 2 fields.
- 26 | DATA LIST FIXED/x y 1-3.
- | ^~~"
-
-"data-list.sps:27.24-27.28: error: DATA LIST: Unknown format type `xyzzy'.
- 27 | DATA LIST FIXED/x 1-5 (xyzzy).
- | ^~~~~"
-
-"data-list.sps:28.24-28.25: error: DATA LIST: Syntax error expecting `)'.
- 28 | DATA LIST FIXED/x 1-5 (**).
- | ^~"
-
-"data-list.sps:29.19-29.25: error: DATA LIST: Input format F1.5 specifies 5 decimal places, but width 1 allows at most 1 decimals.
- 29 | DATA LIST FIXED/x 1 (F,5).
- | ^~~~~~~"
-
-"data-list.sps:30.20-30.25: error: DATA LIST: Number of variables specified (1) differs from number of variable formats (2).
- 30 | DATA LIST FIXED/x (2F8.0).
- | ^~~~~~"
-
-"data-list.sps:31.19-31.20: error: DATA LIST: SPSS-like or Fortran-like format specification expected after variable names.
- 31 | DATA LIST FIXED/x **.
- | ^~"
-
-"data-list.sps:32.21: error: DATA LIST: x is a duplicate variable name.
- 32 | DATA LIST FIXED/x 1 x 2.
- | ^"
-
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-x,1,1-1,F1.0
-
-"data-list.sps:35.17-35.23: error: DATA LIST: There is already a variable x of a different type.
- 35 | DATA LIST FIXED/x 1 (a).
- | ^~~~~~~"
-
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-y,1,2-2,A1
-
-"data-list.sps:39.17-39.25: error: DATA LIST: There is already a string variable y of a different width.
- 39 | DATA LIST FIXED/y 3-4 (a).
- | ^~~~~~~~~"
-
-"data-list.sps:41.26-41.29: error: DATA LIST: Cannot place variable x on record 2 when RECORDS=1 is specified.
- 41 | DATA LIST FIXED RECORDS=1/2 x 1-2.
- | ^~~~"
-])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([BEGIN DATA])
-
-# BEGIN DATA can run as a command in itself, or it can appear as part
-# of the first procedure. First, test it after a procedure.
-AT_SETUP([BEGIN DATA as part of a procedure])
-AT_DATA([begin-data.sps], [dnl
-TITLE 'Test BEGIN DATA ... END DATA'.
-
-DATA LIST /a b 1-2.
-LIST.
-BEGIN DATA.
-12
-34
-56
-78
-90
-END DATA.
-])
-AT_CHECK([pspp -O format=csv begin-data.sps], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-a,1,1-1,F1.0
-b,1,2-2,F1.0
-
-Table: Data List
-a,b
-1,2
-3,4
-5,6
-7,8
-9,0
-])
-AT_CLEANUP
-
-# Also test BEGIN DATA as an independent command.
-AT_SETUP([BEGIN DATA as an independent command])
-AT_DATA([begin-data.sps], [dnl
-data list /A B 1-2.
-begin data.
-09
-87
-65
-43
-21
-end data.
-list.
-])
-AT_CHECK([pspp -O format=csv begin-data.sps], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-A,1,1-1,F1.0
-B,1,2-2,F1.0
-
-Table: Data List
-A,B
-0,9
-8,7
-6,5
-4,3
-2,1
-])
-AT_CLEANUP
-
-m4_define([DATA_READER_BINARY],
- [AT_SETUP([read and write files with $1])
-$3
- AT_DATA([input.txt], [dnl
-07-22-2007
-10-06-2007
-321
-07-14-1789
-08-26-1789
-4
-01-01-1972
-12-31-1999
-682
-])
- AT_DATA([make-binary.py], [[
-#! /usr/bin/python3
-
-import struct
-import sys
-
-# This random number generator and the test for it below are drawn
-# from Park and Miller, "Random Number Generators: Good Ones are Hard
-# to Come By", Communications of the ACM 31:10 (October 1988). It is
-# documented to function properly on systems with a 46-bit or longer
-# real significand, which includes systems that have 64-bit IEEE reals
-# (with 53-bit significand). The test should catch any systems for
-# which this is not true, in any case.
-def my_rand(modulus):
- global seed
- a = 16807
- m = 2147483647
- tmp = a * seed
- seed = tmp - m * (tmp // m)
- return seed % modulus
-
-# Test the random number generator for reproducibility,
-# then reset the seed
-seed = 1
-for i in range(10000):
- my_rand(1)
-assert seed == 1043618065
-seed = 1
-
-# ASCII to EBCDIC translation table
-ascii2ebcdic = (
- 0x00, 0x01, 0x02, 0x03, 0x37, 0x2d, 0x2e, 0x2f,
- 0x16, 0x05, 0x25, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
- 0x10, 0x11, 0x12, 0x13, 0x3c, 0x3d, 0x32, 0x26,
- 0x18, 0x19, 0x3f, 0x27, 0x1c, 0x1d, 0x1e, 0x1f,
- 0x40, 0x5a, 0x7f, 0x7b, 0x5b, 0x6c, 0x50, 0x7d,
- 0x4d, 0x5d, 0x5c, 0x4e, 0x6b, 0x60, 0x4b, 0x61,
- 0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7,
- 0xf8, 0xf9, 0x7a, 0x5e, 0x4c, 0x7e, 0x6e, 0x6f,
- 0x7c, 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7,
- 0xc8, 0xc9, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6,
- 0xd7, 0xd8, 0xd9, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6,
- 0xe7, 0xe8, 0xe9, 0xad, 0xe0, 0xbd, 0x9a, 0x6d,
- 0x79, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,
- 0x88, 0x89, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96,
- 0x97, 0x98, 0x99, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6,
- 0xa7, 0xa8, 0xa9, 0xc0, 0x4f, 0xd0, 0x5f, 0x07,
- 0x20, 0x21, 0x22, 0x23, 0x24, 0x15, 0x06, 0x17,
- 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x09, 0x0a, 0x1b,
- 0x30, 0x31, 0x1a, 0x33, 0x34, 0x35, 0x36, 0x08,
- 0x38, 0x39, 0x3a, 0x3b, 0x04, 0x14, 0x3e, 0xe1,
- 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48,
- 0x49, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57,
- 0x58, 0x59, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67,
- 0x68, 0x69, 0x70, 0x71, 0x72, 0x73, 0x74, 0x75,
- 0x76, 0x77, 0x78, 0x80, 0x8a, 0x8b, 0x8c, 0x8d,
- 0x8e, 0x8f, 0x90, 0x6a, 0x9b, 0x9c, 0x9d, 0x9e,
- 0x9f, 0xa0, 0xaa, 0xab, 0xac, 0x4a, 0xae, 0xaf,
- 0xb0, 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7,
- 0xb8, 0xb9, 0xba, 0xbb, 0xbc, 0xa1, 0xbe, 0xbf,
- 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf, 0xda, 0xdb,
- 0xdc, 0xdd, 0xde, 0xdf, 0xea, 0xeb, 0xec, 0xed,
- 0xee, 0xef, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff)
-assert len(ascii2ebcdic) == 256
-
-def a2e(s):
- return bytearray((ascii2ebcdic[ord(c)] for c in s))
-
-def dump_records(out, records):
- while records:
- n = min(my_rand(5) + 1, len(records))
- r = records[:n]
- records[:n] = []
-
- count = sum((len(rec) for rec in r))
- out.buffer.write(struct.pack(">H xx", count + 4))
- for rec in r:
- out.buffer.write(rec)
-
-data = []
-for line in open('input.txt', 'r'):
- data += [line.rstrip('\r\n')]
-
-# MODE=BINARY
-out = open('binary.bin', 'w')
-for item in data:
- reclen = struct.pack("<I", len(item))
- out.buffer.write(reclen)
- out.buffer.write(bytearray([ord(c) for c in item]))
- out.buffer.write(reclen)
-out.close()
-
-# MODE=360 /RECFORM=FIXED /LRECL=32
-out = open('fixed.bin', 'w')
-lrecl = 32
-for item in data:
- s = item[:lrecl]
- s += ' ' * (lrecl - len(s))
- assert len(s) == 32
- out.buffer.write(a2e(s))
-out.close()
-
-# MODE=360 /RECFORM=VARIABLE
-out = open('variable.bin', 'w')
-records = []
-for item in data:
- records += [struct.pack('>H xx', len(item) + 4) + a2e(item)]
-dump_records(out, records)
-out.close()
-
-# MODE=360 /RECFORM=SPANNED
-out = open('spanned.bin', 'w')
-records = []
-for line in data:
- r = []
- while line:
- n = min(my_rand(5), len(line))
- r += [line[:n]]
- line = line[n:]
- for i, s in enumerate(r):
- scc = (0 if len(r) == 1
- else 1 if i == 0
- else 2 if i == len(r) - 1
- else 3)
- records += [struct.pack('>H B x', len(s) + 4, scc) + a2e(s)]
-dump_records(out, records)
-out.close()
-]])
- AT_CHECK([$PYTHON3 make-binary.py])
- AT_DATA([data-reader.sps], [dnl
-FILE HANDLE input/NAME='$2'/$1.
-DATA LIST FIXED FILE=input NOTABLE
- /1 start 1-10 (ADATE)
- /2 end 1-10 (ADATE)
- /3 count 1-3.
-LIST.
-
-* Output the data to a new file in the same format.
-FILE HANDLE OUTPUT/NAME='output.bin'/$1.
-COMPUTE count=count + 1.
-PRINT OUTFILE=output/start end count.
-EXECUTE.
-])
- AT_CHECK([pspp -O format=csv data-reader.sps], [0], [dnl
-Table: Data List
-start,end,count
-07/22/2007,10/06/2007,321
-07/14/1789,08/26/1789,4
-01/01/1972,12/31/1999,682
-])
- AT_CHECK([test -s output.bin])
- AT_DATA([data-reader-2.sps], [dnl
-* Re-read the new data and list it, to verify that it was written correctly.
-FILE HANDLE OUTPUT/NAME='output.bin'/$1.
-DATA LIST FIXED FILE=output NOTABLE/
- start 2-11 (ADATE)
- end 13-22 (ADATE)
- count 24-26.
-LIST.
-])
- AT_CHECK([pspp -O format=csv data-reader-2.sps], [0], [dnl
-Table: Data List
-start,end,count
-07/22/2007,10/06/2007,322
-07/14/1789,08/26/1789,5
-01/01/1972,12/31/1999,683
-])
- AT_CLEANUP])
-
-DATA_READER_BINARY([MODE=BINARY], [binary.bin])
-DATA_READER_BINARY([MODE=360 /RECFORM=FIXED /LRECL=32], [fixed.bin],
- [AT_CHECK([i18n-test supports_encodings EBCDIC-US])])
-DATA_READER_BINARY([MODE=360 /RECFORM=VARIABLE], [variable.bin],
- [AT_CHECK([i18n-test supports_encodings EBCDIC-US])])
-DATA_READER_BINARY([MODE=360 /RECFORM=SPANNED], [spanned.bin],
- [AT_CHECK([i18n-test supports_encodings EBCDIC-US])])
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DATASET commands])
-
-AT_SETUP([DATASET COPY])
-AT_DATA([dataset.pspp], [dnl
-DATASET NAME initial.
-DATA LIST NOTABLE /x 1.
-COMPUTE x = x + 1.
-DATASET COPY clone.
-BEGIN DATA.
-1
-2
-3
-4
-5
-END DATA.
-
-NEW FILE.
-DATA LIST NOTABLE /y 1.
-BEGIN DATA.
-6
-7
-8
-END DATA.
-LIST.
-DATASET DISPLAY.
-
-DATASET ACTIVATE clone.
-DATASET DISPLAY.
-LIST.
-
-DATASET ACTIVATE initial.
-DATASET DISPLAY.
-LIST.
-
-COMPUTE z=y.
-DATASET COPY clone.
-
-DATASET ACTIVATE clone.
-LIST.
-DATASET COPY clone.
-DATASET DISPLAY.
-
-DATASET CLOSE initial.
-DATASET DISPLAY.
-])
-AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
-Table: Data List
-y
-6
-7
-8
-
-Table: Datasets
-clone
-initial (active dataset)
-
-Table: Datasets
-clone (active dataset)
-initial
-
-Table: Data List
-x
-2
-3
-4
-5
-6
-
-Table: Datasets
-clone
-initial (active dataset)
-
-Table: Data List
-y
-6
-7
-8
-
-Table: Data List
-y,z
-6,6.00
-7,7.00
-8,8.00
-
-Table: Datasets
-unnamed dataset (active dataset)
-initial
-
-Table: Datasets
-unnamed dataset (active dataset)
-])
-AT_CLEANUP
-
-AT_SETUP([DATASET DECLARE])
-AT_DATA([dataset.pspp], [dnl
-DATASET DECLARE second.
-DATASET DISPLAY.
-DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-1
-END DATA.
-LIST.
-DATASET ACTIVATE second.
-DATASET DISPLAY.
-LIST.
-])
-AT_CHECK([pspp -O format=csv dataset.pspp], [1], [dnl
-Table: Datasets
-unnamed dataset (active dataset)
-second
-
-Table: Data List
-x
-1
-
-Table: Datasets
-second (active dataset)
-
-"dataset.pspp:10.1-10.4: error: LIST: LIST is allowed only after the active dataset has been defined.
- 10 | LIST.
- | ^~~~"
-])
-AT_CLEANUP
-
-AT_SETUP([DATASET NAME deletes duplicate name])
-AT_DATA([dataset.pspp], [dnl
-DATASET NAME a.
-DATASET DECLARE b.
-DATASET DECLARE c.
-DATASET DISPLAY.
-
-DATASET NAME b.
-DATASET NAME c.
-DATASET DISPLAY.
-])
-AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
-Table: Datasets
-a (active dataset)
-b
-c
-
-Table: Datasets
-c (active dataset)
-])
-AT_CLEANUP
-
-AT_SETUP([DATASET ACTIVATE deletes unnamed dataset])
-AT_DATA([dataset.pspp], [dnl
-DATASET DECLARE x.
-DATASET DISPLAY.
-
-DATASET ACTIVATE x.
-DATASET DISPLAY.
-])
-AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
-Table: Datasets
-unnamed dataset (active dataset)
-x
-
-Table: Datasets
-x (active dataset)
-])
-AT_CLEANUP
-
-AT_SETUP([DATASET ACTIVATE executes pending transformations])
-AT_DATA([dataset.pspp], [dnl
-DATASET NAME one.
-DATASET DECLARE another.
-DATASET DISPLAY.
-
-DATA LIST NOTABLE /x 1.
-PRINT/x.
-DATASET ACTIVATE another.
-BEGIN DATA.
-1
-2
-3
-4
-5
-END DATA.
-
-LIST.
-
-DATASET ACTIVATE one.
-LIST.
-])
-AT_CHECK([pspp -O format=csv dataset.pspp], [1], [dnl
-Table: Datasets
-another
-one (active dataset)
-
-1 @&t@
-
-2 @&t@
-
-3 @&t@
-
-4 @&t@
-
-5 @&t@
-
-"dataset.pspp:16.1-16.4: error: LIST: LIST is allowed only after the active dataset has been defined.
- 16 | LIST.
- | ^~~~"
-
-Table: Data List
-x
-1
-2
-3
-4
-5
-])
-AT_CLEANUP
-
-AT_SETUP([DATASET CLOSE])
-AT_DATA([dataset.pspp], [dnl
-DATASET DISPLAY
-DATASET CLOSE *.
-DATASET DISPLAY.
-
-DATASET NAME this.
-DATASET DISPLAY.
-DATASET CLOSE this.
-DATASET DISPLAY.
-
-DATASET NAME this.
-DATASET DISPLAY.
-DATASET CLOSE *.
-DATASET DISPLAY.
-
-DATASET DECLARE that.
-DATASET DECLARE theother.
-DATASET DECLARE yetanother.
-DATASET DISPLAY.
-DATASET CLOSE ALL.
-DATASET DISPLAY.
-
-DATASET NAME this.
-DATASET DECLARE that.
-DATASET DECLARE theother.
-DATASET DECLARE yetanother.
-DATASET DISPLAY.
-DATASET CLOSE ALL.
-DATASET DISPLAY.
-])
-AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
-Table: Datasets
-unnamed dataset (active dataset)
-
-Table: Datasets
-unnamed dataset (active dataset)
-
-Table: Datasets
-this (active dataset)
-
-Table: Datasets
-unnamed dataset (active dataset)
-
-Table: Datasets
-this (active dataset)
-
-Table: Datasets
-unnamed dataset (active dataset)
-
-Table: Datasets
-unnamed dataset (active dataset)
-that
-theother
-yetanother
-
-Table: Datasets
-unnamed dataset (active dataset)
-
-Table: Datasets
-that
-theother
-this (active dataset)
-yetanother
-
-Table: Datasets
-unnamed dataset (active dataset)
-])
-AT_CLEANUP
-
-
-
-dnl The bug for which the following test checks, is apparent only
-dnl when compiled with -fsanitize=address or run under valgrind
-AT_SETUP([DATASET heap overflow])
-AT_DATA([dataset.pspp], [dnl
-DATASET DECLARE initial.
-DATA LIST /x 1.
-
-DATASET COPY subsq.
-
-DATA LIST /y 2-4.
-BEGIN DATA.
-7
-END DATA.
-
-DATASET ACTIVATE subsq.
-
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv dataset.pspp], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-x,1,1-1,F1.0
-
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-y,1,2-4,F3.0
-])
-
-AT_CLEANUP
-
-AT_SETUP([DATASET syntax errors])
-AT_DATA([dataset.sps], [dnl
-DATASET NAME **.
-DATASET NAME xyzzy WINDOW **.
-
-DATASET NAME xyzzy.
-DATASET ACTIVATE **.
-DATASET ACTIVATE xyzzy WINDOW **.
-
-DATASET COPY **.
-DATASET COPY quux WINDOW **.
-
-DATASET DECLARE **.
-DATASET DECLARE foo WINDOW **.
-
-DATASET CLOSE **.
-])
-AT_CHECK([pspp dataset.sps], [1], [dnl
-dataset.sps:1.14-1.15: error: DATASET NAME: Syntax error expecting identifier.
- 1 | DATASET NAME **.
- | ^~
-
-dataset.sps:2.27-2.28: error: DATASET NAME: Syntax error expecting ASIS or
-FRONT.
- 2 | DATASET NAME xyzzy WINDOW **.
- | ^~
-
-dataset.sps:5.18-5.19: error: DATASET ACTIVATE: Syntax error expecting
-identifier.
- 5 | DATASET ACTIVATE **.
- | ^~
-
-dataset.sps:6.31-6.32: error: DATASET ACTIVATE: Syntax error expecting ASIS or
-FRONT.
- 6 | DATASET ACTIVATE xyzzy WINDOW **.
- | ^~
-
-dataset.sps:8.14-8.15: error: DATASET COPY: Syntax error expecting identifier.
- 8 | DATASET COPY **.
- | ^~
-
-dataset.sps:9.26-9.27: error: DATASET COPY: Syntax error expecting MINIMIZED,
-FRONT, or HIDDEN.
- 9 | DATASET COPY quux WINDOW **.
- | ^~
-
-dataset.sps:11.17-11.18: error: DATASET DECLARE: Syntax error expecting
-identifier.
- 11 | DATASET DECLARE **.
- | ^~
-
-dataset.sps:12.28-12.29: error: DATASET DECLARE: Syntax error expecting
-MINIMIZED, FRONT, or HIDDEN.
- 12 | DATASET DECLARE foo WINDOW **.
- | ^~
-
-dataset.sps:14.15-14.16: error: DATASET CLOSE: Syntax error expecting
-identifier.
- 14 | DATASET CLOSE **.
- | ^~
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([FILE HANDLE])
-
-AT_SETUP([FILE HANDLE])
-AT_DATA([wiggle.txt], [dnl
-1
-2
-5
-109
-])
-AT_DATA([file-handle.sps], [dnl
-FILE HANDLE myhandle /NAME='wiggle.txt'.
-DATA LIST LIST FILE=myhandle /x *.
-LIST.
-])
-AT_CHECK([pspp -O format=csv file-handle.sps], [0], [dnl
-Table: Reading free-form data from myhandle.
-Variable,Format
-x,F8.0
-
-Table: Data List
-x
-1.00
-2.00
-5.00
-109.00
-])
-AT_CLEANUP
-
-AT_SETUP([FILE HANDLE syntax errors])
-AT_DATA([file-handle.sps], [dnl
-FILE HANDLE **.
-FILE HANDLE x/NAME='x.txt'.
-FILE HANDLE x/NAME='x.txt'.
-FILE HANDLE y **.
-FILE HANDLE y/NAME=**.
-FILE HANDLE y/LRECL=8/LRECL=8.
-FILE HANDLE y/LRECL=**.
-FILE HANDLE y/TABWIDTH=8/TABWIDTH=8.
-FILE HANDLE y/TABWIDTH=**.
-FILE HANDLE y/MODE=CHARACTER/MODE=CHARACTER.
-FILE HANDLE y/MODE=**.
-FILE HANDLE y/ENDS=LF/ENDS=LF.
-FILE HANDLE y/ENDS=**.
-FILE HANDLE y/RECFORM=FIXED/RECFORM=FIXED.
-FILE HANDLE y/RECFORM=**.
-FILE HANDLE y/ENCODING='UTF-8'/ENCODING='UTF-8'.
-FILE HANDLE y/ENCODING=**.
-FILE HANDLE y/TABWIDTH=8.
-FILE HANDLE y/NAME='x.txt'/MODE=360.
-FILE HANDLE y/NAME='x.txt'/MODE=FIXED.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='file-handle.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"file-handle.sps:1.13-1.14: error: FILE HANDLE: Syntax error expecting identifier.
- 1 | FILE HANDLE **.
- | ^~"
-
-"file-handle.sps:3.13: error: FILE HANDLE: File handle x is already defined. Use CLOSE FILE HANDLE before redefining a file handle.
- 3 | FILE HANDLE x/NAME='x.txt'.
- | ^"
-
-"file-handle.sps:4.15-4.16: error: FILE HANDLE: Syntax error expecting `/'.
- 4 | FILE HANDLE y **.
- | ^~"
-
-"file-handle.sps:5.20-5.21: error: FILE HANDLE: Syntax error expecting string.
- 5 | FILE HANDLE y/NAME=**.
- | ^~"
-
-"file-handle.sps:6.23-6.27: error: FILE HANDLE: Subcommand LRECL may only be specified once.
- 6 | FILE HANDLE y/LRECL=8/LRECL=8.
- | ^~~~~"
-
-"file-handle.sps:7.21-7.22: error: FILE HANDLE: Syntax error expecting positive integer for LRECL.
- 7 | FILE HANDLE y/LRECL=**.
- | ^~"
-
-"file-handle.sps:8.26-8.33: error: FILE HANDLE: Subcommand TABWIDTH may only be specified once.
- 8 | FILE HANDLE y/TABWIDTH=8/TABWIDTH=8.
- | ^~~~~~~~"
-
-"file-handle.sps:9.24-9.25: error: FILE HANDLE: Syntax error expecting positive integer for TABWIDTH.
- 9 | FILE HANDLE y/TABWIDTH=**.
- | ^~"
-
-"file-handle.sps:10.30-10.33: error: FILE HANDLE: Subcommand MODE may only be specified once.
- 10 | FILE HANDLE y/MODE=CHARACTER/MODE=CHARACTER.
- | ^~~~"
-
-"file-handle.sps:11.20-11.21: error: FILE HANDLE: Syntax error expecting CHARACTER, BINARY, IMAGE, or 360.
- 11 | FILE HANDLE y/MODE=**.
- | ^~"
-
-"file-handle.sps:12.23-12.26: error: FILE HANDLE: Subcommand ENDS may only be specified once.
- 12 | FILE HANDLE y/ENDS=LF/ENDS=LF.
- | ^~~~"
-
-"file-handle.sps:13.20-13.21: error: FILE HANDLE: Syntax error expecting LF or CRLF.
- 13 | FILE HANDLE y/ENDS=**.
- | ^~"
-
-"file-handle.sps:14.29-14.35: error: FILE HANDLE: Subcommand RECFORM may only be specified once.
- 14 | FILE HANDLE y/RECFORM=FIXED/RECFORM=FIXED.
- | ^~~~~~~"
-
-"file-handle.sps:15.23-15.24: error: FILE HANDLE: Syntax error expecting FIXED, VARIABLE, or SPANNED.
- 15 | FILE HANDLE y/RECFORM=**.
- | ^~"
-
-"file-handle.sps:16.32-16.39: error: FILE HANDLE: Subcommand ENCODING may only be specified once.
- 16 | FILE HANDLE y/ENCODING='UTF-8'/ENCODING='UTF-8'.
- | ^~~~~~~~"
-
-"file-handle.sps:17.24-17.25: error: FILE HANDLE: Syntax error expecting string.
- 17 | FILE HANDLE y/ENCODING=**.
- | ^~"
-
-"file-handle.sps:18.1-18.25: error: FILE HANDLE: Required subcommand NAME was not specified.
- 18 | FILE HANDLE y/TABWIDTH=8.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"file-handle.sps:19.28-19.35: error: FILE HANDLE: RECFORM must be specified with MODE=360.
- 19 | FILE HANDLE y/NAME='x.txt'/MODE=360.
- | ^~~~~~~~"
-
-"file-handle.sps:20.33-20.37: error: FILE HANDLE: Syntax error expecting CHARACTER, BINARY, IMAGE, or 360.
- 20 | FILE HANDLE y/NAME='x.txt'/MODE=FIXED.
- | ^~~~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([GET DATA /TYPE=PSQL])
-
-m4_define([INIT_PSQL],
- [AT_SKIP_IF([test "$PSQL_SUPPORT" = no])
- PGDATA=`pwd`/cluster
- export PGDATA
- PGPORT=$PG_PORT
- export PGPORT
- socket_dir=`mktemp -d`
- PGHOST="$socket_dir"
- export PGHOST
- AT_CHECK([PATH=$PG_PATH:$PATH initdb -A trust], [0], [ignore])
- AT_CHECK([PATH=$PG_PATH:$PATH pg_ctl start -w -o "-k $socket_dir -h ''"], [0], [ignore])
- trap 'CLEANUP_PSQL' 0
- AT_CHECK([PATH=$PG_PATH:$PATH createdb -h "$socket_dir" -p $PG_PORT $PG_DBASE],
- [0], [ignore], [ignore])
- AT_DATA([populate.sql],
- [CREATE TABLE empty (a int, b date, c numeric(23, 4));
-
- -- a largeish table to check big queries work ok.
- CREATE TABLE large (x int);
- INSERT INTO large (select * from generate_series(1, 1000));
-
-
- CREATE TABLE thing (
- bool bool ,
- bytea bytea ,
- char char ,
- int8 int8 ,
- int2 int2 ,
- int4 int4 ,
- numeric numeric(50,6) ,
- text text ,
- oid oid ,
- float4 float4 ,
- float8 float8 ,
- money money ,
- pbchar bpchar ,
- varchar varchar ,
- date date ,
- time time ,
- timestamp timestamp ,
- timestamptz timestamptz ,
- interval interval ,
- timetz timetz
- );
-
- INSERT INTO thing VALUES (
- false,
- '0',
- 'a',
- '0',
- 0,
- 0,
- -256.098,
- 'this-long-text',
- 0,
- 0,
- 0,
- '0.01',
- 'a',
- 'A',
- '1-Jan-2000',
- '00:00',
- 'January 8 04:05:06 1999',
- 'January 8 04:05:06 1999 PST',
- '1 minutes',
- '10:09 UTC+4'
- );
-
- INSERT INTO thing VALUES (
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null,
- null
- );
-
- INSERT INTO thing VALUES (
- true,
- '1',
- 'b',
- '1',
- 1,
- 1,
- 65535.00001,
- 'that-long-text',
- 1,
- 1,
- 1,
- '1.23',
- 'b',
- 'B',
- '10-Jan-1963',
- '01:05:02',
- '10-Jan-1963 23:58:00',
- '10-Jan-1963 23:58:00 CET',
- '2 year 1 month 12 days 1 hours 3 minutes 4 seconds',
- '01:05:02 UTC-7'
- );
-])
-
- # On Debian, the psql binary in the postgres bindir won't work because
- # it needs libreadline to be LD_PRELOADed into it. The psql in the
- # normal $PATH works fine though.
- if (PATH=$PG_PATH:$PATH psql -V) >/dev/null 2>&1; then
- psql () {
- PATH=$PG_PATH:$PATH command psql "$$@@"
- }
- fi
- AT_CHECK([psql -h "$socket_dir" -p $PG_PORT $PG_DBASE < populate.sql],
- [0], [ignore])])
-
-m4_define([CLEANUP_PSQL], [PATH=$PG_PATH:$PATH pg_ctl stop -W -o "-k $socket_dir -h ''"])
-
-AT_SETUP([GET DATA /TYPE=PSQL])
-AT_KEYWORDS([slow])
-INIT_PSQL
-
-dnl Test with an ordinary query.
-AT_CHECK([cat > ordinary-query.sps <<EOF
-GET DATA /TYPE=psql
- /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
- /UNENCRYPTED
- /SQL="select * from thing".
-
-DISPLAY DICTIONARY.
-
-LIST.
-EOF
-])
-AT_CHECK([pspp -o pspp.csv ordinary-query.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-bool,1,Scale,Input,8,Right,F8.2,F8.2
-bytea,2,Nominal,Input,1,Left,AHEX2,AHEX2
-char,3,Nominal,Input,8,Left,A8,A8
-int8,4,Scale,Input,8,Right,F8.2,F8.2
-int2,5,Scale,Input,8,Right,F8.2,F8.2
-int4,6,Scale,Input,8,Right,F8.2,F8.2
-numeric,7,Scale,Input,8,Right,E40.6,E40.6
-text,8,Nominal,Input,16,Left,A16,A16
-oid,9,Scale,Input,8,Right,F8.2,F8.2
-float4,10,Scale,Input,8,Right,F8.2,F8.2
-float8,11,Scale,Input,8,Right,F8.2,F8.2
-money,12,Scale,Input,8,Right,DOLLAR8.2,DOLLAR8.2
-pbchar,13,Nominal,Input,8,Left,A8,A8
-varchar,14,Nominal,Input,8,Left,A8,A8
-date,15,Scale,Input,8,Right,DATE11,DATE11
-time,16,Scale,Input,8,Right,TIME11.0,TIME11.0
-timestamp,17,Scale,Input,8,Right,DATETIME22.0,DATETIME22.0
-timestamptz,18,Scale,Input,8,Right,DATETIME22.0,DATETIME22.0
-interval,19,Scale,Input,8,Right,DTIME13.0,DTIME13.0
-interval_months,20,Scale,Input,8,Right,F3.0,F3.0
-timetz,21,Scale,Input,8,Right,TIME11.0,TIME11.0
-timetz_zone,22,Scale,Input,8,Right,F8.2,F8.2
-
-Table: Data List
-bool,bytea,char,int8,int2,int4,numeric,text,oid,float4,float8,money,pbchar,varchar,date,time,timestamp,timestamptz,interval,interval_months,timetz,timetz_zone
-.00,30,a,.00,.00,.00,-2.560980E+002,this-long-text,.00,.00,.00,$.01,a,A,01-JAN-2000,00:00:00,08-JAN-1999 04:05:06,08-JAN-1999 12:05:06,0 00:01:00,0,10:09:00,4.00
-. ,,,. ,. ,. ,. ,,. ,. ,. ,. ,,,.,.,.,.,.,.,.,. @&t@
-1.00,31,b,1.00,1.00,1.00,6.553500E+004,that-long-text,.00,1.00,1.00,$1.23,b,B,10-JAN-1963,01:05:02,10-JAN-1963 23:58:00,10-JAN-1963 22:58:00,12 01:03:04,25,01:05:02,-7.00
-])
-
-dnl Test query with empty result set.
-AT_CHECK([cat > empty-result.sps <<EOF
-GET DATA /TYPE=psql
- /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
- /UNENCRYPTED
- /SQL="select * from empty".
-
-DISPLAY DICTIONARY.
-
-LIST.
-EOF
-])
-AT_CHECK([pspp -o pspp.csv empty-result.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-a,1,Scale,Input,8,Right,F8.2,F8.2
-b,2,Scale,Input,8,Right,DATE11,DATE11
-c,3,Scale,Input,8,Right,E40.2,E40.2
-])
-
-dnl Test query with large result set.
-AT_CHECK([cat > large-result.sps <<EOF
-GET DATA /TYPE=psql
- /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
- /UNENCRYPTED
- /SQL="select * from large".
-
-NUMERIC diff.
-COMPUTE diff = x - lag (x).
-
-TEMPORARY.
-SELECT IF (diff <> 1).
-LIST.
-
-TEMPORARY.
-N OF CASES 6.
-LIST.
-
-SORT CASES BY x (D).
-
-TEMPORARY.
-N OF CASES 6.
-LIST.
-EOF
-])
-AT_CHECK([pspp -o pspp.csv large-result.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-x,diff
-1.00,. @&t@
-2.00,1.00
-3.00,1.00
-4.00,1.00
-5.00,1.00
-6.00,1.00
-
-Table: Data List
-x,diff
-1000.00,1.00
-999.00,1.00
-998.00,1.00
-997.00,1.00
-996.00,1.00
-995.00,1.00
-])
-
-dnl Check for a bug caused by having string variables in the database,
-dnl all of which are null.
-AT_DATA([all-null-string.sql],
- [-- a table which has a text field containing only null, or zero
- -- length entries.
-
- CREATE TABLE foo (int4 int4, text text);
-
- INSERT INTO foo VALUES ('12', '');
-
- INSERT INTO foo VALUES (null, '');
-])
-AT_CHECK([psql -h "$socket_dir" -p $PG_PORT $PG_DBASE < all-null-string.sql],
- [0], [ignore])
-AT_CAPTURE_FILE([get-data.sps])
-AT_CHECK([cat > get-data.sps <<EOF
-GET DATA /TYPE=psql
- /CONNECT="host=$socket_dir port=$PGPORT dbname=$PG_DBASE"
- /UNENCRYPTED
- /SQL="select * from foo".
-
-DISPLAY DICTIONARY.
-
-LIST.
-EOF
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CAPTURE_FILE([pspp.csv])
-rm -rf "$socket_dir"
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-m4_define([SPREADSHEET_TEST_PREP],[dnl
- AT_KEYWORDS([spreadsheet])
- m4_if($1,[GNM],[dnl
- AT_CHECK([gzip -c $top_srcdir/tests/language/data-io/Book1.gnm.unzipped > Book1.gnumeric])dnl
- m4_define([testsheet],[Book1.gnumeric])dnl
- ]) dnl
- m4_if($1,[ODS],[dnl
- AT_CHECK([cp $top_srcdir/tests/language/data-io/test.ods test.ods])dnl
- m4_define([testsheet],[test.ods])dnl
- ])dnl
-])
-
-m4_define([CHECK_SPREADSHEET_READER],
- [dnl
-AT_SETUP([GET DATA /TYPE=$1 with CELLRANGE])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-GET DATA /TYPE=$1 /FILE='testsheet' /READNAMES=off /SHEET=name 'This' /CELLRANGE=range 'g9:i13' .
-DISPLAY VARIABLES.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Print Format,Write Format
-VAR001,1,F8.2,F8.2
-VAR002,2,A8,A8
-VAR003,3,F8.2,F8.2
-
-Table: Data List
-VAR001,VAR002,VAR003
-.00,fred,20.00
-1.00,11,21.00
-2.00,twelve,22.00
-3.00,13,23.00
-4.00,14,24.00
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=$1 with CELLRANGE and READNAMES])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-GET DATA /TYPE=$1 /FILE='testsheet' /READNAMES=on /SHEET=name 'This' /CELLRANGE=range 'g8:i13' .
-DISPLAY VARIABLES.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Print Format,Write Format
-V1,1,F8.2,F8.2
-V2,2,A8,A8
-VAR001,3,F8.2,F8.2
-
-Table: Data List
-V1,V2,VAR001
-.00,fred,20.00
-1.00,11,21.00
-2.00,twelve,22.00
-3.00,13,23.00
-4.00,14,24.00
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=$1 without CELLRANGE])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-GET DATA /TYPE=$1 /FILE='testsheet' /SHEET=index 3.
-DISPLAY VARIABLES.
-LIST.
-])
-AT_CHECK([pspp -O format=csv get-data.sps], [0], [dnl
-Table: Variables
-Name,Position,Print Format,Write Format
-name,1,A8,A8
-id,2,F8.2,F8.2
-height,3,F8.2,F8.2
-
-warning: Cannot convert the value in the spreadsheet cell C4 to format (F8.2): Field contents are not numeric.
-
-Table: Data List
-name,id,height
-fred,.00,23.40
-bert,1.00,.56
-charlie,2.00,. @&t@
-dick,3.00,-34.09
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=$1 with missing data])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-* This sheet has no data in one of its variables
-GET DATA /TYPE=$1 /FILE='testsheet' /READNAMES=on /SHEET=index 5.
-DISPLAY VARIABLES.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Print Format,Write Format
-vone,1,F8.2,F8.2
-vtwo,2,F8.2,F8.2
-vthree,3,A8,A8
-v4,4,F8.2,F8.2
-
-Table: Data List
-vone,vtwo,vthree,v4
-1.00,3.00,,5.00
-2.00,4.00,,6.00
-])
-AT_CLEANUP
-
-dnl This syntax doesn't do anything particularly useful.
-dnl It has been seen to cause a few crashes, so we check here that it
-dnl doesn't do anthing bad.
-AT_SETUP([GET DATA /TYPE=$1 with no options])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-* This sheet is empty
-GET DATA /TYPE=$1 /FILE='testsheet'.
-DISPLAY DICTIONARY.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [ignore])
-AT_CLEANUP
-
-
-
-AT_SETUP([GET DATA /TYPE=$1 with empty sheet])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-* This sheet is empty
-GET DATA /TYPE=$1 /FILE='testsheet' /SHEET=name 'Empty'.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [dnl
-warning: Selected sheet or range of spreadsheet `testsheet' is empty.
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=$1 with nonexistent sheet])
-SPREADSHEET_TEST_PREP($1)
-AT_DATA([get-data.sps], [dnl
-* This sheet doesnt exist.
-GET DATA /TYPE=$1 /FILE='testsheet' /SHEET=name 'foobarxx'.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [dnl
-warning: Selected sheet or range of spreadsheet `testsheet' is empty.
-])
-AT_CLEANUP
-])
-
-
-AT_BANNER([GET DATA Spreadsheet /TYPE=GNM])
-
-CHECK_SPREADSHEET_READER([GNM])
-
-dnl Check for a bug where gnumeric files were interpreted incorrectly
-AT_SETUP([GET DATA /TYPE=GNM sheet index bug])
-AT_KEYWORDS([spreadsheet])
-AT_DATA([minimal3.gnumeric],[dnl
-<?xml version="1.0" encoding="UTF-8"?>
-<gnm:Workbook xmlns:gnm="http://www.gnumeric.org/v10.dtd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gnumeric.org/v9.xsd">
- <gnm:Version Epoch="1" Major="10" Minor="8" Full="1.10.8"/>
- <gnm:SheetNameIndex>
- <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet1</gnm:SheetName>
- <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet2</gnm:SheetName>
- <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet3</gnm:SheetName>
- </gnm:SheetNameIndex>
- <gnm:Sheets>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
- <gnm:Name>Sheet1</gnm:Name>
- <gnm:MaxCol>2</gnm:MaxCol>
- <gnm:MaxRow>3</gnm:MaxRow>
- <gnm:Names>
- <gnm:Name>
- <gnm:name>Print_Area</gnm:name>
- <gnm:value>#REF!</gnm:value>
- <gnm:position>A1</gnm:position>
- </gnm:Name>
- <gnm:Name>
- <gnm:name>Sheet_Title</gnm:name>
- <gnm:value>"Sheet1"</gnm:value>
- <gnm:position>A1</gnm:position>
- </gnm:Name>
- </gnm:Names>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="94.5" HardSize="1"/>
- <gnm:ColInfo No="1" Unit="48" Count="2"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="13.5" Count="4"/>
- </gnm:Rows>
- <gnm:Cells>
- <gnm:Cell Row="0" Col="0" ValueType="60">Name</gnm:Cell>
- <gnm:Cell Row="0" Col="1" ValueType="60">x</gnm:Cell>
- <gnm:Cell Row="0" Col="2" ValueType="60">y</gnm:Cell>
- <gnm:Cell Row="1" Col="0" ValueType="60">Sheet One</gnm:Cell>
- <gnm:Cell Row="1" Col="1" ValueType="40">1</gnm:Cell>
- <gnm:Cell Row="1" Col="2" ValueType="40">2</gnm:Cell>
- <gnm:Cell Row="2" Col="0" ValueType="60">foo</gnm:Cell>
- <gnm:Cell Row="2" Col="1" ValueType="40">3</gnm:Cell>
- <gnm:Cell Row="2" Col="2" ValueType="40">4</gnm:Cell>
- <gnm:Cell Row="3" Col="0" ValueType="60">bar</gnm:Cell>
- <gnm:Cell Row="3" Col="1" ValueType="40">5</gnm:Cell>
- <gnm:Cell Row="3" Col="2" ValueType="40">6</gnm:Cell>
- </gnm:Cells>
- </gnm:Sheet>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
- <gnm:Name>Sheet2</gnm:Name>
- <gnm:MaxCol>2</gnm:MaxCol>
- <gnm:MaxRow>2</gnm:MaxRow>
- <gnm:Names>
- <gnm:Name>
- <gnm:name>Print_Area</gnm:name>
- <gnm:value>#REF!</gnm:value>
- <gnm:position>A1</gnm:position>
- </gnm:Name>
- <gnm:Name>
- <gnm:name>Sheet_Title</gnm:name>
- <gnm:value>"Sheet2"</gnm:value>
- <gnm:position>A1</gnm:position>
- </gnm:Name>
- </gnm:Names>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="48"/>
- <gnm:ColInfo No="1" Unit="57.75"/>
- <gnm:ColInfo No="2" Unit="54.75"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="13.5" Count="3"/>
- </gnm:Rows>
- <gnm:Cells>
- <gnm:Cell Row="0" Col="0" ValueType="60">Comment</gnm:Cell>
- <gnm:Cell Row="0" Col="1" ValueType="60">DOB</gnm:Cell>
- <gnm:Cell Row="0" Col="2" ValueType="60">wealth</gnm:Cell>
- <gnm:Cell Row="1" Col="0" ValueType="60">Sheet Two</gnm:Cell>
- <gnm:Cell Row="1" Col="1" ValueType="60">24/5/1966</gnm:Cell>
- <gnm:Cell Row="1" Col="2" ValueType="40" ValueFormat="_($* 0.00_);_($* (0.00);_($* "-"??_);_(@_)">0.02</gnm:Cell>
- <gnm:Cell Row="2" Col="0" ValueType="60">wee</gnm:Cell>
- <gnm:Cell Row="2" Col="1" ValueType="40" ValueFormat="dd/mm/yyyy">37145</gnm:Cell>
- <gnm:Cell Row="2" Col="2" ValueType="40" ValueFormat="_($* 0.00_);_($* (0.00);_($* "-"??_);_(@_)">3000</gnm:Cell>
- </gnm:Cells>
- </gnm:Sheet>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
- <gnm:Name>Sheet3</gnm:Name>
- <gnm:MaxCol>2</gnm:MaxCol>
- <gnm:MaxRow>2</gnm:MaxRow>
- <gnm:Names>
- <gnm:Name>
- <gnm:name>Print_Area</gnm:name>
- <gnm:value>#REF!</gnm:value>
- <gnm:position>A1</gnm:position>
- </gnm:Name>
- <gnm:Name>
- <gnm:name>Sheet_Title</gnm:name>
- <gnm:value>"Sheet3"</gnm:value>
- <gnm:position>A1</gnm:position>
- </gnm:Name>
- </gnm:Names>
- <gnm:Cols DefaultSizePts="48">
- <gnm:ColInfo No="0" Unit="48" Count="3"/>
- </gnm:Cols>
- <gnm:Rows DefaultSizePts="12.75">
- <gnm:RowInfo No="0" Unit="13.5"/>
- <gnm:RowInfo No="1" Unit="12.75" Count="2"/>
- </gnm:Rows>
- <gnm:Cells>
- <gnm:Cell Row="0" Col="0" ValueType="40">3</gnm:Cell>
- <gnm:Cell Row="0" Col="1" ValueType="40">4</gnm:Cell>
- <gnm:Cell Row="0" Col="2" ValueType="40">5</gnm:Cell>
- <gnm:Cell Row="1" Col="0" ValueType="40">6</gnm:Cell>
- <gnm:Cell Row="1" Col="1" ValueType="40">7</gnm:Cell>
- <gnm:Cell Row="1" Col="2" ValueType="40">8</gnm:Cell>
- <gnm:Cell Row="2" Col="0" ValueType="40">9</gnm:Cell>
- <gnm:Cell Row="2" Col="1" ValueType="40">10</gnm:Cell>
- <gnm:Cell Row="2" Col="2" ValueType="40">11</gnm:Cell>
- </gnm:Cells>
- </gnm:Sheet>
- </gnm:Sheets>
-</gnm:Workbook>
-])
-
-AT_DATA([gnum.sps], [dnl
-GET DATA
- /TYPE=GNM
- /FILE='minimal3.gnumeric'
- /SHEET=index 3
- /READNAMES=off
- .
-
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv gnum.sps], [0], [dnl
-Table: Data List
-VAR001,VAR002,VAR003
-3,4.00,5.00
-6,7.00,8.00
-9,10.00,11.00
-])
-
-
-AT_CLEANUP
-
-
-dnl Check for a bug where certain gnumeric files failed an assertion
-AT_SETUP([GET DATA /TYPE=GNM assert-fail])
-AT_KEYWORDS([spreadsheet])
-AT_DATA([read.sps],[dnl
-GET DATA
- /TYPE=GNM
- /FILE='crash.gnumeric'
- .
-list.
-])
-
-
-AT_DATA([crash.gnumeric],[dnl
-<?xml version="1.0" encoding="UTF-8"?>
-<gnm:Workbook xmlns:gnm="http://www.gnumeric.org/v10.dtd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.gnumeric.org/v9.xsd">
- <office:document-meta xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:meta="urn:oasis:names:tc:opendocument:xmlns:meta:1.0" xmlns:ooo="http://openoffice.org/2004/office" office:version="1.1">
- </office:document-meta>
- <gnm:SheetNameIndex>
- <gnm:SheetName gnm:Cols="256" gnm:Rows="65536">Sheet1</gnm:SheetName>
- </gnm:SheetNameIndex>
- <gnm:Sheets>
- <gnm:Sheet DisplayFormulas="0" HideZero="0" HideGrid="0" HideColHeader="0" HideRowHeader="0" DisplayOutlines="1" OutlineSymbolsBelow="1" OutlineSymbolsRight="1" Visibility="GNM_SHEET_VISIBILITY_VISIBLE" GridColor="0:0:0">
- <gnm:Name>Sheet1</gnm:Name>
- <gnm:MaxCol>2</gnm:MaxCol>
- <gnm:MaxRow>4</gnm:MaxRow>
- <gnm:Styles>
- <gnm:StyleRegion startCol="0" startRow="0" endCol="255" endRow="65535">
- <gnm:Style HAlign="1" VAlign="2" WrapText="0" ShrinkToFit="0" Rotation="0" Shade="0" Indent="0" Locked="1" Hidden="0" Fore="0:0:0" Back="FFFF:FFFF:FFFF" PatternColor="0:0:0" Format="General">
- </gnm:Style>
- </gnm:StyleRegion>
- </gnm:Styles>
- <gnm:Cells>
- <gnm:Cell Row="1" Col="1" ValueType="60">one</gnm:Cell>
- <gnm:Cell Row="1" Col="2" ValueType="60">two</gnm:Cell>
- <gnm:Cell Row="2" Col="1" ValueType="40">1</gnm:Cell>
- <gnm:Cell Row="2" Col="2" ValueType="40">2</gnm:Cell>
- <gnm:Cell Row="3" Col="1" ValueType="40">1</gnm:Cell>
- <gnm:Cell Row="3" Col="2" ValueType="40">2</gnm:Cell>
- <gnm:Cell Row="4" Col="1" ValueType="40">1</gnm:Cell>
- <gnm:Cell Row="4" Col="2" ValueType="40">2</gnm:Cell>
- </gnm:Cells>
- </gnm:Sheet>
- </gnm:Sheets>
-</gnm:Workbook>
-])
-
-AT_CHECK([pspp -O format=csv read.sps], [0], [ignore])
-
-
-AT_CLEANUP
-
-
-
-AT_BANNER([GET DATA Spreadsheet /TYPE=ODS])
-
-CHECK_SPREADSHEET_READER([ODS])
-
-
-AT_SETUP([GET DATA /TYPE=ODS crash])
-AT_KEYWORDS([spreadsheet])
-
-
-AT_CHECK([cp $top_srcdir/tests/language/data-io/newone.ods this.ods])dnl
-
-AT_DATA([crash.sps],[dnl
-GET DATA /TYPE=ODS /FILE='this.ods' /CELLRANGE=RANGE 'A1:C8' /READNAMES=ON
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv crash.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([GET DATA /TYPE=ODS readnames])
-AT_KEYWORDS([spreadsheet])
-
-dnl Check for a bug where in the ODS reader /READNAMES incorrectly
-dnl dealt with repeated names.
-AT_CHECK([cp $top_srcdir/tests/language/data-io/readnames.ods this.ods])dnl
-
-AT_DATA([readnames.sps],[dnl
-GET DATA /TYPE=ODS /FILE='this.ods' /CELLRANGE=RANGE 'A1:H8' /READNAMES=ON
-EXECUTE.
-DISPLAY DICTIONARY.
-LIST.
-])
-
-
-AT_CHECK([pspp -O format=csv readnames.sps], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-freda,1,Nominal,Input,8,Right,F8.2,F8.2
-fred,2,Nominal,Input,8,Right,F8.2,F8.2
-fred_A,3,Nominal,Input,8,Right,F8.2,F8.2
-fred_B,4,Nominal,Input,8,Right,F8.2,F8.2
-fred_C,5,Nominal,Input,8,Right,F8.2,F8.2
-fred_D,6,Nominal,Input,8,Right,F8.2,F8.2
-fred_E,7,Nominal,Input,8,Right,F8.2,F8.2
-
-Table: Data List
-freda,fred,fred_A,fred_B,fred_C,fred_D,fred_E
-1.00,2.00,3.00,4.00,5.00,6.00,7.00
-8.00,9.00,10.00,11.00,12.00,13.00,14.00
-])
-
-AT_CLEANUP
-
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([GET DATA /TYPE=TXT])
-
-dnl These tests exercise features of GET DATA /TYPE=TXT that
-dnl it has in common with DATA LIST, using tests drawn from
-dnl DATA LIST.
-
-AT_SETUP([GET DATA /TYPE=TXT with explicit delimiters])
-AT_DATA([get-data.sps], [dnl
-get data /type=txt /file=inline /delimiters="|X"
- /variables=A f7.2 B f7.2 C f7.2 D f7.2.
-begin data.
-1|23X45|2.03
-2X22|34|23|
-3|34|34X34
-end data.
-
-list.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-A,B,C,D
-1.00,23.00,45.00,2.03
-2.00,22.00,34.00,23.00
-3.00,34.00,34.00,34.00
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=TXT with FIRSTCASE])
-AT_DATA([get-data.sps], [dnl
-get data /type=txt /file=inline /delimiters=', ' /delcase=variables 4
- /firstcase=2 /variables=A f7.2 B f7.2 C f7.2 D f7.2.
-begin data.
-# This record is ignored.
-,1,2,3
-,4,,5
-6
-7,
-
-8 9
-0,1,,,
-
-,,,,
-
-2
-
-3
-4
-5
-end data.
-list.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps], [0], [ignore])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-A,B,C,D
-. ,1.00,2.00,3.00
-. ,4.00,. ,5.00
-6.00,7.00,. ,8.00
-9.00,.00,1.00,. @&t@
-. ,. ,. ,. @&t@
-. ,. ,. ,2.00
-. ,3.00,4.00,5.00
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=TXT with FIRSTCASE and tab delimiter])
-AT_DATA([get-data.sps], [dnl
-get data /type=txt /file=inline /delimiters='\t' /delcase=variables 4
- /firstcase=3 /variables=A f7.2 B f7.2 C f7.2 D f7.2.
-begin data.
-# These records
-# are skipped.
-1 2 3 4
-1 2 3 4 @&t@
-1 2 4
-1 2 4 @&t@
-1 3 4
-1 3 4 @&t@
-1 4
-1 4 @&t@
- 2 3 4
- 2 3 4 @&t@
- 2 4
- 2 4 @&t@
- 3 4
- 3 4 @&t@
- 4
- 4 @&t@
-end data.
-list.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-A,B,C,D
-1.00,2.00,3.00,4.00
-1.00,2.00,3.00,4.00
-1.00,2.00,. ,4.00
-1.00,2.00,. ,4.00
-1.00,. ,3.00,4.00
-1.00,. ,3.00,4.00
-1.00,. ,. ,4.00
-1.00,. ,. ,4.00
-. ,2.00,3.00,4.00
-. ,2.00,3.00,4.00
-. ,2.00,. ,4.00
-. ,2.00,. ,4.00
-. ,. ,3.00,4.00
-. ,. ,3.00,4.00
-. ,. ,. ,4.00
-. ,. ,. ,4.00
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=TXT with multiple records per case])
-AT_DATA([get-data.sps], [dnl
-get data /type=txt /file=inline /arrangement=fixed /fixcase=3 /variables=
- /1 start 0-19 adate8
- /2 end 0-19 adate
- /3 count 0-2 f.
-begin data.
-07-22-2007
-10-06-2007
-321
-07-14-1789
-08-26-1789
-4
-01-01-1972
-12-31-1999
-682
-end data.
-list.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-start,end,count
-07/22/07,10/06/2007,321
-********,08/26/1789,4
-01/01/72,12/31/1999,682
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=TXT with empty trailing record])
-AT_DATA([get-data.sps], [dnl
-get data /type=txt /file=inline /arrangement=fixed /fixcase=2 /variables=
- /1 x 0 f
- y 1 f.
-begin data.
-12
-
-34
-
-56
-
-78
-
-90
-
-end data.
-list.
-])
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-x,y
-1,2
-3,4
-5,6
-7,8
-9,0
-])
-AT_CLEANUP
-
-dnl This test is a copy of an example given in the manual
-dnl in doc/files.texi.
-AT_SETUP([GET DATA /TYPE=TXT password example])
-AT_DATA([passwd.data], [dnl
-root:$1$nyeSP5gD$pDq/:0:0:,,,:/root:/bin/bash
-blp:$1$BrP/pFg4$g7OG:1000:1000:Ben Pfaff,,,:/home/blp:/bin/bash
-john:$1$JBuq/Fioq$g4A:1001:1001:John Darrington,,,:/home/john:/bin/bash
-jhs:$1$D3li4hPL$88X1:1002:1002:Jason Stover,,,:/home/jhs:/bin/csh
-])
-AT_DATA([passwd.sps], [dnl
-GET DATA /TYPE=TXT /FILE='passwd.data' /DELIMITERS=':'
- /VARIABLES=username A20
- password A40
- uid F10
- gid F10
- gecos A40
- home A40
- shell A40.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv passwd.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-username,password,uid,gid,gecos,home,shell
-root,$1$nyeSP5gD$pDq/,0,0,",,,",/root,/bin/bash
-blp,$1$BrP/pFg4$g7OG,1000,1000,"Ben Pfaff,,,",/home/blp,/bin/bash
-john,$1$JBuq/Fioq$g4A,1001,1001,"John Darrington,,,",/home/john,/bin/bash
-jhs,$1$D3li4hPL$88X1,1002,1002,"Jason Stover,,,",/home/jhs,/bin/csh
-])
-AT_CLEANUP
-
-dnl This test is a copy of an example given in the manual
-dnl in doc/files.texi.
-AT_SETUP([GET DATA /TYPE=TXT cars example])
-AT_DATA([cars.data], [dnl
-model year mileage price type age
-Civic 2002 29883 15900 Si 2
-Civic 2003 13415 15900 EX 1
-Civic 1992 107000 3800 n/a 12
-Accord 2002 26613 17900 EX 1
-])
-AT_DATA([cars.sps], [dnl
-GET DATA /TYPE=TXT /FILE='cars.data' /DELIMITERS=' ' /FIRSTCASE=2
- /VARIABLES=model A8
- year F4
- mileage F6
- price F5
- type A4
- age F2.
-LIST.
-
-GET DATA /TYPE=TXT /FILE='cars.data' /ARRANGEMENT=FIXED /FIRSTCASE=2
- /VARIABLES=model 0-7 A
- year 8-15 F
- mileage 16-23 F
- price 24-31 F
- type 32-39 A
- age 40-47 F.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv cars.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-model,year,mileage,price,type,age
-Civic,2002,29883,15900,Si,2
-Civic,2003,13415,15900,EX,1
-Civic,1992,107000,3800,n/a,12
-Accord,2002,26613,17900,EX,1
-
-Table: Data List
-model,year,mileage,price,type,age
-Civic,2002,29883,15900,Si,2
-Civic,2003,13415,15900,EX,1
-Civic,1992,107000,3800,n/a,12
-Accord,2002,26613,17900,EX,1
-])
-AT_CLEANUP
-
-dnl This test is a copy of an example given in the manual
-dnl in doc/files.texi.
-AT_SETUP([GET DATA /TYPE=TXT pets example])
-AT_DATA([pets.data], [dnl
-'Pet''s Name', "Age", "Color", "Date Received", "Price", "Height", "Type"
-, (Years), , , (Dollars), ,
-"Rover", 4.5, Brown, "12 Feb 2004", 80, '1''4"', "Dog"
-"Charlie", , Gold, "5 Apr 2007", 12.3, "3""", "Fish"
-"Molly", 2, Black, "12 Dec 2006", 25, '5"', "Cat"
-"Gilly", , White, "10 Apr 2007", 10, "3""", "Guinea Pig"
-])
-AT_DATA([pets.sps], [dnl
-GET DATA /TYPE=TXT /FILE='pets.data' /DELIMITERS=', ' /QUALIFIER='''"'
- /FIRSTCASE=3
- /VARIABLES=name A10
- age F3.1
- color A5
- received EDATE10
- price F5.2
- height a5
- type a10.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv pets.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-name,age,color,received,price,height,type
-Rover,4.5,Brown,12.02.2004,80.00,"1'4""",Dog
-Charlie,. ,Gold,05.04.2007,12.30,"3""",Fish
-Molly,2.0,Black,12.12.2006,25.00,"5""",Cat
-Gilly,. ,White,10.04.2007,10.00,"3""",Guinea Pig
-])
-AT_CLEANUP
-dnl " (fixes Emacs highlighting)
-
-AT_SETUP([GET DATA /TYPE=TXT with IMPORTCASE])
-AT_CHECK([$PYTHON3 > test.data -c '
-for i in range(1, 101):
- print("%02d" % i)
-'])
-AT_DATA([get-data.sps], [dnl
-get data /type=txt /file='test.data' /importcase=first 10 /variables x f8.0.
-get data /type=txt /file='test.data' /importcase=percent 1 /variables x f8.0.
-get data /type=txt /file='test.data' /importcase=percent 35 /variables x f8.0.
-get data /type=txt /file='test.data' /importcase=percent 95 /variables x f8.0.
-get data /type=txt /file='test.data' /importcase=percent 100 /variables x f8.0.
-])
-AT_CHECK([pspp -O format=csv get-data.sps], [0], [dnl
-"get-data.sps:1.39-1.57: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
- 1 | get data /type=txt /file='test.data' /importcase=first 10 /variables x f8.0.
- | ^~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:2.39-2.58: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
- 2 | get data /type=txt /file='test.data' /importcase=percent 1 /variables x f8.0.
- | ^~~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:3.39-3.59: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
- 3 | get data /type=txt /file='test.data' /importcase=percent 35 /variables x f8.0.
- | ^~~~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:4.39-4.59: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
- 4 | get data /type=txt /file='test.data' /importcase=percent 95 /variables x f8.0.
- | ^~~~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:5.39-5.60: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
- 5 | get data /type=txt /file='test.data' /importcase=percent 100 /variables x f8.0.
- | ^~~~~~~~~~~~~~~~~~~~~~"
-])
-AT_CLEANUP
-
-AT_SETUP([GET DATA /TYPE=TXT with ENCODING subcommand])
-AT_CHECK([i18n-test supports_encodings UTF-8 ISO-8859-1])
-AT_DATA([get-data.sps], [dnl
-set locale='utf-8'
-get data /type=txt /file='data.txt' /encoding='iso-8859-1'
- /delimiters="," /variables=s a8.
-list.
-])
-printf '\351' > data.txt # é in ISO-8859-1.
-AT_CHECK([pspp -o pspp.csv get-data.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-s
-é
-])
-AT_CLEANUP
-
-
-AT_SETUP([GET DATA /TYPE= truncated])
-
-AT_DATA([x.sps], [dnl
-GET DATA /TYPE=
-.
-])
-
-AT_CHECK([pspp -o pspp.csv x.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([GET DATA /TYPE=txt bug])
-
-
-AT_DATA([thing.txt], [dnl
-foo, title, last
-1, this, 1
-2, that, 2
-3, other, 3
-])
-
-AT_DATA([x.sps], [dnl
-GET DATA
- /TYPE=TXT
- /FILE="thing.txt"
- /ARRANGEMENT=DELIMITED
- /DELCASE=LINE
- /FIRSTCASE=2
- /DELIMITERS=","
- /VARIABLES=foo F1.0
- title A8
- last F2.0.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv x.sps], [0], [dnl
-Table: Data List
-foo,title,last
-1,this,1
-2,that,2
-3,other,3
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([GET DATA /TYPE=txt another bug])
-
-AT_DATA([crash.sps], [dnl
-get data /type=txt /file=inline /variables=C f7.2 D f7>2.
-begin data.
-3 2
-4 2
-5 2
-end data.
-])
-
-AT_CHECK([pspp -O format=csv crash.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-
-
-
+++ /dev/null
-AT_BANNER([GET DATA])
-
-AT_SETUP([GET DATA syntax errors])
-AT_DATA([get-data.sps], [dnl
-GET DATA **.
-GET DATA / **.
-GET DATA /TYPE **.
-
-GET DATA /TYPE=TXT **.
-GET DATA /TYPE=TXT/ **.
-GET DATA /TYPE=TXT/FILE **.
-GET DATA /TYPE=TXT/FILE='x.txt' **.
-GET DATA /TYPE=TXT/FILE='x.txt' /ENCODING=**.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=**.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /ARRANGEMENT=DELIMITED.
-GET DATA /TYPE=TXT/FILE='x.txt' /FIRSTCASE=**.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELCASE=LINE.
-GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=VARIABLES **.
-GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=**.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=DELIMITED /FIXCASE=1.
-GET DATA /TYPE=TXT/FILE='x.txt' /FIXCASE=**.
-GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=FIRST **.
-GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=PERCENT **.
-GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=ALL.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELIMITERS=' '.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /QUALIFIER='"'.
-GET DATA /TYPE=TXT/FILE='x.txt' /QUALIFIER='"' + "'".
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES / **.
-GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES **.
-GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES a_very_long_name_that_exceeds_the_64_byte_limit_for_variable_names.
-GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x **.
-GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x F1.2.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x **.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 **.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 FOO.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DATE.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DOLLAR1.2.
-GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 F x 6-10 F.
-
-GET DATA /TYPE=PSQL **.
-GET DATA /TYPE=PSQL/ **.
-GET DATA /TYPE=PSQL/CONNECT **.
-GET DATA /TYPE=PSQL/CONNECT='db'/ASSUMEDSTRWIDTH=**.
-GET DATA /TYPE=PSQL/CONNECT='db'/BSIZE=**.
-GET DATA /TYPE=PSQL/CONNECT='db'/SQL=**.
-
-GET DATA /TYPE=GNM **.
-GET DATA /TYPE=GNM/ **.
-GET DATA /TYPE=GNM/FILE **.
-GET DATA /TYPE=GNM/FILE= **.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/ASSUMEDSTRWIDTH=**.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=NAME **.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=INDEX **.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=**.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=RANGE **.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=**.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/READNAMES=**.
-GET DATA /TYPE=GNM/FILE='x.gnumeric'/ **.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='get-data.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp -x compatible --testing-mode -O format=csv insert.sps], [1], [dnl
-"get-data.sps:1.10-1.11: error: GET DATA: Syntax error expecting `/TYPE='.
- 1 | GET DATA **.
- | ^~"
-
-"get-data.sps:2.10-2.13: error: GET DATA: Syntax error expecting `/TYPE='.
- 2 | GET DATA / **.
- | ^~~~"
-
-"get-data.sps:3.10-3.17: error: GET DATA: Syntax error expecting `/TYPE='.
- 3 | GET DATA /TYPE **.
- | ^~~~~~~~"
-
-"get-data.sps:5.20-5.21: error: GET DATA: Syntax error expecting `/FILE='.
- 5 | GET DATA /TYPE=TXT **.
- | ^~"
-
-"get-data.sps:6.19-6.22: error: GET DATA: Syntax error expecting `/FILE='.
- 6 | GET DATA /TYPE=TXT/ **.
- | ^~~~"
-
-"get-data.sps:7.19-7.26: error: GET DATA: Syntax error expecting `/FILE='.
- 7 | GET DATA /TYPE=TXT/FILE **.
- | ^~~~~~~~"
-
-"get-data.sps:8.33-8.34: error: GET DATA: Syntax error expecting `/'.
- 8 | GET DATA /TYPE=TXT/FILE='x.txt' **.
- | ^~"
-
-"get-data.sps:9.43-9.44: error: GET DATA: Syntax error expecting string.
- 9 | GET DATA /TYPE=TXT/FILE='x.txt' /ENCODING=**.
- | ^~"
-
-"get-data.sps:10.46-10.47: error: GET DATA: Syntax error expecting FIXED or DELIMITED.
- 10 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=**.
- | ^~"
-
-get-data.sps:11: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
-
-"get-data.sps:11.53-11.73: note: GET DATA: This syntax requires DELIMITED arrangement.
- 11 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /ARRANGEMENT=DELIMITED.
- | ^~~~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:11.34-11.50: note: GET DATA: This syntax requires FIXED arrangement.
- 11 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /ARRANGEMENT=DELIMITED.
- | ^~~~~~~~~~~~~~~~~"
-
-"get-data.sps:12.44-12.45: error: GET DATA: Syntax error expecting positive integer for FIRSTCASE.
- 12 | GET DATA /TYPE=TXT/FILE='x.txt' /FIRSTCASE=**.
- | ^~"
-
-get-data.sps:13: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
-
-"get-data.sps:13.53-13.59: note: GET DATA: This syntax requires DELIMITED arrangement.
- 13 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELCASE=LINE.
- | ^~~~~~~"
-
-"get-data.sps:13.34-13.50: note: GET DATA: This syntax requires FIXED arrangement.
- 13 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELCASE=LINE.
- | ^~~~~~~~~~~~~~~~~"
-
-"get-data.sps:14.52-14.53: error: GET DATA: Syntax error expecting integer.
- 14 | GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=VARIABLES **.
- | ^~"
-
-"get-data.sps:15.42-15.43: error: GET DATA: Syntax error expecting LINE or VARIABLES.
- 15 | GET DATA /TYPE=TXT/FILE='x.txt' /DELCASE=**.
- | ^~"
-
-get-data.sps:16: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
-
-"get-data.sps:16.57-16.63: note: GET DATA: This syntax requires FIXED arrangement.
- 16 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=DELIMITED /FIXCASE=1.
- | ^~~~~~~"
-
-"get-data.sps:16.34-16.54: note: GET DATA: This syntax requires DELIMITED arrangement.
- 16 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=DELIMITED /FIXCASE=1.
- | ^~~~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:17.42-17.43: error: GET DATA: Syntax error expecting positive integer for FIXCASE.
- 17 | GET DATA /TYPE=TXT/FILE='x.txt' /FIXCASE=**.
- | ^~"
-
-"get-data.sps:18.52-18.53: error: GET DATA: Syntax error expecting integer.
- 18 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=FIRST **.
- | ^~"
-
-"get-data.sps:19.54-19.55: error: GET DATA: Syntax error expecting integer.
- 19 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=PERCENT **.
- | ^~"
-
-"get-data.sps:20.34-20.48: warning: GET DATA: Ignoring obsolete IMPORTCASES subcommand. (N OF CASES or SAMPLE may be used to substitute.).
- 20 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=ALL.
- | ^~~~~~~~~~~~~~~"
-
-"get-data.sps:20.49: error: GET DATA: Syntax error expecting `/'.
- 20 | GET DATA /TYPE=TXT/FILE='x.txt' /IMPORTCASES=ALL.
- | ^"
-
-get-data.sps:21: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
-
-"get-data.sps:21.53-21.62: note: GET DATA: This syntax requires DELIMITED arrangement.
- 21 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELIMITERS=' '.
- | ^~~~~~~~~~"
-
-"get-data.sps:21.34-21.50: note: GET DATA: This syntax requires FIXED arrangement.
- 21 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /DELIMITERS=' '.
- | ^~~~~~~~~~~~~~~~~"
-
-get-data.sps:22: error: GET DATA: FIXED and DELIMITED arrangements are mutually exclusive.
-
-"get-data.sps:22.53-22.61: note: GET DATA: This syntax requires DELIMITED arrangement.
- 22 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /QUALIFIER='""'.
- | ^~~~~~~~~"
-
-"get-data.sps:22.34-22.50: note: GET DATA: This syntax requires FIXED arrangement.
- 22 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /QUALIFIER='""'.
- | ^~~~~~~~~~~~~~~~~"
-
-"get-data.sps:23.44-23.52: error: GET DATA: In compatible syntax mode, the QUALIFIER string must contain exactly one character.
- 23 | GET DATA /TYPE=TXT/FILE='x.txt' /QUALIFIER='""' + ""'"".
- | ^~~~~~~~~"
-
-"get-data.sps:24.65-24.66: error: GET DATA: Syntax error expecting integer.
- 24 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES / **.
- | ^~"
-
-"get-data.sps:25.44-25.45: error: GET DATA: Syntax error expecting identifier.
- 25 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES **.
- | ^~"
-
-"get-data.sps:26.44-26.109: error: GET DATA: Identifier `a_very_long_name_that_exceeds_the_64_byte_limit_for_variable_names' exceeds 64-byte limit.
- 26 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES a_very_long_name_that_exceeds_the_64_byte_limit_for_variable_names.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"get-data.sps:27.46-27.47: error: GET DATA: Syntax error expecting valid format specifier.
- 27 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x **.
- | ^~"
-
-"get-data.sps:28.46-28.49: error: GET DATA: Input format F1.2 specifies 2 decimal places, but width 1 allows at most 1 decimals.
- 28 | GET DATA /TYPE=TXT/FILE='x.txt' /VARIABLES x F1.2.
- | ^~~~"
-
-"get-data.sps:29.65-29.66: error: GET DATA: Syntax error expecting integer.
- 29 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x **.
- | ^~"
-
-"get-data.sps:30.69-30.70: error: GET DATA: Syntax error expecting valid format specifier.
- 30 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 **.
- | ^~"
-
-"get-data.sps:31.69-31.71: error: GET DATA: Unknown format type `FOO'.
- 31 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 FOO.
- | ^~~"
-
-"get-data.sps:32.65-32.72: error: GET DATA: Input format DATE5 specifies width 5, but DATE requires a width between 8 and 40.
- 32 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DATE.
- | ^~~~~~~~"
-
-"get-data.sps:33.65-33.77: error: GET DATA: Output format DOLLAR1.2 specifies width 1, but DOLLAR requires a width between 2 and 40.
- 33 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 DOLLAR1.2.
- | ^~~~~~~~~~~~~"
-
-"get-data.sps:34.71: error: GET DATA: x is a duplicate variable name.
- 34 | GET DATA /TYPE=TXT/FILE='x.txt' /ARRANGEMENT=FIXED /VARIABLES x 1-5 F x 6-10 F.
- | ^"
-
-"get-data.sps:36.21-36.22: error: GET DATA: Syntax error expecting `/CONNECT='.
- 36 | GET DATA /TYPE=PSQL **.
- | ^~"
-
-"get-data.sps:37.20-37.23: error: GET DATA: Syntax error expecting `/CONNECT='.
- 37 | GET DATA /TYPE=PSQL/ **.
- | ^~~~"
-
-"get-data.sps:38.20-38.30: error: GET DATA: Syntax error expecting `/CONNECT='.
- 38 | GET DATA /TYPE=PSQL/CONNECT **.
- | ^~~~~~~~~~~"
-
-"get-data.sps:39.50-39.51: error: GET DATA: Syntax error expecting integer between 1 and 32767 for ASSUMEDSTRWIDTH.
- 39 | GET DATA /TYPE=PSQL/CONNECT='db'/ASSUMEDSTRWIDTH=**.
- | ^~"
-
-"get-data.sps:40.40-40.41: error: GET DATA: Syntax error expecting positive integer for BSIZE.
- 40 | GET DATA /TYPE=PSQL/CONNECT='db'/BSIZE=**.
- | ^~"
-
-"get-data.sps:41.38-41.39: error: GET DATA: Syntax error expecting string.
- 41 | GET DATA /TYPE=PSQL/CONNECT='db'/SQL=**.
- | ^~"
-
-"get-data.sps:43.20-43.21: error: GET DATA: Syntax error expecting `/FILE='.
- 43 | GET DATA /TYPE=GNM **.
- | ^~"
-
-"get-data.sps:44.19-44.22: error: GET DATA: Syntax error expecting `/FILE='.
- 44 | GET DATA /TYPE=GNM/ **.
- | ^~~~"
-
-"get-data.sps:45.19-45.26: error: GET DATA: Syntax error expecting `/FILE='.
- 45 | GET DATA /TYPE=GNM/FILE **.
- | ^~~~~~~~"
-
-"get-data.sps:46.26-46.27: error: GET DATA: Syntax error expecting string.
- 46 | GET DATA /TYPE=GNM/FILE= **.
- | ^~"
-
-"get-data.sps:47.54-47.55: error: GET DATA: Syntax error expecting integer between 1 and 32767 for ASSUMEDSTRWIDTH.
- 47 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/ASSUMEDSTRWIDTH=**.
- | ^~"
-
-"get-data.sps:48.49-48.50: error: GET DATA: Syntax error expecting string.
- 48 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=NAME **.
- | ^~"
-
-"get-data.sps:49.50-49.51: error: GET DATA: Syntax error expecting positive integer for INDEX.
- 49 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=INDEX **.
- | ^~"
-
-"get-data.sps:50.44-50.45: error: GET DATA: Syntax error expecting NAME or INDEX.
- 50 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/SHEET=**.
- | ^~"
-
-"get-data.sps:51.54-51.55: error: GET DATA: Syntax error expecting string.
- 51 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=RANGE **.
- | ^~"
-
-"get-data.sps:52.48-52.49: error: GET DATA: Syntax error expecting FULL or RANGE.
- 52 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/CELLRANGE=**.
- | ^~"
-
-"get-data.sps:53.48-53.49: error: GET DATA: Syntax error expecting ON or OFF.
- 53 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/READNAMES=**.
- | ^~"
-
-"get-data.sps:54.39-54.40: error: GET DATA: Syntax error expecting ASSUMEDSTRWIDTH, SHEET, CELLRANGE, or READNAMES.
- 54 | GET DATA /TYPE=GNM/FILE='x.gnumeric'/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([GET])
-
-dnl Tests for a bug which caused the second procedure
-dnl after GET to have corrupt input.
-AT_SETUP([GET data works in multiple procedures])
-AT_DATA([get.sps], [dnl
-DATA LIST LIST NOTABLE /LOCATION * EDITOR * SHELL * FREQ * .
-BEGIN DATA.
- 1.00 1.00 1.0 2.00
- 1.00 1.00 2.0 30.00
- 1.00 2.00 1.0 8.00
- 1.00 2.00 2.0 20.00
- 2.00 1.00 1.0 2.00
- 2.00 1.00 2.0 22.00
- 2.00 2.00 1.0 1.00
- 2.00 2.00 2.0 3.00
-END DATA.
-
-SAVE /OUTFILE='foo.sav'.
-
-GET /FILE='foo.sav'.
-
-* This one's ok
-LIST.
-
-* But this one get rubbish
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv get.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-LOCATION,EDITOR,SHELL,FREQ
-1.00,1.00,1.00,2.00
-1.00,1.00,2.00,30.00
-1.00,2.00,1.00,8.00
-1.00,2.00,2.00,20.00
-2.00,1.00,1.00,2.00
-2.00,1.00,2.00,22.00
-2.00,2.00,1.00,1.00
-2.00,2.00,2.00,3.00
-
-Table: Data List
-LOCATION,EDITOR,SHELL,FREQ
-1.00,1.00,1.00,2.00
-1.00,1.00,2.00,30.00
-1.00,2.00,1.00,8.00
-1.00,2.00,2.00,20.00
-2.00,1.00,1.00,2.00
-2.00,1.00,2.00,22.00
-2.00,2.00,1.00,1.00
-2.00,2.00,2.00,3.00
-])
-AT_CLEANUP
-
-dnl Tests for a bug that crashed when GET specified a nonexistent file.
-AT_SETUP([GET nonexistent file doesn't crash])
-dnl We use stdin here, because the bug seems to manifest itself only in
-dnl interactive mode.
-AT_CHECK([echo "GET /FILE='nonexistent.sav'." | pspp -O format=csv], [1], [dnl
-error: An error occurred while opening `nonexistent.sav': No such file or directory.
-])
-AT_CLEANUP
-
-dnl Tests for bug #15766 (/KEEP subcommand on SAVE doesn't
-dnl fully support ALL) and underlying problems.
-m4_define([GET_KEEP_ALL],
- [AT_SETUP([GET with /KEEP=ALL crashes -- $1])
- AT_DATA([get.sps], [dnl
-DATA LIST LIST NOTABLE
- /a b c d e f g h i j k l m n o p q r s t u v w x y z (F2.0).
-BEGIN DATA.
-1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
-END DATA.
-LIST.
-SAVE OUTFILE='test.sav'/$1.
-GET FILE='test.sav'/KEEP=x y z all.
-LIST.
-])
- AT_CHECK([pspp -o pspp.csv get.sps])
- AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z
-1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26
-
-Table: Data List
-x,y,z,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w
-24,25,26,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23
-])
- AT_CLEANUP])
-GET_KEEP_ALL([uncompressed])
-GET_KEEP_ALL([compressed])
-
-
-dnl Test for a crash when no /TYPE was provided
-AT_SETUP([GET data no type])
-AT_DATA([get.sps], [dnl
-get data /file='anything'.
-])
-
-AT_CHECK([pspp get.sps], [1], [ignore])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([INPUT PROGRAM])
-
-dnl Tests for a bug which caused a crash when
-dnl reading invalid INPUT PROGRAM syntax.
-AT_SETUP([INPUT PROGRAM invalid syntax crash])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
-DATA LIST NOTABLE /a 1-9.
-BEGIN DATA
-123456789
-END DATA.
-END INPUT PROGRAM.
-])
-AT_CHECK([pspp -O format=csv input-program.sps], [1], [dnl
-"input-program.sps:3.1-3.10: error: BEGIN DATA: BEGIN DATA is not allowed inside INPUT PROGRAM.
- 3 | BEGIN DATA
- | ^~~~~~~~~~"
-])
-AT_CLEANUP
-
-dnl Tests for bug #21108, a crash when
-dnl reading invalid INPUT PROGRAM syntax.
-AT_SETUP([INPUT PROGRAM invalid syntax crash])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
-DATA LIST LIST NOTABLE /x.
-END FILE.
-END INPUT PROGRAM.
-
-DESCRIPTIVES x.
-])
-AT_CHECK([pspp -O format=csv input-program.sps], [1], [dnl
-error: DESCRIPTIVES: At end of input: Syntax error expecting `BEGIN DATA'.
-])
-AT_CLEANUP
-
-dnl Tests for bug #38782, an infinite loop processing an empty input program.
-AT_SETUP([INPUT PROGRAM infinite loop])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
-STRING firstname lastname (a24) / address (a80).
-END INPUT PROGRAM.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv input-program.sps], [1], [dnl
-"input-program.sps:1.1-3.17: error: INPUT PROGRAM: Input program does not contain DATA LIST or END FILE.
- 1 | INPUT PROGRAM.
- 2 | STRING firstname lastname (a24) / address (a80).
- 3 | END INPUT PROGRAM."
-
-"input-program.sps:4.1-4.7: error: EXECUTE: EXECUTE is allowed only after the active dataset has been defined.
- 4 | EXECUTE.
- | ^~~~~~~"
-])
-AT_CLEANUP
-
-dnl Tests for bug #39097, a bug when an INPUT PROGRAM used VECTOR, was
-dnl followed immediately by a call to proc_execute() (here via DATASET
-dnl COPY), and then the input was actually used.
-AT_SETUP([INPUT PROGRAM with VECTOR and EXECUTE])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
-VECTOR vec(5).
-LOOP #c = 1 to 10.
- LOOP #v = 1 to 5.
- COMPUTE vec(#v) = #v.
- END LOOP.
- END CASE.
-END LOOP.
-END FILE.
-END INPUT PROGRAM.
-DATASET COPY x.
-LIST.
-])
-AT_CHECK([pspp -O format=csv input-program.sps], [0], [dnl
-Table: Data List
-vec1,vec2,vec3,vec4,vec5
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-1.00,2.00,3.00,4.00,5.00
-])
-AT_CLEANUP
-
-AT_SETUP([INPUT PROGRAM taking shorter of two files])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
- DATA LIST NOTABLE FILE='a.txt'/X 1-10.
- DATA LIST NOTABLE FILE='b.txt'/Y 1-10.
-END INPUT PROGRAM.
-LIST.
-])
-AT_DATA([short.txt], [dnl
-1
-2
-3
-])
-AT_DATA([long.txt], [dnl
-4
-5
-6
-7
-])
-
-cp short.txt a.txt
-cp long.txt b.txt
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X,Y
-1,4
-2,5
-3,6
-])
-
-cp short.txt b.txt
-cp long.txt a.txt
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X,Y
-4,1
-5,2
-6,3
-])
-AT_CLEANUP
-
-AT_SETUP([INPUT PROGRAM taking longer of two files])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
- NUMERIC #A #B.
-
- DO IF NOT #A.
- DATA LIST NOTABLE END=#A FILE='a.txt'/X 1-10.
- END IF.
- DO IF NOT #B.
- DATA LIST NOTABLE END=#B FILE='b.txt'/Y 1-10.
- END IF.
- DO IF #A AND #B.
- END FILE.
- END IF.
- END CASE.
-END INPUT PROGRAM.
-LIST.
-])
-AT_DATA([short.txt], [dnl
-1
-2
-3
-])
-AT_DATA([long.txt], [dnl
-4
-5
-6
-7
-8
-])
-
-cp short.txt a.txt
-cp long.txt b.txt
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X,Y
-1,4
-2,5
-3,6
-.,7
-.,8
-])
-
-cp short.txt b.txt
-cp long.txt a.txt
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X,Y
-4,1
-5,2
-6,3
-7,.
-8,.
-])
-AT_CLEANUP
-
-AT_SETUP([INPUT PROGRAM concatenating two files - version 1])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
- NUMERIC #A #B.
-
- DO IF #A.
- DATA LIST NOTABLE END=#B FILE='b.txt'/X 1-10.
- DO IF #B.
- END FILE.
- ELSE.
- END CASE.
- END IF.
- ELSE.
- DATA LIST NOTABLE END=#A FILE='a.txt'/X 1-10.
- DO IF NOT #A.
- END CASE.
- END IF.
- END IF.
-END INPUT PROGRAM.
-LIST.
-])
-AT_DATA([a.txt], [dnl
-1
-2
-3
-])
-AT_DATA([b.txt], [dnl
-4
-5
-6
-7
-8
-])
-
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X
-1
-2
-3
-4
-5
-6
-7
-8
-])
-AT_CLEANUP
-
-AT_SETUP([INPUT PROGRAM concatenating two files - version 2])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
- NUMERIC #EOF.
-
- LOOP IF NOT #EOF.
- DATA LIST NOTABLE END=#EOF FILE='a.txt'/X 1-10.
- DO IF NOT #EOF.
- END CASE.
- END IF.
- END LOOP.
-
- COMPUTE #EOF = 0.
- LOOP IF NOT #EOF.
- DATA LIST NOTABLE END=#EOF FILE='b.txt'/X 1-10.
- DO IF NOT #EOF.
- END CASE.
- END IF.
- END LOOP.
-
- END FILE.
-END INPUT PROGRAM.
-LIST.
-])
-AT_DATA([a.txt], [dnl
-1
-2
-3
-])
-AT_DATA([b.txt], [dnl
-4
-5
-6
-7
-8
-])
-
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X
-1
-2
-3
-4
-5
-6
-7
-8
-])
-AT_CLEANUP
-
-AT_SETUP([INPUT PROGRAM generating data])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
- LOOP #I=1 TO 10.
- COMPUTE X=#I.
- END CASE.
- END LOOP.
- END FILE.
-END INPUT PROGRAM.
-FORMAT X(F2).
-LIST.
-])
-AT_CHECK([pspp -O format=csv input-program.sps], 0, [dnl
-Table: Data List
-X
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-])
-AT_CLEANUP
-
-AT_SETUP([INPUT PROGRAM unexpected end of file])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
-])
-AT_CHECK([pspp input-program.sps], 1, [dnl
-error: INPUT PROGRAM: Unexpected end-of-file within INPUT PROGRAM.
-])
-AT_CLEANUP
-
-
-AT_SETUP([INPUT PROGRAM no variables])
-AT_DATA([input-program.sps], [dnl
-INPUT PROGRAM.
-END FILE.
-END INPUT PROGRAM.
-])
-AT_CHECK([pspp input-program.sps], 1, [dnl
-input-program.sps:1.1-3.17: error: INPUT PROGRAM: Input program did not create
-any variables.
- 1 | INPUT PROGRAM.
- 2 | END FILE.
- 3 | END INPUT PROGRAM.
-])
-AT_CLEANUP
-
-AT_SETUP([REREAD syntax errors])
-AT_DATA([reread.sps], [dnl
-INPUT PROGRAM.
-REREAD COLUMN=1 COLUMN=**.
-END INPUT PROGRAM.
-
-INPUT PROGRAM.
-REREAD COLUMN=**.
-END INPUT PROGRAM.
-
-INPUT PROGRAM.
-REREAD FILE=**.
-END INPUT PROGRAM.
-
-INPUT PROGRAM.
-REREAD ENCODING=**.
-END INPUT PROGRAM.
-
-INPUT PROGRAM.
-REREAD **.
-END INPUT PROGRAM.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='reread.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"reread.sps:2.17-2.22: error: REREAD: Subcommand COLUMN may only be specified once.
- 2 | REREAD COLUMN=1 COLUMN=**.
- | ^~~~~~"
-
-"reread.sps:6.15-6.16: error: REREAD: Syntax error parsing expression.
- 6 | REREAD COLUMN=**.
- | ^~"
-
-"reread.sps:10.13-10.14: error: REREAD: Syntax error expecting a file name or handle name.
- 10 | REREAD FILE=**.
- | ^~"
-
-"reread.sps:14.17-14.18: error: REREAD: Syntax error expecting string.
- 14 | REREAD ENCODING=**.
- | ^~"
-
-"reread.sps:18.8-18.9: error: REREAD: Syntax error expecting COLUMN, FILE, or ENCODING.
- 18 | REREAD **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([LIST])
-
-AT_SETUP([LIST plain cases])
-AT_DATA([data.txt], [dnl
- 18 1
- 19 7
- 20 26
- 21 76
- 22 57
- 23 58
- 24 38
- 25 38
- 26 30
- 27 21
- 28 23
-])
-AT_DATA([list.sps], [dnl
-DATA LIST FILE='data.txt'/avar 1-5 bvar 6-10.
-WEIGHT BY bvar.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading 1 record from `data.txt'.
-Variable,Record,Columns,Format
-avar,1,1-5,F5.0
-bvar,1,6-10,F5.0
-
-Table: Data List
-avar,bvar
-18,1
-19,7
-20,26
-21,76
-22,57
-23,58
-24,38
-25,38
-26,30
-27,21
-28,23
-])
-AT_CLEANUP
-
-AT_SETUP([LIST numbered cases])
-AT_DATA([data.txt], [dnl
- 18 1
- 19 7
- 20 26
- 21 76
- 22 57
- 23 58
- 24 38
- 25 38
- 26 30
- 27 21
- 28 23
-])
-AT_DATA([list.sps], [dnl
-DATA LIST FILE='data.txt'/avar 1-5 bvar 6-10.
-WEIGHT BY bvar.
-LIST/FORMAT NUMBERED.
-LIST/FORMAT NUMBERED/CASES FROM 2 TO 9 BY 2.
-])
-AT_CHECK([pspp -o pspp.csv list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading 1 record from `data.txt'.
-Variable,Record,Columns,Format
-avar,1,1-5,F5.0
-bvar,1,6-10,F5.0
-
-Table: Data List
-Case Number,avar,bvar
-1,18,1
-2,19,7
-3,20,26
-4,21,76
-5,22,57
-6,23,58
-7,24,38
-8,25,38
-9,26,30
-10,27,21
-11,28,23
-
-Table: Data List
-Case Number,avar,bvar
-2,19,7
-4,21,76
-6,23,58
-8,25,38
-])
-AT_CLEANUP
-
-# Checks for a crash when LIST did not include the variables from SPLIT
-# FILE in the same positions (bug #30684).
-AT_SETUP([LIST with split file])
-AT_DATA([data.txt], [dnl
-a 1
-a 2
-a 3
-b 1
-c 4
-c 5
-])
-AT_DATA([list.sps], [dnl
-DATA LIST LIST NOTABLE FILE='data.txt'/s (a1) n.
-SPLIT FILE BY s.
-LIST n.
-])
-AT_CHECK([pspp -o pspp.csv list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Split Values
-Variable,Value
-s,a
-
-Table: Data List
-n
-1.00
-2.00
-3.00
-
-Table: Split Values
-Variable,Value
-s,b
-
-Table: Data List
-n
-1.00
-
-Table: Split Values
-Variable,Value
-s,c
-
-Table: Data List
-n
-4.00
-5.00
-])
-AT_CLEANUP
-
-AT_SETUP([LIST lots of variables])
-AT_DATA([data.txt], [dnl
-767532466348513789073483106409
-888693089424177542378334186760
-492611507909187152726427852242
-819848892023195875879332001491
-452777898709563729845541516650
-239961967077732760663525115073
-])
-AT_DATA([list.sps], [dnl
-DATA LIST FILE='data.txt' NOTABLE/x01 to x30 1-30.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-x01,x02,x03,x04,x05,x06,x07,x08,x09,x10,x11,x12,x13,x14,x15,x16,x17,x18,x19,x20,x21,x22,x23,x24,x25,x26,x27,x28,x29,x30
-7,6,7,5,3,2,4,6,6,3,4,8,5,1,3,7,8,9,0,7,3,4,8,3,1,0,6,4,0,9
-8,8,8,6,9,3,0,8,9,4,2,4,1,7,7,5,4,2,3,7,8,3,3,4,1,8,6,7,6,0
-4,9,2,6,1,1,5,0,7,9,0,9,1,8,7,1,5,2,7,2,6,4,2,7,8,5,2,2,4,2
-8,1,9,8,4,8,8,9,2,0,2,3,1,9,5,8,7,5,8,7,9,3,3,2,0,0,1,4,9,1
-4,5,2,7,7,7,8,9,8,7,0,9,5,6,3,7,2,9,8,4,5,5,4,1,5,1,6,6,5,0
-2,3,9,9,6,1,9,6,7,0,7,7,7,3,2,7,6,0,6,6,3,5,2,5,1,1,5,0,7,3
-])
-AT_CLEANUP
-
-AT_SETUP([LIST selected cases])
-AT_DATA([data.txt], [dnl
-7675324663
-8886930894
-4926115079
-8198488920
-4527778987
-2399619670
-1667799691
-1623914684
-3681393233
-6418731145
-2284534083
-6617637452
-9865713582
-1163234537
-9981663637
-6821567746
-0952774952
-1641790193
-3763182871
-2046820753
-7970620091
-4841176017
-6949973797
-1396285996
-0700489524
-])
-AT_DATA([list.sps], [dnl
-DATA LIST FILE='data.txt' NOTABLE/x0 to x9 1-10.
-LIST /CASES=FROM 6 TO 20 BY 5.
-LIST /CASES=4.
-LIST /CASES=BY 10.
-LIST /CASES=FROM 25.
-LIST /CASES=FROM 26.
-])
-AT_CHECK([pspp -o pspp.csv list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
-2,3,9,9,6,1,9,6,7,0
-2,2,8,4,5,3,4,0,8,3
-6,8,2,1,5,6,7,7,4,6
-
-Table: Data List
-x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
-7,6,7,5,3,2,4,6,6,3
-8,8,8,6,9,3,0,8,9,4
-4,9,2,6,1,1,5,0,7,9
-8,1,9,8,4,8,8,9,2,0
-
-Table: Data List
-x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
-7,6,7,5,3,2,4,6,6,3
-2,2,8,4,5,3,4,0,8,3
-7,9,7,0,6,2,0,0,9,1
-
-Table: Data List
-x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
-0,7,0,0,4,8,9,5,2,4
-
-Table: Data List
-x0,x1,x2,x3,x4,x5,x6,x7,x8,x9
-])
-AT_CLEANUP
-
-dnl This program tests for a bug which caused a buffer overflow
-dnl when the list command attempted to write very long strings.
-AT_SETUP([LIST very long string])
-AT_DATA([list.sps], [dnl
-INPUT PROGRAM.
-STRING foo (a2000).
-+ COMPUTE foo=CONCAT(RPAD('A',1999, 'x'), 'Z').
-END CASE.
-END FILE.
-END INPUT PROGRAM.
-
-EXECUTE.
-
-DISPLAY VARIABLES.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Print Format,Write Format
-foo,1,A2000,A2000
-
-Table: Data List
-foo
-AxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxZ
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([LIST crash on invalid input])
-
-AT_DATA([list.sps], [dnl
-DATA LIST LIST /`$b.
-BEGIN DATA.
-1 3
-4 6
-7 9
-END DATA.
-
-LIST.
-])
-
-AT_CHECK([pspp -o pspp.csv list.sps], [1], [ignore])
-
-AT_CLEANUP
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([LIST tutorial example])
-AT_DATA([list.sps], [dnl
-data list list /forename (A12) height.
-begin data.
-Ahmed 188
-Bertram 167
-Catherine 134.231
-David 109.1
-end data
-
-list /format=numbered.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt list.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-forename,A12
-height,F8.0
-
-Table: Data List
-Case Number,forename,height
-1,Ahmed,188.00
-2,Bertram,167.00
-3,Catherine,134.23
-4,David,109.10
-])
-AT_CLEANUP
-
-AT_SETUP([LIST syntax errors])
-AT_DATA([list.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-LIST VARIABLES=**.
-LIST FORMAT=**.
-LIST CASES=FROM -1.
-LIST CASES=FROM 5 TO 4.
-LIST CASES=BY 0.
-LIST **.
-])
-AT_CHECK([pspp -O format=csv list.sps], [1], [dnl
-"list.sps:2.16-2.17: error: LIST: Syntax error expecting variable name.
- 2 | LIST VARIABLES=**.
- | ^~"
-
-"list.sps:3.13-3.14: error: LIST: Syntax error expecting NUMBERED or UNNUMBERED.
- 3 | LIST FORMAT=**.
- | ^~"
-
-"list.sps:4.17-4.18: error: LIST: Syntax error expecting positive integer for FROM.
- 4 | LIST CASES=FROM -1.
- | ^~"
-
-"list.sps:5.22: error: LIST: Syntax error expecting integer 5 or greater for TO.
- 5 | LIST CASES=FROM 5 TO 4.
- | ^"
-
-"list.sps:6.15: error: LIST: Syntax error expecting positive integer for TO.
- 6 | LIST CASES=BY 0.
- | ^"
-
-"list.sps:7.6-7.7: error: LIST: Syntax error expecting variable name.
- 7 | LIST **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([MATCH FILES])
-
-m4_define([PREPARE_MATCH_FILES],
- [AT_DATA([data1.txt], [dnl
-1aB
-8aM
-3aE
-5aG
-0aA
-5aH
-6aI
-7aJ
-2aD
-7aK
-1aC
-7aL
-4aF
-])
-
- AT_DATA([data2.txt], [dnl
-1bN
-3bO
-4bP
-6bQ
-7bR
-9bS
-])
-
- AT_DATA([prepare.sps], [dnl
-DATA LIST NOTABLE FILE='data1.txt' /a b c 1-3 (A).
-SAVE OUTFILE='data1.sav'.
-DATA LIST NOTABLE FILE='data2.txt' /a b c 1-3 (A).
-SAVE OUTFILE='data2.sav'.
-])
- AT_CHECK([pspp -O format=csv prepare.sps])
- AT_CHECK([test -f data1.sav && test -f data2.sav])])
-
-dnl CHECK_MATCH_FILES(TYPE2, SOURCE1, SOURCE2)
-dnl
-dnl Checks the MATCH FILES procedure with the specified combination of:
-dnl
-dnl - TYPE2: Either "file" or "table" for the type of matching used for
-dnl the second data source. (The first data source is always "file").
-dnl
-dnl - SOURCE1: Either "system" or "active" for the source of data for
-dnl the first data source.
-dnl
-dnl - SOURCE2: Either "system" or "active" for the source of data for
-dnl the second data source. (SOURCE1 and SOURCE2 may not both be
-dnl "active".)
-m4_define([CHECK_MATCH_FILES],
- [AT_SETUP([MATCH FILES -- $2 file and $3 $1])
- PREPARE_MATCH_FILES
- AT_DATA([expout],
- [m4_if([$1], [file], [dnl
-Table: Data List
-a,b,c,d,ina,inb,first,last
-0,a,A,,1,0,1,1
-1,a,B,N,1,1,1,0
-1,a,C,,1,0,0,1
-2,a,D,,1,0,1,1
-3,a,E,O,1,1,1,1
-4,a,F,P,1,1,1,1
-5,a,G,,1,0,1,0
-5,a,H,,1,0,0,1
-6,a,I,Q,1,1,1,1
-7,a,J,R,1,1,1,0
-7,a,K,,1,0,0,0
-7,a,L,,1,0,0,1
-8,a,M,,1,0,1,1
-9,b,,S,0,1,1,1
-], [dnl
-Table: Data List
-a,b,c,d,ina,inb,first,last
-0,a,A,,1,0,1,1
-1,a,B,N,1,1,1,0
-1,a,C,N,1,1,0,1
-2,a,D,,1,0,1,1
-3,a,E,O,1,1,1,1
-4,a,F,P,1,1,1,1
-5,a,G,,1,0,1,0
-5,a,H,,1,0,0,1
-6,a,I,Q,1,1,1,1
-7,a,J,R,1,1,1,0
-7,a,K,R,1,1,0,0
-7,a,L,R,1,1,0,1
-8,a,M,,1,0,1,1
-])])
-
- AT_DATA([match-files.sps], [dnl
-m4_if([$2], [active], [GET FILE='data1.sav'.],
- [$3], [active], [GET FILE='data2.sav'.],
- [])
-MATCH FILES
- FILE=m4_if([$2], [active], [*], ['data1.sav']) /IN=ina /SORT
- $1=m4_if([$3], [active], [*], ['data2.sav']) /in=inb /rename c=d
- /BY a /FIRST=first /LAST=last.
-LIST.
-])
- AT_CHECK([pspp -o pspp.csv match-files.sps])
- AT_CHECK([cat pspp.csv], [0], [expout])
- AT_CLEANUP])
-
-CHECK_MATCH_FILES([file], [system], [system])
-CHECK_MATCH_FILES([file], [system], [active])
-CHECK_MATCH_FILES([file], [active], [system])
-CHECK_MATCH_FILES([table], [system], [system])
-CHECK_MATCH_FILES([table], [system], [active])
-CHECK_MATCH_FILES([table], [active], [system])
-
-AT_SETUP([MATCH FILES parallel match])
-PREPARE_MATCH_FILES
-AT_DATA([match-files.sps], [dnl
-MATCH FILES FILE='data1.sav' /FILE='data2.sav' /RENAME (a b c=d e f).
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv match-files.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-a,b,c,d,e,f
-1,a,B,1,b,N
-8,a,M,3,b,O
-3,a,E,4,b,P
-5,a,G,6,b,Q
-0,a,A,7,b,R
-5,a,H,9,b,S
-6,a,I,,,
-7,a,J,,,
-2,a,D,,,
-7,a,K,,,
-1,a,C,,,
-7,a,L,,,
-4,a,F,,,
-])
-AT_CLEANUP
-
-dnl Test bug handling TABLE from active dataset found by John Darrington.
-AT_SETUP([MATCH FILES bug with TABLE from active dataset])
-AT_DATA([match-files.sps], [dnl
-DATA LIST LIST NOTABLE /x * y *.
-BEGIN DATA
-3 30
-2 21
-1 22
-END DATA.
-
-SAVE OUTFILE='bar.sav'.
-
-DATA LIST LIST NOTABLE /x * z *.
-BEGIN DATA
-3 8
-2 9
-END DATA.
-
-MATCH FILES TABLE=* /FILE='bar.sav' /BY=x.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv match-files.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-x,z,y
-3.00,8.00,30.00
-2.00,. ,21.00
-1.00,. ,22.00
-])
-AT_CLEANUP
-
-dnl Tests for a bug which caused MATCH FILES to crash
-dnl when used with scratch variables.
-AT_SETUP([MATCH FILES bug with scratch variables])
-AT_DATA([match-files.sps], [dnl
-DATA LIST LIST /w * x * y * .
-BEGIN DATA
-4 5 6
-1 2 3
-END DATA.
-
-COMPUTE j=0.
-LOOP #k = 1 to 10.
-COMPUTE j=#k + j.
-END LOOP.
-
-MATCH FILES FILE=* /DROP=w.
-LIST.
-FINISH.
-])
-AT_CHECK([pspp -o pspp.csv match-files.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-w,F8.0
-x,F8.0
-y,F8.0
-
-Table: Data List
-x,y,j
-5.00,6.00,55.00
-2.00,3.00,55.00
-])
-AT_CLEANUP
-
-dnl Tests for a bug that caused MATCH FILES to crash
-dnl with incompatible variables, especially but not
-dnl exclusively when one variable came from the active
-dnl file.
-AT_SETUP([MATCH FILES with incompatible variable types])
-AT_DATA([match-files.sps], [dnl
-DATA LIST LIST NOTABLE/name (A6) x.
-BEGIN DATA.
-al,7
-brad,8
-carl,9
-END DATA.
-SAVE OUTFILE='x.sav'.
-
-DATA LIST LIST NOTABLE/name (A7) y.
-BEGIN DATA.
-al,1
-carl,2
-dan,3
-END DATA.
-MATCH FILES/FILE='x.sav'/FILE=*/BY name.
-LIST.
-])
-AT_CHECK([pspp -O format=csv match-files.sps], [1], [dnl
-match-files.sps:15: error: MATCH FILES: Variable name has different type or width in different files.
-
-"match-files.sps:15.13-15.24: note: MATCH FILES: In file `x.sav', name is a string with width 6.
- 15 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
- | ^~~~~~~~~~~~"
-
-"match-files.sps:15.26-15.31: note: MATCH FILES: In file *, name is a string with width 7.
- 15 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
- | ^~~~~~"
-
-match-files.sps:16: error: Stopping syntax file processing here to avoid a cascade of dependent command failures.
-])
-AT_CLEANUP
-
-AT_SETUP([MATCH FILES syntax errors])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='match-files.sps' ERROR=IGNORE.
-])
-AT_DATA([match-files.sps], [dnl
-MATCH FILES/FILE=*.
-
-DATA LIST LIST NOTABLE/name (A6) x.
-BEGIN DATA.
-al,7
-brad,8
-carl,9
-END DATA.
-SAVE OUTFILE='x.sav'.
-
-TEMPORARY.
-MATCH FILES/FILE=*.
-
-DATA LIST LIST NOTABLE/name (A7) y.
-BEGIN DATA.
-al,1
-carl,2
-dan,3
-END DATA.
-MATCH FILES/FILE='x.sav'/FILE=*/BY name.
-MATCH FILES/FILE='x.sav'/IN=**.
-MATCH FILES/FILE='x.sav'/IN=x/IN=y.
-MATCH FILES/FILE='x.sav'/BY=x/BY=y.
-MATCH FILES/FILE='x.sav'/BY=**.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY y.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY x.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST x/FIRST y.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=**.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST x/LAST y.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=**.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=xyzzy.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=ALL.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/KEEP=xyzzy.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x **.
-MATCH FILES/FILE='x.sav'/TABLE=*/RENAME(name=name2).
-MATCH FILES/FILE='x.sav'/SORT/FILE=*/RENAME(name=name2)/SORT.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=x.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x.
-MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/IN=x.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"match-files.sps:1.18: error: MATCH FILES: Cannot specify the active dataset since none has been defined.
- 1 | MATCH FILES/FILE=*.
- | ^"
-
-"match-files.sps:12.18: error: MATCH FILES: This command may not be used after TEMPORARY when the active dataset is an input source. Temporary transformations will be made permanent.
- 12 | MATCH FILES/FILE=*.
- | ^"
-
-match-files.sps:20: error: MATCH FILES: Variable name has different type or width in different files.
-
-"match-files.sps:20.13-20.24: note: MATCH FILES: In file `x.sav', name is a string with width 6.
- 20 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
- | ^~~~~~~~~~~~"
-
-"match-files.sps:20.26-20.31: note: MATCH FILES: In file *, name is a string with width 7.
- 20 | MATCH FILES/FILE='x.sav'/FILE=*/BY name.
- | ^~~~~~"
-
-"match-files.sps:21.29-21.30: error: MATCH FILES: Syntax error expecting identifier.
- 21 | MATCH FILES/FILE='x.sav'/IN=**.
- | ^~"
-
-"match-files.sps:22.34: error: MATCH FILES: Multiple IN subcommands for a single FILE or TABLE.
- 22 | MATCH FILES/FILE='x.sav'/IN=x/IN=y.
- | ^"
-
-"match-files.sps:23.31-23.32: error: MATCH FILES: Subcommand BY may only be specified once.
- 23 | MATCH FILES/FILE='x.sav'/BY=x/BY=y.
- | ^~"
-
-"match-files.sps:24.29-24.30: error: MATCH FILES: Syntax error expecting variable name.
- 24 | MATCH FILES/FILE='x.sav'/BY=**.
- | ^~"
-
-"match-files.sps:25.13-25.24: error: MATCH FILES: File `x.sav' lacks BY variable y.
- 25 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY y.
- | ^~~~~~~~~~~~"
-
-"match-files.sps:26.26-26.31: error: MATCH FILES: File * lacks BY variable x.
- 26 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/BY x.
- | ^~~~~~"
-
-"match-files.sps:27.60-27.64: error: MATCH FILES: Subcommand FIRST may only be specified once.
- 27 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST x/FIRST y.
- | ^~~~~"
-
-"match-files.sps:28.58-28.59: error: MATCH FILES: Syntax error expecting identifier.
- 28 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=**.
- | ^~"
-
-"match-files.sps:29.59-29.62: error: MATCH FILES: Subcommand LAST may only be specified once.
- 29 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST x/LAST y.
- | ^~~~"
-
-"match-files.sps:30.57-30.58: error: MATCH FILES: Syntax error expecting identifier.
- 30 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=**.
- | ^~"
-
-"match-files.sps:31.57-31.61: error: MATCH FILES: xyzzy is not a variable name.
- 31 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=xyzzy.
- | ^~~~~"
-
-"match-files.sps:32.52-32.59: error: MATCH FILES: Cannot DROP all variables from dictionary.
- 32 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/DROP=ALL.
- | ^~~~~~~~"
-
-"match-files.sps:33.57-33.61: error: MATCH FILES: xyzzy is not a variable name.
- 33 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/KEEP=xyzzy.
- | ^~~~~"
-
-"match-files.sps:34.59-34.60: error: MATCH FILES: Syntax error expecting end of command.
- 34 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x **.
- | ^~"
-
-"match-files.sps:35.26-35.32: error: MATCH FILES: BY is required when TABLE is specified.
- 35 | MATCH FILES/FILE='x.sav'/TABLE=*/RENAME(name=name2).
- | ^~~~~~~"
-
-"match-files.sps:36.26-36.29: error: MATCH FILES: BY is required when SORT is specified.
- 36 | MATCH FILES/FILE='x.sav'/SORT/FILE=*/RENAME(name=name2)/SORT.
- | ^~~~"
-
-"match-files.sps:37.58: error: MATCH FILES: Variable name x specified on FIRST subcommand duplicates an existing variable name.
- 37 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/FIRST=x.
- | ^"
-
-"match-files.sps:38.57: error: MATCH FILES: Variable name x specified on LAST subcommand duplicates an existing variable name.
- 38 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/LAST=x.
- | ^"
-
-"match-files.sps:39.55: error: MATCH FILES: Variable name x specified on IN subcommand duplicates an existing variable name.
- 39 | MATCH FILES/FILE='x.sav'/FILE=*/RENAME(name=name2)/IN=x.
- | ^"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017, 2020 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([MATRIX DATA])
-
-dnl Keep this test in sync with Example 1 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - LOWER DIAGONAL with ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ var01 TO var08
- /FILE='matrix-data.txt'.
-FORMATS var01 TO var08(F5.2).
-LIST.
-])
-AT_DATA([matrix-data.txt], [dnl
-MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
-SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
-N 92 92 92 92 92 92 92 92
-'CORR' 1.00
-CORR .18 1.00
-CORR -.22 -.17 1.00
-"CORR" .36 .31 -.14 1.00
-COR .27 .16 -.12 .22 1.00
-CORR .33 .15 -.17 .24 .21 1.00
-CORR .50 .29 -.20 .32 .12 .38 1.00
-CORR .17 .29 -.05 .20 .27 .20 .04 1.00
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
-MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
-STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
-N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
-CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
-CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
-CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
-CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
-CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
-CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
-CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
-CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - UPPER DIAGONAL with ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var02 var03 var04
- /format = upper diagonal.
-
-begin data
-mean 34 35 36 37
-sd 22 11 55 66
-n_ve 100 101 102 103
-corr 1 9 8 7
-corr 1 6 5
-corr 1 4
-corr 1
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-MEAN,,34.0000,35.0000,36.0000,37.0000
-STDDEV,,22.0000,11.0000,55.0000,66.0000
-N,,100.0000,101.0000,102.0000,103.0000
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var02,9.0000,1.0000,6.0000,5.0000
-CORR,var03,8.0000,6.0000,1.0000,4.0000
-CORR,var04,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - FULL with ROWTYPE_])
-dnl Just for fun, this one is in a different case.
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = ROWTYPE_ var01 var02 var03 var04
- /format = full diagonal.
-
-begin data
-MEAN 34 35 36 37
-SD 22 11 55 66
-N 100 101 102 103
-CORR 1 9 8 7
-CORR 9 1 6 5
-CORR 8 6 1 4
-CORR 7 5 4 1
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-MEAN,,34.0000,35.0000,36.0000,37.0000
-STDDEV,,22.0000,11.0000,55.0000,66.0000
-N,,100.0000,101.0000,102.0000,103.0000
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var02,9.0000,1.0000,6.0000,5.0000
-CORR,var03,8.0000,6.0000,1.0000,4.0000
-CORR,var04,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-
-AT_SETUP([MATRIX DATA - UPPER NODIAGONAL with ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var02 var03 var04
- /format = upper nodiagonal.
-
-begin data
-mean 34 35 36 37
-sd 22 11 55 66
-n 100 101 102 103
-corr 9 8 7
-corr 6 5
-corr 4
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-MEAN,,34.0000,35.0000,36.0000,37.0000
-STDDEV,,22.0000,11.0000,55.0000,66.0000
-N,,100.0000,101.0000,102.0000,103.0000
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var02,9.0000,1.0000,6.0000,5.0000
-CORR,var03,8.0000,6.0000,1.0000,4.0000
-CORR,var04,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 2 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - UPPER NODIAGONAL with ROWTYPE_ - 2])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ var01 TO var08
- /FORMAT=UPPER NODIAGONAL.
-BEGIN DATA.
-MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
-SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
-N 92 92 92 92 92 92 92 92
-CORR .17 .50 -.33 .27 .36 -.22 .18
-CORR .29 .29 -.20 .32 .12 .38
-CORR .05 .20 -.15 .16 .21
-CORR .20 .32 -.17 .12
-CORR .27 .12 -.24
-CORR -.20 -.38
-CORR .04
-END DATA.
-FORMATS var01 TO var08(F6.2).
-LIST.
-])
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
-MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
-STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
-N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
-CORR,var01,1.00,.17,.50,-.33,.27,.36,-.22,.18
-CORR,var02,.17,1.00,.29,.29,-.20,.32,.12,.38
-CORR,var03,.50,.29,1.00,.05,.20,-.15,.16,.21
-CORR,var04,-.33,.29,.05,1.00,.20,.32,-.17,.12
-CORR,var05,.27,-.20,.20,.20,1.00,.27,.12,-.24
-CORR,var06,.36,.32,-.15,.32,.27,1.00,-.20,-.38
-CORR,var07,-.22,.12,.16,-.17,.12,-.20,1.00,.04
-CORR,var08,.18,.38,.21,.12,-.24,-.38,.04,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - LOWER NODIAGONAL with ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var02 var03 var04
- /format = lower nodiagonal
- /cells = 2.
-
-begin data
-mean 34 35 36 37
-sd 22 11 55 66
-n 100 101 102 103
-corr 9
-corr 8 6
-corr 7 5 4
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-"matrix-data.sps:4.6-4.10: warning: MATRIX DATA: CELLS is ignored when VARIABLES includes ROWTYPE_.
- 4 | /cells = 2.
- | ^~~~~"
-
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-MEAN,,34.0000,35.0000,36.0000,37.0000
-STDDEV,,22.0000,11.0000,55.0000,66.0000
-N,,100.0000,101.0000,102.0000,103.0000
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var02,9.0000,1.0000,6.0000,5.0000
-CORR,var03,8.0000,6.0000,1.0000,4.0000
-CORR,var04,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - split data])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = s1 s2 rowtype_ var01 var02 var03
- /split=s1 s2.
-
-begin data
-8 0 mean 21.4 5.0 72.9
-8 0 sd 6.5 1.6 22.8
-8 0 n 106 106 106
-8 0 corr 1
-8 0 corr .41 1
-8 0 corr -.16 -.22 1
-8 1 mean 11.4 1.0 52.9
-8 1 sd 9.5 8.6 12.8
-8 1 n 10 11 12
-8 1 corr 1
-8 1 corr .51 1
-8 1 corr .36 -.41 1
-end data.
-
-display dictionary.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-s1,1,Nominal,Input,8,Right,F4.0,F4.0
-s2,2,Nominal,Input,8,Right,F4.0,F4.0
-ROWTYPE_,3,Nominal,Input,8,Left,A8,A8
-VARNAME_,4,Nominal,Input,8,Left,A8,A8
-var01,5,Scale,Input,8,Right,F10.4,F10.4
-var02,6,Scale,Input,8,Right,F10.4,F10.4
-var03,7,Scale,Input,8,Right,F10.4,F10.4
-
-Table: Split Values
-Variable,Value
-s1,8
-s2,0
-
-Table: Data List
-s1,s2,ROWTYPE_,VARNAME_,var01,var02,var03
-8,0,MEAN,,21.4000,5.0000,72.9000
-8,0,STDDEV,,6.5000,1.6000,22.8000
-8,0,N,,106.0000,106.0000,106.0000
-8,0,CORR,var01,1.0000,.4100,-.1600
-8,0,CORR,var02,.4100,1.0000,-.2200
-8,0,CORR,var03,-.1600,-.2200,1.0000
-
-Table: Split Values
-Variable,Value
-s1,8
-s2,1
-
-Table: Data List
-s1,s2,ROWTYPE_,VARNAME_,var01,var02,var03
-8,1,MEAN,,11.4000,1.0000,52.9000
-8,1,STDDEV,,9.5000,8.6000,12.8000
-8,1,N,,10.0000,11.0000,12.0000
-8,1,CORR,var01,1.0000,.5100,.3600
-8,1,CORR,var02,.5100,1.0000,-.4100
-8,1,CORR,var03,.3600,-.4100,1.0000
-])
-
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 4 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - split data - 2])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=s1 ROWTYPE_ var01 TO var04
- /SPLIT=s1
- /FORMAT=FULL.
-BEGIN DATA.
-0 MEAN 34 35 36 37
-0 SD 22 11 55 66
-0 N 99 98 99 92
-0 CORR 1 .9 .8 .7
-0 CORR .9 1 .6 .5
-0 CORR .8 .6 1 .4
-0 CORR .7 .5 .4 1
-1 MEAN 44 45 34 39
-1 SD 23 15 51 46
-1 N 98 34 87 23
-1 CORR 1 .2 .3 .4
-1 CORR .2 1 .5 .6
-1 CORR .3 .5 1 .7
-1 CORR .4 .6 .7 1
-END DATA.
-FORMATS var01 TO var04(F5.1).
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Split Values
-Variable,Value
-s1,0
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-0,MEAN,,34.0,35.0,36.0,37.0
-0,STDDEV,,22.0,11.0,55.0,66.0
-0,N,,99.0,98.0,99.0,92.0
-0,CORR,var01,1.0,.9,.8,.7
-0,CORR,var02,.9,1.0,.6,.5
-0,CORR,var03,.8,.6,1.0,.4
-0,CORR,var04,.7,.5,.4,1.0
-
-Table: Split Values
-Variable,Value
-s1,1
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-1,MEAN,,44.0,45.0,34.0,39.0
-1,STDDEV,,23.0,15.0,51.0,46.0
-1,N,,98.0,34.0,87.0,23.0
-1,CORR,var01,1.0,.2,.3,.4
-1,CORR,var02,.2,1.0,.5,.6
-1,CORR,var03,.3,.5,1.0,.7
-1,CORR,var04,.4,.6,.7,1.0
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 5 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - factor variables])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ f1 var01 TO var04
- /FACTOR=f1.
-BEGIN DATA.
-MEAN 0 34 35 36 37
-SD 0 22 11 55 66
-N 0 99 98 99 92
-MEAN 1 44 45 34 39
-SD 1 23 15 51 46
-N 1 98 34 87 23
-CORR . 1
-CORR . .9 1
-CORR . .8 .6 1
-CORR . .7 .5 .4 1
-END DATA.
-FORMATS var01 TO var04(F5.1).
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
-MEAN,0,,34.0,35.0,36.0,37.0
-STDDEV,0,,22.0,11.0,55.0,66.0
-N,0,,99.0,98.0,99.0,92.0
-MEAN,1,,44.0,45.0,34.0,39.0
-STDDEV,1,,23.0,15.0,51.0,46.0
-N,1,,98.0,34.0,87.0,23.0
-CORR,.,var01,1.0,.9,.8,.7
-CORR,.,var02,.9,1.0,.6,.5
-CORR,.,var03,.8,.6,1.0,.4
-CORR,.,var04,.7,.5,.4,1.0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - factors and splits])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = s f rowtype_ var01 var02 var03
- /split=s
- /factor=f.
-
-begin data
-8 0 mean 21.4 5.0 72.9
-8 0 sd 6.5 1.6 22.8
-8 0 n 106 106 106
-8 . corr 1
-8 . corr .41 1
-8 . corr -.16 -.22 1
-9 1 mean 11.4 1.0 52.9
-9 1 sd 9.5 8.6 12.8
-9 1 n 10 11 12
-9 . corr 1
-9 . corr .51 1
-9 . corr .36 -.41 1
-end data.
-
-display dictionary.
-
-list.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-s,1,Nominal,Input,8,Right,F4.0,F4.0
-ROWTYPE_,2,Nominal,Input,8,Left,A8,A8
-f,3,Nominal,Input,8,Right,F4.0,F4.0
-VARNAME_,4,Nominal,Input,8,Left,A8,A8
-var01,5,Scale,Input,8,Right,F10.4,F10.4
-var02,6,Scale,Input,8,Right,F10.4,F10.4
-var03,7,Scale,Input,8,Right,F10.4,F10.4
-
-Table: Split Values
-Variable,Value
-s,8
-
-Table: Data List
-s,ROWTYPE_,f,VARNAME_,var01,var02,var03
-8,MEAN,0,,21.4000,5.0000,72.9000
-8,STDDEV,0,,6.5000,1.6000,22.8000
-8,N,0,,106.0000,106.0000,106.0000
-8,CORR,.,var01,1.0000,.4100,-.1600
-8,CORR,.,var02,.4100,1.0000,-.2200
-8,CORR,.,var03,-.1600,-.2200,1.0000
-
-Table: Split Values
-Variable,Value
-s,9
-
-Table: Data List
-s,ROWTYPE_,f,VARNAME_,var01,var02,var03
-9,MEAN,1,,11.4000,1.0000,52.9000
-9,STDDEV,1,,9.5000,8.6000,12.8000
-9,N,1,,10.0000,11.0000,12.0000
-9,CORR,.,var01,1.0000,.5100,.3600
-9,CORR,.,var02,.5100,1.0000,-.4100
-9,CORR,.,var03,.3600,-.4100,1.0000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - bad ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var02 var03 var04
- /format = upper diagonal.
-
-begin data
-cork 1 9 8 7
-corr 1 6 5
-corr 1 4
-corr 1
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
-"matrix-data.sps:6.1-6.4: error: Unknown row type ""cork""."
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - unexpected ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ f1 var01 var02 var03 var04
- /content = corr (sd)
- /factor = f1
- /format = upper diagonal.
-
-begin data
-corr . 1 9 8 7
-corr . 1 6 5
-corr . 1 4
-corr . 1
-sd . 1 2 3 4
-
-corr 0 1 9 8 7
-corr 0 1 6 5
-corr 0 1 4
-corr 0 1
-sd 0 1 2 3 4
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-matrix-data.sps:12: warning: Data contains pooled row type STDDEV not included in CONTENTS.
-
-matrix-data.sps:14: warning: Data contains with-factors row type CORR not included in CONTENTS.
-
-Table: Data List
-ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
-CORR,.,var01,1.0000,9.0000,8.0000,7.0000
-CORR,.,var02,9.0000,1.0000,6.0000,5.0000
-CORR,.,var03,8.0000,6.0000,1.0000,4.0000
-CORR,.,var04,7.0000,5.0000,4.0000,1.0000
-STDDEV,.,,1.0000,2.0000,3.0000,4.0000
-CORR,0,var01,1.0000,9.0000,8.0000,7.0000
-CORR,0,var02,9.0000,1.0000,6.0000,5.0000
-CORR,0,var03,8.0000,6.0000,1.0000,4.0000
-CORR,0,var04,7.0000,5.0000,4.0000,1.0000
-STDDEV,0,,1.0000,2.0000,3.0000,4.0000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - bad number])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var02 var03 var04
- /format = upper diagonal.
-
-begin data
-corr 1 9 8 7
-corr 1 x 5
-corr 1 4
-corr 1
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
-matrix-data.sps:7.15: error: Field contents are not numeric.
-
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var02,9.0000,1.0000,. ,5.0000
-CORR,var03,8.0000,. ,1.0000,4.0000
-CORR,var04,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - long variable names])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var_two variable_number_three variableFour
- /format = upper diagonal.
-
-begin data
-mean 34 35 36 37
-sd 22 11 55 66
-n_vector 100 101 102 103
-corr 1 9 8 7
-corr 1 6 5
-corr 1 4
-corr 1
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var_two,variable_number_three,variableFour
-MEAN,,34.0000,35.0000,36.0000,37.0000
-STDDEV,,22.0000,11.0000,55.0000,66.0000
-N,,100.0000,101.0000,102.0000,103.0000
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var_two,9.0000,1.0000,6.0000,5.0000
-CORR,variable_number_three,8.0000,6.0000,1.0000,4.0000
-CORR,variableFour,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - read integrity])
-dnl Check that matrices presented are read correctly.
-dnl The example below is an unlikely one since all
-dnl covariance/correlation matrices must be symmetrical
-dnl but it serves a purpose for this test.
-AT_DATA([matrix-reader.sps], [dnl
-matrix data
- variables = rowtype_ var01 to var9
- /format = full.
-
-begin data
-n 1 2 3 4 5 6 7 8 9
-sd 100 200 300 400 500 600 700 800 900
-corr 11 12 13 14 15 16 17 18 19
-corr 21 22 23 24 25 26 27 28 29
-corr 31 32 33 34 35 36 37 38 39
-corr 41 42 43 44 45 46 47 48 49
-corr 51 52 53 54 55 56 57 58 59
-corr 61 62 63 64 65 66 67 68 69
-corr 71 72 73 74 75 76 77 78 79
-corr 81 82 83 84 85 86 87 88 89
-corr 91 92 93 94 95 96 97 98 99
-end data.
-DEBUG MATRIX READ.
-FORMATS var01 to var09(F3.0).
-list.
-factor /matrix = in (corr = *)
- /analysis var02 var04 var06
- /method = correlation
- /rotation = norotate
- /print correlation.
-])
-
-AT_CHECK([pspp --testing-mode -O format=csv matrix-reader.sps], [0], [dnl
-Table: Debug Matrix Reader
-,,,var01,var02,var03,var04,var05,var06,var07,var08,var09
-1,Correlation,var01,11.000,12.000,13.000,14.000,15.000,16.000,17.000,18.000,19.000
-,,var02,21.000,22.000,23.000,24.000,25.000,26.000,27.000,28.000,29.000
-,,var03,31.000,32.000,33.000,34.000,35.000,36.000,37.000,38.000,39.000
-,,var04,41.000,42.000,43.000,44.000,45.000,46.000,47.000,48.000,49.000
-,,var05,51.000,52.000,53.000,54.000,55.000,56.000,57.000,58.000,59.000
-,,var06,61.000,62.000,63.000,64.000,65.000,66.000,67.000,68.000,69.000
-,,var07,71.000,72.000,73.000,74.000,75.000,76.000,77.000,78.000,79.000
-,,var08,81.000,82.000,83.000,84.000,85.000,86.000,87.000,88.000,89.000
-,,var09,91.000,92.000,93.000,94.000,95.000,96.000,97.000,98.000,99.000
-,N,Value,1.000,2.000,3.000,4.000,5.000,6.000,7.000,8.000,9.000
-,Standard Deviation,Value,100.000,200.000,300.000,400.000,500.000,600.000,700.000,800.000,900.000
-
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08,var09
-N,,1,2,3,4,5,6,7,8,9
-STDDEV,,100,200,300,400,500,600,700,800,900
-CORR,var01,11,12,13,14,15,16,17,18,19
-CORR,var02,21,22,23,24,25,26,27,28,29
-CORR,var03,31,32,33,34,35,36,37,38,39
-CORR,var04,41,42,43,44,45,46,47,48,49
-CORR,var05,51,52,53,54,55,56,57,58,59
-CORR,var06,61,62,63,64,65,66,67,68,69
-CORR,var07,71,72,73,74,75,76,77,78,79
-CORR,var08,81,82,83,84,85,86,87,88,89
-CORR,var09,91,92,93,94,95,96,97,98,99
-
-Table: Correlation Matrix
-,,var02,var04,var06
-Correlation,var02,22.000,24.000,26.000
-,var04,42.000,44.000,46.000
-,var06,62.000,64.000,66.000
-
-Table: Component Matrix
-,Component,
-,1,2
-var02,6.73,-2.23
-var04,6.95,2.15
-var06,9.22,.01
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - too many rows])
-dnl Test for a crash which occurred when the matrix had more rows declared
-dnl than variables to hold them.
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_
- var01 var02 var03 var04
- / format = upper diagonal .
-begin data
- mean 21.4 5.0 72.9 17.4
- sd 6.5 1.6 22.8 5.7
- n 106 106 106 106
- corr 1.00 .32 .48 .28
- corr 1.00 .72 .54 .44
- corr 1.00 .50 .59 .64
- corr 1.00 .62 .49 -.30
- corr 1.00 .56 -.38 .52
- corr 1.00 -.73 .91 .80
- corr 1.00 -.65 -.60
- corr 1.00 .70
- corr 1.00
-end data .
-FORMATS var01 TO var04 (F6.2).
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
-matrix-data.sps:10.29-10.31: error: Extraneous data expecting end of line.
-
-matrix-data.sps:11.24-11.31: error: Extraneous data expecting end of line.
-
-matrix-data.sps:12.19-12.32: error: Extraneous data expecting end of line.
-
-matrix-data.sps:18: error: Matrix CORR had 9 rows but 4 rows were expected.
-
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-MEAN,,21.40,5.00,72.90,17.40
-STDDEV,,6.50,1.60,22.80,5.70
-N,,106.00,106.00,106.00,106.00
-CORR,var01,1.00,.32,.48,.28
-CORR,var02,.32,1.00,.72,.54
-CORR,var03,.48,.72,1.00,.50
-CORR,var04,.28,.54,.50,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - too few rows])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ s1 var01 var02 var03 var04
- /split s1
- /format = upper diagonal
- /file='matrix-data.txt'.
-FORMATS var01 TO var04 (F6.2).
-LIST.
-])
-AT_DATA([matrix-data.txt], [dnl
-mean 1 21.4 5.0 72.9 17.4
-sd 1 6.5 1.6 22.8 5.7
-n 1 106 106 106 106
-corr 1 1.00 .32 .48 .28
-corr 2 1.00 .32 .48 .28
-corr 2 2.00 .72 .54
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
-matrix-data.txt:5: error: Matrix CORR had 1 rows but 4 rows were expected.
-
-matrix-data.txt:6: error: Matrix CORR had 2 rows but 4 rows were expected.
-
-Table: Split Values
-Variable,Value
-s1,1
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-1,MEAN,,21.40,5.00,72.90,17.40
-1,STDDEV,,6.50,1.60,22.80,5.70
-1,N,,106.00,106.00,106.00,106.00
-1,CORR,var01,1.00,.32,.48,.28
-1,CORR,var02,.32,1.00,. ,. @&t@
-1,CORR,var03,.48,. ,1.00,. @&t@
-1,CORR,var04,.28,. ,. ,1.00
-
-Table: Split Values
-Variable,Value
-s1,2
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-2,CORR,var01,1.00,.32,.48,.28
-2,CORR,var02,.32,2.00,.72,.54
-2,CORR,var03,.48,.72,1.00,. @&t@
-2,CORR,var04,.28,.54,. ,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - badly formed])
-AT_DATA([data.sps], [dnl
-data list list NOTABLE /ROWTYPE_ (a8) VARNAME_(a4) v1 v2 v3 v4xxxxxxxxxxxxxxxxxxxxxzzzzzzzzzzzzzxxxxxxxxx.
-begin data
-mean "" 1 2 3 4
-sd "" 5 6 7 8
-n "" 2 3 4 5
-corr v1 11 22 33 44
-corr v2 55 66 77 88
-corr v3 111 222 333 444
-corr v4 4 3 21 1
-end data.
-
-DEBUG MATRIX READ.
-])
-
-AT_CHECK([pspp --testing-mode -O format=csv data.sps], [0], [dnl
-data.sps:12: warning: DEBUG MATRIX READ: CORR matrix has 4 columns but 3 rows named variables to be analyzed (and 1 rows named unknown variables).
-
-Table: Debug Matrix Reader
-,,,v1,v2,v3,v4xxxxxxxxxxxxxxxxxxxxxzzzzzzzzzzzzzxxxxxxxxx
-1,Correlation,v1,11.000,22.000,33.000,44.000
-,,v2,55.000,66.000,77.000,88.000
-,,v3,111.000,222.000,333.000,444.000
-,,v4xxxxxxxxxxxxxxxxxxxxxzzzzzzzzzzzzzxxxxxxxxx,. ,. ,. ,. @&t@
-,N,Value,2.000,3.000,4.000,5.000
-,Mean,Value,1.000,2.000,3.000,4.000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - N subcommand])
-AT_DATA([matrix-data.sps], [dnl
-matrix data
- variables = rowtype_ var01 var02 var03 var04
- /n = 99
- /format = upper nodiagonal.
-begin data
-mean 34 35 36 37
-sd 22 11 55 66
-n_vector 1 2 3 4
-corr 9 8 7
-corr 6 5
-corr 4
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [dnl
-matrix-data.sps:8: error: N record is not allowed with N subcommand. Ignoring N record.
-
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04
-N,,99.0000,99.0000,99.0000,99.0000
-MEAN,,34.0000,35.0000,36.0000,37.0000
-STDDEV,,22.0000,11.0000,55.0000,66.0000
-CORR,var01,1.0000,9.0000,8.0000,7.0000
-CORR,var02,9.0000,1.0000,6.0000,5.0000
-CORR,var03,8.0000,6.0000,1.0000,4.0000
-CORR,var04,7.0000,5.0000,4.0000,1.0000
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 3 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - N subcommand - 2])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ var01 TO var08
- /FORMAT=UPPER NODIAGONAL
- /N 92.
-BEGIN DATA.
-MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
-SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
-CORR .17 .50 -.33 .27 .36 -.22 .18
-CORR .29 .29 -.20 .32 .12 .38
-CORR .05 .20 -.15 .16 .21
-CORR .20 .32 -.17 .12
-CORR .27 .12 -.24
-CORR -.20 -.38
-CORR .04
-END DATA.
-FORMATS var01 TO var08(F6.2).
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
-N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
-MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
-STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
-CORR,var01,1.00,.17,.50,-.33,.27,.36,-.22,.18
-CORR,var02,.17,1.00,.29,.29,-.20,.32,.12,.38
-CORR,var03,.50,.29,1.00,.05,.20,-.15,.16,.21
-CORR,var04,-.33,.29,.05,1.00,.20,.32,-.17,.12
-CORR,var05,.27,-.20,.20,.20,1.00,.27,.12,-.24
-CORR,var06,.36,.32,-.15,.32,.27,1.00,-.20,-.38
-CORR,var07,-.22,.12,.16,-.17,.12,-.20,1.00,.04
-CORR,var08,.18,.38,.21,.12,-.24,-.38,.04,1.00
-])
-AT_CLEANUP
-
-dnl A "no-crash" test. This was observed to cause problems.
-dnl See bug #58596
-AT_SETUP([MATRIX DATA - crash])
-
-AT_DATA([matrix-data.sps], [dnl
-begin data
-corr 31
-
-matrix data
- var1
-begin data
- corr 1.00
-end data .
-
-matrix data
- variables = roxtype_ var01
- /format = upper nodiagonal.
-begin data
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [1], [ignore])
-AT_CLEANUP
-\f
-dnl Keep this test in sync with Example 6 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - LOWER DIAGONAL without ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=var01 TO var08
- /CONTENTS=MEAN SD N CORR.
-BEGIN DATA.
-24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
- 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
- 92 92 92 92 92 92 92 92
-1.00
- .18 1.00
--.22 -.17 1.00
- .36 .31 -.14 1.00
- .27 .16 -.12 .22 1.00
- .33 .15 -.17 .24 .21 1.00
- .50 .29 -.20 .32 .12 .38 1.00
- .17 .29 -.05 .20 .27 .20 .04 1.00
-END DATA.
-FORMATS var01 TO var08(F5.2).
-LIST.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
-MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
-STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
-N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
-CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
-CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
-CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
-CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
-CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
-CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
-CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
-CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - extraneous data without ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=var01 TO var08
- /CONTENTS=MEAN SD N CORR.
-BEGIN DATA.
-24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
- 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
- 92 92 92 92 92 92 92 92
-1.00 .18
- .18 1.00
--.22 -.17 1.00
- .36 .31 -.14 1.00
- .27 .16 -.12 .22 1.00
- .33 .15 -.17 .24 .21 1.00
- .50 .29 -.20 .32 .12 .38 1.00
- .17 .29 -.05 .20 .27 .20 .04 1.00
-END DATA.
-FORMATS var01 TO var08(F5.2).
-LIST.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [1], [dnl
-matrix-data.sps:8.8-8.10: error: Extraneous data expecting end of line.
-
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
-MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
-STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
-N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
-CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
-CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
-CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
-CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
-CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
-CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
-CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
-CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 7 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - Split variables with explicit values without ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=s1 var01 TO var04
- /SPLIT=s1
- /FORMAT=FULL
- /CONTENTS=MEAN SD N CORR.
-BEGIN DATA.
-0 34 35 36 37
-0 22 11 55 66
-0 99 98 99 92
-0 1 .9 .8 .7
-0 .9 1 .6 .5
-0 .8 .6 1 .4
-0 .7 .5 .4 1
-1 44 45 34 39
-1 23 15 51 46
-1 98 34 87 23
-1 1 .2 .3 .4
-1 .2 1 .5 .6
-1 .3 .5 1 .7
-1 .4 .6 .7 1
-END DATA.
-FORMATS var01 TO var04(F5.2).
-LIST.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
-Table: Split Values
-Variable,Value
-s1,0
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-0,MEAN,,34.00,35.00,36.00,37.00
-0,STDDEV,,22.00,11.00,55.00,66.00
-0,N,,99.00,98.00,99.00,92.00
-0,CORR,var01,1.00,.90,.80,.70
-0,CORR,var02,.90,1.00,.60,.50
-0,CORR,var03,.80,.60,1.00,.40
-0,CORR,var04,.70,.50,.40,1.00
-
-Table: Split Values
-Variable,Value
-s1,1
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-1,MEAN,,44.00,45.00,34.00,39.00
-1,STDDEV,,23.00,15.00,51.00,46.00
-1,N,,98.00,34.00,87.00,23.00
-1,CORR,var01,1.00,.20,.30,.40
-1,CORR,var02,.20,1.00,.50,.60
-1,CORR,var03,.30,.50,1.00,.70
-1,CORR,var04,.40,.60,.70,1.00
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 8 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - Split variable with sequential values without ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=var01 TO var04
- /SPLIT=s1
- /FORMAT=FULL
- /CONTENTS=MEAN SD N CORR.
-BEGIN DATA.
-34 35 36 37
-22 11 55 66
-99 98 99 92
- 1 .9 .8 .7
-.9 1 .6 .5
-.8 .6 1 .4
-.7 .5 .4 1
-44 45 34 39
-23 15 51 46
-98 34 87 23
- 1 .2 .3 .4
-.2 1 .5 .6
-.3 .5 1 .7
-.4 .6 .7 1
-END DATA.
-FORMATS var01 TO var04(F5.2).
-LIST.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
-Table: Split Values
-Variable,Value
-s1,1
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-1,MEAN,,34.00,35.00,36.00,37.00
-1,STDDEV,,22.00,11.00,55.00,66.00
-1,N,,99.00,98.00,99.00,92.00
-1,CORR,var01,1.00,.90,.80,.70
-1,CORR,var02,.90,1.00,.60,.50
-1,CORR,var03,.80,.60,1.00,.40
-1,CORR,var04,.70,.50,.40,1.00
-
-Table: Split Values
-Variable,Value
-s1,2
-
-Table: Data List
-s1,ROWTYPE_,VARNAME_,var01,var02,var03,var04
-2,MEAN,,44.00,45.00,34.00,39.00
-2,STDDEV,,23.00,15.00,51.00,46.00
-2,N,,98.00,34.00,87.00,23.00
-2,CORR,var01,1.00,.20,.30,.40
-2,CORR,var02,.20,1.00,.50,.60
-2,CORR,var03,.30,.50,1.00,.70
-2,CORR,var04,.40,.60,.70,1.00
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 9 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - Factor variables grouping within-cell records by factor without ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=f1 var01 TO var04
- /FACTOR=f1
- /CELLS=2
- /CONTENTS=(MEAN SD N) CORR.
-BEGIN DATA.
-0 34 35 36 37
-0 22 11 55 66
-0 99 98 99 92
-1 44 45 34 39
-1 23 15 51 46
-1 98 34 87 23
- 1
- .9 1
- .8 .6 1
- .7 .5 .4 1
-END DATA.
-FORMATS var01 TO var04(F5.1).
-LIST.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
-Table: Data List
-ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
-MEAN,0,,34.0,35.0,36.0,37.0
-STDDEV,0,,22.0,11.0,55.0,66.0
-N,0,,99.0,98.0,99.0,92.0
-MEAN,1,,44.0,45.0,34.0,39.0
-STDDEV,1,,23.0,15.0,51.0,46.0
-N,1,,98.0,34.0,87.0,23.0
-CORR,.,var01,1.0,.9,.8,.7
-CORR,.,var02,.9,1.0,.6,.5
-CORR,.,var03,.8,.6,1.0,.4
-CORR,.,var04,.7,.5,.4,1.0
-])
-AT_CLEANUP
-
-dnl Keep this test in sync with Example 10 in doc/matrices.texi.
-AT_SETUP([MATRIX DATA - Factor variables grouping within-cell records by row type without ROWTYPE_])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=f1 var01 TO var04
- /FACTOR=f1
- /CELLS=2
- /CONTENTS=(MEAN) (SD) (N) CORR.
-BEGIN DATA.
-0 34 35 36 37
-1 44 45 34 39
-0 22 11 55 66
-1 23 15 51 46
-0 99 98 99 92
-1 98 34 87 23
- 1
- .9 1
- .8 .6 1
- .7 .5 .4 1
-END DATA.
-FORMATS var01 TO var04(F5.1).
-LIST.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [0], [dnl
-Table: Data List
-ROWTYPE_,f1,VARNAME_,var01,var02,var03,var04
-MEAN,0,,34.0,35.0,36.0,37.0
-MEAN,1,,44.0,45.0,34.0,39.0
-STDDEV,0,,22.0,11.0,55.0,66.0
-STDDEV,1,,23.0,15.0,51.0,46.0
-N,0,,99.0,98.0,99.0,92.0
-N,1,,98.0,34.0,87.0,23.0
-CORR,.,var01,1.0,.9,.8,.7
-CORR,.,var02,.9,1.0,.6,.5
-CORR,.,var03,.8,.6,1.0,.4
-CORR,.,var04,.7,.5,.4,1.0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX DATA - syntax errors])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA VARIABLES=var01 varname_.
-MATRIX DATA VARIABLES=v v v.
-MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/SPLIT=rowtype_.
-MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/FACTORS=rowtype_.
-MATRIX DATA VARIABLES=rowtype_ s1 v1 v2 v3/SPLIT=v1/FACTORS=v1.
-
-MATRIX DATA VARIABLES=v1 v2 v3/FORMAT=FULL NODIAGONAL.
-MATRIX DATA VARIABLES=v1 v2 v3/FACTORS=v1.
-MATRIX DATA VARIABLES=v1 v2 v3.
-BEGIN DATA.
-END DATA.
-MATRIX DATA VARIABLES=v1/FACTORS=v1.
-MATRIX DATA VARIABLES=v1 v2 v3 ROWTYPE_.
-MATRIX DATA VARIABLES=v1 v2 v3/CONTENTS=N/N=5.
-MATRIX DATA VARIABLES=v1/CONTENTS=XYZZY.
-MATRIX DATA VARIABLES=v1/CONTENTS=(.
-MATRIX DATA VARIABLES=v1/CONTENTS=(CORR.
-MATRIX DATA VARIABLES=v1/CONTENTS=).
-MATRIX DATA.
-MATRIX DATA VARIABLES=v*.
-MATRIX DATA VARIABLES=v/N=-1.
-MATRIX DATA VARIABLES=v/FORMAT=XYZZY.
-MATRIX DATA VARIABLES=v/FILE=123.
-MATRIX DATA VARIABLES=v/SPLIT=123.
-MATRIX DATA VARIABLES=v/CELLS=-1.
-MATRIX DATA VARIABLES=v/XYZZY.
-])
-AT_CHECK([pspp matrix-data.sps -O format=csv], [1], [dnl
-"matrix-data.sps:1.23-1.36: error: MATRIX DATA: VARIABLES may not include VARNAME_.
- 1 | MATRIX DATA VARIABLES=var01 varname_.
- | ^~~~~~~~~~~~~~"
-
-"matrix-data.sps:2.25: error: MATRIX DATA: Variable v appears twice in variable list.
- 2 | MATRIX DATA VARIABLES=v v v.
- | ^"
-
-"matrix-data.sps:3.47-3.54: error: MATRIX DATA: ROWTYPE_ is not allowed on SPLIT or FACTORS.
- 3 | MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/SPLIT=rowtype_.
- | ^~~~~~~~"
-
-"matrix-data.sps:4.49-4.56: error: MATRIX DATA: ROWTYPE_ is not allowed on SPLIT or FACTORS.
- 4 | MATRIX DATA VARIABLES=rowtype_ v1 v2 v3/FACTORS=rowtype_.
- | ^~~~~~~~"
-
-"matrix-data.sps:5.61-5.62: error: MATRIX DATA: v1 may not appear on both SPLIT and FACTORS.
- 5 | MATRIX DATA VARIABLES=rowtype_ s1 v1 v2 v3/SPLIT=v1/FACTORS=v1.
- | ^~"
-
-"matrix-data.sps:7.32-7.53: error: MATRIX DATA: FORMAT=FULL and FORMAT=NODIAGONAL are mutually exclusive.
- 7 | MATRIX DATA VARIABLES=v1 v2 v3/FORMAT=FULL NODIAGONAL.
- | ^~~~~~~~~~~~~~~~~~~~~~"
-
-matrix-data.sps:8: error: MATRIX DATA: CELLS is required when factor variables are specified and VARIABLES does not include ROWTYPE_.
-
-matrix-data.sps:9: warning: MATRIX DATA: CONTENTS was not specified and VARIABLES does not include ROWTYPE_. Assuming CONTENTS=CORR.
-
-matrix-data.sps:12: error: MATRIX DATA: CELLS is required when factor variables are specified and VARIABLES does not include ROWTYPE_.
-
-"matrix-data.sps:13.13-13.39: error: MATRIX DATA: VARIABLES includes ROWTYPE_ but the continuous variables are not the last ones on VARIABLES.
- 13 | MATRIX DATA VARIABLES=v1 v2 v3 ROWTYPE_.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"matrix-data.sps:14.43-14.45: error: MATRIX DATA: Cannot specify N on CONTENTS along with the N subcommand.
- 14 | MATRIX DATA VARIABLES=v1 v2 v3/CONTENTS=N/N=5.
- | ^~~"
-
-"matrix-data.sps:15.35-15.39: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
- 15 | MATRIX DATA VARIABLES=v1/CONTENTS=XYZZY.
- | ^~~~~"
-
-"matrix-data.sps:16.36: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
- 16 | MATRIX DATA VARIABLES=v1/CONTENTS=@{:@.
- | ^"
-
-"matrix-data.sps:17.40: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
- 17 | MATRIX DATA VARIABLES=v1/CONTENTS=@{:@CORR.
- | ^"
-
-"matrix-data.sps:18.35: error: MATRIX DATA: Syntax error expecting one of the following: CORR, COV, MAT, N_MATRIX, PROX, COUNT, DFE, MEAN, MSE, STDDEV, N, N_SCALAR, N_VECTOR, SD.
- 18 | MATRIX DATA VARIABLES=v1/CONTENTS=@:}@.
- | ^"
-
-"matrix-data.sps:19.12: error: MATRIX DATA: Syntax error expecting VARIABLES.
- 19 | MATRIX DATA.
- | ^"
-
-"matrix-data.sps:20.24: error: MATRIX DATA: Syntax error expecting `/'.
- 20 | MATRIX DATA VARIABLES=v*.
- | ^"
-
-"matrix-data.sps:21.27-21.28: error: MATRIX DATA: Syntax error expecting non-negative integer for N.
- 21 | MATRIX DATA VARIABLES=v/N=-1.
- | ^~"
-
-"matrix-data.sps:22.32-22.36: error: MATRIX DATA: Syntax error expecting LIST, FREE, UPPER, LOWER, FULL, DIAGONAL, or NODIAGONAL.
- 22 | MATRIX DATA VARIABLES=v/FORMAT=XYZZY.
- | ^~~~~"
-
-"matrix-data.sps:23.30-23.32: error: MATRIX DATA: Syntax error expecting a file name or handle name.
- 23 | MATRIX DATA VARIABLES=v/FILE=123.
- | ^~~"
-
-"matrix-data.sps:24.31-24.33: error: MATRIX DATA: Syntax error expecting variable name.
- 24 | MATRIX DATA VARIABLES=v/SPLIT=123.
- | ^~~"
-
-"matrix-data.sps:25.31-25.32: error: MATRIX DATA: Syntax error expecting non-negative integer for CELLS.
- 25 | MATRIX DATA VARIABLES=v/CELLS=-1.
- | ^~"
-
-"matrix-data.sps:26.25-26.29: error: MATRIX DATA: Syntax error expecting N, FORMAT, FILE, SPLIT, FACTORS, CELLS, or CONTENTS.
- 26 | MATRIX DATA VARIABLES=v/XYZZY.
- | ^~~~~"
-])
-AT_CLEANUP
-
-dnl I don't know what lunatic thought this was OK, but we strive to be
-dnl compatible.
-AT_SETUP([MATRIX DATA - plus and minus as delimiters])
-AT_DATA([matrix-data.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ var01 TO var08.
-BEGIN DATA.
-MEAN+24.3+5.4+69.7+20.1+13.4+2.7+27.9+3.7
-SD +5.7+1.5+23.5+5.8+2.8+4.5+5.4+1.5
-N+92+92+92+92+92+92+92+92
-CORR+1.00
-CORR+.18+1.00
-CORR-.22e+0-.17+1.00
-CORR+.36d-0+.31-.14+1.00
-CORR+.27+.16-.12+.22+1.00
-CORR+.33+.15-.17+.24+.21+1.00
-CORR+.50+.29-.20+.32+.12+.38+1.00
-CORR+.17+.29-.05+.20+.27+.20+.04+1.00
-END DATA.
-FORMATS var01 TO var08(F5.2).
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv matrix-data.sps], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,var01,var02,var03,var04,var05,var06,var07,var08
-MEAN,,24.30,5.40,69.70,20.10,13.40,2.70,27.90,3.70
-STDDEV,,5.70,1.50,23.50,5.80,2.80,4.50,5.40,1.50
-N,,92.00,92.00,92.00,92.00,92.00,92.00,92.00,92.00
-CORR,var01,1.00,.18,-.22,.36,.27,.33,.50,.17
-CORR,var02,.18,1.00,-.17,.31,.16,.15,.29,.29
-CORR,var03,-.22,-.17,1.00,-.14,-.12,-.17,-.20,-.05
-CORR,var04,.36,.31,-.14,1.00,.22,.24,.32,.20
-CORR,var05,.27,.16,-.12,.22,1.00,.21,.12,.27
-CORR,var06,.33,.15,-.17,.24,.21,1.00,.38,.20
-CORR,var07,.50,.29,-.20,.32,.12,.38,1.00,.04
-CORR,var08,.17,.29,-.05,.20,.27,.20,.04,1.00
-])
-AT_CLEANUP
+++ /dev/null
-AT_BANNER([Matrix reader])
-
-AT_SETUP([Matrix reader - negative tests])
-AT_DATA([matrix-reader.sps], [dnl
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) VARNAME_(a8) f1 (f1) ROWTYPE_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(f8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (f8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTPYE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARANME_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (a1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (a1) VARNAME_ (a8) c1 to c5 (f8.2).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8) c1 to c5 (a8).
-DEBUG MATRIX READ NODATA.
-
-DATA LIST LIST NOTABLE /s1 (f1) ROWTYPE_(a8) f1 (f1) VARNAME_ (a8).
-DEBUG MATRIX READ NODATA.
-])
-AT_CHECK([pspp matrix-reader.sps --testing-mode -O format=csv], [1], [dnl
-matrix-reader.sps:5: error: DEBUG MATRIX READ: Variable ROWTYPE_ must precede VARNAME_ in matrix file dictionary.
-
-matrix-reader.sps:8: error: DEBUG MATRIX READ: Matrix dataset variable ROWTYPE_ should be of string type.
-
-matrix-reader.sps:11: error: DEBUG MATRIX READ: Matrix dataset variable VARNAME_ should be of string type.
-
-matrix-reader.sps:14: error: DEBUG MATRIX READ: Matrix dataset lacks a variable called ROWTYPE_.
-
-matrix-reader.sps:17: error: DEBUG MATRIX READ: Matrix dataset lacks a variable called VARNAME_.
-
-matrix-reader.sps:20: error: DEBUG MATRIX READ: Matrix dataset variable s1 should be numeric.
-
-matrix-reader.sps:23: error: DEBUG MATRIX READ: Matrix dataset variable f1 should be numeric.
-
-matrix-reader.sps:26: error: DEBUG MATRIX READ: Matrix dataset variable c1 should be numeric.
-
-matrix-reader.sps:29: error: DEBUG MATRIX READ: Matrix dataset does not have any continuous variables.
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2021 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([MCONVERT])
-
-AT_SETUP([MCONVERT])
-AT_DATA([mconvert.sps], [dnl
-MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
-BEGIN DATA.
-0 COV 1.0
-0 COV 1.0 16.0
-0 COV 8.1 18.0 81.0
-1 CORR 1
-1 CORR .25 1
-1 CORR .9 .5 1
-1 STDDEV 1 4 9
-END DATA.
-FORMATS var01 TO var03(F5.2).
-SPLIT FILE OFF.
-MCONVERT.
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
-Table: Data List
-s,ROWTYPE_,VARNAME_,var01,var02,var03
-0,CORR,var01,1.00,.25,.90
-0,CORR,var02,.25,1.00,.50
-0,CORR,var03,.90,.50,1.00
-0,STDDEV,,1.00,4.00,9.00
-1,STDDEV,,1.00,4.00,9.00
-1,COV,var01,1.00,1.00,8.10
-1,COV,var02,1.00,16.00,18.00
-1,COV,var03,8.10,18.00,81.00
-])
-AT_CLEANUP
-
-AT_SETUP([MCONVERT from .sav file])
-AT_DATA([input.sps], [dnl
-MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
-BEGIN DATA.
-0 COV 1.0
-0 COV 1.0 16.0
-0 COV 8.1 18.0 81.0
-1 CORR 1
-1 CORR .25 1
-1 CORR .9 .5 1
-1 STDDEV 1 4 9
-END DATA.
-FORMATS var01 TO var03(F5.2).
-SPLIT FILE OFF.
-SAVE OUTFILE='input.sav'.
-])
-AT_DATA([mconvert.sps], [dnl
-MCONVERT MATRIX=IN('input.sav').
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv input.sps])
-AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
-Table: Data List
-s,ROWTYPE_,VARNAME_,var01,var02,var03
-0,CORR,var01,1.00,.25,.90
-0,CORR,var02,.25,1.00,.50
-0,CORR,var03,.90,.50,1.00
-0,STDDEV,,1.00,4.00,9.00
-1,STDDEV,,1.00,4.00,9.00
-1,COV,var01,1.00,1.00,8.10
-1,COV,var02,1.00,16.00,18.00
-1,COV,var03,8.10,18.00,81.00
-])
-AT_CLEANUP
-
-AT_SETUP([MCONVERT to .sav file])
-AT_DATA([mconvert.sps], [dnl
-MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
-BEGIN DATA.
-0 COV 1.0
-0 COV 1.0 16.0
-0 COV 8.1 18.0 81.0
-1 CORR 1
-1 CORR .25 1
-1 CORR .9 .5 1
-1 STDDEV 1 4 9
-END DATA.
-FORMATS var01 TO var03(F5.2).
-SPLIT FILE OFF.
-MCONVERT/REPLACE/OUT('output.sav').
-LIST.
-])
-AT_DATA([output.sps], [dnl
-GET 'output.sav'.
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
-Table: Data List
-s,ROWTYPE_,VARNAME_,var01,var02,var03
-0,COV,var01,1.00,1.00,8.10
-0,COV,var02,1.00,16.00,18.00
-0,COV,var03,8.10,18.00,81.00
-1,CORR,var01,1.00,.25,.90
-1,CORR,var02,.25,1.00,.50
-1,CORR,var03,.90,.50,1.00
-1,STDDEV,,1.00,4.00,9.00
-])
-AT_CHECK([pspp -O format=csv output.sps], [0], [dnl
-Table: Data List
-s,ROWTYPE_,VARNAME_,var01,var02,var03
-0,CORR,var01,1.00,.25,.90
-0,CORR,var02,.25,1.00,.50
-0,CORR,var03,.90,.50,1.00
-0,STDDEV,,1.00,4.00,9.00
-1,STDDEV,,1.00,4.00,9.00
-1,COV,var01,1.00,1.00,8.10
-1,COV,var02,1.00,16.00,18.00
-1,COV,var03,8.10,18.00,81.00
-])
-AT_CLEANUP
-
-AT_SETUP([MCONVERT from .sav file to .sav file])
-AT_DATA([input.sps], [dnl
-MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
-BEGIN DATA.
-0 COV 1.0
-0 COV 1.0 16.0
-0 COV 8.1 18.0 81.0
-1 CORR 1
-1 CORR .25 1
-1 CORR .9 .5 1
-1 STDDEV 1 4 9
-END DATA.
-FORMATS var01 TO var03(F5.2).
-SPLIT FILE OFF.
-SAVE OUTFILE='input.sav'.
-])
-AT_DATA([mconvert.sps], [dnl
-MCONVERT MATRIX=IN('input.sav') OUT('output.sav')/REPLACE.
-LIST.
-])
-AT_DATA([output.sps], [dnl
-GET 'output.sav'.
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv input.sps])
-AT_CHECK([pspp -O format=csv mconvert.sps], [1], [dnl
-"mconvert.sps:2.1-2.4: error: LIST: LIST is allowed only after the active dataset has been defined.
- 2 | LIST.
- | ^~~~"
-])
-AT_CHECK([pspp -O format=csv output.sps], [0], [dnl
-Table: Data List
-s,ROWTYPE_,VARNAME_,var01,var02,var03
-0,CORR,var01,1.00,.25,.90
-0,CORR,var02,.25,1.00,.50
-0,CORR,var03,.90,.50,1.00
-0,STDDEV,,1.00,4.00,9.00
-1,STDDEV,,1.00,4.00,9.00
-1,COV,var01,1.00,1.00,8.10
-1,COV,var02,1.00,16.00,18.00
-1,COV,var03,8.10,18.00,81.00
-])
-AT_CLEANUP
-
-AT_SETUP([MCONVERT with APPEND])
-AT_DATA([mconvert.sps], [dnl
-MATRIX DATA VARIABLES=s ROWTYPE_ var01 TO var03/SPLIT s.
-BEGIN DATA.
-0 COV 1.0
-0 COV 1.0 16.0
-0 COV 8.1 18.0 81.0
-1 CORR 1
-1 CORR .25 1
-1 CORR .9 .5 1
-1 STDDEV 1 4 9
-END DATA.
-FORMATS var01 TO var03(F5.2).
-SPLIT FILE OFF.
-MCONVERT/APPEND.
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv mconvert.sps], [0], [dnl
-Table: Data List
-s,ROWTYPE_,VARNAME_,var01,var02,var03
-0,COV,var01,1.00,1.00,8.10
-0,COV,var02,1.00,16.00,18.00
-0,COV,var03,8.10,18.00,81.00
-0,CORR,var01,1.00,.25,.90
-0,CORR,var02,.25,1.00,.50
-0,CORR,var03,.90,.50,1.00
-0,STDDEV,,1.00,4.00,9.00
-1,CORR,var01,1.00,.25,.90
-1,CORR,var02,.25,1.00,.50
-1,CORR,var03,.90,.50,1.00
-1,STDDEV,,1.00,4.00,9.00
-1,COV,var01,1.00,1.00,8.10
-1,COV,var02,1.00,16.00,18.00
-1,COV,var03,8.10,18.00,81.00
-])
-AT_CLEANUP
-
-AT_SETUP([MCONVERT negative test])
-AT_DATA([mconvert.sps], [MCONVERT.
-])
-AT_CHECK([pspp mconvert.sps], [1], [dnl
-mconvert.sps:1: error: MCONVERT: No active file is defined and no external file
-is specified on MATRIX=IN.
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([PRINT SPACE])
-
-AT_SETUP([PRINT SPACE without arguments])
-AT_DATA([print-space.sps], [dnl
-DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-1
-2
-END DATA.
-PRINT/x.
-PRINT SPACE.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print-space.sps], [0], [dnl
-1 @&t@
-
-2 @&t@
-
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT SPACE with number])
-AT_DATA([print-space.sps], [dnl
-DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-1
-2
-END DATA.
-PRINT/x.
-PRINT SPACE 2.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print-space.sps], [0], [dnl
-1 @&t@
-
-
-2 @&t@
-
-
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT SPACE to file])
-AT_CAPTURE_FILE([output.txt])
-AT_DATA([print-space.sps], [dnl
-DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-1
-2
-END DATA.
-PRINT OUTFILE='output.txt'/x.
-PRINT SPACE OUTFILE='output.txt'.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print-space.sps])
-AT_CHECK([cat output.txt], [0], [dnl
- 1 @&t@
- @&t@
- 2 @&t@
- @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT SPACE to file with number])
-AT_CAPTURE_FILE([output.txt])
-AT_DATA([print-space.sps], [dnl
-DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-1
-2
-END DATA.
-PRINT OUTFILE='output.txt'/x.
-PRINT SPACE OUTFILE='output.txt' 2.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print-space.sps])
-AT_CHECK([cat output.txt], [0], [dnl
- 1 @&t@
- @&t@
- @&t@
- 2 @&t@
- @&t@
- @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT SPACE syntax errors])
-AT_DATA([print-space.sps], [dnl
-DATA LIST NOTABLE /x 1.
-PRINT SPACE OUTFILE=**.
-PRINT SPACE OUTFILE='out.txt' ENCODING=**.
-PRINT SPACE **.
-PRINT SPACE 1 'xyzzy'.
-])
-AT_CHECK([pspp -O format=csv print-space.sps], [1], [dnl
-"print-space.sps:2.21-2.22: error: PRINT SPACE: Syntax error expecting a file name or handle name.
- 2 | PRINT SPACE OUTFILE=**.
- | ^~"
-
-"print-space.sps:3.40-3.41: error: PRINT SPACE: Syntax error expecting string.
- 3 | PRINT SPACE OUTFILE='out.txt' ENCODING=**.
- | ^~"
-
-"print-space.sps:4.13-4.14: error: PRINT SPACE: Syntax error parsing expression.
- 4 | PRINT SPACE **.
- | ^~"
-
-"print-space.sps:5.15-5.21: error: PRINT SPACE: Syntax error expecting end of command.
- 5 | PRINT SPACE 1 'xyzzy'.
- | ^~~~~~~"
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT SPACE evaluation errors])
-AT_DATA([print-space.sps], [dnl
-DATA LIST NOTABLE /x 1.
-BEGIN DATA.
-1
-END DATA.
-PRINT SPACE $SYSMIS.
-PRINT SPACE -1.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print-space.sps], [0], [dnl
-"print-space.sps:5.13-5.19: warning: EXECUTE: The expression on PRINT SPACE evaluated to the system-missing value.
- 5 | PRINT SPACE $SYSMIS.
- | ^~~~~~~"
-
-
-
-"print-space.sps:6.13-6.14: warning: EXECUTE: The expression on PRINT SPACE evaluated to -1.
- 6 | PRINT SPACE -1.
- | ^~"
-
-
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([PRINT and WRITE])
-
-dnl These tests check unusual aspects of the PRINT and WRITE
-dnl transformations:
-dnl
-dnl - PRINT puts spaces between variables, unless a format
-dnl is specified explicitly.
-dnl
-dnl - WRITE doesn't put space between variables.
-dnl
-dnl - PRINT to an external file prefixes each line with a space.
-dnl
-dnl - PRINT EJECT to an external file indicates a formfeed by a "1"
-dnl in the first column.
-dnl
-dnl - WRITE writes out spaces for system-missing values, not a period.
-dnl
-dnl - When no output is specified, an empty record is output.
-
-AT_SETUP([PRINT numeric variables])
-AT_DATA([print.sps], [dnl
-data list notable /x y 1-2.
-begin data.
-12
-34
- 6
-7
-90
-end data.
-
-print /x y.
-print eject /x y 1-2.
-print /x '-' y.
-print.
-
-execute.
-])
-AT_CHECK([pspp -O format=csv print.sps], [0], [dnl
-1 2 @&t@
-
-
-
-12
-1 -2 @&t@
-
-3 4 @&t@
-
-
-
-34
-3 -4 @&t@
-
-. 6 @&t@
-
-
-
-.6
-. -6 @&t@
-
-7 . @&t@
-
-
-
-7.
-7 -. @&t@
-
-9 0 @&t@
-
-
-
-90
-9 -0 @&t@
-
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT string variables])
-AT_DATA([print.sps], [dnl
-DATA LIST FREE /s8 (a8) s10 (a10) vl255 (a255) vl258 (a258).
-BEGIN DATA.
-12345678
-AaaaaaaaaZ
-AbbbbMaryHadALittleLambbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbZ
-AccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccZ
-87654321
-AnnnnnnnnZ
-AmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmZ
-AoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooWhyIsItSoZ
-END DATA.
-
-print
- outfile='print.txt'
- /s10 * vl255 * vl258 *.
-
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print.sps])
-AT_CHECK([cat print.txt], [0], [dnl
- AaaaaaaaaZ AbbbbMaryHadALittleLambbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbZ AccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccZ @&t@
- AnnnnnnnnZ AmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmZ AoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooWhyIsItSoZ @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT to file])
-AT_DATA([print.sps], [dnl
-data list notable /x y 1-2.
-begin data.
-12
-34
- 6
-7
-90
-end data.
-
-print outfile='print.out' /x y.
-print eject outfile='print.out' /x y (f1,f1).
-print outfile='print.out' /x '-' y.
-print outfile='print.out'.
-
-execute.
-])
-AT_CHECK([pspp -O format=csv print.sps])
-AT_CHECK([cat print.out], [0], [dnl
- 1 2 @&t@
-112
- 1 -2 @&t@
- @&t@
- 3 4 @&t@
-134
- 3 -4 @&t@
- @&t@
- . 6 @&t@
-1.6
- . -6 @&t@
- @&t@
- 7 . @&t@
-17.
- 7 -. @&t@
- @&t@
- 9 0 @&t@
-190
- 9 -0 @&t@
- @&t@
-])
-AT_CLEANUP
-
-dnl Tests for a bug which crashed when deallocating after a bad PRINT
-dnl command.
-AT_SETUP([PRINT crash bug])
-AT_DATA([print.sps], [dnl
-DATA LIST LIST NOTABLE /a * b *.
-BEGIN DATA.
-1 2
-3 4
-END DATA.
-
-PRINT F8.2
-LIST.
-])
-AT_CHECK([pspp -O format=csv print.sps], [1], [dnl
-"print.sps:7.7-7.10: error: PRINT: Syntax error expecting OUTFILE, ENCODING, RECORDS, TABLE, or NOTABLE.
- 7 | PRINT F8.2
- | ^~~~"
-
-Table: Data List
-a,b
-1.00,2.00
-3.00,4.00
-])
-AT_CLEANUP
-
-AT_SETUP([WRITE to file])
-AT_DATA([write.sps], [dnl
-data list notable /x y 1-2.
-begin data.
-12
-34
- 6
-7
-90
-end data.
-
-write outfile='write.out' /x y.
-write outfile='write.out' /x y (2(f1)).
-write outfile='write.out' /x '-' y.
-write outfile='write.out'.
-
-execute.
-])
-AT_CHECK([pspp -O format=csv write.sps])
-AT_CHECK([cat write.out], [0], [dnl
-12
-12
-1-2
-
-34
-34
-3-4
-
- 6
- 6
- -6
-
-7 @&t@
-7 @&t@
-7- @&t@
-
-90
-90
-9-0
-
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT overwrites file])
-AT_DATA([output.txt], [abcdef
-])
-AT_DATA([print.sps], [dnl
-DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-5
-END DATA.
-PRINT OUTFILE='output.txt'/x.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print.sps])
-AT_CHECK([cat output.txt], [0], [ 5 @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT overwrites file atomically])
-AT_DATA([output.txt], [abcdef
-])
-AT_DATA([print.sps],
-[[DATA LIST NOTABLE/x 1.
-BEGIN DATA.
-5
-END DATA.
-PRINT OUTFILE='output.txt'/x.
-HOST COMMAND=['kill -TERM $PPID'].
-]])
-AT_CHECK([pspp -O format=csv print.sps], [143], [], [ignore])
-AT_CHECK([cat output.txt], [0], [abcdef
-])
-AT_CHECK(
- [for file in *.tmp*; do if test -e $file; then echo $file; exit 1; fi; done])
-AT_CLEANUP
-
-AT_SETUP([PRINT to same file being read])
-AT_DATA([data.txt], [5
-])
-AT_DATA([print.sps], [dnl
-DATA LIST FILE='data.txt' NOTABLE/x 1.
-COMPUTE y = x + 1.
-PRINT OUTFILE='data.txt'/y.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print.sps])
-AT_CHECK([cat data.txt], [0], [ 6.00 @&t@
-])
-AT_CHECK(
- [for file in *.tmp*; do if test -e $file; then echo $file; exit 1; fi; done])
-AT_CLEANUP
-
-AT_SETUP([PRINT to special file])
-AT_SKIP_IF([test ! -c /dev/null])
-AT_CHECK([ln -s /dev/null foo.out || exit 77])
-AT_SKIP_IF([test ! -c foo.out])
-AT_DATA([print.sps], [dnl
-DATA LIST NOTABLE /x 1.
-BEGIN DATA.
-1
-2
-3
-4
-5
-END DATA.
-PRINT OUTFILE='foo.out'/x.
-PRINT OUTFILE='foo2.out'/x.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print.sps])
-AT_CHECK([cat foo2.out], [0], [dnl
- 1 @&t@
- 2 @&t@
- 3 @&t@
- 4 @&t@
- 5 @&t@
-])
-ls -l foo.out foo2.out
-AT_CHECK([test -c foo.out])
-AT_CLEANUP
-
-AT_SETUP([PRINT with special line ends])
-AT_DATA([print.sps], [dnl
-FILE HANDLE lf /NAME='lf.txt' /ENDS=LF.
-FILE HANDLE crlf /NAME='crlf.txt' /ENDS=CRLF.
-DATA LIST NOTABLE /x 1.
-BEGIN DATA.
-1
-2
-3
-4
-5
-END DATA.
-PRINT OUTFILE=lf/x.
-PRINT OUTFILE=crlf/x.
-EXECUTE.
-])
-AT_CHECK([pspp -O format=csv print.sps])
-AT_CHECK([cat lf.txt], [0], [dnl
- 1 @&t@
- 2 @&t@
- 3 @&t@
- 4 @&t@
- 5 @&t@
-])
-AT_CHECK([tr '\r' R < crlf.txt], [0], [dnl
- 1 R
- 2 R
- 3 R
- 4 R
- 5 R
-])
-AT_CLEANUP
-
-AT_SETUP([PRINT syntax errors])
-AT_DATA([print.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-PRINT OUTFILE=**.
-PRINT ENCODING=**.
-PRINT RECORDS=-1.
-PRINT **.
-PRINT/ **.
-PRINT/'string' 0.
-PRINT/'string' 5-3.
-PRINT/y.
-PRINT/x 0.
-PRINT/x (A8).
-])
-AT_CHECK([pspp -O format=csv print.sps], [1], [dnl
-"print.sps:2.15-2.16: error: PRINT: Syntax error expecting a file name or handle name.
- 2 | PRINT OUTFILE=**.
- | ^~"
-
-"print.sps:3.16-3.17: error: PRINT: Syntax error expecting string.
- 3 | PRINT ENCODING=**.
- | ^~"
-
-"print.sps:4.15-4.16: error: PRINT: Syntax error expecting non-negative integer for RECORDS.
- 4 | PRINT RECORDS=-1.
- | ^~"
-
-"print.sps:5.7-5.8: error: PRINT: Syntax error expecting OUTFILE, ENCODING, RECORDS, TABLE, or NOTABLE.
- 5 | PRINT **.
- | ^~"
-
-"print.sps:6.8-6.9: error: PRINT: Syntax error expecting variable name.
- 6 | PRINT/ **.
- | ^~"
-
-"print.sps:7.16: error: PRINT: Column positions for fields must be positive.
- 7 | PRINT/'string' 0.
- | ^"
-
-"print.sps:8.16-8.18: error: PRINT: The ending column for a field must be greater than the starting column.
- 8 | PRINT/'string' 5-3.
- | ^~~"
-
-"print.sps:9.7: error: PRINT: y is not a variable name.
- 9 | PRINT/y.
- | ^"
-
-"print.sps:10.9: error: PRINT: Column positions for fields must be positive.
- 10 | PRINT/x 0.
- | ^"
-
-"print.sps:11.9-11.12: error: PRINT: Numeric variable x is not compatible with string format A8.
- 11 | PRINT/x (A8).
- | ^~~~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SAVE TRANSLATE /TYPE=CSV])
-
-m4_define([PREPARE_SAVE_TRANSLATE_CSV], [dnl
-AT_KEYWORDS([SAVE TRANSLATE])
-AT_DATA([data.txt], [dnl
-0 '1 9:30:05' 1/2/2003 "25/8/1995 15:30:00" "'a,b,c'",0
-, '-0 5:17' 10/31/2010 "9/4/2008 9:29:00" " xxx ",1
-1.625,'0 12:00',,,xyzzy,1
-])
-AT_DATA([save-translate.pspp], [dnl
-SET DECIMAL=DOT.
-DATA LIST LIST NOTABLE FILE="data.txt"
- /number(F8.3) time(DTIME10) date(ADATE10) datetime(DATETIME20) string(A8)
- filter(F1.0).
-MISSING VALUES number(0) time('0 12:00') string('xyzzy').
-FILTER BY filter.
-SAVE TRANSLATE /OUTFILE="data.csv" /TYPE=m4_if([$2], [], [CSV], [$2])
- $1.
-])
-AT_CHECK([pspp -O format=csv save-translate.pspp], [0])
-])
-
-AT_SETUP([CSV output -- defaults])
-PREPARE_SAVE_TRANSLATE_CSV
-AT_CHECK([cat data.csv], [0], [dnl
-0,33:30:05,01/02/2003,08/25/1995 15:30:00,"'a,b,c'",0
- ,-05:17:00,10/31/2010,04/09/2008 09:29:00, xxx,1
-1.625,12:00:00, , ,xyzzy,1
-])
-AT_CLEANUP
-
-AT_SETUP([CSV output -- recode missing, delete unselected])
-PREPARE_SAVE_TRANSLATE_CSV([/MISSING=RECODE /UNSELECTED=DELETE])
-AT_CHECK([cat data.csv], [0], [dnl
- ,-05:17:00,10/31/2010,04/09/2008 09:29:00, xxx,1
-1.625, , , ,,1
-])
-AT_CLEANUP
-
-AT_SETUP([CSV output -- var names, formats])
-PREPARE_SAVE_TRANSLATE_CSV(
- [/FIELDNAMES /TEXTOPTIONS FORMAT=VARIABLE /UNSELECTED=RETAIN])
-AT_CHECK([cat data.csv], [0], [dnl
-number,time,date,datetime,string,filter
-.000,1 09:30:05,01/02/2003,25-AUG-1995 15:30:00,"'a,b,c'",0
- ,-0 05:17,10/31/2010,09-APR-2008 09:29:00, xxx,1
-1.625,0 12:00:00, , ,xyzzy,1
-])
-AT_CLEANUP
-
-AT_SETUP([CSV output -- comma as decimal point])
-PREPARE_SAVE_TRANSLATE_CSV([/FIELDNAMES /TEXTOPTIONS DECIMAL=COMMA])
-AT_CHECK([cat data.csv], [0], [dnl
-number;time;date;datetime;string;filter
-0;33:30:05;01/02/2003;08/25/1995 15:30:00;'a,b,c';0
- ;-05:17:00;10/31/2010;04/09/2008 09:29:00; xxx;1
-1,625;12:00:00; ; ;xyzzy;1
-])
-AT_CLEANUP
-
-AT_SETUP([CSV output -- custom delimiter, qualifier])
-PREPARE_SAVE_TRANSLATE_CSV(
- [/FIELDNAMES /TEXTOPTIONS DELIMITER=':' QUALIFIER="'"])
-AT_CHECK([cat data.csv], [0], [dnl
-number:time:date:datetime:string:filter
-0:'33:30:05':01/02/2003:'08/25/1995 15:30:00':'''a,b,c''':0
- :'-05:17:00':10/31/2010:'04/09/2008 09:29:00': xxx:1
-1.625:'12:00:00': : :xyzzy:1
-])
-AT_CLEANUP
-
-AT_SETUP([CSV output -- KEEP, RENAME quoted])
-PREPARE_SAVE_TRANSLATE_CSV(
- [/FIELDNAMES /KEEP=time string /RENAME string='long name with spaces' /UNSELECTED=DELETE])
-AT_CHECK([cat data.csv], [0], [dnl
-time,long name with spaces
--05:17:00, xxx
-12:00:00,xyzzy
-])
-AT_CLEANUP
-
-
-AT_SETUP([CSV output -- KEEP, RENAME multi quoted])
-PREPARE_SAVE_TRANSLATE_CSV(
- [/FIELDNAMES
- /RENAME =
- number = "this one"
- time = "that one"
- date = "which one?"
- datetime = "another variable replacement"
- string="long name with spaces"
- /UNSELECTED=DELETE])
-AT_CHECK([cat data.csv], [0], [dnl
-this one,that one,which one?,another variable replacement,long name with spaces,filter
- ,-05:17:00,10/31/2010,04/09/2008 09:29:00, xxx,1
-1.625,12:00:00, , ,xyzzy,1
-])
-AT_CLEANUP
-
-
-AT_SETUP([CSV output -- KEEP, RENAME bad name ])
-AT_KEYWORDS([SAVE TRANSLATE])
-AT_DATA([bad.sps], [
-data list notable list /Var1 Var2 Var3 Var4 Var5 *.
-begin data
-1 2 3 4 5
-end data.
-
-SAVE TRANSLATE
-/OUTFILE="foo.csv"
- /TYPE=CSV
- /MAP
- /REPLACE
- /FIELDNAMES
- /Unselected=DELETE
- /RENAME =
- Var4 = Var5
- (Var1 Var2 = one Var3 )
- (Var3 = "The second")
- /CELLS=VALUES
-.
-])
-
-AT_CHECK([pspp -O format=csv bad.sps], [1], [dnl
-"bad.sps:15.9-17.29: error: SAVE TRANSLATE: Requested renaming duplicates variable name Var5.
- 15 | Var4 = Var5
- | ^~~~~~~~~~~
- 16 | (Var1 Var2 = one Var3 )
- | -------------------------------
- 17 | (Var3 = ""The second"")
- | -----------------------------"
-])
-
-
-AT_CLEANUP
-
-
-
-AT_BANNER([SAVE TRANSLATE /TYPE=TAB])
-
-AT_SETUP([TAB output])
-PREPARE_SAVE_TRANSLATE_CSV([/FIELDNAMES], [TAB])
-AT_CHECK([cat data.csv], [0], [dnl
-number time date datetime string filter
-0 33:30:05 01/02/2003 08/25/1995 15:30:00 'a,b,c' 0
- -05:17:00 10/31/2010 04/09/2008 09:29:00 xxx 1
-1.625 12:00:00 xyzzy 1
-])
-AT_CLEANUP
-
-AT_SETUP([SAVE TRANSLATE syntax errors])
-: > xyzzy.csv
-AT_DATA([save-translate.sps], [dnl
-DATA LIST LIST NOTABLE /v1 to v10.
-SAVE TRANSLATE **.
-SAVE TRANSLATE/OUTFILE=**.
-SAVE TRANSLATE/OUTFILE='xyzzy.txt'/OUTFILE='xyzzy.txt'.
-SAVE TRANSLATE/TYPE=CSV/TYPE=**.
-SAVE TRANSLATE/TYPE=**.
-SAVE TRANSLATE/MISSING=**.
-SAVE TRANSLATE/CELLS=**.
-SAVE TRANSLATE/TEXTOPTIONS DELIMITER=**.
-SAVE TRANSLATE/TEXTOPTIONS DELIMITER='ab'.
-SAVE TRANSLATE/TEXTOPTIONS QUALIFIER=**.
-SAVE TRANSLATE/TEXTOPTIONS QUALIFIER='ab'.
-SAVE TRANSLATE/TEXTOPTIONS DECIMAL=**.
-SAVE TRANSLATE/UNSELECTED=**.
-SAVE TRANSLATE/ **.
-SAVE TRANSLATE/OUTFILE='xyzzy.csv'.
-SAVE TRANSLATE/TYPE=CSV.
-SAVE TRANSLATE/OUTFILE='xyzzy.csv'/TYPE=CSV.
-SAVE TRANSLATE/RENAME **.
-SAVE TRANSLATE/RENAME v1**.
-SAVE TRANSLATE/RENAME(v1**).
-SAVE TRANSLATE/RENAME v1=.
-SAVE TRANSLATE/RENAME v1=**.
-SAVE TRANSLATE/RENAME v1 to v5=v6.
-SAVE TRANSLATE/RENAME (v1=v2 v3).
-SAVE TRANSLATE/RENAME (v1 v2=v3).
-SAVE TRANSLATE/RENAME (v1=v3**.
-SAVE TRANSLATE/RENAME v1=v5.
-SAVE TRANSLATE/RENAME v1 v5=v5 v1.
-SAVE TRANSLATE/RENAME(v1 v5=v5 v1).
-SAVE TRANSLATE/RENAME(v1 to v10=v01 to v10).
-SAVE TRANSLATE/RENAME=v1=v1.
-SAVE TRANSLATE/DROP=ALL.
-SAVE TRANSLATE/DROP=**.
-SAVE TRANSLATE/KEEP=**.
-])
-AT_CHECK([pspp -O format=csv save-translate.sps], [1], [dnl
-"save-translate.sps:2.16-2.17: error: SAVE TRANSLATE: Syntax error expecting `/'.
- 2 | SAVE TRANSLATE **.
- | ^~"
-
-"save-translate.sps:3.24-3.25: error: SAVE TRANSLATE: Syntax error expecting a file name or handle name.
- 3 | SAVE TRANSLATE/OUTFILE=**.
- | ^~"
-
-"save-translate.sps:4.36-4.42: error: SAVE TRANSLATE: Subcommand OUTFILE may only be specified once.
- 4 | SAVE TRANSLATE/OUTFILE='xyzzy.txt'/OUTFILE='xyzzy.txt'.
- | ^~~~~~~"
-
-"save-translate.sps:5.25-5.28: error: SAVE TRANSLATE: Subcommand TYPE may only be specified once.
- 5 | SAVE TRANSLATE/TYPE=CSV/TYPE=**.
- | ^~~~"
-
-"save-translate.sps:6.21-6.22: error: SAVE TRANSLATE: Syntax error expecting CSV or TAB.
- 6 | SAVE TRANSLATE/TYPE=**.
- | ^~"
-
-"save-translate.sps:7.24-7.25: error: SAVE TRANSLATE: Syntax error expecting IGNORE or RECODE.
- 7 | SAVE TRANSLATE/MISSING=**.
- | ^~"
-
-"save-translate.sps:8.22-8.23: error: SAVE TRANSLATE: Syntax error expecting VALUES or LABELS.
- 8 | SAVE TRANSLATE/CELLS=**.
- | ^~"
-
-"save-translate.sps:9.38-9.39: error: SAVE TRANSLATE: Syntax error expecting string.
- 9 | SAVE TRANSLATE/TEXTOPTIONS DELIMITER=**.
- | ^~"
-
-"save-translate.sps:10.38-10.41: error: SAVE TRANSLATE: The DELIMITER string must contain exactly one character.
- 10 | SAVE TRANSLATE/TEXTOPTIONS DELIMITER='ab'.
- | ^~~~"
-
-"save-translate.sps:11.38-11.39: error: SAVE TRANSLATE: Syntax error expecting string.
- 11 | SAVE TRANSLATE/TEXTOPTIONS QUALIFIER=**.
- | ^~"
-
-"save-translate.sps:12.38-12.41: error: SAVE TRANSLATE: The QUALIFIER string must contain exactly one character.
- 12 | SAVE TRANSLATE/TEXTOPTIONS QUALIFIER='ab'.
- | ^~~~"
-
-"save-translate.sps:13.36-13.37: error: SAVE TRANSLATE: Syntax error expecting DOT or COMMA.
- 13 | SAVE TRANSLATE/TEXTOPTIONS DECIMAL=**.
- | ^~"
-
-"save-translate.sps:14.27-14.28: error: SAVE TRANSLATE: Syntax error expecting RETAIN or DELETE.
- 14 | SAVE TRANSLATE/UNSELECTED=**.
- | ^~"
-
-"save-translate.sps:15.17-15.18: error: SAVE TRANSLATE: Syntax error expecting MAP, DROP, KEEP, or RENAME.
- 15 | SAVE TRANSLATE/ **.
- | ^~"
-
-"save-translate.sps:16.1-16.35: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
- 16 | SAVE TRANSLATE/OUTFILE='xyzzy.csv'.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"save-translate.sps:17.1-17.24: error: SAVE TRANSLATE: Required subcommand OUTFILE was not specified.
- 17 | SAVE TRANSLATE/TYPE=CSV.
- | ^~~~~~~~~~~~~~~~~~~~~~~~"
-
-"save-translate.sps:18.16-18.34: error: SAVE TRANSLATE: Output file `xyzzy.csv' exists but REPLACE was not specified.
- 18 | SAVE TRANSLATE/OUTFILE='xyzzy.csv'/TYPE=CSV.
- | ^~~~~~~~~~~~~~~~~~~"
-
-"save-translate.sps:19.23-19.24: error: SAVE TRANSLATE: Syntax error expecting variable name.
- 19 | SAVE TRANSLATE/RENAME **.
- | ^~"
-
-"save-translate.sps:20.25-20.26: error: SAVE TRANSLATE: Syntax error expecting `='.
- 20 | SAVE TRANSLATE/RENAME v1**.
- | ^~"
-
-"save-translate.sps:21.25-21.26: error: SAVE TRANSLATE: Syntax error expecting `='.
- 21 | SAVE TRANSLATE/RENAME(v1**).
- | ^~"
-
-"save-translate.sps:22.26: error: SAVE TRANSLATE: Syntax error expecting variable name.
- 22 | SAVE TRANSLATE/RENAME v1=.
- | ^"
-
-"save-translate.sps:23.26-23.27: error: SAVE TRANSLATE: Syntax error expecting variable name.
- 23 | SAVE TRANSLATE/RENAME v1=**.
- | ^~"
-
-save-translate.sps:24: error: SAVE TRANSLATE: Old and new variable counts do not match.
-
-"save-translate.sps:24.23-24.30: note: SAVE TRANSLATE: There are 5 old variables.
- 24 | SAVE TRANSLATE/RENAME v1 to v5=v6.
- | ^~~~~~~~"
-
-"save-translate.sps:24.32-24.33: note: SAVE TRANSLATE: There is 1 new variable name.
- 24 | SAVE TRANSLATE/RENAME v1 to v5=v6.
- | ^~"
-
-save-translate.sps:25: error: SAVE TRANSLATE: Old and new variable counts do not match.
-
-"save-translate.sps:25.24-25.25: note: SAVE TRANSLATE: There is 1 old variable.
- 25 | SAVE TRANSLATE/RENAME (v1=v2 v3).
- | ^~"
-
-"save-translate.sps:25.27-25.31: note: SAVE TRANSLATE: There are 2 new variable names.
- 25 | SAVE TRANSLATE/RENAME (v1=v2 v3).
- | ^~~~~"
-
-save-translate.sps:26: error: SAVE TRANSLATE: Old and new variable counts do not match.
-
-"save-translate.sps:26.24-26.28: note: SAVE TRANSLATE: There are 2 old variables.
- 26 | SAVE TRANSLATE/RENAME (v1 v2=v3).
- | ^~~~~"
-
-"save-translate.sps:26.30-26.31: note: SAVE TRANSLATE: There is 1 new variable name.
- 26 | SAVE TRANSLATE/RENAME (v1 v2=v3).
- | ^~"
-
-"save-translate.sps:27.29-27.30: error: SAVE TRANSLATE: Syntax error expecting `)'.
- 27 | SAVE TRANSLATE/RENAME (v1=v3**.
- | ^~"
-
-"save-translate.sps:28.23-28.27: error: SAVE TRANSLATE: Requested renaming duplicates variable name v5.
- 28 | SAVE TRANSLATE/RENAME v1=v5.
- | ^~~~~"
-
-"save-translate.sps:29.26-29.27: error: SAVE TRANSLATE: Syntax error expecting `='.
- 29 | SAVE TRANSLATE/RENAME v1 v5=v5 v1.
- | ^~"
-
-"save-translate.sps:30.1-30.35: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
- 30 | SAVE TRANSLATE/RENAME(v1 v5=v5 v1).
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"save-translate.sps:31.1-31.44: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
- 31 | SAVE TRANSLATE/RENAME(v1 to v10=v01 to v10).
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"save-translate.sps:32.1-32.28: error: SAVE TRANSLATE: Required subcommand TYPE was not specified.
- 32 | SAVE TRANSLATE/RENAME=v1=v1.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"save-translate.sps:33.16-33.23: error: SAVE TRANSLATE: Cannot DROP all variables from dictionary.
- 33 | SAVE TRANSLATE/DROP=ALL.
- | ^~~~~~~~"
-
-"save-translate.sps:34.21-34.22: error: SAVE TRANSLATE: Syntax error expecting variable name.
- 34 | SAVE TRANSLATE/DROP=**.
- | ^~"
-
-"save-translate.sps:35.21-35.22: error: SAVE TRANSLATE: Syntax error expecting variable name.
- 35 | SAVE TRANSLATE/KEEP=**.
- | ^~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([IMPORT and EXPORT])
-
-AT_SETUP([IMPORT and EXPORT])
-AT_DATA([import-export.sps], [dnl
-DATA LIST LIST NOTABLE /X Y.
-BEGIN DATA.
-1 2
-3 .
-5 6
-END DATA.
-
-EXPORT /OUTFILE='wiz.por'.
-IMPORT /FILE='wiz.por'.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv import-export.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-X,Y
-1.00,2.00
-3.00,. @&t@
-5.00,6.00
-])
-AT_CLEANUP
-
-AT_BANNER([SAVE])
-
-# UNSELECTED=DELETE used to cause a crash if there was actually a
-# filter variable.
-AT_SETUP([SAVE -- delete unselected])
-AT_DATA([data.txt], [dnl
-0 '1 9:30:05' 1/2/2003 "25/8/1995 15:30:00" "'a,b,c'",0
-, '-0 5:17' 10/31/2010 "9/4/2008 9:29:00" " xxx ",1
-1.625,'0 12:00',,,xyzzy,1
-])
-AT_DATA([save.pspp], [dnl
-SET DECIMAL=DOT.
-DATA LIST LIST NOTABLE FILE="data.txt"
- /number(F8.3) time(DTIME10) date(ADATE10) datetime(DATETIME20) string(A8)
- filter(F1.0).
-MISSING VALUES number(0) time('0 12:00') string('xyzzy').
-FILTER BY filter.
-SAVE /OUTFILE="data.sav" /UNSELECTED=DELETE.
-])
-AT_DATA([get.pspp], [dnl
-GET FILE='data.sav'.
-LIST.
-])
-AT_CHECK([pspp -O format=csv save.pspp])
-AT_CHECK([pspp -O format=csv get.pspp], [0], [dnl
-Table: Data List
-number,time,date,datetime,string,filter
-. ,-0 05:17,10/31/2010,09-APR-2008 09:29:00,xxx,1
-1.625,0 12:00:00,.,.,xyzzy,1
-])
-AT_CLEANUP
-
-AT_SETUP([SAVE RENAME with TO])
-AT_DATA([save-rename-to.sps], [dnl
-data list notable list /a b c fxo9*.
-begin data
-1 2 3 8
-end data.
-
-SAVE OUTFILE = "renamed.sav"
- /RENAME=(A B C = fdo9 TO fdo11).
-
-
-NEW FILE.
-GET FILE = "renamed.sav".
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv save-rename-to.sps], [0], [dnl
-Table: Data List
-fdo9,fdo10,fdo11,fxo9
-1.00,2.00,3.00,8.00
-])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017, 2020 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-m4_define([CHECK_UPDATE],
- [AT_SETUP([UPDATE $1 with $2])
- AT_DATA([a.data], [dnl
-1aB
-8aM
-3aE
-5aG
-0aA
-5aH
-6aI
-7aJ
-2aD
-7aK
-1aC
-7aL
-4aF
-])
- AT_DATA([b.data], [dnl
-1bN
-3 O
-4bP
-6bQ
-7bR
-9bS
-])
- m4_if([$1], [sav],
- [AT_DATA([save-a.sps], [dnl
-DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).
-SAVE OUTFILE='a.sav'.
-])
- AT_CHECK([pspp -O format=csv save-a.sps])])
- m4_if([$2], [sav],
- [AT_DATA([save-b.sps], [dnl
-DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).
-SAVE OUTFILE='b.sav'.
-])
- AT_CHECK([pspp -O format=csv save-b.sps])])
- AT_DATA([update.sps], [dnl
-m4_if([$1], [sav], [], [DATA LIST NOTABLE FILE='a.data' /a b c 1-3 (A).])
-m4_if([$2], [sav], [], [DATA LIST NOTABLE FILE='b.data' /a b c 1-3 (A).])
-UPDATE
- m4_if([$1], [sav], [FILE='a.sav'], [FILE=*]) /IN=InA /SORT
- m4_if([$2], [sav], [FILE='b.sav'], [FILE=*]) /IN=InB /RENAME c=d
- /BY a.
-LIST.
-])
- cat update.sps
- AT_CHECK([pspp -O format=csv update.sps], [0], [dnl
-update.sps:6: warning: UPDATE: Encountered 3 sets of duplicate cases in the master file.
-
-Table: Data List
-a,b,c,d,InA,InB
-0,a,A,,1,0
-1,b,B,N,1,1
-1,a,C,,1,0
-2,a,D,,1,0
-3,a,E,O,1,1
-4,b,F,P,1,1
-5,a,G,,1,0
-5,a,H,,1,0
-6,b,I,Q,1,1
-7,b,J,R,1,1
-7,a,K,,1,0
-7,a,L,,1,0
-8,a,M,,1,0
-9,b,,S,0,1
-])
-AT_CLEANUP
-])
-
-AT_BANNER([UPDATE])
-
-CHECK_UPDATE([sav], [sav])
-CHECK_UPDATE([sav], [inline])
-CHECK_UPDATE([inline], [sav])
-
-dnl Far more syntax errors are possible, but the rest are all covered
-dnl by the MATCH FILES tests.
-AT_SETUP([UPDATE syntax errors])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='update.sps' ERROR=IGNORE.
-])
-AT_DATA([update.sps], [dnl
-DATA LIST LIST NOTABLE/name (A6) x.
-BEGIN DATA.
-al,7
-brad,8
-carl,9
-END DATA.
-SAVE OUTFILE='x.sav'.
-
-DATA LIST LIST NOTABLE/name (A7) y.
-BEGIN DATA.
-al,1
-carl,2
-dan,3
-END DATA.
-UPDATE/FILE='x.sav'/FILE=*/RENAME(name=name2).
-UPDATE/xyzzy.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"update.sps:15.1-15.46: error: UPDATE: Required subcommand BY was not specified.
- 15 | UPDATE/FILE='x.sav'/FILE=*/RENAME(name=name2).
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"update.sps:16.8-16.12: error: UPDATE: Syntax error expecting BY, MAP, DROP, or KEEP.
- 16 | UPDATE/xyzzy.
- | ^~~~~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2019 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([APPLY DICTIONARY])
-
-AT_SETUP([APPLY DICTIONARY])
-AT_DATA([apply-dict.sps], [dnl
-data list notable list /foo (TIME22.0) bar (a22).
-begin data
-end data.
-Variable label foo "This is a label".
-save outfile='ugg.sav'.
-
-new file.
-data list notable list /foo bar *.
-begin data
-end data.
-display dictionary.
-apply dictionary from = 'ugg.sav'.
-display dictionary.
-])
-
-AT_CHECK([pspp -O format=csv apply-dict.sps], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-foo,1,Nominal,Input,8,Right,F8.2,F8.2
-bar,2,Nominal,Input,8,Right,F8.2,F8.2
-
-"apply-dict.sps:12: warning: APPLY DICTIONARY: Variable bar is numeric in target file, but string in source file."
-
-Table: Variables
-Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-foo,1,This is a label,Nominal,Input,8,Right,TIME22.0,TIME22.0
-bar,2,,Nominal,Input,8,Right,F8.2,F8.2
-])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([VARIABLE ATTRIBUTES and DATAFILE ATTRIBUTES])
-
-AT_SETUP([VARIABLE ATTRIBUTES and DATAFILE ATTRIBUTES])
-AT_DATA([save-attrs.pspp],
- [[DATA LIST FREE/a b c.
-BEGIN DATA.
-1 2 3
-END DATA.
-
-DATAFILE ATTRIBUTE
- ATTRIBUTE=key('value')
- array('array element 1')
- Array[2]('array element 2').
-VARIABLE ATTRIBUTE
- VARIABLES=a b
- ATTRIBUTE=ValidationRule[2]("a + b > 2")
- ValidationRule[1]('a * b > 3')
- /VARIABLES=c
- ATTRIBUTE=QuestionWording('X or Y?').
-DISPLAY ATTRIBUTES.
-
-SAVE OUTFILE='attributes.sav'.
-]])
-AT_DATA([get-attrs.pspp],
- [[GET FILE='attributes.sav'.
-
-DATAFILE ATTRIBUTE
- DELETE=Array[1] Array[2].
-VARIABLE ATTRIBUTE
- VARIABLES=a
- DELETE=ValidationRule
- /VARIABLE=b
- DELETE=validationrule[2].
-
-DISPLAY ATTRIBUTES.
-]])
-AT_CHECK([pspp -O format=csv save-attrs.pspp], [0],
- [[Table: Variable and Dataset Attributes
-Variable and Name,,Value
-(dataset),array[1],array element 1
-,array[2],array element 2
-,key,value
-a,ValidationRule[1],a * b > 3
-,ValidationRule[2],a + b > 2
-b,ValidationRule[1],a * b > 3
-,ValidationRule[2],a + b > 2
-c,QuestionWording,X or Y?
-]])
-AT_CHECK([pspp -O format=csv get-attrs.pspp], [0], [dnl
-Table: Variable and Dataset Attributes
-Variable and Name,,Value
-(dataset),array,array element 2
-,key,value
-b,ValidationRule,a * b > 3
-c,QuestionWording,X or Y?
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DELETE VARIABLES])
-
-dnl Checks for regressions against a crash reported in bug #38843.
-AT_SETUP([DELETE VARIABLES with FILTER])
-AT_DATA([delete-variables.sps], [dnl
-DATA LIST LIST /a b.
-BEGIN DATA.
-1 3
-4 6
-7 9
-END DATA.
-
-FILTER BY b.
-DELETE VARIABLES a.
-LIST.
-])
-AT_CHECK([pspp -O format=csv delete-variables.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-a,F8.0
-b,F8.0
-
-Table: Data List
-b
-3.00
-6.00
-9.00
-])
-AT_CLEANUP
-
-dnl Checks for regression against a crash reported on pspp-users:
-dnl https://lists.gnu.org/archive/html/pspp-users/2021-03/msg00025.html
-AT_SETUP([DELETE VARIABLES with string variables])
-AT_DATA([delete-variables.sps], [dnl
-DATA LIST NOTABLE /s1 TO s2 1-2(A).
-BEGIN DATA
-12
-END DATA.
-DELETE VARIABLES s1.
-NUMERIC n1.
-LIST.
-])
-AT_CHECK([pspp -O format=csv delete-variables.sps], [0], [dnl
-Table: Data List
-s2,n1
-2,. @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([DELETE VARIABLES syntax errors])
-AT_DATA([delete-variables.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-BEGIN DATA.
-1 2 3
-END DATA.
-TEMPORARY.
-DELETE VARIABLES x.
-DELETE VARIABLES y z.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='delete-variables.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"delete-variables.sps:6.1-6.16: error: DELETE VARIABLES: DELETE VARIABLES may not be used after TEMPORARY. Temporary transformations will be made permanent.
- 6 | DELETE VARIABLES x.
- | ^~~~~~~~~~~~~~~~"
-
-"delete-variables.sps:7.1-7.20: error: DELETE VARIABLES: DELETE VARIABLES may not be used to delete all variables from the active dataset dictionary. Use NEW FILE instead.
- 7 | DELETE VARIABLES y z.
- | ^~~~~~~~~~~~~~~~~~~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([FORMATS])
-
-AT_SETUP([FORMATS positive tests])
-AT_DATA([formats.sps], [dnl
-DATA LIST LIST /a b c * x (A1) y (A2) z (A3).
-DISPLAY VARIABLES.
-FORMATS /a (COMMA10) b (N4).
-DISPLAY VARIABLES.
-FORMATS c (E8.1) x (A1) /y (AHEX4) z (A3).
-DISPLAY VARIABLES.
-])
-AT_CHECK([pspp -o pspp.csv formats.sps])
-AT_CHECK([grep -E -v 'Measure|Display' pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-a,F8.0
-b,F8.0
-c,F8.0
-x,A1
-y,A2
-z,A3
-
-Table: Variables
-Name,Position,Print Format,Write Format
-a,1,F8.2,F8.2
-b,2,F8.2,F8.2
-c,3,F8.2,F8.2
-x,4,A1,A1
-y,5,A2,A2
-z,6,A3,A3
-
-Table: Variables
-Name,Position,Print Format,Write Format
-a,1,COMMA10.0,COMMA10.0
-b,2,N4.0,N4.0
-c,3,F8.2,F8.2
-x,4,A1,A1
-y,5,A2,A2
-z,6,A3,A3
-
-Table: Variables
-Name,Position,Print Format,Write Format
-a,1,COMMA10.0,COMMA10.0
-b,2,N4.0,N4.0
-c,3,E8.1,E8.1
-x,4,A1,A1
-y,5,AHEX4,AHEX4
-z,6,A3,A3
-])
-AT_CLEANUP
-
-AT_SETUP([FORMATS negative tests])
-AT_DATA([formats.sps], [dnl
-DATA LIST LIST /a b c * x (A1) y (A2) z (A3).
-FORMATS a (E6.1).
-FORMATS a y (F4).
-FORMATS x (A2).
-FORMATS y (AHEX2).
-FORMATS x y (A2).
-])
-AT_CHECK([pspp -O format=csv formats.sps], [1], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-a,F8.0
-b,F8.0
-c,F8.0
-x,A1
-y,A2
-z,A3
-
-"formats.sps:2.12-2.15: error: FORMATS: Output format E6.1 specifies 1 decimal place, but width 6 does not allow for any decimals.
- 2 | FORMATS a (E6.1).
- | ^~~~"
-
-"formats.sps:3.11: error: FORMATS: a and y are not the same type. All variables in this variable list must be of the same type. y will be omitted from the list.
- 3 | FORMATS a y (F4).
- | ^"
-
-"formats.sps:4.12-4.13: error: FORMATS: String variable x with width 1 is not compatible with format A2. Use format A1 instead.
- 4 | FORMATS x (A2).
- | ^~"
-
-"formats.sps:5.12-5.16: error: FORMATS: String variable y with width 2 is not compatible with format AHEX2. Use format AHEX4 instead.
- 5 | FORMATS y (AHEX2).
- | ^~~~~"
-
-"formats.sps:6.11: error: FORMATS: x and y are string variables with different widths. All variables in this variable list must have the same width. y will be omitted from the list.
- 6 | FORMATS x y (A2).
- | ^"
-
-"formats.sps:6.14-6.15: error: FORMATS: String variable x with width 1 is not compatible with format A2. Use format A1 instead.
- 6 | FORMATS x y (A2).
- | ^~"
-])
-AT_CLEANUP
+++ /dev/null
-AT_BANNER([LEAVE])
-
-AT_SETUP([LEAVE])
-AT_DATA([leave.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-LEAVE x y.
-])
-AT_CHECK([pspp -O format=csv leave.sps])
-AT_CLEANUP
-
-AT_SETUP([LEAVE syntax errors])
-AT_DATA([leave.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-LEAVE **.
-LEAVE x **.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='leave.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"leave.sps:2.7-2.8: error: LEAVE: Syntax error expecting variable name.
- 2 | LEAVE **.
- | ^~"
-
-"leave.sps:3.9-3.10: error: LEAVE: Syntax error expecting end of command.
- 3 | LEAVE x **.
- | ^~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([MISSING VALUES])
-
-AT_SETUP([MISSING VALUES valid cases])
-AT_DATA([missing-values.sps], [dnl
-DATA LIST NOTABLE/str1 1-5 (A) str2 6-8 (A) date1 9-19 (DATE) num1 20-25
- longstr 26-36 (A).
-
-* Numeric missing values.
-MISSING VALUES date1 num1 (1).
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES date1 num1 (1, 2).
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES date1 num1 (1, 2, 3).
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES date1 num1 (9999998, 9999984, 3).
-DISPLAY DICTIONARY date1 num1.
-
-* Numeric missing values using the first variable's format.
-MISSING VALUES num1 date1 ('1').
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES num1 date1 ('1', '2').
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES num1 date1 ('1', '2', '3').
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES date1 num1 ('06-AUG-05').
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES date1 num1 ('06-AUG-05', '01-OCT-78').
-DISPLAY DICTIONARY date1 num1.
-MISSING VALUES date1 num1 ('06-AUG-05', '01-OCT-78', '14-FEB-81').
-DISPLAY DICTIONARY date1 num1.
-
-* Ranges of numeric missing values.
-MISSING VALUES num1 (1 THRU 2).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (LO THRU 2).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (LOWEST THRU 2).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (1 THRU HI).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (1 THRU HIGHEST).
-DISPLAY DICTIONARY num1.
-
-* A range of numeric missing values, plus an individual value.
-MISSING VALUES num1 (1 THRU 2, 3).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (LO THRU 2, 3).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (LOWEST THRU 2, 3).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (1 THRU HI, -1).
-DISPLAY DICTIONARY num1.
-MISSING VALUES num1 (1 THRU HIGHEST, -1).
-DISPLAY DICTIONARY num1.
-
-* String missing values.
-MISSING VALUES str1 str2 longstr ('abc ','def').
-DISPLAY DICTIONARY str1 str2 longstr.
-
-* May mix variable types when clearing missing values.
-MISSING VALUES ALL ().
-MISSING VALUES num1 (1).
-DISPLAY DICTIONARY
-])
-AT_CHECK([pspp -o pspp.csv missing-values.sps])
-AT_CHECK([cat pspp.csv | sed '/^Table/d
-/^Name/d
-s/^\([[a-z0-9]]*\),.*,\([[^,]]*\)$/\1: \2/'], [0], [dnl
-date1: 1
-num1: 1
-
-date1: 1; 2
-num1: 1; 2
-
-date1: 1; 2; 3
-num1: 1; 2; 3
-
-date1: 9999998; 9999984; 3
-num1: 9999998; 9999984; 3
-
-date1: 1
-num1: 1
-
-date1: 1; 2
-num1: 1; 2
-
-date1: 1; 2; 3
-num1: 1; 2; 3
-
-date1: 13342665600
-num1: 13342665600
-
-date1: 13342665600; 12495427200
-num1: 13342665600; 12495427200
-
-date1: 13342665600; 12495427200; 12570336000
-num1: 13342665600; 12495427200; 12570336000
-
-num1: 1 THRU 2
-
-num1: LOWEST THRU 2
-
-num1: LOWEST THRU 2
-
-num1: 1 THRU HIGHEST
-
-num1: 1 THRU HIGHEST
-
-num1: 1 THRU 2; 3
-
-num1: LOWEST THRU 2; 3
-
-num1: LOWEST THRU 2; 3
-
-num1: 1 THRU HIGHEST; -1
-
-num1: 1 THRU HIGHEST; -1
-
-str1: """abc ""; ""def """
-str2: """abc""; ""def"""
-longstr: """abc ""; ""def """
-
-str1: @&t@
-str2: @&t@
-date1: @&t@
-num1: 1
-longstr: @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([MISSING VALUES invalid cases])
-AT_DATA([missing-values.sps], [dnl
-DATA LIST NOTABLE/str1 1-5 (A) str2 6-8 (A) date1 9-19 (DATE) num1 20-25
- longstr 26-36 (A).
-
-* Too long for str2.
-MISSING VALUES str1 str2 longstr ('abcde').
-
-* Long string missing value longer than 8 bytes.
-MISSING VALUES longstr ('abcdefghijk').
-
-* No string ranges.
-MISSING VALUES str1 ('a' THRU 'z').
-
-* Mixing string and numeric variables.
-MISSING VALUES str1 num1 ('123').
-
-* Too many values.
-MISSING VALUES num1 (1, 2, 3, 4).
-MISSING VALUES num1 (1 THRU 2, 3 THRU 4).
-MISSING VALUES num1 (1, 2 THRU 3, 4).
-MISSING VALUES str1 ('abc', 'def', 'ghi', 'jkl').
-
-* Bad range.
-MISSING VALUES num1 (2 THRU 1).
-])
-AT_CHECK([pspp -O format=csv missing-values.sps], [1], [dnl
-"missing-values.sps:5.35-5.41: error: MISSING VALUES: Missing values are too long to assign to variable str2 with width 3.
- 5 | MISSING VALUES str1 str2 longstr ('abcde').
- | ^~~~~~~"
-
-"missing-values.sps:8.25-8.37: error: MISSING VALUES: Truncating missing value to maximum acceptable length (8 bytes).
- 8 | MISSING VALUES longstr ('abcdefghijk').
- | ^~~~~~~~~~~~~"
-
-"missing-values.sps:11.26-11.29: error: MISSING VALUES: Syntax error expecting string.
- 11 | MISSING VALUES str1 ('a' THRU 'z').
- | ^~~~"
-
-"missing-values.sps:14.27-14.31: error: MISSING VALUES: Cannot assign string missing values to numeric variable num1.
- 14 | MISSING VALUES str1 num1 ('123').
- | ^~~~~"
-
-"missing-values.sps:17.22-17.31: error: MISSING VALUES: Too many numeric missing values. At most three individual values or one value and one range are allowed.
- 17 | MISSING VALUES num1 (1, 2, 3, 4).
- | ^~~~~~~~~~"
-
-"missing-values.sps:18.22-18.39: error: MISSING VALUES: Too many numeric missing values. At most three individual values or one value and one range are allowed.
- 18 | MISSING VALUES num1 (1 THRU 2, 3 THRU 4).
- | ^~~~~~~~~~~~~~~~~~"
-
-"missing-values.sps:19.22-19.35: error: MISSING VALUES: Too many numeric missing values. At most three individual values or one value and one range are allowed.
- 19 | MISSING VALUES num1 (1, 2 THRU 3, 4).
- | ^~~~~~~~~~~~~~"
-
-"missing-values.sps:20.22-20.47: error: MISSING VALUES: Too many string missing values. At most three individual values are allowed.
- 20 | MISSING VALUES str1 ('abc', 'def', 'ghi', 'jkl').
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"missing-values.sps:23.22-23.29: warning: MISSING VALUES: The high end of the range (1) is below the low end (2). The range will be treated as if reversed.
- 23 | MISSING VALUES num1 (2 THRU 1).
- | ^~~~~~~~"
-])
-AT_CLEANUP
-
-AT_SETUP([MISSING VALUES syntax errors])
-AT_DATA([missing-values.sps], [dnl
-DATA LIST LIST NOTABLE/n1 to n10 (F8.2) s1 to s10 (A8).
-MISSING VALUES **.
-MISSING VALUES n1 **.
-MISSING VALUES s1 (1).
-MISSING VALUES n1 (1**).
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='missing-values.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"missing-values.sps:2.16-2.17: error: MISSING VALUES: Syntax error expecting variable name.
- 2 | MISSING VALUES **.
- | ^~"
-
-"missing-values.sps:3.19-3.20: error: MISSING VALUES: Syntax error expecting `@{:@'.
- 3 | MISSING VALUES n1 **.
- | ^~"
-
-"missing-values.sps:4.20: error: MISSING VALUES: Syntax error expecting string.
- 4 | MISSING VALUES s1 (1).
- | ^"
-
-"missing-values.sps:5.21-5.22: error: MISSING VALUES: Syntax error expecting number.
- 5 | MISSING VALUES n1 (1**).
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([MRSETS])
-
-m4_define([DEFINE_MRSETS_DATA],
- [DATA LIST NOTABLE /w x y z 1-4 a b c d 5-8 (a).
-BEGIN DATA.
-1234acbd
-5678efgh
-END DATA.])
-
-m4_define([DEFINE_MRSETS],
- [DEFINE_MRSETS_DATA
-
-[VARIABLE LABEL
- w 'duplicate variable label'
- x 'Variable x'
- z 'Duplicate variable label'.
-VALUE LABELS
- /w 1 'w value 1'
- /y 1 'duplicate Value label'
- /z 1 'duplicate value Label'
- /a b c d 'a' 'burger' 'b' 'fries' 'c' 'shake' 'd' 'taco'.
-ADD VALUE LABELS
- /b 'b' 'Fries'
- /c 'b' 'XXX'.
-MRSETS
- /MDGROUP NAME=$a
- LABEL='First multiple dichotomy group'
- CATEGORYLABELS=VARLABELS
- VARIABLES=w x y z
- VALUE=5
- /MDGROUP NAME=$b
- CATEGORYLABELS=COUNTEDVALUES
- VARIABLES=z y
- VALUE=123
- /MDGROUP NAME=$c
- LABELSOURCE=VARLABEL
- CATEGORYLABELS=COUNTEDVALUES
- VARIABLES=w x y z
- VALUE=1
- /MDGROUP NAME=$d
- LABELSOURCE=VARLABEL
- VARIABLES=a b c d
- VALUE='c'
- /MCGROUP NAME=$e
- LABEL='First multiple category group'
- VARIABLES=w x y z
- /MCGROUP NAME=$f
- VARIABLES=a b c d.
-]])
-
-m4_define([DEFINE_MRSETS_OUTPUT], [dnl
-"mrsets.sps:23.16-23.22: warning: MRSETS: Variables w and z specified as part of multiple dichotomy group $a have the same variable label. Categories represented by these variables will not be distinguishable in output.
- 23 | VARIABLES=w x y z
- | ^~~~~~~"
-
-"mrsets.sps:27.16-27.18: warning: MRSETS: Variable z specified as part of multiple dichotomy group $b (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
- 27 | VARIABLES=z y
- | ^~~"
-
-"mrsets.sps:27.16-27.18: warning: MRSETS: Variable y specified as part of multiple dichotomy group $b (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
- 27 | VARIABLES=z y
- | ^~~"
-
-"mrsets.sps:32.16-32.22: warning: MRSETS: Variable x specified as part of multiple dichotomy group $c (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
- 32 | VARIABLES=w x y z
- | ^~~~~~~"
-
-"mrsets.sps:32.16-32.22: warning: MRSETS: Variables y and z specified as part of multiple dichotomy group $c (which has CATEGORYLABELS=COUNTEDVALUES) have the same value label for the group's counted value. These categories will not be distinguishable in output.
- 32 | VARIABLES=w x y z
- | ^~~~~~~"
-
-"mrsets.sps:35.6-35.25: warning: MRSETS: MDGROUP subcommand for group $d specifies LABELSOURCE=VARLABEL but not CATEGORYLABELS=COUNTEDVALUES. Ignoring LABELSOURCE.
- 35 | LABELSOURCE=VARLABEL
- | ^~~~~~~~~~~~~~~~~~~~"
-
-"mrsets.sps:40.16-40.22: warning: MRSETS: Variables specified on MCGROUP should have the same categories, but w and y (and possibly others) in multiple category group $e have different value labels for value 1.
- 40 | VARIABLES=w x y z
- | ^~~~~~~"
-
-"mrsets.sps:42.16-42.22: warning: MRSETS: Variables specified on MCGROUP should have the same categories, but a and c (and possibly others) in multiple category group $f have different value labels for value b.
- 42 | VARIABLES=a b c d.
- | ^~~~~~~"
-])
-
-m4_define([MRSETS_DISPLAY_OUTPUT], [dnl
-Table: Multiple Response Sets
-Name,Label,Encoding,Counted Value,Member Variables
-$a,First multiple dichotomy group,Dichotomies,5,"w
-x
-y
-z"
-$b,,Dichotomies,123,"z
-y"
-$c,duplicate variable label,Dichotomies,1,"w
-x
-y
-z"
-$d,,Dichotomies,c,"a
-b
-c
-d"
-$e,First multiple category group,Categories,,"w
-x
-y
-z"
-$f,,Categories,,"a
-b
-c
-d"
-])
-
-AT_SETUP([MRSETS add, display, delete])
-AT_DATA([mrsets.sps],
- [DEFINE_MRSETS
-[MRSETS
- /DISPLAY NAME=[$a]
- /DISPLAY NAME=ALL
- /DELETE NAME=[$c]
- /DISPLAY NAME=ALL
- /DELETE NAME=ALL
- /DISPLAY NAME=ALL.
-]])
-AT_CHECK([pspp -o - -O format=csv -o mrsets.csv -o mrsets.txt mrsets.sps], [0],
- [DEFINE_MRSETS_OUTPUT
-Table: Multiple Response Sets
-Name,Label,Encoding,Counted Value,Member Variables
-$a,First multiple dichotomy group,Dichotomies,5,"w
-x
-y
-z"
-
-Table: Multiple Response Sets
-Name,Label,Encoding,Counted Value,Member Variables
-$a,First multiple dichotomy group,Dichotomies,5,"w
-x
-y
-z"
-$b,,Dichotomies,123,"z
-y"
-$c,duplicate variable label,Dichotomies,1,"w
-x
-y
-z"
-$d,,Dichotomies,c,"a
-b
-c
-d"
-$e,First multiple category group,Categories,,"w
-x
-y
-z"
-$f,,Categories,,"a
-b
-c
-d"
-
-Table: Multiple Response Sets
-Name,Label,Encoding,Counted Value,Member Variables
-$a,First multiple dichotomy group,Dichotomies,5,"w
-x
-y
-z"
-$b,,Dichotomies,123,"z
-y"
-$d,,Dichotomies,c,"a
-b
-c
-d"
-$e,First multiple category group,Categories,,"w
-x
-y
-z"
-$f,,Categories,,"a
-b
-c
-d"
-
-"mrsets.sps:50.19-50.21: note: MRSETS: The active dataset dictionary does not contain any multiple response sets.
- 50 | /DISPLAY NAME=ALL.
- | ^~~"
-])
-AT_CLEANUP
-
-AT_SETUP([MRSETS read and write])
-AT_DATA([mrsets.sps],
- [DEFINE_MRSETS
-SAVE OUTFILE='mrsets.sav'.
-])
-AT_CHECK([pspp -O format=csv mrsets.sps], [0], [DEFINE_MRSETS_OUTPUT])
-AT_DATA([mrsets2.sps],
- [GET FILE='mrsets.sav'.
-MRSETS /DISPLAY NAME=ALL.
-])
-AT_CHECK([pspp -O format=csv mrsets2.sps], [0], [MRSETS_DISPLAY_OUTPUT],
- [], [hd mrsets.sav])
-AT_CLEANUP
-
-AT_SETUP([MRSETS syntax errors])
-AT_DATA([mrsets.sps], [dnl
-DATA LIST NOTABLE /w x y z 1-4 a b c d 5-8 (a).
-BEGIN DATA.
-1234acbd
-5678efgh
-END DATA.
-VARIABLE LABEL
- w 'duplicate variable label'
- x 'Variable x'
- z 'Duplicate variable label'.
-VALUE LABELS
- /w 1 'w value 1'
- /y 1 'duplicate Value label'
- /z 1 'duplicate value Label'
- /a b c d 'a' 'burger' 'b' 'fries' 'c' 'shake' 'd' 'taco'.
-ADD VALUE LABELS
- /b 'b' 'Fries'
- /c 'b' 'XXX'.
-
-MRSETS /MDGROUP NAME **.
-MRSETS /MDGROUP NAME=**.
-MRSETS /MDGROUP NAME=x.
-MRSETS /MDGROUP VARIABLES **.
-MRSETS /MDGROUP VARIABLES=**.
-MRSETS /MDGROUP VARIABLES=a.
-MRSETS /MDGROUP LABEL **.
-MRSETS /MDGROUP LABEL=**.
-MRSETS /MDGROUP LABELSOURCE=**.
-MRSETS /MDGROUP VALUE **.
-MRSETS /MDGROUP VALUE=1.5.
-MRSETS /MDGROUP VALUE=**.
-MRSETS /MDGROUP CATEGORYLABELS **.
-MRSETS /MDGROUP CATEGORYLABELS=**.
-MRSETS /MDGROUP **.
-MRSETS /MCGROUP **.
-MRSETS /MDGROUP.
-MRSETS /MDGROUP NAME=[$x].
-MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE=1.
-MRSETS /MDGROUP NAME=[$x] VARIABLES=x y VALUE='a'.
-MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE='xyzzy'.
-MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE='y' LABELSOURCE=VARLABEL.
-MRSETS /MDGROUP NAME=[$x] VARIABLES=w z VALUE=1 CATEGORYLABELS=VARLABELS.
-MRSETS /MDGROUP NAME=[$x] VARIABLES=a b VALUE='y'
- LABELSOURCE=VARLABEL CATEGORYLABELS=COUNTEDVALUES
- LABEL='foo'.
-MRSETS /MDGROUP NAME=[$x] VARIABLES=y z VALUE=1
- LABELSOURCE=VARLABEL CATEGORYLABELS=COUNTEDVALUES.
-MRSETS /MCGROUP NAME=[$x] VARIABLES=w x y z.
-
-MRSETS /DELETE **.
-MRSETS /DELETE NAME**.
-MRSETS /DELETE NAME=[[**]].
-MRSETS /DELETE NAME=[[$x]].
-MRSETS /DELETE NAME=**.
-
-MRSETS /DISPLAY NAME=ALL.
-])
-AT_CHECK([pspp -O format=csv mrsets.sps], [1], [dnl
-"mrsets.sps:19.22-19.23: error: MRSETS: Syntax error expecting `='.
- 19 | MRSETS /MDGROUP NAME **.
- | ^~"
-
-"mrsets.sps:20.22-20.23: error: MRSETS: Syntax error expecting identifier.
- 20 | MRSETS /MDGROUP NAME=**.
- | ^~"
-
-"mrsets.sps:21.22: error: MRSETS: x is not a valid name for a multiple response set. Multiple response set names must begin with `$'.
- 21 | MRSETS /MDGROUP NAME=x.
- | ^"
-
-"mrsets.sps:22.27-22.28: error: MRSETS: Syntax error expecting `='.
- 22 | MRSETS /MDGROUP VARIABLES **.
- | ^~"
-
-"mrsets.sps:23.27-23.28: error: MRSETS: Syntax error expecting variable name.
- 23 | MRSETS /MDGROUP VARIABLES=**.
- | ^~"
-
-"mrsets.sps:24.27: error: MRSETS: At least two variables are required.
- 24 | MRSETS /MDGROUP VARIABLES=a.
- | ^"
-
-"mrsets.sps:25.23-25.24: error: MRSETS: Syntax error expecting `='.
- 25 | MRSETS /MDGROUP LABEL **.
- | ^~"
-
-"mrsets.sps:26.23-26.24: error: MRSETS: Syntax error expecting string.
- 26 | MRSETS /MDGROUP LABEL=**.
- | ^~"
-
-"mrsets.sps:27.28-27.30: error: MRSETS: Syntax error expecting `=VARLABEL'.
- 27 | MRSETS /MDGROUP LABELSOURCE=**.
- | ^~~"
-
-"mrsets.sps:28.23-28.24: error: MRSETS: Syntax error expecting `='.
- 28 | MRSETS /MDGROUP VALUE **.
- | ^~"
-
-"mrsets.sps:29.23-29.25: error: MRSETS: Numeric VALUE must be an integer.
- 29 | MRSETS /MDGROUP VALUE=1.5.
- | ^~~"
-
-"mrsets.sps:30.23-30.24: error: MRSETS: Syntax error expecting integer or string.
- 30 | MRSETS /MDGROUP VALUE=**.
- | ^~"
-
-"mrsets.sps:31.32-31.33: error: MRSETS: Syntax error expecting `='.
- 31 | MRSETS /MDGROUP CATEGORYLABELS **.
- | ^~"
-
-"mrsets.sps:32.32-32.33: error: MRSETS: Syntax error expecting VARLABELS or COUNTEDVALUES.
- 32 | MRSETS /MDGROUP CATEGORYLABELS=**.
- | ^~"
-
-"mrsets.sps:33.17-33.18: error: MRSETS: Syntax error expecting NAME, VARIABLES, LABEL, LABELSOURCE, VALUE, or CATEGORYLABELS.
- 33 | MRSETS /MDGROUP **.
- | ^~"
-
-"mrsets.sps:34.17-34.18: error: MRSETS: Syntax error expecting NAME, VARIABLES, or LABEL.
- 34 | MRSETS /MCGROUP **.
- | ^~"
-
-"mrsets.sps:35.16: error: MRSETS: Required NAME specification missing from MDGROUP subcommand.
- 35 | MRSETS /MDGROUP.
- | ^"
-
-"mrsets.sps:36.24: error: MRSETS: Required VARIABLES specification missing from MDGROUP subcommand.
- 36 | MRSETS /MDGROUP NAME=$x.
- | ^"
-
-mrsets.sps:37: error: MRSETS: VARIABLES and VALUE must have the same type.
-
-"mrsets.sps:37.35-37.37: note: MRSETS: These are string variables.
- 37 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE=1.
- | ^~~"
-
-"mrsets.sps:37.45: note: MRSETS: This is a numeric value.
- 37 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE=1.
- | ^"
-
-mrsets.sps:38: error: MRSETS: VARIABLES and VALUE must have the same type.
-
-"mrsets.sps:38.35-38.37: note: MRSETS: These are numeric variables.
- 38 | MRSETS /MDGROUP NAME=$x VARIABLES=x y VALUE='a'.
- | ^~~"
-
-"mrsets.sps:38.45-38.47: note: MRSETS: This is a string value.
- 38 | MRSETS /MDGROUP NAME=$x VARIABLES=x y VALUE='a'.
- | ^~~"
-
-mrsets.sps:39: error: MRSETS: The VALUE string must be no longer than the narrowest variable in the group.
-
-"mrsets.sps:39.45-39.51: note: MRSETS: The VALUE string is 5 bytes long.
- 39 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='xyzzy'.
- | ^~~~~~~"
-
-"mrsets.sps:39.35-39.37: note: MRSETS: Variable a has a width of 1 bytes.
- 39 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='xyzzy'.
- | ^~~"
-
-"mrsets.sps:40.49-40.68: warning: MRSETS: MDGROUP subcommand for group $x specifies LABELSOURCE=VARLABEL but not CATEGORYLABELS=COUNTEDVALUES. Ignoring LABELSOURCE.
- 40 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='y' LABELSOURCE=VARLABEL.
- | ^~~~~~~~~~~~~~~~~~~~"
-
-"mrsets.sps:41.35-41.37: warning: MRSETS: Variables w and z specified as part of multiple dichotomy group $x have the same variable label. Categories represented by these variables will not be distinguishable in output.
- 41 | MRSETS /MDGROUP NAME=$x VARIABLES=w z VALUE=1 CATEGORYLABELS=VARLABELS.
- | ^~~"
-
-"mrsets.sps:44: warning: MRSETS: MDGROUP subcommand for group $x specifies both LABEL and LABELSOURCE, but only one of these subcommands may be used at a time. Ignoring LABELSOURCE."
-
-"mrsets.sps:44.3-44.13: note: MRSETS: Here is the LABEL setting.
- 44 | LABEL='foo'.
- | ^~~~~~~~~~~"
-
-"mrsets.sps:43.3-43.22: note: MRSETS: Here is the LABELSOURCE setting.
- 43 | LABELSOURCE=VARLABEL CATEGORYLABELS=COUNTEDVALUES
- | ^~~~~~~~~~~~~~~~~~~~"
-
-"mrsets.sps:42.35-42.37: warning: MRSETS: Variable a specified as part of multiple dichotomy group $x (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
- 42 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='y'
- | ^~~"
-
-"mrsets.sps:42.35-42.37: warning: MRSETS: Variable b specified as part of multiple dichotomy group $x (which has CATEGORYLABELS=COUNTEDVALUES) has no value label for its counted value. This category will not be distinguishable in output.
- 42 | MRSETS /MDGROUP NAME=$x VARIABLES=a b VALUE='y'
- | ^~~"
-
-"mrsets.sps:45.35-45.37: warning: MRSETS: Variables y and z specified as part of multiple dichotomy group $x (which has CATEGORYLABELS=COUNTEDVALUES) have the same value label for the group's counted value. These categories will not be distinguishable in output.
- 45 | MRSETS /MDGROUP NAME=$x VARIABLES=y z VALUE=1
- | ^~~"
-
-"mrsets.sps:47.35-47.41: warning: MRSETS: Variables specified on MCGROUP should have the same categories, but w and y (and possibly others) in multiple category group $x have different value labels for value 1.
- 47 | MRSETS /MCGROUP NAME=$x VARIABLES=w x y z.
- | ^~~~~~~"
-
-"mrsets.sps:49.16-49.17: error: MRSETS: Syntax error expecting `NAME='.
- 49 | MRSETS /DELETE **.
- | ^~"
-
-"mrsets.sps:50.16-50.21: error: MRSETS: Syntax error expecting `NAME='.
- 50 | MRSETS /DELETE NAME**.
- | ^~~~~~"
-
-"mrsets.sps:51.22-51.23: error: MRSETS: Syntax error expecting identifier.
- 51 | MRSETS /DELETE NAME=[[**]].
- | ^~"
-
-"mrsets.sps:53.21-53.22: error: MRSETS: Syntax error expecting `@<:@' or ALL.
- 53 | MRSETS /DELETE NAME=**.
- | ^~"
-
-"mrsets.sps:55.22-55.24: note: MRSETS: The active dataset dictionary does not contain any multiple response sets.
- 55 | MRSETS /DISPLAY NAME=ALL.
- | ^~~"
-])
-AT_CLEANUP
+++ /dev/null
-AT_BANNER([NUMERIC])
-
-AT_SETUP([NUMERIC])
-AT_DATA([numeric.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-NUMERIC n/k(F5).
-DISPLAY DICTIONARY.
-])
-AT_CHECK([pspp -O format=csv numeric.sps], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-x,1,Unknown,Input,8,Right,F8.2,F8.2
-y,2,Unknown,Input,8,Right,F8.2,F8.2
-z,3,Unknown,Input,8,Right,F8.2,F8.2
-n,4,Unknown,Input,8,Right,F8.2,F8.2
-k,5,Unknown,Input,8,Right,F5.0,F5.0
-])
-AT_CLEANUP
-
-AT_SETUP([NUMERIC syntax errors])
-AT_DATA([numeric.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-NUMERIC **.
-NUMERIC n **.
-NUMERIC x.
-NUMERIC n (**).
-NUMERIC n (F50).
-NUMERIC n (A8).
-NUMERIC n (F8.0 **).
-])
-AT_CHECK([pspp -O format=csv numeric.sps], [1], [dnl
-"numeric.sps:2.9-2.10: error: NUMERIC: Syntax error expecting variable name.
- 2 | NUMERIC **.
- | ^~"
-
-"numeric.sps:3.11-3.12: error: NUMERIC: Syntax error expecting end of command.
- 3 | NUMERIC n **.
- | ^~"
-
-"numeric.sps:4.9: error: NUMERIC: There is already a variable named x.
- 4 | NUMERIC x.
- | ^"
-
-"numeric.sps:5.12-5.13: error: NUMERIC: Syntax error expecting valid format specifier.
- 5 | NUMERIC n (**).
- | ^~"
-
-"numeric.sps:6.12-6.14: error: NUMERIC: Output format F50.0 specifies width 50, but F requires a width between 1 and 40.
- 6 | NUMERIC n (F50).
- | ^~~"
-
-"numeric.sps:7.12-7.13: error: NUMERIC: Format type A8 may not be used with a numeric variable.
- 7 | NUMERIC n (A8).
- | ^~"
-
-"numeric.sps:8.17-8.18: error: NUMERIC: Syntax error expecting `@:}@'.
- 8 | NUMERIC n (F8.0 **).
- | ^~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([RENAME VARIABLES])
-
-AT_SETUP([RENAME VARIABLES])
-AT_DATA([rename-variables.sps], [dnl
-DATA LIST LIST /brakeFluid y auxiliary warp (F2.0).
-BEGIN DATA.
-1 3 5 9
-2 3 6 10
-3 3 7 11
-4 3 8 11
-END DATA.
-
-LIST.
-
-RENAME VARIABLES brakeFluid=applecarts y=bananamobiles.
-RENAME VARIABLES (warp auxiliary=foobar xyzzy).
-
-LIST.
-
-SAVE /OUTFILE='rename.sav'.
-])
-AT_CHECK([pspp -O format=csv rename-variables.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-brakeFluid,F2.0
-y,F2.0
-auxiliary,F2.0
-warp,F2.0
-
-Table: Data List
-brakeFluid,y,auxiliary,warp
-1,3,5,9
-2,3,6,10
-3,3,7,11
-4,3,8,11
-
-Table: Data List
-applecarts,bananamobiles,xyzzy,foobar
-1,3,5,9
-2,3,6,10
-3,3,7,11
-4,3,8,11
-])
-AT_CHECK([grep '[bB][rR][aA][kK][eE]' rename.sav], [1], [ignore-nolog])
-AT_CLEANUP
-
-
-AT_SETUP([RENAME VARIABLES -- multiple sets])
-AT_DATA([rename-variables.sps], [dnl
-data list list /a b c d e *.
-begin data.
-1 2 3 4 5
-end data.
-
-rename variables (a b=x y) (c d e=z zz zzz).
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv rename-variables.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-a,F8.0
-b,F8.0
-c,F8.0
-d,F8.0
-e,F8.0
-
-Table: Data List
-x,y,z,zz,zzz
-1.00,2.00,3.00,4.00,5.00
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([RENAME VARIABLES -- invalid syntax 1])
-
-AT_DATA([rename-variables.sps], [dnl
-DATA LIST LIST /brakeFluid y auxiliary warp (F2.0).
-RENAME VARIABLES warp auxiliary=foobar xyzzy.
-])
-
-AT_CHECK([pspp -o pspp.csv rename-variables.sps], [1], [dnl
-rename-variables.sps:2.23-2.31: error: RENAME VARIABLES: Syntax error expecting `='.
- 2 | RENAME VARIABLES warp auxiliary=foobar xyzzy.
- | ^~~~~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([RENAME VARIABLES -- invalid syntax 2])
-AT_DATA([rename-variables.sps], [dnl
-DATA LIST LIST /brakeFluid y auxiliary warp (F2.0).
-RENAME VARIABLES (brakeFluid=applecarts y=bananamobiles).
-])
-
-AT_CHECK([pspp -o pspp.csv rename-variables.sps], [1], [dnl
-rename-variables.sps:2.19-2.41: error: RENAME VARIABLES: Differing number of variables in old name list (1) and in new name list (2).
- 2 | RENAME VARIABLES (brakeFluid=applecarts y=bananamobiles).
- | ^~~~~~~~~~~~~~~~~~~~~~~
-])
-AT_CLEANUP
-
-
-
-
-AT_SETUP([RENAME VARIABLES -- invalid syntax 3])
-AT_DATA([rename-variables.sps], [dnl
-DATA LIST NOTABLE LIST /z y p q (F2.0).
-BEGIN DATA.
-4 3 8 11
-END DATA.
-
-RENAME VARIABLES z=a y}bqnanamobiles.
-
-LIST.
-])
-
-AT_CHECK([pspp -O format=csv rename-variables.sps], [1], [ignore])
-
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SORT VARIABLES])
-
-AT_SETUP([SORT VARIABLES])
-# This reverses the order of its input lines.
-# From the GNU sed manual.
-tac () {
- sed -n '1! G
-$ p
-h'
-}
-
-test_sort () {
- cat > sort-variables.sps <<EOF
-DATA LIST FREE/$1.
-$2
-SORT VARIABLES $3.
-DISPLAY NAMES.
-SORT VARIABLES $3(D).
-DISPLAY NAMES.
-EOF
- AT_CHECK_UNQUOTED([pspp -O format=csv sort-variables.sps], [0],
-[Table: Variables
-Name
-`for var in $4; do echo $var; done`
-
-Table: Variables
-Name
-`for var in $4; do echo $var; done | tac`
-])
-}
-
-test_sort 'x100 c b x99 a y400 y5' '' NAME 'a b c x99 x100 y5 y400'
-test_sort 'c(a10) a(a5) b' '' TYPE 'b a c'
-test_sort 'a (datetime) b (f) c (a5) d (a2) e (a1)' '' FORMAT 'e d c b a'
-test_sort 'a b c' \
- 'VARIABLE LABEL a "hi there".' \
- LABEL 'b c a'
-test_sort 'a b c' \
- 'VALUE LABELS a 123 "xyzzy".' \
- VALUES 'b c a'
-test_sort 'a b c' \
- 'MISSING VALUES a (123).' \
- MISSING 'b c a'
-test_sort 'a b c' \
- 'VARIABLE LEVEL a (SCALE) b (ORDINAL) c (NOMINAL).' \
- MEASURE 'c b a'
-test_sort 'b n i s t p' \
- 'VARIABLE ROLE /INPUT i /TARGET t /BOTH b /NONE n /PARTITION p /SPLIT s.' \
- ROLE 'i t b n p s'
-test_sort 'c10 c5 c15 c9' \
- 'VARIABLE WIDTH c10(10) c5(5) c15(15) c9(9).' \
- COLUMNS 'c5 c9 c10 c15'
-test_sort 'c l r' \
- 'VARIABLE ALIGNMENT c (CENTER) l (LEFT) r (RIGHT).' \
- ALIGNMENT 'l r c'
-test_sort 'az ax ay ab' \
- 'VARIABLE ATTRIBUTE VARIABLES=az ATTRIBUTE=key("z").
- VARIABLE ATTRIBUTE VARIABLES=ax ATTRIBUTE=key("x").
- VARIABLE ATTRIBUTE VARIABLES=ay ATTRIBUTE=key("y").
- VARIABLE ATTRIBUTE VARIABLES=ab ATTRIBUTE=key("b").' \
- 'ATTRIBUTE key' 'ab ax ay az'
-AT_CLEANUP
-
-AT_SETUP([SORT VARIABLES syntax errors])
-AT_DATA([sort-variables.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-SORT VARIABLES BY **.
-SORT VARIABLES BY ATTRIBUTE **.
-SORT VARIABLES BY NAME (**).
-SORT VARIABLES BY NAME (A **).
-])
-AT_CHECK([pspp -O format=csv sort-variables.sps], [1], [dnl
-"sort-variables.sps:2.19-2.20: error: SORT VARIABLES: Syntax error expecting one of the following: NAME, TYPE, FORMAT, LABEL, VALUES, MISSING, MEASURE, ROLE, COLUMNS, ALIGNMENT, ATTRIBUTE.
- 2 | SORT VARIABLES BY **.
- | ^~"
-
-"sort-variables.sps:3.29-3.30: error: SORT VARIABLES: Syntax error expecting identifier.
- 3 | SORT VARIABLES BY ATTRIBUTE **.
- | ^~"
-
-"sort-variables.sps:4.25-4.26: error: SORT VARIABLES: Syntax error expecting A or D.
- 4 | SORT VARIABLES BY NAME (**).
- | ^~"
-
-"sort-variables.sps:5.27-5.28: error: SORT VARIABLES: Syntax error expecting `)'.
- 5 | SORT VARIABLES BY NAME (A **).
- | ^~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SPLIT FILE])
-
-AT_SETUP([SPLIT FILE - basic test])
-AT_DATA([split-file.sps], [dnl
-title 'Test SPLIT FILE utility'.
-
-data list notable /X 1 Y 2.
-begin data.
-12
-16
-17
-19
-15
-14
-27
-20
-26
-25
-28
-29
-24
-end data.
-split file by x.
-list.
-])
-AT_CHECK([pspp -o pspp.csv split-file.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Split Values
-Variable,Value
-X,1
-
-Table: Data List
-X,Y
-1,2
-1,6
-1,7
-1,9
-1,5
-1,4
-
-Table: Split Values
-Variable,Value
-X,2
-
-Table: Data List
-X,Y
-2,7
-2,0
-2,6
-2,5
-2,8
-2,9
-2,4
-])
-AT_CLEANUP
-
-AT_SETUP([SPLIT FILE - vs procedures])
-AT_DATA([split-file.sps], [dnl
-
-* This test is a compendium of those procedures which might
-* have problems when run in conjunction with SPLITS.
-
-data list list /a b c q g *.
-begin data.
-1 2 3 1 0
-4 5 6 0 0
-7 8 9 1 0
-10 11 12 0 1
-13 14 15 1 1
-end data.
-
-split file by g.
-
-
-* The commented out lines are ones which currently fail.
-
-AGGREGATE outfile='foo' /break=c /X=sum(a).
-AUTORECODE variables = c into d .
-COUNT e = c (1 2 3 4 5 6 7).
-CROSSTABS a by b.
-CORRELATIONS /VARIABLES = a b.
-DELETE VARIABLES a.
-DESCRIPTIVES e .
-EXAMINE c by b.
-EXPORT outfile='xxx'.
-FACTOR /VARIABLES = b c d.
-FILTER BY c.
-FREQUENCIES b.
-GLM c BY b.
-GRAPH /HISTOGRAM = b .
-GRAPH /SCATTERPLOT(BIVARIATE) = b with c by e .
-*GRAPH /BAR (GROUPED) = MEAN(b) by c by e.
-GRAPH /BAR = COUNT BY b.
-LIST.
-LOGISTIC REGRESSION q WITH b.
-MEANS c b.
-NPAR TESTS /MCNEMAR q.
-ONEWAY c BY b.
-QUICK CLUSTER b c.
-RANK b c.
-REGRESSION /VARIABLES = c /DEPENDENT = q.
-RELIABILITY /VARIABLES = c b d.
-RENAME VARIABLES (b = bb).
-ROC bb by q(1).
-SAMPLE 0.9 .
-SAVE outfile='xx.sav'.
-SORT CASES by bb.
-T-TEST /GROUP=q(0,1) /VARIABLES=bb.
-USE ALL.
-FLIP /VARIABLES = bb, c .
-
-execute.
-finish.
-])
-
-AT_CHECK([pspp -O format=csv split-file.sps], [0],[ignore])
-
-AT_CLEANUP
-
-AT_SETUP([SPLIT FILE - split variable limit])
-AT_DATA([split-file.sps], [dnl
-DATA LIST LIST NOTABLE /V1 TO V9.
-SPLIT FILE BY V1 TO V9.
-])
-AT_CHECK([pspp split-file.sps], [1], [dnl
-split-file.sps:2.15-2.22: error: SPLIT FILE: At most 8 split variables may be
-specified.
- 2 | SPLIT FILE BY V1 TO V9.
- | ^~~~~~~~
-])
-AT_CLEANUP
+++ /dev/null
-AT_BANNER([STRING])
-
-AT_SETUP([STRING])
-AT_DATA([string.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-STRING s1 (A8)/s2 (A1).
-DISPLAY DICTIONARY.
-])
-AT_CHECK([pspp -O format=csv string.sps], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-x,1,Unknown,Input,8,Right,F8.2,F8.2
-y,2,Unknown,Input,8,Right,F8.2,F8.2
-z,3,Unknown,Input,8,Right,F8.2,F8.2
-s1,4,Nominal,Input,8,Left,A8,A8
-s2,5,Nominal,Input,1,Left,A1,A1
-])
-AT_CLEANUP
-
-AT_SETUP([STRING syntax errors])
-AT_DATA([string.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-STRING **.
-STRING s **.
-STRING s (**).
-STRING s (F8).
-STRING s (AHEX1).
-STRING s (A8 **).
-STRING x (A8).
-])
-AT_CHECK([pspp -O format=csv string.sps], [1], [dnl
-"string.sps:2.8-2.9: error: STRING: Syntax error expecting variable name.
- 2 | STRING **.
- | ^~"
-
-"string.sps:3.10-3.11: error: STRING: Syntax error expecting `('.
- 3 | STRING s **.
- | ^~"
-
-"string.sps:4.11-4.12: error: STRING: Syntax error expecting valid format specifier.
- 4 | STRING s (**).
- | ^~"
-
-"string.sps:5.11-5.12: error: STRING: String variables are not compatible with numeric format F8.0.
- 5 | STRING s (F8).
- | ^~"
-
-"string.sps:6.11-6.15: error: STRING: Output format AHEX1 specifies width 1, but AHEX requires an even width.
- 6 | STRING s (AHEX1).
- | ^~~~~"
-
-"string.sps:7.14-7.15: error: STRING: Syntax error expecting `)'.
- 7 | STRING s (A8 **).
- | ^~"
-
-"string.sps:8.8: error: STRING: There is already a variable named x.
- 8 | STRING x (A8).
- | ^"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SYSFILE INFO])
-
-AT_SETUP([SYSFILE INFO])
-AT_DATA([sysfile-info.sps], [dnl
-DATA LIST LIST /x * name (a10) .
-BEGIN DATA
-1 one
-2 two
-3 three
-END DATA.
-DOCUMENT A document.
-SAVE OUTFILE='pro.sav'.
-
-sysfile info file='pro.sav'.
-])
-AT_CHECK([pspp -o pspp.csv sysfile-info.sps])
-AT_CHECK(
- [sed -e '/^Created,/d' \
- -e '/^Endian,/d' \
- -e '/^Integer Format,/d' \
- -e '/^Real Format,/d' \
- -e '/^Encoding,/d' \
- -e 's/(Entered.*)/(Entered <date>)/' pspp.csv],
- [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-name,A10
-
-Table: File Information
-File,pro.sav
-Variables,2
-Cases,3
-Type,SPSS System File
-Weight,Not weighted
-Compression,SAV
-Documents,"DOCUMENT A document.
- (Entered <date>)"
-
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-x,1,Nominal,Input,8,Right,F8.2,F8.2
-name,2,Nominal,Input,10,Left,A10,A10
-])
-AT_CLEANUP
-
-AT_BANNER([DISPLAY])
-
-dnl DISPLAY DOCUMENTS is tested with commands for documents.
-
-AT_SETUP([DISPLAY FILE LABEL])
-AT_DATA([display.sps], [dnl
-DATA LIST LIST NOTABLE /x * name (a10) .
-
-DISPLAY FILE LABEL.
-
-FILE LABEL 'foo bar baz quux'.
-DISPLAY FILE LABEL.
-])
-AT_CHECK([pspp -O format=csv display.sps], [0], [dnl
-Table: File Label
-Label,(none)
-
-Table: File Label
-Label,foo bar baz quux
-])
-AT_CLEANUP
-
-dnl DISPLAY VECTORS is tested with commands for vectors.
-
-dnl DISPLAY ATTRIBUTES and @ATTRIBUTES are tested with commands for attributes.
-
-AT_SETUP([DISPLAY SCRATCH])
-AT_DATA([sysfile-info.sps], [dnl
-DATA LIST LIST NOTABLE /x * name (a10) .
-DISPLAY SCRATCH.
-COMPUTE #x=0.
-DISPLAY SCRATCH.
-])
-AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
-sysfile-info.sps:2: note: DISPLAY: No variables to display.
-
-Table: Variables
-Name
-#x
-])
-AT_CLEANUP
-
-AT_SETUP([DISPLAY INDEX])
-AT_DATA([sysfile-info.sps], [dnl
-DATA LIST LIST NOTABLE /x * name (a10) .
-DISPLAY INDEX.
-])
-AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
-Table: Variables
-Name,Position
-x,1
-name,2
-])
-AT_CLEANUP
-
-AT_SETUP([DISPLAY NAMES])
-AT_DATA([sysfile-info.sps], [dnl
-DATA LIST LIST NOTABLE /x * name (a10) .
-DISPLAY NAMES.
-])
-AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
-Table: Variables
-Name
-x
-name
-])
-AT_CLEANUP
-
-AT_SETUP([DISPLAY LABELS])
-AT_DATA([sysfile-info.sps], [dnl
-DATA LIST LIST NOTABLE /x * name (a10) .
-VARIABLE LABEL x 'variable one' name 'variable two'.
-VALUE LABEL x 1 'asdf' 2 'jkl;'.
-DISPLAY LABELS.
-])
-AT_CHECK([pspp -O format=csv sysfile-info.sps], [0], [dnl
-Table: Variables
-Name,Position,Label
-x,1,variable one
-name,2,variable two
-])
-AT_CLEANUP
-
-dnl DISPLAY VARIABLES Is tested in multiple places.
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([VALUE LABELS])
-
-AT_SETUP([VALUE LABELS date formats])
-AT_DATA([value-labels.sps], [dnl
-DATA LIST LIST NOTABLE /ad (adate10) dt (datetime20).
-VALUE LABELS ad 'july 10, 1982' 'label 1'
- '1-2-93' 'label 2'
- '5-4-2003' 'label 3'
- /dt '12 Apr 2011 06:09:56' 'label 4'.
-DISPLAY DICTIONARY.
-])
-AT_CHECK([pspp -O format=csv value-labels.sps], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-ad,1,Unknown,Input,8,Right,ADATE10,ADATE10
-dt,2,Unknown,Input,8,Right,DATETIME20.0,DATETIME20.0
-
-Table: Value Labels
-Variable Value,,Label
-ad,07/10/1982,label 1
-,01/02/1993,label 2
-,05/04/2003,label 3
-dt,12-APR-2011 06:09:56,label 4
-])
-AT_CLEANUP
-
-AT_SETUP([VALUE LABELS with new-line])
-AT_DATA([value-labels.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-VALUE LABELS x 1 'one' 2 'first line\nsecond line' 3 'three'.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-DISPLAY DICTIONARY.
-FREQUENCIES x/STAT=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt value-labels.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-x,1,Nominal,Input,8,Right,F8.2,F8.2
-
-Table: Value Labels
-Variable Value,,Label
-x,1.00,one
-,2.00,first line\nsecond line
-,3.00,three
-
-Table: x
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,one,1,33.3%,33.3%,33.3%
-,"first line
-second line",1,33.3%,33.3%,66.7%
-,three,1,33.3%,33.3%,100.0%
-Total,,3,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([VALUE LABELS with new-line in system file])
-AT_DATA([save.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-VALUE LABELS x 1 'one' 2 'first line\nsecond line' 3 'three'.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-SAVE OUTFILE='value-labels.sav'.
-])
-AT_CHECK([pspp -O format=csv save.sps])
-AT_DATA([get.sps], [dnl
-GET FILE='value-labels.sav'.
-DISPLAY DICTIONARY.
-FREQUENCIES x/STAT=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt get.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-x,1,Nominal,Input,8,Right,F8.2,F8.2
-
-Table: Value Labels
-Variable Value,,Label
-x,1.00,one
-,2.00,first line\nsecond line
-,3.00,three
-
-Table: x
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,one,1,33.3%,33.3%,33.3%
-,"first line
-second line",1,33.3%,33.3%,66.7%
-,three,1,33.3%,33.3%,100.0%
-Total,,3,100.0%,,
-])
-AT_CLEANUP
-
-dnl Tests for a bug which caused VALUE LABELS to
-dnl crash when given invalid syntax.
-AT_SETUP([VALUE LABELS invalid syntax bug])
-AT_DATA([value-labels.sps], [dnl
-DATA LIST LIST NOTABLE /a * pref * .
-BEGIN DATA.
- 1.00 1.00
- 1.00 2.00
- 2.00 1.00
- 2.00 2.00
-END DATA.
-
-VALUE LABELS /var=a 'label for a'.
-])
-AT_CHECK([pspp -O format=csv value-labels.sps], [1], [dnl
-"value-labels.sps:9.15-9.17: error: VALUE LABELS: var is not a variable name.
- 9 | VALUE LABELS /var=a 'label for a'.
- | ^~~"
-])
-AT_CLEANUP
-
-# Tests for a bug which caused a crash if VALUE LABELS had a trailing /.
-AT_SETUP([VALUE LABELS trailing `/' bug])
-AT_DATA([value-labels.sps], [dnl
-DATA LIST LIST NOTABLE /X * .
-BEGIN DATA.
-1
-2
-3
-4
-END DATA.
-
-
-VALUE LABELS X 1 'one' 2 'two' 3 'three'/
-
-
-LIST VARIABLES=X.
-])
-AT_CHECK([pspp -O format=csv value-labels.sps], [0], [dnl
-Table: Data List
-X
-1.00
-2.00
-3.00
-4.00
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([variable display attributes])
-
-AT_SETUP([variable display attribute commands])
-AT_KEYWORDS([VARIABLE ALIGNMENT])
-AT_KEYWORDS([VARIABLE WIDTH])
-AT_KEYWORDS([VARIABLE LEVEL])
-AT_KEYWORDS([VARIABLE ROLE])
-AT_DATA([var-display.sps], [dnl
-DATA LIST FREE /x y z.
-VARIABLE ALIGNMENT x (LEFT)/y (RIGHT)/z (CENTER).
-VARIABLE WIDTH x (10)/y (12)/z (14).
-VARIABLE LEVEL x (SCALE)/y (ORDINAL)/z (NOMINAL).
-VARIABLE ROLE /TARGET x /BOTH y /NONE z.
-DISPLAY DICTIONARY.
-])
-AT_CHECK([pspp -o pspp.csv var-display.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-x,1,Scale,Output,10,Left,F8.2,F8.2
-y,2,Ordinal,Both,12,Right,F8.2,F8.2
-z,3,Nominal,None,14,Center,F8.2,F8.2
-])
-AT_CLEANUP
-
-AT_SETUP([variable display attribute syntax errors])
-AT_KEYWORDS([VARIABLE ALIGNMENT])
-AT_KEYWORDS([VARIABLE WIDTH])
-AT_KEYWORDS([VARIABLE LEVEL])
-AT_KEYWORDS([VARIABLE ROLE])
-AT_DATA([var-display.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-VARIABLE ALIGNMENT **.
-VARIABLE ALIGNMENT x **.
-VARIABLE ALIGNMENT x (**).
-VARIABLE ALIGNMENT x (LEFT **).
-VARIABLE WIDTH **.
-VARIABLE WIDTH x **.
-VARIABLE WIDTH x (**).
-VARIABLE WIDTH x (10 **).
-VARIABLE LEVEL **.
-VARIABLE LEVEL x **.
-VARIABLE LEVEL x (**).
-VARIABLE LEVEL x (SCALE **).
-VARIABLE ROLE **.
-VARIABLE ROLE / **.
-VARIABLE ROLE /INPUT **.
-VARIABLE ROLE /INPUT x **.
-])
-AT_CHECK([pspp -O format=csv var-display.sps], [1], [dnl
-"var-display.sps:2.20-2.21: error: VARIABLE ALIGNMENT: Syntax error expecting variable name.
- 2 | VARIABLE ALIGNMENT **.
- | ^~"
-
-"var-display.sps:3.22-3.23: error: VARIABLE ALIGNMENT: Syntax error expecting `('.
- 3 | VARIABLE ALIGNMENT x **.
- | ^~"
-
-"var-display.sps:4.23-4.24: error: VARIABLE ALIGNMENT: Syntax error expecting LEFT, RIGHT, or CENTER.
- 4 | VARIABLE ALIGNMENT x (**).
- | ^~"
-
-"var-display.sps:5.28-5.29: error: VARIABLE ALIGNMENT: Syntax error expecting `)'.
- 5 | VARIABLE ALIGNMENT x (LEFT **).
- | ^~"
-
-"var-display.sps:6.16-6.17: error: VARIABLE WIDTH: Syntax error expecting variable name.
- 6 | VARIABLE WIDTH **.
- | ^~"
-
-"var-display.sps:7.18-7.19: error: VARIABLE WIDTH: Syntax error expecting `('.
- 7 | VARIABLE WIDTH x **.
- | ^~"
-
-"var-display.sps:8.19-8.20: error: VARIABLE WIDTH: Syntax error expecting positive integer.
- 8 | VARIABLE WIDTH x (**).
- | ^~"
-
-"var-display.sps:9.22-9.23: error: VARIABLE WIDTH: Syntax error expecting `)'.
- 9 | VARIABLE WIDTH x (10 **).
- | ^~"
-
-"var-display.sps:10.16-10.17: error: VARIABLE LEVEL: Syntax error expecting variable name.
- 10 | VARIABLE LEVEL **.
- | ^~"
-
-"var-display.sps:11.18-11.19: error: VARIABLE LEVEL: Syntax error expecting `('.
- 11 | VARIABLE LEVEL x **.
- | ^~"
-
-"var-display.sps:12.19-12.20: error: VARIABLE LEVEL: Syntax error expecting SCALE, ORDINAL, or NOMINAL.
- 12 | VARIABLE LEVEL x (**).
- | ^~"
-
-"var-display.sps:13.25-13.26: error: VARIABLE LEVEL: Syntax error expecting `)'.
- 13 | VARIABLE LEVEL x (SCALE **).
- | ^~"
-
-"var-display.sps:14.15-14.16: error: VARIABLE ROLE: Syntax error expecting `/'.
- 14 | VARIABLE ROLE **.
- | ^~"
-
-"var-display.sps:15.17-15.18: error: VARIABLE ROLE: Syntax error expecting INPUT, TARGET, BOTH, NONE, PARTITION, or SPLIT.
- 15 | VARIABLE ROLE / **.
- | ^~"
-
-"var-display.sps:16.22-16.23: error: VARIABLE ROLE: Syntax error expecting variable name.
- 16 | VARIABLE ROLE /INPUT **.
- | ^~"
-
-"var-display.sps:17.24-17.25: error: VARIABLE ROLE: Syntax error expecting `/'.
- 17 | VARIABLE ROLE /INPUT x **.
- | ^~"
-])
-AT_CLEANUP
-
-AT_SETUP([variable level inference and SCALEMIN])
-AT_DATA([var-level.sps], [dnl
-DATA LIST LIST NOTABLE /n1 to n3 s1 to s5.
-
-* Nominal formats (copied from data that will default to scale).
-COMPUTE n4=s1.
-COMPUTE n5=s1.
-FORMATS n4(WKDAY5) n5(MONTH5).
-
-* Scale formats (copied from data that will default to nominal).
-COMPUTE s6=n1.
-COMPUTE s7=n1.
-COMPUTE s8=n1.
-FORMATS s6(DOLLAR6.2) s7(CCA8.2) s8(DATETIME17).
-
-STRING string(A8).
-DISPLAY DICTIONARY.
-EXECUTE.
-
-* n1 has 10 unique small values -> nominal.
-* n2 has 23 unique small values -> nominal.
-* n3 is all missing -> nominal.
-* s1 has 24 unique small values -> scale.
-* s2 has one negative value -> scale.
-* s3 has one non-integer value -> scale.
-* s4 has no valid values less than 10 -> scale.
-* s5 has no valid values less than 10,000 -> scale.
-BEGIN DATA.
-1 1 . 1 1 1 10 10001
-2 2 . 2 2 2 11 10002
-3 3 . 3 3 3 12 10003
-4 4 . 4 4 4 13 10004
-5 5 . 5 5 5 14 10005
-6 6 . 6 6 6 15 10006
-7 7 . 7 7 7 16 10007
-8 8 . 8 8 8 17 10008
-9 9 . 9 9 9 18 10009
-10 10 . 10 10 10.5 19 110000
-1 11 . 11 -1 1 11 10001
-2 12 . 12 2 2 12 10002
-3 13 . 13 3 3 13 10003
-4 14 . 14 4 4 14 10004
-5 15 . 15 5 5 15 10005
-6 16 . 16 6 6 16 10006
-7 17 . 17 7 7 17 10007
-8 18 . 18 8 8 18 10008
-9 19 . 19 9 9 19 10009
-1 20 . 20 1 1 20 10001
-2 21 . 21 2 2 21 10002
-3 22 . 22 3 3 22 10003
-4 23 . 23 4 4 23 10004
-5 23 . 24 5 5 24 10005
-6 23 . 24 6 6 25 10006
-END DATA.
-DISPLAY DICTIONARY.
-])
-AT_CHECK([pspp -o pspp.csv var-level.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-n1,1,Unknown,Input,8,Right,F8.2,F8.2
-n2,2,Unknown,Input,8,Right,F8.2,F8.2
-n3,3,Unknown,Input,8,Right,F8.2,F8.2
-s1,4,Unknown,Input,8,Right,F8.2,F8.2
-s2,5,Unknown,Input,8,Right,F8.2,F8.2
-s3,6,Unknown,Input,8,Right,F8.2,F8.2
-s4,7,Unknown,Input,8,Right,F8.2,F8.2
-s5,8,Unknown,Input,8,Right,F8.2,F8.2
-n4,9,Unknown,Input,8,Right,WKDAY5,WKDAY5
-n5,10,Unknown,Input,8,Right,MONTH5,MONTH5
-s6,11,Unknown,Input,8,Right,DOLLAR6.2,DOLLAR6.2
-s7,12,Unknown,Input,8,Right,CCA8.2,CCA8.2
-s8,13,Unknown,Input,8,Right,DATETIME17.0,DATETIME17.0
-string,14,Nominal,Input,8,Left,A8,A8
-
-Table: Variables
-Name,Position,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-n1,1,Nominal,Input,8,Right,F8.2,F8.2
-n2,2,Nominal,Input,8,Right,F8.2,F8.2
-n3,3,Nominal,Input,8,Right,F8.2,F8.2
-s1,4,Scale,Input,8,Right,F8.2,F8.2
-s2,5,Scale,Input,8,Right,F8.2,F8.2
-s3,6,Scale,Input,8,Right,F8.2,F8.2
-s4,7,Scale,Input,8,Right,F8.2,F8.2
-s5,8,Scale,Input,8,Right,F8.2,F8.2
-n4,9,Nominal,Input,8,Right,WKDAY5,WKDAY5
-n5,10,Nominal,Input,8,Right,MONTH5,MONTH5
-s6,11,Scale,Input,8,Right,DOLLAR6.2,DOLLAR6.2
-s7,12,Scale,Input,8,Right,CCA8.2,CCA8.2
-s8,13,Scale,Input,8,Right,DATETIME17.0,DATETIME17.0
-string,14,Nominal,Input,8,Left,A8,A8
-])
-AT_CLEANUP
-
-AT_BANNER([VARIABLE LABELS])
-
-AT_SETUP([variable labels])
-
-dnl The following test is to make sure the TVARS command works and that
-dnl variables are displayed accordingly.
-AT_DATA([var-labels.sps], [dnl
-DATA LIST LIST NOTABLE /x * y *.
-BEGIN DATA.
-1 100
-2 200
-3 300
-4 400
-END DATA.
-
-* While no labels have been set, the TVARS is irrelevant.
-SET TVARS=NAMES.
-DESCRIPTIVES ALL.
-
-SET TVARS=LABELS.
-DESCRIPTIVES ALL.
-
-SET TVARS=BOTH.
-DESCRIPTIVES ALL.
-
-VARIABLE LABEL x 'foo' y 'bar'.
-
-* Now, the TVARS setting should have effect
-
-SET TVARS=NAMES.
-DESCRIPTIVES ALL.
-
-SET TVARS=LABELS.
-DESCRIPTIVES ALL.
-
-SET TVARS=BOTH.
-DESCRIPTIVES ALL.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt var-labels.sps])
-AT_CHECK([cat pspp.csv], [0],[dnl
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-x,4,2.50,1.29,1.00,4.00
-y,4,250.00,129.10,100.00,400.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-x,4,2.50,1.29,1.00,4.00
-y,4,250.00,129.10,100.00,400.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-x,4,2.50,1.29,1.00,4.00
-y,4,250.00,129.10,100.00,400.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-x,4,2.50,1.29,1.00,4.00
-y,4,250.00,129.10,100.00,400.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-foo,4,2.50,1.29,1.00,4.00
-bar,4,250.00,129.10,100.00,400.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-x foo,4,2.50,1.29,1.00,4.00
-y bar,4,250.00,129.10,100.00,400.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([VECTOR])
-
-AT_SETUP([VECTOR short form])
-AT_DATA([vector.sps], [dnl
-data list notable/x 1.
-vector v(4).
-display vector.
-])
-AT_CHECK([pspp -o pspp.csv vector.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Vectors
-Vector and Position,,Variable,Print Format
-v,1,v1,F8.2
-,2,v2,F8.2
-,3,v3,F8.2
-,4,v4,F8.2
-])
-AT_CLEANUP
-
-AT_SETUP([VECTOR short form with format specification])
-AT_DATA([vector.sps], [dnl
-data list notable/x 1.
-vector #vec(4, comma10.2)
- /#svec(3, a8).
-display vector.
-])
-AT_CHECK([pspp -o pspp.csv vector.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Vectors
-Vector and Position,,Variable,Print Format
-#vec,1,#vec1,COMMA10.2
-,2,#vec2,COMMA10.2
-,3,#vec3,COMMA10.2
-,4,#vec4,COMMA10.2
-#svec,1,#svec1,A8
-,2,#svec2,A8
-,3,#svec3,A8
-])
-AT_CLEANUP
-
-AT_SETUP([VECTOR short form in INPUT PROGRAM])
-AT_DATA([vector.sps], [dnl
-input program.
-vector x(5).
-data list notable/x5 x2 x3 x1 x4 1-5.
-end input program.
-display vector.
-])
-AT_CHECK([pspp -o pspp.csv vector.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Vectors
-Vector and Position,,Variable,Print Format
-x,1,x1,F8.2
-,2,x2,F8.2
-,3,x3,F8.2
-,4,x4,F8.2
-,5,x5,F8.2
-])
-AT_CLEANUP
-
-AT_SETUP([VECTOR long form])
-AT_DATA([vector.sps], [dnl
-data list notable/u w x y z 1-5.
-vector a=u to y.
-vector b=x to z.
-vector c=all.
-display vector.
-])
-AT_CHECK([pspp -o pspp.csv vector.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Vectors
-Vector and Position,,Variable,Print Format
-a,1,u,F1.0
-,2,w,F1.0
-,3,x,F1.0
-,4,y,F1.0
-b,1,x,F1.0
-,2,y,F1.0
-,3,z,F1.0
-c,1,u,F1.0
-,2,w,F1.0
-,3,x,F1.0
-,4,y,F1.0
-,5,z,F1.0
-])
-AT_CLEANUP
-
-AT_SETUP([VECTOR syntax errors])
-AT_DATA([vector.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-VECTOR **.
-VECTOR aslkdfjaklsdjfklasdjfklasjdfklasjdfkajsdlkfajsdkfjaksdjfaklsdkasdjfklasdjfklasjdfkldkl.
-VECTOR dup=x y z.
-VECTOR dup.
-VECTOR v v.
-VECTOR u v=x y z.
-VECTOR v(1, 2).
-VECTOR v(0).
-VECTOR v(F8.2, F8.2).
-VECTOR v(asdf).
-VECTOR v(**).
-VECTOR v(F8.2).
-VECTOR xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(5).
-VECTOR v **.
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='vector.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"vector.sps:2.8-2.9: error: VECTOR: Syntax error expecting identifier.
- 2 | VECTOR **.
- | ^~"
-
-"vector.sps:3.8-3.93: error: VECTOR: Identifier `aslkdfjaklsdjfklasdjfklasjdfklasjdfkajsdlkfajsdkfjaksdjfaklsdkasdjfklasdjfklasjdfkldkl' exceeds 64-byte limit.
- 3 | VECTOR aslkdfjaklsdjfklasdjfklasjdfklasjdfkajsdlkfajsdkfjaksdjfaklsdkasdjfklasdjfklasjdfkldkl.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"vector.sps:5.8-5.10: error: VECTOR: A vector named dup already exists.
- 5 | VECTOR dup.
- | ^~~"
-
-"vector.sps:6.8-6.10: error: VECTOR: Vector name v is given twice.
- 6 | VECTOR v v.
- | ^~~"
-
-"vector.sps:7.8-7.11: error: VECTOR: Only a single vector name may be specified when a list of variables is given.
- 7 | VECTOR u v=x y z.
- | ^~~~"
-
-"vector.sps:8.9-8.13: error: VECTOR: Vector length may only be specified once.
- 8 | VECTOR v(1, 2).
- | ^~~~~"
-
-"vector.sps:9.10: error: VECTOR: Syntax error expecting positive integer.
- 9 | VECTOR v(0).
- | ^"
-
-"vector.sps:10.9-10.19: error: VECTOR: Only one format may be specified.
- 10 | VECTOR v(F8.2, F8.2).
- | ^~~~~~~~~~~"
-
-"vector.sps:11.10-11.13: error: VECTOR: Unknown format type `asdf'.
- 11 | VECTOR v(asdf).
- | ^~~~"
-
-"vector.sps:12.10-12.11: error: VECTOR: Syntax error expecting vector length or format.
- 12 | VECTOR v(**).
- | ^~"
-
-"vector.sps:13.9-13.14: error: VECTOR: Vector length is required.
- 13 | VECTOR v(F8.2).
- | ^~~~~~"
-
-"vector.sps:14.8-14.74: error: VECTOR: Identifier `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx1' exceeds 64-byte limit.
- 14 | VECTOR xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(5).
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-"vector.sps:15.10-15.11: error: VECTOR: Syntax error expecting `=' or `@{:@'.
- 15 | VECTOR v **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([WEIGHT])
-
-AT_SETUP([WEIGHT])
-AT_DATA([weight.txt], [dnl
- 18 1
- 19 7
- 20 26
- 21 76
- 22 57
- 23 58
- 24 38
- 25 38
- 26 30
- 27 21
- 28 23
- 29 24
- 30 23
- 31 14
- 32 21
- 33 21
- 34 14
- 35 14
- 36 17
- 37 11
- 38 16
- 39 14
- 40 15
- 41 14
- 42 14
- 43 8
- 44 15
- 45 10
- 46 12
- 47 13
- 48 13
- 49 5
- 50 5
- 51 3
- 52 7
- 53 6
- 54 2
- 55 2
- 56 2
- 57 3
- 58 1
- 59 3
- 61 1
- 62 3
- 63 1
- 64 1
- 65 2
- 70 1
- 78 1
- 79 1
- 80 1
- 94 1
-])
-AT_DATA([weight.sps], [dnl
-SET FORMAT F8.3.
-data list file='weight.txt'/AVAR 1-5 BVAR 6-10.
-weight by BVAR.
-
-descriptives AVAR /statistics all /format serial.
-frequencies AVAR /statistics all.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt weight.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading 1 record from `weight.txt'.
-Variable,Record,Columns,Format
-AVAR,1,1-5,F5.0
-BVAR,1,6-10,F5.0
-
-Table: Descriptive Statistics
-,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
-AVAR,730,31.515,.405,10.937,119.608,2.411,.181,1.345,.090,76.000,18,94,23006.00
-Valid N (listwise),730,,,,,,,,,,,,
-Missing N (listwise),0,,,,,,,,,,,,
-
-Table: Statistics
-,,AVAR
-N,Valid,730
-,Missing,0
-Mean,,31.515
-S.E. Mean,,.405
-Median,,28.000
-Mode,,21
-Std Dev,,10.937
-Variance,,119.608
-Kurtosis,,2.411
-S.E. Kurt,,.181
-Skewness,,1.345
-S.E. Skew,,.090
-Range,,76.000
-Minimum,,18
-Maximum,,94
-Sum,,23006.00
-
-Table: AVAR
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,18,1,.1%,.1%,.1%
-,19,7,1.0%,1.0%,1.1%
-,20,26,3.6%,3.6%,4.7%
-,21,76,10.4%,10.4%,15.1%
-,22,57,7.8%,7.8%,22.9%
-,23,58,7.9%,7.9%,30.8%
-,24,38,5.2%,5.2%,36.0%
-,25,38,5.2%,5.2%,41.2%
-,26,30,4.1%,4.1%,45.3%
-,27,21,2.9%,2.9%,48.2%
-,28,23,3.2%,3.2%,51.4%
-,29,24,3.3%,3.3%,54.7%
-,30,23,3.2%,3.2%,57.8%
-,31,14,1.9%,1.9%,59.7%
-,32,21,2.9%,2.9%,62.6%
-,33,21,2.9%,2.9%,65.5%
-,34,14,1.9%,1.9%,67.4%
-,35,14,1.9%,1.9%,69.3%
-,36,17,2.3%,2.3%,71.6%
-,37,11,1.5%,1.5%,73.2%
-,38,16,2.2%,2.2%,75.3%
-,39,14,1.9%,1.9%,77.3%
-,40,15,2.1%,2.1%,79.3%
-,41,14,1.9%,1.9%,81.2%
-,42,14,1.9%,1.9%,83.2%
-,43,8,1.1%,1.1%,84.2%
-,44,15,2.1%,2.1%,86.3%
-,45,10,1.4%,1.4%,87.7%
-,46,12,1.6%,1.6%,89.3%
-,47,13,1.8%,1.8%,91.1%
-,48,13,1.8%,1.8%,92.9%
-,49,5,.7%,.7%,93.6%
-,50,5,.7%,.7%,94.2%
-,51,3,.4%,.4%,94.7%
-,52,7,1.0%,1.0%,95.6%
-,53,6,.8%,.8%,96.4%
-,54,2,.3%,.3%,96.7%
-,55,2,.3%,.3%,97.0%
-,56,2,.3%,.3%,97.3%
-,57,3,.4%,.4%,97.7%
-,58,1,.1%,.1%,97.8%
-,59,3,.4%,.4%,98.2%
-,61,1,.1%,.1%,98.4%
-,62,3,.4%,.4%,98.8%
-,63,1,.1%,.1%,98.9%
-,64,1,.1%,.1%,99.0%
-,65,2,.3%,.3%,99.3%
-,70,1,.1%,.1%,99.5%
-,78,1,.1%,.1%,99.6%
-,79,1,.1%,.1%,99.7%
-,80,1,.1%,.1%,99.9%
-,94,1,.1%,.1%,100.0%
-Total,,730,100.0%,,
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([AGGREGATE procedure])
-
-dnl CHECK_AGGREGATE(OUTFILE, SORT, MISSING)
-dnl
-dnl Checks the AGGREGATE procedure with the specified combination of:
-dnl
-dnl - OUTFILE: One of "dataset", "active", or "external" according to
-dnl where AGGREGATE's output should be directed.
-dnl
-dnl - SORT: Either "presorted" or "unsorted" according to whether
-dnl AGGREGATE should received presorted input.
-dnl
-dnl - MISSING: Either "itemwise" or "columnwise" according to the basis
-dnl on which missing values should be eliminated.
-dnl
-m4_define([CHECK_AGGREGATE], [
- AT_SETUP([AGGREGATE $2 data to $1 file, $3 missing])
- AT_DATA([aggregate.data],
- [2 42
-1001
-4 41
-3112
-1112
-2661
-1221
-2771
-1331
-1441
-2881
-1551
-])
- AT_DATA([aggregate.sps],
- [DATA LIST NOTABLE FILE='aggregate.data' /G N 1-2 S 3(a) W 4.
-WEIGHT BY w.
-MISSING VALUES n(4) s('4').
-m4_if([$1], [dataset], [DATASET DECLARE aggregate.])
-m4_if([$2], [presorted], [SORT CASES BY g.])
-AGGREGATE dnl
-m4_if([$1], [active], [OUTFILE=*],
- [$1], [external], [OUTFILE='aggregate.sys'],
- [outfile=aggregate]) dnl
-m4_if([$3], [columnwise], [/MISSING=COLUMNWISE])
-m4_if([$2], [presorted], [/PRESORTED]) dnl
- /DOCUMENT
- /BREAK=g
- /N = n
- /NI = n./
- NU = nu
- /NUI = nu./
- NFGT2 = fgt(n, 2)
- /NFGT2I = fgt.(n, 2)
- /SFGT2 = fgt(s, '2')
- /SFGT2I = fgt.(s, '2')
- /NFIN23 = fin(n, 2, 3)
- /NFIN23I = fin.(n, 2, 3)
- /SFIN23 = fin(s, '2', '3')
- /SFIN23I = fin.(s, '2', '3')
- /NFLT2 = flt(n, 2)
- /NFLT2I = flt.(n, 2)
- /SFLT2 = flt(s, '2')
- /SFLT2I = flt.(s, '2')
- /NFIRST = first(n)
- /NFIRSTI = first.(n)
- /SFIRST = first(s)
- /SFIRSTI = first.(s)
- /NFOUT23 = fout(n, 3, 2)
- /NFOUT23I = fout.(n, 3, 2)
- /SFOUT23 = fout(s, '3', '2')
- /SFOUT23I = fout.(s, '3', '2')
- /NLAST = last(n)
- /NLASTI = last.(n)
- /SLAST = last(s)
- /SLASTI = last.(s)
- /NMAX = max(n)
- /NMAXI = max.(n)
- /SMAX = max(s)
- /SMAXI = max.(s)
- /NMEAN = mean(n)
- /NMEANI = mean.(n)
- /NMIN = min(n)
- /NMINI = min.(n)
- /SMIN = min(s)
- /SMINI = min.(s)
- /NN = n(n)
- /NNI = n.(n)
- /SN = n(s)
- /SNI = n.(s)
- /NNMISS = nmiss(n)
- /NNMISSI = nmiss.(n)
- /SNMISS = nmiss(s)
- /SNMISSI = nmiss.(s)
- /NNU = nu(n)
- /NNUI = nu.(n)
- /SNU = nu(s)
- /SNUI = nu.(s)
- /NNUMISS = numiss(n)
- /NNUMISSI = numiss.(n)
- /SNUMISS = numiss(s)
- /SNUMISSI = numiss.(s)
- /NPGT2 = pgt(n, 2)
- /NPGT2I = pgt.(n, 2)
- /SPGT2 = pgt(s, '2')
- /SPGT2I = pgt.(s, '2')
- /NPIN23 = pin(n, 2, 3)
- /NPIN23I = pin.(n, 2, 3)
- /SPIN23 = pin(s, '2', '3')
- /SPIN23I = pin.(s, '2', '3')
- /NPLT2 = plt(n, 2)
- /NPLT2I = plt.(n, 2)
- /SPLT2 = plt(s, '2')
- /SPLT2I = plt.(s, '2')
- /NPOUT23 = pout(n, 2, 3)
- /NPOUT23I = pout.(n, 2, 3)
- /SPOUT23 = pout(s, '2', '3')
- /SPOUT23I = pout.(s, '2', '3')
- /NMEDIAN = median(n)
- /NMEDIANI = median.(n)
- /NSD = sd(n)
- /NSDI = sd.(n)
- /NSUM = sum(n)
- /NSUMI = sum.(n).
-m4_if([$1], [external], [GET FILE='aggregate.sys'.],
- [$1], [dataset], [DATASET ACTIVATE aggregate.])
-LIST.
-])
- AT_CHECK([pspp -O format=csv aggregate.sps], [0],
- [m4_if([$3], [itemwise], [dnl
-"aggregate.sps:29.28-29.31: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 29 | /NFOUT23 = fout(n, 3, 2)
- | ^~~~"
-
-"aggregate.sps:30.30-30.33: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 30 | /NFOUT23I = fout.(n, 3, 2)
- | ^~~~"
-
-"aggregate.sps:31.28-31.35: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 31 | /SFOUT23 = fout(s, '3', '2')
- | ^~~~~~~~"
-
-"aggregate.sps:32.30-32.37: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 32 | /SFOUT23I = fout.(s, '3', '2')
- | ^~~~~~~~"
-
-Table: Data List
-G,N,NI,NU,NUI,NFGT2,NFGT2I,SFGT2,SFGT2I,NFIN23,NFIN23I,SFIN23,SFIN23I,NFLT2,NFLT2I,SFLT2,SFLT2I,NFIRST,NFIRSTI,SFIRST,SFIRSTI,NFOUT23,NFOUT23I,SFOUT23,SFOUT23I,NLAST,NLASTI,SLAST,SLASTI,NMAX,NMAXI,SMAX,SMAXI,NMEAN,NMEANI,NMIN,NMINI,SMIN,SMINI,NN,NNI,SN,SNI,NNMISS,NNMISSI,SNMISS,SNMISSI,NNU,NNUI,SNU,SNUI,NNUMISS,NNUMISSI,SNUMISS,SNUMISSI,NPGT2,NPGT2I,SPGT2,SPGT2I,NPIN23,NPIN23I,SPIN23,SPIN23I,NPLT2,NPLT2I,SPLT2,SPLT2I,NPOUT23,NPOUT23I,SPOUT23,SPOUT23I,NMEDIAN,NMEDIANI,NSD,NSDI,NSUM,NSUMI
-1,7.00,7.00,6,6,.333,.429,.333,.429,.333,.286,.333,.286,.500,.429,.500,.429,0,0,0,0,.667,.714,.667,.714,5,5,5,5,5,5,5,5,2.00,2.29,0,0,0,0,6.00,7.00,6.00,7.00,1.00,.00,1.00,.00,5,6,5,6,1,0,1,0,33.3,42.9,33.3,42.9,33.3,28.6,33.3,28.6,50.0,42.9,50.0,42.9,66.7,71.4,66.7,71.4,1.50,2.00,1.79,1.80,12.00,16.00
-2,5.00,5.00,4,4,1.000,1.000,1.000,1.000,.000,.000,.000,.000,.000,.000,.000,.000,6,6,6,4,1.000,1.000,1.000,1.000,8,8,8,8,8,8,8,8,7.00,7.00,6,6,6,4,3.00,3.00,3.00,5.00,2.00,2.00,2.00,.00,3,3,3,4,1,1,1,0,100.0,100.0,100.0,100.0,.0,.0,.0,.0,.0,.0,.0,.0,100.0,100.0,100.0,100.0,7.00,7.00,1.00,1.00,21.00,21.00
-3,2.00,2.00,1,1,.000,.000,.000,.000,.000,.000,.000,.000,1.000,1.000,1.000,1.000,1,1,1,1,1.000,1.000,1.000,1.000,1,1,1,1,1,1,1,1,1.00,1.00,1,1,1,1,2.00,2.00,2.00,2.00,.00,.00,.00,.00,1,1,1,1,0,0,0,0,.0,.0,.0,.0,.0,.0,.0,.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,1.00,1.00,.00,.00,2.00,2.00
-4,1.00,1.00,1,1,. ,. ,. ,1.000,. ,. ,. ,.000,. ,. ,. ,.000,.,.,,4,. ,. ,. ,1.000,.,.,,4,.,.,,4,. ,. ,.,.,,4,.00,.00,.00,1.00,1.00,1.00,1.00,.00,0,0,0,1,1,1,1,0,. ,. ,. ,100.0,. ,. ,. ,.0,. ,. ,. ,.0,. ,. ,. ,100.0,NaN,NaN,. ,. ,. ,. @&t@
-],
- [dnl
-"aggregate.sps:29.28-29.31: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 29 | /NFOUT23 = fout(n, 3, 2)
- | ^~~~"
-
-"aggregate.sps:30.30-30.33: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 30 | /NFOUT23I = fout.(n, 3, 2)
- | ^~~~"
-
-"aggregate.sps:31.28-31.35: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 31 | /SFOUT23 = fout(s, '3', '2')
- | ^~~~~~~~"
-
-"aggregate.sps:32.30-32.37: warning: AGGREGATE: The value arguments passed to the FOUT function are out of order. They will be treated as if they had been specified in the correct order.
- 32 | /SFOUT23I = fout.(s, '3', '2')
- | ^~~~~~~~"
-
-Table: Data List
-G,N,NI,NU,NUI,NFGT2,NFGT2I,SFGT2,SFGT2I,NFIN23,NFIN23I,SFIN23,SFIN23I,NFLT2,NFLT2I,SFLT2,SFLT2I,NFIRST,NFIRSTI,SFIRST,SFIRSTI,NFOUT23,NFOUT23I,SFOUT23,SFOUT23I,NLAST,NLASTI,SLAST,SLASTI,NMAX,NMAXI,SMAX,SMAXI,NMEAN,NMEANI,NMIN,NMINI,SMIN,SMINI,NN,NNI,SN,SNI,NNMISS,NNMISSI,SNMISS,SNMISSI,NNU,NNUI,SNU,SNUI,NNUMISS,NNUMISSI,SNUMISS,SNUMISSI,NPGT2,NPGT2I,SPGT2,SPGT2I,NPIN23,NPIN23I,SPIN23,SPIN23I,NPLT2,NPLT2I,SPLT2,SPLT2I,NPOUT23,NPOUT23I,SPOUT23,SPOUT23I,NMEDIAN,NMEDIANI,NSD,NSDI,NSUM,NSUMI
-1,7.00,7.00,6,6,. ,.429,. ,.429,. ,.286,. ,.286,. ,.429,. ,.429,.,0,,0,. ,.714,. ,.714,.,5,,5,.,5,,5,. ,2.29,.,0,,0,6.00,7.00,6.00,7.00,1.00,.00,1.00,.00,5,6,5,6,1,0,1,0,. ,42.9,. ,42.9,. ,28.6,. ,28.6,. ,42.9,. ,42.9,. ,71.4,. ,71.4,. ,2.00,. ,1.80,. ,16.00
-2,5.00,5.00,4,4,. ,. ,. ,1.000,. ,. ,. ,.000,. ,. ,. ,.000,.,.,,4,. ,. ,. ,1.000,.,.,,8,.,.,,8,. ,. ,.,.,,4,3.00,3.00,3.00,5.00,2.00,2.00,2.00,.00,3,3,3,4,1,1,1,0,. ,. ,. ,100.0,. ,. ,. ,.0,. ,. ,. ,.0,. ,. ,. ,100.0,. ,. ,. ,. ,. ,. @&t@
-3,2.00,2.00,1,1,.000,.000,.000,.000,.000,.000,.000,.000,1.000,1.000,1.000,1.000,1,1,1,1,1.000,1.000,1.000,1.000,1,1,1,1,1,1,1,1,1.00,1.00,1,1,1,1,2.00,2.00,2.00,2.00,.00,.00,.00,.00,1,1,1,1,0,0,0,0,.0,.0,.0,.0,.0,.0,.0,.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,100.0,1.00,1.00,.00,.00,2.00,2.00
-4,1.00,1.00,1,1,. ,. ,. ,1.000,. ,. ,. ,.000,. ,. ,. ,.000,.,.,,4,. ,. ,. ,1.000,.,.,,4,.,.,,4,. ,. ,.,.,,4,.00,.00,.00,1.00,1.00,1.00,1.00,.00,0,0,0,1,1,1,1,0,. ,. ,. ,100.0,. ,. ,. ,.0,. ,. ,. ,.0,. ,. ,. ,100.0,. ,. ,. ,. ,. ,. @&t@
-])])
- AT_CLEANUP])
-
-CHECK_AGGREGATE([dataset], [presorted], [itemwise])
-CHECK_AGGREGATE([dataset], [presorted], [columnwise])
-CHECK_AGGREGATE([dataset], [unsorted], [itemwise])
-CHECK_AGGREGATE([dataset], [unsorted], [columnwise])
-CHECK_AGGREGATE([active], [presorted], [itemwise])
-CHECK_AGGREGATE([active], [presorted], [columnwise])
-CHECK_AGGREGATE([active], [unsorted], [itemwise])
-CHECK_AGGREGATE([active], [unsorted], [columnwise])
-CHECK_AGGREGATE([external], [presorted], [itemwise])
-CHECK_AGGREGATE([external], [presorted], [columnwise])
-CHECK_AGGREGATE([external], [unsorted], [itemwise])
-CHECK_AGGREGATE([external], [unsorted], [columnwise])
-
-AT_SETUP([AGGREGATE crash with MAX function])
-AT_DATA([aggregate.sps],
- [DATA LIST LIST /X (F8.2) Y (a25).
-
-BEGIN DATA.
-87.50 foo
-87.34 bar
-1 bar
-END DATA.
-
-AGGREGATE OUTFILE=* /BREAK=y /X=MAX(x).
-LIST /x y.
-])
-AT_CHECK([pspp -O format=csv aggregate.sps], [0],
- [Table: Reading free-form data from INLINE.
-Variable,Format
-X,F8.2
-Y,A25
-
-Table: Data List
-X,Y
-87.34,bar
-87.50,foo
-])
-AT_CLEANUP
-
-AT_SETUP([AGGREGATE crash with invalid syntax])
-AT_DATA([aggregate.sps],
- [INPUT PROGRAM.
-LOOP c=1 TO 20.
- COMPUTE x=UNIFORM(10)
- END CASE.
-END LOOP.
-END FILE.
-END INPUT PROGRAM.
-
-AGGREGATE /BREAK=x .
-])
-AT_CHECK([pspp -O format=csv aggregate.sps], [1], [ignore], [])
-AT_CLEANUP
-
-
-AT_SETUP([AGGREGATE mode=addvariables])
-AT_DATA([addvariables.sps],
- [data list notable list /x * cn * y *.
-begin data.
-1 1 2
-3 2 3
-3 3 4
-5 4 6
-7 5 8
-7 6 9
-7 7 20
-9 8 11
-end data.
-
-aggregate outfile=* mode=addvariables
- /break = x
- /sum = sum(y)
- /mean = mean (y)
- /median = median (y).
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv addvariables.sps], [0],
- [Table: Data List
-x,cn,y,sum,mean,median
-1.00,1.00,2.00,2.00,2.00,2.00
-3.00,2.00,3.00,7.00,3.50,3.50
-3.00,3.00,4.00,7.00,3.50,3.50
-5.00,4.00,6.00,6.00,6.00,6.00
-7.00,5.00,8.00,37.00,12.33,9.00
-7.00,6.00,9.00,37.00,12.33,9.00
-7.00,7.00,20.00,37.00,12.33,9.00
-9.00,8.00,11.00,11.00,11.00,11.00
-])
-
-AT_CLEANUP
-
-AT_SETUP([AGGREGATE duplicate variable errors])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='aggregate.sps' ERROR=IGNORE.
-])
-AT_DATA([aggregate.sps], [dnl
-DATA LIST NOTABLE LIST /x.
-AGGREGATE OUTFILE=* /BREAK=x /x=N.
-AGGREGATE OUTFILE=* MODE=ADDVARIABLES /x=N.
-AGGREGATE OUTFILE=* MODE=ADDVARIABLES /y=N /y=N.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"aggregate.sps:2.31: error: AGGREGATE: Variable name x duplicates the name of a break variable.
- 2 | AGGREGATE OUTFILE=* /BREAK=x /x=N.
- | ^"
-
-"aggregate.sps:3.40: error: AGGREGATE: Variable name x duplicates the name of a variable in the active file dictionary.
- 3 | AGGREGATE OUTFILE=* MODE=ADDVARIABLES /x=N.
- | ^"
-
-"aggregate.sps:4.45: error: AGGREGATE: Duplicate target variable name y.
- 4 | AGGREGATE OUTFILE=* MODE=ADDVARIABLES /y=N /y=N.
- | ^"
-])
-AT_CLEANUP
-
-AT_SETUP([AGGREGATE presorted warnings])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='aggregate.sps' ERROR=IGNORE.
-])
-AT_DATA([aggregate.sps], [dnl
-DATA LIST NOTABLE LIST /x.
-AGGREGATE/PRESORTED/BREAK=x(A).
-AGGREGATE/BREAK=x(A).
-AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"aggregate.sps:2.27-2.30: warning: AGGREGATE: When the input data is presorted, specifying sorting directions with (A) or (D) has no effect. Output data will be sorted the same way as the input data.
- 2 | AGGREGATE/PRESORTED/BREAK=x(A).
- | ^~~~"
-
-"aggregate.sps:2.11-2.19: note: AGGREGATE: The PRESORTED subcommand state that the input data is presorted.
- 2 | AGGREGATE/PRESORTED/BREAK=x(A).
- | ^~~~~~~~~"
-
-"aggregate.sps:2.31: error: AGGREGATE: Syntax error expecting `/'.
- 2 | AGGREGATE/PRESORTED/BREAK=x(A).
- | ^"
-
-"aggregate.sps:3.17-3.20: warning: AGGREGATE: When the input data is presorted, specifying sorting directions with (A) or (D) has no effect. Output data will be sorted the same way as the input data.
- 3 | AGGREGATE/BREAK=x(A).
- | ^~~~"
-
-aggregate.sps:3: note: AGGREGATE: The input data must be presorted because the OUTFILE subcommand is not specified.
-
-"aggregate.sps:3.21: error: AGGREGATE: Syntax error expecting `/'.
- 3 | AGGREGATE/BREAK=x(A).
- | ^"
-
-"aggregate.sps:4.45-4.48: warning: AGGREGATE: When the input data is presorted, specifying sorting directions with (A) or (D) has no effect. Output data will be sorted the same way as the input data.
- 4 | AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
- | ^~~~"
-
-"aggregate.sps:4.26-4.37: note: AGGREGATE: ADDVARIABLES implies that the input data is presorted.
- 4 | AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
- | ^~~~~~~~~~~~"
-
-"aggregate.sps:4.49: error: AGGREGATE: Syntax error expecting `/'.
- 4 | AGGREGATE/OUTFILE=* MODE=ADDVARIABLES/BREAK=x(A).
- | ^"
-])
-AT_CLEANUP
-
-AT_SETUP([AGGREGATE - subcommand syntax errors])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='aggregate.sps' ERROR=IGNORE.
-])
-AT_DATA([aggregate.sps], [dnl
-DATA LIST NOTABLE LIST /x.
-AGGREGATE OUTFILE=**.
-AGGREGATE OUTFILE=* MODE=**.
-AGGREGATE /MISSING=**.
-AGGREGATE /BREAK=**.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"aggregate.sps:2.19-2.20: error: AGGREGATE: Syntax error expecting a file name or handle name.
- 2 | AGGREGATE OUTFILE=**.
- | ^~"
-
-"aggregate.sps:3.26-3.27: error: AGGREGATE: Syntax error expecting ADDVARIABLES or REPLACE.
- 3 | AGGREGATE OUTFILE=* MODE=**.
- | ^~"
-
-"aggregate.sps:4.20-4.21: error: AGGREGATE: Syntax error expecting COLUMNWISE.
- 4 | AGGREGATE /MISSING=**.
- | ^~"
-
-"aggregate.sps:5.18-5.19: error: AGGREGATE: Syntax error expecting variable name.
- 5 | AGGREGATE /BREAK=**.
- | ^~"
-])
-AT_CLEANUP
-
-AT_SETUP([AGGREGATE - aggregation function syntax errors])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='aggregate.sps' ERROR=IGNORE.
-])
-AT_DATA([aggregate.sps], [dnl
-DATA LIST NOTABLE LIST /x (f8.2) s (a8).
-AGGREGATE **.
-AGGREGATE / **.
-AGGREGATE /y.
-AGGREGATE /y=**.
-AGGREGATE /y=xyzzy.
-AGGREGATE /y=mean.
-AGGREGATE /y=mean(**).
-AGGREGATE /y=fgt(x **).
-AGGREGATE /y=fgt(x 'xyzzy').
-AGGREGATE /y=fgt(s 1).
-AGGREGATE /y=fgt(s x).
-AGGREGATE /y=sum(s).
-AGGREGATE /y=sum(x. /* )
-AGGREGATE /y=min(x, s).
-AGGREGATE /y t=min(x).
-AGGREGATE /y=pin(x, 2, 1).
-AGGREGATE /y=mean(x)**.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"aggregate.sps:2.11-2.12: error: AGGREGATE: Syntax error expecting `/'.
- 2 | AGGREGATE **.
- | ^~"
-
-"aggregate.sps:3.13-3.14: error: AGGREGATE: Syntax error expecting variable name.
- 3 | AGGREGATE / **.
- | ^~"
-
-"aggregate.sps:4.13: error: AGGREGATE: Syntax error expecting variable name.
- 4 | AGGREGATE /y.
- | ^"
-
-"aggregate.sps:5.14-5.15: error: AGGREGATE: Syntax error expecting aggregation function.
- 5 | AGGREGATE /y=**.
- | ^~"
-
-"aggregate.sps:6.14-6.18: error: AGGREGATE: Unknown aggregation function xyzzy.
- 6 | AGGREGATE /y=xyzzy.
- | ^~~~~"
-
-"aggregate.sps:7.18: error: AGGREGATE: Syntax error expecting `('.
- 7 | AGGREGATE /y=mean.
- | ^"
-
-"aggregate.sps:8.19-8.20: error: AGGREGATE: Syntax error expecting variable name.
- 8 | AGGREGATE /y=mean(**).
- | ^~"
-
-"aggregate.sps:9.20-9.21: error: AGGREGATE: Missing argument 1 to FGT.
- 9 | AGGREGATE /y=fgt(x **).
- | ^~"
-
-aggregate.sps:10: error: AGGREGATE: Arguments to FGT must be of same type as source variables.
-
-"aggregate.sps:10.20-10.26: note: AGGREGATE: The argument is a string.
- 10 | AGGREGATE /y=fgt(x 'xyzzy').
- | ^~~~~~~"
-
-"aggregate.sps:10.18: note: AGGREGATE: The variables are numeric.
- 10 | AGGREGATE /y=fgt(x 'xyzzy').
- | ^"
-
-aggregate.sps:11: error: AGGREGATE: Arguments to FGT must be of same type as source variables.
-
-"aggregate.sps:11.20: note: AGGREGATE: The argument is numeric.
- 11 | AGGREGATE /y=fgt(s 1).
- | ^"
-
-"aggregate.sps:11.18: note: AGGREGATE: The variables have string type.
- 11 | AGGREGATE /y=fgt(s 1).
- | ^"
-
-"aggregate.sps:12.20: error: AGGREGATE: s and x are not the same type. All variables in this variable list must be of the same type. x will be omitted from the list.
- 12 | AGGREGATE /y=fgt(s x).
- | ^"
-
-"aggregate.sps:12.21: error: AGGREGATE: Missing argument 1 to FGT.
- 12 | AGGREGATE /y=fgt(s x).
- | ^"
-
-"aggregate.sps:13.18: warning: AGGREGATE: s is not a numeric variable. It will not be included in the variable list.
- 13 | AGGREGATE /y=sum(s).
- | ^"
-
-"aggregate.sps:14.19: error: AGGREGATE: Syntax error expecting `)'.
- 14 | AGGREGATE /y=sum(x. /* )
- | ^"
-
-aggregate.sps:15: error: AGGREGATE: Number of source variables (2) does not match number of target variables (1).
-
-"aggregate.sps:15.18-15.21: note: AGGREGATE: These are the source variables.
- 15 | AGGREGATE /y=min(x, s).
- | ^~~~"
-
-"aggregate.sps:15.12: note: AGGREGATE: These are the target variables.
- 15 | AGGREGATE /y=min(x, s).
- | ^"
-
-aggregate.sps:16: error: AGGREGATE: Number of source variables (1) does not match number of target variables (2).
-
-"aggregate.sps:16.20: note: AGGREGATE: These are the source variables.
- 16 | AGGREGATE /y t=min(x).
- | ^"
-
-"aggregate.sps:16.12-16.14: note: AGGREGATE: These are the target variables.
- 16 | AGGREGATE /y t=min(x).
- | ^~~"
-
-"aggregate.sps:17.21-17.24: warning: AGGREGATE: The value arguments passed to the PIN function are out of order. They will be treated as if they had been specified in the correct order.
- 17 | AGGREGATE /y=pin(x, 2, 1).
- | ^~~~"
-
-"aggregate.sps:18.1-18.9: error: AGGREGATE: Syntax error expecting `BEGIN DATA'.
- 18 | AGGREGATE /y=mean(x)**.
- | ^~~~~~~~~"
-
-"aggregate.sps:18.1-18.9: error: AGGREGATE: Syntax error expecting end of command.
- 18 | AGGREGATE /y=mean(x)**.
- | ^~~~~~~~~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([AUTORECODE procedure])
-
-AT_SETUP([AUTORECODE multiple missing values])
-AT_DATA([autorecode.sps],
- [DATA LIST LIST NOTABLE /u v w x y z (F2.0).
-BEGIN DATA.
-11 11 11 11 11 11
-12 12 12 12 12 12
-13 13 13 13 13 13
-14 14 14 14 14 14
-15 15 15 15 15 15
-16 16 16 16 16 16
-END DATA.
-
-MISSING VALUES u (11)
- v (11, 12)
- w (11, 12, 13)
- x (11 THRU 14)
- y (11 THRU 15)
- z (11 THRU 16).
-
-AUTORECODE u v w x y z INTO a b c d e f/print.
-LIST.
-DISPLAY VARIABLES/VARIABLES=a TO f.
-])
-AT_CHECK([pspp -O format=csv autorecode.sps], [0], [dnl
-Table: Recoding u into a.
-Old Value,New Value,Value Label
-12,1,12
-13,2,13
-14,3,14
-15,4,15
-16,5,16
-11,6,11
-
-Table: Recoding v into b.
-Old Value,New Value,Value Label
-13,1,13
-14,2,14
-15,3,15
-16,4,16
-11,5,11
-12,6,12
-
-Table: Recoding w into c.
-Old Value,New Value,Value Label
-14,1,14
-15,2,15
-16,3,16
-11,4,11
-12,5,12
-13,6,13
-
-Table: Recoding x into d.
-Old Value,New Value,Value Label
-15,1,15
-16,2,16
-11,3,11
-12,4,12
-13,5,13
-14,6,14
-
-Table: Recoding y into e.
-Old Value,New Value,Value Label
-16,1,16
-11,2,11
-12,3,12
-13,4,13
-14,5,14
-15,6,15
-
-Table: Recoding z into f.
-Old Value,New Value,Value Label
-11,1,11
-12,2,12
-13,3,13
-14,4,14
-15,5,15
-16,6,16
-
-Table: Data List
-u,v,w,x,y,z,a,b,c,d,e,f
-11,11,11,11,11,11,6,5,4,3,2,1
-12,12,12,12,12,12,1,6,5,4,3,2
-13,13,13,13,13,13,2,1,6,5,4,3
-14,14,14,14,14,14,3,2,1,6,5,4
-15,15,15,15,15,15,4,3,2,1,6,5
-16,16,16,16,16,16,5,4,3,2,1,6
-
-Table: Variables
-Name,Position,Print Format,Write Format,Missing Values
-a,7,F1.0,F1.0,6
-b,8,F1.0,F1.0,5; 6
-c,9,F1.0,F1.0,4; 5; 6
-d,10,F1.0,F1.0,3 THRU 6
-e,11,F1.0,F1.0,2 THRU 6
-f,12,F1.0,F1.0,1 THRU 6
-])
-AT_CLEANUP
-
-AT_SETUP([AUTORECODE numbers and short strings])
-AT_DATA([autorecode.sps],
- [data list /X 1-5(a) Y 7.
-begin data.
-lasdj 1
-asdfk 0
-asdfj 2
-asdfj 1
-asdfk 2
-asdfj 9
-lajks 9
-asdfk 0
-asdfk 1
-end data.
-
-missing values x('asdfk') y(9).
-
-autorecode x y into A B/descend/print.
-
-list.
-compute Z=trunc(y/2).
-formats z(F1.0).
-autorecode z into W.
-list.
-])
-AT_CHECK([pspp -O format=csv autorecode.sps], [0],
- [Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-X,1,1-5,A5
-Y,1,7-7,F1.0
-
-Table: Recoding X into A.
-Old Value,New Value,Value Label
-lasdj,1,lasdj
-lajks,2,lajks
-asdfj,3,asdfj
-asdfk,4,asdfk
-
-Table: Recoding Y into B.
-Old Value,New Value,Value Label
-2,1,2
-1,2,1
-0,3,0
-9,4,9
-
-Table: Data List
-X,Y,A,B
-lasdj,1,1,2
-asdfk,0,4,3
-asdfj,2,3,1
-asdfj,1,3,2
-asdfk,2,4,1
-asdfj,9,3,4
-lajks,9,2,4
-asdfk,0,4,3
-asdfk,1,4,2
-
-Table: Data List
-X,Y,A,B,Z,W
-lasdj,1,1,2,0,1
-asdfk,0,4,3,0,1
-asdfj,2,3,1,1,2
-asdfj,1,3,2,0,1
-asdfk,2,4,1,1,2
-asdfj,9,3,4,.,.
-lajks,9,2,4,.,.
-asdfk,0,4,3,0,1
-asdfk,1,4,2,0,1
-])
-AT_CLEANUP
-
-AT_SETUP([AUTORECODE long strings and check the value labels])
-AT_DATA([ar.sps],
- [data list notable list /s (a16) x (f1.0).
-begin data.
-widgets 1
-thingummies 2
-oojars 3
-widgets 4
-oojars 5
-thingummies 6
-oojimiflips 7
-end data.
-
-variable labels s 'tracking my stuff'.
-value labels /s 'thingummies' 'Funny sticky things'.
-
-autorecode s into new/print.
-
-list.
-
-display dictionary/variables=new.
-])
-
-AT_CHECK([pspp -O format=csv ar.sps], [0],
- [Table: Recoding s into new (tracking my stuff).
-Old Value,New Value,Value Label
-oojars,1,oojars
-oojimiflips,2,oojimiflips
-thingummies,3,Funny sticky things
-widgets,4,widgets
-
-Table: Data List
-s,x,new
-widgets,1,4
-thingummies,2,3
-oojars,3,1
-widgets,4,4
-oojars,5,1
-thingummies,6,3
-oojimiflips,7,2
-
-Table: Variables
-Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-new,3,tracking my stuff,Nominal,Input,8,Right,F1.0,F1.0
-
-Table: Value Labels
-Variable Value,,Label
-tracking my stuff,1,oojars
-,2,oojimiflips
-,3,Funny sticky things
-,4,widgets
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([AUTORECODE group subcommand])
-AT_DATA([ar-group.sps],
-[data list notable list /x y (f8.0).
-begin data.
-11 10
-12 12
-13 15
-14 11
-15 12
-16 18
-end data.
-
-missing values y (12).
-
-autorecode
- x y into a b
- /group
- /print.
-
-list.
-display variables /variables=a b.
-])
-
-AT_CHECK([pspp -O format=csv ar-group.sps], [0],
-[Table: Recoding grouped variables.
-Old Value,New Value,Value Label
-10,1,10
-11,2,11
-13,3,13
-14,4,14
-15,5,15
-16,6,16
-18,7,18
-12,8,12
-
-Table: Data List
-x,y,a,b
-11,10,2,1
-12,12,8,8
-13,15,3,5
-14,11,4,2
-15,12,5,8
-16,18,6,7
-
-Table: Variables
-Name,Position,Print Format,Write Format,Missing Values
-a,3,F1.0,F1.0,8
-b,4,F1.0,F1.0,8
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([AUTORECODE group - string variables])
-AT_DATA([strings.sps],
-[data list notable list /x (a8) y (a16).
-begin data.
-fred bert
-charlie " "
-delta echo
-" " windows
-" " nothing
-end data.
-
-
-autorecode x y into a b
- /group
- /print.
-
-delete variables x y.
-
-list.
-
-])
-
-AT_CHECK([pspp -O format=csv strings.sps], [0],
-[Table: Recoding grouped variables.
-Old Value,New Value,Value Label
-,1,
-bert,2,bert
-charlie,3,charlie
-delta,4,delta
-echo,5,echo
-fred,6,fred
-nothing,7,nothing
-windows,8,windows
-
-Table: Data List
-a,b
-6,2
-3,1
-4,5
-1,8
-1,7
-])
-
-AT_CLEANUP
-
-
-dnl Tests for a crash which happened when the /GROUP subcommand
-dnl appeared with string variables of different widths.
-AT_SETUP([AUTORECODE group vs. strings])
-AT_DATA([ar-strings.sps],
- [data list notable list /a (a12) b (a6).
-begin data.
-one nine
-two ten
-three eleven
-four nought
-end data.
-
-autorecode a b into x y
- /group
- /print.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv ar-strings.sps], [0], [dnl
-Table: Recoding grouped variables.
-Old Value,New Value,Value Label
-eleven,1,eleven
-four,2,four
-nine,3,nine
-nought,4,nought
-one,5,one
-ten,6,ten
-three,7,three
-two,8,two
-
-Table: Data List
-a,b,x,y
-one,nine,5,3
-two,ten,8,6
-three,eleven,7,1
-four,nought,2,4
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([AUTORECODE /blank])
-
-AT_DATA([auto-blank.sps], [dnl
-data list notable list /x (a8) y (f8.0) z (a16).
-begin data.
-one 2 fred
-two 4 ""
-"" 4 fred
-"" 2 charliebrown
-three 2 charliebrown
-end data.
-
-autorecode variables x y z into a b c /blank=missing /print.
-
-list a b c y.
-])
-
-AT_CHECK([pspp -O format=csv auto-blank.sps], [0], [dnl
-Table: Recoding x into a.
-Old Value,New Value,Value Label
-one,1,one
-three,2,three
-two,3,two
-
-Table: Recoding y into b.
-Old Value,New Value,Value Label
-2,1,2
-4,2,4
-
-Table: Recoding z into c.
-Old Value,New Value,Value Label
-charliebrown,1,charliebrown
-fred,2,fred
-
-Table: Data List
-a,b,c,y
-1,1,2,2
-3,2,.,4
-.,2,2,4
-.,1,1,2
-2,1,1,2
-])
-
-AT_CLEANUP
-
-dnl AUTORECODE had a use-after-free error when TEMPORARY was in use.
-dnl Bug #32757.
-AT_SETUP([AUTORECODE with TEMPORARY])
-AT_DATA([autorecode.sps],
- [data list /X 1-5(a) Y 7.
-begin data.
-lasdj 1
-asdfk 0
-asdfj 2
-asdfj 1
-asdfk 2
-asdfj 9
-lajks 9
-asdfk 0
-asdfk 1
-end data.
-
-temporary.
-select if y > 1.
-autorecode x y into A B/descend/print.
-list.
-])
-AT_CHECK([pspp -O format=csv autorecode.sps], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-X,1,1-5,A5
-Y,1,7-7,F1.0
-
-Table: Recoding X into A.
-Old Value,New Value,Value Label
-lajks,1,lajks
-asdfk,2,asdfk
-asdfj,3,asdfj
-
-Table: Recoding Y into B.
-Old Value,New Value,Value Label
-9,1,9
-2,2,2
-
-Table: Data List
-X,Y,A,B
-lasdj,1,.,.
-asdfk,0,2,.
-asdfj,2,3,2
-asdfj,1,3,.
-asdfk,2,2,2
-asdfj,9,3,1
-lajks,9,1,1
-asdfk,0,2,.
-asdfk,1,2,.
-])
-AT_CLEANUP
-
-
-dnl For compatibility, make sure that /INTO (with leading slash) is accepted
-dnl (bug #48762)
-AT_SETUP([AUTORECODE with /INTO])
-AT_DATA([autorecode.sps],
- [data list list notable /x (f8.0).
-begin data.
-1
-8
--901
-4
-1
-99
-8
-end data.
-
-autorecode x /into y /print.
-
-list.
-])
-AT_CHECK([pspp -O format=csv autorecode.sps], [0],
-[Table: Recoding x into y.
-Old Value,New Value,Value Label
--901,1,-901
-1,2,1
-4,3,4
-8,4,8
-99,5,99
-
-Table: Data List
-x,y
-1,2
-8,4
--901,1
-4,3
-1,2
-99,5
-8,4
-])
-AT_CLEANUP
-
-
-AT_SETUP([AUTORECODE with /BLANK without specifier])
-
-AT_DATA([autorecode.sps], [data list notable list /x (a18).
-begin data
-one
-two
-three
-end data.
-
-* /BLANK should be either =MISSING or =VALID
-autorecode x /into y
- /blank
-
-execute.
-])
-
-AT_CHECK([pspp -O format=csv autorecode.sps], [1], [ignore])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([CORRELATIONS])
-
-AT_SETUP([CORRELATIONS -- unweighted])
-AT_DATA([correlations.sps], [dnl
-set format = F11.3.
-data list notable list /foo * bar * wiz * bang *.
-begin data.
-1 0 3 1
-3 9 -50 5
-3 4 3 203
-4 -9 0 -4
-98 78 104 2
-3 50 -49 200
-. 4 4 4
-5 3 0 .
-end data.
-
-correlations
- variables = foo bar wiz bang
- /print nosig
- /missing = listwise
- .
-
-correlations
- variables = bar wiz
- /print nosig
- /missing = listwise
- .
-
-correlations
- variables = foo bar wiz bang
- /print nosig
- /missing = pairwise
- .
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt correlations.sps])
-AT_CHECK([cat pspp.csv], [0],
-[[Table: Correlations
-,,foo,bar,wiz,bang
-foo,Pearson Correlation,1.000,.802,.890[a],-.308
-,Sig. (2-tailed),,.055,.017,.553
-bar,Pearson Correlation,.802,1.000,.519,.118
-,Sig. (2-tailed),.055,,.291,.824
-wiz,Pearson Correlation,.890[a],.519,1.000,-.344
-,Sig. (2-tailed),.017,.291,,.505
-bang,Pearson Correlation,-.308,.118,-.344,1.000
-,Sig. (2-tailed),.553,.824,.505,
-Footnote: a. Significant at .05 level
-
-Table: Correlations
-,,bar,wiz
-bar,Pearson Correlation,1.000,.497
-,Sig. (2-tailed),,.210
-wiz,Pearson Correlation,.497,1.000
-,Sig. (2-tailed),.210,
-
-Table: Correlations
-,,foo,bar,wiz,bang
-foo,Pearson Correlation,1.000,.805[a],.883[a],-.308
-,Sig. (2-tailed),,.029,.008,.553
-,N,7,7,7,6
-bar,Pearson Correlation,.805[a],1.000,.497,.164
-,Sig. (2-tailed),.029,,.210,.725
-,N,7,8,8,7
-wiz,Pearson Correlation,.883[a],.497,1.000,-.337
-,Sig. (2-tailed),.008,.210,,.460
-,N,7,8,8,7
-bang,Pearson Correlation,-.308,.164,-.337,1.000
-,Sig. (2-tailed),.553,.725,.460,
-,N,6,7,7,7
-Footnote: a. Significant at .05 level
-]])
-AT_CLEANUP
-
-AT_SETUP([CORRELATIONS -- weighted])
-AT_DATA([correlations1.sps], [dnl
-set format = F11.3.
-data list notable list /foo * bar * wiz * bang * w *.
-begin data.
-1 0 3 1 1
-3 9 -50 5 2
-3 4 3 203 1
-4 -9 0 -4 1
-98 78 104 2 3
-3 50 -49 200 1
-end data.
-
-weight by w.
-
-correlations
- variables = foo bar wiz bang
- /statistics=descriptives xprod
- .
-])
-AT_DATA([correlations2.sps], [dnl
-set format = F11.3.
-data list notable list /foo * bar * wiz * bang * w *.
-begin data.
-1 0 3 1 1
-3 9 -50 5 1
-3 9 -50 5 1
-3 4 3 203 1
-4 -9 0 -4 1
-98 78 104 2 1
-98 78 104 2 1
-98 78 104 2 1
-3 50 -49 200 1
-end data.
-
-weight by w.
-
-correlations
- variables = foo bar wiz bang
- /statistics=descriptives xprod
- .
-])
-AT_CHECK([pspp -O format=csv correlations1.sps], [0], [stdout])
-mv stdout expout
-AT_CHECK([pspp -O format=csv correlations2.sps], [0], [expout])
-AT_CLEANUP
-
-
-AT_SETUP([CORRELATIONS -- non-square])
-AT_DATA([corr-ns.sps], [dnl
-set format = F11.3.
-data list notable list /foo * bar * wiz *.
-begin data.
-1 1 6
-2 2 5
-3 3 4
-4 4 3
-5 5 2
-6 6 1
-end data.
-
-correlations
- variables = foo with bar wiz
- .
-])
-
-AT_CHECK([pspp -O format=csv corr-ns.sps], [0], [dnl
-Table: Correlations
-,,bar,wiz
-foo,Pearson Correlation,1.000,-1.000
-,Sig. (2-tailed),.000,.000
-,N,6,6
-])
-
-AT_CLEANUP
-
-dnl Checks for bug #38661.
-AT_SETUP([CORRELATIONS -- crash with WITH keyword])
-AT_DATA([correlations.sps], [dnl
-DATA LIST LIST NOTABLE /a b c d e f g h i.
-.
-BEGIN DATA.
-20 21 17 28 23 4.35 24 19 25
-28 18 29 30 23 4.55 17 23 28
-47 18 30 30 29 4.35 26 31 31
-20 7 19 22 22 4.80 24 16 27
-19 12 17 27 22 . 22 14 25
-22 9 19 30 33 5 29 30 27
-41 16 22 32 23 3.90 26 27 23
-18 18 20 26 22 5.80 17 20 39
-18 24 25 25 31 5.15 27 27 34
-19 22 26 23 37 6 41 32 27
-23 12 15 29 25 4.10 21 27 20
-21 4 28 37 31 5.65 27 18 42
-19 5 17 17 29 3.10 19 16 19
-21 17 20 35 31 . 28 30 22
-END DATA.
-
-CORRELATIONS VARIABLE=a f b WITH c g h i e d/STATISTICS=DESCRIPTIVES.
-])
-AT_CHECK([pspp -o pspp.csv correlations.sps])
-# Check the output, ignoring the actual correlations values since
-# they look pretty nonsensical to me for this input (they include NaNs).
-AT_CHECK([sed '/a,Pearson/,$s/,\([[^,]]*\),.*/,\1,.../' pspp.csv], [0], [dnl
-Table: Descriptive Statistics
-,Mean,Std. Deviation,N
-a,24.00,8.93,14
-f,4.73,.85,12
-b,14.50,6.41,14
-c,21.71,4.98,14
-g,24.86,6.09,14
-h,23.57,6.30,14
-i,27.79,6.73,14
-e,27.21,4.95,14
-d,27.93,5.23,14
-
-Table: Correlations
-,,c,g,h,i,e,d
-a,Pearson Correlation,...
-,Sig. (2-tailed),...
-,N,...
-f,Pearson Correlation,...
-,Sig. (2-tailed),...
-,N,...
-b,Pearson Correlation,...
-,Sig. (2-tailed),...
-,N,...
-])
-AT_CLEANUP
-
-
-
-dnl Checks for bug #40661
-AT_SETUP([CORRELATIONS -- incorrect subtable selection])
-AT_DATA([correlations.sps], [dnl
-set format = F12.4.
-OUTPUT MODIFY /SELECT TABLES /TABLECELLS SELECT = [[CORRELATIONS]] FORMAT=F12.4.
-set decimal = dot.
-data list notable list /var1 var2 var3 var4 var5 *.
-begin data.
-7,6,9,2,3
-9,12,8,5,8
-8,9,7,8,6
-8,8,9,10,8
-7,6,4,5,3
-7,9,8,2,1
-9,8,11,,10
-8,7,6,,5
-6,7,6,,8
-6,,3,,4
-6,,7,3,3
-5,4,2,7,8
-9,8,6,11,10
-5,6,2,2,4
-8,7,6,8,7
-10,13,8,12,10
-7,8,7,11,2
-8,7,7,9,6
-10,11,11,8,1
-5,8,6,9,9
-8,7,5,5,6
-5,7,2,1,8
-9,8,8,13,6
-5,8,5,6,4
-,7,5,4,5
-,8,4,4,3
-,6,4,9,5
-8,11,9,12,3
-9,11,8,10,6
-10,10,7,8,1
-6,6,3,8,9
-10,9,7,12,2
-6,8,,7,4
-6,8,3,2,9
-7,8,8,2,9
-5,6,5,5,5
-9,9,7,7,5
-9,10,11,7,8
-8,11,9,3,3
-5,4,4,0,5
-9,9,11,14,2
-5,6,2,4,4
-8,8,7,4,1
-9,9,8,14,
-6,8,7,2,
-10,9,9,6,
-8,8,10,9,
-7,8,4,12,
-6,6,6,7,1
-5,7,7,4,10
-9,10,10,13,4
-9,11,9,8,7
-10,13,12,6,8
-8,11,6,8,5
-7,8,7,12,2
-6,7,4,1,10
-5,4,5,6,10
-7,8,6,12,10
-6,5,3,9,2
-7,8,8,7,2
-5,4,4,9,8
-5,7,6,3,9
-10,10,9,13,1
-8,10,9,5,4
-8,9,8,8,7
-7,9,9,6,7
-10,9,7,12,6
-10,13,12,12,4
-7,10,9,7,2
-6,8,7,11,6
-8,11,5,13,2
-7,10,6,12,8
-10,10,9,7,9
-9,12,6,7,10
-6,6,8,2,9
-10,9,12,13,10
-8,9,8,3,6
-8,7,6,4,10
-8,7,10,12,2
-7,6,8,2,7
-8,11,6,9,4
-6,6,7,8,2
-6,7,3,11,4
-5,6,3,0,5
-10,10,11,15,6
-5,4,7,6,8
-5,4,4,1,3
-6,9,8,1,6
-10,11,10,15,8
-7,10,4,11,7
-9,12,8,6,3
-10,10,11,15,2
-10,9,9,15,3
-6,6,8,5,1
-5,7,7,0,3
-9,8,10,6,8
-9,8,11,11,4
-8,10,7,3,4
-7,8,7,3,3
-8,9,10,13,8
-end data.
-
-CORRELATION
- /VARIABLES = var1 var2 var3 WITH var4 var5
- /PRINT = TWOTAIL NOSIG.
-
-CORRELATION
- /VARIABLES = var3 var4 var5 WITH var1 var2
- /PRINT = TWOTAIL NOSIG.
-
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt correlations.sps])
-AT_CHECK([cat pspp.csv], [0],
-[[Table: Correlations
-,,var4,var5
-var1,Pearson Correlation,.5693[a],-.0519
-,Sig. (2-tailed),.000,.623
-,N,93,92
-var2,Pearson Correlation,.3792[a],-.0407
-,Sig. (2-tailed),.000,.698
-,N,95,93
-var3,Pearson Correlation,.3699[a],-.0543
-,Sig. (2-tailed),.000,.603
-,N,95,94
-Footnote: a. Significant at .05 level
-
-Table: Correlations
-,,var1,var2
-var3,Pearson Correlation,.6964[a],.5615[a]
-,Sig. (2-tailed),.000,.000
-,N,96,97
-var4,Pearson Correlation,.5693[a],.3792[a]
-,Sig. (2-tailed),.000,.000
-,N,93,95
-var5,Pearson Correlation,-.0519,-.0407
-,Sig. (2-tailed),.623,.698
-,N,92,93
-Footnote: a. Significant at .05 level
-]])
-
-AT_CLEANUP
-
-
-dnl Crash found by zzuf
-AT_SETUP([CORRELATIONS -- empty dataset])
-
-AT_DATA([correlations.sps], [dnl
-data list list /a b c q g *.
-CORRELATIONS 'VARIABLES = a b.]
-)
-
-AT_CHECK([pspp -o pspp.csv correlations.sps], [1], [ignore])
-
-AT_CLEANUP
-
-dnl Another Crash found by zzuf
-AT_SETUP([CORRELATIONS -- empty dataset 2])
-
-AT_DATA([correlations.sps], [dnl
-data list notable list /foo * bar * wiz bang *.
-begin data.
- 1 00 3 .
- 3 9 -50 .
-98 78 104 .
- . 4 4 .
- 5 3 0 .
-end data.
-
-correlations
- variables = foo bar wiz bang
- /missing = listwise
- .
-])
-
-AT_CHECK([pspp -O format=csv correlations.sps], [1], [dnl
-correlations.sps:13: error: CORRELATIONS: The data for the chosen variables are all missing or empty.
-])
-
-AT_CLEANUP
-
-AT_SETUP([CORRELATIONS -- syntax errors])
-AT_DATA([correlations.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-CORRELATIONS MISSING=**.
-CORRELATIONS PRINT=**.
-CORRELATIONS STATISTICS=**.
-CORRELATIONS **.
-CORRELATIONS x y z WITH **.
-CORRELATIONS.
-])
-AT_CHECK([pspp -O format=csv correlations.sps], [1], [dnl
-"correlations.sps:2.22-2.23: error: CORRELATIONS: Syntax error expecting PAIRWISE, LISTWISE, INCLUDE, or EXCLUDE.
- 2 | CORRELATIONS MISSING=**.
- | ^~"
-
-"correlations.sps:3.20-3.21: error: CORRELATIONS: Syntax error expecting TWOTAIL, ONETAIL, SIG, or NOSIG.
- 3 | CORRELATIONS PRINT=**.
- | ^~"
-
-"correlations.sps:4.25-4.26: error: CORRELATIONS: Syntax error expecting DESCRIPTIVES, XPROD, or ALL.
- 4 | CORRELATIONS STATISTICS=**.
- | ^~"
-
-"correlations.sps:5.14-5.15: error: CORRELATIONS: Syntax error expecting variable name.
- 5 | CORRELATIONS **.
- | ^~"
-
-"correlations.sps:6.25-6.26: error: CORRELATIONS: Syntax error expecting variable name.
- 6 | CORRELATIONS x y z WITH **.
- | ^~"
-
-"correlations.sps:7.1-7.12: error: CORRELATIONS: No variables specified.
- 7 | CORRELATIONS.
- | ^~~~~~~~~~~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([CROSSTABS procedure])
-
-dnl Based on bug #60982.
-AT_SETUP([CROSSTABS residuals])
-AT_DATA([crosstabs.sps],
- [DATASET CLOSE ALL.
-DATA LIST LIST NOTABLE/ r c n.
-BEGIN DATA
-1 1 26
-1 2 31
-2 1 12
-2 2 32
-3 1 27
-3 2 18
-4 1 8
-4 2 7
-END DATA.
-WEIGHT by n.
-CROSSTABS r by c /STATISTICS=CHISQ
-/CELLS=COUNT EXPECTED RESID SRESID ASRESID.
-])
-AT_CHECK([pspp -O format=csv crosstabs.sps], [0],
- [Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-r × c,161.00,100.0%,.00,.0%,161.00,100.0%
-
-Table: r × c
-,,,c,,Total
-,,,1.00,2.00,
-r,1.00,Count,26.00,31.00,57.00
-,,Expected,25.84,31.16,.35
-,,Residual,.16,-.16,
-,,Std. Residual,.03,-.03,
-,,Adjusted Residual,.05,-.05,
-,2.00,Count,12.00,32.00,44.00
-,,Expected,19.95,24.05,.27
-,,Residual,-7.95,7.95,
-,,Std. Residual,-1.78,1.62,
-,,Adjusted Residual,-2.82,2.82,
-,3.00,Count,27.00,18.00,45.00
-,,Expected,20.40,24.60,.28
-,,Residual,6.60,-6.60,
-,,Std. Residual,1.46,-1.33,
-,,Adjusted Residual,2.33,-2.33,
-,4.00,Count,8.00,7.00,15.00
-,,Expected,6.80,8.20,.09
-,,Residual,1.20,-1.20,
-,,Std. Residual,.46,-.42,
-,,Adjusted Residual,.65,-.65,
-Total,,Count,73.00,88.00,161.00
-,,Expected,.45,.55,1.00
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed)
-Pearson Chi-Square,10.09,3.00,.018
-Likelihood Ratio,10.35,3.00,.016
-Linear-by-Linear Association,1.96,1.00,.162
-N of Valid Cases,161.00,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS integer mode crash])
-AT_DATA([crosstabs.sps],
- [DATA LIST LIST /A * B * X * Y * .
-BEGIN DATA.
-2 3 4 5
-END DATA.
-
-CROSSTABS VARIABLES X (1,7) Y (1,7) /TABLES X BY Y.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
-AT_CHECK([cat pspp.csv], [0],
- [[Table: Reading free-form data from INLINE.
-Variable,Format
-A,F8.0
-B,F8.0
-X,F8.0
-Y,F8.0
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X × Y,1,100.0%,0,.0%,1,100.0%
-
-Table: X × Y
-,,,Y,,,,,,,Total
-,,,1.00,2.00,3.00,4.00,5.00,6.00,7.00,
-X,1.00,Count,0,0,0,0,0,0,0,0
-,2.00,Count,0,0,0,0,0,0,0,0
-,3.00,Count,0,0,0,0,0,0,0,0
-,4.00,Count,0,0,0,0,1,0,0,1
-,5.00,Count,0,0,0,0,0,0,0,0
-,6.00,Count,0,0,0,0,0,0,0,0
-,7.00,Count,0,0,0,0,0,0,0,0
-Total,,Count,0,0,0,0,1,0,0,1
-]])
-AT_CLEANUP
-
-# Bug #47600.
-AT_SETUP([CROSSTABS integer mode crash 2])
-AT_DATA([crosstabs.sps], [dnl
-DATA LIST lIST /x y.
-BEGIN DATA.
-4 5
-END DATA.
-
-CROSSTABS
- VARIABLES x (1,3) y (1,7)
- /TABLES x BY y.
-])
-AT_CHECK([pspp -O format=csv crosstabs.sps], [0],
- [[Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-y,F8.0
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,0,.0%,1,100.0%,1,100.0%
-
-Table: x × y
-,,,y,,,,,,,Total
-,,,1.00,2.00,3.00,4.00,5.00,6.00,7.00,
-x,1.00,Count,,,,,,,,
-,2.00,Count,,,,,,,,
-,3.00,Count,,,,,,,,
-Total,,Count,,,,,,,,
-]])
-AT_CLEANUP
-
-# Bug #22037.
-AT_SETUP([CROSSTABS long string crash])
-AT_DATA([crosstabs.sps],
- [data list list /x * y (a18).
-
-begin data.
-
- 1. 'zero none'
-
-1 'one unity'
-2 'two duality'
-3 'three lots'
-end data.
-
-CROSSTABS /TABLES = x BY y.
-])
-AT_CHECK([pspp -o - -O format=csv -o pspp.txt crosstabs.sps], [0],
- [[Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-y,A18
-
-"crosstabs.sps:4: warning: Missing value(s) for all variables from x onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"crosstabs.sps:6: warning: Missing value(s) for all variables from x onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,4,66.7%,2,33.3%,6,100.0%
-
-Table: x × y
-,,,y,,,,Total
-,,,one unity,three lots,two duality,zero none,
-x,1.00,Count,1,0,0,1,2
-,2.00,Count,0,0,1,0,1
-,3.00,Count,0,1,0,0,1
-Total,,Count,1,1,1,1,4
-]])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS crash])
-AT_DATA([crosstabs.sps],
- [[DATA LIST FIXED
- / x 1-2
- y 3
- z 4.
-
-BEGIN DATA.
-0111
-0222
-0311
-0412
-0521
-0612
-0711
-0811
-0912
-END DATA.
-
-LIST.
-
-
-CROSSTABS TABLES y by z.
-]])
-AT_CHECK([pspp -o - -O format=csv -o pspp.txt crosstabs.sps], [0],
- [Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-x,1,1-2,F2.0
-y,1,3-3,F1.0
-z,1,4-4,F1.0
-
-Table: Data List
-x,y,z
-1,1,1
-2,2,2
-3,1,1
-4,1,2
-5,2,1
-6,1,2
-7,1,1
-8,1,1
-9,1,2
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-y × z,9,100.0%,0,.0%,9,100.0%
-
-Table: y × z
-,,,z,,Total
-,,,1,2,
-y,1,Count,4,3,7
-,2,Count,1,1,2
-Total,,Count,5,4,9
-])
-AT_CLEANUP
-
-# Bug #26739, which caused CROSSTABS to crash or to fail to output
-# chi-square results.
-AT_SETUP([CROSSTABS chi-square crash])
-AT_DATA([crosstabs.sps],
- [[DATA LIST LIST /x * y *.
-BEGIN DATA.
-2 2
-3 1
-4 2
-4 1
-END DATA.
-
-CROSSTABS
- /TABLES= x BY y
- /STATISTICS=CHISQ.
-]])
-AT_CHECK([pspp -O format=csv crosstabs.sps], [0],
- [[Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-y,F8.0
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,4,100.0%,0,.0%,4,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,2.00,Count,0,1,1
-,3.00,Count,1,0,1
-,4.00,Count,1,1,2
-Total,,Count,2,2,4
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed)
-Pearson Chi-Square,2.00,2,.368
-Likelihood Ratio,2.77,2,.250
-Linear-by-Linear Association,.27,1,.602
-N of Valid Cases,4,,
-]])
-AT_CLEANUP
-
-# Bug #27883.
-AT_SETUP([CROSSTABS crash with SPLIT FILE])
-AT_DATA([crosstabs.sps],
- [data list notable / v0 to v2 1-6 (A)
-begin data.
-a c e
-a c e
-a c e
-a d e
-a d f
-b d f
-b d f
-b c f
-b d e
-a c f
-end data.
-SORT CASES BY v0.
-SPLIT FILE SEPARATE BY v0.
-
-CROSSTABS
- /TABLES= v1 BY v2
- /FORMAT=AVALUE TABLES
- /STATISTICS=CHISQ
- /CELLS=COUNT ROW COLUMN TOTAL.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Split Values
-Variable,Value
-v0,a
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-v1 × v2,6,100.0%,0,.0%,6,100.0%
-
-Table: v1 × v2
-,,,v2,,Total
-,,,e,f,
-v1,c,Count,3,1,4
-,,Row %,75.0%,25.0%,100.0%
-,,Column %,75.0%,50.0%,66.7%
-,,Total %,50.0%,16.7%,66.7%
-,d,Count,1,1,2
-,,Row %,50.0%,50.0%,100.0%
-,,Column %,25.0%,50.0%,33.3%
-,,Total %,16.7%,16.7%,33.3%
-Total,,Count,4,2,6
-,,Row %,66.7%,33.3%,100.0%
-,,Column %,100.0%,100.0%,100.0%
-,,Total %,66.7%,33.3%,100.0%
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
-Pearson Chi-Square,.38,1,.540,,
-Likelihood Ratio,.37,1,.545,,
-Fisher's Exact Test,,,,1.000,.600
-Continuity Correction,.00,1,1.000,,
-N of Valid Cases,6,,,,
-
-Table: Split Values
-Variable,Value
-v0,b
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-v1 × v2,4,100.0%,0,.0%,4,100.0%
-
-Table: v1 × v2
-,,,v2,,Total
-,,,e,f,
-v1,c,Count,0,1,1
-,,Row %,.0%,100.0%,100.0%
-,,Column %,.0%,33.3%,25.0%
-,,Total %,.0%,25.0%,25.0%
-,d,Count,1,2,3
-,,Row %,33.3%,66.7%,100.0%
-,,Column %,100.0%,66.7%,75.0%
-,,Total %,25.0%,50.0%,75.0%
-Total,,Count,1,3,4
-,,Row %,25.0%,75.0%,100.0%
-,,Column %,100.0%,100.0%,100.0%
-,,Total %,25.0%,75.0%,100.0%
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
-Pearson Chi-Square,.44,1,.505,,
-Likelihood Ratio,.68,1,.410,,
-Fisher's Exact Test,,,,1.000,.750
-Continuity Correction,.00,1,1.000,,
-N of Valid Cases,4,,,,
-])
-AT_CLEANUP
-
-# Bug #24752.
-AT_SETUP([3-way CROSSTABS])
-AT_DATA([crosstabs.sps],
- [[DATA LIST FIXED
- / x 1-2
- y 3
- z 4.
-
-BEGIN DATA.
-0111
-0222
-0311
-0412
-0521
-0612
-0711
-0811
-0912
-END DATA.
-
-LIST.
-
-
-CROSSTABS TABLES x BY y BY z/STATISTICS=ALL.
-]])
-AT_CHECK([pspp -o - -O format=csv -o pspp.csv -o pspp.txt crosstabs.sps], [0],
- [Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-x,1,1-2,F2.0
-y,1,3-3,F1.0
-z,1,4-4,F1.0
-
-Table: Data List
-x,y,z
-1,1,1
-2,2,2
-3,1,1
-4,1,2
-5,2,1
-6,1,2
-7,1,1
-8,1,1
-9,1,2
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y × z,9,100.0%,0,.0%,9,100.0%
-
-Table: x × y × z
-,,,,,y,,Total
-,,,,,1,2,
-z,1,x,1,Count,1,0,1
-,,,3,Count,0,0,1
-,,,5,Count,1,0,1
-,,,7,Count,0,0,1
-,,,8,Count,0,1,1
-,,Total,,Count,4,1,5
-,2,x,2,Count,0,0,1
-,,,4,Count,0,1,1
-,,,6,Count,0,0,1
-,,,9,Count,1,0,1
-,,Total,,Count,3,1,4
-
-Table: Chi-Square Tests
-,,,Value,df,Asymptotic Sig. (2-tailed)
-z,1,Pearson Chi-Square,5.00,4,.287
-,,Likelihood Ratio,5.00,4,.287
-,,Linear-by-Linear Association,.01,1,.938
-,,N of Valid Cases,5,,
-,2,Pearson Chi-Square,4.00,3,.261
-,,Likelihood Ratio,4.50,3,.212
-,,Linear-by-Linear Association,1.58,1,.209
-,,N of Valid Cases,4,,
-
-Table: Symmetric Measures
-,,,,Value,Asymp. Std. Error,Approx. T
-z,1,Nominal by Nominal,Phi,1.00,,
-,,,Cramer's V,1.00,,
-,,,Contingency Coefficient,.71,,
-,,Ordinal by Ordinal,Kendall's tau-b,.00,.32,.00
-,,,Kendall's tau-c,.00,.32,.00
-,,,Gamma,.00,.50,.00
-,,,Spearman Correlation,.00,.22,.00
-,,Interval by Interval,Pearson's R,.04,.22,.07
-,,N of Valid Cases,,5,,
-,2,Nominal by Nominal,Phi,1.00,,
-,,,Cramer's V,1.00,,
-,,,Contingency Coefficient,.71,,
-,,Ordinal by Ordinal,Kendall's tau-b,-.71,.20,-1.73
-,,,Kendall's tau-c,-.75,.43,-1.73
-,,,Gamma,-1.00,.00,-1.73
-,,,Spearman Correlation,-.77,.17,-1.73
-,,Interval by Interval,Pearson's R,-.73,.18,-1.49
-,,N of Valid Cases,,4,,
-
-Table: Directional Measures
-,,,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-z,1,Nominal by Nominal,Lambda,Symmetric,.40,.28,1.12,.264
-,,,,x Dependent,.25,.22,1.12,.264
-,,,,y Dependent,1.00,.00,1.12,.264
-,,,Goodman and Kruskal tau,x Dependent,.25,,,
-,,,,y Dependent,1.00,,,
-,,,Uncertainty Coefficient,Symmetric,.47,.18,,
-,,,,x Dependent,.31,.15,2.02,
-,,,,y Dependent,1.00,.00,2.02,
-,,Ordinal by Ordinal,Somers' d,Symmetric,.00,,.00,1.000
-,,,,x Dependent,.00,.50,.00,1.000
-,,,,y Dependent,.00,.20,.00,1.000
-,,Nominal by Interval,Eta,x Dependent,.04,,,
-,,,,y Dependent,1.00,,,
-,2,Nominal by Nominal,Lambda,Symmetric,.50,.25,2.00,.046
-,,,,x Dependent,.33,.27,1.15,.248
-,,,,y Dependent,1.00,.00,1.15,.248
-,,,Goodman and Kruskal tau,x Dependent,.33,,,
-,,,,y Dependent,1.00,,,
-,,,Uncertainty Coefficient,Symmetric,.58,.17,,
-,,,,x Dependent,.41,.17,2.36,
-,,,,y Dependent,1.00,.00,2.36,
-,,Ordinal by Ordinal,Somers' d,Symmetric,-.67,,-1.73,.083
-,,,,x Dependent,-1.00,.00,-1.73,.083
-,,,,y Dependent,-.50,.29,-1.73,.083
-,,Nominal by Interval,Eta,x Dependent,.73,,,
-,,,,y Dependent,1.00,,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS rounding weights with COUNT])
-AT_DATA([crosstabs.sps],
- [[DATA LIST NOTABLE LIST /x y w.
-BEGIN DATA.
-1 1 1.4
-1 1 1.4
-1 2 1.6
-1 2 1.6
-2 1 1
-2 2 2
-END DATA.
-WEIGHT BY w.
-
-* These should have the same effect (no rounding).
-CROSSTABS /TABLES x BY y.
-CROSSTABS /TABLES x BY y /COUNT ASIS.
-
-* Round input weights.
-CROSSTABS /TABLES x BY y /COUNT CASE ROUND.
-CROSSTABS /TABLES x BY y /COUNT CASE TRUNCATE.
-
-* Round cell weights.
-CROSSTABS /TABLES x BY y /COUNT.
-CROSSTABS /TABLES x BY y /COUNT TRUNCATE.
-]])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
-AT_CHECK([cat pspp.csv], [0],
- [Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,9.00,100.0%,.00,.0%,9.00,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,1.00,Count,2.80,3.20,6.00
-,2.00,Count,1.00,2.00,3.00
-Total,,Count,3.80,5.20,9.00
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,9.00,100.0%,.00,.0%,9.00,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,1.00,Count,2.80,3.20,6.00
-,2.00,Count,1.00,2.00,3.00
-Total,,Count,3.80,5.20,9.00
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,9.00,100.0%,.00,.0%,9.00,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,1.00,Count,2.00,4.00,6.00
-,2.00,Count,1.00,2.00,3.00
-Total,,Count,3.00,6.00,9.00
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,7.00,100.0%,.00,.0%,7.00,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,1.00,Count,2.00,2.00,4.00
-,2.00,Count,1.00,2.00,3.00
-Total,,Count,3.00,4.00,7.00
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,9.00,100.0%,.00,.0%,9.00,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,1.00,Count,3.00,3.00,6.00
-,2.00,Count,1.00,2.00,3.00
-Total,,Count,4.00,5.00,9.00
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,8.00,100.0%,.00,.0%,8.00,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,1.00,2.00,
-x,1.00,Count,2.00,3.00,5.00
-,2.00,Count,1.00,2.00,3.00
-Total,,Count,3.00,5.00,8.00
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS descending sort order])
-AT_DATA([crosstabs-descending.sps],
- [[DATA LIST NOTABLE LIST /x * y *.
-BEGIN DATA.
-2 2
-2 2
-3 1
-4 1
-3 2
-3 2
-END DATA.
-
-CROSSTABS
- /TABLES= x BY y
- /FORMAT = DVALUE.
-]])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs-descending.sps])
-AT_CHECK([cat pspp.csv], [0],
- [Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,6,100.0%,0,.0%,6,100.0%
-
-Table: x × y
-,,,y,,Total
-,,,2.00,1.00,
-x,4.00,Count,0,1,1
-,3.00,Count,2,1,3
-,2.00,Count,2,0,2
-Total,,Count,4,2,6
-])
-AT_CLEANUP
-
-# Bug #31260.
-AT_SETUP([CROSSTABS crash when all cases missing])
-AT_DATA([crosstabs.sps], [dnl
-DATA LIST LIST NOTABLE /X1 X2.
-BEGIN DATA.
-1 1
-END DATA.
-
-MISSING VALUES x2 (1).
-
-CROSSTABS /TABLES= X1 by X2.
-])
-AT_CHECK([pspp -O format=csv crosstabs.sps], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X1 × X2,0,.0%,1,100.0%,1,100.0%
-
-"crosstabs.sps:8.20-8.27: warning: CROSSTABS: Crosstabulation X1 × X2 contained no non-missing cases.
- 8 | CROSSTABS /TABLES= X1 by X2.
- | ^~~~~~~~"
-])
-AT_CLEANUP
-
-
-
-dnl This example comes from http://www.ats.ucla.edu/stat/spss/whatstat/whatstat.htm#chisq
-AT_SETUP([CROSSTABS Fisher Exact Test])
-
-AT_DATA([fisher-exact.sps], [dnl
-SET FORMAT F12.3.
-SET DECIMAL DOT.
-
-DATA LIST notable LIST /schtyp (F9.2) female (F9.2) ses (F9.2) .
-begin data.
- 1.00 .00 1.00
- 1.00 1.00 2.00
- 1.00 .00 3.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 3.00
- 1.00 .00 1.00
- 1.00 .00 1.00
- 1.00 .00 3.00
- 2.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 2.00 .00 2.00
- 2.00 .00 3.00
- 1.00 .00 1.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 2.00 .00 3.00
- 1.00 .00 2.00
- 2.00 .00 3.00
- 1.00 .00 3.00
- 2.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 1.00
- 1.00 .00 2.00
- 2.00 .00 2.00
- 2.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 1.00
- 1.00 .00 3.00
- 1.00 .00 1.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 2.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 2.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 1.00
- 1.00 .00 2.00
- 2.00 .00 2.00
- 1.00 .00 2.00
- 2.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 1.00
- 1.00 .00 2.00
- 2.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 1.00
- 1.00 .00 1.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 2.00
- 1.00 .00 1.00
- 1.00 .00 3.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 3.00
- 1.00 .00 1.00
- 2.00 .00 2.00
- 1.00 .00 1.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 3.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 3.00
- 1.00 .00 2.00
- 1.00 .00 1.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 2.00 1.00 3.00
- 1.00 1.00 3.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 2.00 1.00 1.00
- 2.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 2.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 2.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 3.00
- 2.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 2.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 2.00 1.00 3.00
- 1.00 1.00 2.00
- 2.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 1.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 2.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 1.00
- 2.00 1.00 2.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 2.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 2.00
- 1.00 1.00 2.00
- 2.00 1.00 3.00
- 1.00 1.00 1.00
- 1.00 1.00 1.00
- 2.00 1.00 3.00
- 2.00 1.00 2.00
- 1.00 1.00 3.00
- 2.00 1.00 2.00
- 2.00 1.00 2.00
- 1.00 1.00 2.00
- 2.00 1.00 2.00
- 1.00 1.00 2.00
- 1.00 1.00 3.00
-end data.
-
-VARIABLE LABEL schtyp 'type of school'.
-ADD VALUE LABELS female 0 male 1 female.
-ADD VALUE LABELS ses 1 low 2 middle 3 high.
-ADD VALUE LABELS schtyp 1 public 2 private.
-
-crosstabs /tables = schtyp by female /statistic = chisq.
-crosstabs /tables = female by ses /statistic = chisq.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt fisher-exact.sps])
-AT_CHECK([cat pspp.csv], [0], [Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-type of school × female,200,100.0%,0,.0%,200,100.0%
-
-Table: type of school × female
-,,,female,,Total
-,,,male,female,
-type of school,public,Count,77,91,168
-,private,Count,14,18,32
-Total,,Count,91,109,200
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
-Pearson Chi-Square,.047,1,.828,,
-Likelihood Ratio,.047,1,.828,,
-Fisher's Exact Test,,,,.849,.492
-Continuity Correction,.001,1,.981,,
-Linear-by-Linear Association,.047,1,.829,,
-N of Valid Cases,200,,,,
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-female × ses,200,100.0%,0,.0%,200,100.0%
-
-Table: female × ses
-,,,ses,,,Total
-,,,low,middle,high,
-female,male,Count,15,47,29,91
-,female,Count,32,48,29,109
-Total,,Count,47,95,58,200
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed)
-Pearson Chi-Square,4.577,2,.101
-Likelihood Ratio,4.679,2,.096
-Linear-by-Linear Association,3.110,1,.078
-N of Valid Cases,200,,
-])
-
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Pearson's R - 1])
-AT_DATA([pearson.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://www.statisticslectures.com/topics/pearsonr/.
-DATA LIST FREE/x y.
-BEGIN DATA.
-1 4
-3 6
-5 10
-5 12
-6 13
-END DATA.
-CROSSTABS x BY y/STATISTICS=CORR.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,5,100.0%,0,.0%,5,100.0%
-
-Table: x × y
-,,,y,,,,,Total
-,,,4.000,6.000,10.000,12.000,13.000,
-x,1.000,Count,1,0,0,0,0,1
-,3.000,Count,0,1,0,0,0,1
-,5.000,Count,0,0,1,1,0,2
-,6.000,Count,0,0,0,0,1,1
-Total,,Count,1,1,1,1,1,5
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Spearman Correlation,.975,.022,7.550
-Interval by Interval,Pearson's R,.968,.017,6.708
-N of Valid Cases,,5,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Pearson's R - 2])
-AT_DATA([pearson2.sps], [dnl
-SET FORMAT F8.3.
-
-* Checked with http://www.socscistatistics.com/tests/pearson/Default2.aspx.
-DATA LIST FREE/x y.
-BEGIN DATA.
-1 1.5
-2 1.5
-3 4
-4 6
-5 5
-6 7
-7 6.5
-8 9
-9 10.5
-10 11
-END DATA.
-CROSSTABS x BY y/STATISTICS=CORR.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson2.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,10,100.0%,0,.0%,10,100.0%
-
-Table: x × y
-,,,y,,,,,,,,,Total
-,,,1.500,4.000,5.000,6.000,6.500,7.000,9.000,10.500,11.000,
-x,1.000,Count,1,0,0,0,0,0,0,0,0,1
-,2.000,Count,1,0,0,0,0,0,0,0,0,1
-,3.000,Count,0,1,0,0,0,0,0,0,0,1
-,4.000,Count,0,0,0,1,0,0,0,0,0,1
-,5.000,Count,0,0,1,0,0,0,0,0,0,1
-,6.000,Count,0,0,0,0,0,1,0,0,0,1
-,7.000,Count,0,0,0,0,1,0,0,0,0,1
-,8.000,Count,0,0,0,0,0,0,1,0,0,1
-,9.000,Count,0,0,0,0,0,0,0,1,0,1
-,10.000,Count,0,0,0,0,0,0,0,0,1,1
-Total,,Count,2,1,1,1,1,1,1,1,1,10
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Spearman Correlation,.973,.015,11.844
-Interval by Interval,Pearson's R,.971,.017,11.580
-N of Valid Cases,,10,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Pearson's R - 3])
-AT_DATA([pearson3.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://learntech.uwe.ac.uk/da/Default.aspx?pageid=1442.
-DATA LIST FREE/x y.
-BEGIN DATA.
-56 87
-56 91
-65 85
-65 91
-50 75
-25 28
-87 122
-44 66
-35 58
-END DATA.
-CROSSTABS x BY y/STATISTICS=CORR.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson3.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,9,100.0%,0,.0%,9,100.0%
-
-Table: x × y
-,,,y,,,,,,,,Total
-,,,28.000,58.000,66.000,75.000,85.000,87.000,91.000,122.000,
-x,25.000,Count,1,0,0,0,0,0,0,0,1
-,35.000,Count,0,1,0,0,0,0,0,0,1
-,44.000,Count,0,0,1,0,0,0,0,0,1
-,50.000,Count,0,0,0,1,0,0,0,0,1
-,56.000,Count,0,0,0,0,0,1,1,0,2
-,65.000,Count,0,0,0,0,1,0,1,0,2
-,87.000,Count,0,0,0,0,0,0,0,1,1
-Total,,Count,1,1,1,1,1,1,2,1,9
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Spearman Correlation,.911,.068,5.860
-Interval by Interval,Pearson's R,.966,.017,9.915
-N of Valid Cases,,9,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Pearson's R - 4])
-AT_DATA([pearson4.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://psychology.ucdavis.edu/faculty_sites/sommerb/sommerdemo/correlation/hand/pearson_hand.htm.
-DATA LIST FREE/x y.
-BEGIN DATA.
-5 5
-10 20
-6 4
-8 15
-4 11
-4 9
-3 12
-10 18
-2 7
-6 2
-7 14
-9 17
-END DATA.
-CROSSTABS x BY y/STATISTICS=CORR.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson4.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,12,100.0%,0,.0%,12,100.0%
-
-Table: x × y
-,,,y,,,,,,,,,,,,Total
-,,,2.000,4.000,5.000,7.000,9.000,11.000,12.000,14.000,15.000,17.000,18.000,20.000,
-x,2.000,Count,0,0,0,1,0,0,0,0,0,0,0,0,1
-,3.000,Count,0,0,0,0,0,0,1,0,0,0,0,0,1
-,4.000,Count,0,0,0,0,1,1,0,0,0,0,0,0,2
-,5.000,Count,0,0,1,0,0,0,0,0,0,0,0,0,1
-,6.000,Count,1,1,0,0,0,0,0,0,0,0,0,0,2
-,7.000,Count,0,0,0,0,0,0,0,1,0,0,0,0,1
-,8.000,Count,0,0,0,0,0,0,0,0,1,0,0,0,1
-,9.000,Count,0,0,0,0,0,0,0,0,0,1,0,0,1
-,10.000,Count,0,0,0,0,0,0,0,0,0,0,1,1,2
-Total,,Count,1,1,1,1,1,1,1,1,1,1,1,1,12
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Spearman Correlation,.657,.140,2.758
-Interval by Interval,Pearson's R,.667,.132,2.830
-N of Valid Cases,,12,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Pearson's R - 5])
-AT_DATA([pearson5.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://www.statisticslectures.com/topics/pearsonr/.
-DATA LIST FREE/x y.
-BEGIN DATA.
-18 15000
-25 29000
-57 68000
-45 52000
-26 32000
-64 80000
-37 41000
-40 45000
-24 26000
-33 33000
-END DATA.
-CROSSTABS x BY y/STATISTICS=CORR.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt pearson5.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,10,100.0%,0,.0%,10,100.0%
-
-Table: x × y
-,,,y,,,,,,,,,,Total
-,,,15000.00,26000.00,29000.00,32000.00,33000.00,41000.00,45000.00,52000.00,68000.00,80000.00,
-x,18.000,Count,1,0,0,0,0,0,0,0,0,0,1
-,24.000,Count,0,1,0,0,0,0,0,0,0,0,1
-,25.000,Count,0,0,1,0,0,0,0,0,0,0,1
-,26.000,Count,0,0,0,1,0,0,0,0,0,0,1
-,33.000,Count,0,0,0,0,1,0,0,0,0,0,1
-,37.000,Count,0,0,0,0,0,1,0,0,0,0,1
-,40.000,Count,0,0,0,0,0,0,1,0,0,0,1
-,45.000,Count,0,0,0,0,0,0,0,1,0,0,1
-,57.000,Count,0,0,0,0,0,0,0,0,1,0,1
-,64.000,Count,0,0,0,0,0,0,0,0,0,1,1
-Total,,Count,1,1,1,1,1,1,1,1,1,1,10
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Spearman Correlation,1.000,.000,+Infinit
-Interval by Interval,Pearson's R,.992,.004,22.638
-N of Valid Cases,,10,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - 1])
-AT_DATA([lambda.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://www.csupomona.edu/~jlkorey/POWERMUTT/Topics/contingency_tables.html.
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 424
-1 2 213
-1 3 59
-3 1 55
-3 2 188
-3 3 357
-END DATA.
-
-CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,1296.000,100.0%,.000,.0%,1296.000,100.0%
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.423,.021,16.875,.000
-,,x Dependent,.497,.024,15.986,.000
-,,y Dependent,.370,.020,16.339,.000
-,Goodman and Kruskal tau,x Dependent,.382,,,
-,,y Dependent,.198,,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - 2])
-AT_DATA([lambda.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://vassarstats.net.
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 19
-1 2 26
-1 3 8
-2 1 21
-2 2 13
-2 3 5
-3 1 6
-3 2 12
-3 3 27
-END DATA.
-
-CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,137.000,100.0%,.000,.0%,137.000,100.0%
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.259,.081,2.902,.004
-,,x Dependent,.250,.089,2.479,.013
-,,y Dependent,.267,.085,2.766,.006
-,Goodman and Kruskal tau,x Dependent,.129,,,
-,,y Dependent,.123,,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - 3])
-AT_DATA([lambda.sps], [dnl
-SET FORMAT F8.3.
-
-* From Goodman, L.A., Kruskal, W.H. (1954) "Measures of association for
- cross classifications". Part I. Journal of the American Statistical
- Association, 49, 732-764.
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 1768
-1 2 807
-1 3 189
-1 4 47
-2 1 946
-2 2 1387
-2 3 746
-2 4 53
-3 1 115
-3 2 438
-3 3 288
-3 4 16
-END DATA.
-CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,6800.000,100.0%,.000,.0%,6800.000,100.0%
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.208,.010,18.793,.000
-,,x Dependent,.224,.013,16.076,.000
-,,y Dependent,.192,.012,14.438,.000
-,Goodman and Kruskal tau,x Dependent,.089,,,
-,,y Dependent,.081,,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Goodman and Kruskal's lambda - treatment of ties])
-AT_DATA([lambda.sps], [dnl
-SET FORMAT F8.3.
-
-* From Douglas Bonett.
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 225
-1 2 43
-1 3 216
-2 1 3
-2 2 1
-2 3 12
-END DATA.
-
-CROSSTABS x BY y/CELLS=NONE/STATISTICS=LAMBDA.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt lambda.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,500.000,100.0%,.000,.0%,500.000,100.0%
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.031,.013,2.336,.019
-,,x Dependent,.000,.000,NaN,NaN
-,,y Dependent,.033,.014,2.336,.019
-,Goodman and Kruskal tau,x Dependent,.012,,,
-,,y Dependent,.009,,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Somers' D, Tau-B, Tau-C, Gamma - 1])
-AT_DATA([somersd.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://stats.stackexchange.com/questions/72203/problem-with-calculating-asymptotic-standard-error-for-somers-d.
-DATA LIST LIST NOTABLE/x y * w (F10.6).
-WEIGHT BY w.
-BEGIN DATA.
-1 1 0.000025
-1 2 0.0001
-1 3 0.001
-1 4 0.0025
-1 5 0.004
-1 6 0.0075
-1 7 0.0125
-2 1 0.049975
-2 2 0.0999
-2 3 0.199
-2 4 0.2475
-2 5 0.196
-2 6 0.1425
-2 7 0.0375
-END DATA.
-CROSSTABS x BY y/STATISTICS=D/CELLS=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt somersd.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,1.000000,100.0%,.000000,.0%,1.000000,100.0%
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,-.084,,-.149,.882
-,,x Dependent,-.045,.300,-.149,.882
-,,y Dependent,-.684,2.378,-.149,.882
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Somers' D, Tau-B, Tau-C, Gamma - 2])
-AT_DATA([somersd.sps], [dnl
-SET FORMAT F8.3.
-
-* From http://uregina.ca/~gingrich/gamma.pdf.
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 34
-1 2 24
-1 3 15
-2 1 42
-2 2 74
-2 3 67
-3 1 28
-3 2 111
-3 3 292
-END DATA.
-CROSSTABS x BY y/STATISTICS=BTAU CTAU GAMMA D/CELLS=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt somersd.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,687.000,100.0%,.000,.0%,687.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,.372,.033,10.669
-,Kendall's tau-c,.310,.029,10.669
-,Gamma,.591,.043,10.669
-N of Valid Cases,,687.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,.371,,10.669,.000
-,,x Dependent,.351,.032,10.669,.000
-,,y Dependent,.394,.035,10.669,.000
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Somers' D, Tau-B, Tau-C, Gamma - 3])
-AT_DATA([ordinal.sps], [dnl
-SET FORMAT F8.3.
-
-* From https://www.iup.edu/WorkArea/DownloadAsset.aspx?id=9829, "Case 1".
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 2 40
-2 3 80
-3 4 30
-END DATA.
-CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
-
-* Same site, case 2.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 40
-2 3 80
-3 4 30
-END DATA.
-CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
-
-* Same site, case 3.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 4 40
-2 3 80
-3 2 30
-END DATA.
-CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
-
-* Same site, case 4.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 20
-1 2 20
-2 3 80
-3 4 30
-END DATA.
-CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
-
-* Same site, case 5.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 2 40
-2 2 80
-3 2 29
-3 3 1
-END DATA.
-CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
-
-* Same site, case 6.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 3
-1 2 6
-1 3 28
-1 4 61
-2 1 4
-2 2 5
-2 3 21
-2 4 20
-END DATA.
-CROSSTABS x BY y/STATISTICS=GAMMA D BTAU/CELLS=NONE.
-
-* Same site, case 7.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 38
-1 2 6
-1 3 3
-1 4 51
-2 1 4
-2 2 20
-2 3 21
-2 4 5
-END DATA.
-CROSSTABS x BY y/STATISTICS=LAMBDA D PHI GAMMA/CELLS=NONE.
-
-* Same site, case 8.
-DATA LIST LIST NOTABLE /x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 2
-1 2 3
-1 3 5
-1 4 1
-2 1 2
-2 2 16
-2 3 3
-2 4 6
-3 1 3
-3 2 10
-3 3 35
-3 4 27
-4 1 6
-4 2 15
-4 3 33
-4 4 45
-END DATA.
-CROSSTABS x BY y/STATISTICS=LAMBDA D PHI BTAU/CELLS=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt ordinal.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,150.000,100.0%,.000,.0%,150.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,1.000,.000,24.841
-,Gamma,1.000,.000,24.841
-N of Valid Cases,,150.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,1.000,,24.841,.000
-,,x Dependent,1.000,.000,24.841,.000
-,,y Dependent,1.000,.000,24.841,.000
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,150.000,100.0%,.000,.0%,150.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,1.000,.000,24.841
-,Gamma,1.000,.000,24.841
-N of Valid Cases,,150.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,1.000,,24.841,.000
-,,x Dependent,1.000,.000,24.841,.000
-,,y Dependent,1.000,.000,24.841,.000
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,150.000,100.0%,.000,.0%,150.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,-1.000,.000,-24.841
-,Gamma,-1.000,.000,-24.841
-N of Valid Cases,,150.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,-1.000,,-24.841,.000
-,,x Dependent,-1.000,.000,-24.841,.000
-,,y Dependent,-1.000,.000,-24.841,.000
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,150.000,100.0%,.000,.0%,150.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,.972,.007,24.841
-,Gamma,1.000,.000,24.841
-N of Valid Cases,,150.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,.971,,24.841,.000
-,,x Dependent,.944,.013,24.841,.000
-,,y Dependent,1.000,.000,24.841,.000
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,150.000,100.0%,.000,.0%,150.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,.119,.059,1.009
-,Gamma,1.000,.000,1.009
-N of Valid Cases,,150.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,.035,,1.009,.313
-,,x Dependent,.805,.032,1.009,.313
-,,y Dependent,.018,.017,1.009,.313
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,148.000,100.0%,.000,.0%,148.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Ordinal by Ordinal,Kendall's tau-b,-.208,.078,-2.641
-,Gamma,-.381,.130,-2.641
-N of Valid Cases,,148.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Ordinal by Ordinal,Somers' d,Symmetric,-.206,,-2.641,.008
-,,x Dependent,-.182,.069,-2.641,.008
-,,y Dependent,-.237,.089,-2.641,.008
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,148.000,100.0%,.000,.0%,148.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Nominal by Nominal,Phi,.731,,
-,Cramer's V,.731,,
-Ordinal by Ordinal,Gamma,-.110,.107,-1.022
-N of Valid Cases,,148.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.338,.059,4.743,.000
-,,x Dependent,.640,.085,4.875,.000
-,,y Dependent,.174,.050,3.248,.001
-,Goodman and Kruskal tau,x Dependent,.534,,,
-,,y Dependent,.167,,,
-Ordinal by Ordinal,Somers' d,Symmetric,-.074,,-1.022,.307
-,,x Dependent,-.060,.059,-1.022,.307
-,,y Dependent,-.096,.094,-1.022,.307
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,212.000,100.0%,.000,.0%,212.000,100.0%
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Nominal by Nominal,Phi,.432,,
-,Cramer's V,.249,,
-Ordinal by Ordinal,Kendall's tau-b,.209,.062,3.338
-N of Valid Cases,,212.000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.102,.067,1.473,.141
-,,x Dependent,.027,.087,.302,.763
-,,y Dependent,.165,.065,2.349,.019
-,Goodman and Kruskal tau,x Dependent,.051,,,
-,,y Dependent,.068,,,
-Ordinal by Ordinal,Somers' d,Symmetric,.209,,3.338,.001
-,,x Dependent,.202,.060,3.338,.001
-,,y Dependent,.217,.064,3.338,.001
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS Cohens Kappa])
-
-dnl Example from Wood J. M.
-dnl "Understanding and Computing Cohen's Kappa: A Tutorial"
-dnl WebPsychEmpiricist. Oct 3 2007
-AT_DATA([kappa.sps], [dnl
-SET FORMAT=F8.3.
-
-data list notable list /p1 * p2 * w *.
-begin data.
-0 0 18
-1 0 1
-0 1 1
-end data.
-
-weight by w.
-
-crosstabs /table = p1 by p2
- /statistics = kappa
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt kappa.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-p1 × p2,20.000,100.0%,.000,.0%,20.000,100.0%
-
-Table: p1 × p2
-,,,p2,,Total
-,,,.000,1.000,
-p1,.000,Count,18.000,1.000,19.000
-,1.000,Count,1.000,.000,1.000
-Total,,Count,19.000,1.000,20.000
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Measure of Agreement,Kappa,-.053,.037,-.235
-N of Valid Cases,,20.000,,
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([CROSSTABS many statistics])
-AT_DATA([crosstabs.sps], [dnl
-SET FORMAT=F8.4.
-
-* From http://www4.stat.ncsu.edu/~dzhang2/st744/table3.9.lst.txt.
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 25
-1 2 25
-1 3 12
-2 2 1
-2 3 3
-END DATA.
-CROSSTABS x BY y/STATISTICS=CHISQ PHI CC LAMBDA UC BTAU CTAU GAMMA D CORR/CELLS=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt crosstabs.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,66.0000,100.0%,.0000,.0%,66.0000,100.0%
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed)
-Pearson Chi-Square,6.9562,2.0000,.031
-Likelihood Ratio,6.6901,2.0000,.035
-Linear-by-Linear Association,5.8450,1.0000,.016
-N of Valid Cases,66.0000,,
-
-Table: Symmetric Measures
-,,Value,Asymp. Std. Error,Approx. T
-Nominal by Nominal,Phi,.3246,,
-,Cramer's V,.3246,,
-,Contingency Coefficient,.3088,,
-Ordinal by Ordinal,Kendall's tau-b,.2752,.0856,1.9920
-,Kendall's tau-c,.1497,.0751,1.9920
-,Gamma,.8717,.1250,1.9920
-,Spearman Correlation,.2908,.0906,2.4311
-Interval by Interval,Pearson's R,.2999,.0973,2.5147
-N of Valid Cases,,66.0000,,
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.0455,.1629,.2723,.785
-,,x Dependent,.0000,.0000,NaN,NaN
-,,y Dependent,.0500,.1791,.2723,.785
-,Goodman and Kruskal tau,x Dependent,.1054,,,
-,,y Dependent,.0434,,,
-,Uncertainty Coefficient,Symmetric,.0780,.0474,,
-,,x Dependent,.2217,.1062,1.5373,
-,,y Dependent,.0473,.0306,1.5373,
-Ordinal by Ordinal,Somers' d,Symmetric,.1960,,1.9920,.046
-,,x Dependent,.1152,.0572,1.9920,.046
-,,y Dependent,.6573,.1417,1.9920,.046
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS uncertainy coefficient])
-AT_DATA([uc.sps], [dnl
-* From http://groups.chass.utoronto.ca/pol242/5bMeasuringAssociation.htm.
-SET FORMAT=F8.3.
-
-DATA LIST LIST NOTABLE/x y w.
-WEIGHT BY w.
-BEGIN DATA.
-1 1 416
-1 2 121
-2 1 335
-2 2 2
-3 1 112
-3 2 1
-END DATA.
-CROSSTABS x BY y/STATISTICS=LAMBDA UC/CELLS=NONE.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt uc.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x × y,987.000,100.0%,.000,.0%,987.000,100.0%
-
-Table: Directional Measures
-,,,Value,Asymp. Std. Error,Approx. T,Approx. Sig.
-Nominal by Nominal,Lambda,Symmetric,.000,.000,NaN,NaN
-,,x Dependent,.000,.000,NaN,NaN
-,,y Dependent,.000,.000,NaN,NaN
-,Goodman and Kruskal tau,x Dependent,.076,,,
-,,y Dependent,.108,,,
-,Uncertainty Coefficient,Symmetric,.105,.012,,
-,,x Dependent,.073,.009,7.890,
-,,y Dependent,.184,.019,7.890,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS estimated risk])
-dnl Example data and expected output from
-dnl http://www.psychology.nottingham.ac.uk/staff/ddc/c8cxpa/further/Project_resources/SPSSCrosstabW.pdf
-AT_DATA([risk.sps], [dnl
-DATA LIST LIST /factor disease count (F8.0).
-WEIGHT BY count.
-VALUE LABELS /factor 0 'Placebo' 1 'Aspirin'
- /disease 1 'No' 0 'Yes'.
-BEGIN DATA.
-0 1 80
-0 0 20
-1 1 135
-1 0 15
-END DATA.
-CROSSTABS factor BY disease/STATISTICS=RISK CHISQ.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt risk.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-factor,F8.0
-disease,F8.0
-count,F8.0
-
-Table: Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-factor × disease,250,100.0%,0,.0%,250,100.0%
-
-Table: factor × disease
-,,,disease,,Total
-,,,Yes,No,
-factor,Placebo,Count,20,80,100
-,Aspirin,Count,15,135,150
-Total,,Count,35,215,250
-
-Table: Chi-Square Tests
-,Value,df,Asymptotic Sig. (2-tailed),Exact Sig. (2-tailed),Exact Sig. (1-tailed)
-Pearson Chi-Square,4.98,1,.026,,
-Likelihood Ratio,4.88,1,.027,,
-Fisher's Exact Test,,,,.039,.021
-Continuity Correction,4.19,1,.041,,
-Linear-by-Linear Association,4.96,1,.026,,
-N of Valid Cases,250,,,,
-
-Table: Risk Estimate
-,Value,95% Confidence Interval,
-,,Lower,Upper
-Odds Ratio for factor (Placebo / Aspirin),2.25,2.25,2.25
-For cohort disease = Yes,1.08,1.08,1.08
-For cohort disease = No,.99,.99,.99
-N of Valid Cases,250.00,,
-])
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS barchart])
-AT_DATA([bc.sps], [dnl
-SET FORMAT=F8.3.
-
-DATA LIST LIST NOTABLE /x (a20) y (f8) z (f8) w (f8) .
-BEGIN DATA.
-This 1 0 416
-That 2 0 121
-Other 2 0 335
-This 2 0 231
-That 3 0 112
-Other 4 0 130
-This 1 1 160
-That 2 1 211
-Other 2 1 352
-This 2 1 212
-That 3 1 121
-Other 4 1 101
-END DATA.
-
-WEIGHT BY w.
-
-CROSSTABS
- /table x BY y BY z
- /table x BY y
- /barchart.
-])
-
-AT_CHECK([pspp -O format=txt -o xxx bc.sps], [0], [ignore])
-
-AT_CHECK([test -e xxx-1.png], [0], [ignore])
-AT_CHECK([test -e xxx-2.png], [0], [ignore])
-
-AT_CHECK([diff xxx-1.png xxx-2.png], [0], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([CROSSTABS syntax errors])
-AT_DATA([crosstabs.sps], [dnl
-DATA LIST LIST NOTABLE/x y v1 to v100.
-CROSSTABS TABLES=x BY y/VARIABLES **.
-CROSSTABS VARIABLES=**.
-CROSSTABS VARIABLES=x **.
-CROSSTABS VARIABLES=x (**).
-CROSSTABS VARIABLES=x (1,**).
-CROSSTABS VARIABLES=x (1,5**).
-CROSSTABS MISSING=**.
-CROSSTABS COUNT=**.
-CROSSTABS FORMAT=**.
-CROSSTABS CELLS=**.
-CROSSTABS STATISTICS=**.
-CROSSTABS **.
-CROSSTABS
- v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100.
-CROSSTABS BARCHART.
-CROSSTABS x BY y/MISSING=REPORT.
-])
-AT_CHECK([pspp -O format=csv crosstabs.sps], [1], [dnl
-"crosstabs.sps:2.25-2.33: error: CROSSTABS: VARIABLES must be specified before TABLES.
- 2 | CROSSTABS TABLES=x BY y/VARIABLES **.
- | ^~~~~~~~~"
-
-"crosstabs.sps:3.21-3.22: error: CROSSTABS: Syntax error expecting variable name.
- 3 | CROSSTABS VARIABLES=**.
- | ^~"
-
-"crosstabs.sps:4.23-4.24: error: CROSSTABS: Syntax error expecting `('.
- 4 | CROSSTABS VARIABLES=x **.
- | ^~"
-
-"crosstabs.sps:5.24-5.25: error: CROSSTABS: Syntax error expecting integer.
- 5 | CROSSTABS VARIABLES=x (**).
- | ^~"
-
-"crosstabs.sps:6.26-6.27: error: CROSSTABS: Syntax error expecting positive integer.
- 6 | CROSSTABS VARIABLES=x (1,**).
- | ^~"
-
-"crosstabs.sps:7.27-7.28: error: CROSSTABS: Syntax error expecting `)'.
- 7 | CROSSTABS VARIABLES=x (1,5**).
- | ^~"
-
-"crosstabs.sps:8.19-8.20: error: CROSSTABS: Syntax error expecting TABLE, INCLUDE, or REPORT.
- 8 | CROSSTABS MISSING=**.
- | ^~"
-
-"crosstabs.sps:9.17-9.18: error: CROSSTABS: Syntax error expecting ASIS, CASE, CELL, ROUND, or TRUNCATE.
- 9 | CROSSTABS COUNT=**.
- | ^~"
-
-"crosstabs.sps:10.18-10.19: error: CROSSTABS: Syntax error expecting AVALUE, DVALUE, TABLES, or NOTABLES.
- 10 | CROSSTABS FORMAT=**.
- | ^~"
-
-"crosstabs.sps:11.17-11.18: error: CROSSTABS: Syntax error expecting COUNT, EXPECTED, ROW, COLUMN, TOTAL, RESIDUAL, SRESIDUAL, or ASRESIDUAL.
- 11 | CROSSTABS CELLS=**.
- | ^~"
-
-"crosstabs.sps:12.22-12.23: error: CROSSTABS: Syntax error expecting one of the following: CHISQ, PHI, CC, LAMBDA, UC, BTAU, CTAU, RISK, GAMMA, D, KAPPA, ETA, CORR.
- 12 | CROSSTABS STATISTICS=**.
- | ^~"
-
-"crosstabs.sps:13.11-13.12: error: CROSSTABS: Syntax error expecting subcommand name or variable name.
- 13 | CROSSTABS **.
- | ^~"
-
-"crosstabs.sps:15.8-16.59: error: CROSSTABS: Too many cross-tabulation variables or dimensions.
- 15 | v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- 16 | BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100 BY v1 to v100
- | -----------------------------------------------------------"
-
-crosstabs.sps:21: error: CROSSTABS: At least one crosstabulation must be requested (using the TABLES subcommand).
-
-"crosstabs.sps:22.26-22.31: warning: CROSSTABS: Missing mode REPORT not allowed in general mode. Assuming MISSING=TABLE.
- 22 | CROSSTABS x BY y/MISSING=REPORT.
- | ^~~~~~"
-
-error: CROSSTABS: At end of input: Syntax error expecting `BEGIN DATA'.
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-AT_BANNER([CTABLES])
-
-dnl Known bugs:
-dnl TOTAL interaction with PCOMPUTE, e.g. the following
-dnl CTABLES
-dnl /PCOMPUTE &all_drivers=EXPR([1 THRU 2] + [3 THRU 4])
-dnl /PPROPERTIES &all_drivers LABEL='All Drivers'
-dnl /PCOMPUTE &pct_not_drivers=EXPR([5] / ([1 THRU 2] + [3 THRU 4] + [5]) * 100)
-dnl /PPROPERTIES &pct_not_drivers LABEL='% Not Drivers' FORMAT=COUNT PCT40.1
-dnl /TABLE=qn1 BY qns3a
-dnl /CATEGORIES VARIABLES=qns3a TOTAL=YES
-dnl /CATEGORIES VARIABLES=qn1 [1 THRU 2, SUBTOTAL='Frequent Drivers',
-dnl 3 THRU 4, SUBTOTAL='Infrequent Drivers',
-dnl &all_drivers, 5, &pct_not_drivers,
-dnl MISSING, SUBTOTAL='Not Drivers or Missing'].
-dnl yields gaps in the Total column:
-dnl ╭─────────────────────────────────────────────────────────────────────────┬──────────────────╮
-dnl │ │ S3a. GENDER: │
-dnl │ ├─────┬──────┬─────┤
-dnl │ │ Male│Female│Total│
-dnl │ ├─────┼──────┼─────┤
-dnl │ │Count│ Count│Count│
-dnl ├─────────────────────────────────────────────────────────────────────────┼─────┼──────┼─────┤
-dnl │ 1. How often do you usually drive a car or other Every day │ 2305│ 2362│ 4667│
-dnl │motor vehicle? Several days a week │ 440│ 834│ 1274│
-dnl │ Frequent Drivers │ 2745│ 3196│ │
-dnl │ Once a week or less │ 125│ 236│ 361│
-dnl │ Only certain times a │ 58│ 72│ 130│
-dnl │ year │ │ │ │
-dnl │ Infrequent Drivers │ 183│ 308│ │
-dnl │ All Drivers │ 2928│ 3504│ │
-dnl │ Never │ 192│ 348│ 540│
-dnl │ % Not Drivers │ 6.2%│ 9.0%│ │
-dnl │ Don't know │ 3│ 5│ 8│
-dnl │ Refused │ 9│ 10│ 19│
-dnl │ Not Drivers or │ 204│ 363│ │
-dnl │ Missing │ │ │ │
-dnl ╰─────────────────────────────────────────────────────────────────────────┴─────┴──────┴─────╯
-dnl Features not yet implemented:
-dnl - Multiple response sets
-dnl - MRSETS subcommand.
-dnl - CATEGORIES: Special case for explicit category specifications and multiple dichotomy sets.
-dnl - SIGTEST
-dnl - COMPARETEST
-dnl - Summary functions:
-dnl * .LCL and .UCL suffixes.
-dnl * .SE suffixes.
-dnl - CATEGORIES:
-dnl * Data-dependent sorting.
-
-AT_SETUP([CTABLES parsing])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /FORMAT MINCOLWIDTH=10 MAXCOLWIDTH=20 UNITS=POINTS EMPTY=ZERO MISSING="x"
- /FORMAT MINCOLWIDTH=DEFAULT MAXCOLWIDTH=DEFAULT UNITS=INCHES EMPTY=BLANK MISSING="."
- /FORMAT UNITS=CM EMPTY="(-)"
- /VLABELS VARIABLES=qn1 DISPLAY=DEFAULT
- /VLABELS VARIABLES=qn17 DISPLAY=NAME
- /VLABELS VARIABLES=qns3a DISPLAY=LABEL
- /VLABELS VARIABLES=qnd1 DISPLAY=BOTH
- /VLABELS VARIABLES=qn20 DISPLAY=NONE
- /MRSETS COUNTDUPLICATES=NO
- /MRSETS COUNTDUPLICATES=YES
- /SMISSING VARIABLE
- /SMISSING LISTWISE
- /WEIGHT VARIABLE=qns3a
- /HIDESMALLCOUNTS
- /HIDESMALLCOUNTS COUNT=10
- /TABLE qnsa1
- /SLABELS POSITION=COLUMN VISIBLE=YES
- /SLABELS VISIBLE=NO POSITION=ROW
- /SLABELS POSITION=LAYER
- /CLABELS AUTO
- /CLABELS ROWLABELS=OPPOSITE
- /CRITERIA CILEVEL=50
- /CATEGORIES VARIABLES=qn1 qn17
- ORDER=A KEY=VALUE MISSING=INCLUDE TOTAL=YES LABEL="xyzzy"
- POSITION=BEFORE EMPTY=INCLUDE.
-CTABLES /TABLE qnsa1 /CLABELS ROWLABELS=LAYER.
-CTABLES /TABLE qnsa1 /CLABELS COLLABELS=OPPOSITE.
-CTABLES /TABLE qnsa1 /CLABELS COLLABELS=LAYER.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-Count
-╭───────────────────┬────┬────╮
-│ │ RDD│CELL│
-├───────────────────┼────┼────┤
-│Sa1. SAMPLE SOURCE:│5392│1607│
-╰───────────────────┴────┴────╯
-
- Custom Tables
-RDD
-╭───────────────────┬─────╮
-│ │Count│
-├───────────────────┼─────┤
-│Sa1. SAMPLE SOURCE:│ 5392│
-╰───────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────┬─────╮
-│ │Count│
-├────────────────────────┼─────┤
-│Sa1. SAMPLE SOURCE: RDD │ 5392│
-│ CELL│ 1607│
-╰────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────┬─────╮
-│ │Count│
-├────────────────────────┼─────┤
-│Sa1. SAMPLE SOURCE: RDD │ 5392│
-│ CELL│ 1607│
-╰────────────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES parsing - negative])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES.
-CTABLES /FORMAT MINCOLWIDTH='foo'.
-CTABLES /TABLE qn1 [**].
-CTABLES /TABLE qn1 [NOTAFUNCTION].
-CTABLES /TABLE (qn1.
-CTABLES /TABLE **.
-CTABLES /TABLE NOTAVAR.
-STRING string(A8).
-CTABLES /TABLE string[S].
-CTABLES /TABLE qn1 [PTILE 101].
-CTABLES /TABLE qn1 [MEAN F0.1].
-CTABLES /TABLE qn1 [MEAN NEGPAREN1.2].
-CTABLES /TABLE qn1 [MEAN NEGPAREN3.4].
-CTABLES /TABLE qn1 [MEAN TOTALS].
-CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [SUBTOTAL=x].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO **].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO THRU x].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1 THRU **].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['x' THRU **].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&**].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&x].
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 101).
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 MISSING=**.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 TOTAL=**.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 LABEL=**.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 POSITION=**.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 EMPTY=**.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 **.
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1,2,3] **.
-CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[0]).
-CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[1**]).
-CTABLES /PCOMPUTE &k=EXPR([LO **]).
-CTABLES /PCOMPUTE &k=EXPR([LO THRU **]).
-CTABLES /PCOMPUTE &k=EXPR([1 THRU **]).
-CTABLES /PCOMPUTE &k=EXPR([1**]).
-CTABLES /PCOMPUTE &k=EXPR((1x)).
-CTABLES /PCOMPUTE **k.
-CTABLES /PCOMPUTE &1.
-CTABLES /PCOMPUTE &k**.
-CTABLES /PCOMPUTE &k=**.
-CTABLES /PCOMPUTE &k=EXPR**.
-CTABLES /PCOMPUTE &k=EXPR(1x).
-CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
-CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=NOTAFUNCTION.
-CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=PTILE **.
-CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k LABEL=**.
-CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k HIDESOURCECATS=**.
-CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k **.
-CTABLES /FORMAT EMPTY=**.
-CTABLES /FORMAT MISSING=**.
-CTABLES /FORMAT **.
-CTABLES /FORMAT MINCOLWIDTH=20 MAXCOLWIDTH=10/.
-CTABLES /VLABELS **.
-CTABLES /VLABELS VARIABLES=NOTAVAR.
-CTABLES /VLABELS VARIABLES=qn1 **.
-CTABLES /VLABELS VARIABLES=qn1 DISPLAY=**.
-CTABLES /MRSETS **.
-CTABLES /MRSETS COUNTDUPLICATES=**.
-CTABLES /SMISSING **.
-CTABLES /WEIGHT **.
-CTABLES /WEIGHT VARIABLE=NOTAVAR.
-CTABLES /HIDESMALLCOUNTS COUNT=1.
-CTABLES /QUUX.
-CTABLES /HIDESMALLCOUNTS COUNT=2.
-CTABLES /TABLE qn1**.
-CTABLES /TABLE qn1 /SLABELS POSITION=**.
-CTABLES /TABLE qn1 /SLABELS VISIBLE=**.
-CTABLES /TABLE qn1 /SLABELS **.
-CTABLES /TABLE qn1 /CLABELS ROWLABELS=**.
-CTABLES /TABLE qn1 /CLABELS COLLABELS=**.
-CTABLES /TABLE qn1 /CLABELS **.
-CTABLES /TABLE qn1 /CRITERIA **.
-CTABLES /TABLE qn1 /CRITERIA CILEVEL=101.
-CTABLES /TABLE qn1 /TITLES **.
-CTABLES /TABLE qn1 /SIGTEST TYPE=**.
-CTABLES /TABLE qn1 /SIGTEST ALPHA=**.
-CTABLES /TABLE qn1 /SIGTEST INCLUDEMRSETS=**.
-CTABLES /TABLE qn1 /SIGTEST CATEGORIES=**.
-CTABLES /TABLE qn1 /SIGTEST **.
-CTABLES /TABLE qn1 /COMPARETEST TYPE=**.
-CTABLES /TABLE qn1 /COMPARETEST ALPHA=**.
-CTABLES /TABLE qn1 /COMPARETEST ALPHA=0,5.
-CTABLES /TABLE qn1 /COMPARETEST ADJUST=**.
-CTABLES /TABLE qn1 /COMPARETEST INCLUDEMRSETS=**.
-CTABLES /TABLE qn1 /COMPARETEST MEANSVARIANCE=**.
-CTABLES /TABLE qn1 /COMPARETEST CATEGORIES=**.
-CTABLES /TABLE qn1 /COMPARETEST MERGE=**.
-CTABLES /TABLE qn1 /COMPARETEST STYLE=**.
-CTABLES /TABLE qn1 /COMPARETEST SHOWSIG=**.
-CTABLES /TABLE qn1 /COMPARETEST **.
-CTABLES /TABLE qn1 /FORMAT.
-CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CLABELS COLLABELS=OPPOSITE.
-CTABLES /TABLE qn20 > qnd1.
-CTABLES /TABLE qn1 [ROWPCT] > qnsa1.
-NUMERIC datetime (DATETIME17.0).
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=datetime ['123'].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [1],
-[[ctables.sps:2.8: error: CTABLES: Syntax error expecting `/'.
- 2 | CTABLES.
- | ^
-
-ctables.sps:3.29-3.33: error: CTABLES: Syntax error expecting non-negative
-number for MINCOLWIDTH.
- 3 | CTABLES /FORMAT MINCOLWIDTH='foo'.
- | ^~~~~
-
-ctables.sps:4.21-4.22: error: CTABLES: Syntax error expecting identifier.
- 4 | CTABLES /TABLE qn1 [**].
- | ^~
-
-ctables.sps:5.21-5.32: error: CTABLES: Syntax error expecting summary function
-name.
- 5 | CTABLES /TABLE qn1 [NOTAFUNCTION].
- | ^~~~~~~~~~~~
-
-ctables.sps:6.20: error: CTABLES: Syntax error expecting `@:}@'.
- 6 | CTABLES /TABLE @{:@qn1.
- | ^
-
-ctables.sps:7.16-7.17: error: CTABLES: Syntax error expecting identifier.
- 7 | CTABLES /TABLE **.
- | ^~
-
-ctables.sps:8.16-8.22: error: CTABLES: NOTAVAR is not a variable name.
- 8 | CTABLES /TABLE NOTAVAR.
- | ^~~~~~~
-
-ctables.sps:10.16-10.24: error: CTABLES: Cannot use string variable string as a
-scale variable.
- 10 | CTABLES /TABLE string[S].
- | ^~~~~~~~~
-
-ctables.sps:11.27-11.29: error: CTABLES: Syntax error expecting number between 0
-and 100 for PTILE.
- 11 | CTABLES /TABLE qn1 [PTILE 101].
- | ^~~
-
-ctables.sps:12.26-12.29: error: CTABLES: Output format F0.1 specifies width 0,
-but F requires a width between 1 and 40.
- 12 | CTABLES /TABLE qn1 [MEAN F0.1].
- | ^~~~
-
-ctables.sps:13.26-13.36: error: CTABLES: Output format NEGPAREN requires width 2
-or greater.
- 13 | CTABLES /TABLE qn1 [MEAN NEGPAREN1.2].
- | ^~~~~~~~~~~
-
-ctables.sps:14.26-14.36: error: CTABLES: Output format NEGPAREN requires width
-greater than decimals.
- 14 | CTABLES /TABLE qn1 [MEAN NEGPAREN3.4].
- | ^~~~~~~~~~~
-
-ctables.sps:15.21-15.24: error: CTABLES: Summary function MEAN applies only to
-scale variables.
- 15 | CTABLES /TABLE qn1 [MEAN TOTALS].
- | ^~~~
-
-ctables.sps:15.16-15.18: note: CTABLES: 'QN1' is not a scale variable.
- 15 | CTABLES /TABLE qn1 [MEAN TOTALS].
- | ^~~
-
-ctables.sps:15.32: error: CTABLES: Syntax error expecting `@<:@'.
- 15 | CTABLES /TABLE qn1 [MEAN TOTALS].
- | ^
-
-ctables.sps:16.21-16.24: error: CTABLES: Summary function MEAN applies only to
-scale variables.
- 16 | CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
- | ^~~~
-
-ctables.sps:16.16-16.18: note: CTABLES: 'QN1' is not a scale variable.
- 16 | CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
- | ^~~
-
-ctables.sps:16.40: error: CTABLES: Syntax error expecting `@:>@'.
- 16 | CTABLES /TABLE qn1 [MEAN TOTALS[STDDEV]%].
- | ^
-
-ctables.sps:17.56: error: CTABLES: Syntax error expecting string.
- 17 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [SUBTOTAL=x].
- | ^
-
-ctables.sps:18.50-18.51: error: CTABLES: Syntax error expecting THRU.
- 18 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO **].
- | ^~
-
-ctables.sps:19.55: error: CTABLES: Syntax error expecting number.
- 19 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [LO THRU x].
- | ^
-
-ctables.sps:20.54-20.55: error: CTABLES: Syntax error expecting number.
- 20 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1 THRU **].
- | ^~
-
-ctables.sps:21.56-21.57: error: CTABLES: Syntax error expecting string.
- 21 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['x' THRU **].
- | ^~
-
-ctables.sps:22.48-22.49: error: CTABLES: Syntax error expecting identifier.
- 22 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&**].
- | ^~
-
-ctables.sps:23.47-23.48: error: CTABLES: Unknown postcompute &x.
- 23 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&x].
- | ^~
-
-ctables.sps:24.61-24.63: error: CTABLES: Syntax error expecting number between 0
-and 100 for PTILE.
- 24 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 101).
- | ^~~
-
-ctables.sps:25.58: error: CTABLES: Syntax error expecting `@:}@'.
- 25 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1.
- | ^
-
-ctables.sps:26.54: error: CTABLES: Syntax error expecting `@{:@'.
- 26 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=MEAN.
- | ^
-
-ctables.sps:27.54-27.55: error: CTABLES: Syntax error expecting INCLUDE or
-EXCLUDE.
- 27 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 MISSING=**.
- | ^~
-
-ctables.sps:28.52-28.53: error: CTABLES: Syntax error expecting YES or NO.
- 28 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 TOTAL=**.
- | ^~
-
-ctables.sps:29.52-29.53: error: CTABLES: Syntax error expecting string.
- 29 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 LABEL=**.
- | ^~
-
-ctables.sps:30.55-30.56: error: CTABLES: Syntax error expecting BEFORE or AFTER.
- 30 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 POSITION=**.
- | ^~
-
-ctables.sps:31.52-31.53: error: CTABLES: Syntax error expecting INCLUDE or
-EXCLUDE.
- 31 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 EMPTY=**.
- | ^~
-
-ctables.sps:32.46-32.47: error: CTABLES: Syntax error expecting ORDER, KEY,
-MISSING, TOTAL, LABEL, POSITION, or EMPTY.
- 32 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 **.
- | ^~
-
-ctables.sps:33.54-33.55: error: CTABLES: Syntax error expecting TOTAL, LABEL,
-POSITION, or EMPTY.
- 33 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1,2,3] **.
- | ^~
-
-ctables.sps:34.36: error: CTABLES: Syntax error expecting positive integer for
-SUBTOTAL.
- 34 | CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[0]).
- | ^
-
-ctables.sps:35.37-35.38: error: CTABLES: Syntax error expecting `@:>@'.
- 35 | CTABLES /PCOMPUTE &k=EXPR(SUBTOTAL[1**]).
- | ^~
-
-ctables.sps:36.31-36.32: error: CTABLES: Syntax error expecting THRU.
- 36 | CTABLES /PCOMPUTE &k=EXPR([LO **]).
- | ^~
-
-ctables.sps:37.36-37.37: error: CTABLES: Syntax error expecting number.
- 37 | CTABLES /PCOMPUTE &k=EXPR([LO THRU **]).
- | ^~
-
-ctables.sps:38.35-38.36: error: CTABLES: Syntax error expecting number.
- 38 | CTABLES /PCOMPUTE &k=EXPR([1 THRU **]).
- | ^~
-
-ctables.sps:39.29-39.30: error: CTABLES: Syntax error expecting `@:>@'.
- 39 | CTABLES /PCOMPUTE &k=EXPR([1**]).
- | ^~
-
-ctables.sps:40.29: error: CTABLES: Syntax error expecting `@:}@'.
- 40 | CTABLES /PCOMPUTE &k=EXPR((1x)).
- | ^
-
-ctables.sps:41.19-41.20: error: CTABLES: Syntax error expecting &.
- 41 | CTABLES /PCOMPUTE **k.
- | ^~
-
-ctables.sps:42.20: error: CTABLES: Syntax error expecting identifier.
- 42 | CTABLES /PCOMPUTE &1.
- | ^
-
-ctables.sps:43.21-43.22: error: CTABLES: Syntax error expecting `=EXPR@{:@'.
- 43 | CTABLES /PCOMPUTE &k**.
- | ^~
-
-ctables.sps:44.21-44.23: error: CTABLES: Syntax error expecting `=EXPR@{:@'.
- 44 | CTABLES /PCOMPUTE &k=**.
- | ^~~
-
-ctables.sps:45.21-45.27: error: CTABLES: Syntax error expecting `=EXPR@{:@'.
- 45 | CTABLES /PCOMPUTE &k=EXPR**.
- | ^~~~~~~
-
-ctables.sps:46.28: error: CTABLES: Syntax error expecting `@:}@'.
- 46 | CTABLES /PCOMPUTE &k=EXPR(1x).
- | ^
-
-ctables.sps:47.31-47.49: warning: CTABLES: New definition of &k will override
-the previous definition.
- 47 | CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
- | ^~~~~~~~~~~~~~~~~~~
-
-ctables.sps:47.10-47.28: note: CTABLES: This is the previous definition.
- 47 | CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
- | ^~~~~~~~~~~~~~~~~~~
-
-ctables.sps:47.50: error: CTABLES: Syntax error expecting `/'.
- 47 | CTABLES /PCOMPUTE &k=EXPR(1) /PCOMPUTE &k=EXPR(2).
- | ^
-
-ctables.sps:48.53-48.64: error: CTABLES: Syntax error expecting summary function
-name.
- 48 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=NOTAFUNCTION.
- | ^~~~~~~~~~~~
-
-ctables.sps:49.59-49.60: error: CTABLES: Syntax error expecting number between 0
-and 100 for PTILE.
- 49 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k FORMAT=PTILE **.
- | ^~
-
-ctables.sps:50.52-50.53: error: CTABLES: Syntax error expecting string.
- 50 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k LABEL=**.
- | ^~
-
-ctables.sps:51.61-51.62: error: CTABLES: Syntax error expecting YES or NO.
- 51 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k HIDESOURCECATS=**.
- | ^~
-
-ctables.sps:52.46-52.47: error: CTABLES: Syntax error expecting LABEL, FORMAT,
-or HIDESOURCECATS.
- 52 | CTABLES /PCOMPUTE &k=EXPR(1) /PPROPERTIES &k **.
- | ^~
-
-ctables.sps:53.23-53.24: error: CTABLES: Syntax error expecting string.
- 53 | CTABLES /FORMAT EMPTY=**.
- | ^~
-
-ctables.sps:54.25-54.26: error: CTABLES: Syntax error expecting string.
- 54 | CTABLES /FORMAT MISSING=**.
- | ^~
-
-ctables.sps:55.17-55.18: error: CTABLES: Syntax error expecting MINCOLWIDTH,
-MAXCOLWIDTH, UNITS, EMPTY, or MISSING.
- 55 | CTABLES /FORMAT **.
- | ^~
-
-ctables.sps:56.17-56.45: error: CTABLES: MINCOLWIDTH must not be greater than
-MAXCOLWIDTH.
- 56 | CTABLES /FORMAT MINCOLWIDTH=20 MAXCOLWIDTH=10/.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:57.18-57.19: error: CTABLES: Syntax error expecting VARIABLES.
- 57 | CTABLES /VLABELS **.
- | ^~
-
-ctables.sps:58.28-58.34: error: CTABLES: NOTAVAR is not a variable name.
- 58 | CTABLES /VLABELS VARIABLES=NOTAVAR.
- | ^~~~~~~
-
-ctables.sps:59.32-59.33: error: CTABLES: Syntax error expecting DISPLAY.
- 59 | CTABLES /VLABELS VARIABLES=qn1 **.
- | ^~
-
-ctables.sps:60.40-60.41: error: CTABLES: Syntax error expecting DEFAULT, NAME,
-LABEL, BOTH, or NONE.
- 60 | CTABLES /VLABELS VARIABLES=qn1 DISPLAY=**.
- | ^~
-
-ctables.sps:61.17-61.18: error: CTABLES: Syntax error expecting COUNTDUPLICATES.
- 61 | CTABLES /MRSETS **.
- | ^~
-
-ctables.sps:62.33-62.34: error: CTABLES: Syntax error expecting YES or NO.
- 62 | CTABLES /MRSETS COUNTDUPLICATES=**.
- | ^~
-
-ctables.sps:63.19-63.20: error: CTABLES: Syntax error expecting VARIABLE or
-LISTWISE.
- 63 | CTABLES /SMISSING **.
- | ^~
-
-ctables.sps:64.17-64.18: error: CTABLES: Syntax error expecting VARIABLE.
- 64 | CTABLES /WEIGHT **.
- | ^~
-
-ctables.sps:65.26-65.32: error: CTABLES: NOTAVAR is not a variable name.
- 65 | CTABLES /WEIGHT VARIABLE=NOTAVAR.
- | ^~~~~~~
-
-ctables.sps:66.32: error: CTABLES: Syntax error expecting integer 2 or greater
-for HIDESMALLCOUNTS COUNT.
- 66 | CTABLES /HIDESMALLCOUNTS COUNT=1.
- | ^
-
-ctables.sps:67.10-67.13: error: CTABLES: Syntax error expecting one of the
-following: FORMAT, VLABELS, MRSETS, SMISSING, PCOMPUTE, PPROPERTIES, WEIGHT,
-HIDESMALLCOUNTS, TABLE.
- 67 | CTABLES /QUUX.
- | ^~~~
-
-ctables.sps:68.33: error: CTABLES: Syntax error expecting `/'.
- 68 | CTABLES /HIDESMALLCOUNTS COUNT=2.
- | ^
-
-ctables.sps:69.19-69.20: error: CTABLES: Syntax error expecting `/'.
- 69 | CTABLES /TABLE qn1**.
- | ^~
-
-ctables.sps:70.38-70.39: error: CTABLES: Syntax error expecting COLUMN, ROW, or
-LAYER.
- 70 | CTABLES /TABLE qn1 /SLABELS POSITION=**.
- | ^~
-
-ctables.sps:71.37-71.38: error: CTABLES: Syntax error expecting YES or NO.
- 71 | CTABLES /TABLE qn1 /SLABELS VISIBLE=**.
- | ^~
-
-ctables.sps:72.29-72.30: error: CTABLES: Syntax error expecting POSITION or
-VISIBLE.
- 72 | CTABLES /TABLE qn1 /SLABELS **.
- | ^~
-
-ctables.sps:73.39-73.40: error: CTABLES: Syntax error expecting OPPOSITE or
-LAYER.
- 73 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=**.
- | ^~
-
-ctables.sps:74.39-74.40: error: CTABLES: Syntax error expecting OPPOSITE or
-LAYER.
- 74 | CTABLES /TABLE qn1 /CLABELS COLLABELS=**.
- | ^~
-
-ctables.sps:75.29-75.30: error: CTABLES: Syntax error expecting AUTO, ROWLABELS,
-or COLLABELS.
- 75 | CTABLES /TABLE qn1 /CLABELS **.
- | ^~
-
-ctables.sps:76.30-76.31: error: CTABLES: Syntax error expecting CILEVEL.
- 76 | CTABLES /TABLE qn1 /CRITERIA **.
- | ^~
-
-ctables.sps:77.38-77.40: error: CTABLES: Syntax error expecting number in
-@<:@0,100@:}@ for CILEVEL.
- 77 | CTABLES /TABLE qn1 /CRITERIA CILEVEL=101.
- | ^~~
-
-ctables.sps:78.28-78.29: error: CTABLES: Syntax error expecting CAPTION, CORNER,
-or TITLE.
- 78 | CTABLES /TABLE qn1 /TITLES **.
- | ^~
-
-ctables.sps:79.34-79.35: error: CTABLES: Syntax error expecting CHISQUARE.
- 79 | CTABLES /TABLE qn1 /SIGTEST TYPE=**.
- | ^~
-
-ctables.sps:80.35-80.36: error: CTABLES: Syntax error expecting number in @<:@0,1@:}@
-for ALPHA.
- 80 | CTABLES /TABLE qn1 /SIGTEST ALPHA=**.
- | ^~
-
-ctables.sps:81.43-81.44: error: CTABLES: Syntax error expecting YES or NO.
- 81 | CTABLES /TABLE qn1 /SIGTEST INCLUDEMRSETS=**.
- | ^~
-
-ctables.sps:82.40-82.41: error: CTABLES: Syntax error expecting ALLVISIBLE or
-SUBTOTALS.
- 82 | CTABLES /TABLE qn1 /SIGTEST CATEGORIES=**.
- | ^~
-
-ctables.sps:83.29-83.30: error: CTABLES: Syntax error expecting TYPE, ALPHA,
-INCLUDEMRSETS, or CATEGORIES.
- 83 | CTABLES /TABLE qn1 /SIGTEST **.
- | ^~
-
-ctables.sps:84.38-84.39: error: CTABLES: Syntax error expecting PROP or MEAN.
- 84 | CTABLES /TABLE qn1 /COMPARETEST TYPE=**.
- | ^~
-
-ctables.sps:85.39-85.40: error: CTABLES: Syntax error expecting number in (0,1)
-for ALPHA.
- 85 | CTABLES /TABLE qn1 /COMPARETEST ALPHA=**.
- | ^~
-
-ctables.sps:86.39: error: CTABLES: Syntax error expecting number in (0,1) for
-ALPHA.
- 86 | CTABLES /TABLE qn1 /COMPARETEST ALPHA=0,5.
- | ^
-
-ctables.sps:87.40-87.41: error: CTABLES: Syntax error expecting BONFERRONI, BH,
-or NONE.
- 87 | CTABLES /TABLE qn1 /COMPARETEST ADJUST=**.
- | ^~
-
-ctables.sps:88.47-88.48: error: CTABLES: Syntax error expecting YES or NO.
- 88 | CTABLES /TABLE qn1 /COMPARETEST INCLUDEMRSETS=**.
- | ^~
-
-ctables.sps:89.47-89.48: error: CTABLES: Syntax error expecting ALLCATS or
-TESTEDCATS.
- 89 | CTABLES /TABLE qn1 /COMPARETEST MEANSVARIANCE=**.
- | ^~
-
-ctables.sps:90.44-90.45: error: CTABLES: Syntax error expecting ALLVISIBLE or
-SUBTOTALS.
- 90 | CTABLES /TABLE qn1 /COMPARETEST CATEGORIES=**.
- | ^~
-
-ctables.sps:91.39-91.40: error: CTABLES: Syntax error expecting YES or NO.
- 91 | CTABLES /TABLE qn1 /COMPARETEST MERGE=**.
- | ^~
-
-ctables.sps:92.39-92.40: error: CTABLES: Syntax error expecting APA or SIMPLE.
- 92 | CTABLES /TABLE qn1 /COMPARETEST STYLE=**.
- | ^~
-
-ctables.sps:93.41-93.42: error: CTABLES: Syntax error expecting YES or NO.
- 93 | CTABLES /TABLE qn1 /COMPARETEST SHOWSIG=**.
- | ^~
-
-ctables.sps:94.33-94.34: error: CTABLES: Syntax error expecting one of the
-following: TYPE, ALPHA, ADJUST, INCLUDEMRSETS, MEANSVARIANCE, CATEGORIES, MERGE,
-STYLE, SHOWSIG.
- 94 | CTABLES /TABLE qn1 /COMPARETEST **.
- | ^~
-
-ctables.sps:95.21-95.26: error: CTABLES: Syntax error expecting TABLE, SLABELS,
-CLABELS, CRITERIA, CATEGORIES, TITLES, SIGTEST, or COMPARETEST.
- 95 | CTABLES /TABLE qn1 /FORMAT.
- | ^~~~~~
-
-ctables.sps:95.21-95.26: note: CTABLES: This subcommand must appear before
-TABLE.
- 95 | CTABLES /TABLE qn1 /FORMAT.
- | ^~~~~~
-
-ctables.sps:96: error: CTABLES: ROWLABELS and COLLABELS may not both be
-specified.
-
-ctables.sps:96.21-96.46: note: CTABLES: This is the first specification.
- 96 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CLABELS
-COLLABELS=OPPOSITE.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:96.49-96.74: note: CTABLES: This is the second specification.
- 96 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CLABELS
-COLLABELS=OPPOSITE.
- |
-^~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:97.16-97.26: error: CTABLES: Cannot nest scale variables.
- 97 | CTABLES /TABLE qn20 > qnd1.
- | ^~~~~~~~~~~
-
-ctables.sps:97.16-97.19: note: CTABLES: This is an outer scale variable.
- 97 | CTABLES /TABLE qn20 > qnd1.
- | ^~~~
-
-ctables.sps:97.23-97.26: note: CTABLES: This is an inner scale variable.
- 97 | CTABLES /TABLE qn20 > qnd1.
- | ^~~~
-
-ctables.sps:98.16-98.35: error: CTABLES: Summaries may only be requested for
-categorical variables at the innermost nesting level.
- 98 | CTABLES /TABLE qn1 [ROWPCT] > qnsa1.
- | ^~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:98.16-98.18: note: CTABLES: This outer categorical variable has a
-summary.
- 98 | CTABLES /TABLE qn1 [ROWPCT] > qnsa1.
- | ^~~
-
-ctables.sps:100.52-100.56: error: CTABLES: Failed to parse category
-specification as format DATETIME: Day (123) must be between 1 and 31..
- 100 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=datetime ['123'].
- | ^~~~~
-]])
-AT_CLEANUP
-
-AT_SETUP([CTABLES parsing - more negative])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
-CTABLES /PCOMPUTE &pc=EXPR(TOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
-CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc, SUBTOTAL, SUBTOTAL].
-
-STRING string(A8).
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['string'].
-CTABLES /TABLE string /CATEGORIES VARIABLES=string [1].
-
-CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1).
-
-CTABLES /TABLE qnd1 /CLABELS ROWLABELS=OPPOSITE.
-CTABLES /TABLE qn1 + string /CLABELS ROWLABELS=OPPOSITE.
-CTABLES /TABLE qn1 + qnsa1 /CLABELS ROWLABELS=OPPOSITE.
-CTABLES /TABLE qn105ba + qn105bb /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn105ba [1,2,3].
-
-CTABLES /PCOMPUTE &x=EXPR(1**2**3).
-CTABLES /PCOMPUTE &x=EXPR([**]).
-CTABLES /PCOMPUTE &x=EXPR(**).
-
-CTABLES /TABLE.
-
-CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
-
-CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 50).
-
-CTABLES /TABLE $mrset.
-
-CTABLES /TABLE qn113 /SIGTEST TYPE=CHISQUARE.
-CTABLES /TABLE qn113 /COMPARETEST TYPE=PROP.
-
-CTABLES /TABLE qn113 [COUNT.UCL].
-
-CTABLES /TABLE qn1 /CATEGORIES **.
-
-CTABLES /TITLES.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [1],
-[[ctables.sps:2.76-2.78: error: CTABLES: Computed category &pc references a category not included in the category list.
- 2 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
- | ^~~
-
-ctables.sps:2.28-2.35: note: CTABLES: This is the missing category.
- 2 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
- | ^~~~~~~~
-
-ctables.sps:2.76-2.79: note: CTABLES: To fix the problem, add subtotals to the list of categories here.
- 2 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
- | ^~~~
-
-ctables.sps:3.73-3.75: error: CTABLES: Computed category &pc references a category not included in the category list.
- 3 | CTABLES /PCOMPUTE &pc=EXPR(TOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
- | ^~~
-
-ctables.sps:3.28-3.32: note: CTABLES: This is the missing category.
- 3 | CTABLES /PCOMPUTE &pc=EXPR(TOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc].
- | ^~~~~
-
-ctables.sps:3: note: CTABLES: To fix the problem, add TOTAL=YES to the variable's CATEGORIES specification.
-
-ctables.sps:4.76-4.99: error: CTABLES: These categories include 2 instances of SUBTOTAL or HSUBTOTAL, so references from
-computed categories must refer to subtotals by position, e.g. SUBTOTAL[1].
- 4 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc, SUBTOTAL, SUBTOTAL].
- | ^~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:4.28-4.35: note: CTABLES: This is the reference that lacks a position.
- 4 | CTABLES /PCOMPUTE &pc=EXPR(SUBTOTAL) /TABLE qn1 /CATEGORIES VARIABLES=qn1 [&pc, SUBTOTAL, SUBTOTAL].
- | ^~~~~~~~
-
-ctables.sps:7.47-7.54: error: CTABLES: This category specification may be applied only to string variables, but this
-subcommand tries to apply it to numeric variable QN1.
- 7 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 ['string'].
- | ^~~~~~~~
-
-ctables.sps:8.53: error: CTABLES: This category specification may be applied only to numeric variables, but this
-subcommand tries to apply it to string variable string.
- 8 | CTABLES /TABLE string /CATEGORIES VARIABLES=string [1].
- | ^
-
-ctables.sps:10.74-10.86: error: CTABLES: Data-dependent sorting is not implemented.
- 10 | CTABLES /TABLE qn1 /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn1 KEY=MEAN(qn1).
- | ^~~~~~~~~~~~~
-
-ctables.sps:12: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
-moved must be categorical, but qnd1 is scale.
-
-ctables.sps:12.22-12.47: note: CTABLES: This syntax moves category labels to another axis.
- 12 | CTABLES /TABLE qnd1 /CLABELS ROWLABELS=OPPOSITE.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:13: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
-moved must all have the same width, but QN1 has width 0 and string has width 8.
-
-ctables.sps:13.30-13.55: note: CTABLES: This syntax moves category labels to another axis.
- 13 | CTABLES /TABLE qn1 + string /CLABELS ROWLABELS=OPPOSITE.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:14: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
-moved must all have the same value labels, but QN1 and QNSA1 have different value labels.
-
-ctables.sps:14.29-14.54: note: CTABLES: This syntax moves category labels to another axis.
- 14 | CTABLES /TABLE qn1 + qnsa1 /CLABELS ROWLABELS=OPPOSITE.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:15: error: CTABLES: To move category labels from one axis to another, the variables whose labels are to be
-moved must all have the same category specifications, but QN105BA and QN105BB have different category specifications.
-
-ctables.sps:15.35-15.60: note: CTABLES: This syntax moves category labels to another axis.
- 15 | CTABLES /TABLE qn105ba + qn105bb /CLABELS ROWLABELS=OPPOSITE /CATEGORIES VARIABLES=qn105ba [1,2,3].
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:17.27-17.33: warning: CTABLES: The exponentiation operator (`**') is left-associative: `a**b**c' equals
-`(a**b)**c', not `a**(b**c)'. To disable this warning, insert parentheses.
- 17 | CTABLES /PCOMPUTE &x=EXPR(1**2**3).
- | ^~~~~~~
-
-ctables.sps:17.35: error: CTABLES: Syntax error expecting `/'.
- 17 | CTABLES /PCOMPUTE &x=EXPR(1**2**3).
- | ^
-
-ctables.sps:18.28-18.29: error: CTABLES: Syntax error expecting number or string or range.
- 18 | CTABLES /PCOMPUTE &x=EXPR([**]).
- | ^~
-
-ctables.sps:19.27-19.28: error: CTABLES: Syntax error in postcompute expression.
- 19 | CTABLES /PCOMPUTE &x=EXPR(**).
- | ^~
-
-ctables.sps:21.15: error: CTABLES: At least one variable must be specified.
- 21 | CTABLES /TABLE.
- | ^
-
-ctables.sps:23: error: CTABLES: Summaries may appear only on one axis.
-
-ctables.sps:23.50-23.54: note: CTABLES: This variable on the layers axis has a summary.
- 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
- | ^~~~~
-
-ctables.sps:23.16-23.20: note: CTABLES: This variable on the rows axis has a summary.
- 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
- | ^~~~~
-
-ctables.sps:23.33-23.37: note: CTABLES: This variable on the columns axis has a summary.
- 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
- | ^~~~~
-
-ctables.sps:23.33-23.37: note: CTABLES: This is a scale variable, so it always has a summary even if the syntax does not
-explicitly specify one.
- 23 | CTABLES /TABLE qn113 [COUNT] BY qn114 [COUNT] BY qn116 [COUNT].
- | ^~~~~
-
-ctables.sps:25.46-25.63: error: CTABLES: Data-dependent sorting is not implemented.
- 25 | CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 KEY=PTILE(qn1, 50).
- | ^~~~~~~~~~~~~~~~~~
-
-ctables.sps:27.16-27.21: error: CTABLES: Multiple response set support not implemented.
- 27 | CTABLES /TABLE $mrset.
- | ^~~~~~
-
-ctables.sps:29.23-29.44: error: CTABLES: Support for SIGTEST not yet implemented.
- 29 | CTABLES /TABLE qn113 /SIGTEST TYPE=CHISQUARE.
- | ^~~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:30.23-30.43: error: CTABLES: Support for COMPARETEST not yet implemented.
- 30 | CTABLES /TABLE qn113 /COMPARETEST TYPE=PROP.
- | ^~~~~~~~~~~~~~~~~~~~~
-
-ctables.sps:32.23-32.31: error: CTABLES: Support for LCL, UCL, and SE summary functions is not yet implemented.
- 32 | CTABLES /TABLE qn113 [COUNT.UCL].
- | ^~~~~~~~~
-
-ctables.sps:34.32-34.33: error: CTABLES: Syntax error expecting VARIABLES.
- 34 | CTABLES /TABLE qn1 /CATEGORIES **.
- | ^~
-
-ctables.sps:36.10-36.15: error: CTABLES: Syntax error expecting one of the following: FORMAT, VLABELS, MRSETS, SMISSING,
-PCOMPUTE, PPROPERTIES, WEIGHT, HIDESMALLCOUNTS, TABLE.
- 36 | CTABLES /TITLES.
- | ^~~~~~
-
-ctables.sps:36.10-36.15: note: CTABLES: TABLE must appear before this subcommand.
- 36 | CTABLES /TITLES.
- | ^~~~~~
-]])
-AT_CLEANUP
-
-AT_SETUP([CTABLES one categorical variable])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE qn1.
-CTABLES /TABLE BY qn1.
-CTABLES /TABLE BY BY qn1.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│ 1. How often do you usually drive a car or other Every day │ 4667│
-│motor vehicle? Several days a week │ 1274│
-│ Once a week or less │ 361│
-│ Only certain times a │ 130│
-│ year │ │
-│ Never │ 540│
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────────────────────────────╮
-│ 1. How often do you usually drive a car or other motor vehicle? │
-├─────────┬──────────────────┬──────────────────┬────────────────────────┬─────┤
-│ │ Several days a │ Once a week or │ Only certain times a │ │
-│Every day│ week │ less │ year │Never│
-├─────────┼──────────────────┼──────────────────┼────────────────────────┼─────┤
-│ Count │ Count │ Count │ Count │Count│
-├─────────┼──────────────────┼──────────────────┼────────────────────────┼─────┤
-│ 4667│ 1274│ 361│ 130│ 540│
-╰─────────┴──────────────────┴──────────────────┴────────────────────────┴─────╯
-
-Custom Tables
-Every day
-╭─────╮
-│Count│
-├─────┤
-│ 4667│
-╰─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES one string variable])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-STRING licensed(A8).
-MISSING VALUES licensed('DontKnow', 'Refused').
-RECODE qnd7a(1='Yes')(2='No')(3='DontKnow')(4='Refused') INTO licensed.
-CTABLES /TABLE licensed.
-CTABLES /TABLE licensed [COUNT, TOTALS[COUNT, VALIDN]] /CATEGORIES VARIABLES=ALL TOTAL=YES MISSING=INCLUDE.
-CTABLES /TABLE licensed /CATEGORIES VARIABLES=licensed ['Yes', 'No'] TOTAL=YES.
-* Notice that the string matching is case-sensitive.
-CTABLES /TABLE licensed /CATEGORIES VARIABLES=licensed ['Yes', 'no'] TOTAL=YES.
-CTABLES /TABLE licensed /CATEGORIES VARIABLES=licensed ['No' THRU 'yes'] TOTAL=YES.
-CTABLES
- /PCOMPUTE ¬yes=EXPR(['No']+['DontKnow']+['Refused'])
- /PPROPERTIES ¬yes LABEL='Not Yes' HIDESOURCECATS=YES
- /TABLE licensed
- /CATEGORIES VARIABLES=licensed ['Yes', ¬yes, 'No', 'DontKnow', 'Refused'].
-CTABLES
- /PCOMPUTE ¬yes=EXPR(['DontKnow' THRU 'No'] + ['Refused'])
- /PPROPERTIES ¬yes LABEL='Not Yes' HIDESOURCECATS=YES
- /TABLE licensed
- /CATEGORIES VARIABLES=licensed ['Yes', ¬yes, 'DontKnow' THRU 'No', 'Refused'].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭────────────┬─────╮
-│ │Count│
-├────────────┼─────┤
-│licensed No │ 572│
-│ Yes│ 6379│
-╰────────────┴─────╯
-
- Custom Tables
-╭─────────────────┬─────┬───────╮
-│ │Count│Valid N│
-├─────────────────┼─────┼───────┤
-│licensed DontKnow│ 4│ │
-│ No │ 572│ │
-│ Refused │ 44│ │
-│ Yes │ 6379│ │
-│ Total │ 6999│ 6951│
-╰─────────────────┴─────┴───────╯
-
- Custom Tables
-╭──────────────┬─────╮
-│ │Count│
-├──────────────┼─────┤
-│licensed Yes │ 6379│
-│ No │ 572│
-│ Total│ 6951│
-╰──────────────┴─────╯
-
- Custom Tables
-╭──────────────┬─────╮
-│ │Count│
-├──────────────┼─────┤
-│licensed Yes │ 6379│
-│ no │ 0│
-│ Total│ 6379│
-╰──────────────┴─────╯
-
- Custom Tables
-╭────────────────┬─────╮
-│ │Count│
-├────────────────┼─────┤
-│licensed No │ 572│
-│ Refused│ 44│
-│ Yes │ 6379│
-│ Total │ 6995│
-╰────────────────┴─────╯
-
- Custom Tables
-╭────────────────┬─────╮
-│ │Count│
-├────────────────┼─────┤
-│licensed Yes │ 6379│
-│ Not Yes│ 620│
-╰────────────────┴─────╯
-
- Custom Tables
-╭────────────────┬─────╮
-│ │Count│
-├────────────────┼─────┤
-│licensed Yes │ 6379│
-│ Not Yes│ 620│
-╰────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES one scale variable])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-DESCRIPTIVES qnd1.
-CTABLES /TABLE qnd1[COUNT, VALIDN, TOTALN, MEAN, STDDEV, MINIMUM, MAXIMUM].
-CTABLES /TABLE BY qnd1.
-CTABLES /TABLE BY BY qnd1.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Descriptive Statistics
-╭──────────────────────────┬────┬─────┬───────┬───────┬───────────╮
-│ │ N │ Mean│Std Dev│Minimum│ Maximum │
-├──────────────────────────┼────┼─────┼───────┼───────┼───────────┤
-│D1. AGE: What is your age?│6930│48.26│ 19.01│ 16│86 or older│
-│Valid N (listwise) │6999│ │ │ │ │
-│Missing N (listwise) │ 69│ │ │ │ │
-╰──────────────────────────┴────┴─────┴───────┴───────┴───────────╯
-
- Custom Tables
-╭──────────────────────┬─────┬───────┬───────┬────┬────────────┬───────┬───────╮
-│ │ │ │ │ │ Std │ │ │
-│ │Count│Valid N│Total N│Mean│ Deviation │Minimum│Maximum│
-├──────────────────────┼─────┼───────┼───────┼────┼────────────┼───────┼───────┤
-│D1. AGE: What is your │ 6999│ 6930│ 6999│ 48│ 19│ 16│ 86│
-│age? │ │ │ │ │ │ │ │
-╰──────────────────────┴─────┴───────┴───────┴────┴────────────┴───────┴───────╯
-
- Custom Tables
-╭──────────────────────────╮
-│D1. AGE: What is your age?│
-├──────────────────────────┤
-│ Mean │
-├──────────────────────────┤
-│ 48│
-╰──────────────────────────╯
-
-Custom Tables
-D1. AGE: What is your age?
-╭────╮
-│Mean│
-├────┤
-│ 48│
-╰────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES simple stacking])
-AT_KEYWORDS([stack stacked])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE qn105ba + qn105bb + qn105bc + qn105bd BY qns3a [COLPCT PCT8.0].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭───────────────────────────────────────────────────────────────┬──────────────╮
-│ │ S3a. GENDER: │
-│ ├──────┬───────┤
-│ │ Male │ Female│
-│ ├──────┼───────┤
-│ │Column│ Column│
-│ │ % │ % │
-├───────────────────────────────────────────────────────────────┼──────┼───────┤
-│105b. How likely is it that drivers who have had Almost │ 10%│ 11%│
-│too much to drink to drive safely will A. Get certain │ │ │
-│stopped by the police? Very likely │ 21%│ 22%│
-│ Somewhat │ 38%│ 42%│
-│ likely │ │ │
-│ Somewhat │ 21%│ 18%│
-│ unlikely │ │ │
-│ Very │ 10%│ 8%│
-│ unlikely │ │ │
-├───────────────────────────────────────────────────────────────┼──────┼───────┤
-│105b. How likely is it that drivers who have had Almost │ 14%│ 18%│
-│too much to drink to drive safely will B. Have an certain │ │ │
-│accident? Very likely │ 36%│ 45%│
-│ Somewhat │ 39%│ 32%│
-│ likely │ │ │
-│ Somewhat │ 9%│ 4%│
-│ unlikely │ │ │
-│ Very │ 3%│ 2%│
-│ unlikely │ │ │
-├───────────────────────────────────────────────────────────────┼──────┼───────┤
-│105b. How likely is it that drivers who have had Almost │ 18%│ 16%│
-│too much to drink to drive safely will C. Be certain │ │ │
-│convicted for drunk driving? Very likely │ 32%│ 28%│
-│ Somewhat │ 27%│ 32%│
-│ likely │ │ │
-│ Somewhat │ 15%│ 15%│
-│ unlikely │ │ │
-│ Very │ 9%│ 9%│
-│ unlikely │ │ │
-├───────────────────────────────────────────────────────────────┼──────┼───────┤
-│105b. How likely is it that drivers who have had Almost │ 16%│ 16%│
-│too much to drink to drive safely will D. Be certain │ │ │
-│arrested for drunk driving? Very likely │ 26%│ 27%│
-│ Somewhat │ 32%│ 35%│
-│ likely │ │ │
-│ Somewhat │ 17%│ 15%│
-│ unlikely │ │ │
-│ Very │ 9%│ 7%│
-│ unlikely │ │ │
-╰───────────────────────────────────────────────────────────────┴──────┴───────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES show or hide empty categories])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-IF (qn105ba = 2) qn105ba = 1.
-IF (qns3a = 1) qns3a = 2.
-CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0].
-CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0]
- /CATEGORIES VAR=qn105ba EMPTY=EXCLUDE.
-CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0]
- /CATEGORIES VAR=qns3a EMPTY=EXCLUDE.
-CTABLES /TABLE qn105ba BY qns3a [COLPCT PCT8.0]
- /CATEGORIES VAR=ALL EMPTY=EXCLUDE.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭──────────────────────────────────────────────────────────────┬───────────────╮
-│ │ S3a. GENDER: │
-│ ├───────┬───────┤
-│ │ Male │ Female│
-│ ├───────┼───────┤
-│ │ Column│ Column│
-│ │ % │ % │
-├──────────────────────────────────────────────────────────────┼───────┼───────┤
-│105b. How likely is it that drivers who have had Almost │ .│ 32%│
-│too much to drink to drive safely will A. Get certain │ │ │
-│stopped by the police? Very likely│ .│ 0%│
-│ Somewhat │ .│ 40%│
-│ likely │ │ │
-│ Somewhat │ .│ 19%│
-│ unlikely │ │ │
-│ Very │ .│ 9%│
-│ unlikely │ │ │
-╰──────────────────────────────────────────────────────────────┴───────┴───────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────────────┬───────────────╮
-│ │ S3a. GENDER: │
-│ ├───────┬───────┤
-│ │ Male │ Female│
-│ ├───────┼───────┤
-│ │ Column│ Column│
-│ │ % │ % │
-├──────────────────────────────────────────────────────────────┼───────┼───────┤
-│105b. How likely is it that drivers who have had Almost │ .│ 32%│
-│too much to drink to drive safely will A. Get certain │ │ │
-│stopped by the police? Somewhat │ .│ 40%│
-│ likely │ │ │
-│ Somewhat │ .│ 19%│
-│ unlikely │ │ │
-│ Very │ .│ 9%│
-│ unlikely │ │ │
-╰──────────────────────────────────────────────────────────────┴───────┴───────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────┬─────────╮
-│ │ S3a. │
-│ │ GENDER: │
-│ ├─────────┤
-│ │ Female │
-│ ├─────────┤
-│ │ Column %│
-├────────────────────────────────────────────────────────────────────┼─────────┤
-│105b. How likely is it that drivers who have had too Almost │ 32%│
-│much to drink to drive safely will A. Get stopped by certain │ │
-│the police? Very likely │ 0%│
-│ Somewhat │ 40%│
-│ likely │ │
-│ Somewhat │ 19%│
-│ unlikely │ │
-│ Very │ 9%│
-│ unlikely │ │
-╰────────────────────────────────────────────────────────────────────┴─────────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────┬─────────╮
-│ │ S3a. │
-│ │ GENDER: │
-│ ├─────────┤
-│ │ Female │
-│ ├─────────┤
-│ │ Column %│
-├────────────────────────────────────────────────────────────────────┼─────────┤
-│105b. How likely is it that drivers who have had too Almost │ 32%│
-│much to drink to drive safely will A. Get stopped by certain │ │
-│the police? Somewhat │ 40%│
-│ likely │ │
-│ Somewhat │ 19%│
-│ unlikely │ │
-│ Very │ 9%│
-│ unlikely │ │
-╰────────────────────────────────────────────────────────────────────┴─────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES categories and EMPTY])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-DATA LIST LIST NOTABLE /class datum size.
-BEGIN DATA
-1 1 1
-2 2 1
-1 3 1
-2 4 2
-1 5 2
-2 6 2
-END DATA.
-VARIABLE LEVEL class datum size (NOMINAL).
-FORMATS class datum size (F1.0).
-
-* The following are the same except for the order of the CATEGORIES commands.
-* The test checks that they produce the same resuls.
-CTABLES /TABLE=class > datum BY size
- /CATEGORIES VARIABLES=ALL EMPTY=EXCLUDE
- /CATEGORIES VARIABLES=size TOTAL=YES.
-CTABLES /TABLE=class > datum BY size
- /CATEGORIES VARIABLES=size TOTAL=YES
- /CATEGORIES VARIABLES=ALL EMPTY=EXCLUDE.
-])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭───────────────┬─────────────────╮
-│ │ size │
-│ ├─────┬─────┬─────┤
-│ │ 1 │ 2 │Total│
-│ ├─────┼─────┼─────┤
-│ │Count│Count│Count│
-├───────────────┼─────┼─────┼─────┤
-│class 1 datum 1│ 1│ │ 1│
-│ 3│ 1│ │ 1│
-│ 5│ │ 1│ 1│
-│ ╶─────────┼─────┼─────┼─────┤
-│ 2 datum 2│ 1│ │ 1│
-│ 4│ │ 1│ 1│
-│ 6│ │ 1│ 1│
-╰───────────────┴─────┴─────┴─────╯
-
- Custom Tables
-╭───────────────┬─────────────────╮
-│ │ size │
-│ ├─────┬─────┬─────┤
-│ │ 1 │ 2 │Total│
-│ ├─────┼─────┼─────┤
-│ │Count│Count│Count│
-├───────────────┼─────┼─────┼─────┤
-│class 1 datum 1│ 1│ │ 1│
-│ 3│ 1│ │ 1│
-│ 5│ │ 1│ 1│
-│ ╶─────────┼─────┼─────┼─────┤
-│ 2 datum 2│ 1│ │ 1│
-│ 4│ │ 1│ 1│
-│ 6│ │ 1│ 1│
-╰───────────────┴─────┴─────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES sorting categories])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-IF (QND5A=6) QND5A=-1.
-IF (QND5A=5) QND5A=-2.
-CTABLES /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=VALUE ORDER=A
- /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=VALUE ORDER=D
- /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=LABEL ORDER=A
- /TABLE qnd5a /CATEGORIES VARIABLES=qnd5a KEY=LABEL ORDER=D.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary ethnic -2.00 │ 52│
-│background? -1.00 │ 78│
-│ Cuban │ 20│
-│ Mexican │ 311│
-│ Spanish │ 48│
-│ South American │ 34│
-│ Central American │ 0│
-│ Puerto Rican, OR │ 0│
-│ Something else │ 68│
-│ Multiple - cannot choose│ 7│
-│ one │ │
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary ethnic Multiple - cannot choose│ 7│
-│background? one │ │
-│ Something else │ 68│
-│ Puerto Rican, OR │ 0│
-│ Central American │ 0│
-│ South American │ 34│
-│ Spanish │ 48│
-│ Mexican │ 311│
-│ Cuban │ 20│
-│ -1.00 │ 78│
-│ -2.00 │ 52│
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary ethnic Central American │ 0│
-│background? Cuban │ 20│
-│ Mexican │ 311│
-│ Multiple - cannot choose│ 7│
-│ one │ │
-│ Puerto Rican, OR │ 0│
-│ Something else │ 68│
-│ South American │ 34│
-│ Spanish │ 48│
-│ -2.00 │ 52│
-│ -1.00 │ 78│
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary ethnic Spanish │ 48│
-│background? South American │ 34│
-│ Something else │ 68│
-│ Puerto Rican, OR │ 0│
-│ Multiple - cannot choose│ 7│
-│ one │ │
-│ Mexican │ 311│
-│ Cuban │ 20│
-│ Central American │ 0│
-│ -1.00 │ 78│
-│ -2.00 │ 52│
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES simple nesting])
-AT_KEYWORDS([nest nested])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE (qn105ba + qn105bb + qn105bc + qn105bd) > qns3a [COUNT, TABLEPCT PCT8.0]
- /CATEGORIES VARIABLES=qns3a TOTAL=YES.
-CTABLES /TABLE qns3a > (qn105ba + qn105bb + qn105bc + qn105bd) [TABLEPCT PCT8.0]
- /CATEGORIES VARIABLES=qns3a TOTAL=YES
- /CLABELS ROW=OPPOSITE.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭─────────────────────────────────────────────────────────────────┬─────┬──────╮
-│ │ │ Table│
-│ │Count│ % │
-├─────────────────────────────────────────────────────────────────┼─────┼──────┤
-│105b. How likely is it that drivers Almost S3a. Male │ 297│ 4%│
-│who have had too much to drink to certain GENDER: Female│ 403│ 6%│
-│drive safely will A. Get stopped by Total │ 700│ 10%│
-│the police? ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 660│ 10%│
-│ likely GENDER: Female│ 842│ 12%│
-│ Total │ 1502│ 22%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 1174│ 17%│
-│ likely GENDER: Female│ 1589│ 23%│
-│ Total │ 2763│ 40%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 640│ 9%│
-│ unlikely GENDER: Female│ 667│ 10%│
-│ Total │ 1307│ 19%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 311│ 5%│
-│ unlikely GENDER: Female│ 298│ 4%│
-│ Total │ 609│ 9%│
-├─────────────────────────────────────────────────────────────────┼─────┼──────┤
-│105b. How likely is it that drivers Almost S3a. Male │ 429│ 6%│
-│who have had too much to drink to certain GENDER: Female│ 671│ 10%│
-│drive safely will B. Have an accident? Total │ 1100│ 16%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 1104│ 16%│
-│ likely GENDER: Female│ 1715│ 25%│
-│ Total │ 2819│ 41%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 1203│ 17%│
-│ likely GENDER: Female│ 1214│ 18%│
-│ Total │ 2417│ 35%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 262│ 4%│
-│ unlikely GENDER: Female│ 168│ 2%│
-│ Total │ 430│ 6%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 81│ 1%│
-│ unlikely GENDER: Female│ 59│ 1%│
-│ Total │ 140│ 2%│
-├─────────────────────────────────────────────────────────────────┼─────┼──────┤
-│105b. How likely is it that drivers Almost S3a. Male │ 539│ 8%│
-│who have had too much to drink to certain GENDER: Female│ 610│ 9%│
-│drive safely will C. Be convicted for Total │ 1149│ 17%│
-│drunk driving? ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 988│ 14%│
-│ likely GENDER: Female│ 1049│ 15%│
-│ Total │ 2037│ 30%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 822│ 12%│
-│ likely GENDER: Female│ 1210│ 18%│
-│ Total │ 2032│ 30%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 446│ 7%│
-│ unlikely GENDER: Female│ 548│ 8%│
-│ Total │ 994│ 15%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 268│ 4%│
-│ unlikely GENDER: Female│ 354│ 5%│
-│ Total │ 622│ 9%│
-├─────────────────────────────────────────────────────────────────┼─────┼──────┤
-│105b. How likely is it that drivers Almost S3a. Male │ 498│ 7%│
-│who have had too much to drink to certain GENDER: Female│ 603│ 9%│
-│drive safely will D. Be arrested for Total │ 1101│ 16%│
-│drunk driving? ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 805│ 12%│
-│ likely GENDER: Female│ 1029│ 15%│
-│ Total │ 1834│ 27%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 975│ 14%│
-│ likely GENDER: Female│ 1332│ 19%│
-│ Total │ 2307│ 34%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Somewhat S3a. Male │ 535│ 8%│
-│ unlikely GENDER: Female│ 560│ 8%│
-│ Total │ 1095│ 16%│
-│ ╶──────────────────────────┼─────┼──────┤
-│ Very S3a. Male │ 270│ 4%│
-│ unlikely GENDER: Female│ 279│ 4%│
-│ Total │ 549│ 8%│
-╰─────────────────────────────────────────────────────────────────┴─────┴──────╯
-
- Custom Tables
-╭─────────────────────────────────┬────────┬──────┬─────────┬─────────┬────────╮
-│ │ Almost │ Very │ Somewhat│ Somewhat│ Very │
-│ │ certain│likely│ likely │ unlikely│unlikely│
-│ ├────────┼──────┼─────────┼─────────┼────────┤
-│ │ │ Table│ │ │ │
-│ │ Table %│ % │ Table % │ Table % │ Table %│
-├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│S3a. Male 105b. How likely │ 4%│ 10%│ 17%│ 9%│ 5%│
-│GENDER: is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ A. Get stopped by │ │ │ │ │ │
-│ the police? │ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Female 105b. How likely │ 6%│ 12%│ 23%│ 10%│ 4%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ A. Get stopped by │ │ │ │ │ │
-│ the police? │ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Total 105b. How likely │ 10%│ 22%│ 40%│ 19%│ 9%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ A. Get stopped by │ │ │ │ │ │
-│ the police? │ │ │ │ │ │
-├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│S3a. Male 105b. How likely │ 6%│ 16%│ 17%│ 4%│ 1%│
-│GENDER: is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ B. Have an │ │ │ │ │ │
-│ accident? │ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Female 105b. How likely │ 10%│ 25%│ 18%│ 2%│ 1%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ B. Have an │ │ │ │ │ │
-│ accident? │ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Total 105b. How likely │ 16%│ 41%│ 35%│ 6%│ 2%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ B. Have an │ │ │ │ │ │
-│ accident? │ │ │ │ │ │
-├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│S3a. Male 105b. How likely │ 8%│ 14%│ 12%│ 7%│ 4%│
-│GENDER: is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ C. Be convicted │ │ │ │ │ │
-│ for drunk driving?│ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Female 105b. How likely │ 9%│ 15%│ 18%│ 8%│ 5%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ C. Be convicted │ │ │ │ │ │
-│ for drunk driving?│ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Total 105b. How likely │ 17%│ 30%│ 30%│ 15%│ 9%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ C. Be convicted │ │ │ │ │ │
-│ for drunk driving?│ │ │ │ │ │
-├─────────────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│S3a. Male 105b. How likely │ 7%│ 12%│ 14%│ 8%│ 4%│
-│GENDER: is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ D. Be arrested for│ │ │ │ │ │
-│ drunk driving? │ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Female 105b. How likely │ 9%│ 15%│ 19%│ 8%│ 4%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ D. Be arrested for│ │ │ │ │ │
-│ drunk driving? │ │ │ │ │ │
-│ ╶─────────────────────────┼────────┼──────┼─────────┼─────────┼────────┤
-│ Total 105b. How likely │ 16%│ 27%│ 34%│ 16%│ 8%│
-│ is it that drivers│ │ │ │ │ │
-│ who have had too │ │ │ │ │ │
-│ much to drink to │ │ │ │ │ │
-│ drive safely will │ │ │ │ │ │
-│ D. Be arrested for│ │ │ │ │ │
-│ drunk driving? │ │ │ │ │ │
-╰─────────────────────────────────┴────────┴──────┴─────────┴─────────┴────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES nesting and scale variables])
-AT_KEYWORDS([nest nested])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE=qnd1 > qn1 BY qns3a.
-CTABLES /TABLE=qnd1 [MINIMUM, MAXIMUM, MEAN] > qns3a > (qn26 + qn27).
-CTABLES /TABLE=qnsa1 > qn105ba [COLPCT] BY qns1
- /CATEGORIES VAR=qnsa1 EMPTY=EXCLUDE.
-CTABLES /TABLE=AgeGroup > qn20 [MEAN F8.1, STDDEV F8.1].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭─────────────────────────────────────────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │ Mean│ Mean │
-├─────────────────────────────────────────────────────────────────┼─────┼──────┤
-│D1. AGE: What 1. How often do you usually drive Every day │ 46│ 46│
-│is your age? a car or other motor vehicle? Several days a │ 51│ 59│
-│ week │ │ │
-│ Once a week or │ 44│ 54│
-│ less │ │ │
-│ Only certain │ 34│ 41│
-│ times a year │ │ │
-│ Never │ 39│ 55│
-╰─────────────────────────────────────────────────────────────────┴─────┴──────╯
-
- Custom Tables
-╭─────────────────────────────────────────────────────────┬───────┬───────┬────╮
-│ │Minimum│Maximum│Mean│
-├─────────────────────────────────────────────────────────┼───────┼───────┼────┤
-│D1. AGE: S3a. Male 26. During the last 12 Yes│ 16│ 86│ 42│
-│What is GENDER: months, has there been a │ │ │ │
-│your time when you felt you │ │ │ │
-│age? should cut down on your No │ 16│ 86│ 46│
-│ drinking? │ │ │ │
-│ ╶───────────────────────────────────────┼───────┼───────┼────┤
-│ Female 26. During the last 12 Yes│ 16│ 86│ 43│
-│ months, has there been a │ │ │ │
-│ time when you felt you │ │ │ │
-│ should cut down on your No │ 16│ 86│ 48│
-│ drinking? │ │ │ │
-├─────────────────────────────────────────────────────────┼───────┼───────┼────┤
-│D1. AGE: S3a. Male 27. During the last 12 Yes│ 16│ 86│ 38│
-│What is GENDER: months, has there been a │ │ │ │
-│your time when people criticized No │ 16│ 86│ 46│
-│age? your drinking? │ │ │ │
-│ ╶───────────────────────────────────────┼───────┼───────┼────┤
-│ Female 27. During the last 12 Yes│ 17│ 69│ 37│
-│ months, has there been a │ │ │ │
-│ time when people criticized No │ 16│ 86│ 48│
-│ your drinking? │ │ │ │
-╰─────────────────────────────────────────────────────────┴───────┴───────┴────╯
-
- Custom Tables
-╭─────────────────────────────┬────────────────────────────────────────────────╮
-│ │S1. Including yourself, how many members of this│
-│ │ household are age 16 or older? │
-│ ├──────┬──────┬──────┬──────┬──────┬──────┬──────┤
-│ │ │ │ │ │ │ │ 6 or │
-│ │ None │ 1 │ 2 │ 3 │ 4 │ 5 │ more │
-│ ├──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ │Column│Column│Column│Column│Column│Column│Column│
-│ │ % │ % │ % │ % │ % │ % │ % │
-├─────────────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Sa1. RDD 105b. Almost │ .│ 9.5%│ 8.2%│ 12.4%│ 9.9%│ 20.0%│ 23.8%│
-│SAMPLE How certain │ │ │ │ │ │ │ │
-│SOURCE: likely │ │ │ │ │ │ │ │
-│ is it Very │ .│ 24.9%│ 18.5%│ 24.0%│ 26.6%│ 25.5%│ 33.3%│
-│ that likely │ │ │ │ │ │ │ │
-│ drivers │ │ │ │ │ │ │ │
-│ who have │ │ │ │ │ │ │ │
-│ had too Somewhat│ .│ 38.3%│ 41.9%│ 38.6%│ 37.5%│ 36.4%│ 23.8%│
-│ much to likely │ │ │ │ │ │ │ │
-│ drink to │ │ │ │ │ │ │ │
-│ drive │ │ │ │ │ │ │ │
-│ safely Somewhat│ .│ 18.1%│ 21.7%│ 16.8%│ 16.7%│ 10.9%│ 9.5%│
-│ will A. unlikely│ │ │ │ │ │ │ │
-│ Get │ │ │ │ │ │ │ │
-│ stopped Very │ .│ 9.2%│ 9.7%│ 8.2%│ 9.4%│ 7.3%│ 9.5%│
-│ by the unlikely│ │ │ │ │ │ │ │
-│ police? │ │ │ │ │ │ │ │
-╰─────────────────────────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────────────┬────┬──────────╮
-│ │ │ Std │
-│ │Mean│ Deviation│
-├──────────────────────────────────────────────────────────────┼────┼──────────┤
-│Age 16 to 25 20. On how many of the thirty days in this │ 5.2│ 6.0│
-│group typical month did you have one or more │ │ │
-│ alcoholic beverages to drink? │ │ │
-│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
-│ 26 to 35 20. On how many of the thirty days in this │ 4.7│ 5.9│
-│ typical month did you have one or more │ │ │
-│ alcoholic beverages to drink? │ │ │
-│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
-│ 36 to 45 20. On how many of the thirty days in this │ 5.5│ 6.8│
-│ typical month did you have one or more │ │ │
-│ alcoholic beverages to drink? │ │ │
-│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
-│ 46 to 55 20. On how many of the thirty days in this │ 5.8│ 7.7│
-│ typical month did you have one or more │ │ │
-│ alcoholic beverages to drink? │ │ │
-│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
-│ 56 to 65 20. On how many of the thirty days in this │ 6.3│ 8.2│
-│ typical month did you have one or more │ │ │
-│ alcoholic beverages to drink? │ │ │
-│ ╶───────────────────────────────────────────────────────┼────┼──────────┤
-│ 66 or 20. On how many of the thirty days in this │ 7.1│ 9.2│
-│ older typical month did you have one or more │ │ │
-│ alcoholic beverages to drink? │ │ │
-╰──────────────────────────────────────────────────────────────┴────┴──────────╯
-])
-AT_CLEANUP
-
-
-AT_SETUP([CTABLES SLABELS])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE qn1 [COUNT COLPCT].
-CTABLES /TABLE qn1 [COUNT COLPCT]
- /SLABELS POSITION=ROW.
-CTABLES /TABLE qn1 [COUNT COLPCT]
- /SLABELS POSITION=ROW VISIBLE=NO.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭────────────────────────────────────────────────────────────────┬─────┬───────╮
-│ │ │ Column│
-│ │Count│ % │
-├────────────────────────────────────────────────────────────────┼─────┼───────┤
-│ 1. How often do you usually drive a car or Every day │ 4667│ 66.9%│
-│other motor vehicle? Several days a week│ 1274│ 18.3%│
-│ Once a week or less│ 361│ 5.2%│
-│ Only certain times │ 130│ 1.9%│
-│ a year │ │ │
-│ Never │ 540│ 7.7%│
-╰────────────────────────────────────────────────────────────────┴─────┴───────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ 1. How often do you usually drive a car or Every day Count │ 4667│
-│other motor vehicle? Column │66.9%│
-│ % │ │
-│ ╶───────────────────────────┼─────┤
-│ Several days a week Count │ 1274│
-│ Column │18.3%│
-│ % │ │
-│ ╶───────────────────────────┼─────┤
-│ Once a week or less Count │ 361│
-│ Column │ 5.2%│
-│ % │ │
-│ ╶───────────────────────────┼─────┤
-│ Only certain times Count │ 130│
-│ a year Column │ 1.9%│
-│ % │ │
-│ ╶───────────────────────────┼─────┤
-│ Never Count │ 540│
-│ Column │ 7.7%│
-│ % │ │
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ 1. How often do you usually drive a car or other Every day │ 4667│
-│motor vehicle? │66.9%│
-│ Several days a week │ 1274│
-│ │18.3%│
-│ Once a week or less │ 361│
-│ │ 5.2%│
-│ Only certain times a │ 130│
-│ year │ 1.9%│
-│ Never │ 540│
-│ │ 7.7%│
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES SLABELS with stacking different summaries])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=ALL DISPLAY=NAME
- /TABLE qn1 [COUNT] + qnd1 [MEAN] + qn17 [UCOUNT] BY qns3a
- /SLABELS POSITION=ROW.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭─────────────────────────────────────────────────┬───────────╮
-│ │ QNS3A │
-│ ├────┬──────┤
-│ │Male│Female│
-├─────────────────────────────────────────────────┼────┼──────┤
-│QN1 Every day Count │2305│ 2362│
-│ Unweighted Count│ │ │
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Several days a week Count │ 440│ 834│
-│ Unweighted Count│ │ │
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Once a week or less Count │ 125│ 236│
-│ Unweighted Count│ │ │
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Only certain times a year Count │ 58│ 72│
-│ Unweighted Count│ │ │
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Never Count │ 192│ 348│
-│ Unweighted Count│ │ │
-│ Mean │ │ │
-├─────────────────────────────────────────────────┼────┼──────┤
-│qnd1 Count │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Unweighted Count │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Mean │ 46│ 50│
-├─────────────────────────────────────────────────┼────┼──────┤
-│QN17 OR, something else Count │ │ │
-│ Unweighted Count│ 1│ 1│
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Beer Count │ │ │
-│ Unweighted Count│ 817│ 256│
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Light beer Count │ │ │
-│ Unweighted Count│ 406│ 214│
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Wine Count │ │ │
-│ Unweighted Count│ 390│ 1028│
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Wine coolers Count │ │ │
-│ Unweighted Count│ 20│ 117│
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Hard liquor or mixed drinks Count │ │ │
-│ Unweighted Count│ 392│ 496│
-│ Mean │ │ │
-│ ╶────────────────────────────────────────────┼────┼──────┤
-│ Flavored malt drinks Count │ │ │
-│ Unweighted Count│ 20│ 63│
-│ Mean │ │ │
-╰─────────────────────────────────────────────────┴────┴──────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES simple totals])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE=qn17
- /CATEGORIES VARIABLES=qn17 TOTAL=YES LABEL='Number responding'.
-DESCRIPTIVES qn18/STATISTICS=MEAN.
-CTABLES /TABLE=region > qn18 [MEAN, COUNT, VALIDN, TOTALN]
- /CATEGORIES VARIABLES=region TOTAL=YES LABEL='All regions'.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│17. When you drink alcoholic beverages, which ONE of OR, something else│ 2│
-│the following beverages do you drink MOST OFTEN? Beer │ 1073│
-│ Light beer │ 620│
-│ Wine │ 1418│
-│ Wine coolers │ 137│
-│ Hard liquor or │ 888│
-│ mixed drinks │ │
-│ Flavored malt │ 83│
-│ drinks │ │
-│ Number responding │ 4221│
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Descriptive Statistics
-╭────────────────────────────────────────────────────────────────────┬────┬────╮
-│ │ N │Mean│
-├────────────────────────────────────────────────────────────────────┼────┼────┤
-│18. When you drink ANSWERFROM(QN17R1), about how many │4218│4.62│
-│ANSWERFROM(QN17R2) do you usually drink per sitting? │ │ │
-│Valid N (listwise) │6999│ │
-│Missing N (listwise) │2781│ │
-╰────────────────────────────────────────────────────────────────────┴────┴────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────┬────┬─────┬──────┬─────╮
-│ │ │ │ Valid│Total│
-│ │Mean│Count│ N │ N │
-├──────────────────────────────────────────────────────┼────┼─────┼──────┼─────┤
-│Region NE 18. When you drink ANSWERFROM(QN17R1),│4.36│ 1409│ 949│ 1409│
-│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
-│ you usually drink per sitting? │ │ │ │ │
-│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
-│ MW 18. When you drink ANSWERFROM(QN17R1),│4.67│ 1654│ 1027│ 1654│
-│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
-│ you usually drink per sitting? │ │ │ │ │
-│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
-│ S 18. When you drink ANSWERFROM(QN17R1),│4.71│ 2390│ 1287│ 2390│
-│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
-│ you usually drink per sitting? │ │ │ │ │
-│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
-│ W 18. When you drink ANSWERFROM(QN17R1),│4.69│ 1546│ 955│ 1546│
-│ about how many ANSWERFROM(QN17R2) do │ │ │ │ │
-│ you usually drink per sitting? │ │ │ │ │
-│ ╶───────────────────────────────────────────────┼────┼─────┼──────┼─────┤
-│ All 18. When you drink ANSWERFROM(QN17R1),│4.62│ 6999│ 4218│ 6999│
-│ regions about how many ANSWERFROM(QN17R2) do │ │ │ │ │
-│ you usually drink per sitting? │ │ │ │ │
-╰──────────────────────────────────────────────────────┴────┴─────┴──────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES subtotals])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE=qn105ba BY qns1
- /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, SUBTOTAL].
-CTABLES /TABLE=qn105ba [COLPCT] BY qns1
- /CATEGORIES VARIABLES=qn105ba [1, 2, 3, SUBTOTAL, 4, 5, SUBTOTAL].
-CTABLES /TABLE=qn105ba BY qns1
- /CATEGORIES VARIABLES=qn105ba [1, 2, 3, SUBTOTAL, 4, 5, SUBTOTAL]
- /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, SUBTOTAL].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-╭─────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────╮
-│ │ S1. Including yourself, how many members of this household │
-│ │ are age 16 or older? │
-│ ├───────┬───────┬─────────┬───────┬────────┬──────┬──────────┤
-│ │ 1 │ 2 │ Subtotal│ 3 │ 4 │ 5 │ Subtotal │
-│ ├───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
-│ │ Count │ Count │ Count │ Count │ Count │ Count│ Count │
-├─────────────────────────────────────────────────────────┼───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
-│105b. How likely is it that drivers who have Almost │ 147│ 246│ 393│ 62│ 19│ 11│ 92│
-│had too much to drink to drive safely will A. certain │ │ │ │ │ │ │ │
-│Get stopped by the police? Very likely│ 384│ 552│ 936│ 120│ 51│ 14│ 185│
-│ Somewhat │ 590│ 1249│ 1839│ 193│ 72│ 20│ 285│
-│ likely │ │ │ │ │ │ │ │
-│ Somewhat │ 278│ 647│ 925│ 84│ 32│ 6│ 122│
-│ unlikely │ │ │ │ │ │ │ │
-│ Very │ 141│ 290│ 431│ 41│ 18│ 4│ 63│
-│ unlikely │ │ │ │ │ │ │ │
-╰─────────────────────────────────────────────────────────┴───────┴───────┴─────────┴───────┴────────┴──────┴──────────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────╮
-│ │ S1. Including yourself, how many members of this household │
-│ │ are age 16 or older? │
-│ ├────────┬────────┬────────┬────────┬───────┬────────┬────────┤
-│ │ │ │ │ │ │ │ 6 or │
-│ │ None │ 1 │ 2 │ 3 │ 4 │ 5 │ more │
-│ ├────────┼────────┼────────┼────────┼───────┼────────┼────────┤
-│ │ │ │ │ │ Column│ │ │
-│ │Column %│Column %│Column %│Column %│ % │Column %│Column %│
-├────────────────────────────────────────────────────────┼────────┼────────┼────────┼────────┼───────┼────────┼────────┤
-│105b. How likely is it that drivers who have Almost │ .│ 9.5%│ 8.2%│ 12.4%│ 9.9%│ 20.0%│ 23.8%│
-│had too much to drink to drive safely will certain │ │ │ │ │ │ │ │
-│A. Get stopped by the police? Very likely│ .│ 24.9%│ 18.5%│ 24.0%│ 26.6%│ 25.5%│ 33.3%│
-│ Somewhat │ .│ 38.3%│ 41.9%│ 38.6%│ 37.5%│ 36.4%│ 23.8%│
-│ likely │ │ │ │ │ │ │ │
-│ Subtotal │ │ 72.8%│ 68.6%│ 75.0%│ 74.0%│ 81.8%│ 81.0%│
-│ Somewhat │ .│ 18.1%│ 21.7%│ 16.8%│ 16.7%│ 10.9%│ 9.5%│
-│ unlikely │ │ │ │ │ │ │ │
-│ Very │ .│ 9.2%│ 9.7%│ 8.2%│ 9.4%│ 7.3%│ 9.5%│
-│ unlikely │ │ │ │ │ │ │ │
-│ Subtotal │ │ 27.2%│ 31.4%│ 25.0%│ 26.0%│ 18.2%│ 19.0%│
-╰────────────────────────────────────────────────────────┴────────┴────────┴────────┴────────┴───────┴────────┴────────╯
-
- Custom Tables
-╭─────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────╮
-│ │ S1. Including yourself, how many members of this household │
-│ │ are age 16 or older? │
-│ ├───────┬───────┬─────────┬───────┬────────┬──────┬──────────┤
-│ │ 1 │ 2 │ Subtotal│ 3 │ 4 │ 5 │ Subtotal │
-│ ├───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
-│ │ Count │ Count │ Count │ Count │ Count │ Count│ Count │
-├─────────────────────────────────────────────────────────┼───────┼───────┼─────────┼───────┼────────┼──────┼──────────┤
-│105b. How likely is it that drivers who have Almost │ 147│ 246│ 393│ 62│ 19│ 11│ 92│
-│had too much to drink to drive safely will A. certain │ │ │ │ │ │ │ │
-│Get stopped by the police? Very likely│ 384│ 552│ 936│ 120│ 51│ 14│ 185│
-│ Somewhat │ 590│ 1249│ 1839│ 193│ 72│ 20│ 285│
-│ likely │ │ │ │ │ │ │ │
-│ Subtotal │ 1121│ 2047│ 3168│ 375│ 142│ 45│ 562│
-│ Somewhat │ 278│ 647│ 925│ 84│ 32│ 6│ 122│
-│ unlikely │ │ │ │ │ │ │ │
-│ Very │ 141│ 290│ 431│ 41│ 18│ 4│ 63│
-│ unlikely │ │ │ │ │ │ │ │
-│ Subtotal │ 419│ 937│ 1356│ 125│ 50│ 10│ 185│
-╰─────────────────────────────────────────────────────────┴───────┴───────┴─────────┴───────┴────────┴──────┴──────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES PCOMPUTE])
-AT_KEYWORDS([postcompute])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /PCOMPUTE &x=EXPR([3] + [4])
- /PCOMPUTE &y=EXPR([4] + [5])
- /PPROPERTIES &x LABEL='3+4' FORMAT=COUNT F8.2
- /PPROPERTIES &y LABEL=')LABEL[5]+)LABEL[6]'
- /TABLE=qn105ba [COUNT, ROWPCT] BY qns1
- /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, &x, &y, SUBTOTAL] TOTAL=YES
-
-* Adding HIDESOURCECATS=YES for one PPROPERTIES.
-CTABLES
- /PCOMPUTE &x=EXPR([3] + [4])
- /PCOMPUTE &y=EXPR([4] + [5])
- /PPROPERTIES &x LABEL='3+4' FORMAT=COUNT F8.2
- /PPROPERTIES &y LABEL=')LABEL[5]+)LABEL[6]' HIDESOURCECATS=YES
- /TABLE=qn105ba [COUNT, ROWPCT] BY qns1
- /CATEGORIES VARIABLES=qns1 [1, 2, SUBTOTAL, 3, 4, 5, &x, &y, SUBTOTAL] TOTAL=YES
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=140], [0], [dnl
- Custom Tables
-╭───────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ │ S1. Including yourself, how many members of this household are age 16 or older? │
-│ ├───────────┬───────────┬───────────┬───────────┬──────────┬──────────┬────────────┬──────────┬───────────┬────────────┤
-│ │ 1 │ 2 │ Subtotal │ 3 │ 4 │ 5 │ 3+4 │ 4+5 │ Subtotal │ Total │
-│ ├─────┬─────┼─────┬─────┼─────┬─────┼─────┬─────┼─────┬────┼─────┬────┼──────┬─────┼─────┬────┼─────┬─────┼─────┬──────┤
-│ │ │ │ │ │ │ │ │ │ │ Row│ │ Row│ │ │ │ Row│ │ │ │ │
-│ │Count│Row %│Count│Row %│Count│Row %│Count│Row %│Count│ % │Count│ % │ Count│Row %│Count│ % │Count│Row %│Count│ Row %│
-├───────────────────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼────┼─────┼────┼──────┼─────┼─────┼────┼─────┼─────┼─────┼──────┤
-│105b. How Almost │ 147│30.3%│ 246│50.7%│ 393│81.0%│ 62│12.8%│ 19│3.9%│ 11│2.3%│ 81.00│16.7%│ 30│6.2%│ 92│19.0%│ 485│100.0%│
-│likely is certain │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│it that │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│drivers Very │ 384│34.3%│ 552│49.2%│ 936│83.5%│ 120│10.7%│ 51│4.5%│ 14│1.2%│171.00│15.3%│ 65│5.8%│ 185│16.5%│ 1121│100.0%│
-│who have likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│had too │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│much to Somewhat│ 590│27.8%│ 1249│58.8%│ 1839│86.6%│ 193│ 9.1%│ 72│3.4%│ 20│ .9%│265.00│12.5%│ 92│4.3%│ 285│13.4%│ 2124│100.0%│
-│drink to likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│drive │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│safely Somewhat│ 278│26.6%│ 647│61.8%│ 925│88.3%│ 84│ 8.0%│ 32│3.1%│ 6│ .6%│116.00│11.1%│ 38│3.6%│ 122│11.7%│ 1047│100.0%│
-│will A. unlikely│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│Get │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│stopped by Very │ 141│28.5%│ 290│58.7%│ 431│87.2%│ 41│ 8.3%│ 18│3.6%│ 4│ .8%│ 59.00│11.9%│ 22│4.5%│ 63│12.8%│ 494│100.0%│
-│the unlikely│ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│police? │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-╰───────────────────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴────┴─────┴────┴──────┴─────┴─────┴────┴─────┴─────┴─────┴──────╯
-
- Custom Tables
-╭─────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ │ S1. Including yourself, how many members of this household are age 16 or older? │
-│ ├───────────┬───────────┬───────────┬───────────┬────────────┬──────────┬───────────┬────────────┤
-│ │ 1 │ 2 │ Subtotal │ 3 │ 3+4 │ 4+5 │ Subtotal │ Total │
-│ ├─────┬─────┼─────┬─────┼─────┬─────┼─────┬─────┼──────┬─────┼─────┬────┼─────┬─────┼─────┬──────┤
-│ │ │ │ │ │ │ │ │ │ │ │ │ Row│ │ │ │ │
-│ │Count│Row %│Count│Row %│Count│Row %│Count│Row %│ Count│Row %│Count│ % │Count│Row %│Count│ Row %│
-├─────────────────────────────────────────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼─────┼──────┼─────┼─────┼────┼─────┼─────┼─────┼──────┤
-│105b. How likely is it that Almost │ 147│30.3%│ 246│50.7%│ 393│81.0%│ 62│12.8%│ 81.00│16.7%│ 30│6.2%│ 92│19.0%│ 485│100.0%│
-│drivers who have had too much certain │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│to drink to drive safely will Very │ 384│34.3%│ 552│49.2%│ 936│83.5%│ 120│10.7%│171.00│15.3%│ 65│5.8%│ 185│16.5%│ 1121│100.0%│
-│A. Get stopped by the police? likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│ Somewhat │ 590│27.8%│ 1249│58.8%│ 1839│86.6%│ 193│ 9.1%│265.00│12.5%│ 92│4.3%│ 285│13.4%│ 2124│100.0%│
-│ likely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│ Somewhat │ 278│26.6%│ 647│61.8%│ 925│88.3%│ 84│ 8.0%│116.00│11.1%│ 38│3.6%│ 122│11.7%│ 1047│100.0%│
-│ unlikely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-│ Very │ 141│28.5%│ 290│58.7%│ 431│87.2%│ 41│ 8.3%│ 59.00│11.9%│ 22│4.5%│ 63│12.8%│ 494│100.0%│
-│ unlikely │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-╰─────────────────────────────────────────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┴──────┴─────┴─────┴────┴─────┴─────┴─────┴──────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES PCOMPUTE - OTHERNM and MISSING])
-AT_KEYWORDS([postcompute])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /PCOMPUTE &x=EXPR(OTHERNM)
- /PCOMPUTE &y=EXPR(MISSING)
- /PPROPERTIES &x LABEL='Drivers'
- /PPROPERTIES &y LABEL='Missing Values 2'
- /TABLE=qn1 BY qns3a
- /CATEGORIES VARIABLES=qn1 [OTHERNM, 5, &x, SUBTOTAL='Valid Values', MISSING, SUBTOTAL='Missing Values', &y]
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=140], [0], [dnl
- Custom Tables
-╭──────────────────────────────────────────────────────────────────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├──────────────────────────────────────────────────────────────────────────────────────────┼─────┼──────┤
-│ 1. How often do you usually drive a car or other motor vehicle? Every day │ 2305│ 2362│
-│ Several days a week │ 440│ 834│
-│ Once a week or less │ 125│ 236│
-│ Only certain times a year│ 58│ 72│
-│ Never │ 192│ 348│
-│ Drivers │ 2928│ 3504│
-│ Valid Values │ 3120│ 3852│
-│ Don't know │ 3│ 5│
-│ Refused │ 9│ 10│
-│ Missing Values │ 12│ 15│
-│ Missing Values 2 │ 12│ 15│
-╰──────────────────────────────────────────────────────────────────────────────────────────┴─────┴──────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES PCOMPUTE - THRU])
-AT_KEYWORDS([postcompute])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /PCOMPUTE &x=EXPR([1 THRU 2])
- /PCOMPUTE &y=EXPR([3 THRU 4])
- /PCOMPUTE &z=EXPR([5] + MISSING)
- /PPROPERTIES &x LABEL='Frequent Drivers'
- /PPROPERTIES &y LABEL='Infrequent Drivers'
- /PPROPERTIES &z LABEL='Not Drivers or Missing'
- /TABLE=qn1 BY qns3a
- /CATEGORIES VARIABLES=qn1 [1 THRU 2, &x, 3 THRU 4, &y, SUBTOTAL='Drivers', 5, MISSING, &z]
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=140], [0], [dnl
- Custom Tables
-╭──────────────────────────────────────────────────────────────────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├──────────────────────────────────────────────────────────────────────────────────────────┼─────┼──────┤
-│ 1. How often do you usually drive a car or other motor vehicle? Every day │ 2305│ 2362│
-│ Several days a week │ 440│ 834│
-│ Frequent Drivers │ 2745│ 3196│
-│ Once a week or less │ 125│ 236│
-│ Only certain times a year│ 58│ 72│
-│ Infrequent Drivers │ 183│ 308│
-│ Drivers │ 2928│ 3504│
-│ Never │ 192│ 348│
-│ Don't know │ 3│ 5│
-│ Refused │ 9│ 10│
-│ Not Drivers or Missing │ 204│ 363│
-╰──────────────────────────────────────────────────────────────────────────────────────────┴─────┴──────╯
-])
-AT_CLEANUP
-
-dnl I'm not sure that this is the correct behavior (see
-dnl https://mail.gnu.org/archive/html/pspp-users/2022-07/msg00002.html)
-dnl but at least this test will notify us if the behavior changes.
-AT_SETUP([CTABLES intersecting PCOMPUTEs])
-AT_KEYWORDS([PCOMPUTE postcompute])
-AT_DATA([ctables.sps],
-[[DATA LIST LIST NOTABLE/x y z.
-WEIGHT by z.
-FORMATS ALL (F1.0).
-VARIABLE LEVEL x y (NOMINAL).
-BEGIN DATA.
-1 4 5
-1 5 2
-1 6 9
-2 4 2
-2 5 3
-2 6 4
-3 4 1
-3 5 6
-3 6 1
-END DATA.
-
-CTABLES
- /PCOMPUTE &a = EXPR([1] + [2])
- /PCOMPUTE &b = EXPR([2] + [3])
- /PCOMPUTE &c = EXPR([4] * [5])
- /PCOMPUTE &d = EXPR([5] * [6])
- /TABLE x BY y
- /CATEGORIES VARIABLES=x [1, &a, 2, &b, 3]
- /CATEGORIES VARIABLES=y [4, &c, 5, &d, 6].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0],
-[[ Custom Tables
-╭───────────┬─────────────────────────────────────╮
-│ │ y │
-│ ├─────┬─────────┬─────┬─────────┬─────┤
-│ │ 4 │[4] * [5]│ 5 │[5] * [6]│ 6 │
-│ ├─────┼─────────┼─────┼─────────┼─────┤
-│ │Count│ Count │Count│ Count │Count│
-├───────────┼─────┼─────────┼─────┼─────────┼─────┤
-│x 1 │ 5│ 10│ 2│ 18│ 9│
-│ [1] + [2]│ 7│ .│ 5│ .│ 13│
-│ 2 │ 2│ 6│ 3│ 12│ 4│
-│ [2] + [3]│ 3│ .│ 9│ .│ 5│
-│ 3 │ 1│ 6│ 6│ 6│ 1│
-╰───────────┴─────┴─────────┴─────┴─────────┴─────╯
-]])
-AT_CLEANUP
-
-AT_SETUP([CTABLES string and date and time])
-
-weight=1
-for gender in F M; do
- for month in Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec; do
- for wkday in Sun Mon Tue Wed Thu Fri Sat Sun; do
- printf "$weight $gender $month $wkday\n"
- weight=$(expr \( $weight + 3 \) % 7 + 2)
- done
- done
-done > ctables.txt
-
-AT_DATA([ctables.sps],
-[[DATA LIST LIST NOTABLE FILE='ctables.txt'
- /w (F5.0) gender (A1) fmon (MONTH3) fday (WKDAY3).
-WEIGHT by w.
-VARIABLE LEVEL w (SCALE).
-VARIABLE LEVEL gender fmon fday (NOMINAL).
-VARIABLE LABEL
- gender 'Gender'
- fmon 'Favorite month'
- fday 'Favorite day of the week'.
-VALUE LABELS /gender 'M' 'Male' 'F' 'Female'.
-CTABLES
- /PCOMPUTE &q2 = EXPR(['APR' THRU 'June'])
- /PPROPERTIES &q2 LABEL='Q2'
- /PCOMPUTE &weekend = EXPR(['sun'] + ['Sat'])
- /PPROPERTIES &weekend LABEL='Weekend'
- /TABLE fmon BY gender > fday
- /CATEGORIES VARIABLES=fmon ['JAN', 'FEB', 'Mar', SUBTOTAL="Q1",
- 4 THRU 6, &q2,
- 'JUL' THRU 'sep', SUBTOTAL="Q3",
- OTHERNM, SUBTOTAL='Q4']
- /CATEGORIES VARIABLES=gender ['M', 'F']
- /CATEGORIES VARIABLE=fday ['Sun', 2 THRU 6, 'Sat', &weekend] TOTAL=YES
- /SLABELS VISIBLE=NO.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-╭──────────────────┬───────────────────────────────────────────────────────────────────────────────────╮
-│ │ Gender │
-│ ├─────────────────────────────────────────┬─────────────────────────────────────────┤
-│ │ Male │ Female │
-│ ├─────────────────────────────────────────┼─────────────────────────────────────────┤
-│ │ Favorite day of the week │ Favorite day of the week │
-│ ├───┬───┬───┬───┬───┬───┬───┬───────┬─────┼───┬───┬───┬───┬───┬───┬───┬───────┬─────┤
-│ │SUN│MON│TUE│WED│THU│FRI│SAT│Weekend│Total│SUN│MON│TUE│WED│THU│FRI│SAT│Weekend│Total│
-├──────────────────┼───┼───┼───┼───┼───┼───┼───┼───────┼─────┼───┼───┼───┼───┼───┼───┼───┼───────┼─────┤
-│Favorite month JAN│ 10│ 3│ 8│ 6│ 4│ 2│ 7│ 17│ 40│ 9│ 6│ 4│ 2│ 7│ 5│ 3│ 12│ 36│
-│ FEB│ 6│ 8│ 6│ 4│ 2│ 7│ 5│ 11│ 38│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│
-│ MAR│ 16│ 6│ 4│ 2│ 7│ 5│ 3│ 19│ 43│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│
-│ Q1 │ 32│ 17│ 18│ 12│ 13│ 14│ 15│ │ │ 29│ 12│ 13│ 14│ 15│ 16│ 17│ │ │
-│ APR│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│ 4│ 7│ 5│ 3│ 8│ 6│ 4│ 8│ 37│
-│ MAY│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│ 14│ 5│ 3│ 8│ 6│ 4│ 2│ 16│ 42│
-│ JUN│ 4│ 7│ 5│ 3│ 8│ 6│ 4│ 8│ 37│ 10│ 3│ 8│ 6│ 4│ 2│ 7│ 17│ 40│
-│ Q2 │ 24│ 13│ 14│ 15│ 16│ 17│ 18│ .│ │ 28│ 15│ 16│ 17│ 18│ 12│ 13│ .│ │
-│ JUL│ 14│ 5│ 3│ 8│ 6│ 4│ 2│ 16│ 42│ 6│ 8│ 6│ 4│ 2│ 7│ 5│ 11│ 38│
-│ AUG│ 10│ 3│ 8│ 6│ 4│ 2│ 7│ 17│ 40│ 16│ 6│ 4│ 2│ 7│ 5│ 3│ 19│ 43│
-│ SEP│ 6│ 8│ 6│ 4│ 2│ 7│ 5│ 11│ 38│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│
-│ Q3 │ 54│ 29│ 31│ 33│ 28│ 30│ 32│ │ │ 62│ 33│ 28│ 30│ 32│ 27│ 29│ │ │
-│ OCT│ 16│ 6│ 4│ 2│ 7│ 5│ 3│ 19│ 43│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│
-│ NOV│ 12│ 4│ 2│ 7│ 5│ 3│ 8│ 20│ 41│ 4│ 7│ 5│ 3│ 8│ 6│ 4│ 8│ 37│
-│ DEC│ 8│ 2│ 7│ 5│ 3│ 8│ 6│ 14│ 39│ 14│ 5│ 3│ 8│ 6│ 4│ 2│ 16│ 42│
-│ Q4 │ 36│ 12│ 13│ 14│ 15│ 16│ 17│ │ │ 26│ 14│ 15│ 16│ 17│ 18│ 12│ │ │
-╰──────────────────┴───┴───┴───┴───┴───┴───┴───┴───────┴─────┴───┴───┴───┴───┴───┴───┴───┴───────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES CLABELS])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE AgeGroup BY qns3a.
-CTABLES /TABLE AgeGroup BY qns3a /CLABELS ROWLABELS=OPPOSITE.
-CTABLES /TABLE AgeGroup BY qns3a /CLABELS COLLABELS=OPPOSITE.
-CTABLES /TABLE AgeGroup BY qns3a /CLABELS ROWLABELS=LAYER.
-CTABLES /TABLE AgeGroup BY qns3a /CLABELS COLLABELS=LAYER.
-]])
-AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-╭───────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├───────────────────────┼─────┼──────┤
-│Age group 15 or younger│ 0│ 0│
-│ 16 to 25 │ 594│ 505│
-│ 26 to 35 │ 476│ 491│
-│ 36 to 45 │ 489│ 548│
-│ 46 to 55 │ 526│ 649│
-│ 56 to 65 │ 516│ 731│
-│ 66 or older │ 531│ 943│
-╰───────────────────────┴─────┴──────╯
-
- Custom Tables
-╭───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ │ S3a. GENDER: │
-│ ├──────────────────────────────────────────────────────┬───────────────────────────────────────────────────────┤
-│ │ Male │ Female │
-│ ├─────────┬───────┬──────┬──────┬──────┬───────┬───────┼──────────┬──────┬───────┬──────┬──────┬──────┬────────┤
-│ │ 15 or │ 16 to │ 26 to│ 36 to│ 46 to│ 56 to │ 66 or │ 15 or │ 16 to│ 26 to │ 36 to│ 46 to│ 56 to│ 66 or │
-│ │ younger │ 25 │ 35 │ 45 │ 55 │ 65 │ older │ younger │ 25 │ 35 │ 45 │ 55 │ 65 │ older │
-│ ├─────────┼───────┼──────┼──────┼──────┼───────┼───────┼──────────┼──────┼───────┼──────┼──────┼──────┼────────┤
-│ │ Count │ Count │ Count│ Count│ Count│ Count │ Count │ Count │ Count│ Count │ Count│ Count│ Count│ Count │
-├───────┼─────────┼───────┼──────┼──────┼──────┼───────┼───────┼──────────┼──────┼───────┼──────┼──────┼──────┼────────┤
-│Age │ 0│ 594│ 476│ 489│ 526│ 516│ 531│ 0│ 505│ 491│ 548│ 649│ 731│ 943│
-│group │ │ │ │ │ │ │ │ │ │ │ │ │ │ │
-╰───────┴─────────┴───────┴──────┴──────┴──────┴───────┴───────┴──────────┴──────┴───────┴──────┴──────┴──────┴────────╯
-
- Custom Tables
-╭──────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├────────────┤
-│ │ Count │
-├──────────────────────────────┼────────────┤
-│Age group 15 or younger Male │ 0│
-│ Female│ 0│
-│ ╶────────────────────┼────────────┤
-│ 16 to 25 Male │ 594│
-│ Female│ 505│
-│ ╶────────────────────┼────────────┤
-│ 26 to 35 Male │ 476│
-│ Female│ 491│
-│ ╶────────────────────┼────────────┤
-│ 36 to 45 Male │ 489│
-│ Female│ 548│
-│ ╶────────────────────┼────────────┤
-│ 46 to 55 Male │ 526│
-│ Female│ 649│
-│ ╶────────────────────┼────────────┤
-│ 56 to 65 Male │ 516│
-│ Female│ 731│
-│ ╶────────────────────┼────────────┤
-│ 66 or older Male │ 531│
-│ Female│ 943│
-╰──────────────────────────────┴────────────╯
-
- Custom Tables
-15 or younger
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 0│ 0│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-16 to 25
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 594│ 505│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-26 to 35
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 476│ 491│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-36 to 45
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 489│ 548│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-46 to 55
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 526│ 649│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-56 to 65
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 516│ 731│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-66 or older
-╭─────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────┼─────┼──────┤
-│Age group│ 531│ 943│
-╰─────────┴─────┴──────╯
-
- Custom Tables
-Male
-╭───────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├────────────┤
-│ │ Count │
-├───────────────────────┼────────────┤
-│Age group 15 or younger│ 0│
-│ 16 to 25 │ 594│
-│ 26 to 35 │ 476│
-│ 36 to 45 │ 489│
-│ 46 to 55 │ 526│
-│ 56 to 65 │ 516│
-│ 66 or older │ 531│
-╰───────────────────────┴────────────╯
-
- Custom Tables
-Female
-╭───────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├────────────┤
-│ │ Count │
-├───────────────────────┼────────────┤
-│Age group 15 or younger│ 0│
-│ 16 to 25 │ 505│
-│ 26 to 35 │ 491│
-│ 36 to 45 │ 548│
-│ 46 to 55 │ 649│
-│ 56 to 65 │ 731│
-│ 66 or older │ 943│
-╰───────────────────────┴────────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES missing values])
-AT_DATA([ctables.sps],
-[[DATA LIST LIST NOTABLE/x y.
-BEGIN DATA.
-1 1
-1 2
-1 3
-1 4
-1 5
-1 .
-2 1
-2 2
-2 3
-2 4
-2 5
-2 .
-3 1
-3 2
-3 3
-3 4
-3 5
-3 .
-4 1
-4 2
-4 3
-4 4
-4 5
-4 .
-5 1
-5 2
-5 3
-5 4
-5 5
-5 .
-. 1
-. 2
-. 3
-. 4
-. 5
-. .
-END DATA.
-MISSING VALUES x (1, 2) y (2, 3).
-VARIABLE LEVEL ALL (NOMINAL).
-
-CTABLES /TABLE x[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN,
- TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, VALIDN, TOTALN]]
- /CATEGORIES VARIABLES=ALL TOTAL=YES.
-CTABLES /TABLE x[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN,
- TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, VALIDN, TOTALN]]
- /CATEGORIES VARIABLES=ALL TOTAL=YES MISSING=INCLUDE.
-CTABLES /TABLE x BY y[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN,
- TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, VALIDN, TOTALN]]
- /CATEGORIES VARIABLES=ALL TOTAL=YES
- /SLABELS POSITION=ROW.
-CTABLES /TABLE x BY y[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN,
- TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, VALIDN, TOTALN]]
- /CATEGORIES VARIABLES=ALL TOTAL=YES MISSING=INCLUDE
- /SLABELS POSITION=ROW.
-CTABLES /TABLE x BY y[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN,
- TOTALS[COUNT, COLPCT, COLPCT.VALIDN, COLPCT.TOTALN, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, VALIDN, TOTALN]]
- /CATEGORIES VARIABLES=x [1, 2, 3, 4] TOTAL=YES
- /CATEGORIES VARIABLES=y [1, 3, 4, 5] TOTAL=YES
- /SLABELS POSITION=ROW.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-╭───────┬─────┬────────┬────────────────┬────────────────┬───────┬───────╮
-│ │Count│Column %│Column Valid N %│Column Total N %│Valid N│Total N│
-├───────┼─────┼────────┼────────────────┼────────────────┼───────┼───────┤
-│x 3.00 │ 6│ 33.3%│ 33.3%│ 16.7%│ │ │
-│ 4.00 │ 6│ 33.3%│ 33.3%│ 16.7%│ │ │
-│ 5.00 │ 6│ 33.3%│ 33.3%│ 16.7%│ │ │
-│ Total│ 18│ 100.0%│ 100.0%│ 100.0%│ 18│ 36│
-╰───────┴─────┴────────┴────────────────┴────────────────┴───────┴───────╯
-dnl Note that Column Total N % doesn't add up to 100 because missing
-dnl values are included in the total but not shown as a category and this
-dnl is expected behavior.
-
- Custom Tables
-╭───────┬─────┬────────┬────────────────┬────────────────┬───────┬───────╮
-│ │Count│Column %│Column Valid N %│Column Total N %│Valid N│Total N│
-├───────┼─────┼────────┼────────────────┼────────────────┼───────┼───────┤
-│x 1.00 │ 6│ 20.0%│ .0%│ 16.7%│ │ │
-│ 2.00 │ 6│ 20.0%│ .0%│ 16.7%│ │ │
-│ 3.00 │ 6│ 20.0%│ 33.3%│ 16.7%│ │ │
-│ 4.00 │ 6│ 20.0%│ 33.3%│ 16.7%│ │ │
-│ 5.00 │ 6│ 20.0%│ 33.3%│ 16.7%│ │ │
-│ Total│ 30│ 100.0%│ 100.0%│ 100.0%│ 18│ 36│
-╰───────┴─────┴────────┴────────────────┴────────────────┴───────┴───────╯
-dnl Note that Column Total N % doesn't add up to 100 because system-missing
-dnl values are included in the total but not shown as a category and this
-dnl is expected behavior.
-
- Custom Tables
-╭────────────────────────┬───────────────────────────╮
-│ │ y │
-│ ├──────┬──────┬──────┬──────┤
-│ │ 1.00 │ 4.00 │ 5.00 │ Total│
-├────────────────────────┼──────┼──────┼──────┼──────┤
-│x 3.00 Count │ 1│ 1│ 1│ 3│
-│ Column % │ 33.3%│ 33.3%│ 33.3%│ .│
-│ Column Valid N %│ 33.3%│ 33.3%│ 33.3%│ .│
-│ Column Total N %│ 33.3%│ 33.3%│ 33.3%│ .│
-│ Row % │ 33.3%│ 33.3%│ 33.3%│100.0%│
-│ Row Valid N % │ 33.3%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ 3│
-│ Total N │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┤
-│ 4.00 Count │ 1│ 1│ 1│ 3│
-│ Column % │ 33.3%│ 33.3%│ 33.3%│ .│
-│ Column Valid N %│ 33.3%│ 33.3%│ 33.3%│ .│
-│ Column Total N %│ 33.3%│ 33.3%│ 33.3%│ .│
-│ Row % │ 33.3%│ 33.3%│ 33.3%│100.0%│
-│ Row Valid N % │ 33.3%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ 3│
-│ Total N │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┤
-│ 5.00 Count │ 1│ 1│ 1│ 3│
-│ Column % │ 33.3%│ 33.3%│ 33.3%│ .│
-│ Column Valid N %│ 33.3%│ 33.3%│ 33.3%│ .│
-│ Column Total N %│ 33.3%│ 33.3%│ 33.3%│ .│
-│ Row % │ 33.3%│ 33.3%│ 33.3%│100.0%│
-│ Row Valid N % │ 33.3%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ 3│
-│ Total N │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┤
-│ Total Count │ 3│ 3│ 3│ 9│
-│ Column % │100.0%│100.0%│100.0%│ .│
-│ Column Valid N %│100.0%│100.0%│100.0%│ .│
-│ Column Total N %│100.0%│100.0%│100.0%│ .│
-│ Row % │ .│ .│ .│ .│
-│ Row Valid N % │ .│ .│ .│ .│
-│ Row Total N % │ .│ .│ .│ .│
-│ Valid N │ 3│ 3│ 3│ 9│
-│ Total N │ 3│ 3│ 3│ 18│
-╰────────────────────────┴──────┴──────┴──────┴──────╯
-
- Custom Tables
-╭────────────────────────┬─────────────────────────────────────────╮
-│ │ y │
-│ ├──────┬──────┬──────┬──────┬──────┬──────┤
-│ │ 1.00 │ 2.00 │ 3.00 │ 4.00 │ 5.00 │ Total│
-├────────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│x 1.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
-│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
-│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ 2.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
-│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
-│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ 3.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
-│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
-│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ 4.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
-│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
-│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ 5.00 Count │ 1│ 1│ 1│ 1│ 1│ 5│
-│ Column % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Column Valid N %│ 20.0%│ .│ .│ 20.0%│ 20.0%│ .│
-│ Column Total N %│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│ .│
-│ Row % │ 20.0%│ 20.0%│ 20.0%│ 20.0%│ 20.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ Total Count │ 5│ 5│ 5│ 5│ 5│ 25│
-│ Column % │100.0%│100.0%│100.0%│100.0%│100.0%│ .│
-│ Column Valid N %│100.0%│ .│ .│100.0%│100.0%│ .│
-│ Column Total N %│100.0%│100.0%│100.0%│100.0%│100.0%│ .│
-│ Row % │ .│ .│ .│ .│ .│ .│
-│ Row Valid N % │ .│ .│ .│ .│ .│ .│
-│ Row Total N % │ .│ .│ .│ .│ .│ .│
-│ Valid N │ 5│ 0│ 0│ 5│ 5│ 15│
-│ Total N │ 5│ 5│ 5│ 5│ 5│ 30│
-╰────────────────────────┴──────┴──────┴──────┴──────┴──────┴──────╯
-
- Custom Tables
-╭────────────────────────┬──────────────────────────────────╮
-│ │ y │
-│ ├──────┬──────┬──────┬──────┬──────┤
-│ │ 1.00 │ 3.00 │ 4.00 │ 5.00 │ Total│
-├────────────────────────┼──────┼──────┼──────┼──────┼──────┤
-│x 1.00 Count │ 1│ 1│ 1│ 1│ 4│
-│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
-│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
-│ 2.00 Count │ 1│ 1│ 1│ 1│ 4│
-│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
-│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
-│ 3.00 Count │ 1│ 1│ 1│ 1│ 4│
-│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
-│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
-│ 4.00 Count │ 1│ 1│ 1│ 1│ 4│
-│ Column % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Column Valid N %│ 25.0%│ .│ 25.0%│ 25.0%│ .│
-│ Column Total N %│ 25.0%│ 25.0%│ 25.0%│ 25.0%│ .│
-│ Row % │ 25.0%│ 25.0%│ 25.0%│ 25.0%│100.0%│
-│ Row Valid N % │ 33.3%│ .0%│ 33.3%│ 33.3%│100.0%│
-│ Row Total N % │ 16.7%│ 16.7%│ 16.7%│ 16.7%│100.0%│
-│ Valid N │ │ │ │ │ 3│
-│ Total N │ │ │ │ │ 6│
-│ ╶──────────────────────┼──────┼──────┼──────┼──────┼──────┤
-│ Total Count │ 4│ 4│ 4│ 4│ 16│
-│ Column % │100.0%│100.0%│100.0%│100.0%│ .│
-│ Column Valid N %│100.0%│ .│100.0%│100.0%│ .│
-│ Column Total N %│100.0%│100.0%│100.0%│100.0%│ .│
-│ Row % │ .│ .│ .│ .│ .│
-│ Row Valid N % │ .│ .│ .│ .│ .│
-│ Row Total N % │ .│ .│ .│ .│ .│
-│ Valid N │ 4│ 0│ 4│ 4│ 12│
-│ Total N │ 4│ 4│ 4│ 4│ 24│
-╰────────────────────────┴──────┴──────┴──────┴──────┴──────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES SMISSING=LISTWISE])
-AT_KEYWORDS([SMISSING LISTWISE])
-AT_DATA([ctables.sps],
-[[DATA LIST LIST NOTABLE/x y z.
-BEGIN DATA.
-1 . 40
-1 10 50
-1 20 60
-1 . .
-1 30 .
-END DATA.
-VARIABLE LEVEL x (NOMINAL).
-
-CTABLES /TABLE (y + z) > x.
-CTABLES /SMISSING LISTWISE /TABLE (y + z) > x.
-
-* The following doesn't come out as listwise because the tables are
-separate, not linked by an > operator.
-CTABLES /SMISSING LISTWISE /TABLE (y > x) + (z > x).
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-╭────────┬─────╮
-│ │ Mean│
-├────────┼─────┤
-│y x 1.00│20.00│
-├────────┼─────┤
-│z x 1.00│50.00│
-╰────────┴─────╯
-
- Custom Tables
-╭────────┬─────╮
-│ │ Mean│
-├────────┼─────┤
-│y x 1.00│15.00│
-├────────┼─────┤
-│z x 1.00│55.00│
-╰────────┴─────╯
-
- Custom Tables
-╭────────┬─────╮
-│ │ Mean│
-├────────┼─────┤
-│y x 1.00│20.00│
-├────────┼─────┤
-│z x 1.00│50.00│
-╰────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES VLABELS - variables on different axes])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=DEFAULT /TABLE qnd5a BY qns3a.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NAME /TABLE qnd5a BY qns3a.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=LABEL /TABLE qnd5a BY qns3a.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=BOTH /TABLE qnd5a BY qns3a.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NONE /TABLE qnd5a BY qns3a.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭────────────────────────────────────────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├────────────────────────────────────────────────────────────────┼─────┼──────┤
-│D5a. What would you say is your primary Cuban │ 13│ 7│
-│ethnic background? Mexican │ 175│ 136│
-│ Spanish │ 20│ 28│
-│ South American │ 21│ 13│
-│ Central American │ 27│ 25│
-│ Puerto Rican, OR │ 37│ 41│
-│ Something else │ 35│ 33│
-│ Multiple - cannot │ 2│ 5│
-│ choose one │ │ │
-╰────────────────────────────────────────────────────────────────┴─────┴──────╯
-
- Custom Tables
-╭──────────────────────────────────┬────────────╮
-│ │ QNS3A │
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├──────────────────────────────────┼─────┼──────┤
-│QND5A Cuban │ 13│ 7│
-│ Mexican │ 175│ 136│
-│ Spanish │ 20│ 28│
-│ South American │ 21│ 13│
-│ Central American │ 27│ 25│
-│ Puerto Rican, OR │ 37│ 41│
-│ Something else │ 35│ 33│
-│ Multiple - cannot choose one│ 2│ 5│
-╰──────────────────────────────────┴─────┴──────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├────────────────────────────────────────────────────────────────┼─────┼──────┤
-│D5a. What would you say is your primary Cuban │ 13│ 7│
-│ethnic background? Mexican │ 175│ 136│
-│ Spanish │ 20│ 28│
-│ South American │ 21│ 13│
-│ Central American │ 27│ 25│
-│ Puerto Rican, OR │ 37│ 41│
-│ Something else │ 35│ 33│
-│ Multiple - cannot │ 2│ 5│
-│ choose one │ │ │
-╰────────────────────────────────────────────────────────────────┴─────┴──────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────┬────────────────╮
-│ │ QNS3A S3a. │
-│ │ GENDER: │
-│ ├───────┬────────┤
-│ │ Male │ Female │
-│ ├───────┼────────┤
-│ │ Count │ Count │
-├────────────────────────────────────────────────────────────┼───────┼────────┤
-│QND5A D5a. What would you say is your Cuban │ 13│ 7│
-│primary ethnic background? Mexican │ 175│ 136│
-│ Spanish │ 20│ 28│
-│ South American │ 21│ 13│
-│ Central American │ 27│ 25│
-│ Puerto Rican, OR │ 37│ 41│
-│ Something else │ 35│ 33│
-│ Multiple - cannot │ 2│ 5│
-│ choose one │ │ │
-╰────────────────────────────────────────────────────────────┴───────┴────────╯
-
- Custom Tables
-╭────────────────────────────┬─────┬──────╮
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├────────────────────────────┼─────┼──────┤
-│Cuban │ 13│ 7│
-│Mexican │ 175│ 136│
-│Spanish │ 20│ 28│
-│South American │ 21│ 13│
-│Central American │ 27│ 25│
-│Puerto Rican, OR │ 37│ 41│
-│Something else │ 35│ 33│
-│Multiple - cannot choose one│ 2│ 5│
-╰────────────────────────────┴─────┴──────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES VLABELS - stacked variables])
-AT_KEYWORDS([stack stacking])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NAME /TABLE qnd5a + qns3a.
-CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NAME /TABLE qnd5a + qns3a.
-CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NONE /TABLE qnd5a + qns3a.
-CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NONE /TABLE qnd5a + qns3a.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NONE /TABLE qnd5a + qns3a.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭───────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary ethnic Cuban │ 20│
-│background? Mexican │ 311│
-│ Spanish │ 48│
-│ South American │ 34│
-│ Central American │ 52│
-│ Puerto Rican, OR │ 78│
-│ Something else │ 68│
-│ Multiple - cannot │ 7│
-│ choose one │ │
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│QNS3A Male │ 3132│
-│ Female │ 3867│
-╰───────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭─────────────────────────────────────────┬─────╮
-│ │Count│
-├─────────────────────────────────────────┼─────┤
-│QND5A Cuban │ 20│
-│ Mexican │ 311│
-│ Spanish │ 48│
-│ South American │ 34│
-│ Central American │ 52│
-│ Puerto Rican, OR │ 78│
-│ Something else │ 68│
-│ Multiple - cannot choose one│ 7│
-├─────────────────────────────────────────┼─────┤
-│S3a. GENDER: Male │ 3132│
-│ Female │ 3867│
-╰─────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭───────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary ethnic Cuban │ 20│
-│background? Mexican │ 311│
-│ Spanish │ 48│
-│ South American │ 34│
-│ Central American │ 52│
-│ Puerto Rican, OR │ 78│
-│ Something else │ 68│
-│ Multiple - cannot │ 7│
-│ choose one │ │
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│Male │ 3132│
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│Female │ 3867│
-╰───────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭─────────────────────────────┬─────╮
-│ │Count│
-├─────────────────────────────┼─────┤
-│Cuban │ 20│
-├─────────────────────────────┼─────┤
-│Mexican │ 311│
-├─────────────────────────────┼─────┤
-│Spanish │ 48│
-├─────────────────────────────┼─────┤
-│South American │ 34│
-├─────────────────────────────┼─────┤
-│Central American │ 52│
-├─────────────────────────────┼─────┤
-│Puerto Rican, OR │ 78│
-├─────────────────────────────┼─────┤
-│Something else │ 68│
-├─────────────────────────────┼─────┤
-│Multiple - cannot choose one │ 7│
-├─────────────────────────────┼─────┤
-│S3a. GENDER: Male │ 3132│
-│ Female │ 3867│
-╰─────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────┼─────┤
-│Cuban │ 20│
-│Mexican │ 311│
-│Spanish │ 48│
-│South American │ 34│
-│Central American │ 52│
-│Puerto Rican, OR │ 78│
-│Something else │ 68│
-│Multiple - cannot choose one│ 7│
-│Male │ 3132│
-│Female │ 3867│
-╰────────────────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES VLABELS - nested variables])
-AT_KEYWORDS([nest nesting])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NAME /TABLE qnd5a > qns3a.
-CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NAME /TABLE qnd5a > qns3a.
-CTABLES /VLABELS VARIABLES=qns3a DISPLAY=NONE /TABLE qnd5a > qns3a.
-CTABLES /VLABELS VARIABLES=qnd5a DISPLAY=NONE /TABLE qnd5a > qns3a.
-CTABLES /VLABELS VARIABLES=qns3a qnd5a DISPLAY=NONE /TABLE qnd5a > qns3a.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭───────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your Cuban QNS3A Male │ 13│
-│primary ethnic background? Female│ 7│
-│ ╶─────────────────────────────────┼─────┤
-│ Mexican QNS3A Male │ 175│
-│ Female│ 136│
-│ ╶─────────────────────────────────┼─────┤
-│ Spanish QNS3A Male │ 20│
-│ Female│ 28│
-│ ╶─────────────────────────────────┼─────┤
-│ South American QNS3A Male │ 21│
-│ Female│ 13│
-│ ╶─────────────────────────────────┼─────┤
-│ Central American QNS3A Male │ 27│
-│ Female│ 25│
-│ ╶─────────────────────────────────┼─────┤
-│ Puerto Rican, OR QNS3A Male │ 37│
-│ Female│ 41│
-│ ╶─────────────────────────────────┼─────┤
-│ Something else QNS3A Male │ 35│
-│ Female│ 33│
-│ ╶─────────────────────────────────┼─────┤
-│ Multiple - cannot QNS3A Male │ 2│
-│ choose one Female│ 5│
-╰───────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├──────────────────────────────────────────────────────┼─────┤
-│QND5A Cuban S3a. GENDER: Male │ 13│
-│ Female│ 7│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ Mexican S3a. GENDER: Male │ 175│
-│ Female│ 136│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ Spanish S3a. GENDER: Male │ 20│
-│ Female│ 28│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ South American S3a. GENDER: Male │ 21│
-│ Female│ 13│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ Central American S3a. GENDER: Male │ 27│
-│ Female│ 25│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ Puerto Rican, OR S3a. GENDER: Male │ 37│
-│ Female│ 41│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ Something else S3a. GENDER: Male │ 35│
-│ Female│ 33│
-│ ╶────────────────────────────────────────────────┼─────┤
-│ Multiple - cannot choose one S3a. GENDER: Male │ 2│
-│ Female│ 5│
-╰──────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭───────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├───────────────────────────────────────────────────────────────────────┼─────┤
-│D5a. What would you say is your primary Cuban Male │ 13│
-│ethnic background? Female│ 7│
-│ ╶────────────────────────────┼─────┤
-│ Mexican Male │ 175│
-│ Female│ 136│
-│ ╶────────────────────────────┼─────┤
-│ Spanish Male │ 20│
-│ Female│ 28│
-│ ╶────────────────────────────┼─────┤
-│ South American Male │ 21│
-│ Female│ 13│
-│ ╶────────────────────────────┼─────┤
-│ Central American Male │ 27│
-│ Female│ 25│
-│ ╶────────────────────────────┼─────┤
-│ Puerto Rican, OR Male │ 37│
-│ Female│ 41│
-│ ╶────────────────────────────┼─────┤
-│ Something else Male │ 35│
-│ Female│ 33│
-│ ╶────────────────────────────┼─────┤
-│ Multiple - cannot Male │ 2│
-│ choose one Female│ 5│
-╰───────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────┼─────┤
-│Cuban S3a. GENDER: Male │ 13│
-│ Female│ 7│
-├────────────────────────────────────────────────┼─────┤
-│Mexican S3a. GENDER: Male │ 175│
-│ Female│ 136│
-├────────────────────────────────────────────────┼─────┤
-│Spanish S3a. GENDER: Male │ 20│
-│ Female│ 28│
-├────────────────────────────────────────────────┼─────┤
-│South American S3a. GENDER: Male │ 21│
-│ Female│ 13│
-├────────────────────────────────────────────────┼─────┤
-│Central American S3a. GENDER: Male │ 27│
-│ Female│ 25│
-├────────────────────────────────────────────────┼─────┤
-│Puerto Rican, OR S3a. GENDER: Male │ 37│
-│ Female│ 41│
-├────────────────────────────────────────────────┼─────┤
-│Something else S3a. GENDER: Male │ 35│
-│ Female│ 33│
-├────────────────────────────────────────────────┼─────┤
-│Multiple - cannot choose one S3a. GENDER: Male │ 2│
-│ Female│ 5│
-╰────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭───────────────────────────────────┬─────╮
-│ │Count│
-├───────────────────────────────────┼─────┤
-│Cuban Male │ 13│
-│ Female│ 7│
-├───────────────────────────────────┼─────┤
-│Mexican Male │ 175│
-│ Female│ 136│
-├───────────────────────────────────┼─────┤
-│Spanish Male │ 20│
-│ Female│ 28│
-├───────────────────────────────────┼─────┤
-│South American Male │ 21│
-│ Female│ 13│
-├───────────────────────────────────┼─────┤
-│Central American Male │ 27│
-│ Female│ 25│
-├───────────────────────────────────┼─────┤
-│Puerto Rican, OR Male │ 37│
-│ Female│ 41│
-├───────────────────────────────────┼─────┤
-│Something else Male │ 35│
-│ Female│ 33│
-├───────────────────────────────────┼─────┤
-│Multiple - cannot choose one Male │ 2│
-│ Female│ 5│
-╰───────────────────────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES FORMAT EMPTY])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /FORMAT EMPTY=ZERO /TABLE qnd5a BY qnd5.
-CTABLES /FORMAT EMPTY=BLANK /TABLE qnd5a BY qnd5.
-CTABLES /FORMAT EMPTY='n/a' /TABLE qnd5a BY qnd5.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭─────────────────────────────────────────────┬───────────────────────────────╮
-│ │ D5. ETHNICITY: Are you of │
-│ │ Hispanic or Latino origin or │
-│ │ descent? │
-│ ├───────────────┬───────────────┤
-│ │ Yes │ No │
-│ ├───────────────┼───────────────┤
-│ │ Count │ Count │
-├─────────────────────────────────────────────┼───────────────┼───────────────┤
-│D5a. What would you say is Cuban │ 20│ 0│
-│your primary ethnic Mexican │ 311│ 0│
-│background? Spanish │ 48│ 0│
-│ South American │ 34│ 0│
-│ Central American│ 52│ 0│
-│ Puerto Rican, OR│ 78│ 0│
-│ Something else │ 68│ 0│
-│ Multiple - │ 7│ 0│
-│ cannot choose │ │ │
-│ one │ │ │
-╰─────────────────────────────────────────────┴───────────────┴───────────────╯
-
- Custom Tables
-╭─────────────────────────────────────────────┬───────────────────────────────╮
-│ │ D5. ETHNICITY: Are you of │
-│ │ Hispanic or Latino origin or │
-│ │ descent? │
-│ ├───────────────┬───────────────┤
-│ │ Yes │ No │
-│ ├───────────────┼───────────────┤
-│ │ Count │ Count │
-├─────────────────────────────────────────────┼───────────────┼───────────────┤
-│D5a. What would you say is Cuban │ 20│ │
-│your primary ethnic Mexican │ 311│ │
-│background? Spanish │ 48│ │
-│ South American │ 34│ │
-│ Central American│ 52│ │
-│ Puerto Rican, OR│ 78│ │
-│ Something else │ 68│ │
-│ Multiple - │ 7│ │
-│ cannot choose │ │ │
-│ one │ │ │
-╰─────────────────────────────────────────────┴───────────────┴───────────────╯
-
- Custom Tables
-╭─────────────────────────────────────────────┬───────────────────────────────╮
-│ │ D5. ETHNICITY: Are you of │
-│ │ Hispanic or Latino origin or │
-│ │ descent? │
-│ ├───────────────┬───────────────┤
-│ │ Yes │ No │
-│ ├───────────────┼───────────────┤
-│ │ Count │ Count │
-├─────────────────────────────────────────────┼───────────────┼───────────────┤
-│D5a. What would you say is Cuban │ 20│n/a │
-│your primary ethnic Mexican │ 311│n/a │
-│background? Spanish │ 48│n/a │
-│ South American │ 34│n/a │
-│ Central American│ 52│n/a │
-│ Puerto Rican, OR│ 78│n/a │
-│ Something else │ 68│n/a │
-│ Multiple - │ 7│n/a │
-│ cannot choose │ │ │
-│ one │ │ │
-╰─────────────────────────────────────────────┴───────────────┴───────────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES FORMAT MISSING])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /FORMAT MISSING='(no data)' /TABLE qnd5a[COLPCT] BY qnd5.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭─────────────────────────────────────────────┬───────────────────────────────╮
-│ │ D5. ETHNICITY: Are you of │
-│ │ Hispanic or Latino origin or │
-│ │ descent? │
-│ ├───────────────┬───────────────┤
-│ │ Yes │ No │
-│ ├───────────────┼───────────────┤
-│ │ Column % │ Column % │
-├─────────────────────────────────────────────┼───────────────┼───────────────┤
-│D5a. What would you say is Cuban │ 3.2%│(no data) │
-│your primary ethnic Mexican │ 50.3%│(no data) │
-│background? Spanish │ 7.8%│(no data) │
-│ South American │ 5.5%│(no data) │
-│ Central American│ 8.4%│(no data) │
-│ Puerto Rican, OR│ 12.6%│(no data) │
-│ Something else │ 11.0%│(no data) │
-│ Multiple - │ 1.1%│(no data) │
-│ cannot choose │ │ │
-│ one │ │ │
-╰─────────────────────────────────────────────┴───────────────┴───────────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES HIDESMALLCOUNTS])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /TABLE qn38[c][COUNT, COLPCT].
-CTABLES /HIDESMALLCOUNTS /TABLE qn38[c][COUNT, COLPCT].
-CTABLES /HIDESMALLCOUNTS COUNT=10 /TABLE qn38[c][COUNT, COLPCT].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭──────────────────────────────────────────────────────────────┬─────┬────────╮
-│ │Count│Column %│
-├──────────────────────────────────────────────────────────────┼─────┼────────┤
-│38. How many drinks did you have on that Less than one│ 7│ .5%│
-│occasion? 1 │ 491│ 34.9%│
-│ 2 │ 462│ 32.9%│
-│ 3 │ 229│ 16.3%│
-│ 4 │ 82│ 5.8%│
-│ 5 │ 56│ 4.0%│
-│ 6 │ 32│ 2.3%│
-│ 7 │ 9│ .6%│
-│ 8 │ 8│ .6%│
-│ 9 │ 4│ .3%│
-│ 10 │ 6│ .4%│
-│ 11 │ 2│ .1%│
-│ 12 │ 5│ .4%│
-│ 14 │ 1│ .1%│
-│ 15 │ 1│ .1%│
-│ 18 │ 1│ .1%│
-│ 20 │ 4│ .3%│
-│ 25 │ 1│ .1%│
-│ 30 │ 3│ .2%│
-│ 60 │ 1│ .1%│
-│ 99+ │ 0│ .0%│
-╰──────────────────────────────────────────────────────────────┴─────┴────────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────────────┬─────┬────────╮
-│ │Count│Column %│
-├──────────────────────────────────────────────────────────────┼─────┼────────┤
-│38. How many drinks did you have on that Less than one│ 7│ .5%│
-│occasion? 1 │ 491│ 34.9%│
-│ 2 │ 462│ 32.9%│
-│ 3 │ 229│ 16.3%│
-│ 4 │ 82│ 5.8%│
-│ 5 │ 56│ 4.0%│
-│ 6 │ 32│ 2.3%│
-│ 7 │ 9│ .6%│
-│ 8 │ 8│ .6%│
-│ 9 │<5 │ .3%│
-│ 10 │ 6│ .4%│
-│ 11 │<5 │ .1%│
-│ 12 │ 5│ .4%│
-│ 14 │<5 │ .1%│
-│ 15 │<5 │ .1%│
-│ 18 │<5 │ .1%│
-│ 20 │<5 │ .3%│
-│ 25 │<5 │ .1%│
-│ 30 │<5 │ .2%│
-│ 60 │<5 │ .1%│
-│ 99+ │<5 │ .0%│
-╰──────────────────────────────────────────────────────────────┴─────┴────────╯
-
- Custom Tables
-╭──────────────────────────────────────────────────────────────┬─────┬────────╮
-│ │Count│Column %│
-├──────────────────────────────────────────────────────────────┼─────┼────────┤
-│38. How many drinks did you have on that Less than one│<10 │ .5%│
-│occasion? 1 │ 491│ 34.9%│
-│ 2 │ 462│ 32.9%│
-│ 3 │ 229│ 16.3%│
-│ 4 │ 82│ 5.8%│
-│ 5 │ 56│ 4.0%│
-│ 6 │ 32│ 2.3%│
-│ 7 │<10 │ .6%│
-│ 8 │<10 │ .6%│
-│ 9 │<10 │ .3%│
-│ 10 │<10 │ .4%│
-│ 11 │<10 │ .1%│
-│ 12 │<10 │ .4%│
-│ 14 │<10 │ .1%│
-│ 15 │<10 │ .1%│
-│ 18 │<10 │ .1%│
-│ 20 │<10 │ .3%│
-│ 25 │<10 │ .1%│
-│ 30 │<10 │ .2%│
-│ 60 │<10 │ .1%│
-│ 99+ │<10 │ .0%│
-╰──────────────────────────────────────────────────────────────┴─────┴────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES FORMAT MINCOLWIDTH MAXCOLWIDTH])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES /FORMAT MINCOLWIDTH=1 MAXCOLWIDTH=2 UNITS=INCHES /TABLE BY qns3a.
-]])
-AT_CHECK([pspp ctables.sps -o - -O box=unicode -o pspp.spv], [0], [dnl
- Custom Tables
-╭────────────╮
-│S3a. GENDER:│
-├─────┬──────┤
-│ Male│Female│
-├─────┼──────┤
-│Count│ Count│
-├─────┼──────┤
-│ 3132│ 3867│
-╰─────┴──────╯
-])
-AT_CHECK([pspp-output get-table-look pspp.spv pspp.stt])
-AT_CHECK([sed 's/ /\n/g' pspp.stt | grep ColumnWidth | sort], [0], [dnl
-maximumColumnWidth="192"
-minimumColumnWidth="96"
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES special formats])
-AT_KEYWORDS([NEGPAREN NEQUAL PAREN PCTPAREN])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-COMPUTE x = qnd3 - 4.
-CTABLES /TABLE x[MINIMUM NEGPAREN8.1, MINIMUM NEQUAL8.1, MINIMUM PAREN8.1, MINIMUM PCTPAREN8.1, MAXIMUM NEGPAREN8.1, MAXIMUM NEQUAL8.1, MAXIMUM PAREN8.1, MAXIMUM PCTPAREN8.1].
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭─┬───────┬───────┬───────┬───────┬───────┬───────┬───────┬───────╮
-│ │Minimum│Minimum│Minimum│Minimum│Maximum│Maximum│Maximum│Maximum│
-├─┼───────┼───────┼───────┼───────┼───────┼───────┼───────┼───────┤
-│x│(3.0) │N=-3.0 │(-3.0) │(-3.0%)│8.0 │N=8.0 │(8.0) │(8.0%) │
-╰─┴───────┴───────┴───────┴───────┴───────┴───────┴───────┴───────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES TITLES])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=qn1 DISPLAY=NONE
- /TABLE ((qn1[c][COUNT])) BY qns3a[c] > qnd5
- /TITLES TITLE='How often do you drive?' 'second line of title'
- CAPTION='Generated )TIME on )DATE'
- CORNER=')TABLE'.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode | sed 's/..:..:../HH:MM:SS/
-s&../../..&MM/DD/YY&'], [0], [dnl
- How often do you drive?
- second line of title
-╭───────────────────────────────────┬─────────────────────────────────────────╮
-│ │ S3a. GENDER: │
-│ ├────────────────────┬────────────────────┤
-│ │ Male │ Female │
-│ ├────────────────────┼────────────────────┤
-│ │ D5. ETHNICITY: Are │ D5. ETHNICITY: Are │
-│ │ you of Hispanic or │ you of Hispanic or │
-│ │ Latino origin or │ Latino origin or │
-│( ( 1. How often do you usually │ descent? │ descent? │
-│drive a car or other motor ├─────────┬──────────┼─────────┬──────────┤
-│vehicle?) ) BY S3a. GENDER: > D5. │ Yes │ No │ Yes │ No │
-│ETHNICITY: Are you of Hispanic or ├─────────┼──────────┼─────────┼──────────┤
-│Latino origin or descent? │ Count │ Count │ Count │ Count │
-├───────────────────────────────────┼─────────┼──────────┼─────────┼──────────┤
-│Every day │ 218│ 2066│ 166│ 2175│
-│Several days a week │ 44│ 391│ 45│ 782│
-│Once a week or less │ 16│ 109│ 12│ 223│
-│Only certain times a year │ 15│ 41│ 11│ 61│
-│Never │ 39│ 150│ 56│ 278│
-╰───────────────────────────────────┴─────────┴──────────┴─────────┴──────────╯
-Generated HH:MM:SS on MM/DD/YY
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES area definitions])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=ALL DISPLAY=NAME
- /TABLE qn61 > qn57 BY qnd7a > qn86 + qn64b BY qns3a[TABLEID, LAYERID, SUBTABLEID]
- /SLABELS POSITION=ROW
- /TABLE qn61 > qn57 BY qnd7a > qn86 + qn64b BY qns3a[ROWID, LAYERROWID]
- /SLABELS POSITION=ROW
- /TABLE qn61 > qn57 BY qnd7a > qn86 + qn64b BY qns3a[COLID, LAYERCOLID]
- /SLABELS POSITION=ROW.
-]])
-AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-Male
-╭─────────────────────────────┬─────────────┬──────╮
-│ │ QND7A │ QN64B│
-│ ├──────┬──────┼───┬──┤
-│ │ Yes │ No │ │ │
-│ ├──────┼──────┤ │ │
-│ │ QN86 │ QN86 │ │ │
-│ ├───┬──┼───┬──┤ │ │
-│ │Yes│No│Yes│No│Yes│No│
-├─────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│
-│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
-│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Subtable ID│ 4│ 4│ 5│ 5│ 6│ 6│
-│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
-│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Subtable ID│ 4│ 4│ 5│ 5│ 6│ 6│
-╰─────────────────────────────┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-╭─────────────────────────────┬─────────────┬──────╮
-│ │ QND7A │ QN64B│
-│ ├──────┬──────┼───┬──┤
-│ │ Yes │ No │ │ │
-│ ├──────┼──────┤ │ │
-│ │ QN86 │ QN86 │ │ │
-│ ├───┬──┼───┬──┤ │ │
-│ │Yes│No│Yes│No│Yes│No│
-├─────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
-│ Subtable ID│ 7│ 7│ 8│ 8│ 9│ 9│
-│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
-│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
-│ Subtable ID│ 7│ 7│ 8│ 8│ 9│ 9│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
-│ Subtable ID│ 10│10│ 11│11│ 12│12│
-│ ╶───────────────┼───┼──┼───┼──┼───┼──┤
-│ No Table ID │ 1│ 1│ 1│ 1│ 2│ 2│
-│ Layer ID │ 3│ 3│ 3│ 3│ 4│ 4│
-│ Subtable ID│ 10│10│ 11│11│ 12│12│
-╰─────────────────────────────┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-╭──────────────────────────────┬─────────────┬──────╮
-│ │ QND7A │ QN64B│
-│ ├──────┬──────┼───┬──┤
-│ │ Yes │ No │ │ │
-│ ├──────┼──────┤ │ │
-│ │ QN86 │ QN86 │ │ │
-│ ├───┬──┼───┬──┤ │ │
-│ │Yes│No│Yes│No│Yes│No│
-├──────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│QN61 Yes QN57 Yes Row ID │ 1│ 1│ 2│ 2│ 3│ 3│
-│ Layer Row ID│ 1│ 1│ 1│ 1│ 2│ 2│
-│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Row ID │ 4│ 4│ 5│ 5│ 6│ 6│
-│ Layer Row ID│ 3│ 3│ 3│ 3│ 4│ 4│
-│ ╶─────────────────────────┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Yes Row ID │ 7│ 7│ 8│ 8│ 9│ 9│
-│ Layer Row ID│ 5│ 5│ 5│ 5│ 6│ 6│
-│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Row ID │ 10│10│ 11│11│ 12│12│
-│ Layer Row ID│ 7│ 7│ 7│ 7│ 8│ 8│
-╰──────────────────────────────┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-╭──────────────────────────────┬─────────────┬──────╮
-│ │ QND7A │ QN64B│
-│ ├──────┬──────┼───┬──┤
-│ │ Yes │ No │ │ │
-│ ├──────┼──────┤ │ │
-│ │ QN86 │ QN86 │ │ │
-│ ├───┬──┼───┬──┤ │ │
-│ │Yes│No│Yes│No│Yes│No│
-├──────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│QN61 Yes QN57 Yes Row ID │ 13│13│ 14│14│ 15│15│
-│ Layer Row ID│ 9│ 9│ 9│ 9│ 10│10│
-│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Row ID │ 16│16│ 17│17│ 18│18│
-│ Layer Row ID│ 11│11│ 11│11│ 12│12│
-│ ╶─────────────────────────┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Yes Row ID │ 19│19│ 20│20│ 21│21│
-│ Layer Row ID│ 13│13│ 13│13│ 14│14│
-│ ╶────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Row ID │ 22│22│ 23│23│ 24│24│
-│ Layer Row ID│ 15│15│ 15│15│ 16│16│
-╰──────────────────────────────┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-╭─────────────────────────────────┬─────────────┬──────╮
-│ │ QND7A │ QN64B│
-│ ├──────┬──────┼───┬──┤
-│ │ Yes │ No │ │ │
-│ ├──────┼──────┤ │ │
-│ │ QN86 │ QN86 │ │ │
-│ ├───┬──┼───┬──┤ │ │
-│ │Yes│No│Yes│No│Yes│No│
-├─────────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│QN61 Yes QN57 Yes Column ID │ 1│ 2│ 3│ 4│ 5│ 6│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
-│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Column ID │ 1│ 2│ 3│ 4│ 5│ 6│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
-│ ╶────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Yes Column ID │ 7│ 8│ 9│10│ 11│12│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
-│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Column ID │ 7│ 8│ 9│10│ 11│12│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│
-╰─────────────────────────────────┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-╭─────────────────────────────────┬─────────────┬──────╮
-│ │ QND7A │ QN64B│
-│ ├──────┬──────┼───┬──┤
-│ │ Yes │ No │ │ │
-│ ├──────┼──────┤ │ │
-│ │ QN86 │ QN86 │ │ │
-│ ├───┬──┼───┬──┤ │ │
-│ │Yes│No│Yes│No│Yes│No│
-├─────────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│QN61 Yes QN57 Yes Column ID │ 13│14│ 15│16│ 17│18│
-│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
-│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Column ID │ 13│14│ 15│16│ 17│18│
-│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
-│ ╶────────────────────────────┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Yes Column ID │ 19│20│ 21│22│ 23│24│
-│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
-│ ╶───────────────────┼───┼──┼───┼──┼───┼──┤
-│ No Column ID │ 19│20│ 21│22│ 23│24│
-│ Layer Column ID│ 7│ 8│ 9│10│ 11│12│
-╰─────────────────────────────────┴───┴──┴───┴──┴───┴──╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES area definitions with CLABELS ROWLABELS=OPPOSITE])
-AT_KEYWORDS([ROWLABELS OPPOSITE])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=ALL DISPLAY=NAME
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=OPPOSITE
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=OPPOSITE
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=OPPOSITE.
-]])
-AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-Male
-╭──────────────────────────────────┬───────────────────────────────────────────────────────╮
-│ │ QN27 │
-│ ├───────────────────────────┬───────────────────────────┤
-│ │ Yes │ No │
-│ ├───────────────────────────┼───────────────────────────┤
-│ │ QND7A │ QND7A │
-│ ├─────────────┬─────────────┼─────────────┬─────────────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├─────────────┼─────────────┼─────────────┼─────────────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│ 4│ 4│ 5│ 5│ 6│ 6│ 7│ 7│ 8│ 8│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 2│ 2│ 3│ 3│ 4│ 4│ 5│ 5│ 6│ 6│ 7│ 7│ 8│ 8│
-│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 10│10│ 11│11│ 12│12│ 13│13│ 14│14│ 15│15│ 16│16│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 10│10│ 11│11│ 12│12│ 13│13│ 14│14│ 15│15│ 16│16│
-╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-╭──────────────────────────────────┬───────────────────────────────────────────────────────╮
-│ │ QN27 │
-│ ├───────────────────────────┬───────────────────────────┤
-│ │ Yes │ No │
-│ ├───────────────────────────┼───────────────────────────┤
-│ │ QND7A │ QND7A │
-│ ├─────────────┬─────────────┼─────────────┬─────────────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├─────────────┼─────────────┼─────────────┼─────────────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 17│17│ 18│18│ 19│19│ 20│20│ 21│21│ 22│22│ 23│23│ 24│24│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 17│17│ 18│18│ 19│19│ 20│20│ 21│21│ 22│22│ 23│23│ 24│24│
-│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 25│25│ 26│26│ 27│27│ 28│28│ 29│29│ 30│30│ 31│31│ 32│32│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 25│25│ 26│26│ 27│27│ 28│28│ 29│29│ 30│30│ 31│31│ 32│32│
-╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-╭───────────────────────────────────┬───────────────────────────────────────────────────────╮
-│ │ QN27 │
-│ ├───────────────────────────┬───────────────────────────┤
-│ │ Yes │ No │
-│ ├───────────────────────────┼───────────────────────────┤
-│ │ QND7A │ QND7A │
-│ ├─────────────┬─────────────┼─────────────┬─────────────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├─────────────┼─────────────┼─────────────┼─────────────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
-├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Row ID │ 1│ 1│ 2│ 2│ 3│ 3│ 4│ 4│ 5│ 5│ 6│ 6│ 7│ 7│ 8│ 8│
-│ Layer Row ID│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 9│ 9│ 10│10│ 11│11│ 12│12│ 13│13│ 14│14│ 15│15│ 16│16│
-│ Layer Row ID│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Row ID │ 17│17│ 18│18│ 19│19│ 20│20│ 21│21│ 22│22│ 23│23│ 24│24│
-│ Layer Row ID│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 25│25│ 26│26│ 27│27│ 28│28│ 29│29│ 30│30│ 31│31│ 32│32│
-│ Layer Row ID│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
-╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-╭───────────────────────────────────┬───────────────────────────────────────────────────────╮
-│ │ QN27 │
-│ ├───────────────────────────┬───────────────────────────┤
-│ │ Yes │ No │
-│ ├───────────────────────────┼───────────────────────────┤
-│ │ QND7A │ QND7A │
-│ ├─────────────┬─────────────┼─────────────┬─────────────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├─────────────┼─────────────┼─────────────┼─────────────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
-├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Row ID │ 33│33│ 34│34│ 35│35│ 36│36│ 37│37│ 38│38│ 39│39│ 40│40│
-│ Layer Row ID│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 41│41│ 42│42│ 43│43│ 44│44│ 45│45│ 46│46│ 47│47│ 48│48│
-│ Layer Row ID│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│
-│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Row ID │ 49│49│ 50│50│ 51│51│ 52│52│ 53│53│ 54│54│ 55│55│ 56│56│
-│ Layer Row ID│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 57│57│ 58│58│ 59│59│ 60│60│ 61│61│ 62│62│ 63│63│ 64│64│
-│ Layer Row ID│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│
-╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-╭──────────────────────────────────────┬───────────────────────────────────────────────────────╮
-│ │ QN27 │
-│ ├───────────────────────────┬───────────────────────────┤
-│ │ Yes │ No │
-│ ├───────────────────────────┼───────────────────────────┤
-│ │ QND7A │ QND7A │
-│ ├─────────────┬─────────────┼─────────────┬─────────────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├─────────────┼─────────────┼─────────────┼─────────────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Column ID │ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
-│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Column ID │ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
-│ Layer Column ID│ 1│ 2│ 3│ 4│ 5│ 6│ 7│ 8│ 9│10│ 11│12│ 13│14│ 15│16│
-╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-╭──────────────────────────────────────┬───────────────────────────────────────────────────────╮
-│ │ QN27 │
-│ ├───────────────────────────┬───────────────────────────┤
-│ │ Yes │ No │
-│ ├───────────────────────────┼───────────────────────────┤
-│ │ QND7A │ QND7A │
-│ ├─────────────┬─────────────┼─────────────┬─────────────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├─────────────┼─────────────┼─────────────┼─────────────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├──────┬──────┼──────┬──────┼──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │ Yes │ No │ Yes │ No │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Column ID │ 33│34│ 35│36│ 37│38│ 39│40│ 41│42│ 43│44│ 45│46│ 47│48│
-│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 33│34│ 35│36│ 37│38│ 39│40│ 41│42│ 43│44│ 45│46│ 47│48│
-│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
-│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Column ID │ 49│50│ 51│52│ 53│54│ 55│56│ 57│58│ 59│60│ 61│62│ 63│64│
-│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 49│50│ 51│52│ 53│54│ 55│56│ 57│58│ 59│60│ 61│62│ 63│64│
-│ Layer Column ID│ 17│18│ 19│20│ 21│22│ 23│24│ 25│26│ 27│28│ 29│30│ 31│32│
-╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──┴───┴──╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES area definitions with CLABELS COLLABELS=OPPOSITE])
-AT_KEYWORDS([COLLABELS OPPOSITE])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=ALL DISPLAY=NAME
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
- /SLABELS POSITION=ROW
- /CLABELS COLLABELS=OPPOSITE
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
- /SLABELS POSITION=ROW
- /CLABELS COLLABELS=OPPOSITE
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
- /SLABELS POSITION=ROW
- /CLABELS COLLABELS=OPPOSITE.
-]])
-AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-Male
-╭──────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 2│ 2│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 2│ 2│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 3│ 3│ 4│ 4│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 3│ 3│ 4│ 4│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 5│ 5│ 6│ 6│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 5│ 5│ 6│ 6│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 7│ 7│ 8│ 8│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 7│ 7│ 8│ 8│
-│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 10│ 10│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 10│ 10│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 11│ 11│ 12│ 12│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 11│ 11│ 12│ 12│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 13│ 13│ 14│ 14│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 13│ 13│ 14│ 14│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 15│ 15│ 16│ 16│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 15│ 15│ 16│ 16│
-╰──────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-╭──────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 17│ 17│ 18│ 18│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 17│ 17│ 18│ 18│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 19│ 19│ 20│ 20│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 19│ 19│ 20│ 20│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 21│ 21│ 22│ 22│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 21│ 21│ 22│ 22│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 23│ 23│ 24│ 24│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 23│ 23│ 24│ 24│
-│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 25│ 25│ 26│ 26│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 25│ 25│ 26│ 26│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 27│ 27│ 28│ 28│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 27│ 27│ 28│ 28│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 29│ 29│ 30│ 30│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 29│ 29│ 30│ 30│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 31│ 31│ 32│ 32│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 31│ 31│ 32│ 32│
-╰──────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-╭───────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├───────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Yes Row ID │ 1│ 1│ 3│ 3│
-│ Layer Row ID│ 1│ 1│ 1│ 1│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 2│ 2│ 4│ 4│
-│ Layer Row ID│ 2│ 2│ 2│ 2│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 5│ 5│ 7│ 7│
-│ Layer Row ID│ 3│ 3│ 3│ 3│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 6│ 6│ 8│ 8│
-│ Layer Row ID│ 4│ 4│ 4│ 4│
-│ ╶─────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Row ID │ 9│ 9│ 11│ 11│
-│ Layer Row ID│ 5│ 5│ 5│ 5│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 10│ 10│ 12│ 12│
-│ Layer Row ID│ 6│ 6│ 6│ 6│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 13│ 13│ 15│ 15│
-│ Layer Row ID│ 7│ 7│ 7│ 7│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 14│ 14│ 16│ 16│
-│ Layer Row ID│ 8│ 8│ 8│ 8│
-│ ╶──────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Yes Row ID │ 17│ 17│ 19│ 19│
-│ Layer Row ID│ 9│ 9│ 9│ 9│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 18│ 18│ 20│ 20│
-│ Layer Row ID│ 10│ 10│ 10│ 10│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 21│ 21│ 23│ 23│
-│ Layer Row ID│ 11│ 11│ 11│ 11│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 22│ 22│ 24│ 24│
-│ Layer Row ID│ 12│ 12│ 12│ 12│
-│ ╶─────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Row ID │ 25│ 25│ 27│ 27│
-│ Layer Row ID│ 13│ 13│ 13│ 13│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 26│ 26│ 28│ 28│
-│ Layer Row ID│ 14│ 14│ 14│ 14│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 29│ 29│ 31│ 31│
-│ Layer Row ID│ 15│ 15│ 15│ 15│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 30│ 30│ 32│ 32│
-│ Layer Row ID│ 16│ 16│ 16│ 16│
-╰───────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-╭───────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├───────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Yes Row ID │ 33│ 33│ 35│ 35│
-│ Layer Row ID│ 17│ 17│ 17│ 17│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 34│ 34│ 36│ 36│
-│ Layer Row ID│ 18│ 18│ 18│ 18│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 37│ 37│ 39│ 39│
-│ Layer Row ID│ 19│ 19│ 19│ 19│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 38│ 38│ 40│ 40│
-│ Layer Row ID│ 20│ 20│ 20│ 20│
-│ ╶─────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Row ID │ 41│ 41│ 43│ 43│
-│ Layer Row ID│ 21│ 21│ 21│ 21│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 42│ 42│ 44│ 44│
-│ Layer Row ID│ 22│ 22│ 22│ 22│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 45│ 45│ 47│ 47│
-│ Layer Row ID│ 23│ 23│ 23│ 23│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 46│ 46│ 48│ 48│
-│ Layer Row ID│ 24│ 24│ 24│ 24│
-│ ╶──────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Yes Row ID │ 49│ 49│ 51│ 51│
-│ Layer Row ID│ 25│ 25│ 25│ 25│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 50│ 50│ 52│ 52│
-│ Layer Row ID│ 26│ 26│ 26│ 26│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 53│ 53│ 55│ 55│
-│ Layer Row ID│ 27│ 27│ 27│ 27│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 54│ 54│ 56│ 56│
-│ Layer Row ID│ 28│ 28│ 28│ 28│
-│ ╶─────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Row ID │ 57│ 57│ 59│ 59│
-│ Layer Row ID│ 29│ 29│ 29│ 29│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 58│ 58│ 60│ 60│
-│ Layer Row ID│ 30│ 30│ 30│ 30│
-│ ╶────────────────────┼────┼────┼────┼────┤
-│ No Yes Row ID │ 61│ 61│ 63│ 63│
-│ Layer Row ID│ 31│ 31│ 31│ 31│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 62│ 62│ 64│ 64│
-│ Layer Row ID│ 32│ 32│ 32│ 32│
-╰───────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-╭──────────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Yes Column ID │ 1│ 2│ 3│ 4│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 1│ 2│ 3│ 4│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 5│ 6│ 7│ 8│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 5│ 6│ 7│ 8│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶────────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Column ID │ 9│ 10│ 11│ 12│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 9│ 10│ 11│ 12│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 13│ 14│ 15│ 16│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 13│ 14│ 15│ 16│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶─────────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Yes Column ID │ 17│ 18│ 19│ 20│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 17│ 18│ 19│ 20│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 21│ 22│ 23│ 24│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 21│ 22│ 23│ 24│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶────────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Column ID │ 25│ 26│ 27│ 28│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 25│ 26│ 27│ 28│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 29│ 30│ 31│ 32│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 29│ 30│ 31│ 32│
-│ Layer Column ID│ 1│ 2│ 3│ 4│
-╰──────────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-╭──────────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Yes Column ID │ 33│ 34│ 35│ 36│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 33│ 34│ 35│ 36│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 37│ 38│ 39│ 40│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 37│ 38│ 39│ 40│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶────────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Column ID │ 41│ 42│ 43│ 44│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 41│ 42│ 43│ 44│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 45│ 46│ 47│ 48│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 45│ 46│ 47│ 48│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶─────────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Yes Column ID │ 49│ 50│ 51│ 52│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 49│ 50│ 51│ 52│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 53│ 54│ 55│ 56│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 53│ 54│ 55│ 56│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶────────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Yes Column ID │ 57│ 58│ 59│ 60│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 57│ 58│ 59│ 60│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────────┼────┼────┼────┼────┤
-│ No Yes Column ID │ 61│ 62│ 63│ 64│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 61│ 62│ 63│ 64│
-│ Layer Column ID│ 5│ 6│ 7│ 8│
-╰──────────────────────────────────────────────┴────┴────┴────┴────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES area definitions with CLABELS ROWLABELS=LAYER])
-AT_KEYWORDS([ROWLABELS LAYER])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=ALL DISPLAY=NAME
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=LAYER
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=LAYER
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=LAYER.
-]])
-AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-Male
-Yes
-╭──────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 3│ 3│ 5│ 5│ 7│ 7│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 3│ 3│ 5│ 5│ 7│ 7│
-│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 11│11│ 13│13│ 15│15│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 11│11│ 13│13│ 15│15│
-╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-No
-╭──────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 2│ 2│ 4│ 4│ 6│ 6│ 8│ 8│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 2│ 2│ 4│ 4│ 6│ 6│ 8│ 8│
-│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 10│10│ 12│12│ 14│14│ 16│16│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ Subtable ID│ 10│10│ 12│12│ 14│14│ 16│16│
-╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-Yes
-╭──────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
-│ Subtable ID│ 17│17│ 19│19│ 21│21│ 23│23│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
-│ Subtable ID│ 17│17│ 19│19│ 21│21│ 23│23│
-│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
-│ Subtable ID│ 25│25│ 27│27│ 29│29│ 31│31│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
-│ Subtable ID│ 25│25│ 27│27│ 29│29│ 31│31│
-╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-No
-╭──────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
-│ Subtable ID│ 18│18│ 20│20│ 22│22│ 24│24│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
-│ Subtable ID│ 18│18│ 20│20│ 22│22│ 24│24│
-│ ╶─────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
-│ Subtable ID│ 26│26│ 28│28│ 30│30│ 32│32│
-│ ╶────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Table ID │ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
-│ Subtable ID│ 26│26│ 28│28│ 30│30│ 32│32│
-╰──────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-Yes
-╭───────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Row ID │ 1│ 1│ 3│ 3│ 5│ 5│ 7│ 7│
-│ Layer Row ID│ 1│ 1│ 1│ 1│ 1│ 1│ 1│ 1│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 9│ 9│ 11│11│ 13│13│ 15│15│
-│ Layer Row ID│ 3│ 3│ 3│ 3│ 3│ 3│ 3│ 3│
-│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Row ID │ 17│17│ 19│19│ 21│21│ 23│23│
-│ Layer Row ID│ 5│ 5│ 5│ 5│ 5│ 5│ 5│ 5│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 25│25│ 27│27│ 29│29│ 31│31│
-│ Layer Row ID│ 7│ 7│ 7│ 7│ 7│ 7│ 7│ 7│
-╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-No
-╭───────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Row ID │ 2│ 2│ 4│ 4│ 6│ 6│ 8│ 8│
-│ Layer Row ID│ 2│ 2│ 2│ 2│ 2│ 2│ 2│ 2│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 10│10│ 12│12│ 14│14│ 16│16│
-│ Layer Row ID│ 4│ 4│ 4│ 4│ 4│ 4│ 4│ 4│
-│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Row ID │ 18│18│ 20│20│ 22│22│ 24│24│
-│ Layer Row ID│ 6│ 6│ 6│ 6│ 6│ 6│ 6│ 6│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 26│26│ 28│28│ 30│30│ 32│32│
-│ Layer Row ID│ 8│ 8│ 8│ 8│ 8│ 8│ 8│ 8│
-╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-Yes
-╭───────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Row ID │ 33│33│ 35│35│ 37│37│ 39│39│
-│ Layer Row ID│ 9│ 9│ 9│ 9│ 9│ 9│ 9│ 9│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 41│41│ 43│43│ 45│45│ 47│47│
-│ Layer Row ID│ 11│11│ 11│11│ 11│11│ 11│11│
-│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Row ID │ 49│49│ 51│51│ 53│53│ 55│55│
-│ Layer Row ID│ 13│13│ 13│13│ 13│13│ 13│13│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 57│57│ 59│59│ 61│61│ 63│63│
-│ Layer Row ID│ 15│15│ 15│15│ 15│15│ 15│15│
-╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-No
-╭───────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├───────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Row ID │ 34│34│ 36│36│ 38│38│ 40│40│
-│ Layer Row ID│ 10│10│ 10│10│ 10│10│ 10│10│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 42│42│ 44│44│ 46│46│ 48│48│
-│ Layer Row ID│ 12│12│ 12│12│ 12│12│ 12│12│
-│ ╶──────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Row ID │ 50│50│ 52│52│ 54│54│ 56│56│
-│ Layer Row ID│ 14│14│ 14│14│ 14│14│ 14│14│
-│ ╶─────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Row ID │ 58│58│ 60│60│ 62│62│ 64│64│
-│ Layer Row ID│ 16│16│ 16│16│ 16│16│ 16│16│
-╰───────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-Yes
-╭──────────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Column ID │ 1│ 3│ 5│ 7│ 9│11│ 13│15│
-│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 1│ 3│ 5│ 7│ 9│11│ 13│15│
-│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
-│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Column ID │ 17│19│ 21│23│ 25│27│ 29│31│
-│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 17│19│ 21│23│ 25│27│ 29│31│
-│ Layer Column ID│ 1│ 3│ 5│ 7│ 9│11│ 13│15│
-╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Male
-No
-╭──────────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Column ID │ 2│ 4│ 6│ 8│ 10│12│ 14│16│
-│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 2│ 4│ 6│ 8│ 10│12│ 14│16│
-│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
-│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Column ID │ 18│20│ 22│24│ 26│28│ 30│32│
-│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 18│20│ 22│24│ 26│28│ 30│32│
-│ Layer Column ID│ 2│ 4│ 6│ 8│ 10│12│ 14│16│
-╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-Yes
-╭──────────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Column ID │ 33│35│ 37│39│ 41│43│ 45│47│
-│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 33│35│ 37│39│ 41│43│ 45│47│
-│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
-│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Column ID │ 49│51│ 53│55│ 57│59│ 61│63│
-│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 49│51│ 53│55│ 57│59│ 61│63│
-│ Layer Column ID│ 17│19│ 21│23│ 25│27│ 29│31│
-╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-
- Custom Tables
-Female
-No
-╭──────────────────────────────────────┬───────────────────────────╮
-│ │ QN27 │
-│ ├─────────────┬─────────────┤
-│ │ Yes │ No │
-│ ├─────────────┼─────────────┤
-│ │ QND7A │ QND7A │
-│ ├──────┬──────┼──────┬──────┤
-│ │ Yes │ No │ Yes │ No │
-│ ├──────┼──────┼──────┼──────┤
-│ │ QN86 │ QN86 │ QN86 │ QN86 │
-│ ├───┬──┼───┬──┼───┬──┼───┬──┤
-│ │Yes│No│Yes│No│Yes│No│Yes│No│
-├──────────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│QN26 Yes QN61 Yes QN57 Column ID │ 34│36│ 38│40│ 42│44│ 46│48│
-│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 34│36│ 38│40│ 42│44│ 46│48│
-│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
-│ ╶─────────────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN61 Yes QN57 Column ID │ 50│52│ 54│56│ 58│60│ 62│64│
-│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
-│ ╶────────────────────────┼───┼──┼───┼──┼───┼──┼───┼──┤
-│ No QN57 Column ID │ 50│52│ 54│56│ 58│60│ 62│64│
-│ Layer Column ID│ 18│20│ 22│24│ 26│28│ 30│32│
-╰──────────────────────────────────────┴───┴──┴───┴──┴───┴──┴───┴──╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES area definitions with CLABELS COLLABELS=LAYER])
-AT_KEYWORDS([COLLABELS LAYER])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /VLABELS VARIABLES=ALL DISPLAY=NAME
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[TABLEID, LAYERID, SUBTABLEID]
- /SLABELS POSITION=ROW
- /CLABELS COLLABELS=LAYER
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[ROWID, LAYERROWID]
- /SLABELS POSITION=ROW
- /CLABELS COLLABELS=LAYER
- /TABLE qn26 > qn61 > qn57 BY qn27 > qnd7a > qn86 BY qns3a[COLID, LAYERCOLID]
- /SLABELS POSITION=ROW
- /CLABELS COLLABELS=LAYER.
-]])
-AT_CHECK([pspp ctables.sps --table-look="$builddir"/all-layers.stt -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-Male
-Yes
-╭──────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 3│ 3│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 1│ 1│ 3│ 3│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 5│ 5│ 7│ 7│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 5│ 5│ 7│ 7│
-│ ╶─────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 11│ 11│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 9│ 9│ 11│ 11│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 13│ 13│ 15│ 15│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 1│ 1│ 1│ 1│
-│ Subtable ID│ 13│ 13│ 15│ 15│
-╰──────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-No
-╭──────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 2│ 2│ 4│ 4│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 2│ 2│ 4│ 4│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 6│ 6│ 8│ 8│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 6│ 6│ 8│ 8│
-│ ╶─────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 10│ 10│ 12│ 12│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 10│ 10│ 12│ 12│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 14│ 14│ 16│ 16│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 2│ 2│ 2│ 2│
-│ Subtable ID│ 14│ 14│ 16│ 16│
-╰──────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-Yes
-╭──────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 17│ 17│ 19│ 19│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 17│ 17│ 19│ 19│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 21│ 21│ 23│ 23│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 21│ 21│ 23│ 23│
-│ ╶─────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 25│ 25│ 27│ 27│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 25│ 25│ 27│ 27│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 29│ 29│ 31│ 31│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 3│ 3│ 3│ 3│
-│ Subtable ID│ 29│ 29│ 31│ 31│
-╰──────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-No
-╭──────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 18│ 18│ 20│ 20│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 18│ 18│ 20│ 20│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 22│ 22│ 24│ 24│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 22│ 22│ 24│ 24│
-│ ╶─────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 26│ 26│ 28│ 28│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 26│ 26│ 28│ 28│
-│ ╶────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 30│ 30│ 32│ 32│
-│ ╶───────────────┼────┼────┼────┼────┤
-│ No Table ID │ 1│ 1│ 1│ 1│
-│ Layer ID │ 4│ 4│ 4│ 4│
-│ Subtable ID│ 30│ 30│ 32│ 32│
-╰──────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-Yes
-╭───────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├───────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Row ID │ 1│ 1│ 3│ 3│
-│ Layer Row ID│ 1│ 1│ 1│ 1│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 5│ 5│ 7│ 7│
-│ Layer Row ID│ 3│ 3│ 3│ 3│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 9│ 9│ 11│ 11│
-│ Layer Row ID│ 5│ 5│ 5│ 5│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 13│ 13│ 15│ 15│
-│ Layer Row ID│ 7│ 7│ 7│ 7│
-│ ╶──────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Row ID │ 17│ 17│ 19│ 19│
-│ Layer Row ID│ 9│ 9│ 9│ 9│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 21│ 21│ 23│ 23│
-│ Layer Row ID│ 11│ 11│ 11│ 11│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 25│ 25│ 27│ 27│
-│ Layer Row ID│ 13│ 13│ 13│ 13│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 29│ 29│ 31│ 31│
-│ Layer Row ID│ 15│ 15│ 15│ 15│
-╰───────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-No
-╭───────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├───────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Row ID │ 2│ 2│ 4│ 4│
-│ Layer Row ID│ 2│ 2│ 2│ 2│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 6│ 6│ 8│ 8│
-│ Layer Row ID│ 4│ 4│ 4│ 4│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 10│ 10│ 12│ 12│
-│ Layer Row ID│ 6│ 6│ 6│ 6│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 14│ 14│ 16│ 16│
-│ Layer Row ID│ 8│ 8│ 8│ 8│
-│ ╶──────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Row ID │ 18│ 18│ 20│ 20│
-│ Layer Row ID│ 10│ 10│ 10│ 10│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 22│ 22│ 24│ 24│
-│ Layer Row ID│ 12│ 12│ 12│ 12│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 26│ 26│ 28│ 28│
-│ Layer Row ID│ 14│ 14│ 14│ 14│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 30│ 30│ 32│ 32│
-│ Layer Row ID│ 16│ 16│ 16│ 16│
-╰───────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-Yes
-╭───────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├───────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Row ID │ 33│ 33│ 35│ 35│
-│ Layer Row ID│ 17│ 17│ 17│ 17│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 37│ 37│ 39│ 39│
-│ Layer Row ID│ 19│ 19│ 19│ 19│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 41│ 41│ 43│ 43│
-│ Layer Row ID│ 21│ 21│ 21│ 21│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 45│ 45│ 47│ 47│
-│ Layer Row ID│ 23│ 23│ 23│ 23│
-│ ╶──────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Row ID │ 49│ 49│ 51│ 51│
-│ Layer Row ID│ 25│ 25│ 25│ 25│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 53│ 53│ 55│ 55│
-│ Layer Row ID│ 27│ 27│ 27│ 27│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 57│ 57│ 59│ 59│
-│ Layer Row ID│ 29│ 29│ 29│ 29│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 61│ 61│ 63│ 63│
-│ Layer Row ID│ 31│ 31│ 31│ 31│
-╰───────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-No
-╭───────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├───────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Row ID │ 34│ 34│ 36│ 36│
-│ Layer Row ID│ 18│ 18│ 18│ 18│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 38│ 38│ 40│ 40│
-│ Layer Row ID│ 20│ 20│ 20│ 20│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 42│ 42│ 44│ 44│
-│ Layer Row ID│ 22│ 22│ 22│ 22│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 46│ 46│ 48│ 48│
-│ Layer Row ID│ 24│ 24│ 24│ 24│
-│ ╶──────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Row ID │ 50│ 50│ 52│ 52│
-│ Layer Row ID│ 26│ 26│ 26│ 26│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 54│ 54│ 56│ 56│
-│ Layer Row ID│ 28│ 28│ 28│ 28│
-│ ╶─────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Row ID │ 58│ 58│ 60│ 60│
-│ Layer Row ID│ 30│ 30│ 30│ 30│
-│ ╶────────────────┼────┼────┼────┼────┤
-│ No Row ID │ 62│ 62│ 64│ 64│
-│ Layer Row ID│ 32│ 32│ 32│ 32│
-╰───────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-Yes
-╭──────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Column ID │ 1│ 3│ 5│ 7│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 1│ 3│ 5│ 7│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 9│ 11│ 13│ 15│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 9│ 11│ 13│ 15│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Column ID │ 17│ 19│ 21│ 23│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 17│ 19│ 21│ 23│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 25│ 27│ 29│ 31│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 25│ 27│ 29│ 31│
-│ Layer Column ID│ 1│ 3│ 5│ 7│
-╰──────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Male
-No
-╭──────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Column ID │ 2│ 4│ 6│ 8│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 2│ 4│ 6│ 8│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 10│ 12│ 14│ 16│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 10│ 12│ 14│ 16│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Column ID │ 18│ 20│ 22│ 24│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 18│ 20│ 22│ 24│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 26│ 28│ 30│ 32│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 26│ 28│ 30│ 32│
-│ Layer Column ID│ 2│ 4│ 6│ 8│
-╰──────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-Yes
-╭──────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Column ID │ 33│ 35│ 37│ 39│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 33│ 35│ 37│ 39│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 41│ 43│ 45│ 47│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 41│ 43│ 45│ 47│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Column ID │ 49│ 51│ 53│ 55│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 49│ 51│ 53│ 55│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 57│ 59│ 61│ 63│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 57│ 59│ 61│ 63│
-│ Layer Column ID│ 9│ 11│ 13│ 15│
-╰──────────────────────────────────────────┴────┴────┴────┴────╯
-
- Custom Tables
-Female
-No
-╭──────────────────────────────────────────┬───────────────────╮
-│ │ QN27 │
-│ ├─────────┬─────────┤
-│ │ Yes │ No │
-│ ├─────────┼─────────┤
-│ │ QND7A │ QND7A │
-│ ├────┬────┼────┬────┤
-│ │ Yes│ No │ Yes│ No │
-│ ├────┼────┼────┼────┤
-│ │QN86│QN86│QN86│QN86│
-├──────────────────────────────────────────┼────┼────┼────┼────┤
-│QN26 Yes QN61 Yes QN57 Yes Column ID │ 34│ 36│ 38│ 40│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 34│ 36│ 38│ 40│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 42│ 44│ 46│ 48│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 42│ 44│ 46│ 48│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶─────────────────────────────────────┼────┼────┼────┼────┤
-│ No QN61 Yes QN57 Yes Column ID │ 50│ 52│ 54│ 56│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 50│ 52│ 54│ 56│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶────────────────────────────┼────┼────┼────┼────┤
-│ No QN57 Yes Column ID │ 58│ 60│ 62│ 64│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-│ ╶───────────────────┼────┼────┼────┼────┤
-│ No Column ID │ 58│ 60│ 62│ 64│
-│ Layer Column ID│ 10│ 12│ 14│ 16│
-╰──────────────────────────────────────────┴────┴────┴────┴────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES categorical summary functions])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /TABLE region BY qnd5a[COUNT, ROWPCT, ROWPCT.VALIDN, ROWPCT.TOTALN, TOTAL[COUNT, VALIDN, TOTALN]]
- /CATEGORIES VARIABLES=qnd5a TOTAL=YES MISSING=INCLUDE
- /SLABELS POSITION=ROW.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Custom Tables
-╭─────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────╮
-│ │ D5a. What would you say is your primary ethnic background? │
-│ ├─────┬───────┬───────┬──────────┬──────────┬─────────┬──────────┬──────────────┬──────┬───────┬─────┤
-│ │ │ │ │ │ │ │ │ Multiple - │ │ │ │
-│ │ │ │ │ South │ Central │ Puerto │ Something│ cannot choose│ Don't│ │ │
-│ │Cuban│Mexican│Spanish│ American │ American │Rican, OR│ else │ one │ know │Refused│Total│
-├─────────────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
-│Region NE Count │ 6│ 8│ 8│ 11│ 7│ 39│ 23│ 2│ 0│ 1│ 105│
-│ Row % │ 5.7%│ 7.6%│ 7.6%│ 10.5%│ 6.7%│ 37.1%│ 21.9%│ 1.9%│ .0%│ 1.0%│ │
-│ Row │ 5.8%│ 7.7%│ 7.7%│ 10.6%│ 6.7%│ 37.5%│ 22.1%│ 1.9%│ .0%│ .0%│ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Row │ .4%│ .6%│ .6%│ .8%│ .5%│ 2.8%│ 1.6%│ .1%│ .0%│ .1%│ │
-│ Total N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ 104│
-│ Total N│ │ │ │ │ │ │ │ │ │ │ 1409│
-│ ╶──────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
-│ MW Count │ 3│ 24│ 1│ 4│ 5│ 9│ 6│ 0│ 0│ 1│ 53│
-│ Row % │ 5.7%│ 45.3%│ 1.9%│ 7.5%│ 9.4%│ 17.0%│ 11.3%│ .0%│ .0%│ 1.9%│ │
-│ Row │ 5.8%│ 46.2%│ 1.9%│ 7.7%│ 9.6%│ 17.3%│ 11.5%│ .0%│ .0%│ .0%│ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Row │ .2%│ 1.5%│ .1%│ .2%│ .3%│ .5%│ .4%│ .0%│ .0%│ .1%│ │
-│ Total N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ 52│
-│ Total N│ │ │ │ │ │ │ │ │ │ │ 1654│
-│ ╶──────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
-│ S Count │ 10│ 113│ 11│ 14│ 25│ 23│ 20│ 2│ 3│ 2│ 223│
-│ Row % │ 4.5%│ 50.7%│ 4.9%│ 6.3%│ 11.2%│ 10.3%│ 9.0%│ .9%│ 1.3%│ .9%│ │
-│ Row │ 4.6%│ 51.8%│ 5.0%│ 6.4%│ 11.5%│ 10.6%│ 9.2%│ .9%│ .0%│ .0%│ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Row │ .4%│ 4.7%│ .5%│ .6%│ 1.0%│ 1.0%│ .8%│ .1%│ .1%│ .1%│ │
-│ Total N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ 218│
-│ Total N│ │ │ │ │ │ │ │ │ │ │ 2390│
-│ ╶──────────┼─────┼───────┼───────┼──────────┼──────────┼─────────┼──────────┼──────────────┼──────┼───────┼─────┤
-│ W Count │ 1│ 166│ 28│ 5│ 15│ 7│ 19│ 3│ 0│ 1│ 245│
-│ Row % │ .4%│ 67.8%│ 11.4%│ 2.0%│ 6.1%│ 2.9%│ 7.8%│ 1.2%│ .0%│ .4%│ │
-│ Row │ .4%│ 68.0%│ 11.5%│ 2.0%│ 6.1%│ 2.9%│ 7.8%│ 1.2%│ .0%│ .0%│ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Row │ .1%│ 10.7%│ 1.8%│ .3%│ 1.0%│ .5%│ 1.2%│ .2%│ .0%│ .1%│ │
-│ Total N│ │ │ │ │ │ │ │ │ │ │ │
-│ % │ │ │ │ │ │ │ │ │ │ │ │
-│ Valid N│ │ │ │ │ │ │ │ │ │ │ 244│
-│ Total N│ │ │ │ │ │ │ │ │ │ │ 1546│
-╰─────────────────┴─────┴───────┴───────┴──────────┴──────────┴─────────┴──────────┴──────────────┴──────┴───────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES scale summary functions])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-SET TVAR=NAME.
-
-* Use SPLIT FILE with FREQUENCIES to generate output equivalent to
- CTABLES later, to make the results easier to verify.
-SPLIT FILE BY REGION.
-FREQUENCIES
- qn19a
- /STATISTICS=MEAN SEMEAN MEDIAN MODE STDDEV VARIANCE RANGE MINIMUM MAXIMUM SUM
- /FORMAT NOTABLE /MISSING=INCLUDE.
-SPLIT FILE OFF.
-
-CTABLES
- /VLABELS VARIABLE=qn19a DISPLAY=NONE
- /TABLE region BY qn19a[VALIDN, MISSING, MEAN, SEMEAN, MEDIAN, MODE, STDDEV, VARIANCE, RANGE, MINIMUM, MAXIMUM, SUM, COUNT, TOTALN, ROWPCT.SUM]
- /CATEGORIES VARIABLES=qn19a TOTAL=YES MISSING=INCLUDE
- /SLABELS POSITION=ROW
- /CLABELS ROWLABELS=OPPOSITE.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Statistics
-╭─────────┬───────────────────────────────────╮
-│ │ REGION │
-│ ├────────┬────────┬────────┬────────┤
-│ │ NE │ MW │ S │ W │
-│ ├────────┼────────┼────────┼────────┤
-│ │ QN19A │ QN19A │ QN19A │ QN19A │
-├─────────┼────────┼────────┼────────┼────────┤
-│N Valid │ 936│ 1019│ 1276│ 950│
-│ Missing│ 473│ 635│ 1114│ 596│
-├─────────┼────────┼────────┼────────┼────────┤
-│Mean │ 19.33│ 19.83│ 20.29│ 19.87│
-├─────────┼────────┼────────┼────────┼────────┤
-│S.E. Mean│ .14│ .16│ .18│ .17│
-├─────────┼────────┼────────┼────────┼────────┤
-│Median │ 18.00│ 19.00│ 19.00│ 19.00│
-├─────────┼────────┼────────┼────────┼────────┤
-│Mode │ 18.00│ 18.00│ 18.00│ 18.00│
-├─────────┼────────┼────────┼────────┼────────┤
-│Std Dev │ 4.41│ 5.15│ 6.44│ 5.25│
-├─────────┼────────┼────────┼────────┼────────┤
-│Variance │ 19.41│ 26.47│ 41.43│ 27.59│
-├─────────┼────────┼────────┼────────┼────────┤
-│Range │ 59.00│ 71.00│ 75.00│ 61.00│
-├─────────┼────────┼────────┼────────┼────────┤
-│Minimum │ .00│ 4.00│ 4.00│ 4.00│
-├─────────┼────────┼────────┼────────┼────────┤
-│Maximum │ 59.00│ 75.00│ 79.00│ 65.00│
-├─────────┼────────┼────────┼────────┼────────┤
-│Sum │18092.00│20206.00│25886.00│18877.00│
-╰─────────┴────────┴────────┴────────┴────────╯
-
- Custom Tables
-╭────────────────────────┬────────┬────────┬────────┬────────╮
-│ │ NE │ MW │ S │ W │
-├────────────────────────┼────────┼────────┼────────┼────────┤
-│REGION Valid N │ 936│ 1019│ 1276│ 950│
-│ Missing │ 473.00│ 635.00│ 1114.00│ 596.00│
-│ Mean │ 19.33│ 19.83│ 20.29│ 19.87│
-│ Std Error of Mean│ .14│ .16│ .18│ .17│
-│ Median │ 18.00│ 19.00│ 19.00│ 19.00│
-│ Mode │ 18.00│ 18.00│ 18.00│ 18.00│
-│ Std Deviation │ 4.41│ 5.15│ 6.44│ 5.25│
-│ Variance │ 19.41│ 26.47│ 41.43│ 27.59│
-│ Range │ 59.00│ 71.00│ 75.00│ 61.00│
-│ Minimum │ .00│ 4.00│ 4.00│ 4.00│
-│ Maximum │ 59.00│ 75.00│ 79.00│ 65.00│
-│ Sum │18092.00│20206.00│25886.00│18877.00│
-│ Count │ 1409│ 1654│ 2390│ 1546│
-│ Total N │ 1409│ 1654│ 2390│ 1546│
-│ Row Sum % │ 21.8%│ 24.3%│ 31.2%│ 22.7%│
-╰────────────────────────┴────────┴────────┴────────┴────────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES scale summary functions - weighting])
-weight=1
-c=10
-for a in 1 2 9; do
- for b in 3 4 9; do
- for n in 1 2 3 4 5 6 7 8 9 10; do
- if test $c -lt 15; then
- cval=.
- else
- cval=$c
- fi
- printf "$weight $a $b $cval\n"
- weight=$(expr \( $weight + 3 \) % 7 + 2)
- c=$(expr \( $c + 13 \) % 29 + 7)
- done
- done
-done > ctables.txt
-
-AT_DATA([analysis.sps],
-[[* Use SPLIT FILE with FREQUENCIES to generate output equivalent to
- CTABLES later, to make the results easier to verify.
-SPLIT FILE BY a b.
-FREQUENCIES
- c
- /STATISTICS=MEAN SEMEAN MEDIAN MODE STDDEV VARIANCE RANGE MINIMUM MAXIMUM SUM
- /FORMAT NOTABLE /MISSING=INCLUDE.
-SPLIT FILE OFF.
-
-CTABLES
- /TABLE c[VALIDN, MISSING, MEAN F8.2, SEMEAN F8.2, MEDIAN F8.2, MODE, STDDEV F8.2, VARIANCE F8.2, RANGE F8.2, MINIMUM, MAXIMUM, SUM F8.2, COUNT, TOTALN, LAYERROWPCT.SUM] BY a>b
- /SLABELS POSITION=ROW
- /CATEGORIES VARIABLES=a b MISSING=INCLUDE.
-]])
-
-AT_DATA([ctables.sps],
-[[DATA LIST LIST NOTABLE FILE='ctables.txt'
- /w (F5.0) a b c (f2.0).
-VAR LEVEL w c (SCALE) a b (NOMINAL).
-MISSING VALUES a b (9).
-
-INCLUDE 'analysis.sps'.
-
-WEIGHT BY w.
-INCLUDE 'analysis.sps'.
-
-* Same as original analysis using unweighted versions of summaries.
-CTABLES
- /TABLE c[UVALIDN, UMISSING, UMEAN F8.2, USEMEAN F8.2, UMEDIAN F8.2, UMODE, USTDDEV F8.2, UVARIANCE F8.2, USUM F8.2, UCOUNT, UTOTALN, ULAYERROWPCT.SUM] BY a>b
- /SLABELS POSITION=ROW
- /CATEGORIES VARIABLES=a b MISSING=INCLUDE.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=120], [0], [dnl
- Statistics
-╭─────────┬──────────────────────────────────────────────────────────────╮
-│ │ a │
-│ ├────────────────────┬────────────────────┬────────────────────┤
-│ │ 1 │ 2 │ 9 │
-│ ├────────────────────┼────────────────────┼────────────────────┤
-│ │ b │ b │ b │
-│ ├──────┬──────┬──────┼──────┬──────┬──────┼──────┬──────┬──────┤
-│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
-│ ├──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│ │ c │ c │ c │ c │ c │ c │ c │ c │ c │
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│N Valid │ 7│ 6│ 8│ 7│ 7│ 8│ 7│ 7│ 8│
-│ Missing│ 3│ 4│ 2│ 3│ 3│ 2│ 3│ 3│ 2│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Mean │ 25.86│ 24.50│ 24.63│ 25.86│ 25.71│ 24.25│ 25.43│ 25.29│ 23.88│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│S.E. Mean│ 2.44│ 2.14│ 2.58│ 2.44│ 2.18│ 2.43│ 2.36│ 2.18│ 2.47│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Median │ 25.00│ 24.50│ 25.00│ 25.00│ 27.00│ 25.00│ 25.00│ 24.00│ 23.50│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Mode │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Std Dev │ 6.47│ 5.24│ 7.31│ 6.47│ 5.77│ 6.88│ 6.24│ 5.77│ 6.98│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Variance │ 41.81│ 27.50│ 53.41│ 41.81│ 33.24│ 47.36│ 38.95│ 33.24│ 48.70│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
-├─────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│Sum │181.00│147.00│197.00│181.00│180.00│194.00│178.00│177.00│191.00│
-╰─────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
-
- Custom Tables
-╭───────────────────┬──────────────────────────────────────────────────────────────╮
-│ │ a │
-│ ├────────────────────┬────────────────────┬────────────────────┤
-│ │ 1 │ 2 │ 9 │
-│ ├────────────────────┼────────────────────┼────────────────────┤
-│ │ b │ b │ b │
-│ ├──────┬──────┬──────┼──────┬──────┬──────┼──────┬──────┬──────┤
-│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
-├───────────────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│c Valid N │ 7│ 6│ 8│ 7│ 7│ 8│ 7│ 7│ 8│
-│ Missing │ 3│ 4│ 2│ 3│ 3│ 2│ 3│ 3│ 2│
-│ Mean │ 25.86│ 24.50│ 24.63│ 25.86│ 25.71│ 24.25│ 25.43│ 25.29│ 23.88│
-│ Std Error of Mean│ 2.44│ 2.14│ 2.58│ 2.44│ 2.18│ 2.43│ 2.36│ 2.18│ 2.47│
-│ Median │ 25.00│ 24.50│ 25.00│ 25.00│ 27.00│ 25.00│ 25.00│ 24.00│ 23.50│
-│ Mode │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-│ Std Deviation │ 6.47│ 5.24│ 7.31│ 6.47│ 5.77│ 6.88│ 6.24│ 5.77│ 6.98│
-│ Variance │ 41.81│ 27.50│ 53.41│ 41.81│ 33.24│ 47.36│ 38.95│ 33.24│ 48.70│
-│ Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
-│ Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-│ Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
-│ Sum │181.00│147.00│197.00│181.00│180.00│194.00│178.00│177.00│191.00│
-│ Count │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
-│ Total N │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
-│ Layer Row Sum % │ 11.1%│ 9.0%│ 12.1%│ 11.1%│ 11.1%│ 11.9%│ 10.9%│ 10.9%│ 11.7%│
-╰───────────────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
-
- Statistics
-╭─────────┬─────────────────────────────────────────────────────────────────╮
-│ │ a │
-│ ├─────────────────────┬─────────────────────┬─────────────────────┤
-│ │ 1 │ 2 │ 9 │
-│ ├─────────────────────┼─────────────────────┼─────────────────────┤
-│ │ b │ b │ b │
-│ ├───────┬──────┬──────┼──────┬───────┬──────┼──────┬──────┬───────┤
-│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
-│ ├───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│ │ c │ c │ c │ c │ c │ c │ c │ c │ c │
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│N Valid │ 40│ 35│ 41│ 26│ 38│ 40│ 34│ 32│ 39│
-│ Missing│ 6│ 14│ 11│ 22│ 13│ 7│ 16│ 21│ 10│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Mean │ 27.23│ 24.20│ 22.63│ 27.96│ 27.21│ 23.48│ 23.71│ 25.47│ 26.03│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│S.E. Mean│ .93│ .75│ 1.03│ 1.12│ .84│ .87│ 1.01│ 1.05│ 1.01│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Median │ 30.00│ 22.00│ 19.00│ 30.00│ 29.00│ 24.00│ 23.00│ 24.00│ 28.00│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Mode │ 34│ 29│ 19│ 34│ 33│ 28│ 23│ 18│ 30│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Std Dev │ 5.89│ 4.42│ 6.59│ 5.69│ 5.16│ 5.50│ 5.87│ 5.94│ 6.30│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Variance │ 34.64│ 19.52│ 43.39│ 32.36│ 26.66│ 30.20│ 34.46│ 35.29│ 39.71│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
-├─────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│Sum │1089.00│847.00│928.00│727.00│1034.00│939.00│806.00│815.00│1015.00│
-╰─────────┴───────┴──────┴──────┴──────┴───────┴──────┴──────┴──────┴───────╯
-
- Custom Tables
-╭───────────────────┬─────────────────────────────────────────────────────────────────╮
-│ │ a │
-│ ├─────────────────────┬─────────────────────┬─────────────────────┤
-│ │ 1 │ 2 │ 9 │
-│ ├─────────────────────┼─────────────────────┼─────────────────────┤
-│ │ b │ b │ b │
-│ ├───────┬──────┬──────┼──────┬───────┬──────┼──────┬──────┬───────┤
-│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
-├───────────────────┼───────┼──────┼──────┼──────┼───────┼──────┼──────┼──────┼───────┤
-│c Valid N │ 40│ 35│ 41│ 26│ 38│ 40│ 34│ 32│ 39│
-│ Missing │ 6│ 14│ 11│ 22│ 13│ 7│ 16│ 21│ 10│
-│ Mean │ 27.22│ 24.20│ 22.63│ 27.96│ 27.21│ 23.48│ 23.71│ 25.47│ 26.03│
-│ Std Error of Mean│ .93│ .75│ 1.03│ 1.12│ .84│ .87│ 1.01│ 1.05│ 1.01│
-│ Median │ 30.00│ 22.00│ 19.00│ 30.00│ 29.00│ 24.00│ 23.00│ 24.00│ 28.00│
-│ Mode │ 34│ 29│ 19│ 34│ 33│ 28│ 23│ 18│ 30│
-│ Std Deviation │ 5.89│ 4.42│ 6.59│ 5.69│ 5.16│ 5.50│ 5.87│ 5.94│ 6.30│
-│ Variance │ 34.64│ 19.52│ 43.39│ 32.36│ 26.66│ 30.20│ 34.46│ 35.29│ 39.71│
-│ Range │ 18.00│ 13.00│ 20.00│ 18.00│ 15.00│ 20.00│ 18.00│ 15.00│ 20.00│
-│ Minimum │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-│ Maximum │ 34│ 31│ 35│ 34│ 33│ 35│ 34│ 33│ 35│
-│ Sum │1089.00│847.00│928.00│727.00│1034.00│939.00│806.00│815.00│1015.00│
-│ Count │ 46│ 49│ 52│ 48│ 51│ 47│ 50│ 53│ 49│
-│ Total N │ 46│ 49│ 52│ 48│ 51│ 47│ 50│ 53│ 49│
-│ Layer Row Sum % │ 13.3%│ 10.3%│ 11.3%│ 8.9%│ 12.6%│ 11.5%│ 9.8%│ 9.9%│ 12.4%│
-╰───────────────────┴───────┴──────┴──────┴──────┴───────┴──────┴──────┴──────┴───────╯
-
- Custom Tables
-╭──────────────────────────────┬──────────────────────────────────────────────────────────────╮
-│ │ a │
-│ ├────────────────────┬────────────────────┬────────────────────┤
-│ │ 1 │ 2 │ 9 │
-│ ├────────────────────┼────────────────────┼────────────────────┤
-│ │ b │ b │ b │
-│ ├──────┬──────┬──────┼──────┬──────┬──────┼──────┬──────┬──────┤
-│ │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │ 3 │ 4 │ 9 │
-├──────────────────────────────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┼──────┤
-│c Unweighted Valid N │ 7│ 6│ 8│ 7│ 7│ 8│ 7│ 7│ 8│
-│ Unweighted Missing │ 3│ 4│ 2│ 3│ 3│ 2│ 3│ 3│ 2│
-│ Unweighted Mean │ 25.86│ 24.50│ 24.63│ 25.86│ 25.71│ 24.25│ 25.43│ 25.29│ 23.88│
-│ Unweighted Std Error of Mean│ 2.44│ 2.14│ 2.58│ 2.44│ 2.18│ 2.43│ 2.36│ 2.18│ 2.47│
-│ Unweighted Median │ 25.00│ 24.50│ 25.00│ 25.00│ 27.00│ 25.00│ 25.00│ 24.00│ 23.50│
-│ Unweighted Mode │ 16│ 18│ 15│ 16│ 18│ 15│ 16│ 18│ 15│
-│ Unweighted Std Deviation │ 6.47│ 5.24│ 7.31│ 6.47│ 5.77│ 6.88│ 6.24│ 5.77│ 6.98│
-│ Unweighted Variance │ 41.81│ 27.50│ 53.41│ 41.81│ 33.24│ 47.36│ 38.95│ 33.24│ 48.70│
-│ Unweighted Sum │181.00│147.00│197.00│181.00│180.00│194.00│178.00│177.00│191.00│
-│ Unweighted Count │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
-│ Unweighted Total N │ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│ 10│
-│ Unweighted Layer Row Sum % │ 11.1%│ 9.0%│ 12.1%│ 11.1%│ 11.1%│ 11.9%│ 10.9%│ 10.9%│ 11.7%│
-╰──────────────────────────────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────┴──────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES hidden scale VLABELS])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-CTABLES
- /TABLE region BY qn19a + qn35
- /SLABELS POSITION=ROW.
-CTABLES
- /VLABELS VARIABLE=qn19a DISPLAY=NONE
- /TABLE region BY qn19a + qn35
- /SLABELS POSITION=ROW.
-CTABLES
- /VLABELS VARIABLE=qn35 DISPLAY=NONE
- /TABLE region BY qn19a + qn35
- /SLABELS POSITION=ROW.
-
-* This one in particular caused a crash because no categories were
- created on the column axis, so passing in 0 for the index was still
- too big for that number of categories. It was fixed by creating a
- name-only category for each variable despite the "NONE" request,
- then hiding the entire dimension's labels if all its labels were
- set to "NONE".
-CTABLES
- /VLABELS VARIABLE=qn19a qn35 DISPLAY=NONE
- /TABLE region BY qn19a + qn35
- /SLABELS POSITION=ROW.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode], [0], [dnl
- Custom Tables
-╭──────────────┬────────────────────────────┬─────────────────────────────────╮
-│ │ 19a. About how old were you│ 35. In the past thirty days, how│
-│ │ when you first starting │ many times have you driven a │
-│ │ drinking alcohol, not │ motor vehicle WITHIN TWO HOURS │
-│ │ counting small tastes or │ AFTER drinking alcoholic │
-│ │ sips of alcohol. │ beverages? │
-├──────────────┼────────────────────────────┼─────────────────────────────────┤
-│Region NE Mean│ 19.33│ 2│
-│ ╶───────┼────────────────────────────┼─────────────────────────────────┤
-│ MW Mean│ 19.83│ 2│
-│ ╶───────┼────────────────────────────┼─────────────────────────────────┤
-│ S Mean│ 20.29│ 2│
-│ ╶───────┼────────────────────────────┼─────────────────────────────────┤
-│ W Mean│ 19.87│ 2│
-╰──────────────┴────────────────────────────┴─────────────────────────────────╯
-
- Custom Tables
-╭──────────────┬─────┬────────────────────────────────────────────────────────╮
-│ │ │ 35. In the past thirty days, how many times have you │
-│ │ │ driven a motor vehicle WITHIN TWO HOURS AFTER drinking │
-│ │QN19A│ alcoholic beverages? │
-├──────────────┼─────┼────────────────────────────────────────────────────────┤
-│Region NE Mean│19.33│ 2│
-│ ╶───────┼─────┼────────────────────────────────────────────────────────┤
-│ MW Mean│19.83│ 2│
-│ ╶───────┼─────┼────────────────────────────────────────────────────────┤
-│ S Mean│20.29│ 2│
-│ ╶───────┼─────┼────────────────────────────────────────────────────────┤
-│ W Mean│19.87│ 2│
-╰──────────────┴─────┴────────────────────────────────────────────────────────╯
-
- Custom Tables
-╭──────────────┬─────────────────────────────────────────────────────────┬────╮
-│ │ 19a. About how old were you when you first starting │ │
-│ │ drinking alcohol, not counting small tastes or sips of │ │
-│ │ alcohol. │qn35│
-├──────────────┼─────────────────────────────────────────────────────────┼────┤
-│Region NE Mean│ 19.33│ 2│
-│ ╶───────┼─────────────────────────────────────────────────────────┼────┤
-│ MW Mean│ 19.83│ 2│
-│ ╶───────┼─────────────────────────────────────────────────────────┼────┤
-│ S Mean│ 20.29│ 2│
-│ ╶───────┼─────────────────────────────────────────────────────────┼────┤
-│ W Mean│ 19.87│ 2│
-╰──────────────┴─────────────────────────────────────────────────────────┴────╯
-
- Custom Tables
-╭──────────────┬───────╮
-│Region NE Mean│19.33 2│
-│ ╶───────┼───────┤
-│ MW Mean│19.83 2│
-│ ╶───────┼───────┤
-│ S Mean│20.29 2│
-│ ╶───────┼───────┤
-│ W Mean│19.87 2│
-╰──────────────┴───────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES with SPLIT FILE])
-AT_CHECK([ln $top_srcdir/tests/language/stats/nhtsa.sav . || cp $top_srcdir/tests/language/stats/nhtsa.sav .])
-AT_DATA([ctables.sps],
-[[GET 'nhtsa.sav'.
-
-SORT CASES BY qns3a.
-
-CTABLES /TABLE qn105ba.
-
-* Layered split has no effect on output.
-SPLIT FILE BY qns3a.
-CTABLES /TABLE qn105ba.
-
-* Add column variable qns3a to compare against separate splits.
-CTABLES /TABLE qn105ba BY qns3a.
-
-* Separate splits are truly output separately.
-SPLIT FILE SEPARATE BY qns3a.
-CTABLES /TABLE qn105ba.
-]])
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│105b. How likely is it that drivers who have had too much Almost │ 700│
-│to drink to drive safely will A. Get stopped by the police? certain │ │
-│ Very likely │ 1502│
-│ Somewhat │ 2763│
-│ likely │ │
-│ Somewhat │ 1307│
-│ unlikely │ │
-│ Very │ 609│
-│ unlikely │ │
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│105b. How likely is it that drivers who have had too much Almost │ 700│
-│to drink to drive safely will A. Get stopped by the police? certain │ │
-│ Very likely │ 1502│
-│ Somewhat │ 2763│
-│ likely │ │
-│ Somewhat │ 1307│
-│ unlikely │ │
-│ Very │ 609│
-│ unlikely │ │
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Custom Tables
-╭─────────────────────────────────────────────────────────────────┬────────────╮
-│ │S3a. GENDER:│
-│ ├─────┬──────┤
-│ │ Male│Female│
-│ ├─────┼──────┤
-│ │Count│ Count│
-├─────────────────────────────────────────────────────────────────┼─────┼──────┤
-│105b. How likely is it that drivers who have had too Almost │ 297│ 403│
-│much to drink to drive safely will A. Get stopped by certain │ │ │
-│the police? Very likely │ 660│ 842│
-│ Somewhat │ 1174│ 1589│
-│ likely │ │ │
-│ Somewhat │ 640│ 667│
-│ unlikely │ │ │
-│ Very │ 311│ 298│
-│ unlikely │ │ │
-╰─────────────────────────────────────────────────────────────────┴─────┴──────╯
-
- Split Values
-╭────────────┬─────╮
-│Variable │Value│
-├────────────┼─────┤
-│S3a. GENDER:│Male │
-╰────────────┴─────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│105b. How likely is it that drivers who have had too much Almost │ 297│
-│to drink to drive safely will A. Get stopped by the police? certain │ │
-│ Very likely │ 660│
-│ Somewhat │ 1174│
-│ likely │ │
-│ Somewhat │ 640│
-│ unlikely │ │
-│ Very │ 311│
-│ unlikely │ │
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-
- Split Values
-╭────────────┬──────╮
-│Variable │ Value│
-├────────────┼──────┤
-│S3a. GENDER:│Female│
-╰────────────┴──────╯
-
- Custom Tables
-╭────────────────────────────────────────────────────────────────────────┬─────╮
-│ │Count│
-├────────────────────────────────────────────────────────────────────────┼─────┤
-│105b. How likely is it that drivers who have had too much Almost │ 403│
-│to drink to drive safely will A. Get stopped by the police? certain │ │
-│ Very likely │ 842│
-│ Somewhat │ 1589│
-│ likely │ │
-│ Somewhat │ 667│
-│ unlikely │ │
-│ Very │ 298│
-│ unlikely │ │
-╰────────────────────────────────────────────────────────────────────────┴─────╯
-])
-AT_CLEANUP
-
-AT_SETUP([CTABLES variable level inference])
-AT_DATA([data.txt], [dnl
-dnl n1 has 10 unique small values -> nominal.
-dnl n2 has 23 unique small values -> nominal.
-dnl n3 is all missing -> nominal.
-dnl s1 has 24 unique small values -> scale.
-dnl s2 has one negative value -> scale.
-dnl s3 has one non-integer value -> scale.
-dnl s4 has no valid values less than 10 -> scale.
-dnl s5 has no valid values less than 10,000 -> scale.
-1 1 . 1 1 1 10 10001
-2 2 . 2 2 2 11 10002
-3 3 . 3 3 3 12 10003
-4 4 . 4 4 4 13 10004
-5 5 . 5 5 5 14 10005
-6 6 . 6 6 6 15 10006
-7 7 . 7 7 7 16 10007
-8 8 . 8 8 8 17 10008
-9 9 . 9 9 9 18 10009
-10 10 . 10 10 10.5 19 110000
-1 11 . 11 -1 1 11 10001
-2 12 . 12 2 2 12 10002
-3 13 . 13 3 3 13 10003
-4 14 . 14 4 4 14 10004
-5 15 . 15 5 5 15 10005
-6 16 . 16 6 6 16 10006
-7 17 . 17 7 7 17 10007
-8 18 . 18 8 8 18 10008
-9 19 . 19 9 9 19 10009
-1 20 . 20 1 1 20 10001
-2 21 . 21 2 2 21 10002
-3 22 . 22 3 3 22 10003
-4 23 . 23 4 4 23 10004
-5 23 . 24 5 5 24 10005
-6 23 . 24 6 6 25 10006
-])
-
-AT_DATA([ctables.sps], [dnl
-DATA LIST LIST file='data.txt' NOTABLE /n1 to n3 s1 to s5.
-
-* Nominal formats (copied from data that will default to scale).
-COMPUTE n4=s1.
-COMPUTE n5=s1.
-FORMATS n4(WKDAY5) n5(MONTH5).
-
-* Scale formats (copied from data that will default to nominal).
-COMPUTE s6=n1.
-COMPUTE s7=n1.
-COMPUTE s8=n1.
-FORMATS s6(DOLLAR6.2) s7(CCA8.2) s8(DATETIME17).
-
-STRING string(A8).
-
-DISPLAY DICTIONARY.
-CTABLES /TABLE n1 + n2 + n3 + string + s1 + s2 + s3 + s4 + s5.
-DISPLAY DICTIONARY.
-])
-
-AT_CHECK([pspp ctables.sps -O box=unicode -O width=80], [0], [dnl
- Variables
-╭──────┬────────┬──────────────┬─────┬─────┬─────────┬────────────┬────────────╮
-│ │ │ Measurement │ │ │ │ │ │
-│Name │Position│ Level │ Role│Width│Alignment│Print Format│Write Format│
-├──────┼────────┼──────────────┼─────┼─────┼─────────┼────────────┼────────────┤
-│n1 │ 1│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│n2 │ 2│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│n3 │ 3│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│s1 │ 4│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│s2 │ 5│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│s3 │ 6│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│s4 │ 7│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│s5 │ 8│Unknown │Input│ 8│Right │F8.2 │F8.2 │
-│n4 │ 9│Unknown │Input│ 8│Right │WKDAY5 │WKDAY5 │
-│n5 │ 10│Unknown │Input│ 8│Right │MONTH5 │MONTH5 │
-│s6 │ 11│Unknown │Input│ 8│Right │DOLLAR6.2 │DOLLAR6.2 │
-│s7 │ 12│Unknown │Input│ 8│Right │CCA8.2 │CCA8.2 │
-│s8 │ 13│Unknown │Input│ 8│Right │DATETIME17.0│DATETIME17.0│
-│string│ 14│Nominal │Input│ 8│Left │A8 │A8 │
-╰──────┴────────┴──────────────┴─────┴─────┴─────────┴────────────┴────────────╯
-
- Custom Tables
-╭────────────┬─────┬────────╮
-│ │Count│ Mean │
-├────────────┼─────┼────────┤
-│n1 1.00 │ 3│ │
-│ 2.00 │ 3│ │
-│ 3.00 │ 3│ │
-│ 4.00 │ 3│ │
-│ 5.00 │ 3│ │
-│ 6.00 │ 3│ │
-│ 7.00 │ 2│ │
-│ 8.00 │ 2│ │
-│ 9.00 │ 2│ │
-│ 10.00│ 1│ │
-├────────────┼─────┼────────┤
-│n2 1.00 │ 1│ │
-│ 2.00 │ 1│ │
-│ 3.00 │ 1│ │
-│ 4.00 │ 1│ │
-│ 5.00 │ 1│ │
-│ 6.00 │ 1│ │
-│ 7.00 │ 1│ │
-│ 8.00 │ 1│ │
-│ 9.00 │ 1│ │
-│ 10.00│ 1│ │
-│ 11.00│ 1│ │
-│ 12.00│ 1│ │
-│ 13.00│ 1│ │
-│ 14.00│ 1│ │
-│ 15.00│ 1│ │
-│ 16.00│ 1│ │
-│ 17.00│ 1│ │
-│ 18.00│ 1│ │
-│ 19.00│ 1│ │
-│ 20.00│ 1│ │
-│ 21.00│ 1│ │
-│ 22.00│ 1│ │
-│ 23.00│ 3│ │
-├────────────┼─────┼────────┤
-│string │ 25│ │
-├────────────┼─────┼────────┤
-│s1 │ │ 12.96│
-├────────────┼─────┼────────┤
-│s2 │ │ 4.76│
-├────────────┼─────┼────────┤
-│s3 │ │ 4.86│
-├────────────┼─────┼────────┤
-│s4 │ │ 16.60│
-├────────────┼─────┼────────┤
-│s5 │ │14004.44│
-╰────────────┴─────┴────────╯
-
- Variables
-╭──────┬────────┬──────────────┬─────┬─────┬─────────┬────────────┬────────────╮
-│ │ │ Measurement │ │ │ │ │ │
-│Name │Position│ Level │ Role│Width│Alignment│Print Format│Write Format│
-├──────┼────────┼──────────────┼─────┼─────┼─────────┼────────────┼────────────┤
-│n1 │ 1│Nominal │Input│ 8│Right │F8.2 │F8.2 │
-│n2 │ 2│Nominal │Input│ 8│Right │F8.2 │F8.2 │
-│n3 │ 3│Nominal │Input│ 8│Right │F8.2 │F8.2 │
-│s1 │ 4│Scale │Input│ 8│Right │F8.2 │F8.2 │
-│s2 │ 5│Scale │Input│ 8│Right │F8.2 │F8.2 │
-│s3 │ 6│Scale │Input│ 8│Right │F8.2 │F8.2 │
-│s4 │ 7│Scale │Input│ 8│Right │F8.2 │F8.2 │
-│s5 │ 8│Scale │Input│ 8│Right │F8.2 │F8.2 │
-│n4 │ 9│Nominal │Input│ 8│Right │WKDAY5 │WKDAY5 │
-│n5 │ 10│Nominal │Input│ 8│Right │MONTH5 │MONTH5 │
-│s6 │ 11│Scale │Input│ 8│Right │DOLLAR6.2 │DOLLAR6.2 │
-│s7 │ 12│Scale │Input│ 8│Right │CCA8.2 │CCA8.2 │
-│s8 │ 13│Scale │Input│ 8│Right │DATETIME17.0│DATETIME17.0│
-│string│ 14│Nominal │Input│ 8│Left │A8 │A8 │
-╰──────┴────────┴──────────────┴─────┴─────┴─────────┴────────────┴────────────╯
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([DESCRIPTIVES procedure])
-
-AT_SETUP([DESCRIPTIVES basics])
-AT_DATA([descriptives.sps],
- [title 'Test DESCRIPTIVES procedure'.
-
-data list / V0 to V16 1-17.
-begin data.
-12128989012389023
-34128080123890128
-56127781237893217
-78127378123793112
-90913781237892318
-37978547878935789
-52878237892378279
-12377912789378932
-26787654347894348
-29137178947891888
-end data.
-
-descript all/stat=all/format=serial.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-V0,1,1-1,F1.0
-V1,1,2-2,F1.0
-V2,1,3-3,F1.0
-V3,1,4-4,F1.0
-V4,1,5-5,F1.0
-V5,1,6-6,F1.0
-V6,1,7-7,F1.0
-V7,1,8-8,F1.0
-V8,1,9-9,F1.0
-V9,1,10-10,F1.0
-V10,1,11-11,F1.0
-V11,1,12-12,F1.0
-V12,1,13-13,F1.0
-V13,1,14-14,F1.0
-V14,1,15-15,F1.0
-V15,1,16-16,F1.0
-V16,1,17-17,F1.0
-
-Table: Descriptive Statistics
-,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
-V0,10,3.80,.84,2.66,7.07,-.03,1.33,.89,.69,8.00,1,9,38.00
-V1,10,4.60,.96,3.03,9.16,-1.39,1.33,-.03,.69,9.00,0,9,46.00
-V2,10,4.10,1.16,3.67,13.43,-2.02,1.33,.48,.69,8.00,1,9,41.00
-V3,10,4.10,.87,2.77,7.66,-2.05,1.33,.42,.69,7.00,1,8,41.00
-V4,10,7.00,.47,1.49,2.22,7.15,1.33,-2.52,.69,5.00,3,8,70.00
-V5,10,4.90,1.03,3.25,10.54,-1.40,1.33,-.20,.69,9.00,0,9,49.00
-V6,10,5.90,.80,2.51,6.32,-.29,1.33,-.96,.69,7.00,1,8,59.00
-V7,10,4.70,1.10,3.47,12.01,-1.99,1.33,-.16,.69,9.00,0,9,47.00
-V8,10,4.10,1.10,3.48,12.10,-1.93,1.33,.37,.69,9.00,0,9,41.00
-V9,10,4.30,.87,2.75,7.57,-.87,1.33,.73,.69,8.00,1,9,43.00
-V10,10,5.50,.85,2.68,7.17,-1.84,1.33,-.33,.69,7.00,2,9,55.00
-V11,10,6.50,.78,2.46,6.06,-1.28,1.33,-.89,.69,6.00,3,9,65.00
-V12,10,7.90,.60,1.91,3.66,5.24,1.33,-2.21,.69,6.00,3,9,79.00
-V13,10,4.30,.99,3.13,9.79,-1.25,1.33,.33,.69,9.00,0,9,43.00
-V14,10,3.60,1.01,3.20,10.27,-.96,1.33,.81,.69,9.00,0,9,36.00
-V15,10,3.70,.92,2.91,8.46,-1.35,1.33,.71,.69,7.00,1,8,37.00
-V16,10,6.40,.91,2.88,8.27,-1.14,1.33,-.92,.69,7.00,2,9,64.00
-Valid N (listwise),10,,,,,,,,,,,,
-Missing N (listwise),0,,,,,,,,,,,,
-])
-AT_CLEANUP
-
-m4_define([DESCRIPTIVES_MISSING_DATA],
- [data list notable / V1 TO V3 1-3.
-mis val v1 to v3 (1).
-begin data.
-111
-
- 1
-1 1
-112
-123
-234
-end data.
-])
-
-AT_SETUP([DESCRIPTIVES -- excluding missing data])
-AT_DATA([descriptives.sps],
- [DESCRIPTIVES_MISSING_DATA
-descript all/stat=all/format=serial.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
-V1,1,2.00,. ,. ,. ,. ,. ,. ,. ,.00,2,2,2.00
-V2,2,2.50,.50,.71,.50,. ,. ,. ,. ,1.00,2,3,5.00
-V3,3,3.00,.58,1.00,1.00,. ,. ,.00,1.22,2.00,2,4,9.00
-Valid N (listwise),7,,,,,,,,,,,,
-Missing N (listwise),6,,,,,,,,,,,,
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES -- including missing data])
-AT_DATA([descriptives.sps],
- [DESCRIPTIVES_MISSING_DATA
-descript all/stat=all/format=serial/missing=include.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
-V1,5,1.20,.20,.45,.20,5.00,2.00,2.24,.91,1.00,1,2,6.00
-V2,5,1.60,.40,.89,.80,.31,2.00,1.26,.91,2.00,1,3,8.00
-V3,5,2.20,.58,1.30,1.70,-1.49,2.00,.54,.91,3.00,1,4,11.00
-Valid N (listwise),7,,,,,,,,,,,,
-Missing N (listwise),3,,,,,,,,,,,,
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES -- excluding missing data listwise])
-AT_DATA([descriptives.sps],
- [DESCRIPTIVES_MISSING_DATA
-descript all/stat=all/format=serial/missing=listwise.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
-V1,1,2.00,. ,. ,. ,. ,. ,. ,. ,.00,2,2,2.00
-V2,1,3.00,. ,. ,. ,. ,. ,. ,. ,.00,3,3,3.00
-V3,1,4.00,. ,. ,. ,. ,. ,. ,. ,.00,4,4,4.00
-Valid N (listwise),1,,,,,,,,,,,,
-Missing N (listwise),6,,,,,,,,,,,,
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES -- including missing data listwise])
-AT_DATA([descriptives.sps],
- [DESCRIPTIVES_MISSING_DATA
-descript all/stat=all/format=serial/missing=listwise include.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean,S.E. Mean,Std Dev,Variance,Kurtosis,S.E. Kurt,Skewness,S.E. Skew,Range,Minimum,Maximum,Sum
-V1,4,1.25,.25,.50,.25,4.00,2.62,2.00,1.01,1.00,1,2,5.00
-V2,4,1.75,.48,.96,.92,-1.29,2.62,.85,1.01,2.00,1,3,7.00
-V3,4,2.50,.65,1.29,1.67,-1.20,2.62,.00,1.01,3.00,1,4,10.00
-Valid N (listwise),4,,,,,,,,,,,,
-Missing N (listwise),3,,,,,,,,,,,,
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES bug calculating mean only])
-AT_DATA([descriptives.sps],
- [SET FORMAT F8.3.
-
-data list notable / X 1.
-begin data.
-0
-1
-2
-3
-4
-5
-end data.
-
-descript all/stat=mean.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean
-X,6,2.500
-Valid N (listwise),6,
-Missing N (listwise),0,
-])
-AT_CLEANUP
-
-dnl Git history shows that this was probably a bug in the PSPP
-dnl core regarding multipass procedures, not anything specific
-dnl to DESCRIPTIVES.
-AT_SETUP([DESCRIPTIVES bug with TEMPORARY])
-AT_DATA([descriptives.sps], [dnl
-DATA LIST LIST NOTABLE /id * abc *.
-BEGIN DATA.
-1 3.5
-2 2.0
-3 2.0
-4 3.5
-5 3.0
-6 4.0
-7 5.0
-END DATA.
-
-TEMPORARY.
-SELECT IF id < 7 .
-
-DESCRIPTIVES /VAR=abc.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-abc,6,3.00,.84,2.00,4.00
-Valid N (listwise),6,,,,
-Missing N (listwise),0,,,,
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES -- Z scores])
-AT_DATA([descriptives.sps], [dnl
-DATA LIST LIST NOTABLE /a b.
-BEGIN DATA.
-1 50
-2 60
-3 70
-END DATA.
-
-DESCRIPTIVES /VAR=a b /SAVE.
-LIST.
-])
-AT_CHECK([pspp -O format=csv descriptives.sps], [0], [dnl
-Table: Mapping of Variables to Z-scores
-Source,Target
-a,Za
-b,Zb
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-a,3,2.00,1.00,1.00,3.00
-b,3,60.00,10.00,50.00,70.00
-Valid N (listwise),3,,,,
-Missing N (listwise),0,,,,
-
-Table: Data List
-a,b,Za,Zb
-1.00,50.00,-1.00,-1.00
-2.00,60.00,.00,.00
-3.00,70.00,1.00,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES -- Z scores with SPLIT FILE])
-AT_DATA([descriptives.sps], [dnl
-DATA LIST LIST NOTABLE /group a b.
-BEGIN DATA.
-1 1 50
-1 2 60
-1 3 70
-2 100 6000
-2 200 7000
-2 400 9000
-2 500 10000
-END DATA.
-
-SPLIT FILE BY group.
-DESCRIPTIVES /VAR=a b /SAVE.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt descriptives.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Mapping of Variables to Z-scores
-Source,Target
-a,Za
-b,Zb
-
-Table: Split Values
-Variable,Value
-group,1.00
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-a,3,2.00,1.00,1.00,3.00
-b,3,60.00,10.00,50.00,70.00
-Valid N (listwise),3,,,,
-Missing N (listwise),0,,,,
-
-Table: Split Values
-Variable,Value
-group,2.00
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-a,4,300.00,182.57,100.00,500.00
-b,4,8000.00,1825.74,6000.00,10000.00
-Valid N (listwise),4,,,,
-Missing N (listwise),0,,,,
-
-Table: Split Values
-Variable,Value
-group,1.00
-
-Table: Data List
-group,a,b,Za,Zb
-1.00,1.00,50.00,-1.00,-1.00
-1.00,2.00,60.00,.00,.00
-1.00,3.00,70.00,1.00,1.00
-
-Table: Split Values
-Variable,Value
-group,2.00
-
-Table: Data List
-group,a,b,Za,Zb
-2.00,100.00,6000.00,-1.10,-1.10
-2.00,200.00,7000.00,-.55,-.55
-2.00,400.00,9000.00,.55,.55
-2.00,500.00,10000.00,1.10,1.10
-])
-AT_CLEANUP
-
-dnl Ideally DESCRIPTIVES would not make temporary transformations permanent
-dnl as it does now (bug #38786), so these results are imperfect. However,
-dnl this test does verify that DESCRIPTIVES does not crash in this situation
-dnl (as it once did).
-AT_SETUP([DESCRIPTIVES -- Z scores bug with TEMPORARY])
-AT_DATA([descriptives.sps], [dnl
-DATA LIST LIST NOTABLE /id abc.
-BEGIN DATA.
-1 3.5
-2 2.0
-3 2.0
-4 3.5
-5 3.0
-6 4.0
-7 5.0
-END DATA.
-
-TEMPORARY.
-SELECT IF id < 7 .
-
-DESCRIPTIVES /VAR=abc/SAVE.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt descriptives.sps], [0], [dnl
-descriptives.sps:15.23-15.26: warning: DESCRIPTIVES: DESCRIPTIVES with Z scores ignores TEMPORARY. Temporary transformations will be made permanent.
- 15 | DESCRIPTIVES /VAR=abc/SAVE.
- | ^~~~
-])
-AT_CHECK([cat pspp.csv], [0], [dnl
-"descriptives.sps:15.23-15.26: warning: DESCRIPTIVES: DESCRIPTIVES with Z scores ignores TEMPORARY. Temporary transformations will be made permanent.
- 15 | DESCRIPTIVES /VAR=abc/SAVE.
- | ^~~~"
-
-Table: Mapping of Variables to Z-scores
-Source,Target
-abc,Zabc
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-abc,6,3.00,.84,2.00,4.00
-Valid N (listwise),6,,,,
-Missing N (listwise),0,,,,
-
-Table: Data List
-id,abc,Zabc
-1.00,3.50,.60
-2.00,2.00,-1.20
-3.00,2.00,-1.20
-4.00,3.50,.60
-5.00,3.00,.00
-6.00,4.00,1.20
-])
-AT_CLEANUP
-
-dnl This test was supplied by Mindaugus as part of the report for bug #42012.
-AT_SETUP([DESCRIPTIVES -- Z scores with FILTER])
-AT_DATA([descriptives.sps], [dnl
-DATA LIST LIST/filter1 filter2 x.
-BEGIN DATA.
-0,0,300
-0,1,200
-0,1,100
-1,0,5
-1,0,4
-1,1,3
-1,1,2
-1,1,1
-END DATA.
-
-FILTER OFF.
-SPLIT FILE OFF.
-DESCRIPTIVES /VARIABLES=X /SAVE.
-
-FILTER BY filter1.
-SPLIT FILE OFF.
-DESCRIPTIVES /VARIABLES=X /SAVE.
-
-FILTER OFF.
-SORT CASES BY filter1.
-SPLIT FILE BY filter1.
-DESCRIPTIVES /VARIABLES=X /SAVE.
-
-FILTER BY filter2.
-SPLIT FILE BY filter1.
-DESCRIPTIVES /VARIABLES=X /SAVE.
-
-FILTER OFF.
-SORT CASES BY filter1 filter2.
-SPLIT FILE BY filter1 filter2.
-DESCRIPTIVES /VARIABLES=X /SAVE.
-EXECUTE.
-
-SPLIT FILE OFF.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv descriptives.sps])
-AT_CHECK([sed -n '/Table: Data List/,$p' < pspp.csv], [0], [dnl
-Table: Data List
-filter1,filter2,x,Zx,ZSC001,ZSC002,ZSC003,ZSC004
-.00,.00,300.00,1.94,. ,1.00,. ,. @&t@
-.00,1.00,200.00,1.07,. ,.00,.71,.71
-.00,1.00,100.00,.20,. ,-1.00,-.71,-.71
-1.00,.00,5.00,-.62,1.26,1.26,. ,.71
-1.00,.00,4.00,-.63,.63,.63,. ,-.71
-1.00,1.00,3.00,-.64,.00,.00,1.00,1.00
-1.00,1.00,2.00,-.65,-.63,-.63,.00,.00
-1.00,1.00,1.00,-.66,-1.26,-1.26,-1.00,-1.00
-])
-AT_CLEANUP
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([DESCRIPTIVES tutorial example])
-cp $top_srcdir/examples/physiology.sav .
-AT_DATA([descriptives.sps], [dnl
-GET FILE='physiology.sav'.
-DESCRIPTIVES sex, weight, height.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt descriptives.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-Sex of subject ,40,.45,.50,Male,Female
-Weight in kilograms ,40,72.12,26.70,-55.6,92.1
-Height in millimeters ,40,1677.12,262.87,179,1903
-Valid N (listwise),40,,,,
-Missing N (listwise),0,,,,
-])
-AT_CLEANUP
-
-AT_SETUP([DESCRIPTIVES syntax errors])
-AT_DATA([descriptives.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-DESCRIPTIVES MISSING=**.
-DESCRIPTIVES FORMAT=**.
-DESCRIPTIVES STATISTICS=**.
-DESCRIPTIVES SORT=**.
-DESCRIPTIVES SORT=NAME (**).
-DESCRIPTIVES SORT=NAME (A **).
-DESCRIPTIVES **.
-DESCRIPTIVES x/ **.
-DESCRIPTIVES MISSING=INCLUDE.
-TEMPORARY.
-NUMERIC Zx ZSC001 TO ZSC099 STDZ01 TO STDZ09 ZZZZ01 TO ZZZZ09 ZQZQ01 TO ZQZQ09.
-DESCRIPTIVES x/SAVE.
-])
-AT_CHECK([pspp descriptives.sps], [1], [dnl
-descriptives.sps:2.22-2.23: error: DESCRIPTIVES: Syntax error expecting
-VARIABLE, LISTWISE, or INCLUDE.
- 2 | DESCRIPTIVES MISSING=**.
- | ^~
-
-descriptives.sps:3.21-3.22: error: DESCRIPTIVES: Syntax error expecting LABELS,
-NOLABELS, INDEX, NOINDEX, LINE, or SERIAL.
- 3 | DESCRIPTIVES FORMAT=**.
- | ^~
-
-descriptives.sps:5.19-5.20: error: DESCRIPTIVES: Syntax error expecting
-variable name.
- 5 | DESCRIPTIVES SORT=**.
- | ^~
-
-descriptives.sps:6.25-6.26: error: DESCRIPTIVES: Syntax error expecting A or D.
- 6 | DESCRIPTIVES SORT=NAME (**).
- | ^~
-
-descriptives.sps:7.27-7.28: error: DESCRIPTIVES: Syntax error expecting `)'.
- 7 | DESCRIPTIVES SORT=NAME (A **).
- | ^~
-
-descriptives.sps:8.14-8.15: error: DESCRIPTIVES: Syntax error expecting
-variable name.
- 8 | DESCRIPTIVES **.
- | ^~
-
-descriptives.sps:9.17-9.18: error: DESCRIPTIVES: Syntax error expecting
-MISSING, SAVE, FORMAT, STATISTICS, SORT, or VARIABLES.
- 9 | DESCRIPTIVES x/ **.
- | ^~
-
-descriptives.sps:10: error: DESCRIPTIVES: No variables specified.
-
-descriptives.sps:13: error: DESCRIPTIVES: Ran out of generic names for Z-score
-variables. There are only 126 generic names: ZSC001-ZSC099, STDZ01-STDZ09,
-ZZZZ01-ZZZZ09, ZQZQ01-ZQZQ09.
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017, 2019 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([EXAMINE])
-
-AT_SETUP([EXAMINE])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [
-DATA LIST LIST /QUALITY * W * BRAND * .
-BEGIN DATA
-3 1 1
-2 2 1
-1 2 1
-1 1 1
-4 1 1
-4 1 1
-5 1 2
-2 1 2
-4 4 2
-2 1 2
-3 1 2
-7 1 3
-4 2 3
-5 3 3
-3 1 3
-6 1 3
-END DATA
-
-WEIGHT BY w.
-
-VARIABLE LABELS brand 'Manufacturer'.
-VARIABLE LABELS quality 'Breaking Strain'.
-
-VALUE LABELS /brand 1 'Aspeger' 2 'Bloggs' 3 'Charlies'.
-
-LIST /FORMAT=NUMBERED.
-
-EXAMINE
- quality BY brand
- /STATISTICS descriptives extreme(3)
- .
-])
-
-
-dnl In the following data, only the extreme values have been checked.
-dnl The descriptives have been blindly pasted.
-AT_CHECK([pspp -O format=csv examine.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-QUALITY,F8.0
-W,F8.0
-BRAND,F8.0
-
-Table: Data List
-Case Number,QUALITY,W,BRAND
-1,3.00,1.00,1.00
-2,2.00,2.00,1.00
-3,1.00,2.00,1.00
-4,1.00,1.00,1.00
-5,4.00,1.00,1.00
-6,4.00,1.00,1.00
-7,5.00,1.00,2.00
-8,2.00,1.00,2.00
-9,4.00,4.00,2.00
-10,2.00,1.00,2.00
-11,3.00,1.00,2.00
-12,7.00,1.00,3.00
-13,4.00,2.00,3.00
-14,5.00,3.00,3.00
-15,3.00,1.00,3.00
-16,6.00,1.00,3.00
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-Breaking Strain,24.00,100.0%,.00,.0%,24.00,100.0%
-
-Table: Extreme Values
-,,,Case Number,Value
-Breaking Strain,Highest,1,12,7.00
-,,2,16,6.00
-,,3,14,5.00
-,Lowest,1,3,1.00
-,,2,4,1.00
-,,3,2,2.00
-
-Table: Descriptives
-,,,Statistic,Std. Error
-Breaking Strain,Mean,,3.54,.32
-,95% Confidence Interval for Mean,Lower Bound,2.87,
-,,Upper Bound,4.21,
-,5% Trimmed Mean,,3.50,
-,Median,,4.00,
-,Variance,,2.52,
-,Std. Deviation,,1.59,
-,Minimum,,1.00,
-,Maximum,,7.00,
-,Range,,6.00,
-,Interquartile Range,,2.75,
-,Skewness,,.06,.47
-,Kurtosis,,-.36,.92
-
-Table: Case Processing Summary
-,Manufacturer,Cases,,,,,
-,,Valid,,Missing,,Total,
-,,N,Percent,N,Percent,N,Percent
-Breaking Strain,Aspeger,8.00,100.0%,.00,.0%,8.00,100.0%
-,Bloggs,8.00,100.0%,.00,.0%,8.00,100.0%
-,Charlies,8.00,100.0%,.00,.0%,8.00,100.0%
-
-Table: Extreme Values
-,Manufacturer,,,Case Number,Value
-Breaking Strain,Aspeger,Highest,1,6,4.00
-,,,2,5,4.00
-,,,3,1,3.00
-,,Lowest,1,3,1.00
-,,,2,4,1.00
-,,,3,2,2.00
-,Bloggs,Highest,1,7,5.00
-,,,2,9,4.00
-,,,3,11,3.00
-,,Lowest,1,8,2.00
-,,,2,10,2.00
-,,,3,11,3.00
-,Charlies,Highest,1,12,7.00
-,,,2,16,6.00
-,,,3,14,5.00
-,,Lowest,1,15,3.00
-,,,2,13,4.00
-,,,3,14,5.00
-
-Table: Descriptives
-,Manufacturer,,,Statistic,Std. Error
-Breaking Strain,Aspeger,Mean,,2.25,.45
-,,95% Confidence Interval for Mean,Lower Bound,1.18,
-,,,Upper Bound,3.32,
-,,5% Trimmed Mean,,2.22,
-,,Median,,2.00,
-,,Variance,,1.64,
-,,Std. Deviation,,1.28,
-,,Minimum,,1.00,
-,,Maximum,,4.00,
-,,Range,,3.00,
-,,Interquartile Range,,2.75,
-,,Skewness,,.47,.75
-,,Kurtosis,,-1.55,1.48
-,Bloggs,Mean,,3.50,.38
-,,95% Confidence Interval for Mean,Lower Bound,2.61,
-,,,Upper Bound,4.39,
-,,5% Trimmed Mean,,3.50,
-,,Median,,4.00,
-,,Variance,,1.14,
-,,Std. Deviation,,1.07,
-,,Minimum,,2.00,
-,,Maximum,,5.00,
-,,Range,,3.00,
-,,Interquartile Range,,1.75,
-,,Skewness,,-.47,.75
-,,Kurtosis,,-.83,1.48
-,Charlies,Mean,,4.88,.44
-,,95% Confidence Interval for Mean,Lower Bound,3.83,
-,,,Upper Bound,5.92,
-,,5% Trimmed Mean,,4.86,
-,,Median,,5.00,
-,,Variance,,1.55,
-,,Std. Deviation,,1.25,
-,,Minimum,,3.00,
-,,Maximum,,7.00,
-,,Range,,4.00,
-,,Interquartile Range,,1.75,
-,,Skewness,,.30,.75
-,,Kurtosis,,.15,1.48
-])
-
-AT_CLEANUP
-
-AT_SETUP([EXAMINE -- extremes])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-data list free /V1 W
-begin data.
-1 1
-2 1
-3 2
-3 1
-4 1
-5 1
-6 1
-7 1
-8 1
-9 1
-10 1
-11 1
-12 1
-13 1
-14 1
-15 1
-16 1
-17 1
-18 2
-19 1
-20 1
-end data.
-
-weight by w.
-
-examine v1
- /statistics=extreme(6)
- .
-])
-
-AT_CHECK([pspp -O format=csv examine.sps], [0],[dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-V1,23.00,100.0%,.00,.0%,23.00,100.0%
-
-Table: Extreme Values
-,,,Case Number,Value
-V1,Highest,1,21,20.00
-,,2,20,19.00
-,,3,19,18.00
-,,4,18,17.00
-,,5,17,16.00
-,,6,16,15.00
-,Lowest,1,1,1.00
-,,2,2,2.00
-,,3,3,3.00
-,,4,4,3.00
-,,5,5,4.00
-,,6,6,5.00
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([EXAMINE -- extremes with fractional weights])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([extreme.sps], [dnl
-set format=F20.3.
-data list notable list /w * x *.
-begin data.
- 0.88 300000
- 0.86 320000
- 0.98 480000
- 0.93 960000
- 1.35 960000
- 1.31 960000
- 0.88 960000
- 0.88 1080000
- 0.88 1080000
- 0.95 1200000
- 1.47 1200000
- 0.93 1200000
- 0.98 1320000
- 1.31 1380000
- 0.93 1440000
- 0.88 1560000
- 1.56 1560000
- 1.47 1560000
-end data.
-
-weight by w.
-
-
-EXAMINE
- x
- /STATISTICS = DESCRIPTIVES EXTREME (5)
- .
-])
-
-AT_CHECK([pspp -O format=csv extreme.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x,19.430,100.0%,.000,.0%,19.430,100.0%
-
-Table: Extreme Values
-,,,Case Number,Value
-x,Highest,1,18,1560000.000
-,,2,17,1560000.000
-,,3,16,1560000.000
-,,4,15,1440000.000
-,,5,14,1380000.000
-,Lowest,1,1,300000.000
-,,2,2,320000.000
-,,3,3,480000.000
-,,4,4,960000.000
-,,5,5,960000.000
-
-Table: Descriptives
-,,,Statistic,Std. Error
-x,Mean,,1120010.293,86222.178
-,95% Confidence Interval for Mean,Lower Bound,939166.693,
-,,Upper Bound,1300853.894,
-,5% Trimmed Mean,,1141017.899,
-,Median,,1200000.000,
-,Variance,,144447748124.869,
-,Std. Deviation,,380062.821,
-,Minimum,,300000.000,
-,Maximum,,1560000.000,
-,Range,,1260000.000,
-,Interquartile Range,,467258.065,
-,Skewness,,-.887,.519
-,Kurtosis,,.340,1.005
-])
-
-AT_CLEANUP
-
-dnl Test the PERCENTILES subcommand of the EXAMINE command.
-dnl In particular test that it behaves properly when there are only
-dnl a few cases.
-AT_SETUP([EXAMINE -- percentiles])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST /X *.
-BEGIN DATA.
-2.00
-8.00
-5.00
-END DATA.
-
-EXAMINE /x
- /PERCENTILES=HAVERAGE.
-
-EXAMINE /x
- /PERCENTILES=WAVERAGE.
-
-EXAMINE /x
- /PERCENTILES=ROUND.
-
-EXAMINE /x
- /PERCENTILES=EMPIRICAL.
-
-EXAMINE /x
- /PERCENTILES=AEMPIRICAL.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt examine.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-X,F8.0
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,3,100.0%,0,.0%,3,100.0%
-
-Table: Percentiles
-,,Percentiles,,,,,,
-,,5,10,25,50,75,90,95
-X,Weighted Average,.40,.80,2.00,5.00,8.00,8.00,8.00
-,Tukey's Hinges,,,3.50,5.00,6.50,,
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,3,100.0%,0,.0%,3,100.0%
-
-Table: Percentiles
-,,Percentiles,,,,,,
-,,5,10,25,50,75,90,95
-X,Weighted Average,.30,.60,1.50,3.50,5.75,7.10,7.55
-,Tukey's Hinges,,,3.50,5.00,6.50,,
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,3,100.0%,0,.0%,3,100.0%
-
-Table: Percentiles
-,,Percentiles,,,,,,
-,,5,10,25,50,75,90,95
-X,Weighted Average,.00,.00,2.00,5.00,5.00,8.00,8.00
-,Tukey's Hinges,,,3.50,5.00,6.50,,
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,3,100.0%,0,.0%,3,100.0%
-
-Table: Percentiles
-,,Percentiles,,,,,,
-,,5,10,25,50,75,90,95
-X,Weighted Average,2.00,2.00,2.00,5.00,8.00,8.00,8.00
-,Tukey's Hinges,,,3.50,5.00,6.50,,
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,3,100.0%,0,.0%,3,100.0%
-
-Table: Percentiles
-,,Percentiles,,,,,,
-,,5,10,25,50,75,90,95
-X,Weighted Average,2.00,2.00,2.00,5.00,8.00,8.00,8.00
-,Tukey's Hinges,,,3.50,5.00,6.50,,
-])
-AT_CLEANUP
-
-AT_SETUP([EXAMINE -- missing values])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST /x * y *.
-BEGIN DATA.
-1 1
-2 1
-3 1
-4 1
-5 2
-6 2
-. 2
-END DATA
-
-EXAMINE /x by y
- /MISSING = PAIRWISE
- .
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-y,F8.0
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x,6,85.7%,1,14.3%,7,100.0%
-
-Table: Case Processing Summary
-,y,Cases,,,,,
-,,Valid,,Missing,,Total,
-,,N,Percent,N,Percent,N,Percent
-x,1.00,4,100.0%,0,.0%,4,100.0%
-,2.00,2,66.7%,1,33.3%,3,100.0%
-])
-AT_CLEANUP
-
-
-AT_SETUP([EXAMINE -- user missing values])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine-m.sps], [dnl
-DATA LIST notable LIST /x * y *.
-BEGIN DATA.
-1 2
-9999999999 2
-9999999999 99
-END DATA.
-
-MISSING VALUES x (9999999999).
-MISSING VALUES y (99).
-
-EXAMINE
- /VARIABLES= x y
- /MISSING=PAIRWISE.
-])
-AT_CHECK([pspp -O format=csv examine-m.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x,1,33.3%,2,66.7%,3,100.0%
-y,2,66.7%,1,33.3%,3,100.0%
-])
-AT_CLEANUP
-
-AT_SETUP([EXAMINE -- missing values and percentiles])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST /X *.
-BEGIN DATA.
-99
-99
-5.00
-END DATA.
-
-MISSING VALUE X (99).
-
-EXAMINE /x
- /PERCENTILES=HAVERAGE.
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Tests the trimmed mean calculation in the case
-dnl where the data is weighted towards the centre.
-AT_SETUP([EXAMINE -- trimmed mean])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST /X * C *.
-BEGIN DATA.
-1 1
-2 49
-3 2
-END DATA.
-
-WEIGHT BY c.
-
-EXAMINE
- x
- /STATISTICS=DESCRIPTIVES
- .
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-X,F8.0
-C,F8.0
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,52.00,100.0%,.00,.0%,52.00,100.0%
-
-Table: Descriptives
-,,,Statistic,Std. Error
-X,Mean,,2.02,.03
-,95% Confidence Interval for Mean,Lower Bound,1.95,
-,,Upper Bound,2.09,
-,5% Trimmed Mean,,2.00,
-,Median,,2.00,
-,Variance,,.06,
-,Std. Deviation,,.24,
-,Minimum,,1.00,
-,Maximum,,3.00,
-,Range,,2.00,
-,Interquartile Range,,.00,
-,Skewness,,1.19,.33
-,Kurtosis,,15.73,.65
-])
-AT_CLEANUP
-
-AT_SETUP([EXAMINE -- crash bug])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-data list list /a * x * y *.
-begin data.
-3 1 3
-5 1 4
-7 2 3
-end data.
-
-examine a by x by y
- /statistics=DESCRIPTIVES
- .
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Test that two consecutive EXAMINE commands don't crash PSPP.
-AT_SETUP([EXAMINE -- consecutive runs don't crash])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-data list list /y * z *.
-begin data.
-6 4
-5 3
-7 6
-end data.
-
-EXAMINE /VARIABLES= z BY y.
-
-EXAMINE /VARIABLES= z.
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Test that /DESCRIPTIVES does not crash in presence of missing values.
-AT_SETUP([EXAMINE -- missing values don't crash])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-data list list /x * y *.
-begin data.
-1 0
-2 0
-. 0
-3 1
-4 1
-end data.
-examine x by y /statistics=descriptives.
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Test that having only a single case doesn't crash.
-AT_SETUP([EXAMINE -- single case doesn't crash])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST /quality * .
-BEGIN DATA
-3
-END DATA
-
-
-EXAMINE
- quality
- /STATISTICS descriptives
- /PLOT = histogram
- .
-])
-AT_CHECK([pspp -o pspp.csv examine.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Test that all-missing data doesn't crash.
-AT_SETUP([EXAMINE -- all-missing data doesn't crash])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST /x *.
-BEGIN DATA.
-.
-.
-.
-.
-END DATA.
-
-EXAMINE /x
- PLOT=HISTOGRAM BOXPLOT NPPLOT SPREADLEVEL(1) ALL
- /ID=x
- /STATISTICS = DESCRIPTIVES EXTREME (5) ALL
- /PERCENTILE=AEMPIRICAL
- .
-])
-AT_CHECK([pspp -o pspp.csv examine.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Test that big input doesn't crash (bug 11307).
-AT_SETUP([EXAMINE -- big input doesn't crash])
-AT_KEYWORDS([categorical categoricals slow])
-AT_DATA([examine.sps], [dnl
-INPUT PROGRAM.
- LOOP #I=1 TO 50000.
- COMPUTE X=NORMAL(10).
- END CASE.
- END LOOP.
- END FILE.
-END INPUT PROGRAM.
-
-
-EXAMINE /x
- /STATISTICS=DESCRIPTIVES.
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-dnl Another test that big input doesn't crash.
-dnl The actual bug that this checks for has been lost.
-AT_SETUP([EXAMINE -- big input doesn't crash 2])
-AT_KEYWORDS([categorical categoricals slow])
-AT_CHECK([$PYTHON3 -c '
-for i in range(100000): print("AB12")
-for i in range(100000): print("AB04")
-' > large.txt])
-AT_DATA([examine.sps], [dnl
-DATA LIST FILE='large.txt' /S 1-2 (A) X 3 .
-
-
-AGGREGATE OUTFILE=* /BREAK=X /A=N.
-
-
-EXAMINE /A BY X.
-])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CHECK([$PYTHON3 -c 'for i in range(25000): print("AB04\nAB12")' >> large.txt])
-AT_CHECK([pspp -o pspp.csv examine.sps])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-
-dnl Test that the ID command works with non-numberic variables
-AT_SETUP([EXAMINE -- non-numeric ID])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([examine-id.sps], [dnl
-data list notable list /x * y (a12).
-begin data.
-1 one
-2 two
-3 three
-4 four
-5 five
-6 six
-7 seven
-8 eight
-9 nine
-10 ten
-11 eleven
-12 twelve
-30 thirty
-300 threehundred
-end data.
-
-set small=0.
-examine x
- /statistics = extreme
- /id = y
- /plot = boxplot
- .
-])
-
-AT_CHECK([pspp -O format=csv examine-id.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x,14,100.0%,0,.0%,14,100.0%
-
-Table: Extreme Values
-,,,y,Value
-x,Highest,1,threehundred,300.00
-,,2,thirty,30.00
-,,3,twelve,12.00
-,,4,eleven,11.00
-,,5,ten,10.00
-,Lowest,1,one,1.00
-,,2,two,2.00
-,,3,three,3.00
-,,4,four,4.00
-,,5,five,5.00
-
-Table: Tests of Normality
-,Shapiro-Wilk,,
-,Statistic,df,Sig.
-x,.37,14,.00
-])
-
-AT_CLEANUP
-
-dnl Test for a crash which happened on cleanup from a bad input syntax
-AT_SETUP([EXAMINE -- Bad Input])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([examine-bad.sps], [dnl
-data list list /h * g *.
-begin data.
-1 1
-2 1
-3 1
-4 1
-5 2
-6 2
-7 2
-8 2
-9 2
-end data.
-
-EXAMINE
- /VARIABLES= h
- BY g
- /STATISTICS = DESCRIPTIVES EXTREME
- /PLOT = lkajsdas
- .
-])
-
-AT_CHECK([pspp -o pspp.csv examine-bad.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-dnl Check the MISSING=REPORT option
-AT_SETUP([EXAMINE -- MISSING=REPORT])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([examine-report.sps], [dnl
-set format = F22.0.
-data list list /x * g *.
-begin data.
-1 1
-2 1
-3 1
-4 1
-5 1
-6 1
-7 1
-8 1
-9 1
-10 2
-20 2
-30 2
-40 2
-50 2
-60 2
-70 2
-80 2
-90 2
-101 9
-201 9
-301 9
-401 9
-501 99
-601 99
-701 99
-801 99
-901 99
-1001 .
-2002 .
-3003 .
-4004 .
-end data.
-
-MISSING VALUES g (9, 99, 999).
-
-EXAMINE
- /VARIABLES = x
- BY g
- /STATISTICS = EXTREME
- /NOTOTAL
- /MISSING = REPORT.
-])
-
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt examine-report.sps])
-AT_CHECK([cat pspp.csv], [0],
- [[Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-g,F8.0
-
-Table: Case Processing Summary
-,g,Cases,,,,,
-,,Valid,,Missing,,Total,
-,,N,Percent,N,Percent,N,Percent
-x,.,4,100.0%,0,.0%,4,100.0%
-,1,9,100.0%,0,.0%,9,100.0%
-,2,9,100.0%,0,.0%,9,100.0%
-,9[a],4,100.0%,0,.0%,4,100.0%
-,99[a],5,100.0%,0,.0%,5,100.0%
-Footnote: a. User-missing value.
-
-Table: Extreme Values
-,g,,,Case Number,Value
-x,.,Highest,1,31,4004
-,,,2,30,3003
-,,,3,29,2002
-,,,4,28,1001
-,,,5,0,0
-,,Lowest,1,28,1001
-,,,2,29,2002
-,,,3,30,3003
-,,,4,31,4004
-,,,5,31,4004
-,1,Highest,1,9,9
-,,,2,8,8
-,,,3,7,7
-,,,4,6,6
-,,,5,5,5
-,,Lowest,1,1,1
-,,,2,2,2
-,,,3,3,3
-,,,4,4,4
-,,,5,5,5
-,2,Highest,1,18,90
-,,,2,17,80
-,,,3,16,70
-,,,4,15,60
-,,,5,14,50
-,,Lowest,1,10,10
-,,,2,11,20
-,,,3,12,30
-,,,4,13,40
-,,,5,14,50
-,9[a],Highest,1,22,401
-,,,2,21,301
-,,,3,20,201
-,,,4,19,101
-,,,5,0,0
-,,Lowest,1,19,101
-,,,2,20,201
-,,,3,21,301
-,,,4,22,401
-,,,5,22,401
-,99[a],Highest,1,27,901
-,,,2,26,801
-,,,3,25,701
-,,,4,24,601
-,,,5,23,501
-,,Lowest,1,23,501
-,,,2,24,601
-,,,3,25,701
-,,,4,26,801
-,,,5,27,901
-Footnote: a. User-missing value.
-]])
-
-AT_CLEANUP
-
-
-dnl Run a test of the basic STATISTICS using a "real"
-dnl dataset and comparing with "real" results kindly
-dnl provided by Olaf Nöhring
-AT_SETUP([EXAMINE -- sample unweighted])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([sample.sps], [dnl
-set format = F22.4.
-DATA LIST notable LIST /X *
-BEGIN DATA.
-461.19000000
-466.38000000
-479.46000000
-480.10000000
-483.43000000
-488.30000000
-489.00000000
-491.62000000
-505.62000000
-511.30000000
-521.53000000
-526.70000000
-528.25000000
-538.70000000
-540.22000000
-540.58000000
-546.10000000
-548.17000000
-553.99000000
-566.21000000
-575.90000000
-584.38000000
-593.40000000
-357.05000000
-359.73000000
-360.48000000
-373.98000000
-374.13000000
-381.45000000
-383.72000000
-390.00000000
-400.34000000
-415.32000000
-415.91000000
-418.30000000
-421.03000000
-422.43000000
-426.93000000
-433.25000000
-436.89000000
-445.33000000
-446.33000000
-446.55000000
-456.44000000
-689.49000000
-691.92000000
-695.00000000
-695.36000000
-698.21000000
-699.46000000
-706.61000000
-710.69000000
-715.82000000
-715.82000000
-741.39000000
-752.27000000
-756.73000000
-757.74000000
-759.57000000
-796.07000000
-813.78000000
-817.25000000
-825.48000000
-831.28000000
-849.24000000
-890.00000000
-894.78000000
-935.65000000
-935.90000000
-945.90000000
-1012.8600000
-1022.6000000
-1061.8100000
-1063.5000000
-1077.2300000
-1151.6300000
-1355.2800000
-598.88000000
-606.91000000
-621.60000000
-624.80000000
-636.13000000
-637.38000000
-640.32000000
-649.35000000
-656.51000000
-662.55000000
-664.69000000
-106.22000000
-132.24000000
-174.76000000
-204.85000000
-264.93000000
-264.99000000
-269.84000000
-325.12000000
-331.67000000
-337.26000000
-347.68000000
-354.91000000
-END DATA.
-
-EXAMINE
- x
- /STATISTICS=DESCRIPTIVES
- .
-])
-
-AT_CHECK([pspp -O format=csv sample.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-X,100,100.0%,0,.0%,100,100.0%
-
-Table: Descriptives
-,,,Statistic,Std. Error
-X,Mean,,587.6603,23.2665
-,95% Confidence Interval for Mean,Lower Bound,541.4946,
-,,Upper Bound,633.8260,
-,5% Trimmed Mean,,579.7064,
-,Median,,547.1350,
-,Variance,,54132.8466,
-,Std. Deviation,,232.6647,
-,Minimum,,106.2200,
-,Maximum,,1355.2800,
-,Range,,1249.0600,
-,Interquartile Range,,293.1575,
-,Skewness,,.6331,.2414
-,Kurtosis,,.5300,.4783
-])
-
-AT_CLEANUP
-
-
-
-dnl Test for a crash which happened on bad input syntax
-AT_SETUP([EXAMINE -- Empty Parentheses])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([examine-empty-parens.sps], [dnl
-DATA LIST notable LIST /X *
-BEGIN DATA.
-2
-3
-END DATA.
-
-
-EXAMINE
- x
- /PLOT = SPREADLEVEL()
- .
-])
-
-AT_CHECK([pspp -o pspp.csv examine-empty-parens.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-
-dnl Test for another crash which happened on bad input syntax
-AT_SETUP([EXAMINE -- Bad variable])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([examine-bad-variable.sps], [dnl
-data list list /h * g *.
-begin data.
-3 1
-4 1
-5 2
-end data.
-
-EXAMINE
- /VARIABLES/ h
- BY g
- .
-])
-
-AT_CHECK([pspp -o pspp.csv examine-bad-variable.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-dnl Test for yet another crash. This time for extremes vs. missing weight values.\0
-AT_SETUP([EXAMINE -- Extremes vs. Missing Weights])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([examine-missing-weights.sps], [dnl
-data list notable list /h * g *.
-begin data.
-3 1
-4 .
-5 1
-2 1
-end data.
-
-WEIGHT BY g.
-
-EXAMINE h
- /STATISTICS extreme(3)
- .
-])
-
-AT_CHECK([pspp -O format=csv examine-missing-weights.sps], [0], [dnl
-"examine-missing-weights.sps:13: warning: EXAMINE: At least one case in the data file had a weight value that was user-missing, system-missing, zero, or negative. These case(s) were ignored."
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-h,3.00,100.0%,.00,.0%,3.00,100.0%
-
-Table: Extreme Values
-,,,Case Number,Value
-h,Highest,1,3,5.00
-,,2,2,4.00
-,,3,1,3.00
-,Lowest,1,4,2.00
-,,2,1,3.00
-,,3,2,4.00
-])
-
-AT_CLEANUP
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([EXAMINE tutorial example 1])
-cp $top_srcdir/examples/repairs.sav .
-AT_DATA([repairs.sps], [dnl
-GET FILE='repairs.sav'.
-EXAMINE mtbf /STATISTICS=DESCRIPTIVES.
-COMPUTE mtbf_ln = LN (mtbf).
-EXAMINE mtbf_ln /STATISTICS=DESCRIPTIVES.
-])
-
-AT_CHECK([pspp -O format=csv repairs.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-Mean time between failures (months) ,30,100.0%,0,.0%,30,100.0%
-
-Table: Descriptives
-,,,Statistic,Std. Error
-Mean time between failures (months) ,Mean,,8.78,1.10
-,95% Confidence Interval for Mean,Lower Bound,6.53,
-,,Upper Bound,11.04,
-,5% Trimmed Mean,,8.20,
-,Median,,8.29,
-,Variance,,36.34,
-,Std. Deviation,,6.03,
-,Minimum,,1.63,
-,Maximum,,26.47,
-,Range,,24.84,
-,Interquartile Range,,6.03,
-,Skewness,,1.65,.43
-,Kurtosis,,3.41,.83
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-mtbf_ln,30,100.0%,0,.0%,30,100.0%
-
-Table: Descriptives
-,,,Statistic,Std. Error
-mtbf_ln,Mean,,1.95,.13
-,95% Confidence Interval for Mean,Lower Bound,1.69,
-,,Upper Bound,2.22,
-,5% Trimmed Mean,,1.96,
-,Median,,2.11,
-,Variance,,.49,
-,Std. Deviation,,.70,
-,Minimum,,.49,
-,Maximum,,3.28,
-,Range,,2.79,
-,Interquartile Range,,.88,
-,Skewness,,-.37,.43
-,Kurtosis,,.01,.83
-])
-
-AT_CLEANUP
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([EXAMINE tutorial example 2])
-cp $top_srcdir/examples/physiology.sav .
-AT_DATA([examine.sps], [dnl
-GET FILE='physiology.sav'.
-EXAMINE height, weight /STATISTICS=EXTREME(3).
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt examine.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-Height in millimeters ,40,100.0%,0,.0%,40,100.0%
-Weight in kilograms ,40,100.0%,0,.0%,40,100.0%
-
-Table: Extreme Values
-,,,Case Number,Value
-Height in millimeters ,Highest,1,14,1903
-,,2,15,1884
-,,3,12,1802
-,Lowest,1,30,179
-,,2,31,1598
-,,3,28,1601
-Weight in kilograms ,Highest,1,13,92.1
-,,2,5,92.1
-,,3,17,91.7
-,Lowest,1,38,-55.6
-,,2,39,54.5
-,,3,33,55.4
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([EXAMINE -- Crash on unrepresentable graphs])
-AT_DATA([examine.sps], [dnl
-data list notable list /x * g *.
-begin data.
-96 1
-end data.
-
-examine x by g
- /nototal
- /plot = all.
-])
-dnl This bug only manifested itself on cairo based drivers.
-AT_CHECK([pspp -O format=pdf examine.sps], [0], [ignore], [ignore])
-AT_CLEANUP
-
-
-dnl This example comes from the web site:
-dnl https://www.spsstests.com/2018/11/shapiro-wilk-normality-test-spss.html
-AT_SETUP([EXAMINE -- shapiro-wilk 1])
-AT_KEYWORDS([shapiro wilk])
-AT_DATA([shapiro-wilk.sps], [dnl
-data list notable list /x * g *.
-begin data.
-96 1
-98 1
-95 1
-89 1
-90 1
-92 1
-94 1
-93 1
-97 1
-100 1
-99 2
-96 2
-80 2
-89 2
-91 2
-92 2
-93 2
-94 2
-99 2
-80 2
-end data.
-
-set format F22.3.
-
-examine x by g
- /nototal
- /plot = all.
-])
-
-AT_CHECK([pspp -O format=csv shapiro-wilk.sps], [0],[dnl
-Table: Case Processing Summary
-,g,Cases,,,,,
-,,Valid,,Missing,,Total,
-,,N,Percent,N,Percent,N,Percent
-x,1.00,10,100.0%,0,.0%,10,100.0%
-,2.00,10,100.0%,0,.0%,10,100.0%
-
-Table: Tests of Normality
-,g,Shapiro-Wilk,,
-,,Statistic,df,Sig.
-x,1.00,.984,10,.983
-,2.00,.882,10,.136
-])
-
-AT_CLEANUP
-
-
-dnl This example comes from the web site:
-dnl http://www.real-statistics.com/tests-normality-and-symmetry/statistical-tests-normality-symmetry/shapiro-wilk-expanded-test/
-dnl It uses a dataset larger than 11 samples. Hence the alternative method for
-dnl signficance is used.
-AT_SETUP([EXAMINE -- shapiro-wilk 2])
-AT_KEYWORDS([shapiro wilk])
-AT_DATA([shapiro-wilk2.sps], [dnl
-data list notable list /x *.
-begin data.
-65
-61
-63
-86
-70
-55
-74
-35
-72
-68
-45
-58
-end data.
-
-set format F22.3.
-
-examine x
- /plot = boxplot.
-])
-
-AT_CHECK([pspp -O format=csv shapiro-wilk2.sps], [0],[dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Valid,,Missing,,Total,
-,N,Percent,N,Percent,N,Percent
-x,12,100.0%,0,.0%,12,100.0%
-
-Table: Tests of Normality
-,Shapiro-Wilk,,
-,Statistic,df,Sig.
-x,.971,12,.922
-])
-
-AT_CLEANUP
-
-AT_SETUP([EXAMINE syntax errors])
-AT_DATA([examine.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-EXAMINE VARIABLES **.
-EXAMINE **.
-EXAMINE x BY **.
-EXAMINE x/STATISTICS EXTREME (**).
-EXAMINE x/STATISTICS EXTREME (5 **).
-EXAMINE x/STATISTICS **.
-EXAMINE x/PERCENTILES(111).
-EXAMINE x/PERCENTILES(**).
-EXAMINE x/PERCENTILES **.
-EXAMINE x/MISSING **.
-EXAMINE x/COMPARE **.
-EXAMINE x/PLOT SPREADLEVEL(**).
-EXAMINE x/PLOT SPREADLEVEL(123 **).
-EXAMINE x/PLOT **.
-EXAMINE x/CINTERVAL **.
-EXAMINE x/ **.
-EXAMINE x/TOTAL/NOTOTAL.
-])
-AT_CHECK([pspp -O format=csv examine.sps], [1], [dnl
-"examine.sps:2.19-2.20: error: EXAMINE: Syntax error expecting `='.
- 2 | EXAMINE VARIABLES **.
- | ^~"
-
-"examine.sps:3.9-3.10: error: EXAMINE: Syntax error expecting variable name.
- 3 | EXAMINE **.
- | ^~"
-
-"examine.sps:4.14-4.15: error: EXAMINE: Syntax error expecting one of the following: STATISTICS, PERCENTILES, TOTAL, NOTOTAL, MISSING, COMPARE, PLOT, CINTERVAL, ID.
- 4 | EXAMINE x BY **.
- | ^~"
-
-"examine.sps:5.31-5.32: error: EXAMINE: Syntax error expecting non-negative integer for EXTREME.
- 5 | EXAMINE x/STATISTICS EXTREME (**).
- | ^~"
-
-"examine.sps:6.33-6.34: error: EXAMINE: Syntax error expecting `@:}@'.
- 6 | EXAMINE x/STATISTICS EXTREME (5 **).
- | ^~"
-
-"examine.sps:7.22-7.23: error: EXAMINE: Syntax error expecting DESCRIPTIVES, EXTREME, NONE, or ALL.
- 7 | EXAMINE x/STATISTICS **.
- | ^~"
-
-"examine.sps:8.23-8.25: error: EXAMINE: Syntax error expecting number in (0,100) for PERCENTILES.
- 8 | EXAMINE x/PERCENTILES(111).
- | ^~~"
-
-"examine.sps:9.23-9.24: error: EXAMINE: Syntax error expecting `@:}@'.
- 9 | EXAMINE x/PERCENTILES(**).
- | ^~"
-
-"examine.sps:10.23-10.24: error: EXAMINE: Syntax error expecting HAVERAGE, WAVERAGE, ROUND, EMPIRICAL, AEMPIRICAL, or NONE.
- 10 | EXAMINE x/PERCENTILES **.
- | ^~"
-
-"examine.sps:11.19-11.20: error: EXAMINE: Syntax error expecting LISTWISE, PAIRWISE, EXCLUDE, INCLUDE, REPORT, or NOREPORT.
- 11 | EXAMINE x/MISSING **.
- | ^~"
-
-"examine.sps:12.19-12.20: error: EXAMINE: Syntax error expecting VARIABLES or GROUPS.
- 12 | EXAMINE x/COMPARE **.
- | ^~"
-
-"examine.sps:13.28-13.29: error: EXAMINE: Syntax error expecting number.
- 13 | EXAMINE x/PLOT SPREADLEVEL(**).
- | ^~"
-
-"examine.sps:13.28-13.29: error: EXAMINE: Syntax error expecting BOXPLOT, NPPLOT, HISTOGRAM, SPREADLEVEL, NONE, or ALL.
- 13 | EXAMINE x/PLOT SPREADLEVEL(**).
- | ^~"
-
-"examine.sps:14.32-14.33: error: EXAMINE: Syntax error expecting `@:}@'.
- 14 | EXAMINE x/PLOT SPREADLEVEL(123 **).
- | ^~"
-
-"examine.sps:15.16-15.17: error: EXAMINE: Syntax error expecting BOXPLOT, NPPLOT, HISTOGRAM, SPREADLEVEL, NONE, or ALL.
- 15 | EXAMINE x/PLOT **.
- | ^~"
-
-"examine.sps:16.21-16.22: error: EXAMINE: Syntax error expecting number.
- 16 | EXAMINE x/CINTERVAL **.
- | ^~"
-
-"examine.sps:17.12-17.13: error: EXAMINE: Syntax error expecting one of the following: STATISTICS, PERCENTILES, TOTAL, NOTOTAL, MISSING, COMPARE, PLOT, CINTERVAL, ID.
- 17 | EXAMINE x/ **.
- | ^~"
-
-"examine.sps:18.17-18.23: error: EXAMINE: TOTAL and NOTOTAL are mutually exclusive.
- 18 | EXAMINE x/TOTAL/NOTOTAL.
- | ^~~~~~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([FACTOR procedure])
-
-AT_SETUP([FACTOR extraction=paf method=correlation])
-dnl This example is based on data from http://www.ats.ucla.edu/stat/Spss/output/factor1.htm
-
-AT_DATA([factor.sps],
- [set format = F11.3.
-
-data list notable fixed /question13 to question24 1-12.
-begin data.
-555555535543
-544453434443
-545555544444
-444442444433
-554545554554
-554455454455
-555554555244
-554455544443
-555554434344
-544454544344
-555545555555
-555454544455
-555445544455
-455544443343
-544454344344
-555555555455
-555554454455
-555555554445
-555555554555
-545534553343
-555555535554
-555544444445
-545544334433
-554555434443
-555555544454
-555445545453
-555554434244
-444444433233
-545555454443
-554443434243
-444534334333
-454534444332
-555455353444
-555544443243
-555554543243
-555544444343
-445444434443
-555555555544
-444444434340
-455044434334
-555555533433
-554554535040
-434533334232
-443232444432
-555555555555
-555555554544
-555544444445
-444224343344
-444554454355
-444434332433
-555555555555
-043243432433
-444443334333
-453443433434
-443342332232
-554434434533
-444344434443
-444444434443
-554552434133
-453334332432
-444445554444
-431232332223
-555555555544
-544445543443
-444455535543
-444444444433
-444444543243
-555431443333
-444443433433
-444433433443
-454334444433
-111111111544
-444423442433
-555443333353
-555543334344
-055454400000
-555454444355
-555555555555
-055544533333
-555554555554
-555555535554
-555555545355
-555555555455
-555544545543
-555554404455
-555454435454
-555555535554
-555555555555
-555443433544
-555554435454
-555555545344
-555555535454
-555445535453
-444444333544
-555554434444
-455454434454
-555555535555
-554545534455
-555555555443
-454443434444
-555553334444
-555554545454
-555555555555
-555554554454
-555555555555
-544545534544
-555555534454
-555555555455
-555554535544
-555555535555
-555451234443
-555444444544
-555544434354
-555545533444
-555554534443
-545554434554
-554433444433
-544432233524
-411111111111
-445423442233
-555543433344
-444443342233
-555555534455
-545442434223
-554553352333
-544554554445
-555555435455
-444334304234
-455453444434
-444443443245
-555552232132
-555434324345
-544444434344
-534344344444
-445555555344
-444343442132
-444444434344
-444444554334
-444545444333
-434442343224
-443443433233
-555551555554
-544544434444
-454544434433
-555555535434
-555555555555
-455544444444
-454444434233
-555555544344
-454445544445
-444444554434
-555455455443
-555454425444
-444454434443
-544443433233
-444543434433
-555553545354
-544444444433
-444445434433
-555533353333
-555434234333
-444314222411
-555555555555
-455545534443
-455045534433
-555545444444
-455544243543
-444421113343
-554444534444
-555555544554
-555334434452
-555544543455
-555554434554
-555445554454
-555555545344
-555555555555
-554543334245
-554441233333
-555554334545
-555555535544
-555555554554
-555445444543
-444424544432
-444425544333
-555434344443
-555533233223
-544433442343
-555555555555
-555445452234
-555444555444
-555444544455
-555544354554
-545445555555
-555555555555
-455443342232
-555555544454
-455534443455
-555555555055
-444554333244
-444445434543
-555554544455
-444443444434
-444444434445
-334231314323
-444444434433
-555554444443
-444444434443
-555455434444
-444444444444
-555455435455
-444444434344
-444543433232
-544443334454
-555544433244
-534443324224
-555555544444
-444443434444
-555553444344
-443434443333
-444444434333
-554445444343
-444443344434
-445555544543
-555554443443
-322232403322
-444444444433
-444445444443
-444454534445
-544344444344
-554445534544
-555555555554
-555544432333
-444444444443
-555555555554
-455555555554
-555555555543
-555555555544
-555554445555
-444335544455
-555555555555
-555454444454
-555455544454
-444445444444
-555555555555
-555455454554
-555454544554
-555555555455
-555555554444
-444444544434
-334334443333
-422224222211
-121512011111
-443444432332
-434335533344
-444443543433
-554454443453
-555555545454
-555555555344
-555555555554
-555555334555
-454445554444
-454545534444
-555554555444
-455444534455
-555544554544
-444444444444
-555424423133
-222433443224
-544544532344
-545554544544
-555532544144
-444432343433
-434545554545
-444344505443
-555555555544
-555554555344
-444531113112
-555554333133
-455433233233
-354354434345
-454534433433
-433112332321
-555445433333
-454343434143
-555554445555
-455423334322
-443333332222
-444443444443
-553432224134
-444223432233
-454324425444
-344434514443
-555552415255
-354332543353
-444531333233
-443433334133
-444444444433
-444444445533
-445133212223
-343433432213
-442333333332
-345455535244
-443211121122
-555445444444
-555555544344
-453243332232
-443543422533
-554444434344
-554444333444
-555555335343
-444231121133
-454433332233
-455524434244
-554433224354
-555455555444
-454444424444
-554242232134
-555553315443
-555553333454
-455421434211
-111111151111
-454443203123
-454243334132
-355332213144
-554534334134
-555543434355
-555543433255
-444441223334
-555443143255
-444444444454
-444442323544
-454443434343
-445453434444
-454455554433
-554532342234
-444442243233
-444442343334
-555443433444
-555543253444
-554554444455
-555543355344
-555444445444
-434443204222
-444432444234
-222142441111
-111111141111
-334334454433
-555354552543
-433411122112
-455534504444
-543211224233
-344333332532
-344443233333
-444424432434
-555555544454
-555555554544
-555444444444
-555443533443
-555554544344
-555554444444
-544543433343
-444445434444
-555555555555
-555443333233
-545444534454
-555454434343
-555453434544
-544334444333
-555443443444
-555555534544
-444444444433
-555543433343
-555444445334
-555543442433
-455444334443
-554443434443
-555254500544
-555444433344
-555555535544
-444443532232
-544443433433
-555555535544
-555455454444
-555455445544
-544444534433
-555555555554
-555553533444
-555555555554
-555554434343
-555455544443
-555554444443
-555454444445
-333222333223
-444443544233
-543443534433
-555545454533
-444444404033
-455454504543
-455555555454
-454443344343
-555553435244
-555543544444
-555553343044
-444443444433
-445543434434
-555554433545
-555554455445
-455553333234
-555552344243
-444141212213
-444443504234
-445544253444
-555554354555
-534552234543
-555554544544
-444233404224
-555554534444
-444443444344
-422442434324
-554434434344
-444444434334
-443433432444
-555454435344
-252423332214
-454544434434
-444444444244
-555554544445
-443222432333
-555544444332
-545555543445
-554544334444
-555445555544
-554343434433
-555555535554
-444554444333
-553544343534
-555553254433
-555555554554
-544443443344
-555443444344
-544432144123
-555555555555
-555555555354
-555555535545
-455454434444
-455455534445
-455555554455
-555553545445
-445545544444
-545345553555
-444445444433
-443435433433
-555554545353
-444443232223
-444444434433
-555555545454
-555554444343
-555554444443
-434544333422
-444443434343
-555555455443
-555544434343
-455545534444
-555555555553
-444443343434
-555555555445
-433444444434
-344221512132
-333421132223
-333444443444
-435544545533
-444335544443
-333323223323
-334434332333
-444422323213
-324433443423
-555555554555
-555452424444
-555544434444
-555544443343
-555445545453
-555555344453
-555544545550
-454443434334
-444332233344
-555554332344
-554444433444
-555554555554
-555555543544
-444442443132
-555555545555
-544553435533
-044044444444
-444443434533
-555454433434
-555555555454
-555555555555
-454544434444
-444444434333
-554555455554
-555555543444
-423331124132
-445445534455
-555555555554
-444535554434
-555555555554
-354443422232
-555545535443
-555555555555
-555455444453
-455434443333
-455444443233
-343322434411
-555555555534
-453442432333
-445554434544
-345444443333
-454554554355
-444434453434
-555555454443
-555443343341
-545553343433
-222343222201
-444433434443
-535555555544
-552541442423
-545433343334
-455445433443
-444444433433
-455543343433
-434444432333
-444545444444
-555554534444
-452444434433
-454443432533
-555453334433
-444442233432
-555555553433
-545555545445
-335543222333
-554554344445
-243424442212
-133222432411
-343434534233
-222222212211
-445455554434
-455554453344
-223334424434
-555355555544
-455544554434
-544455555444
-444444444444
-444444444344
-444334444444
-444444444434
-555455534243
-555555545555
-555555555555
-444443534343
-555554544444
-555555555454
-444434433433
-455445544444
-555544544455
-444333433333
-455443445343
-444432442444
-443334451543
-555554455555
-555444444444
-555555555455
-555555555554
-555454345154
-555555555555
-555555555455
-555554454544
-454444454544
-554443334544
-455555402535
-555554531534
-455545544554
-444423444223
-555444334533
-455554445243
-453444334344
-555555555554
-555555555455
-555555555554
-455443334344
-555555253555
-555554433454
-555444344455
-550030034433
-555444233343
-555343222133
-555555555554
-555555243243
-555555555355
-555554345555
-555443434454
-545543133133
-555443334154
-444444344454
-555555242254
-555554444344
-545443334454
-554444132454
-555455143154
-555554453044
-555555555455
-555534335454
-555555245555
-545543333444
-555555355454
-555353145133
-555553043454
-555555354554
-555543434454
-555444324454
-444444203443
-555552233355
-555555445455
-555500034354
-555354354444
-555555555555
-555543334144
-555555005254
-454444344254
-555555555554
-555555555255
-555555555455
-444444242243
-555554445154
-444444234333
-555555553455
-444422224243
-555545443344
-545552133143
-555455044344
-555555455555
-555555545454
-454433343144
-555555555555
-555455255155
-555555454455
-555555424455
-555555355555
-555444444455
-555555454455
-544411211314
-223322441123
-444223434233
-444441121232
-555555535555
-454445533444
-434442434433
-545355554454
-555542544333
-445545555444
-321000001011
-444444444444
-333223321322
-444232433233
-425432523122
-455555555544
-555555545555
-555555534354
-554554444243
-555554343443
-444443434333
-555224252443
-555544433433
-344544434423
-555554434344
-555542134233
-444344434444
-445443433233
-455343333434
-455443333445
-355344434433
-234234433333
-445444444445
-455535535545
-443423234443
-455544334544
-345441333323
-445444533433
-455554443355
-445444433243
-455454453444
-444244444444
-554244544154
-555555555555
-454343444444
-555444444344
-545455534454
-555555555555
-555554534454
-555254555444
-544354544453
-553454534445
-555454445355
-545253554454
-433342322233
-544444444443
-555455445544
-424322433233
-444424222233
-555355555544
-000000000000
-555455345344
-000055505450
-434333444333
-444445444444
-555555555545
-555554455545
-555455435533
-544544444243
-444331232323
-555555554444
-443332323233
-444224342433
-555555555544
-433443342333
-555445504554
-555545555544
-555555535555
-555555535345
-454443433333
-555444444555
-444443232435
-555433444443
-555443434543
-555555535555
-555555533544
-555543234444
-433432332221
-443433544233
-443443333334
-444424433444
-552444333123
-233332232211
-223422221122
-434433414133
-332323333102
-552544223222
-542423343232
-555552534132
-455554544134
-433523533132
-433333433433
-435434543333
-434553433444
-555544434345
-443543443433
-555555542344
-544444553144
-555544544243
-535443441342
-344555444333
-444444443333
-443443433432
-545554534544
-443533433433
-333443432223
-333333433123
-322432122213
-555555555555
-554555444433
-444543443234
-444444433433
-452555534433
-244444444233
-433442422232
-555555535555
-555555554455
-555554545244
-555555355455
-555555555555
-555555555555
-444433323233
-555455455455
-434445444433
-555554444455
-454444543445
-555555535455
-555545554455
-555555555555
-555555555555
-555443344353
-455543304132
-444443444433
-555555555355
-434433443333
-444434444433
-444444444433
-555555555555
-445553443323
-445444444444
-555554444054
-455555554543
-555555535555
-555554445454
-555444444443
-555555555454
-555554344455
-555555454454
-554444444454
-555555555353
-555545544454
-553545332223
-444424444332
-545555543433
-555444433444
-555555444445
-444444424433
-444444543434
-124113531311
-555554534543
-555343333333
-545444544344
-555444534444
-555544543444
-555455544443
-444324402121
-555554534544
-555455544444
-555555544454
-444334404433
-555443534444
-555545554444
-555555555555
-455333233433
-455444433433
-455444444444
-444235442443
-555443343433
-444445453444
-454345453432
-555555453444
-344433322323
-444443444244
-444442343133
-554445432233
-555555544444
-555555534444
-555554455554
-454443334244
-544443333233
-444445534445
-444432134121
-555555332243
-555555544433
-555554434444
-454543534233
-454432432343
-444424432433
-545553335344
-555443434344
-454443433333
-555553534444
-554544434355
-552532421235
-454543433434
-544544343234
-555552334125
-555543455555
-443442334222
-554443444344
-555554543334
-555552342444
-554443433333
-443444434445
-555554533344
-442412242121
-454543343244
-445554433344
-444443333433
-444443433333
-444433333334
-444444334444
-432321102223
-444444434443
-444444434343
-454531432331
-445543433434
-554554434554
-334253232333
-444443434244
-444433443234
-444433334334
-444443433333
-553434303222
-454443434244
-444445544444
-443441133433
-444432232133
-444444404344
-444444333243
-455543124243
-555544532344
-444432333132
-554553434244
-454443443333
-433111121111
-555555555544
-444432433222
-444443433233
-443332332133
-445344453243
-444444405434
-554554434343
-455344534443
-444444400434
-444444344344
-544554533443
-555554443455
-555555544544
-555554534444
-323123232311
-444344344443
-555554555544
-343323332333
-444443434444
-444442214340
-555434434444
-455543343444
-445432434433
-455553434455
-243321332322
-444422332332
-555533454444
-555544443433
-354422431422
-333322421211
-444443432434
-344422431322
-333342222321
-444443454433
-443443444433
-553434531334
-554434552343
-545455553544
-554455554443
-555555555533
-555555554543
-454454544433
-555444532143
-554545544443
-444233442434
-544444434234
-554344432233
-555345533355
-554554544433
-455444444344
-555554554554
-555554545555
-444433534434
-444444421134
-334333333333
-334443443343
-122333441413
-434444333333
-444344433233
-444333332143
-555154344133
-324344333223
-244444402233
-454443433543
-444344433344
-455555445555
-555555544433
-454544434443
-344535554533
-333435443433
-444444554544
-343434443333
-544553544455
-555444444455
-244333332222
-333441232233
-544433433433
-555544343344
-344211142124
-442442232113
-433432332223
-333424322222
-444443333233
-344321232223
-442434342422
-545555535555
-455454434454
-455355544444
-454444444445
-555554432430
-555444344144
-455534342234
-555555554354
-213332443111
-555545434433
-555554424444
-443434443433
-555555554443
-555555555544
-435535554433
-555455545443
-555555554533
-554545443455
-555553333233
-434432232323
-443443333433
-443544231534
-434533334334
-555555535544
-555544444445
-555555555555
-555555555555
-555555555555
-555555555555
-544555544544
-555555545555
-545555534354
-445444344344
-555555555555
-444443343334
-555444440000
-555555555554
-555555545455
-454434444345
-555445544443
-555554535433
-454445555555
-555444444355
-555555555555
-555555555555
-555555555555
-555555554555
-555455554455
-454435544255
-545543342243
-555544355345
-555555455354
-435553244333
-555555443454
-444444433445
-121422433111
-555554543244
-555444554444
-444203444433
-344342553322
-554445554344
-545445454454
-444245504233
-334335555533
-554355544444
-444445545444
-555555555344
-454544543233
-444455532434
-555544354243
-535444554433
-444444444444
-455555553243
-244442343235
-554544504043
-444435553433
-455553434354
-555444343314
-555553344453
-555555555544
-544444444433
-434434550033
-555455544444
-455445534344
-454445444554
-555555555555
-555555555555
-555555555454
-555455545412
-334433343132
-555454455455
-555554454443
-555555555555
-555555553433
-555555555555
-555555445555
-555545445455
-555555545554
-555555555555
-445455554443
-555445554433
-555555554445
-555553333144
-555554455143
-554454445444
-555555554533
-334422433422
-555554434444
-433531133222
-443432342224
-555544554433
-553434333333
-555543334443
-444443344323
-555555555555
-555554545344
-555555545554
-555342434333
-444443333233
-444544435444
-555555555555
-555455554354
-555555405054
-555555555555
-554555545545
-555555555544
-555545454344
-555555455555
-433444444343
-454555554444
-453555554544
-444445554444
-544455555455
-555343434343
-555553444454
-555444434443
-555555555444
-555555545443
-555444534455
-555555555555
-444443443434
-555445533543
-555555555444
-555544444354
-445444544243
-555554555455
-555555455444
-555555555544
-555555555454
-554445545454
-555555555543
-344444434443
-555555454453
-455444443433
-344244434433
-355234452132
-555445545455
-444444444444
-455444454433
-555555555555
-344233341155
-333334433233
-444433434333
-444233443334
-454344544444
-555554245253
-444444333344
-554544434333
-555444443343
-555554555445
-005555555544
-554444445244
-555455555555
-555555555444
-444444444433
-555555555444
-555555544454
-555555555544
-444444434444
-555555535553
-545534543334
-554554534533
-555555555554
-554554544544
-454541231221
-555555535544
-445344334432
-444444344333
-555555534444
-555555545555
-443434544232
-545544433343
-343234434333
-444444443233
-555454444455
-555455535455
-554433442243
-444444544454
-554545554544
-454444444444
-554455534455
-555555544355
-555445555555
-555444534444
-455454534444
-555555555544
-444444444455
-454455554344
-443244442233
-350554554434
-455444535343
-344233443433
-454444535545
-244222232232
-245345554344
-355344444443
-555455555444
-444434444233
-334323444322
-333234443233
-455455554454
-555554544434
-555355555444
-444444544443
-555554555544
-344231224131
-444443434433
-454344444431
-555554555553
-454544444443
-444443443532
-334323423222
-455555545554
-555542434443
-554444444443
-554444534344
-555443454555
-555555534554
-444434443343
-444234432233
-334323311333
-444443443233
-355424552242
-233335323322
-234233443333
-123353532334
-444345555244
-332222433422
-545445555443
-555555555555
-555554444443
-455445554455
-455455555455
-555555555555
-444443433445
-555455555544
-355453434232
-555555554344
-444433433433
-434322242112
-444444433433
-444445444544
-555444535444
-544444444443
-454544344334
-454444334333
-434433433332
-334434423423
-444455552233
-442442342233
-445433433343
-555555444355
-555555555555
-455554555545
-555445544444
-544444434443
-555555535333
-554444444344
-555554554445
-345433334121
-555555555554
-545444444233
-555555544455
-555554545555
-555555455555
-555555545555
-555554544455
-444444444333
-544544544445
-555445544544
-555455444455
-555455555454
-554455535444
-453423442244
-444443444444
-454444333333
-555555555542
-555555555543
-555555555544
-555555535534
-555555555444
-444443423333
-555444444444
-555445544453
-444444444443
-555555443444
-444443444333
-554554154344
-555543353333
-554445443333
-555555553455
-554534444243
-554555554545
-555544443443
-555554555555
-555425552422
-555555555555
-454421121321
-555454453433
-555555554443
-244224431223
-455444453444
-454345544455
-344235545044
-555555555544
-555553325554
-554554444244
-555545544544
-454555554545
-444444545444
-545544444455
-454344445443
-545555554453
-444444444443
-545554554453
-555453434444
-005434434454
-455555455544
-555555555554
-455542444433
-545543444555
-445545445444
-555553325454
-555554555554
-444544444443
-555555444454
-443332443222
-454444443444
-344333433332
-555455444455
-555555555445
-555555534454
-445433432343
-555554434444
-555555545444
-544545434455
-454435543444
-555555555555
-555455554555
-555555544554
-555553554455
-555555555555
-545454545444
-555555445555
-554534343444
-555545443343
-545454334444
-554445544544
-555455354344
-555555555355
-554445544354
-455554444444
-555555553543
-555554444444
-555543233444
-555554344433
-333334441223
-543554344434
-055541243244
-555555555544
-555555533444
-555445444544
-444335343433
-544434344333
-544435533333
-444444443333
-555555555554
-555552343233
-444444444433
-544444434444
-555555545555
-555555555554
-445244434444
-445444434444
-445555554521
-443444543343
-444433343434
-355444433442
-555543444455
-555444334544
-555555555555
-554555545555
-555555554555
-555555555555
-454444343445
-444444433444
-354445443444
-555334242132
-455445555543
-254153343433
-354244443333
-554455444344
-343255535444
-455454555553
-555455545555
-444343433343
-343323443323
-455444444424
-555544444455
-343434543444
-555555555544
-555554534243
-555554543344
-555455555544
-555443344343
-444445533133
-555543534555
-444554444444
-444002323320
-444232322222
-244344424441
-555443344334
-555555555555
-555444534443
-555555555555
-555555555550
-455555554555
-555555555555
-
-end data.
-
-missing values
- question13 question14 question15 question16 question17 question18 question19 question20 question21 question22 question23 question24 (0).
-
-factor
- /variables question13 question14 question15 question16 question17 question18 question19 question20 question21 question22 question23 question24
- /analysis all
- /print univariate det correlation
- /format blank(.30)
- /plot eigen
- /criteria factors(3)
- /extraction paf
- /method = correlation.
-])
-
-AT_CHECK([pspp -O format=csv factor.sps], [0],
- [Table: Descriptive Statistics
-,Mean,Std. Deviation,Analysis N
-question13,4.462,.729,1365
-question14,4.525,.700,1365
-question15,4.445,.732,1365
-question16,4.281,.829,1365
-question17,4.166,.895,1365
-question18,3.930,1.034,1365
-question19,4.077,.963,1365
-question20,3.777,.909,1365
-question21,3.774,.984,1365
-question22,3.607,1.116,1365
-question23,3.813,.957,1365
-question24,3.666,.926,1365
-
-Table: Correlation Matrix
-,,question13,question14,question15,question16,question17,question18,question19,question20,question21,question22,question23,question24
-Correlation,question13,1.000,.661,.600,.566,.577,.409,.286,.304,.476,.333,.564,.454
-,question14,.661,1.000,.635,.500,.552,.433,.320,.315,.449,.333,.565,.443
-,question15,.600,.635,1.000,.505,.587,.457,.359,.356,.509,.369,.582,.435
-,question16,.566,.500,.505,1.000,.586,.405,.335,.317,.452,.363,.459,.430
-,question17,.577,.552,.587,.586,1.000,.555,.449,.417,.595,.450,.613,.521
-,question18,.409,.433,.457,.405,.555,1.000,.627,.521,.554,.536,.569,.474
-,question19,.286,.320,.359,.335,.449,.627,1.000,.446,.499,.484,.444,.374
-,question20,.304,.315,.356,.317,.417,.521,.446,1.000,.425,.383,.410,.357
-,question21,.476,.449,.509,.452,.595,.554,.499,.425,1.000,.507,.598,.500
-,question22,.333,.333,.369,.363,.450,.536,.484,.383,.507,1.000,.493,.444
-,question23,.564,.565,.582,.459,.613,.569,.444,.410,.598,.493,1.000,.705
-,question24,.454,.443,.435,.430,.521,.474,.374,.357,.500,.444,.705,1.000
-Caption: Determinant: 0.00
-
-Table: Factor Matrix
-,Factor,,
-,1,2,3
-question13,.713,.398,
-question14,.703,.339,
-question15,.721,,
-question16,.648,,
-question17,.783,,
-question18,.740,-.345,
-question19,.616,-.415,
-question20,.550,,
-question21,.732,,
-question22,.613,,
-question23,.819,,.345
-question24,.695,,.386
-
-Table: Rotated Factor Matrix
-,Factor,,
-,1,2,3
-question13,.771,,
-question14,.726,,
-question15,.676,,
-question16,.591,,
-question17,.587,.446,
-question18,,.739,
-question19,,.727,
-question20,,.540,
-question21,.402,.533,.321
-question22,,.559,
-question23,.449,.377,.668
-question24,.324,.321,.652
-])
-
-AT_CLEANUP
-
-AT_SETUP([FACTOR extraction=pc method=correlation])
-dnl This example is from http://www.ats.ucla.edu/stat/spss/whatstat/whatstat.htm
-
-AT_DATA([factor2.sps],
- [set format = F11.3.
-
-
-data list notable list /id female race ses schtyp prog read write math science socst.
-begin data.
- 70.00 .00 4.00 1.00 1.00 1.00 57.00 52.00 41.00 47.00 57.00
-121.00 1.00 4.00 2.00 1.00 3.00 68.00 59.00 53.00 63.00 61.00
- 86.00 .00 4.00 3.00 1.00 1.00 44.00 33.00 54.00 58.00 31.00
-141.00 .00 4.00 3.00 1.00 3.00 63.00 44.00 47.00 53.00 56.00
-172.00 .00 4.00 2.00 1.00 2.00 47.00 52.00 57.00 53.00 61.00
-113.00 .00 4.00 2.00 1.00 2.00 44.00 52.00 51.00 63.00 61.00
- 50.00 .00 3.00 2.00 1.00 1.00 50.00 59.00 42.00 53.00 61.00
- 11.00 .00 1.00 2.00 1.00 2.00 34.00 46.00 45.00 39.00 36.00
- 84.00 .00 4.00 2.00 1.00 1.00 63.00 57.00 54.00 58.00 51.00
- 48.00 .00 3.00 2.00 1.00 2.00 57.00 55.00 52.00 50.00 51.00
- 75.00 .00 4.00 2.00 1.00 3.00 60.00 46.00 51.00 53.00 61.00
- 60.00 .00 4.00 2.00 1.00 2.00 57.00 65.00 51.00 63.00 61.00
- 95.00 .00 4.00 3.00 1.00 2.00 73.00 60.00 71.00 61.00 71.00
-104.00 .00 4.00 3.00 1.00 2.00 54.00 63.00 57.00 55.00 46.00
- 38.00 .00 3.00 1.00 1.00 2.00 45.00 57.00 50.00 31.00 56.00
-115.00 .00 4.00 1.00 1.00 1.00 42.00 49.00 43.00 50.00 56.00
- 76.00 .00 4.00 3.00 1.00 2.00 47.00 52.00 51.00 50.00 56.00
-195.00 .00 4.00 2.00 2.00 1.00 57.00 57.00 60.00 58.00 56.00
-114.00 .00 4.00 3.00 1.00 2.00 68.00 65.00 62.00 55.00 61.00
- 85.00 .00 4.00 2.00 1.00 1.00 55.00 39.00 57.00 53.00 46.00
-167.00 .00 4.00 2.00 1.00 1.00 63.00 49.00 35.00 66.00 41.00
-143.00 .00 4.00 2.00 1.00 3.00 63.00 63.00 75.00 72.00 66.00
- 41.00 .00 3.00 2.00 1.00 2.00 50.00 40.00 45.00 55.00 56.00
- 20.00 .00 1.00 3.00 1.00 2.00 60.00 52.00 57.00 61.00 61.00
- 12.00 .00 1.00 2.00 1.00 3.00 37.00 44.00 45.00 39.00 46.00
- 53.00 .00 3.00 2.00 1.00 3.00 34.00 37.00 46.00 39.00 31.00
-154.00 .00 4.00 3.00 1.00 2.00 65.00 65.00 66.00 61.00 66.00
-178.00 .00 4.00 2.00 2.00 3.00 47.00 57.00 57.00 58.00 46.00
-196.00 .00 4.00 3.00 2.00 2.00 44.00 38.00 49.00 39.00 46.00
- 29.00 .00 2.00 1.00 1.00 1.00 52.00 44.00 49.00 55.00 41.00
-126.00 .00 4.00 2.00 1.00 1.00 42.00 31.00 57.00 47.00 51.00
-103.00 .00 4.00 3.00 1.00 2.00 76.00 52.00 64.00 64.00 61.00
-192.00 .00 4.00 3.00 2.00 2.00 65.00 67.00 63.00 66.00 71.00
-150.00 .00 4.00 2.00 1.00 3.00 42.00 41.00 57.00 72.00 31.00
-199.00 .00 4.00 3.00 2.00 2.00 52.00 59.00 50.00 61.00 61.00
-144.00 .00 4.00 3.00 1.00 1.00 60.00 65.00 58.00 61.00 66.00
-200.00 .00 4.00 2.00 2.00 2.00 68.00 54.00 75.00 66.00 66.00
- 80.00 .00 4.00 3.00 1.00 2.00 65.00 62.00 68.00 66.00 66.00
- 16.00 .00 1.00 1.00 1.00 3.00 47.00 31.00 44.00 36.00 36.00
-153.00 .00 4.00 2.00 1.00 3.00 39.00 31.00 40.00 39.00 51.00
-176.00 .00 4.00 2.00 2.00 2.00 47.00 47.00 41.00 42.00 51.00
-177.00 .00 4.00 2.00 2.00 2.00 55.00 59.00 62.00 58.00 51.00
-168.00 .00 4.00 2.00 1.00 2.00 52.00 54.00 57.00 55.00 51.00
- 40.00 .00 3.00 1.00 1.00 1.00 42.00 41.00 43.00 50.00 41.00
- 62.00 .00 4.00 3.00 1.00 1.00 65.00 65.00 48.00 63.00 66.00
-169.00 .00 4.00 1.00 1.00 1.00 55.00 59.00 63.00 69.00 46.00
- 49.00 .00 3.00 3.00 1.00 3.00 50.00 40.00 39.00 49.00 47.00
-136.00 .00 4.00 2.00 1.00 2.00 65.00 59.00 70.00 63.00 51.00
-189.00 .00 4.00 2.00 2.00 2.00 47.00 59.00 63.00 53.00 46.00
- 7.00 .00 1.00 2.00 1.00 2.00 57.00 54.00 59.00 47.00 51.00
- 27.00 .00 2.00 2.00 1.00 2.00 53.00 61.00 61.00 57.00 56.00
-128.00 .00 4.00 3.00 1.00 2.00 39.00 33.00 38.00 47.00 41.00
- 21.00 .00 1.00 2.00 1.00 1.00 44.00 44.00 61.00 50.00 46.00
-183.00 .00 4.00 2.00 2.00 2.00 63.00 59.00 49.00 55.00 71.00
-132.00 .00 4.00 2.00 1.00 2.00 73.00 62.00 73.00 69.00 66.00
- 15.00 .00 1.00 3.00 1.00 3.00 39.00 39.00 44.00 26.00 42.00
- 67.00 .00 4.00 1.00 1.00 3.00 37.00 37.00 42.00 33.00 32.00
- 22.00 .00 1.00 2.00 1.00 3.00 42.00 39.00 39.00 56.00 46.00
-185.00 .00 4.00 2.00 2.00 2.00 63.00 57.00 55.00 58.00 41.00
- 9.00 .00 1.00 2.00 1.00 3.00 48.00 49.00 52.00 44.00 51.00
-181.00 .00 4.00 2.00 2.00 2.00 50.00 46.00 45.00 58.00 61.00
-170.00 .00 4.00 3.00 1.00 2.00 47.00 62.00 61.00 69.00 66.00
-134.00 .00 4.00 1.00 1.00 1.00 44.00 44.00 39.00 34.00 46.00
-108.00 .00 4.00 2.00 1.00 1.00 34.00 33.00 41.00 36.00 36.00
-197.00 .00 4.00 3.00 2.00 2.00 50.00 42.00 50.00 36.00 61.00
-140.00 .00 4.00 2.00 1.00 3.00 44.00 41.00 40.00 50.00 26.00
-171.00 .00 4.00 2.00 1.00 2.00 60.00 54.00 60.00 55.00 66.00
-107.00 .00 4.00 1.00 1.00 3.00 47.00 39.00 47.00 42.00 26.00
- 81.00 .00 4.00 1.00 1.00 2.00 63.00 43.00 59.00 65.00 44.00
- 18.00 .00 1.00 2.00 1.00 3.00 50.00 33.00 49.00 44.00 36.00
-155.00 .00 4.00 2.00 1.00 1.00 44.00 44.00 46.00 39.00 51.00
- 97.00 .00 4.00 3.00 1.00 2.00 60.00 54.00 58.00 58.00 61.00
- 68.00 .00 4.00 2.00 1.00 2.00 73.00 67.00 71.00 63.00 66.00
-157.00 .00 4.00 2.00 1.00 1.00 68.00 59.00 58.00 74.00 66.00
- 56.00 .00 4.00 2.00 1.00 3.00 55.00 45.00 46.00 58.00 51.00
- 5.00 .00 1.00 1.00 1.00 2.00 47.00 40.00 43.00 45.00 31.00
-159.00 .00 4.00 3.00 1.00 2.00 55.00 61.00 54.00 49.00 61.00
-123.00 .00 4.00 3.00 1.00 1.00 68.00 59.00 56.00 63.00 66.00
-164.00 .00 4.00 2.00 1.00 3.00 31.00 36.00 46.00 39.00 46.00
- 14.00 .00 1.00 3.00 1.00 2.00 47.00 41.00 54.00 42.00 56.00
-127.00 .00 4.00 3.00 1.00 2.00 63.00 59.00 57.00 55.00 56.00
-165.00 .00 4.00 1.00 1.00 3.00 36.00 49.00 54.00 61.00 36.00
-174.00 .00 4.00 2.00 2.00 2.00 68.00 59.00 71.00 66.00 56.00
- 3.00 .00 1.00 1.00 1.00 2.00 63.00 65.00 48.00 63.00 56.00
- 58.00 .00 4.00 2.00 1.00 3.00 55.00 41.00 40.00 44.00 41.00
-146.00 .00 4.00 3.00 1.00 2.00 55.00 62.00 64.00 63.00 66.00
-102.00 .00 4.00 3.00 1.00 2.00 52.00 41.00 51.00 53.00 56.00
-117.00 .00 4.00 3.00 1.00 3.00 34.00 49.00 39.00 42.00 56.00
-133.00 .00 4.00 2.00 1.00 3.00 50.00 31.00 40.00 34.00 31.00
- 94.00 .00 4.00 3.00 1.00 2.00 55.00 49.00 61.00 61.00 56.00
- 24.00 .00 2.00 2.00 1.00 2.00 52.00 62.00 66.00 47.00 46.00
-149.00 .00 4.00 1.00 1.00 1.00 63.00 49.00 49.00 66.00 46.00
- 82.00 1.00 4.00 3.00 1.00 2.00 68.00 62.00 65.00 69.00 61.00
- 8.00 1.00 1.00 1.00 1.00 2.00 39.00 44.00 52.00 44.00 48.00
-129.00 1.00 4.00 1.00 1.00 1.00 44.00 44.00 46.00 47.00 51.00
-173.00 1.00 4.00 1.00 1.00 1.00 50.00 62.00 61.00 63.00 51.00
- 57.00 1.00 4.00 2.00 1.00 2.00 71.00 65.00 72.00 66.00 56.00
-100.00 1.00 4.00 3.00 1.00 2.00 63.00 65.00 71.00 69.00 71.00
- 1.00 1.00 1.00 1.00 1.00 3.00 34.00 44.00 40.00 39.00 41.00
-194.00 1.00 4.00 3.00 2.00 2.00 63.00 63.00 69.00 61.00 61.00
- 88.00 1.00 4.00 3.00 1.00 2.00 68.00 60.00 64.00 69.00 66.00
- 99.00 1.00 4.00 3.00 1.00 1.00 47.00 59.00 56.00 66.00 61.00
- 47.00 1.00 3.00 1.00 1.00 2.00 47.00 46.00 49.00 33.00 41.00
-120.00 1.00 4.00 3.00 1.00 2.00 63.00 52.00 54.00 50.00 51.00
-166.00 1.00 4.00 2.00 1.00 2.00 52.00 59.00 53.00 61.00 51.00
- 65.00 1.00 4.00 2.00 1.00 2.00 55.00 54.00 66.00 42.00 56.00
-101.00 1.00 4.00 3.00 1.00 2.00 60.00 62.00 67.00 50.00 56.00
- 89.00 1.00 4.00 1.00 1.00 3.00 35.00 35.00 40.00 51.00 33.00
- 54.00 1.00 3.00 1.00 2.00 1.00 47.00 54.00 46.00 50.00 56.00
-180.00 1.00 4.00 3.00 2.00 2.00 71.00 65.00 69.00 58.00 71.00
-162.00 1.00 4.00 2.00 1.00 3.00 57.00 52.00 40.00 61.00 56.00
- 4.00 1.00 1.00 1.00 1.00 2.00 44.00 50.00 41.00 39.00 51.00
-131.00 1.00 4.00 3.00 1.00 2.00 65.00 59.00 57.00 46.00 66.00
-125.00 1.00 4.00 1.00 1.00 2.00 68.00 65.00 58.00 59.00 56.00
- 34.00 1.00 1.00 3.00 2.00 2.00 73.00 61.00 57.00 55.00 66.00
-106.00 1.00 4.00 2.00 1.00 3.00 36.00 44.00 37.00 42.00 41.00
-130.00 1.00 4.00 3.00 1.00 1.00 43.00 54.00 55.00 55.00 46.00
- 93.00 1.00 4.00 3.00 1.00 2.00 73.00 67.00 62.00 58.00 66.00
-163.00 1.00 4.00 1.00 1.00 2.00 52.00 57.00 64.00 58.00 56.00
- 37.00 1.00 3.00 1.00 1.00 3.00 41.00 47.00 40.00 39.00 51.00
- 35.00 1.00 1.00 1.00 2.00 1.00 60.00 54.00 50.00 50.00 51.00
- 87.00 1.00 4.00 2.00 1.00 1.00 50.00 52.00 46.00 50.00 56.00
- 73.00 1.00 4.00 2.00 1.00 2.00 50.00 52.00 53.00 39.00 56.00
-151.00 1.00 4.00 2.00 1.00 3.00 47.00 46.00 52.00 48.00 46.00
- 44.00 1.00 3.00 1.00 1.00 3.00 47.00 62.00 45.00 34.00 46.00
-152.00 1.00 4.00 3.00 1.00 2.00 55.00 57.00 56.00 58.00 61.00
-105.00 1.00 4.00 2.00 1.00 2.00 50.00 41.00 45.00 44.00 56.00
- 28.00 1.00 2.00 2.00 1.00 1.00 39.00 53.00 54.00 50.00 41.00
- 91.00 1.00 4.00 3.00 1.00 3.00 50.00 49.00 56.00 47.00 46.00
- 45.00 1.00 3.00 1.00 1.00 3.00 34.00 35.00 41.00 29.00 26.00
-116.00 1.00 4.00 2.00 1.00 2.00 57.00 59.00 54.00 50.00 56.00
- 33.00 1.00 2.00 1.00 1.00 2.00 57.00 65.00 72.00 54.00 56.00
- 66.00 1.00 4.00 2.00 1.00 3.00 68.00 62.00 56.00 50.00 51.00
- 72.00 1.00 4.00 2.00 1.00 3.00 42.00 54.00 47.00 47.00 46.00
- 77.00 1.00 4.00 1.00 1.00 2.00 61.00 59.00 49.00 44.00 66.00
- 61.00 1.00 4.00 3.00 1.00 2.00 76.00 63.00 60.00 67.00 66.00
-190.00 1.00 4.00 2.00 2.00 2.00 47.00 59.00 54.00 58.00 46.00
- 42.00 1.00 3.00 2.00 1.00 3.00 46.00 52.00 55.00 44.00 56.00
- 2.00 1.00 1.00 2.00 1.00 3.00 39.00 41.00 33.00 42.00 41.00
- 55.00 1.00 3.00 2.00 2.00 2.00 52.00 49.00 49.00 44.00 61.00
- 19.00 1.00 1.00 1.00 1.00 1.00 28.00 46.00 43.00 44.00 51.00
- 90.00 1.00 4.00 3.00 1.00 2.00 42.00 54.00 50.00 50.00 52.00
-142.00 1.00 4.00 2.00 1.00 3.00 47.00 42.00 52.00 39.00 51.00
- 17.00 1.00 1.00 2.00 1.00 2.00 47.00 57.00 48.00 44.00 41.00
-122.00 1.00 4.00 2.00 1.00 2.00 52.00 59.00 58.00 53.00 66.00
-191.00 1.00 4.00 3.00 2.00 2.00 47.00 52.00 43.00 48.00 61.00
- 83.00 1.00 4.00 2.00 1.00 3.00 50.00 62.00 41.00 55.00 31.00
-182.00 1.00 4.00 2.00 2.00 2.00 44.00 52.00 43.00 44.00 51.00
- 6.00 1.00 1.00 1.00 1.00 2.00 47.00 41.00 46.00 40.00 41.00
- 46.00 1.00 3.00 1.00 1.00 2.00 45.00 55.00 44.00 34.00 41.00
- 43.00 1.00 3.00 1.00 1.00 2.00 47.00 37.00 43.00 42.00 46.00
- 96.00 1.00 4.00 3.00 1.00 2.00 65.00 54.00 61.00 58.00 56.00
-138.00 1.00 4.00 2.00 1.00 3.00 43.00 57.00 40.00 50.00 51.00
- 10.00 1.00 1.00 2.00 1.00 1.00 47.00 54.00 49.00 53.00 61.00
- 71.00 1.00 4.00 2.00 1.00 1.00 57.00 62.00 56.00 58.00 66.00
-139.00 1.00 4.00 2.00 1.00 2.00 68.00 59.00 61.00 55.00 71.00
-110.00 1.00 4.00 2.00 1.00 3.00 52.00 55.00 50.00 54.00 61.00
-148.00 1.00 4.00 2.00 1.00 3.00 42.00 57.00 51.00 47.00 61.00
-109.00 1.00 4.00 2.00 1.00 1.00 42.00 39.00 42.00 42.00 41.00
- 39.00 1.00 3.00 3.00 1.00 2.00 66.00 67.00 67.00 61.00 66.00
-147.00 1.00 4.00 1.00 1.00 2.00 47.00 62.00 53.00 53.00 61.00
- 74.00 1.00 4.00 2.00 1.00 2.00 57.00 50.00 50.00 51.00 58.00
-198.00 1.00 4.00 3.00 2.00 2.00 47.00 61.00 51.00 63.00 31.00
-161.00 1.00 4.00 1.00 1.00 2.00 57.00 62.00 72.00 61.00 61.00
-112.00 1.00 4.00 2.00 1.00 2.00 52.00 59.00 48.00 55.00 61.00
- 69.00 1.00 4.00 1.00 1.00 3.00 44.00 44.00 40.00 40.00 31.00
-156.00 1.00 4.00 2.00 1.00 2.00 50.00 59.00 53.00 61.00 61.00
-111.00 1.00 4.00 1.00 1.00 1.00 39.00 54.00 39.00 47.00 36.00
-186.00 1.00 4.00 2.00 2.00 2.00 57.00 62.00 63.00 55.00 41.00
- 98.00 1.00 4.00 1.00 1.00 3.00 57.00 60.00 51.00 53.00 37.00
-119.00 1.00 4.00 1.00 1.00 1.00 42.00 57.00 45.00 50.00 43.00
- 13.00 1.00 1.00 2.00 1.00 3.00 47.00 46.00 39.00 47.00 61.00
- 51.00 1.00 3.00 3.00 1.00 1.00 42.00 36.00 42.00 31.00 39.00
- 26.00 1.00 2.00 3.00 1.00 2.00 60.00 59.00 62.00 61.00 51.00
- 36.00 1.00 3.00 1.00 1.00 1.00 44.00 49.00 44.00 35.00 51.00
-135.00 1.00 4.00 1.00 1.00 2.00 63.00 60.00 65.00 54.00 66.00
- 59.00 1.00 4.00 2.00 1.00 2.00 65.00 67.00 63.00 55.00 71.00
- 78.00 1.00 4.00 2.00 1.00 2.00 39.00 54.00 54.00 53.00 41.00
- 64.00 1.00 4.00 3.00 1.00 3.00 50.00 52.00 45.00 58.00 36.00
- 63.00 1.00 4.00 1.00 1.00 1.00 52.00 65.00 60.00 56.00 51.00
- 79.00 1.00 4.00 2.00 1.00 2.00 60.00 62.00 49.00 50.00 51.00
-193.00 1.00 4.00 2.00 2.00 2.00 44.00 49.00 48.00 39.00 51.00
- 92.00 1.00 4.00 3.00 1.00 1.00 52.00 67.00 57.00 63.00 61.00
-160.00 1.00 4.00 2.00 1.00 2.00 55.00 65.00 55.00 50.00 61.00
- 32.00 1.00 2.00 3.00 1.00 3.00 50.00 67.00 66.00 66.00 56.00
- 23.00 1.00 2.00 1.00 1.00 2.00 65.00 65.00 64.00 58.00 71.00
-158.00 1.00 4.00 2.00 1.00 1.00 52.00 54.00 55.00 53.00 51.00
- 25.00 1.00 2.00 2.00 1.00 1.00 47.00 44.00 42.00 42.00 36.00
-188.00 1.00 4.00 3.00 2.00 2.00 63.00 62.00 56.00 55.00 61.00
- 52.00 1.00 3.00 1.00 1.00 2.00 50.00 46.00 53.00 53.00 66.00
-124.00 1.00 4.00 1.00 1.00 3.00 42.00 54.00 41.00 42.00 41.00
-175.00 1.00 4.00 3.00 2.00 1.00 36.00 57.00 42.00 50.00 41.00
-184.00 1.00 4.00 2.00 2.00 3.00 50.00 52.00 53.00 55.00 56.00
- 30.00 1.00 2.00 3.00 1.00 2.00 41.00 59.00 42.00 34.00 51.00
-179.00 1.00 4.00 2.00 2.00 2.00 47.00 65.00 60.00 50.00 56.00
- 31.00 1.00 2.00 2.00 2.00 1.00 55.00 59.00 52.00 42.00 56.00
-145.00 1.00 4.00 2.00 1.00 3.00 42.00 46.00 38.00 36.00 46.00
-187.00 1.00 4.00 2.00 2.00 1.00 57.00 41.00 57.00 55.00 52.00
-118.00 1.00 4.00 2.00 1.00 1.00 55.00 62.00 58.00 58.00 61.00
-137.00 1.00 4.00 3.00 1.00 2.00 63.00 65.00 65.00 53.00 61.00
-end data.
-
-factor
- /variables read write math science socst
- /analysis read write math science socst
- /extraction pc
- /plot eigen
- /criteria mineigen (.557)
- .
-])
-
-AT_CHECK([pspp -O format=csv factor2.sps], [0],
- [Table: Communalities
-,Initial,Extraction
-read,1.000,.736
-write,1.000,.704
-math,1.000,.750
-science,1.000,.849
-socst,1.000,.900
-
-Table: Total Variance Explained
-,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,,Rotation Sums of Squared Loadings,,
-,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
-1,3.381,67.6%,67.6%,3.381,67.6%,67.6%,2.113,42.3%,42.3%
-2,.557,11.1%,78.8%,.557,11.1%,78.8%,1.825,36.5%,78.8%
-3,.407,8.1%,86.9%,,,,,,
-4,.356,7.1%,94.0%,,,,,,
-5,.299,6.0%,100.0%,,,,,,
-
-Table: Component Matrix
-,Component,
-,1,2
-read,.858,.020
-write,.824,-.155
-math,.844,.195
-science,.801,.456
-socst,.783,-.536
-
-Table: Rotated Component Matrix
-,Component,
-,1,2
-read,.650,.559
-write,.508,.667
-math,.757,.421
-science,.900,.198
-socst,.222,.922
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([FACTOR empty dataset])
-dnl Test that something sane happens when the dataset contains no complete observations
-
-AT_DATA([factor-empty.sps],
- [data list notable list /x * y * z *.
-begin data.
-3.4 . 92.9
-. 32.09 34.2
-1.00 19.80 .
-2.00 . 3.6
-end data.
-
-factor /variables = ALL.
-])
-
-AT_CHECK([pspp -O format=csv factor-empty.sps], [0], [ignore])
-AT_CLEANUP
-
-
-dnl Fixes a crash reported at
-dnl http://lists.gnu.org/archive/html/bug-gnu-pspp/2012-04/msg00001.html
-AT_SETUP([FACTOR /ROTATION=NOROTATE])
-AT_DATA([factor-norotate.sps], [dnl
-DATA LIST FREE / TRAIT1 TO TRAIT5 (F8.2).
-BEGIN DATA
-1 5 5 1 1
-8 9 7 9 8
-9 8 9 9 8
-9 9 9 9 9
-1 9 1 1 9
-9 7 7 9 9
-9 7 9 9 7
-END DATA
-
-SET SMALL=0.
-FACTOR /VARIABLES=TRAIT1 TO TRAIT5
- /ROTATION=NOROTATE /* NOROTATE may have caused the problem. */
- /EXTRACTION=PC
- /PRINT=DEFAULT DET UNIVARIATE ROTATION SIG CORRELATION.
-])
-
-AT_CHECK([pspp -O format=csv factor-norotate.sps], [0], [dnl
-Table: Descriptive Statistics
-,Mean,Std. Deviation,Analysis N
-TRAIT1,6.57,3.54,7
-TRAIT2,7.71,1.39,7
-TRAIT3,6.71,2.71,7
-TRAIT4,6.71,3.61,7
-TRAIT5,7.29,2.66,7
-
-Table: Correlation Matrix
-,,TRAIT1,TRAIT2,TRAIT3,TRAIT4,TRAIT5
-Correlation,TRAIT1,1.000,.296,.881,.995,.545
-,TRAIT2,.296,1.000,-.022,.326,.837
-,TRAIT3,.881,-.022,1.000,.867,.130
-,TRAIT4,.995,.326,.867,1.000,.544
-,TRAIT5,.545,.837,.130,.544,1.000
-Sig. (1-tailed),TRAIT1,,.260,.004,.000,.103
-,TRAIT2,.260,,.482,.238,.009
-,TRAIT3,.004,.482,,.006,.390
-,TRAIT4,.000,.238,.006,,.103
-,TRAIT5,.103,.009,.390,.103,
-Caption: Determinant: 0.00
-
-Table: Communalities
-,Initial,Extraction
-TRAIT1,1.00,1.00
-TRAIT2,1.00,1.00
-TRAIT3,1.00,.99
-TRAIT4,1.00,.99
-TRAIT5,1.00,.99
-
-Table: Total Variance Explained
-,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,
-,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
-1,3.26,65.3%,65.3%,3.26,65.3%,65.3%
-2,1.54,30.8%,96.0%,1.54,30.8%,96.0%
-3,.17,3.4%,99.4%,.17,3.4%,99.4%
-4,.03,.6%,100.0%,.03,.6%,100.0%
-5,.00,.0%,100.0%,,,
-
-Table: Component Matrix
-,Component,,,
-,1,2,3,4
-TRAIT1,.97,.23,-.08,.00
-TRAIT2,.52,-.81,.28,.00
-TRAIT3,.78,.59,.17,.00
-TRAIT4,.97,.21,-.04,.00
-TRAIT5,.70,-.67,-.23,.00
-])
-AT_CLEANUP
-
-
-
-dnl Fixes a bug in the way that the /CRITERIA = ITERATE option was interpreted.
-dnl http://lists.gnu.org/archive/html/bug-gnu-pspp/2013-09/msg00036.html
-AT_SETUP([FACTOR /CRITERIA=ITERATE])
-AT_DATA([factor-iterate.sps], [dnl
-set format = F20.3.
-data list notable list /x y z *.
-begin data.
-1.00 5.00 3.00
-2.00 2.00 2.00
-3.00 1.00 1.00
-4.00 4.00 5.00
-5.00 3.00 9.00
-6.00 6.00 4.00
-7.00 7.00 6.00
-8.00 8.00 8.00
-9.00 9.00 7.00
-end data.
-
-FACTOR
- /VARIABLES= x y z
- /CRITERIA = MINEIGEN (1) ITERATE (25)
- /EXTRACTION =PAF
- /METHOD = CORRELATION
- /PRINT = INITIAL EXTRACTION
- /CRITERIA = ITERATE (0)
- /ROTATION = NOROTATE.
-])
-
-AT_CHECK([pspp -O format=csv factor-iterate.sps], [0], [dnl
-Table: Communalities
-,Initial,Extraction
-x,.735,.979
-y,.640,.653
-z,.514,.523
-
-Table: Total Variance Explained
-,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,
-,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
-1,2.404,80.1%,80.1%,2.155,71.8%,71.8%
-2,.425,14.2%,94.3%,,,
-3,.171,5.7%,100.0%,,,
-
-Table: Factor Matrix
-,Factor
-,1
-x,.990
-y,.808
-z,.723
-])
-AT_CLEANUP
-
-
-AT_SETUP([FACTOR promax])
-AT_DATA([factor-promax.sps], [dnl
-set decimal=dot.
-set format=F22.3.
-
-get file='llz.zsav'.
-
-factor
- /variables pz pn ps nz nn ns tz tn ts oz on os sz sn ss zz zn zs
- /missing listwise
- /print initial extraction rotation
- /criteria mineigen(1) iterate(25)
- /extraction paf
- /method correlation
- /rotation promax (5).
-])
-
-AT_CHECK([ln -s $top_srcdir/tests/language/stats/llz.zsav .], [0], [ignore])
-
-AT_CHECK([pspp -O format=csv factor-promax.sps], [0], [dnl
-Table: Communalities
-,Initial,Extraction
-PZ,.191,.375
-PN,.042,.102
-PS,.458,.623
-NZ,.100,.163
-NN,.065,.079
-NS,.129,.148
-TZ,.181,.344
-TN,.102,.142
-TS,.310,.372
-OZ,.097,.158
-ON,.323,.410
-OS,.469,.617
-SZ,.104,.170
-SN,.154,.267
-SS,.081,.180
-ZZ,.123,.192
-ZN,.208,.412
-ZS,.130,.158
-
-Table: Total Variance Explained
-,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,,Rotation Sums of Squared Loadings,,
-,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
-1,2.968,16.5%,16.5%,2.411,13.4%,13.4%,. ,. ,. @&t@
-2,2.026,11.3%,27.7%,1.271,7.1%,20.5%,. ,. ,-Infinity
-3,1.622,9.0%,36.8%,.948,5.3%,25.7%,. ,. ,-Infinity
-4,1.086,6.0%,42.8%,.283,1.6%,27.3%,. ,. ,-Infinity
-5,.996,5.5%,48.3%,,,,,,
-6,.923,5.1%,53.5%,,,,,,
-7,.873,4.9%,58.3%,,,,,,
-8,.856,4.8%,63.1%,,,,,,
-9,.836,4.6%,67.7%,,,,,,
-10,.816,4.5%,72.2%,,,,,,
-11,.785,4.4%,76.6%,,,,,,
-12,.740,4.1%,80.7%,,,,,,
-13,.713,4.0%,84.7%,,,,,,
-14,.653,3.6%,88.3%,,,,,,
-15,.633,3.5%,91.8%,,,,,,
-16,.604,3.4%,95.2%,,,,,,
-17,.484,2.7%,97.9%,,,,,,
-18,.386,2.1%,100.0%,,,,,,
-
-Table: Factor Matrix
-,Factor,,,
-,1,2,3,4
-PZ,-.276,.154,.510,.124
-PN,.096,.129,-.091,.261
-PS,.746,-.085,.234,.063
-NZ,-.111,.323,.206,-.058
-NN,.007,.260,-.083,-.069
-NS,.366,.096,.046,.051
-TZ,-.228,.172,.509,.059
-TN,.131,.345,-.074,.029
-TS,.601,-.005,.098,.030
-OZ,-.145,.166,.322,-.081
-ON,.607,.082,.073,-.173
-OS,.757,-.059,.171,-.104
-SZ,-.142,.307,.226,-.066
-SN,.175,.436,-.183,.115
-SS,.199,.206,-.083,.302
-ZZ,-.074,.411,-.080,-.104
-ZN,.015,.580,-.252,-.114
-ZS,.365,.156,-.004,.015
-
-Table: Pattern Matrix
-,Factor,,,
-,1,2,3,4
-PZ,-.063,-.126,.599,.085
-PN,-.035,.000,-.033,.325
-PS,.762,-.175,.058,.081
-NZ,.027,.230,.327,-.044
-NN,.008,.289,.008,-.026
-NS,.344,.044,.015,.091
-TZ,.004,-.074,.589,.020
-TN,.097,.307,.033,.103
-TS,.585,-.043,-.017,.062
-OZ,.046,.067,.382,-.109
-ON,.654,.151,-.029,-.145
-OS,.803,-.037,-.009,-.092
-SZ,.009,.213,.345,-.060
-SN,.065,.376,-.036,.227
-SS,.054,.042,-.013,.388
-ZZ,-.044,.434,.078,-.046
-ZN,-.025,.646,-.041,-.006
-ZS,.337,.133,-.013,.067
-
-Table: Structure Matrix
-,Factor,,,
-,1,2,3,4
-PZ,-.177,-.058,.598,-.022
-PN,.068,.110,-.049,.317
-PS,.771,-.138,-.136,.240
-NZ,-.060,.236,.339,.019
-NN,.000,.281,.027,.076
-NS,.368,.080,-.068,.207
-TZ,-.127,-.028,.582,-.049
-TN,.122,.345,.023,.235
-TS,.607,-.018,-.160,.221
-OZ,-.074,.055,.384,-.101
-ON,.619,.104,-.160,.102
-OS,.778,-.064,-.190,.132
-SZ,-.086,.215,.361,-.009
-SN,.143,.453,-.044,.380
-SS,.171,.176,-.052,.420
-ZZ,-.073,.422,.120,.085
-ZN,-.013,.641,.008,.214
-ZS,.361,.158,-.088,.213
-
-Table: Factor Correlation Matrix
-Factor,1,2,3,4
-1,1.000,.008,-.232,.294
-2,.008,1.000,.065,.347
-3,-.232,.065,1.000,-.076
-4,.294,.347,-.076,1.000
-])
-
-
-AT_CLEANUP
-
-
-
-
-
-
-AT_SETUP([FACTOR covariance matrix])
-
-AT_DATA([covariance-matrix.sps], [dnl
-set format = F10.3.
-matrix data
- variables = rowtype_ var01 var02 var03 var04 var05 var06 var07 var08
- / format = lower diagonal .
-begin data
-mean 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
-sd 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
-n 92 92 92 92 92 92 92 92
-cov 32.490000
-cov 1.539000 2.250000
-cov -29.469000 -5.992500 552.250000
-cov 11.901600 2.697000 -19.082000 33.640000
-cov 4.309200 0.672000 -7.896000 3.572800 7.840000
-cov 8.464500 1.012500 -17.977500 6.264000 2.646000 20.250000
-cov 15.390000 2.349000 -25.380000 10.022400 1.814400 9.234000 29.160000
-cov 1.453500 0.652500 -1.762500 1.740000 1.134000 1.350000 0.324000 2.250000
-end data.
-
-factor matrix in (cov = *)
- / method = covariance
- / print = initial covariance
- / extraction = pc
- / rotation = norotate.
-])
-
-AT_CHECK([pspp -O format=csv covariance-matrix.sps], [0], [dnl
-Table: Covariance Matrix
-,var01,var02,var03,var04,var05,var06,var07,var08
-var01,32.490,1.539,-29.469,11.902,4.309,8.464,15.390,1.454
-var02,1.539,2.250,-5.992,2.697,.672,1.013,2.349,.653
-var03,-29.469,-5.992,552.250,-19.082,-7.896,-17.977,-25.380,-1.763
-var04,11.902,2.697,-19.082,33.640,3.573,6.264,10.022,1.740
-var05,4.309,.672,-7.896,3.573,7.840,2.646,1.814,1.134
-var06,8.464,1.013,-17.977,6.264,2.646,20.250,9.234,1.350
-var07,15.390,2.349,-25.380,10.022,1.814,9.234,29.160,.324
-var08,1.454,.653,-1.763,1.740,1.134,1.350,.324,2.250
-
-Table: Communalities
-,Initial
-var01,32.490
-var02,2.250
-var03,552.250
-var04,33.640
-var05,7.840
-var06,20.250
-var07,29.160
-var08,2.250
-
-Table: Total Variance Explained
-,Initial Eigenvalues,,
-,Total,% of Variance,Cumulative %
-1,556.895,81.9%,81.9%
-2,57.963,8.5%,90.4%
-3,23.576,3.5%,93.9%
-4,16.446,2.4%,96.3%
-5,14.603,2.1%,98.4%
-6,6.831,1.0%,99.4%
-7,2.375,.3%,99.8%
-8,1.440,.2%,100.0%
-
-Table: Component Matrix
-,Component,,,,,,
-,1,2,3,4,5,6,7
-var01,1.394,4.388,1.513,-2.851,.849,.396,.033
-var02,.269,.460,-.173,.147,-.146,-.213,.872
-var03,-23.489,.725,.058,.003,.022,-.012,.006
-var04,.926,4.007,-4.068,.241,-.253,.218,-.026
-var05,.363,.829,-.172,-.255,.805,-2.492,.058
-var06,.843,2.354,.971,2.425,2.649,.392,.046
-var07,1.205,3.948,1.926,1.515,-2.450,-.317,-.087
-var08,.085,.319,-.157,-.011,.353,-.341,-.816
-])
-
-AT_CLEANUP
-
-
-
-dnl A more realistic example of factor analysis usage.
-AT_SETUP([FACTOR correlation matrix])
-
-AT_DATA([correlation-matrix.sps], [dnl
-set format = F10.3.
-
-matrix data
- variables = rowtype_
- cdi_actws_16 cdi_maxzin_16 rdls_passws_16 rdls_actws_16 cdi_actws_20 cdi_maxzin_20 cdi_actws_26 cdi_maxzin_26 rdls_passws_26 rdls_actws_26
- nepsy_passws_36 morf_verv_36 bnt_actws_36 klankgr_id_36 snelnoe_36 letters_36 ppvt_passws_50 morf_verv_50
- nepsy_passws_56 bnt_actws_56 klank_gr_weg_56 snelnoe_56 letters_56
- leesacc_wo_owo_811 leesacc_tekst_811 leesacc_otekst_811 leessne_wo_owo_811 leesvl_tekst_811 leesvl_otekst_811 leessne_wo_811 spel_wo_owo_811
- / format = upper diagonal .
-begin data
-mean 64.44 1.74 15.30 11.50 269.03 5.37 441.90 8.57 36.59 33.99 11.68 14.74 18.67 6.70 71.57 2.28 70.45 51.82 18.82 34.57 11.68 45.63 12.94 35.08 92.60 79.28 2.78 61.71 29.44 9.46 13.17
-sd 74.93 1.36 5.51 4.17 159.26 2.76 128.77 3.50 6.20 6.50 3.55 8.37 5.90 3.01 24.81 4.09 24.44 18.55 2.90 6.46 3.01 14.06 7.69 4.36 7.10 17.57 1.27 25.68 11.75 3.36 4.13
-n 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150 150
-corr 1.00 .784 .397 .862 .692 .625 .490 .374 .406 .371 .260 .257 .306 .118 -.148 .072 .202 .234 .198 .241 .205 -.054 .246 .166 .143 .155 -.122 .144 -.010 .135 .241
-corr 1.00 .333 .751 .549 .553 .447 .313 .304 .377 .204 .249 .258 .193 -.158 .119 .150 .216 .127 .209 .242 .046 .233 .120 .155 .107 -.126 .147 -.009 .134 .208
-corr 1.00 .469 .433 .381 .442 .307 .462 .391 .378 .293 .369 .191 -.306 .238 .204 .215 .295 .285 .157 .069 .241 .029 .060 .054 -.043 .124 -.069 .054 .136
-corr 1.00 .708 .663 .509 .419 .434 .432 .267 .255 .342 .132 -.192 .142 .228 .203 .248 .260 .200 -.051 .254 .136 .156 .109 -.126 .172 -.004 .157 .268
-corr 1.00 .787 .710 .567 .402 .511 .274 .285 .332 .154 -.096 .247 .253 .235 .245 .257 .261 -.048 .243 .119 .194 .164 -.108 .184 .011 .157 .235
-corr 1.00 .590 .646 .449 .505 .313 .322 .405 .148 -.117 .152 .294 .322 .252 .321 .267 -.055 .255 .118 .178 .137 -.110 .182 .004 .146 .216
-corr 1.00 .548 .343 .619 .296 .260 .456 .149 -.098 .252 .279 .267 .342 .361 .186 -.066 .215 .107 .148 .059 -.114 .156 -.035 .095 .220
-corr 1.00 .406 .509 .397 .236 .416 .037 -.179 .192 .334 .293 .277 .367 .162 -.150 .306 .171 .307 .173 -.128 .255 .075 .224 .315
-corr 1.00 .410 .497 .560 .574 .240 -.301 .204 .508 .351 .457 .428 .242 -.117 .367 .136 .191 .191 -.102 .215 .053 .185 .273
-corr 1.00 .328 .258 .534 .236 -.202 .200 .333 .209 .352 .375 .302 -.119 .272 .062 .203 .042 -.092 .220 .020 .158 .227
-corr 1.00 .439 .488 .323 -.213 .287 .507 .427 .493 .522 .298 -.142 .371 .109 .215 .213 -.048 .228 .009 .133 .267
-corr 1.00 .437 .381 -.158 .153 .403 .430 .383 .379 .150 -.141 .303 .115 .131 .155 -.170 .206 .039 .193 .254
-corr 1.00 .247 -.143 .182 .521 .364 .415 .688 .304 -.185 .327 .188 .211 .202 -.111 .272 .122 .226 .301
-corr 1.00 -.150 .229 .296 .249 .329 .255 .210 -.036 .252 .141 .230 .112 -.195 .309 .135 .250 .195
-corr 1.00 -.132 -.204 -.162 -.284 -.166 -.189 .294 -.339 -.094 -.218 -.144 .153 -.246 -.128 -.192 -.239
-corr 1.00 .151 .132 .166 .195 .387 -.214 .476 .154 .187 .167 -.236 .410 .316 .370 .245
-corr 1.00 .388 .479 .591 .294 -.171 .351 .102 .245 .180 .003 .274 .059 .178 .236
-corr 1.00 .408 .437 .276 -.153 .353 .251 .318 .229 -.111 .263 .042 .203 .349
-corr 1.00 .467 .234 -.249 .382 .199 .313 .197 -.117 .263 .047 .215 .318
-corr 1.00 .368 -.199 .441 .198 .224 .197 -.099 .329 .105 .256 .322
-corr 1.00 -.211 .473 .233 .253 .268 -.198 .397 .229 .309 .277
-corr 1.00 -.310 -.217 -.312 -.203 .227 -.296 -.260 -.276 -.321
-corr 1.00 .368 .350 .311 -.313 .578 .338 .521 .458
-corr 1.00 .415 .580 -.588 .545 .497 .635 .683
-corr 1.00 .570 -.386 .494 .340 .538 .524
-corr 1.00 -.366 .427 .299 .498 .506
-corr 1.00 -.684 -.620 -.746 -.568
-corr 1.00 .759 .900 .555
-corr 1.00 .814 .400
-corr 1.00 .621
-corr 1.00
-end data .
-
-factor matrix in (cor = *)
- / analysis = cdi_actws_16 rdls_actws_16 cdi_actws_20 cdi_actws_26 rdls_actws_26 bnt_actws_36 bnt_actws_56
- / format = default
- / criteria = factors (1)
- / extraction = pc
- / rotation = norotate
- / print = initial extraction .
-
-])
-
-AT_CHECK([pspp -O format=csv correlation-matrix.sps], [0], [dnl
-Table: Communalities
-,Initial,Extraction
-cdi_actws_16,1.000,.614
-rdls_actws_16,1.000,.660
-cdi_actws_20,1.000,.695
-cdi_actws_26,1.000,.650
-rdls_actws_26,1.000,.536
-bnt_actws_36,1.000,.443
-bnt_actws_56,1.000,.316
-
-Table: Total Variance Explained
-,Initial Eigenvalues,,,Extraction Sums of Squared Loadings,,
-,Total,% of Variance,Cumulative %,Total,% of Variance,Cumulative %
-1,3.914,55.9%,55.9%,3.914,55.9%,55.9%
-2,1.320,18.9%,74.8%,,,
-3,.716,10.2%,85.0%,,,
-4,.422,6.0%,91.0%,,,
-5,.278,4.0%,95.0%,,,
-6,.216,3.1%,98.1%,,,
-7,.135,1.9%,100.0%,,,
-
-Table: Component Matrix
-,Component
-,1
-cdi_actws_16,.784
-rdls_actws_16,.812
-cdi_actws_20,.834
-cdi_actws_26,.806
-rdls_actws_26,.732
-bnt_actws_36,.666
-bnt_actws_56,.562
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([FACTOR bad input])
-
-dnl Test for a crash
-AT_DATA([bad-input.sps], [dnl
-set format = F10.3.
-MATRIX DATA VARIABLES S1 ROWTYPE_ V1 TO V3 /SPLIT=S1.
-BEGIN DATA
-0 MEAN 2 4 3
-0 SD 1 2 3
-0 N 9 9 9
-0 KORR 1
-0 CORV .6 1
-0 CORR .7 .8 1
-1 MEAN 9 8 7
-1 SD 5 6 7
-1 N 9 9 9
-1 CORR 1
-X CORR .4 1
-1 CORR .3 .2 1
-END DATA.
-
-EXECUTE.
-
-FACTOR MATRIX IN (CORR =!*)
- /PRINT = CORRELATION
- .
-])
-
-AT_CHECK([pspp -O format=csv bad-input.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([FACTOR anti-image matrix])
-
-AT_DATA([anti-image-matrix.sps], [dnl
-SET FORMAT=F20.3 .
-matrix data
- variables = rowtype_ viq piq pa ran piatwr piatc
- / n = 476
- / format = lower diagonal .
-begin data
-mean 96.88 100.51 -1.73 -0.94 -2.52 -1.85
-sd 10.97 11.19 1.19 0.88 0.85 0.97
-corr 1.00
-corr 0.38 1.00
-corr 0.26 0.24 1.00
-corr 0.16 0.17 0.34 1.00
-corr 0.25 0.07 0.46 0.38 1.00
-corr 0.37 0.22 0.39 0.30 0.59 1.00
-end data.
-
-factor matrix = in (cor = *)
- / analysis = viq piq pa ran piatwr piatc
- / format = sort
- / extraction = pc
- / rotation = norotate
- / print = aic
-])
-
-AT_CHECK([pspp -O format=csv anti-image-matrix.sps], [0], [dnl
-Table: Anti-Image Matrices
-,,viq,piq,pa,ran,piatwr,piatc
-Anti-image Covariance,viq,.762,-.248,-.048,.008,-.031,-.143
-,piq,-.248,.807,-.117,-.081,.108,-.071
-,pa,-.048,-.117,.711,-.125,-.173,-.060
-,ran,.008,-.081,-.125,.808,-.143,-.035
-,piatwr,-.031,.108,-.173,-.143,.551,-.265
-,piatc,-.143,-.071,-.060,-.035,-.265,.581
-Anti-image Correlation,viq,.741,-.316,-.066,.011,-.048,-.215
-,piq,-.316,.624,-.154,-.100,.163,-.103
-,pa,-.066,-.154,.811,-.165,-.277,-.093
-,ran,.011,-.100,-.165,.825,-.214,-.051
-,piatwr,-.048,.163,-.277,-.214,.675,-.469
-,piatc,-.215,-.103,-.093,-.051,-.469,.729
-
-Table: Component Matrix
-,Component,,,,
-,1,2,3,4,5
-piatc,.774,.122,-.368,.365,-.322
-piatwr,.754,.418,.442,.219,-.115
-pa,.707,.124,-.117,-.161,.256
-piq,.456,-.733,.122,-.289,-.377
-viq,.589,-.539,.033,.298,.457
-ran,.592,.262,-.069,-.638,.096
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([FACTOR Kaiser-Meyer-Olkin])
-
-AT_DATA([kmo.sps], [dnl
-SET FORMAT=F20.3 .
-matrix data
- variables = rowtype_ viq piq pa ran piatwr piatc
- / n = 476
- / format = lower diagonal .
-begin data
-mean 96.88 100.51 -1.73 -0.94 -2.52 -1.85
-sd 10.97 11.19 1.19 0.88 0.85 0.97
-corr 1.00
-corr 0.38 1.00
-corr 0.26 0.24 1.00
-corr 0.16 0.17 0.34 1.00
-corr 0.25 0.07 0.46 0.38 1.00
-corr 0.37 0.22 0.39 0.30 0.59 1.00
-end data.
-
-factor matrix = in (cor = *)
- / analysis = viq piq pa ran piatwr piatc
- / extraction = pc
- / rotation = norotate
- / print = kmo
-])
-
-AT_CHECK([pspp -O format=csv kmo.sps], [0], [dnl
-Table: KMO and Bartlett's Test
-Kaiser-Meyer-Olkin Measure of Sampling Adequacy,,.730
-Bartlett's Test of Sphericity,Approx. Chi-Square,602.673
-,df,15
-,Sig.,.000
-
-Table: Component Matrix
-,Component,,,,
-,1,2,3,4,5
-viq,.589,-.539,.033,.298,.457
-piq,.456,-.733,.122,-.289,-.377
-pa,.707,.124,-.117,-.161,.256
-ran,.592,.262,-.069,-.638,.096
-piatwr,.754,.418,.442,.219,-.115
-piatc,.774,.122,-.368,.365,-.322
-])
-
-AT_CLEANUP
-
-AT_SETUP([FACTOR syntax errors])
-AT_DATA([factor.sps], [dnl
-DATA LIST LIST NOTABLE /x y.
-FACTOR VARIABLES=**.
-FACTOR MATRIX=**.
-FACTOR MATRIX=IN **.
-FACTOR MATRIX=IN(**).
-FACTOR MATRIX=IN(CORR **).
-FACTOR MATRIX=IN(CORR=**).
-FACTOR MATRIX=IN(CORR=* **).
-FACTOR **.
-FACTOR VARIABLES=x/ANALYSIS=**.
-FACTOR VARIABLES=x/PLOT=**.
-FACTOR VARIABLES=x/METHOD=**.
-FACTOR VARIABLES=x/ROTATION=PROMAX(**).
-FACTOR VARIABLES=x/ROTATION=PROMAX(123 **).
-FACTOR VARIABLES=x/ROTATION=**.
-FACTOR VARIABLES=x/CRITERIA=FACTORS **.
-FACTOR VARIABLES=x/CRITERIA=FACTORS(**).
-FACTOR VARIABLES=x/CRITERIA=FACTORS(123 **).
-FACTOR VARIABLES=x/CRITERIA=MINEIGEN **.
-FACTOR VARIABLES=x/CRITERIA=MINEIGEN(**).
-FACTOR VARIABLES=x/CRITERIA=MINEIGEN(123 **).
-FACTOR VARIABLES=x/CRITERIA=ECONVERGE **.
-FACTOR VARIABLES=x/CRITERIA=ECONVERGE(**).
-FACTOR VARIABLES=x/CRITERIA=ECONVERGE(123 **).
-FACTOR VARIABLES=x/CRITERIA=RCONVERGE **.
-FACTOR VARIABLES=x/CRITERIA=RCONVERGE(**).
-FACTOR VARIABLES=x/CRITERIA=RCONVERGE(123 **).
-FACTOR VARIABLES=x/CRITERIA=**.
-FACTOR VARIABLES=x/EXTRACTION=**.
-FACTOR VARIABLES=x/FORMAT=BLANK **.
-FACTOR VARIABLES=x/FORMAT=BLANK(**).
-FACTOR VARIABLES=x/FORMAT=BLANK(123 **).
-FACTOR VARIABLES=x/FORMAT=**.
-FACTOR VARIABLES=x/PRINT=**.
-FACTOR VARIABLES=x/MISSING=**.
-FACTOR VARIABLES=x/ **.
-FACTOR VARIABLES=x.
-])
-AT_CHECK([pspp -O format=csv factor.sps], [1], [dnl
-"factor.sps:2.18-2.19: error: FACTOR: Syntax error expecting variable name.
- 2 | FACTOR VARIABLES=**.
- | ^~"
-
-"factor.sps:3.15-3.16: error: FACTOR: Syntax error expecting `IN('.
- 3 | FACTOR MATRIX=**.
- | ^~"
-
-"factor.sps:4.15-4.19: error: FACTOR: Syntax error expecting `IN('.
- 4 | FACTOR MATRIX=IN **.
- | ^~~~~"
-
-"factor.sps:5.18-5.19: error: FACTOR: Matrix input for FACTOR must be either COV or CORR.
- 5 | FACTOR MATRIX=IN(**).
- | ^~"
-
-"factor.sps:6.23-6.24: error: FACTOR: Syntax error expecting `='.
- 6 | FACTOR MATRIX=IN(CORR **).
- | ^~"
-
-"factor.sps:7.23-7.24: error: FACTOR: Syntax error expecting a file name or handle name.
- 7 | FACTOR MATRIX=IN(CORR=**).
- | ^~"
-
-"factor.sps:8.25-8.26: error: FACTOR: Syntax error expecting `)'.
- 8 | FACTOR MATRIX=IN(CORR=* **).
- | ^~"
-
-"factor.sps:10.29-10.30: error: FACTOR: Syntax error expecting variable name.
- 10 | FACTOR VARIABLES=x/ANALYSIS=**.
- | ^~"
-
-"factor.sps:11.25-11.26: error: FACTOR: Syntax error expecting EIGEN.
- 11 | FACTOR VARIABLES=x/PLOT=**.
- | ^~"
-
-"factor.sps:12.27-12.28: error: FACTOR: Syntax error expecting COVARIANCE or CORRELATION.
- 12 | FACTOR VARIABLES=x/METHOD=**.
- | ^~"
-
-"factor.sps:13.36-13.37: error: FACTOR: Syntax error expecting integer.
- 13 | FACTOR VARIABLES=x/ROTATION=PROMAX(**).
- | ^~"
-
-"factor.sps:14.40-14.41: error: FACTOR: Syntax error expecting `)'.
- 14 | FACTOR VARIABLES=x/ROTATION=PROMAX(123 **).
- | ^~"
-
-"factor.sps:15.29-15.30: error: FACTOR: Syntax error expecting DEFAULT, VARIMAX, EQUAMAX, QUARTIMAX, PROMAX, or NOROTATE.
- 15 | FACTOR VARIABLES=x/ROTATION=**.
- | ^~"
-
-"factor.sps:16.37-16.38: error: FACTOR: Syntax error expecting `('.
- 16 | FACTOR VARIABLES=x/CRITERIA=FACTORS **.
- | ^~"
-
-"factor.sps:17.37-17.38: error: FACTOR: Syntax error expecting integer.
- 17 | FACTOR VARIABLES=x/CRITERIA=FACTORS(**).
- | ^~"
-
-"factor.sps:18.41-18.42: error: FACTOR: Syntax error expecting `)'.
- 18 | FACTOR VARIABLES=x/CRITERIA=FACTORS(123 **).
- | ^~"
-
-"factor.sps:19.38-19.39: error: FACTOR: Syntax error expecting `('.
- 19 | FACTOR VARIABLES=x/CRITERIA=MINEIGEN **.
- | ^~"
-
-"factor.sps:20.38-20.39: error: FACTOR: Syntax error expecting number.
- 20 | FACTOR VARIABLES=x/CRITERIA=MINEIGEN(**).
- | ^~"
-
-"factor.sps:21.42-21.43: error: FACTOR: Syntax error expecting `)'.
- 21 | FACTOR VARIABLES=x/CRITERIA=MINEIGEN(123 **).
- | ^~"
-
-"factor.sps:22.39-22.40: error: FACTOR: Syntax error expecting `('.
- 22 | FACTOR VARIABLES=x/CRITERIA=ECONVERGE **.
- | ^~"
-
-"factor.sps:23.39-23.40: error: FACTOR: Syntax error expecting number.
- 23 | FACTOR VARIABLES=x/CRITERIA=ECONVERGE(**).
- | ^~"
-
-"factor.sps:24.43-24.44: error: FACTOR: Syntax error expecting `)'.
- 24 | FACTOR VARIABLES=x/CRITERIA=ECONVERGE(123 **).
- | ^~"
-
-"factor.sps:25.39-25.40: error: FACTOR: Syntax error expecting `('.
- 25 | FACTOR VARIABLES=x/CRITERIA=RCONVERGE **.
- | ^~"
-
-"factor.sps:26.39-26.40: error: FACTOR: Syntax error expecting number.
- 26 | FACTOR VARIABLES=x/CRITERIA=RCONVERGE(**).
- | ^~"
-
-"factor.sps:27.43-27.44: error: FACTOR: Syntax error expecting `)'.
- 27 | FACTOR VARIABLES=x/CRITERIA=RCONVERGE(123 **).
- | ^~"
-
-"factor.sps:28.29-28.30: error: FACTOR: Syntax error expecting FACTORS, MINEIGEN, ECONVERGE, RCONVERGE, ITERATE, or DEFAULT.
- 28 | FACTOR VARIABLES=x/CRITERIA=**.
- | ^~"
-
-"factor.sps:29.31-29.32: error: FACTOR: Syntax error expecting PAF, PC, PA1, or DEFAULT.
- 29 | FACTOR VARIABLES=x/EXTRACTION=**.
- | ^~"
-
-"factor.sps:30.33-30.34: error: FACTOR: Syntax error expecting `('.
- 30 | FACTOR VARIABLES=x/FORMAT=BLANK **.
- | ^~"
-
-"factor.sps:31.33-31.34: error: FACTOR: Syntax error expecting number.
- 31 | FACTOR VARIABLES=x/FORMAT=BLANK(**).
- | ^~"
-
-"factor.sps:32.37-32.38: error: FACTOR: Syntax error expecting `)'.
- 32 | FACTOR VARIABLES=x/FORMAT=BLANK(123 **).
- | ^~"
-
-"factor.sps:33.27-33.28: error: FACTOR: Syntax error expecting SORT, BLANK, or DEFAULT.
- 33 | FACTOR VARIABLES=x/FORMAT=**.
- | ^~"
-
-"factor.sps:34.26-34.27: error: FACTOR: Syntax error expecting one of the following: UNIVARIATE, DET, AIC, SIG, CORRELATION, COVARIANCE, ROTATION, EXTRACTION, INITIAL, KMO, ALL, DEFAULT.
- 34 | FACTOR VARIABLES=x/PRINT=**.
- | ^~"
-
-"factor.sps:35.28-35.29: error: FACTOR: Syntax error expecting INCLUDE, EXCLUDE, LISTWISE, PAIRRWISE, or MEANSUB.
- 35 | FACTOR VARIABLES=x/MISSING=**.
- | ^~"
-
-"factor.sps:36.21-36.22: error: FACTOR: Syntax error expecting one of the following: ANALYSIS, PLOT, METHOD, ROTATION, CRITERIA, EXTRACTION, FORMAT, PRINT, MISSING.
- 36 | FACTOR VARIABLES=x/ **.
- | ^~"
-
-"factor.sps:37.18: warning: FACTOR: Factor analysis on a single variable is not useful.
- 37 | FACTOR VARIABLES=x.
- | ^"
-
-error: FACTOR: At end of input: Syntax error expecting `BEGIN DATA'.
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([FLIP command])
-
-AT_SETUP([FLIP with NEWNAMES])
-AT_DATA([flip.sps], [dnl
-data list notable /N 1 (a) a b c d 2-9.
-list.
-begin data.
-v 1 2 3 4 5
-w 6 7 8 910
-x1112131415
-y1617181920
-z2122232425
-end data.
-temporary.
-compute e = a.
-flip newnames=n.
-list.
-flip.
-list.
-])
-AT_CHECK([pspp -O format=csv flip.sps], [0], [dnl
-Table: Data List
-N,a,b,c,d
-v,1,2,3,4
-w,6,7,8,9
-x,11,12,13,14
-y,16,17,18,19
-z,21,22,23,24
-
-"flip.sps:12.1-12.4: warning: FLIP: FLIP ignores TEMPORARY. Temporary transformations will be made permanent.
- 12 | flip newnames=n.
- | ^~~~"
-
-Table: Data List
-CASE_LBL,v,w,x,y,z
-a,1.00,6.00,11.00,16.00,21.00
-b,2.00,7.00,12.00,17.00,22.00
-c,3.00,8.00,13.00,18.00,23.00
-d,4.00,9.00,14.00,19.00,24.00
-e,1.00,6.00,11.00,16.00,21.00
-
-Table: Data List
-CASE_LBL,a,b,c,d,e
-v,1.00,2.00,3.00,4.00,1.00
-w,6.00,7.00,8.00,9.00,6.00
-x,11.00,12.00,13.00,14.00,11.00
-y,16.00,17.00,18.00,19.00,16.00
-z,21.00,22.00,23.00,24.00,21.00
-])
-AT_CLEANUP
-
-AT_SETUP([FLIP without NEWNAMES])
-AT_DATA([flip.sps], [dnl
-data list list notable /v1 to v10.
-format all(f2).
-begin data.
-1 2 3 4 5 6 7 8 9 10
-4 5 6 7 8 9 10 11 12 13
-end data.
-
-list.
-
-flip.
-list.
-])
-AT_CHECK([pspp -O format=csv flip.sps], [0], [dnl
-Table: Data List
-v1,v2,v3,v4,v5,v6,v7,v8,v9,v10
-1,2,3,4,5,6,7,8,9,10
-4,5,6,7,8,9,10,11,12,13
-
-Table: Data List
-CASE_LBL,VAR000,VAR001
-v1,1.00,4.00
-v2,2.00,5.00
-v3,3.00,6.00
-v4,4.00,7.00
-v5,5.00,8.00
-v6,6.00,9.00
-v7,7.00,10.00
-v8,8.00,11.00
-v9,9.00,12.00
-v10,10.00,13.00
-])
-AT_CLEANUP
-
-
-
-
-AT_SETUP([FLIP badly formed])
-
-AT_DATA([flip.sps], [dnl
-data list notable /N 1 (a) a b c d 2-9.
-
-flip newnames=n.
-list.
-flip.
-])
-
-AT_CHECK([pspp -O format=csv flip.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([FLIP with invalid variable names])
-
-AT_DATA([flip.sps], [dnl
-data list notable list /N (a3) a b c d *.
-begin data.
-"" 1 2 3 4
-BY 1 2 3 4
-end data.
-
-flip newnames=n.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv flip.sps], [0], [dnl
-Table: Data List
-CASE_LBL,v,BY1
-a,1.00,1.00
-b,2.00,2.00
-c,3.00,3.00
-d,4.00,4.00
-])
-
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([FREQUENCIES procedure])
-
-AT_SETUP([FREQUENCIES string variable])
-AT_DATA([frequencies.sps],
- [DATA LIST FREE/
- name (A8) value * quantity .
-BEGIN DATA.
-foo 1 5
-bar 2 6
-baz 1 9
-quux 3 1
-bar 1 2
-baz 4 3
-baz 1 4
-baz 1 1
-foo 6 0
-quux 5 8
-END DATA.
-EXECUTE.
-
-FREQUENCIES /VAR = name/ORDER=ANALYSIS.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: name
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,bar,2,20.0%,20.0%,20.0%
-,baz,4,40.0%,40.0%,60.0%
-,foo,2,20.0%,20.0%,80.0%
-,quux,2,20.0%,20.0%,100.0%
-Total,,10,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES with SPLIT FILE - LAYERED])
-AT_DATA([frequencies.sps], [dnl
-DATA LIST LIST NOTABLE/name (A8) value quantity.
-BEGIN DATA.
-foo 1 5
-bar 2 6
-baz 1 9
-quux 3 1
-bar 1 2
-baz 4 3
-baz 1 4
-baz 1 1
-foo 6 0
-quux 5 8
-END DATA.
-EXECUTE.
-
-SORT CASES BY name.
-SPLIT FILE BY name.
-FREQUENCIES /VARIABLES=value quantity /FORMAT NOTABLE.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,name,,,,,,,
-,,bar,,baz,,foo,,quux,
-,,value,quantity,value,quantity,value,quantity,value,quantity
-N,Valid,2,2,4,4,2,2,2,2
-,Missing,0,0,0,0,0,0,0,0
-Mean,,1.50,4.00,1.75,4.25,3.50,2.50,4.00,4.50
-Std Dev,,.71,2.83,1.50,3.40,3.54,3.54,1.41,4.95
-Minimum,,1.00,2.00,1.00,1.00,1.00,.00,3.00,1.00
-Maximum,,2.00,6.00,4.00,9.00,6.00,5.00,5.00,8.00
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES with SPLIT FILE - SEPARATE])
-AT_DATA([frequencies.sps], [dnl
-DATA LIST LIST NOTABLE/name (A8) value quantity.
-BEGIN DATA.
-foo 1 5
-bar 2 6
-baz 1 9
-quux 3 1
-bar 1 2
-baz 4 3
-baz 1 4
-baz 1 1
-foo 6 0
-quux 5 8
-END DATA.
-EXECUTE.
-
-SORT CASES BY name.
-SPLIT FILE SEPARATE BY name.
-FREQUENCIES /VARIABLES=value quantity /FORMAT NOTABLE.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Split Values
-Variable,Value
-name,bar
-
-Table: Statistics
-,,value,quantity
-N,Valid,2,2
-,Missing,0,0
-Mean,,1.50,4.00
-Std Dev,,.71,2.83
-Minimum,,1.00,2.00
-Maximum,,2.00,6.00
-
-Table: Split Values
-Variable,Value
-name,baz
-
-Table: Statistics
-,,value,quantity
-N,Valid,4,4
-,Missing,0,0
-Mean,,1.75,4.25
-Std Dev,,1.50,3.40
-Minimum,,1.00,1.00
-Maximum,,4.00,9.00
-
-Table: Split Values
-Variable,Value
-name,foo
-
-Table: Statistics
-,,value,quantity
-N,Valid,2,2
-,Missing,0,0
-Mean,,3.50,2.50
-Std Dev,,3.54,3.54
-Minimum,,1.00,.00
-Maximum,,6.00,5.00
-
-Table: Split Values
-Variable,Value
-name,quux
-
-Table: Statistics
-,,value,quantity
-N,Valid,2,2
-,Missing,0,0
-Mean,,4.00,4.50
-Std Dev,,1.41,4.95
-Minimum,,3.00,1.00
-Maximum,,5.00,8.00
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES with SPLIT FILE - LAYERED - unsorted data])
-AT_DATA([frequencies.sps], [dnl
-DATA LIST LIST NOTABLE/name (A8) value quantity.
-BEGIN DATA.
-foo 1 5
-bar 2 6
-baz 1 9
-quux 3 1
-baz 4 3
-bar 1 2
-baz 1 1
-foo 6 0
-baz 1 4
-quux 5 8
-END DATA.
-EXECUTE.
-
-SPLIT FILE BY name.
-FREQUENCIES /VARIABLES=value quantity /FORMAT NOTABLE.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = baz "
-
-"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = bar "
-
-"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = baz "
-
-"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = foo "
-
-"frequencies.sps:17: warning: FREQUENCIES: When SPLIT FILE is in effect, the input data must be sorted by the split variables (for example, using SORT CASES), but multiple runs of cases with the same split values were found separated by cases with different values. Each run will be analyzed separately. The duplicate split values are: name = baz "
-
-Table: Statistics
-,,name,,,,,,,,,,,,,,,,,,,
-,,foo,,bar,,baz,,quux,,baz,,bar,,baz,,foo,,baz,,quux,
-,,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity,value,quantity
-N,Valid,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
-,Missing,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
-Mean,,1.00,5.00,2.00,6.00,1.00,9.00,3.00,1.00,4.00,3.00,1.00,2.00,1.00,1.00,6.00,.00,1.00,4.00,5.00,8.00
-Std Dev,,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN,NaN
-Minimum,,1.00,5.00,2.00,6.00,1.00,9.00,3.00,1.00,4.00,3.00,1.00,2.00,1.00,1.00,6.00,.00,1.00,4.00,5.00,8.00
-Maximum,,1.00,5.00,2.00,6.00,1.00,9.00,3.00,1.00,4.00,3.00,1.00,2.00,1.00,1.00,6.00,.00,1.00,4.00,5.00,8.00
-
-frequencies.sps:17: warning: FREQUENCIES: Suppressed 1 additional warning about duplicate split values.
-])
-AT_CLEANUP
-
-# Tests for a bug where pspp would crash if two FREQUENCIES commands
-# existed in a input file.
-AT_SETUP([FREQUENCIES two runs crash])
-AT_DATA([frequencies.sps],
- [data list free /v1 v2.
-begin data.
-0 1
-2 3
-4 5
-3 4
-end data.
-
-frequencies v1 v2/statistics=none/ORDER=VARIABLE.
-frequencies v1 v2/statistics=none.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: v1
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,.00,1,25.0%,25.0%,25.0%
-,2.00,1,25.0%,25.0%,50.0%
-,3.00,1,25.0%,25.0%,75.0%
-,4.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-
-Table: v2
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1,25.0%,25.0%,25.0%
-,3.00,1,25.0%,25.0%,50.0%
-,4.00,1,25.0%,25.0%,75.0%
-,5.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-
-Table: v1
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,.00,1,25.0%,25.0%,25.0%
-,2.00,1,25.0%,25.0%,50.0%
-,3.00,1,25.0%,25.0%,75.0%
-,4.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-
-Table: v2
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1,25.0%,25.0%,25.0%
-,3.00,1,25.0%,25.0%,50.0%
-,4.00,1,25.0%,25.0%,75.0%
-,5.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-])
-AT_CLEANUP
-
-# Test that the LIMIT specification works.
-AT_SETUP([FREQUENCIES with LIMIT])
-AT_DATA([frequencies.sps],
- [data list free /v1 v2.
-begin data.
-0 1
-2 5
-4 3
-3 5
-end data.
-
-frequencies v1 v2/statistics=none/FORMAT=LIMIT(3).
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: v2
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1,25.0%,25.0%,25.0%
-,3.00,1,25.0%,25.0%,50.0%
-,5.00,2,50.0%,50.0%,100.0%
-Total,,4,100.0%,,
-])
-AT_CLEANUP
-
-# Tests for a bug where PSPP would crash when a FREQUENCIES command
-# was used with the HTML output driver.
-AT_SETUP([FREQUENCIES HTML output crash])
-AT_DATA([frequencies.sps],
- [data list free /v1 v2.
-begin data.
-0 1
-2 3
-4 5
-3 4
-end data.
-
-list.
-
-frequencies v1/statistics=none.
-])
-AT_CHECK([pspp -o - -O format=csv -o pspp.html frequencies.sps], [0],
- [Table: Data List
-v1,v2
-.00,1.00
-2.00,3.00
-4.00,5.00
-3.00,4.00
-
-Table: v1
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,.00,1,25.0%,25.0%,25.0%
-,2.00,1,25.0%,25.0%,50.0%
-,3.00,1,25.0%,25.0%,75.0%
-,4.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-])
-AT_CHECK([test -s pspp.html])
-AT_CLEANUP
-
-# Tests for a bug which crashed PSPP when a piechart with too many
-# segments was requested.
-AT_SETUP([FREQUENCIES pie chart crash])
-AT_DATA([frequencies.sps],
- [data list list /x * w *.
-begin data.
-1 4
-34 10
--9 15
-232 6
-11 4
-134 1
-9 5
-32 16
--2 6
-2 16
-20 6
-end data.
-
-weight by w.
-
-frequencies /x /format=notable /statistics=none
- /piechart.
-])
-# Cannot use the CSV driver for this because it does not output charts
-# at all.
-AT_CHECK([pspp frequencies.sps], [0], [dnl
-Reading free-form data from INLINE.
-+--------+------+
-|Variable|Format|
-+--------+------+
-|x |F8.0 |
-|w |F8.0 |
-+--------+------+
-])
-AT_CLEANUP
-
-dnl Check that histogram subcommand runs wihout crashing
-AT_SETUP([FREQUENCIES histogram crash])
-AT_DATA([frequencies.sps],
- [data list notable list /x * w *.
-begin data.
-1 4
-34 10
--9 15
-232 6
-11 4
-134 1
-9 5
-32 16
--2 6
-2 16
-20 6
-end data.
-
-weight by w.
-
-frequencies /x
- /format=notable
- /statistics=none
- /histogram=minimum(0) maximum(50) percent(5) normal.
-])
-# Cannot use the CSV driver for this because it does not output charts
-# at all.
-AT_CHECK([pspp -O format=pdf frequencies.sps], [0], [ignore], [ignore])
-AT_CLEANUP
-
-# Tests for a bug which crashed PSPP when the median and a histogram
-# were both requested.
-AT_SETUP([FREQUENCIES median with histogram crash])
-AT_DATA([frequencies.sps], [dnl
-data list list notable /x.
-begin data.
-1
-end data.
-
-frequencies /x /histogram /STATISTICS=median.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [ignore])
-dnl Ignore output - No crash test.
-AT_CLEANUP
-
-# Tests for a bug which caused FREQUENCIES following TEMPORARY to
-# crash (bug #11492).
-AT_SETUP([FREQUENCIES crash after TEMPORARY])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST /SEX (A1) X *.
-BEGIN DATA.
-M 31
-F 21
-M 41
-F 31
-M 13
-F 12
-M 14
-F 13
-END DATA.
-
-
-TEMPORARY
-SELECT IF SEX EQ 'F'
-FREQUENCIES /X .
-
-FINISH
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt frequencies.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-SEX,A1
-X,F8.0
-
-Table: Statistics
-,,X
-N,Valid,4
-,Missing,0
-Mean,,19.25
-Std Dev,,8.81
-Minimum,,12.00
-Maximum,,31.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,12.00,1,25.0%,25.0%,25.0%
-,13.00,1,25.0%,25.0%,50.0%
-,21.00,1,25.0%,25.0%,75.0%
-,31.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-])
-AT_CLEANUP
-
-m4_define([FREQUENCIES_NTILES_OUTPUT], [dnl
-Table: Statistics
-,,x,y
-N,Valid,5,5
-,Missing,0,0
-Mean,,3.00,30.00
-Std Dev,,1.58,15.81
-Minimum,,1.00,10.00
-Maximum,,5.00,50.00
-Percentiles,0,1.00,10.00
-,25,2.00,20.00
-,33,2.33,23.33
-,50,3.00,30.00
-,67,3.67,36.67
-,75,4.00,40.00
-,100,5.00,50.00
-])
-AT_SETUP([FREQUENCIES basic percentiles])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /x y.
-BEGIN DATA.
-1 10
-2 20
-3 30
-4 40
-5 50
-END DATA.
-
-FREQUENCIES
- VAR=x y
- /FORMAT=NOTABLE
- /PERCENTILES = 0 25 33.333 50 66.666 75 100.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0],
- [FREQUENCIES_NTILES_OUTPUT])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES basic n-tiles])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /x y.
-BEGIN DATA.
-1 10
-2 20
-3 30
-4 40
-5 50
-END DATA.
-
-FREQUENCIES
- VAR=x y
- /FORMAT=NOTABLE
- /NTILES = 3
- /NTILES = 4.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0],
- [FREQUENCIES_NTILES_OUTPUT])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES compatibility percentiles])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /X * .
-BEGIN DATA.
-1
-2
-3
-4
-5
-END DATA.
-
-FREQUENCIES
- VAR=x
- /ALGORITHM=COMPATIBLE
- /PERCENTILES = 0 25 50 75 100.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,X
-N,Valid,5
-,Missing,0
-Mean,,3.00
-Std Dev,,1.58
-Minimum,,1.00
-Maximum,,5.00
-Percentiles,0,1.00
-,25,1.50
-,50,3.00
-,75,4.50
-,100,5.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1,20.0%,20.0%,20.0%
-,2.00,1,20.0%,20.0%,40.0%
-,3.00,1,20.0%,20.0%,60.0%
-,4.00,1,20.0%,20.0%,80.0%
-,5.00,1,20.0%,20.0%,100.0%
-Total,,5,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES enhanced percentiles])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /X * .
-BEGIN DATA.
-1
-2
-3
-4
-5
-END DATA.
-
-FREQUENCIES
- VAR=x
- /PERCENTILES = 0 25 50 75 100.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,X
-N,Valid,5
-,Missing,0
-Mean,,3.00
-Std Dev,,1.58
-Minimum,,1.00
-Maximum,,5.00
-Percentiles,0,1.00
-,25,2.00
-,50,3.00
-,75,4.00
-,100,5.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1,20.0%,20.0%,20.0%
-,2.00,1,20.0%,20.0%,40.0%
-,3.00,1,20.0%,20.0%,60.0%
-,4.00,1,20.0%,20.0%,80.0%
-,5.00,1,20.0%,20.0%,100.0%
-Total,,5,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES enhanced percentiles, weighted])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /X * F *.
-BEGIN DATA.
-1 2
-2 2
-3 2
-4 1
-4 1
-5 1
-5 1
-END DATA.
-
-WEIGHT BY f.
-
-FREQUENCIES
- VAR=x
- /PERCENTILES = 0 25 50 75 100.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,X
-N,Valid,10.00
-,Missing,.00
-Mean,,3.00
-Std Dev,,1.49
-Minimum,,1.00
-Maximum,,5.00
-Percentiles,0,1.00
-,25,2.00
-,50,3.00
-,75,4.00
-,100,5.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,2.00,20.0%,20.0%,20.0%
-,2.00,2.00,20.0%,20.0%,40.0%
-,3.00,2.00,20.0%,20.0%,60.0%
-,4.00,2.00,20.0%,20.0%,80.0%
-,5.00,2.00,20.0%,20.0%,100.0%
-Total,,10.00,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES enhanced percentiles, weighted (2)])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /X * F *.
-BEGIN DATA.
-1 1
-3 2
-4 1
-5 1
-5 1
-END DATA.
-
-WEIGHT BY f.
-
-FREQUENCIES
- VAR=x
- /PERCENTILES = 0 25 50 75 100.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,X
-N,Valid,6.00
-,Missing,.00
-Mean,,3.50
-Std Dev,,1.52
-Minimum,,1.00
-Maximum,,5.00
-Percentiles,0,1.00
-,25,3.00
-,50,3.50
-,75,4.75
-,100,5.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1.00,16.7%,16.7%,16.7%
-,3.00,2.00,33.3%,33.3%,50.0%
-,4.00,1.00,16.7%,16.7%,66.7%
-,5.00,2.00,33.3%,33.3%,100.0%
-Total,,6.00,100.0%,,
-])
-AT_CLEANUP
-
-dnl Data for this test case from Fabio Bordignon <bordignon@demos.it>.
-AT_SETUP([FREQUENCIES enhanced percentiles, weighted (3)])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /X * F *.
-BEGIN DATA.
-1 7
-2 16
-3 12
-4 5
-END DATA.
-
-WEIGHT BY f.
-
-FREQUENCIES
- VAR=x
- /PERCENTILES = 0 25 50 75 100.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,X
-N,Valid,40.00
-,Missing,.00
-Mean,,2.38
-Std Dev,,.93
-Minimum,,1.00
-Maximum,,4.00
-Percentiles,0,1.00
-,25,2.00
-,50,2.00
-,75,3.00
-,100,4.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,7.00,17.5%,17.5%,17.5%
-,2.00,16.00,40.0%,40.0%,57.5%
-,3.00,12.00,30.0%,30.0%,87.5%
-,4.00,5.00,12.5%,12.5%,100.0%
-Total,,40.00,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES enhanced percentiles, weighted, missing values])
-AT_DATA([frequencies.sps],
- [DATA LIST LIST notable /X * F *.
-BEGIN DATA.
-1 1
-3 2
-4 1
-5 1
-5 1
-99 4
-END DATA.
-
-MISSING VALUE x (99.0) .
-WEIGHT BY f.
-
-FREQUENCIES
- VAR=x
- /PERCENTILES = 0 25 50 75 100.
-])
-
-AT_CHECK([pspp -O format=csv frequencies.sps], [0], [dnl
-Table: Statistics
-,,X
-N,Valid,6.00
-,Missing,4.00
-Mean,,3.50
-Std Dev,,1.52
-Minimum,,1.00
-Maximum,,5.00
-Percentiles,0,1.00
-,25,3.00
-,50,3.50
-,75,4.75
-,100,5.00
-
-Table: X
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1.00,10.0%,16.7%,16.7%
-,3.00,2.00,20.0%,33.3%,50.0%
-,4.00,1.00,10.0%,16.7%,66.7%
-,5.00,2.00,20.0%,33.3%,100.0%
-Missing,99.00,4.00,40.0%,,
-Total,,10.00,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES dichotomous histogram])
-AT_DATA([frequencies.sps], [dnl
-data list notable list /d4 *.
-begin data.
-0
-0
-0
-1
-0
-0
-0
-0
-1
-0
-0
-0
-0
-0
-1
-2
-0
-end data.
-
-FREQUENCIES
- /VARIABLES = d4
- /FORMAT=AVALUE TABLE
- /HISTOGRAM=NORMAL
- .
-])
-
-AT_CHECK([pspp frequencies.sps], [0], [ignore])
-AT_CLEANUP
-
-
-AT_SETUP([FREQUENCIES median])
-AT_DATA([median.sps], [dnl
-data list notable list /x *.
-begin data.
-1
-2
-3000000
-end data.
-
-FREQUENCIES
- /VARIABLES = x
- /STATISTICS = MEDIAN
- .
-])
-
-AT_CHECK([pspp median.sps -O format=csv], [0], [dnl
-Table: Statistics
-,,x
-N,Valid,3
-,Missing,0
-Median,,2.00
-
-Table: x
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,1,33.3%,33.3%,33.3%
-,2.00,1,33.3%,33.3%,66.7%
-,3000000,1,33.3%,33.3%,100.0%
-Total,,3,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES variance])
-AT_DATA([variance.sps], [dnl
-data list notable list /forename (A12) height.
-begin data.
-Ahmed 188
-bertram 167
-Catherine 134
-David 109
-end data.
-
-FREQUENCIES
- /VARIABLES = height
- /STATISTICS = VARIANCE.
-])
-
-AT_CHECK([pspp variance.sps -O format=csv], [0], [dnl
-Table: Statistics
-,,height
-N,Valid,4
-,Missing,0
-Variance,,1223.00
-
-Table: height
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,109.00,1,25.0%,25.0%,25.0%
-,134.00,1,25.0%,25.0%,50.0%
-,167.00,1,25.0%,25.0%,75.0%
-,188.00,1,25.0%,25.0%,100.0%
-Total,,4,100.0%,,
-])
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES default statistics])
-AT_DATA([median.sps], [dnl
-data list notable list /x *.
-begin data.
-10
-20
-3000000
-end data.
-
-FREQUENCIES
- /VARIABLES = x
- /STATISTICS
- .
-
-FREQUENCIES
- /VARIABLES = x
- /STATISTICS = DEFAULT
- .
-])
-
-AT_CHECK([pspp median.sps -o pspp.csv -o pspp.txt])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Statistics
-,,x
-N,Valid,3
-,Missing,0
-Mean,,1000010
-Std Dev,,1732042
-Minimum,,10.00
-Maximum,,3000000
-
-Table: x
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,10.00,1,33.3%,33.3%,33.3%
-,20.00,1,33.3%,33.3%,66.7%
-,3000000,1,33.3%,33.3%,100.0%
-Total,,3,100.0%,,
-
-Table: Statistics
-,,x
-N,Valid,3
-,Missing,0
-Mean,,1000010
-Std Dev,,1732042
-Minimum,,10.00
-Maximum,,3000000
-
-Table: x
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,10.00,1,33.3%,33.3%,33.3%
-,20.00,1,33.3%,33.3%,66.7%
-,3000000,1,33.3%,33.3%,100.0%
-Total,,3,100.0%,,
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([FREQUENCIES no valid data])
-AT_DATA([empty.sps], [dnl
-data list notable list /x *.
-begin data.
-.
-.
-.
-end data.
-
-FREQUENCIES
- /VARIABLES = x
- /STATISTICS = ALL
- .
-])
-
-AT_CHECK([pspp empty.sps -O format=csv], [0], [dnl
-Table: Statistics
-,,x
-N,Valid,0
-,Missing,3
-Mean,,. @&t@
-S.E. Mean,,. @&t@
-Median,,. @&t@
-Mode,,. @&t@
-Std Dev,,. @&t@
-Variance,,. @&t@
-Kurtosis,,. @&t@
-S.E. Kurt,,. @&t@
-Skewness,,. @&t@
-S.E. Skew,,. @&t@
-Range,,. @&t@
-Minimum,,. @&t@
-Maximum,,. @&t@
-Sum,,. @&t@
-
-Table: x
-,,Frequency,Percent
-Missing,. ,3,100.0%
-Total,,3,.0%
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([FREQUENCIES histogram no valid cases])
-AT_DATA([empty.sps], [dnl
-data list notable list /x w *.
-begin data.
-1 .
-2 .
-3 .
-end data.
-
-weight by w.
-
-FREQUENCIES
- /VARIABLES = x
- /histogram
- .
-])
-
-AT_CHECK([pspp empty.sps -O format=csv], [0], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES percentiles + histogram bug#48128])
-AT_DATA([bug.sps], [dnl
-SET FORMAT=F8.0.
-
-INPUT PROGRAM.
- LOOP I=1 TO 10.
- COMPUTE SCORE=EXP(NORMAL(1)).
- END CASE.
- END LOOP.
- END FILE.
-END INPUT PROGRAM.
-
-FREQUENCIES VARIABLES=SCORE
-/FORMAT=NOTABLE
-/STATISTICS=ALL
-/PERCENTILES=1 10 20 30 40 50 60 70 80 90 99
-/HISTOGRAM.
-
-])
-
-AT_CHECK([pspp bug.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([FREQUENCIES vs. missing weights])
-AT_DATA([warn.sps], [dnl
-data list notable list /x w .
-begin data.
-1 1
-2 1
-1 1
-3 1
-3 .
-4 .
-end data.
-
-weight by w.
-
-frequencies /variables=x.
-])
-
-AT_CHECK([pspp warn.sps -O format=csv], [0], [dnl
-"warn.sps:13: warning: FREQUENCIES: At least one case in the data file had a weight value that was user-missing, system-missing, zero, or negative. These case(s) were ignored."
-
-Table: Statistics
-,,x
-N,Valid,4.00
-,Missing,.00
-Mean,,1.75
-Std Dev,,.96
-Minimum,,1.00
-Maximum,,4.00
-
-Table: x
-,,Frequency,Percent,Valid Percent,Cumulative Percent
-Valid,1.00,2.00,50.0%,50.0%,50.0%
-,2.00,1.00,25.0%,25.0%,75.0%
-,3.00,1.00,25.0%,25.0%,100.0%
-,4.00,.00,.0%,.0%,100.0%
-Total,,4.00,100.0%,,
-])
-
-AT_CLEANUP
-
-AT_SETUP([FREQUENCIES syntax errors])
-AT_DATA([frequencies.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-FREQUENCIES VARIABLES **.
-FREQUENCIES **.
-FREQUENCIES x/STATISTICS **.
-FREQUENCIES x/PERCENTILES **.
-FREQUENCIES x/FORMAT LIMIT **.
-FREQUENCIES x/FORMAT LIMIT(**).
-FREQUENCIES x/FORMAT LIMIT(5 **).
-FREQUENCIES x/FORMAT **.
-FREQUENCIES x/NTILES **.
-FREQUENCIES x/ALGORITHM **.
-FREQUENCIES x/HISTOGRAM FREQ(**).
-FREQUENCIES x/HISTOGRAM FREQ(5 **).
-FREQUENCIES x/HISTOGRAM PERCENT(**).
-FREQUENCIES x/HISTOGRAM PERCENT(5 **).
-FREQUENCIES x/HISTOGRAM MINIMUM(**).
-FREQUENCIES x/HISTOGRAM MINIMUM(5 **).
-FREQUENCIES x/HISTOGRAM MAXIMUM(**).
-FREQUENCIES x/HISTOGRAM MAXIMUM(5 **).
-FREQUENCIES x/HISTOGRAM MINIMUM(5) MAXIMUM(1).
-FREQUENCIES x/HISTOGRAM MAXIMUM(5) MINIMUM(10).
-FREQUENCIES x/HISTOGRAM **.
-FREQUENCIES x/PIECHART MINIMUM(**).
-FREQUENCIES x/PIECHART MINIMUM(5 **).
-FREQUENCIES x/PIECHART MAXIMUM(**).
-FREQUENCIES x/PIECHART MAXIMUM(5 **).
-FREQUENCIES x/PIECHART MINIMUM(5) MAXIMUM(1).
-FREQUENCIES x/PIECHART MAXIMUM(5) MINIMUM(10).
-FREQUENCIES x/PIECHART **.
-FREQUENCIES x/BARCHART FREQ(**).
-FREQUENCIES x/BARCHART FREQ(5 **).
-FREQUENCIES x/BARCHART PERCENT(**).
-FREQUENCIES x/BARCHART PERCENT(5 **).
-FREQUENCIES x/BARCHART MINIMUM(**).
-FREQUENCIES x/BARCHART MINIMUM(5 **).
-FREQUENCIES x/BARCHART MAXIMUM(**).
-FREQUENCIES x/BARCHART MAXIMUM(5 **).
-FREQUENCIES x/BARCHART MINIMUM(5) MAXIMUM(1).
-FREQUENCIES x/BARCHART MAXIMUM(5) MINIMUM(10).
-FREQUENCIES x/BARCHART **.
-FREQUENCIES x/MISSING **.
-FREQUENCIES x/ORDER **.
-FREQUENCIES x/ **.
-])
-AT_CHECK([pspp -O format=csv frequencies.sps], [1], [dnl
-"frequencies.sps:2.23-2.24: error: FREQUENCIES: Syntax error expecting `='.
- 2 | FREQUENCIES VARIABLES **.
- | ^~"
-
-"frequencies.sps:3.13-3.14: error: FREQUENCIES: Syntax error expecting variable name.
- 3 | FREQUENCIES **.
- | ^~"
-
-"frequencies.sps:4.26-4.27: error: FREQUENCIES: Syntax error expecting one of the following: MEAN, SEMEAN, MEDIAN, MODE, STDDEV, VARIANCE, KURTOSIS, SEKURTOSIS, SKEWNESS, SESKEWNESS, RANGE, MINIMUM, MAXIMUM, SUM, DEFAULT, ALL, NONE.
- 4 | FREQUENCIES x/STATISTICS **.
- | ^~"
-
-"frequencies.sps:5.27-5.28: error: FREQUENCIES: Syntax error expecting number between 0 and 100 for PERCENTILES.
- 5 | FREQUENCIES x/PERCENTILES **.
- | ^~"
-
-"frequencies.sps:6.28-6.29: error: FREQUENCIES: Syntax error expecting `('.
- 6 | FREQUENCIES x/FORMAT LIMIT **.
- | ^~"
-
-"frequencies.sps:7.28-7.29: error: FREQUENCIES: Syntax error expecting non-negative integer for LIMIT.
- 7 | FREQUENCIES x/FORMAT LIMIT(**).
- | ^~"
-
-"frequencies.sps:8.30-8.31: error: FREQUENCIES: Syntax error expecting `)'.
- 8 | FREQUENCIES x/FORMAT LIMIT(5 **).
- | ^~"
-
-"frequencies.sps:9.22-9.23: error: FREQUENCIES: Syntax error expecting TABLE, NOTABLE, LIMIT, AVALUE, DVALUE, AFREQ, or DFREQ.
- 9 | FREQUENCIES x/FORMAT **.
- | ^~"
-
-"frequencies.sps:10.22-10.23: error: FREQUENCIES: Syntax error expecting non-negative integer for NTILES.
- 10 | FREQUENCIES x/NTILES **.
- | ^~"
-
-"frequencies.sps:11.25-11.26: error: FREQUENCIES: Syntax error expecting COMPATIBLE or ENHANCED.
- 11 | FREQUENCIES x/ALGORITHM **.
- | ^~"
-
-"frequencies.sps:12.30-12.31: error: FREQUENCIES: Syntax error expecting positive integer for FREQ.
- 12 | FREQUENCIES x/HISTOGRAM FREQ(**).
- | ^~"
-
-"frequencies.sps:13.32-13.33: error: FREQUENCIES: Syntax error expecting `)'.
- 13 | FREQUENCIES x/HISTOGRAM FREQ(5 **).
- | ^~"
-
-"frequencies.sps:14.33-14.34: error: FREQUENCIES: Syntax error expecting positive integer for PERCENT.
- 14 | FREQUENCIES x/HISTOGRAM PERCENT(**).
- | ^~"
-
-"frequencies.sps:15.35-15.36: error: FREQUENCIES: Syntax error expecting `)'.
- 15 | FREQUENCIES x/HISTOGRAM PERCENT(5 **).
- | ^~"
-
-"frequencies.sps:16.33-16.34: error: FREQUENCIES: Syntax error expecting number for MINIMUM.
- 16 | FREQUENCIES x/HISTOGRAM MINIMUM(**).
- | ^~"
-
-"frequencies.sps:17.35-17.36: error: FREQUENCIES: Syntax error expecting `)'.
- 17 | FREQUENCIES x/HISTOGRAM MINIMUM(5 **).
- | ^~"
-
-"frequencies.sps:18.33-18.34: error: FREQUENCIES: Syntax error expecting number for MAXIMUM.
- 18 | FREQUENCIES x/HISTOGRAM MAXIMUM(**).
- | ^~"
-
-"frequencies.sps:19.35-19.36: error: FREQUENCIES: Syntax error expecting `)'.
- 19 | FREQUENCIES x/HISTOGRAM MAXIMUM(5 **).
- | ^~"
-
-"frequencies.sps:20.44: error: FREQUENCIES: Syntax error expecting number 5 or greater for MAXIMUM.
- 20 | FREQUENCIES x/HISTOGRAM MINIMUM(5) MAXIMUM(1).
- | ^"
-
-"frequencies.sps:21.44-21.45: error: FREQUENCIES: Syntax error expecting number less than or equal to 5 for MINIMUM.
- 21 | FREQUENCIES x/HISTOGRAM MAXIMUM(5) MINIMUM(10).
- | ^~"
-
-"frequencies.sps:22.25-22.26: error: FREQUENCIES: Syntax error expecting NORMAL, NONORMAL, FREQ, PERCENT, MINIMUM, or MAXIMUM.
- 22 | FREQUENCIES x/HISTOGRAM **.
- | ^~"
-
-"frequencies.sps:23.32-23.33: error: FREQUENCIES: Syntax error expecting number for MINIMUM.
- 23 | FREQUENCIES x/PIECHART MINIMUM(**).
- | ^~"
-
-"frequencies.sps:24.34-24.35: error: FREQUENCIES: Syntax error expecting `)'.
- 24 | FREQUENCIES x/PIECHART MINIMUM(5 **).
- | ^~"
-
-"frequencies.sps:25.32-25.33: error: FREQUENCIES: Syntax error expecting number for MAXIMUM.
- 25 | FREQUENCIES x/PIECHART MAXIMUM(**).
- | ^~"
-
-"frequencies.sps:26.34-26.35: error: FREQUENCIES: Syntax error expecting `)'.
- 26 | FREQUENCIES x/PIECHART MAXIMUM(5 **).
- | ^~"
-
-"frequencies.sps:27.43: error: FREQUENCIES: Syntax error expecting number 5 or greater for MAXIMUM.
- 27 | FREQUENCIES x/PIECHART MINIMUM(5) MAXIMUM(1).
- | ^"
-
-"frequencies.sps:28.43-28.44: error: FREQUENCIES: Syntax error expecting number less than or equal to 5 for MINIMUM.
- 28 | FREQUENCIES x/PIECHART MAXIMUM(5) MINIMUM(10).
- | ^~"
-
-"frequencies.sps:29.24-29.25: error: FREQUENCIES: Syntax error expecting MINIMUM, MAXIMUM, MISSING, or NOMISSING.
- 29 | FREQUENCIES x/PIECHART **.
- | ^~"
-
-"frequencies.sps:30.29-30.30: error: FREQUENCIES: Syntax error expecting positive number for FREQ.
- 30 | FREQUENCIES x/BARCHART FREQ(**).
- | ^~"
-
-"frequencies.sps:31.31-31.32: error: FREQUENCIES: Syntax error expecting `)'.
- 31 | FREQUENCIES x/BARCHART FREQ(5 **).
- | ^~"
-
-"frequencies.sps:32.32-32.33: error: FREQUENCIES: Syntax error expecting positive number for PERCENT.
- 32 | FREQUENCIES x/BARCHART PERCENT(**).
- | ^~"
-
-"frequencies.sps:33.34-33.35: error: FREQUENCIES: Syntax error expecting `)'.
- 33 | FREQUENCIES x/BARCHART PERCENT(5 **).
- | ^~"
-
-"frequencies.sps:34.32-34.33: error: FREQUENCIES: Syntax error expecting number for MINIMUM.
- 34 | FREQUENCIES x/BARCHART MINIMUM(**).
- | ^~"
-
-"frequencies.sps:35.34-35.35: error: FREQUENCIES: Syntax error expecting `)'.
- 35 | FREQUENCIES x/BARCHART MINIMUM(5 **).
- | ^~"
-
-"frequencies.sps:36.32-36.33: error: FREQUENCIES: Syntax error expecting number for MAXIMUM.
- 36 | FREQUENCIES x/BARCHART MAXIMUM(**).
- | ^~"
-
-"frequencies.sps:37.34-37.35: error: FREQUENCIES: Syntax error expecting `)'.
- 37 | FREQUENCIES x/BARCHART MAXIMUM(5 **).
- | ^~"
-
-"frequencies.sps:38.43: error: FREQUENCIES: Syntax error expecting number 5 or greater for MAXIMUM.
- 38 | FREQUENCIES x/BARCHART MINIMUM(5) MAXIMUM(1).
- | ^"
-
-"frequencies.sps:39.43-39.44: error: FREQUENCIES: Syntax error expecting number less than or equal to 5 for MINIMUM.
- 39 | FREQUENCIES x/BARCHART MAXIMUM(5) MINIMUM(10).
- | ^~"
-
-"frequencies.sps:40.24-40.25: error: FREQUENCIES: Syntax error expecting MINIMUM, MAXIMUM, FREQ, or PERCENT.
- 40 | FREQUENCIES x/BARCHART **.
- | ^~"
-
-"frequencies.sps:41.23-41.24: error: FREQUENCIES: Syntax error expecting EXCLUDE or INCLUDE.
- 41 | FREQUENCIES x/MISSING **.
- | ^~"
-
-"frequencies.sps:42.21-42.22: error: FREQUENCIES: Syntax error expecting ANALYSIS or VARIABLE.
- 42 | FREQUENCIES x/ORDER **.
- | ^~"
-
-"frequencies.sps:43.16-43.17: error: FREQUENCIES: Syntax error expecting one of the following: STATISTICS, PERCENTILES, FORMAT, NTILES, ALGORITHM, HISTOGRAM, PIECHART, BARCHART, MISSING, ORDER.
- 43 | FREQUENCIES x/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([GLM procedure])
-
-AT_SETUP([GLM latin square design])
-AT_KEYWORDS([categorical categoricals])
-
-dnl This example comes from :
-dnl http://ssnds.uwo.ca/statsexamples/spssanova/latinsquareresults.html
-AT_DATA([latin.sps], [dnl
-set format = F20.3.
-data list notable fixed /a 1 b 3 c 5 y 7-10(2).
-begin data.
-1 1 6 3.5
-1 2 2 8.9
-1 3 3 9.6
-1 4 4 10.5
-1 5 5 3.1
-1 6 1 5.9
-2 1 2 4.2
-2 2 6 1.9
-2 3 5 3.7
-2 4 3 10.2
-2 5 1 7.2
-2 6 4 7.6
-3 1 1 6.7
-3 2 4 5.8
-3 3 6 -2.7
-3 4 2 4.6
-3 5 3 4.0
-3 6 5 -0.7
-4 1 4 6.6
-4 2 1 4.5
-4 3 2 3.7
-4 4 5 3.7
-4 5 6 -3.3
-4 6 3 3.0
-5 1 3 4.1
-5 2 5 2.4
-5 3 4 6.0
-5 4 1 5.1
-5 5 2 3.5
-5 6 6 4.0
-6 1 5 3.8
-6 2 3 5.8
-6 3 1 7.0
-6 4 6 3.8
-6 5 4 5.0
-6 6 2 8.6
-end data.
-
-variable labels a 'Factor A' b 'Factor B' c 'Factor C' y 'Criterion'.
-
-glm y by b a c
- /intercept=include
- /criteria=alpha(.05)
- /design = a b c
- .
-])
-
-AT_CHECK([pspp -O format=csv latin.sps | sed 's/329.62[[678]]/329.62/'], [0],
- [dnl
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,263.064,15,17.538,5.269,.000
-Intercept,815.103,1,815.103,244.910,.000
-Factor A,78.869,5,15.774,4.739,.005
-Factor B,28.599,5,5.720,1.719,.176
-Factor C,155.596,5,31.119,9.350,.000
-Error,66.563,20,3.328,,
-Total,1144.730,36,,,
-Corrected Total,329.62,35,,,
-])
-
-AT_CLEANUP
-
-AT_SETUP([GLM 2 by 2 factorial design])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([2by2.sps], [dnl
-set format = F20.3.
-data list notable list /Factor0 * Factor1 * points (F10).
-begin data.
-1 4 332
-1 4 380
-1 4 371
-1 4 366
-1 4 354
-1 0 259.5
-1 0 302.5
-1 0 296
-1 0 349
-1 0 309
-2 4 354.67
-2 4 353.5
-2 4 304
-2 4 365
-2 4 339
-2 0 306
-2 0 339
-2 0 353
-2 0 351
-2 0 333
-end data.
-
-glm points by Factor0 Factor1
- /intercept=include
- /criteria=alpha(.05)
- .
-])
-
-
-AT_CHECK([pspp -O format=csv 2by2.sps ], [0],
- [dnl
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,8667.053,3,2889.018,5.043,.012
-Intercept,2256018.640,1,2256018.640,3937.693,.000
-Factor0,313.394,1,313.394,.547,.470
-Factor1,5157.508,1,5157.508,9.002,.008
-Factor0 × Factor1,3196.150,1,3196.150,5.579,.031
-Error,9166.865,16,572.929,,
-Total,2273852.559,20,,,
-Corrected Total,17833.918,19,,,
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([GLM Type I and II Sums of Squares])
-AT_KEYWORDS([categorical categoricals])
-
-dnl The following example comes from
-dnl http://www.uvm.edu/~dhowell/StatPages/More_Stuff/Type1-3.pdf
-AT_DATA([data-inc.sps], [dnl
-set decimal = dot.
-set format=F20.3.
-data list notable list /dv * Agrp * B0 * B1 * B2 * i0 * i1 * i2 * sss *.
-begin data.
-5 1 1 0 0 1 0 0 1.00
-7 1 1 0 0 1 0 0 1.00
-9 1 1 0 0 1 0 0 1.00
-8 1 1 0 0 1 0 0 1.00
-2 1 0 1 0 0 1 0 1.00
-5 1 0 1 0 0 1 0 1.00
-7 1 0 1 0 0 1 0 1.00
-3 1 0 1 0 0 1 0 1.00
-9 1 0 1 0 0 1 0 1.00
-8 1 0 0 1 0 0 1 1.00
-11 1 0 0 1 0 0 1 1.00
-12 1 0 0 1 0 0 1 1.00
-14 1 0 0 1 0 0 1 1.00
-11 1 -1 -1 -1 -1 -1 -1 1.00
-15 1 -1 -1 -1 -1 -1 -1 1.00
-16 1 -1 -1 -1 -1 -1 -1 1.00
-10 1 -1 -1 -1 -1 -1 -1 1.00
-9 1 -1 -1 -1 -1 -1 -1 1.00
-7 -1 1 0 0 -1 0 0 2.00
-9 -1 1 0 0 -1 0 0 2.00
-10 -1 1 0 0 -1 0 0 2.00
-9 -1 1 0 0 -1 0 0 2.00
-3 -1 0 1 0 0 -1 0 2.00
-8 -1 0 1 0 0 -1 0 2.00
-9 -1 0 1 0 0 -1 0 2.00
-11 -1 0 1 0 0 -1 0 2.00
-9 -1 0 0 1 0 0 -1 2.00
-12 -1 0 0 1 0 0 -1 2.00
-14 -1 0 0 1 0 0 -1 2.00
-8 -1 0 0 1 0 0 -1 2.00
-7 -1 0 0 1 0 0 -1 2.00
-11 -1 -1 -1 -1 1 1 1 2.00
-14 -1 -1 -1 -1 1 1 1 2.00
-10 -1 -1 -1 -1 1 1 1 2.00
-12 -1 -1 -1 -1 1 1 1 2.00
-13 -1 -1 -1 -1 1 1 1 2.00
-11 -1 -1 -1 -1 1 1 1 2.00
-12 -1 -1 -1 -1 1 1 1 2.00
-end data.
-
-do if B0 = -1 AND B1 = -1 AND B2 = -1.
-compute Bgrp = 4.
-end if.
-
-do if B0 = 0 AND B1 = 0 AND B2 = 1.
-compute Bgrp = 3.
-end if.
-
-do if B0 = 0 AND B1 = 1 AND B2 = 0.
-compute Bgrp = 2.
-end if.
-
-do if B0 = 1 AND B1 = 0 AND B2 = 0.
-compute Bgrp = 1.
-end if.
-])
-
-AT_DATA([type1.sps], [dnl
-include 'data-inc.sps'.
-
-glm dv by Agrp Bgrp
- /method = sstype (1)
- .
-
-glm dv by Agrp Bgrp
- /method = sstype (1)
- /design Bgrp Agrp Bgrp * Agrp
- .
-])
-
-
-AT_CHECK([pspp -O format=csv type1.sps], [0],
- [dnl
-Table: Tests of Between-Subjects Effects
-,Type I Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,216.017,7,30.860,5.046,.001
-Agrp,9.579,1,9.579,1.566,.220
-Bgrp,186.225,3,62.075,10.151,.000
-Agrp × Bgrp,20.212,3,6.737,1.102,.364
-Error,183.457,30,6.115,,
-Total,3810.000,38,,,
-Corrected Total,399.474,37,,,
-
-Table: Tests of Between-Subjects Effects
-,Type I Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,216.017,7,30.860,5.046,.001
-Bgrp,193.251,3,64.417,10.534,.000
-Agrp,2.553,1,2.553,.418,.523
-Bgrp × Agrp,20.212,3,6.737,1.102,.364
-Error,183.457,30,6.115,,
-Total,3810.000,38,,,
-Corrected Total,399.474,37,,,
-])
-
-
-AT_DATA([type2.sps], [dnl
-include 'data-inc.sps'.
-
-glm dv by Agrp Bgrp
- /method = sstype (2)
- .
-])
-
-
-AT_CHECK([pspp -O format=csv type2.sps], [0],
- [dnl
-Table: Tests of Between-Subjects Effects
-,Type II Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,216.017,7,30.860,5.046,.001
-Agrp,2.553,1,2.553,.418,.523
-Bgrp,186.225,3,62.075,10.151,.000
-Agrp × Bgrp,20.212,3,6.737,1.102,.364
-Error,183.457,30,6.115,,
-Total,3810.000,38,,,
-Corrected Total,399.474,37,,,
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([GLM excluded intercept])
-AT_KEYWORDS([categorical categoricals])
-
-dnl The following example comes from
-dnl
-dnl Rudolf N. Cardinal
-dnl Graduate-level statistics for psychology and neuroscience
-dnl ANOVA in practice, and complex ANOVA designs
-dnl Version of 2 May 2004
-dnl
-dnl Downloaded from: http://egret.psychol.cam.ac.uk/psychology/graduate/Guide_to_ANOVA.pdf
-
-AT_DATA([intercept-exclude.sps], [dnl
-set format = F20.3.
-
-data list notable list /depvar * A *.
-begin data.
-10 1
-14 1
-8 1
-7 1
-2 1
-10 1
-1 1
-3 1
-2 1
-8.5 1
-14.29 2
-18.49 2
-12.46 2
-11.63 2
-6.66 2
-14.02 2
-5.66 2
-7.06 2
-6.37 2
-13.26 2
-end data.
-
-GLM depvar by A
- /intercept = exclude
- .
-
-
-GLM depvar by A
- /intercept = include
- .
-
-])
-
-AT_CHECK([pspp -O format=csv intercept-exclude.sps], [0],
- [dnl
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Model,1636.826,2,818.413,43.556,.000
-A,1636.826,2,818.413,43.556,.000
-Error,338.216,18,18.790,,
-Total,1975.042,20,,,
-
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,98.568,1,98.568,5.246,.034
-Intercept,1538.258,1,1538.258,81.867,.000
-A,98.568,1,98.568,5.246,.034
-Error,338.216,18,18.790,,
-Total,1975.042,20,,,
-Corrected Total,436.784,19,,,
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([GLM missing values])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([glm.data], [dnl
-1 1 6 3.5
-1 2 2 8.9
-1 3 3 9.6
-1 4 4 10.5
-1 5 5 3.1
-1 6 1 5.9
-2 1 2 4.2
-2 2 6 1.9
-2 3 5 3.7
-2 4 3 10.2
-2 5 1 7.2
-2 6 4 7.6
-3 1 1 6.7
-3 2 4 5.8
-3 3 6 -2.7
-3 4 2 4.6
-3 5 3 4.0
-3 6 5 -0.7
-4 1 4 6.6
-4 2 1 4.5
-4 3 2 3.7
-4 4 5 3.7
-4 5 6 -3.3
-4 6 3 3.0
-5 1 3 4.1
-5 2 5 2.4
-5 3 4 6.0
-5 4 1 5.1
-5 5 2 3.5
-5 6 6 4.0
-6 1 5 3.8
-6 2 3 5.8
-6 3 1 7.0
-6 4 6 3.8
-6 5 4 5.0
-6 6 2 8.6
-])
-
-AT_DATA([glm-miss.sps], [dnl
-set format = F20.3.
-data list file='glm.data' notable fixed /a 1 b 3 c 5 y 7-10(2).
-
-do if a=6.
-recode y (else=SYSMIS).
-end if.
-
-glm y by b a c
- /criteria=alpha(.05)
- /design = a b c
- .
-])
-
-AT_CHECK([pspp -O format=csv glm-miss.sps], [0], [dnl
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,251.621,14,17.973,4.969,.002
-Intercept,628.376,1,628.376,173.737,.000
-a,72.929,4,18.232,5.041,.009
-b,20.703,5,4.141,1.145,.380
-c,135.179,5,27.036,7.475,.001
-Error,54.253,15,3.617,,
-Total,934.250,30,,,
-Corrected Total,305.874,29,,,
-])
-
-
-
-AT_DATA([glm-miss2.sps], [dnl
-set format = F20.3.
-data list file='glm.data' notable fixed /a 1 b 3 c 5 y 7-10(2).
-
-select if a <> 6.
-
-glm y by b a c
- /criteria=alpha(.05)
- /design = a b c
- .
-])
-
-AT_CHECK([pspp -O format=csv glm-miss2.sps], [0], [dnl
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,251.621,14,17.973,4.969,.002
-Intercept,628.376,1,628.376,173.737,.000
-a,72.929,4,18.232,5.041,.009
-b,20.703,5,4.141,1.145,.380
-c,135.179,5,27.036,7.475,.001
-Error,54.253,15,3.617,,
-Total,934.250,30,,,
-Corrected Total,305.874,29,,,
-])
-
-
-dnl Now for some missing values in the factor variables.
-
-AT_DATA([glm-miss3.sps], [dnl
-set format = F20.3.
-data list file=glm.data notable fixed /a 1 b 3 c 5 y 7-10(2).
-
-do if a=6.
-recode a (else=SYSMIS).
-end if.
-
-glm y by b a c
- /criteria=alpha(.05)
- /design = a b c
- .
-])
-
-AT_CHECK([pspp -O format=csv glm-miss3.sps], [0], [dnl
-Table: Tests of Between-Subjects Effects
-,Type III Sum Of Squares,df,Mean Square,F,Sig.
-Corrected Model,251.621,14,17.973,4.969,.002
-Intercept,628.376,1,628.376,173.737,.000
-a,72.929,4,18.232,5.041,.009
-b,20.703,5,4.141,1.145,.380
-c,135.179,5,27.036,7.475,.001
-Error,54.253,15,3.617,,
-Total,934.250,30,,,
-Corrected Total,305.874,29,,,
-])
-
-AT_CLEANUP
-
-AT_SETUP([GLM syntax errors])
-AT_DATA([glm.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-GLM **.
-GLM x **.
-GLM x BY **.
-GLM x BY y.
-GLM x y BY z.
-GLM x BY y/MISSING=**.
-GLM x BY y/INTERCEPT=**.
-GLM x BY y/CRITERIA=**.
-GLM x BY y/CRITERIA=ALPHA **.
-GLM x BY y/CRITERIA=ALPHA(**).
-GLM x BY y/CRITERIA=ALPHA(123 **).
-GLM x BY y/METHOD=**.
-GLM x BY y/METHOD=SSTYPE **.
-GLM x BY y/METHOD=SSTYPE(4).
-GLM x BY y/METHOD=SSTYPE(2 **).
-GLM x BY y/DESIGN=**.
-GLM x BY y/DESIGN=x(y).
-GLM x BY y/DESIGN=x WITHIN y.
-])
-AT_CHECK([pspp -O format=csv glm.sps], [1], [dnl
-"glm.sps:2.5-2.6: error: GLM: Syntax error expecting variable name.
- 2 | GLM **.
- | ^~"
-
-"glm.sps:3.7-3.8: error: GLM: Syntax error expecting `BY'.
- 3 | GLM x **.
- | ^~"
-
-"glm.sps:4.10-4.11: error: GLM: Syntax error expecting variable name.
- 4 | GLM x BY **.
- | ^~"
-
-"glm.sps:6.1-6.3: error: GLM: Syntax error expecting `BEGIN DATA'.
- 6 | GLM x y BY z.
- | ^~~"
-
-"glm.sps:6.1-6.3: error: GLM: Syntax error expecting end of command.
- 6 | GLM x y BY z.
- | ^~~"
-
-"glm.sps:7.20-7.21: error: GLM: Syntax error expecting INCLUDE or EXCLUDE.
- 7 | GLM x BY y/MISSING=**.
- | ^~"
-
-"glm.sps:8.22-8.23: error: GLM: Syntax error expecting INCLUDE or EXCLUDE.
- 8 | GLM x BY y/INTERCEPT=**.
- | ^~"
-
-"glm.sps:9.21-9.22: error: GLM: Syntax error expecting `ALPHA@{:@'.
- 9 | GLM x BY y/CRITERIA=**.
- | ^~"
-
-"glm.sps:10.21-10.28: error: GLM: Syntax error expecting `ALPHA@{:@'.
- 10 | GLM x BY y/CRITERIA=ALPHA **.
- | ^~~~~~~~"
-
-"glm.sps:11.27-11.28: error: GLM: Syntax error expecting number.
- 11 | GLM x BY y/CRITERIA=ALPHA(**).
- | ^~"
-
-"glm.sps:12.31-12.32: error: GLM: Syntax error expecting `@:}@'.
- 12 | GLM x BY y/CRITERIA=ALPHA(123 **).
- | ^~"
-
-"glm.sps:13.19-13.20: error: GLM: Syntax error expecting `SSTYPE@{:@'.
- 13 | GLM x BY y/METHOD=**.
- | ^~"
-
-"glm.sps:14.19-14.27: error: GLM: Syntax error expecting `SSTYPE@{:@'.
- 14 | GLM x BY y/METHOD=SSTYPE **.
- | ^~~~~~~~~"
-
-"glm.sps:15.26: error: GLM: Syntax error expecting integer between 1 and 3 for SSTYPE.
- 15 | GLM x BY y/METHOD=SSTYPE(4).
- | ^"
-
-"glm.sps:16.28-16.29: error: GLM: Syntax error expecting `@:}@'.
- 16 | GLM x BY y/METHOD=SSTYPE(2 **).
- | ^~"
-
-"glm.sps:17.19-17.20: error: GLM: Syntax error expecting variable name.
- 17 | GLM x BY y/DESIGN=**.
- | ^~"
-
-"glm.sps:18.20: error: GLM: Nested variables are not yet implemented.
- 18 | GLM x BY y/DESIGN=x(y).
- | ^"
-
-"glm.sps:19.21-19.26: error: GLM: Nested variables are not yet implemented.
- 19 | GLM x BY y/DESIGN=x WITHIN y.
- | ^~~~~~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([GRAPH])
-
-AT_SETUP([GRAPH simple scatterplot])
-AT_DATA([scatterplot.sps],[
-* Simple Scatterplot test
-NEW FILE.
-INPUT PROGRAM.
-LOOP #i = 1 to 100.
-COMPUTE Age = RV.NORMAL(40,10).
-END CASE.
-END LOOP.
-END FILE.
-END INPUT PROGRAM.
-
-COMPUTE Size = Age * 3 + 50.
-
-GRAPH
- /SCATTERPLOT(BIVARIATE) = Age WITH Size.
-
-])
-
-AT_CHECK([pspp -O format=csv scatterplot.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([GRAPH Scatter and Histogram])
-AT_KEYWORDS([slow])
-AT_DATA([scatterlong.sps],[
-NEW FILE.
-INPUT PROGRAM.
-LOOP #i = 1 to 10000.
-COMPUTE Age = RV.NORMAL(40,10).
-COMPUTE CityNum = TRUNC(UNIFORM(2.95)).
-END CASE.
-END LOOP.
-END FILE.
-END INPUT PROGRAM.
-
-COMPUTE Size = Age * 3 + 50 + 50*CityNum.
-
-STRING City (a20).
-
-Recode CityNum
- (0 = "Madrid")
- (1 = "Paris")
- (ELSE = "Stockholm")
- into City.
-
- GRAPH
- /SCATTERPLOT(BIVARIATE) = Age WITH Size
-
- GRAPH
- /SCATTERPLOT(BIVARIATE) = Age WITH CityNum
-
- GRAPH
- /SCATTERPLOT = CityNum WITH Age
-
- GRAPH
- /SCATTERPLOT = CityNum WITH Size
-
- GRAPH
- /SCATTERPLOT(BIVARIATE) = Age WITH Size BY City
-
- GRAPH
- /SCATTERPLOT(BIVARIATE) = Age WITH Size BY CityNum
-
- ADD VALUE LABELS
- /CityNum 1 'Rio' 2 'Tokyo' 0 'Mumbai'.
-
- GRAPH
- /SCATTERPLOT(BIVARIATE) = Age WITH Size BY CityNum
-
- GRAPH
- /HISTOGRAM = Age.
-
-])
-
-AT_CHECK([pspp -O format=pdf scatterlong.sps], [0], [ignore], [ignore])
-AT_CLEANUP
-
-AT_SETUP([GRAPH missing values don't crash])
-AT_DATA([scatter.sps], [dnl
-data list list /x * y *.
-begin data.
-1 0
-2 0
-. 0
-3 1
-4 1
-5 .
-6 1
-end data.
-graph
- /scatterplot = x with y.
-graph
- /histogram = x.
-])
-AT_CHECK([pspp -o pspp.pdf scatter.sps], [], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-AT_SETUP([GRAPH missing=VARIABLE no crash])
-AT_DATA([scatter.sps], [dnl
-data list list /x * y *.
-begin data.
-1 0
-2 0
-. 0
-3 1
-4 1
-5 .
-6 1
-end data.
-graph
- /scatterplot = x with y
- /missing = VARIABLE.
-graph
- /histogram = x
- /missing = VARIABLE.
-])
-AT_CHECK([pspp -o pspp.pdf scatter.sps], [], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-AT_SETUP([GRAPH missing value in by variable])
-AT_DATA([scatter.sps], [dnl
-data list list /x * y * z *.
-begin data.
-1 0 9
-2 0 9
-. 0 9
-3 1 .
-4 1 8
-5 . 8
-6 1 8
-end data.
-graph
- /scatterplot = x with y by z
- /missing = VARIABLE.
-
-graph
- /scatterplot = x with y by z.
-
-])
-AT_CHECK([pspp -o pspp.pdf scatter.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-
-AT_SETUP([GRAPH histogram with null data])
-AT_DATA([null-hist.sps], [dnl
-data list list /x *.
-begin data.
-1109
-.
-end data.
-
-graph
- /histogram = x.
-
-])
-
-AT_CHECK([pspp -o pspp.pdf null-hist.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-
-AT_SETUP([GRAPH histogram all missing])
-AT_DATA([null-hist.sps], [dnl
-data list list /x *.
-begin data.
-.
-end data.
-
-graph
- /histogram = x.
-
-])
-
-AT_CHECK([pspp null-hist.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-AT_CLEANUP
-
-
-
-
-AT_SETUP([GRAPH barcharts])
-AT_CHECK([ln -s $top_srcdir/examples/physiology.sav .], [0])
-AT_CHECK([ln -s $top_srcdir/examples/repairs.sav .], [0])
-
-AT_DATA([barchart.sps], [dnl
-GET FILE="physiology.sav".
-
-GRAPH /BAR = COUNT BY SEX.
-
-GRAPH /BAR = MEAN(height) BY SEX.
-
-NEW FILE.
-
-GET FILE="repairs.sav".
-
-GRAPH /BAR = MEAN (mtbf) BY factory.
-
-COMPUTE R = TRUNC(RV.UNIFORM(1,5)).
-
-GRAPH /BAR = MEAN (mtbf) BY factory BY R.
-])
-
-AT_CHECK([pspp -o pspp.pdf barchart.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-
-AT_CLEANUP
-
-
-
-AT_SETUP([GRAPH barchart arity])
-
-AT_DATA([barchart.sps], [dnl
-data list notable list /x y z*.
-begin data
-1 1 3
-2 1 4
-3 1 3
-4 1 4
-5 . 3
-6 2 4
-7 2 3
-8 2 4
-9 2 3
-10 2 4
-end data.
-
-* This line is invalid
-GRAPH /BAR = COUNT(x) BY y.
-])
-
-AT_CHECK([pspp -o pspp.pdf barchart.sps], [1], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-
-AT_CLEANUP
-
-
-
-
-AT_SETUP([GRAPH barchart bad syntax])
-
-AT_DATA([barchart.sps], [dnl
-data list notable list /x y z*.
-begin data
-1 1 3
-2 1 4
-3 1 3
-4 1 4
-5 . 3
-6 2 4
-7 2 3
-8 2 4
-9 2 3
-10 2 4
-end data.
-
-* This line is invalid
-GRAPH /BAR = SCROD BY y.
-])
-
-AT_CHECK([pspp -o pspp.pdf barchart.sps], [1], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-
-AT_CLEANUP
-
-
-
-AT_SETUP([GRAPH barchart full])
-
-AT_DATA([barchart.sps], [dnl
-data list notable list /x y z*.
-begin data
-1 1 3
-2 1 4
-3 1 3
-4 1 4
-5 . 3
-6 2 4
-7 2 3
-8 2 4
-9 2 3
-10 2 4
-end data.
-
-* This line is invalid
-GRAPH /BAR = COUNT by z.
-GRAPH /BAR = CUFREQ by z.
-GRAPH /BAR = PCT by z.
-GRAPH /BAR = CUPCT by z.
-
-GRAPH /BAR = MEAN(y) BY z.
-GRAPH /BAR = SUM(y) BY z.
-GRAPH /BAR = MAXIMUM(y) BY z.
-GRAPH /BAR = MINIMUM(y) BY z.
-
-GRAPH /BAR = MEAN(y) BY z BY y.
-GRAPH /BAR = SUM(y) BY z BY y.
-GRAPH /BAR = MAXIMUM(y) BY z BY y.
-GRAPH /BAR = MINIMUM(y) BY z BY y.
-])
-
-AT_CHECK([pspp -o pspp.pdf barchart.sps], [0], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-
-AT_CLEANUP
-
-
-
-
-
-AT_SETUP([GRAPH buggy syntax])
-
-AT_DATA([barchart.sps], [dnl
-data list notable list /x y z*.
-begin data
-1 1 3
-2 1 4
-10 2 4
-end data.
-
-GRAPH /BAR = MINIMUM({) BY z BY y.
-])
-
-AT_CHECK([pspp -o pspp.pdf barchart.sps], [1], [ignore])
-dnl Ignore output -- this is just a no-crash check.
-
-AT_CLEANUP
-
-
-dnl Check that percentages are calculated with respect to the
-dnl proper total. See bug #56247
-AT_SETUP([GRAPH barchart percentage sub-categorical])
-AT_DATA([barchart.sps], [dnl
-data list list notable /penalty_favopp_x * XYdem_racethW8 * w *.
-begin data.
-1 0 1960
-1 1 376
-2 0 678
-2 1 147
-4 0 368
-4 1 164
-5 0 427
-5 1 274
-. . 1522
-end data.
-
-weight by w.
-
-* crosstabs
-* /tables=penalty_favopp_x by XYdem_racethW8
-* /format=AVALUE TABLES PIVOT
-* /statistics=CHISQ
-* /cells COUNT COLUMN TOTAL.
-
-graph
- /bar=pct by penalty_favopp_x
- .
-
-graph
- /bar=pct by penalty_favopp_x by XYdem_racethW8
- .
-])
-
-AT_CHECK([pspp --testing barchart.sps], [0], [dnl
-Graphic: Barchart
-Percentage: 0
-Total Categories: 4
-Primary Categories: 4
-Largest Category: 53.1634
-Total Count: 100
-Y Label: "Percentage"
-Categorical Variables:
- Var: "penalty_favopp_x"
-Categories:
- 0 " 1.00"
- 1 " 2.00"
- 2 " 4.00"
- 3 " 5.00"
-All Categories:
-Count: 53.1634; Cat: " 1.00"
-Count: 18.7756; Cat: " 2.00"
-Count: 12.1074; Cat: " 4.00"
-Count: 15.9536; Cat: " 5.00"
-
-Graphic: Barchart
-Percentage: 0
-Total Categories: 8
-Primary Categories: 4
-Largest Category: 57.0929
-Total Count: 200
-Y Label: "Percentage"
-Categorical Variables:
- Var: "penalty_favopp_x"
- Var: "XYdem_racethW8"
-Categories:
- 0 " 1.00"
- 1 " 2.00"
- 2 " 4.00"
- 3 " 5.00"
-Sub-categories:
- 0 " .00"
- 1 " 1.00"
-All Categories:
-Count: 57.0929; Cat: " 1.00", " .00"
-Count: 39.1259; Cat: " 1.00", " 1.00"
-Count: 19.7495; Cat: " 2.00", " .00"
-Count: 15.2966; Cat: " 2.00", " 1.00"
-Count: 10.7195; Cat: " 4.00", " .00"
-Count: 17.0656; Cat: " 4.00", " 1.00"
-Count: 12.4381; Cat: " 5.00", " .00"
-Count: 28.512; Cat: " 5.00", " 1.00"
-
-])
-
-AT_CLEANUP
-
-AT_SETUP([GRAPH syntax errors])
-AT_DATA([graph.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-GRAPH/HISTOGRAM=x/HISTOGRAM=y.
-GRAPH/HISTOGRAM(**).
-GRAPH/HISTOGRAM(NORMAL **).
-GRAPH/HISTOGRAM=**.
-GRAPH/HISTOGRAM=x y z.
-GRAPH/HISTOGRAM=x/BAR=y.
-GRAPH/BAR(GROUPED).
-GRAPH/BAR(STACKED).
-GRAPH/BAR(RANGE).
-GRAPH/BAR(**).
-GRAPH/BAR **.
-GRAPH/BAR=**.
-GRAPH/BAR=MEAN **.
-GRAPH/BAR=MEAN(**).
-GRAPH/BAR=MEAN(x**).
-GRAPH/BAR=MEAN(x) **.
-GRAPH/BAR=MEAN(x) BY **.
-GRAPH/BAR=MEAN(x) BY y BY **.
-GRAPH/HISTOGRAM=x/SCATTERPLOT=y.
-GRAPH/SCATTERPLOT(OVERLAY).
-GRAPH/SCATTERPLOT(MATRIX).
-GRAPH/SCATTERPLOT(XYZ).
-GRAPH/SCATTERPLOT(**).
-GRAPH/SCATTERPLOT(BIVARIATE **).
-GRAPH/SCATTERPLOT **.
-GRAPH/SCATTERPLOT=**.
-GRAPH/SCATTERPLOT=x y z.
-GRAPH/SCATTERPLOT=x **.
-GRAPH/SCATTERPLOT=x WITH **.
-GRAPH/SCATTERPLOT=x WITH y z.
-GRAPH/SCATTERPLOT=x WITH y BY **.
-GRAPH/LINE.
-GRAPH/PIE.
-GRAPH/ERRORBAR.
-GRAPH/PARETO.
-GRAPH/TITLE.
-GRAPH/SUBTITLE.
-GRAPH/FOOTNOTE.
-GRAPH/MISSING=**.
-GRAPH/ **.
-])
-AT_CHECK([pspp -O format=csv graph.sps], [1], [dnl
-"graph.sps:2.19-2.27: error: GRAPH: Only one chart type is allowed.
- 2 | GRAPH/HISTOGRAM=x/HISTOGRAM=y.
- | ^~~~~~~~~"
-
-"graph.sps:3.17-3.18: error: GRAPH: Syntax error expecting `NORMAL)'.
- 3 | GRAPH/HISTOGRAM(**).
- | ^~"
-
-"graph.sps:4.17-4.25: error: GRAPH: Syntax error expecting `NORMAL)'.
- 4 | GRAPH/HISTOGRAM(NORMAL **).
- | ^~~~~~~~~"
-
-"graph.sps:5.17-5.18: error: GRAPH: Syntax error expecting variable name.
- 5 | GRAPH/HISTOGRAM=**.
- | ^~"
-
-"graph.sps:6.17-6.21: error: GRAPH: Only one variable is allowed.
- 6 | GRAPH/HISTOGRAM=x y z.
- | ^~~~~"
-
-"graph.sps:7.19-7.21: error: GRAPH: Only one chart type is allowed.
- 7 | GRAPH/HISTOGRAM=x/BAR=y.
- | ^~~"
-
-"graph.sps:8.11-8.17: error: GRAPH: GROUPED is not yet implemented.
- 8 | GRAPH/BAR(GROUPED).
- | ^~~~~~~"
-
-"graph.sps:9.11-9.17: error: GRAPH: STACKED is not yet implemented.
- 9 | GRAPH/BAR(STACKED).
- | ^~~~~~~"
-
-"graph.sps:10.11-10.15: error: GRAPH: RANGE is not yet implemented.
- 10 | GRAPH/BAR(RANGE).
- | ^~~~~"
-
-"graph.sps:11.11-11.12: error: GRAPH: Syntax error expecting SIMPLE, GROUPED, STACKED, or RANGE.
- 11 | GRAPH/BAR(**).
- | ^~"
-
-"graph.sps:12.11-12.12: error: GRAPH: Syntax error expecting `='.
- 12 | GRAPH/BAR **.
- | ^~"
-
-"graph.sps:13.11-13.12: error: GRAPH: Syntax error expecting COUNT, PCT, CUFREQ, CUPCT, MEAN, SUM, MAXIMUM, or MINIMUM.
- 13 | GRAPH/BAR=**.
- | ^~"
-
-"graph.sps:14.16-14.17: error: GRAPH: Syntax error expecting `('.
- 14 | GRAPH/BAR=MEAN **.
- | ^~"
-
-"graph.sps:15.16-15.17: error: GRAPH: Syntax error expecting variable name.
- 15 | GRAPH/BAR=MEAN(**).
- | ^~"
-
-"graph.sps:16.17-16.18: error: GRAPH: Syntax error expecting `)'.
- 16 | GRAPH/BAR=MEAN(x**).
- | ^~"
-
-"graph.sps:17.19-17.20: error: GRAPH: Syntax error expecting `BY'.
- 17 | GRAPH/BAR=MEAN(x) **.
- | ^~"
-
-"graph.sps:18.22-18.23: error: GRAPH: Syntax error expecting variable name.
- 18 | GRAPH/BAR=MEAN(x) BY **.
- | ^~"
-
-"graph.sps:19.27-19.28: error: GRAPH: Syntax error expecting variable name.
- 19 | GRAPH/BAR=MEAN(x) BY y BY **.
- | ^~"
-
-"graph.sps:20.19-20.29: error: GRAPH: Only one chart type is allowed.
- 20 | GRAPH/HISTOGRAM=x/SCATTERPLOT=y.
- | ^~~~~~~~~~~"
-
-"graph.sps:21.19-21.25: error: GRAPH: OVERLAY is not yet implemented.
- 21 | GRAPH/SCATTERPLOT(OVERLAY).
- | ^~~~~~~"
-
-"graph.sps:22.19-22.24: error: GRAPH: MATRIX is not yet implemented.
- 22 | GRAPH/SCATTERPLOT(MATRIX).
- | ^~~~~~"
-
-"graph.sps:23.19-23.21: error: GRAPH: XYZ is not yet implemented.
- 23 | GRAPH/SCATTERPLOT(XYZ).
- | ^~~"
-
-"graph.sps:24.19-24.20: error: GRAPH: Syntax error expecting BIVARIATE, OVERLAY, MATRIX, or XYZ.
- 24 | GRAPH/SCATTERPLOT(**).
- | ^~"
-
-"graph.sps:25.29-25.30: error: GRAPH: Syntax error expecting `)'.
- 25 | GRAPH/SCATTERPLOT(BIVARIATE **).
- | ^~"
-
-"graph.sps:26.19-26.20: error: GRAPH: Syntax error expecting `='.
- 26 | GRAPH/SCATTERPLOT **.
- | ^~"
-
-"graph.sps:27.19-27.20: error: GRAPH: Syntax error expecting variable name.
- 27 | GRAPH/SCATTERPLOT=**.
- | ^~"
-
-"graph.sps:28.19-28.23: error: GRAPH: Only one variable is allowed.
- 28 | GRAPH/SCATTERPLOT=x y z.
- | ^~~~~"
-
-"graph.sps:29.21-29.22: error: GRAPH: Syntax error expecting `WITH'.
- 29 | GRAPH/SCATTERPLOT=x **.
- | ^~"
-
-"graph.sps:30.26-30.27: error: GRAPH: Syntax error expecting variable name.
- 30 | GRAPH/SCATTERPLOT=x WITH **.
- | ^~"
-
-"graph.sps:31.26-31.28: error: GRAPH: Only one variable is allowed.
- 31 | GRAPH/SCATTERPLOT=x WITH y z.
- | ^~~"
-
-"graph.sps:32.31-32.32: error: GRAPH: Syntax error expecting variable name.
- 32 | GRAPH/SCATTERPLOT=x WITH y BY **.
- | ^~"
-
-"graph.sps:33.7-33.10: error: GRAPH: LINE is not yet implemented.
- 33 | GRAPH/LINE.
- | ^~~~"
-
-"graph.sps:34.7-34.9: error: GRAPH: PIE is not yet implemented.
- 34 | GRAPH/PIE.
- | ^~~"
-
-"graph.sps:35.7-35.14: error: GRAPH: ERRORBAR is not yet implemented.
- 35 | GRAPH/ERRORBAR.
- | ^~~~~~~~"
-
-"graph.sps:36.7-36.12: error: GRAPH: PARETO is not yet implemented.
- 36 | GRAPH/PARETO.
- | ^~~~~~"
-
-"graph.sps:37.7-37.11: error: GRAPH: TITLE is not yet implemented.
- 37 | GRAPH/TITLE.
- | ^~~~~"
-
-"graph.sps:38.7-38.14: error: GRAPH: SUBTITLE is not yet implemented.
- 38 | GRAPH/SUBTITLE.
- | ^~~~~~~~"
-
-"graph.sps:39.7-39.14: error: GRAPH: FOOTNOTE is not yet implemented.
- 39 | GRAPH/FOOTNOTE.
- | ^~~~~~~~"
-
-"graph.sps:40.15-40.16: error: GRAPH: Syntax error expecting LISTWISE, VARIABLE, EXCLUDE, INCLUDE, REPORT, or NOREPORT.
- 40 | GRAPH/MISSING=**.
- | ^~"
-
-"graph.sps:41.8-41.9: error: GRAPH: Syntax error expecting one of the following: HISTOGRAM, BAR, SCATTERPLOT, LINE, PIE, ERRORBAR, PARETO, TITLE, SUBTITLE, FOOTNOTE, MISSING.
- 41 | GRAPH/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([LOGISTIC REGRESSION])
-
-dnl These examples are adapted from
-dnl http://www.uvm.edu/~dhowell/gradstat/psych341/lectures/Logistic%20Regression/LogisticReg1.html
-
-
-
-m4_define([LOGIT_TEST_DATA],
- [AT_DATA([lr-data.txt], dnl
- 105.00 1.00 33.00 3.00 2.00 .35 17.00 20.00 .50110 -2.00440 1
- 106.00 1.00 50.00 2.00 3.00 .38 7.00 15.00 .20168 -1.25264 1
- 107.00 1.00 91.00 3.00 2.00 .28 15.00 7.00 .00897 -1.00905 1
- 108.00 1.00 90.00 3.00 2.00 .20 2.00 2.00 .00972 -1.00982 1
- 109.00 1.00 70.00 3.00 3.00 .38 23.00 27.00 .04745 -1.04981 1
- 111.00 2.00 31.00 2.00 2.00 .00 19.00 10.00 .54159 1.84640 1
- 112.00 1.00 91.00 2.00 3.00 .18 6.00 16.00 .00897 -1.00905 1
- 113.00 1.00 81.00 3.00 2.00 .00 3.00 9.00 .01998 -1.02039 1
- 114.00 2.00 15.00 1.00 2.00 .13 19.00 13.00 .81241 1.23090 1
- 116.00 2.00 1.00 1.00 2.00 .88 15.00 7.00 .93102 1.07410 1
- 117.00 1.00 93.00 3.00 2.00 .18 9.00 15.00 .00764 -1.00770 1
- 118.00 2.00 14.00 1.00 3.00 .15 23.00 18.00 .82447 1.21289 1
- 120.00 1.00 91.00 2.00 2.00 .43 17.00 14.00 .00897 -1.00905 1
- 121.00 1.00 55.00 3.00 2.00 .69 20.00 14.00 .14409 -1.16834 1
- 122.00 1.00 70.00 2.00 3.00 .03 .00 6.00 .04745 -1.04981 1
- 123.00 1.00 25.00 2.00 2.00 .45 4.00 10.00 .65789 -2.92301 1
- 125.00 1.00 91.00 2.00 2.00 .13 .00 3.00 .00897 -1.00905 1
- 126.00 1.00 91.00 3.00 3.00 .23 4.00 6.00 .00897 -1.00905 1
- 127.00 1.00 91.00 3.00 2.00 .00 8.00 8.00 .00897 -1.00905 1
- 128.00 2.00 13.00 2.00 2.00 .65 16.00 14.00 .83592 1.19629 1
- 129.00 1.00 50.00 2.00 2.00 .25 20.00 23.00 .20168 -1.25264 1
- 135.00 1.00 90.00 3.00 3.00 .03 5.00 12.00 .00972 -1.00982 1
- 138.00 1.00 70.00 3.00 3.00 .10 1.00 6.00 .04745 -1.04981 1
- 139.00 2.00 19.00 3.00 3.00 .10 11.00 12.00 .75787 1.31949 1
- 149.00 2.00 50.00 3.00 2.00 .03 .00 .00 .20168 4.95826 1
- 204.00 1.00 50.00 3.00 1.00 .13 .00 1.00 .20168 -1.25264 1
- 205.00 1.00 91.00 3.00 3.00 .72 16.00 18.00 .00897 -1.00905 1
- 206.00 2.00 24.00 1.00 1.00 .10 5.00 21.00 .67592 1.47947 1
- 207.00 1.00 80.00 3.00 3.00 .13 6.00 7.00 .02164 -1.02212 1
- 208.00 1.00 87.00 2.00 2.00 .18 9.00 20.00 .01237 -1.01253 1
- 209.00 1.00 70.00 2.00 2.00 .53 15.00 12.00 .04745 -1.04981 1
- 211.00 1.00 55.00 2.00 1.00 .33 8.00 5.00 .14409 -1.16834 1
- 212.00 1.00 56.00 3.00 1.00 .30 6.00 20.00 .13436 -1.15522 1
- 214.00 1.00 54.00 2.00 2.00 .15 .00 16.00 .15439 -1.18258 1
- 215.00 1.00 71.00 3.00 3.00 .35 12.00 12.00 .04391 -1.04592 1
- 217.00 2.00 36.00 1.00 1.00 .10 12.00 8.00 .44049 2.27020 1
- 218.00 1.00 91.00 2.00 2.00 .05 11.00 25.00 .00897 -1.00905 1
- 219.00 1.00 91.00 2.00 2.00 1.23 11.00 24.00 .00897 -1.00905 1
- 220.00 1.00 91.00 2.00 3.00 .08 8.00 11.00 .00897 -1.00905 1
- 221.00 1.00 91.00 2.00 2.00 .33 5.00 11.00 .00897 -1.00905 1
- 222.00 2.00 36.00 2.00 1.00 .18 5.00 3.00 .44049 2.27020 1
- 223.00 1.00 70.00 2.00 3.00 .18 14.00 3.00 .04745 -1.04981 1
- 224.00 1.00 91.00 2.00 2.00 .43 2.00 10.00 .00897 -1.00905 1
- 225.00 1.00 55.00 2.00 1.00 .18 6.00 11.00 .14409 -1.16834 1
- 229.00 2.00 75.00 2.00 2.00 .40 30.00 25.00 .03212 31.12941 1
- 232.00 1.00 91.00 3.00 2.00 .15 6.00 3.00 .00897 -1.00905 1
- 233.00 1.00 70.00 2.00 1.00 .00 11.00 8.00 .04745 -1.04981 1
- 234.00 1.00 54.00 3.00 2.00 .10 .00 .00 .15439 -1.18258 1
- 237.00 1.00 70.00 3.00 2.00 .18 5.00 25.00 .04745 -1.04981 1
- 241.00 1.00 19.00 2.00 3.00 .33 13.00 9.00 .75787 -4.12995 1
- 304.00 2.00 18.00 2.00 2.00 .26 25.00 6.00 .77245 1.29458 1
- 305.00 1.00 88.00 3.00 2.00 1.35 17.00 29.00 .01142 -1.01155 1
- 306.00 1.00 70.00 2.00 3.00 .63 14.00 33.00 .04745 -1.04981 1
- 307.00 1.00 85.00 2.00 2.00 2.65 18.00 14.00 .01452 -1.01474 1
- 308.00 1.00 13.00 2.00 2.00 .23 5.00 5.00 .83592 -6.09442 1
- 309.00 2.00 13.00 2.00 2.00 .23 7.00 17.00 .83592 1.19629 1
- 311.00 2.00 1.00 2.00 2.00 .50 20.00 14.00 .93102 1.07410 1
- 315.00 1.00 19.00 2.00 3.00 .18 1.00 11.00 .75787 -4.12995 1
- 316.00 1.00 88.00 2.00 2.00 .38 12.00 11.00 .01142 -1.01155 2
- 318.00 1.00 88.00 3.00 2.00 .03 5.00 5.00 .01142 -1.01155 3
- 319.00 2.00 18.00 2.00 3.00 .30 15.00 16.00 .77245 1.29458 1
- 321.00 2.00 15.00 2.00 2.00 .63 15.00 18.00 .81241 1.23090 1
- 322.00 1.00 88.00 3.00 2.00 .40 18.00 15.00 .01142 -1.01155 1
- 325.00 2.00 18.00 2.00 3.00 1.00 28.00 18.00 .77245 1.29458 1
- 329.00 1.00 88.00 3.00 2.00 .03 7.00 11.00 .01142 -1.01155 4
- 332.00 2.00 2.00 2.00 2.00 .05 8.00 9.00 .92562 1.08036 1
-)])
-
-dnl Note: In the above data cases 305, 316 318 and 329 have identical values
-dnl of the 2nd and 3rd variables. We use this for weight testing.
-
-AT_SETUP([LOGISTIC REGRESSION basic test])
-AT_KEYWORDS([categorical categoricals])
-
-LOGIT_TEST_DATA
-
-AT_DATA([lr-data.sps], [dnl
-set format = F12.3.
-set decimal dot.
-data list notable file='lr-data.txt'
- list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
-
-logistic regression
- variables = outcome with survrate
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt lr-data.sps], [0], [dnl
-note: Estimation terminated at iteration number 6 because parameter estimates changed by less than 0.001
-])
-AT_CHECK([cat pspp.csv], [0], [Table: Dependent Variable Encoding
-Original Value,Internal Value
-1.000,.000
-2.000,1.000
-
-Table: Case Processing Summary
-Unweighted Cases,N,Percent
-Included in Analysis,66,100.0%
-Missing Cases,0,.0%
-Total,66,100.0%
-
-note: Estimation terminated at iteration number 6 because parameter estimates changed by less than 0.001
-
-Table: Model Summary
-Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
-1,37.323,.455,.659
-
-Table: Classification Table
-,Observed,,Predicted,,
-,,,outcome,,Percentage Correct
-,,,1.000,2.000,
-Step 1,outcome,1.000,43,5,89.6%
-,,2.000,4,14,77.8%
-,Overall Percentage,,,,86.4%
-
-Table: Variables in the Equation
-,,B,S.E.,Wald,df,Sig.,Exp(B)
-Step 1,survrate,-.081,.019,17.756,1,.000,.922
-,Constant,2.684,.811,10.941,1,.001,14.639
-])
-AT_CLEANUP
-
-AT_SETUP([LOGISTIC REGRESSION missing values])
-AT_KEYWORDS([categorical categoricals])
-
-LOGIT_TEST_DATA
-
-AT_DATA([lr-data.sps], [dnl
-set format = F12.3.
-set decimal dot.
-data list notable file='lr-data.txt'
- list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
-
-missing values survrate (999) avoid (44444) outcome (99).
-
-logistic regression
- variables = outcome with survrate avoid
- .
-])
-
-AT_CHECK([pspp -O format=csv lr-data.sps > run0], [0], [ignore])
-
-dnl Append some cases with missing values into the data.
-cat >> lr-data.txt << HERE
- 105.00 1.00 999.00 3.00 2.00 .35 17.00 20.00 .50110 -2.00440 1
- 106.00 1.00 999.00 2.00 3.00 .38 7.00 15.00 .20168 -1.25264 1
- 107.00 1.00 5.00 3.00 2.00 .28 44444 34 .00897 -1.00905 1
- 108.00 99 5.00 3.00 2.00 .28 4 34 .00897 -1.00905 1
-HERE
-
-AT_CHECK([pspp -O format=csv lr-data.sps > run1], [0], [ignore])
-
-dnl Only the summary information should be different
-AT_CHECK([diff run0 run1], [1], [dnl
-8,10c8,10
-< Included in Analysis,66,100.0%
-< Missing Cases,0,.0%
-< Total,66,100.0%
----
-> Included in Analysis,66,94.3%
-> Missing Cases,4,5.7%
-> Total,70,100.0%
-])
-
-AT_CLEANUP
-
-
-
-dnl Check that a weighted dataset is interpreted correctly
-dnl To do this, the same data set is used, one weighted, one not.
-dnl The weighted dataset omits certain cases which are identical
-AT_SETUP([LOGISTIC REGRESSION weights])
-AT_KEYWORDS([categorical categoricals])
-
-LOGIT_TEST_DATA
-
-AT_DATA([lr-data-unweighted.sps], [dnl
-set format = F12.3.
-set decimal dot.
-data list notable file='lr-data.txt'
- list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
-
-logistic regression
- variables = outcome with survrate
- .
-])
-
-AT_DATA([lr-data-weighted.sps], [dnl
-set format = F12.3.
-set decimal dot.
-data list notable file='lr-data.txt'
- list /id outcome survrate prognos amttreat gsi avoid intrus pre_1 lre_1 w *.
-
-weight by w.
-
-* Omit duplicate cases.
-select if id <> 305 and id <> 316 and id <> 318.
-
-logistic regression
- variables = outcome with survrate
- .
-])
-
-
-AT_CHECK([pspp -O format=csv lr-data-unweighted.sps > unweighted-result], [0], [ignore])
-AT_CHECK([pspp -O format=csv lr-data-weighted.sps > weighted-result], [0], [ignore])
-
-dnl The only difference should be the summary information, since
-dnl this displays the unweighted totals.
-AT_CHECK([diff unweighted-result weighted-result], [1], [dnl
-8c8
-< Included in Analysis,66,100.0%
----
-> Included in Analysis,63,100.0%
-10c10
-< Total,66,100.0%
----
-> Total,63,100.0%
-22,23c22,23
-< Step 1,outcome,1.000,43,5,89.6%
-< ,,2.000,4,14,77.8%
----
-> Step 1,outcome,1.000,43.000,5.000,89.6%
-> ,,2.000,4.000,14.000,77.8%
-])
-
-
-AT_CLEANUP
-
-
-dnl Check that the /NOCONST option works as intended.
-dnl The results this produces are very similar to those
-dnl at the example in http://www.ats.ucla.edu/stat/SPSS/faq/logregconst.htm
-AT_SETUP([LOGISTIC REGRESSION without constant])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([non-const.sps], [dnl
-set format=F20.3.
-
-input program.
- loop #i = 1 to 200.
- compute female = (#i > 91).
- end case.
- end loop.
-end file.
-end input program.
-
-compute constant = 1.
-
-logistic regression female with constant /noconst.
-])
-
-AT_CHECK([pspp -O format=csv non-const.sps], [0], [dnl
-Table: Dependent Variable Encoding
-Original Value,Internal Value
-.00,.000
-1.00,1.000
-
-Table: Case Processing Summary
-Unweighted Cases,N,Percent
-Included in Analysis,200,100.0%
-Missing Cases,0,.0%
-Total,200,100.0%
-
-note: Estimation terminated at iteration number 2 because parameter estimates changed by less than 0.001
-
-Table: Model Summary
-Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
-1,275.637,.008,.011
-
-Table: Classification Table
-,Observed,,Predicted,,
-,,,female,,Percentage Correct
-,,,.00,1.00,
-Step 1,female,.00,0,91,.0%
-,,1.00,0,109,100.0%
-,Overall Percentage,,,,54.5%
-
-Table: Variables in the Equation
-,,B,S.E.,Wald,df,Sig.,Exp(B)
-Step 1,constant,.180,.142,1.616,1,.204,1.198
-])
-
-AT_CLEANUP
-
-
-
-dnl Check that if somebody passes a dependent variable which is not dichtomous,
-dnl then an error is raised.
-AT_SETUP([LOGISTIC REGRESSION non-dichotomous dep var])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([non-dich.sps], [dnl
-data list notable list /y x1 x2 x3 x4.
-begin data.
-1 2 3 4 5
-0 2 3 4 8
-2 3 4 5 6
-end data.
-
-logistic regression y with x1 x2 x3 x4.
-])
-
-AT_CHECK([pspp -O format=csv non-dich.sps], [1],
- [dnl
-error: Dependent variable's values are not dichotomous.
-])
-
-AT_CLEANUP
-
-
-
-dnl An example to check the behaviour of LOGISTIC REGRESSION with a categorical
-dnl variable. This examṕle was inspired from that at:
-dnl http://www.ats.ucla.edu/stat/spss/dae/logit.htm
-AT_SETUP([LOGISTIC REGRESSION with categorical])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([lr-cat.data], [dnl
- 620 3.07 2 4
- 800 4.00 3 9
- 580 3.40 2 4
- 600 3.13 2 4
- 540 2.70 2 4
- 660 3.31 4 4
- 480 3.58 1 9
- 620 4.00 1 9
- 680 3.98 2 9
- 580 3.40 4 4
- 760 3.35 3 4
- 700 3.72 2 4
- 460 3.64 1 9
- 540 3.28 3 4
- 680 3.48 3 4
- 740 3.31 1 4
- 460 3.77 3 4
- 740 3.54 1 4
- 600 3.63 3 4
- 620 3.05 2 4
- 560 3.04 3 4
- 520 2.70 3 4
- 640 3.35 3 4
- 620 3.58 2 4
- 660 3.70 4 9
- 500 2.86 4 4
- 640 3.50 2 4
- 720 4.00 3 4
- 720 3.94 3 4
- 400 3.65 2 4
- 800 2.90 2 4
- 520 2.90 3 4
- 440 3.24 4 4
- 580 3.51 2 4
- 500 3.31 3 4
- 440 3.22 1 4
- 540 3.17 1 9
- 420 3.02 1 4
- 780 3.22 2 9
- 440 3.13 4 4
- 800 3.66 1 9
- 580 3.32 2 9
- 480 2.67 2 9
- 700 4.00 1 9
- 740 2.97 2 9
- 700 3.83 2 4
- 640 3.93 2 4
- 800 3.90 2 4
- 400 3.38 2 4
- 700 3.52 2 4
- 680 3.00 4 9
- 540 3.20 1 4
- 580 4.00 2 4
- 780 4.00 2 9
- 220 2.83 3 4
- 580 3.20 2 9
- 580 3.50 2 4
- 620 3.30 1 4
- 520 3.65 4 9
- 600 3.38 3 9
- 660 3.77 3 4
- 580 2.86 4 9
- 580 3.46 2 9
- 560 3.36 3 4
- 740 4.00 3 9
- 480 3.44 3 4
- 640 3.19 4 9
- 600 3.54 1 9
- 540 3.38 4 4
- 500 2.81 3 4
- 360 2.56 3 4
- 460 3.15 4 4
- 460 2.63 2 4
- 440 2.76 2 4
- 740 3.62 4 4
- 380 3.38 2 4
- 640 3.63 1 9
- 800 3.73 1 4
- 660 3.67 2 4
- 760 3.00 2 9
- 420 2.96 1 4
- 740 3.74 4 4
- 800 3.75 2 4
- 620 3.40 2 4
- 660 3.67 3 9
- 400 3.35 3 4
- 680 3.14 2 4
- 660 3.47 3 9
- 660 3.63 2 9
- 420 3.41 4 4
- 660 4.00 1 4
- 680 3.70 2 4
- 620 3.23 3 9
- 520 3.35 3 4
- 500 4.00 3 4
- 400 3.36 2 4
- 700 3.56 1 9
- 540 3.81 1 9
- 520 2.68 3 9
- 540 3.50 2 4
- 700 4.00 2 4
- 600 3.64 3 9
- 800 3.31 3 4
- 520 3.29 1 4
- 580 3.69 1 4
- 380 3.43 3 4
- 560 3.19 3 4
- 760 2.81 1 9
- 540 3.13 2 4
- 660 3.14 2 9
- 520 3.81 1 9
- 680 3.19 4 4
- 540 3.78 4 4
- 500 3.57 3 4
- 660 3.49 2 4
- 340 3.00 2 9
- 400 3.15 2 9
- 420 3.92 4 4
- 760 3.35 2 9
- 700 2.94 2 4
- 540 3.04 1 4
- 780 3.87 4 4
- 560 3.78 2 4
- 700 3.82 3 4
- 400 2.93 3 4
- 440 3.45 2 9
- 800 3.47 3 4
- 340 3.15 3 4
- 520 4.00 1 9
- 520 3.15 3 4
- 600 2.98 2 9
- 420 2.69 2 4
- 460 3.44 2 4
- 620 3.71 1 9
- 480 3.13 2 4
- 580 3.40 3 4
- 540 3.39 3 9
- 540 3.94 3 4
- 440 2.98 3 4
- 380 3.59 4 4
- 500 2.97 4 4
- 340 2.92 3 4
- 440 3.15 2 4
- 600 3.48 2 4
- 420 2.67 3 4
- 460 3.07 2 4
- 460 3.45 3 9
- 480 3.39 4 4
- 480 2.78 3 4
- 720 3.42 2 9
- 680 3.67 2 9
- 800 3.89 2 4
- 360 3.00 3 4
- 620 3.17 2 9
- 700 3.52 4 9
- 540 3.19 2 4
- 580 3.30 2 4
- 800 4.00 3 9
- 660 3.33 2 4
- 380 3.34 3 4
- 720 3.84 3 4
- 600 3.59 2 4
- 500 3.03 3 4
- 640 3.81 2 4
- 540 3.49 1 9
- 680 3.85 3 9
- 540 3.84 2 9
- 460 2.93 3 4
- 380 2.94 3 4
- 620 3.22 2 4
- 740 3.37 4 4
- 620 4.00 2 4
- 800 3.74 1 9
- 400 3.31 3 4
- 540 3.46 4 4
- 620 3.18 2 9
- 480 2.91 1 9
- 300 2.84 2 9
- 440 2.48 4 4
- 640 2.79 2 4
- 400 3.23 4 9
- 680 3.46 2 9
- 620 3.37 1 9
- 700 3.92 2 4
- 620 3.37 2 9
- 620 3.63 2 4
- 620 3.95 3 9
- 560 2.52 2 4
- 520 2.62 2 4
- 600 3.35 2 4
- 700 4.00 1 4
- 640 3.67 3 4
- 640 4.00 3 4
- 520 2.93 4 4
- 620 3.21 4 4
- 680 3.99 3 4
- 660 3.34 3 4
- 700 3.45 3 4
- 560 3.36 1 9
- 800 2.78 2 4
- 500 3.88 4 4
- 700 3.65 2 4
- 680 3.76 3 9
- 660 3.07 3 4
- 580 3.46 4 4
- 460 2.87 2 4
- 600 3.31 4 4
- 620 3.94 4 4
- 400 3.05 2 4
- 800 3.43 2 9
- 600 3.58 1 9
- 580 3.36 2 4
- 540 3.16 3 4
- 500 2.71 2 4
- 600 3.28 3 4
- 600 2.82 4 4
- 460 3.58 2 4
- 520 2.85 3 4
- 740 3.52 4 9
- 500 3.95 4 4
- 560 3.61 3 4
- 620 3.45 2 9
- 640 3.51 2 4
- 660 3.44 2 9
- 660 2.91 3 9
- 540 3.28 1 4
- 560 2.98 1 9
- 800 3.97 1 4
- 720 3.77 3 4
- 720 3.64 1 9
- 480 3.71 4 9
- 680 3.34 2 4
- 680 3.11 2 4
- 540 2.81 3 4
- 620 3.75 2 9
- 540 3.12 1 4
- 560 3.48 2 9
- 720 3.40 3 4
- 680 3.90 1 4
- 640 3.76 3 4
- 560 3.16 1 4
- 520 3.30 2 9
- 640 3.12 3 4
- 580 3.57 3 4
- 540 3.55 4 9
- 780 3.63 4 9
- 600 3.89 1 9
- 800 4.00 1 9
- 580 3.29 4 4
- 360 3.27 3 4
- 800 4.00 2 9
- 640 3.52 4 4
- 720 3.45 4 4
- 580 3.06 2 4
- 580 3.02 2 4
- 500 3.60 3 9
- 580 3.12 3 9
- 600 2.82 4 4
- 620 3.99 3 4
- 700 4.00 3 4
- 480 4.00 2 4
- 560 2.95 2 4
- 560 4.00 3 4
- 560 2.65 3 9
- 400 3.08 2 4
- 480 2.62 2 9
- 640 3.86 3 4
- 480 3.57 2 4
- 540 3.51 2 4
- 380 3.33 4 4
- 680 3.64 3 4
- 400 3.51 3 4
- 340 2.90 1 4
- 700 3.08 2 4
- 480 3.02 1 9
- 600 3.15 2 9
- 780 3.80 3 9
- 520 3.74 2 9
- 520 3.51 2 4
- 640 3.73 3 4
- 560 3.32 4 4
- 620 2.85 2 4
- 700 3.28 1 4
- 760 4.00 1 9
- 800 3.60 2 4
- 580 3.34 2 4
- 540 3.77 2 9
- 640 3.17 2 4
- 540 3.02 4 4
- 680 3.08 4 4
- 680 3.31 2 4
- 680 2.96 3 9
- 700 2.88 2 4
- 580 3.77 4 4
- 540 3.49 2 9
- 700 3.56 2 9
- 600 3.56 2 9
- 560 3.59 2 4
- 640 2.94 2 9
- 560 3.33 4 4
- 620 3.69 3 4
- 680 3.27 2 9
- 460 3.14 3 4
- 500 3.53 4 4
- 620 3.33 3 4
- 600 3.62 3 4
- 500 3.01 4 4
- 740 3.34 4 4
- 560 3.69 3 9
- 620 3.95 3 9
- 740 3.86 2 9
- 800 3.53 1 9
- 620 3.78 3 4
- 700 3.27 2 4
- 540 3.78 2 9
- 700 3.65 2 4
- 800 3.22 1 9
- 560 3.59 2 9
- 800 3.15 4 4
- 520 3.90 3 9
- 520 3.74 4 9
- 480 2.55 1 4
- 800 4.00 4 4
- 620 3.09 4 4
- 560 3.49 4 4
- 500 3.17 3 4
- 480 3.40 2 4
- 460 2.98 1 4
- 580 3.58 1 9
- 640 3.30 2 4
- 480 3.45 2 4
- 440 3.17 2 4
- 660 3.32 1 4
- 500 3.08 3 4
- 660 3.94 2 4
- 720 3.31 1 4
- 460 3.64 3 9
- 500 2.93 4 4
- 800 3.54 3 4
- 580 2.93 2 4
- 620 3.61 1 9
- 500 2.98 3 4
- 660 4.00 2 9
- 560 3.24 4 4
- 560 2.42 2 4
- 580 3.80 2 4
- 500 3.23 4 4
- 680 2.42 1 9
- 580 3.46 3 4
- 800 3.91 3 4
- 700 2.90 4 4
- 520 3.12 2 4
- 300 2.92 4 4
- 560 3.43 3 4
- 620 3.63 3 4
- 500 2.79 4 4
- 360 3.14 1 4
- 640 3.94 2 9
- 460 3.99 3 9
- 300 3.01 3 4
- 520 2.73 2 4
- 600 3.47 2 9
- 580 3.25 1 4
- 520 3.10 4 4
- 620 3.43 3 4
- 380 2.91 4 4
- 660 3.59 3 4
- 660 3.95 2 9
- 540 3.33 3 4
- 740 4.00 3 4
- 640 3.38 3 4
- 600 3.89 3 4
- 720 3.88 3 4
- 580 4.00 3 4
- 420 2.26 4 4
- 520 4.00 2 9
- 800 3.70 1 9
- 700 4.00 1 9
- 480 3.43 2 4
- 660 3.45 4 4
- 520 3.25 3 4
- 560 2.71 3 4
- 600 3.32 2 4
- 580 2.88 2 4
- 660 3.88 2 9
- 600 3.22 1 4
- 580 4.00 1 4
- 660 3.60 3 9
- 500 3.35 2 4
- 520 2.98 2 4
- 660 3.49 2 9
- 560 3.07 2 4
- 500 3.13 2 9
- 720 3.50 3 9
- 440 3.39 2 9
- 640 3.95 2 9
- 380 3.61 3 4
- 800 3.05 2 9
- 520 3.19 3 9
- 600 3.40 3 4
-])
-
-AT_DATA([lr-cat.sps], [dnl
-set format=F20.3.
-
-data list notable list file='lr-cat.data' /b1 b2 bcat y.
-
-logistic regression
- y with b1 b2 bcat
- /categorical = bcat
- .
-])
-
-AT_CHECK([pspp -O format=csv lr-cat.sps], [0], [dnl
-Table: Dependent Variable Encoding
-Original Value,Internal Value
-4.000,.000
-9.000,1.000
-
-Table: Case Processing Summary
-Unweighted Cases,N,Percent
-Included in Analysis,400,100.0%
-Missing Cases,0,.0%
-Total,400,100.0%
-
-note: Estimation terminated at iteration number 4 because parameter estimates changed by less than 0.001
-
-Table: Model Summary
-Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
-1,458.517,.098,.138
-
-Table: Categorical Variables' Codings
-,,Frequency,Parameter coding,,
-,,,(1),(2),(3)
-bcat,1.000,61,1,0,0
-,2.000,151,0,1,0
-,3.000,121,0,0,1
-,4.000,67,0,0,0
-
-Table: Classification Table
-,Observed,,Predicted,,
-,,,y,,Percentage Correct
-,,,4.000,9.000,
-Step 1,y,4.000,254,19,93.0%
-,,9.000,97,30,23.6%
-,Overall Percentage,,,,71.0%
-
-Table: Variables in the Equation
-,,B,S.E.,Wald,df,Sig.,Exp(B)
-Step 1,b1,.002,.001,4.284,1,.038,1.002
-,b2,.804,.332,5.872,1,.015,2.235
-,bcat,,,20.895,3,.000,
-,bcat(1),1.551,.418,13.788,1,.000,4.718
-,bcat(2),.876,.367,5.706,1,.017,2.401
-,bcat(3),.211,.393,.289,1,.591,1.235
-,Constant,-5.541,1.138,23.709,1,.000,.004
-])
-AT_CLEANUP
-
-
-
-dnl This example is inspired by http://www.ats.ucla.edu/stat/spss/output/logistic.htm
-AT_SETUP([LOGISTIC REGRESSION with cat var 2])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([lr-cat2.data], [dnl
- 60.00 1.00 8.00 50.00
- 47.00 .00 9.00 42.00
- 57.00 1.00 7.00 53.00
- 60.00 .00 8.00 53.00
- 68.00 .00 8.00 66.00
- 63.00 .00 8.00 55.00
- 65.00 .00 8.00 63.00
- 52.00 .00 8.00 61.00
- 34.00 .00 9.00 42.00
- 37.00 .00 8.00 39.00
- 68.00 1.00 9.00 69.00
- 60.00 .00 9.00 61.00
- 44.00 .00 9.00 58.00
- 42.00 .00 8.00 47.00
- 57.00 1.00 7.00 61.00
- 55.00 1.00 8.00 50.00
- 55.00 .00 9.00 58.00
- 44.00 .00 8.00 63.00
- 50.00 1.00 9.00 66.00
- 44.00 .00 8.00 39.00
- 55.00 .00 8.00 58.00
- 44.00 .00 8.00 50.00
- 47.00 1.00 7.00 34.00
- 48.00 .00 8.00 44.00
- 45.00 .00 7.00 31.00
- 43.00 .00 8.00 50.00
- 39.00 .00 8.00 42.00
- 63.00 .00 9.00 50.00
- 47.00 .00 8.00 58.00
- 42.00 .00 7.00 50.00
- 50.00 .00 9.00 36.00
- 47.00 .00 7.00 33.00
- 60.00 .00 9.00 61.00
- 47.00 .00 7.00 42.00
- 68.00 1.00 9.00 69.00
- 52.00 .00 8.00 54.00
- 63.00 1.00 9.00 61.00
- 65.00 1.00 9.00 61.00
- 63.00 1.00 9.00 53.00
- 57.00 .00 8.00 51.00
- 34.00 .00 8.00 36.00
- 50.00 .00 8.00 39.00
- 52.00 1.00 7.00 56.00
- 45.00 .00 7.00 34.00
- 47.00 1.00 7.00 53.00
- 34.00 .00 7.00 39.00
- 50.00 1.00 8.00 55.00
- 60.00 .00 9.00 58.00
- 63.00 .00 8.00 58.00
- 35.00 .00 7.00 51.00
- 50.00 .00 8.00 58.00
- 68.00 .00 8.00 63.00
- 41.00 .00 9.00 34.00
- 47.00 .00 8.00 47.00
- 76.00 .00 9.00 64.00
- 44.00 .00 8.00 44.00
- 36.00 .00 9.00 50.00
- 68.00 1.00 9.00 55.00
- 47.00 1.00 8.00 50.00
- 50.00 .00 7.00 53.00
- 68.00 .00 8.00 74.00
- 39.00 .00 7.00 44.00
- 50.00 .00 8.00 55.00
- 52.00 .00 9.00 61.00
- 47.00 .00 8.00 53.00
- 39.00 .00 7.00 47.00
- 55.00 1.00 9.00 49.00
- 68.00 1.00 8.00 50.00
- 52.00 1.00 9.00 63.00
- 55.00 .00 8.00 58.00
- 57.00 .00 8.00 55.00
- 66.00 1.00 9.00 61.00
- 65.00 1.00 7.00 58.00
- 42.00 .00 7.00 42.00
- 68.00 1.00 7.00 59.00
- 60.00 1.00 9.00 61.00
- 52.00 .00 8.00 55.00
- 57.00 1.00 7.00 54.00
- 42.00 .00 9.00 50.00
- 42.00 .00 8.00 47.00
- 57.00 .00 8.00 50.00
- 47.00 .00 7.00 45.00
- 44.00 .00 7.00 40.00
- 43.00 .00 9.00 55.00
- 31.00 .00 8.00 39.00
- 37.00 .00 7.00 33.00
- 63.00 1.00 7.00 63.00
- 47.00 .00 8.00 39.00
- 57.00 1.00 8.00 63.00
- 52.00 .00 8.00 44.00
- 44.00 .00 7.00 35.00
- 52.00 .00 7.00 55.00
- 55.00 .00 7.00 69.00
- 52.00 .00 8.00 53.00
- 55.00 .00 9.00 61.00
- 65.00 1.00 9.00 63.00
- 55.00 .00 8.00 44.00
- 63.00 .00 7.00 65.00
- 44.00 .00 7.00 39.00
- 47.00 .00 7.00 36.00
- 63.00 1.00 9.00 55.00
- 68.00 .00 8.00 66.00
- 34.00 .00 8.00 39.00
- 47.00 .00 9.00 50.00
- 50.00 .00 9.00 58.00
- 63.00 .00 8.00 66.00
- 44.00 .00 7.00 34.00
- 44.00 .00 8.00 50.00
- 50.00 .00 8.00 53.00
- 47.00 1.00 9.00 69.00
- 65.00 .00 9.00 58.00
- 57.00 .00 8.00 47.00
- 39.00 .00 8.00 39.00
- 47.00 .00 8.00 53.00
- 50.00 1.00 7.00 63.00
- 50.00 .00 8.00 50.00
- 63.00 .00 9.00 53.00
- 73.00 1.00 9.00 61.00
- 44.00 .00 7.00 47.00
- 47.00 .00 8.00 42.00
- 47.00 .00 8.00 58.00
- 36.00 .00 7.00 61.00
- 57.00 1.00 8.00 55.00
- 53.00 1.00 8.00 57.00
- 63.00 .00 7.00 66.00
- 50.00 .00 8.00 34.00
- 47.00 .00 9.00 48.00
- 57.00 1.00 8.00 58.00
- 39.00 .00 8.00 53.00
- 42.00 .00 8.00 42.00
- 42.00 .00 9.00 31.00
- 42.00 .00 8.00 72.00
- 46.00 .00 8.00 44.00
- 55.00 .00 8.00 42.00
- 42.00 .00 8.00 47.00
- 50.00 .00 8.00 44.00
- 44.00 .00 9.00 39.00
- 73.00 1.00 8.00 69.00
- 71.00 1.00 9.00 58.00
- 50.00 .00 9.00 49.00
- 63.00 1.00 7.00 54.00
- 42.00 .00 8.00 36.00
- 47.00 .00 7.00 42.00
- 39.00 .00 9.00 26.00
- 63.00 .00 8.00 58.00
- 50.00 .00 8.00 55.00
- 65.00 1.00 8.00 55.00
- 76.00 1.00 9.00 67.00
- 71.00 1.00 8.00 66.00
- 39.00 .00 9.00 47.00
- 47.00 1.00 9.00 63.00
- 60.00 .00 7.00 50.00
- 63.00 .00 9.00 55.00
- 54.00 1.00 9.00 55.00
- 55.00 1.00 8.00 58.00
- 57.00 .00 8.00 61.00
- 55.00 1.00 9.00 63.00
- 42.00 .00 7.00 50.00
- 50.00 .00 8.00 44.00
- 55.00 .00 8.00 42.00
- 42.00 .00 7.00 50.00
- 34.00 .00 8.00 39.00
- 65.00 .00 9.00 46.00
- 52.00 .00 7.00 58.00
- 44.00 .00 8.00 39.00
- 65.00 1.00 9.00 66.00
- 47.00 .00 8.00 42.00
- 41.00 .00 7.00 39.00
- 68.00 .00 9.00 63.00
- 63.00 1.00 8.00 72.00
- 52.00 .00 8.00 53.00
- 57.00 .00 8.00 50.00
- 68.00 .00 8.00 55.00
- 42.00 .00 8.00 56.00
- 47.00 .00 8.00 48.00
- 73.00 1.00 9.00 58.00
- 39.00 .00 8.00 50.00
- 63.00 1.00 9.00 69.00
- 60.00 .00 8.00 55.00
- 65.00 1.00 9.00 66.00
- 73.00 1.00 8.00 63.00
- 52.00 .00 8.00 55.00
- 36.00 .00 8.00 42.00
- 28.00 .00 7.00 44.00
- 47.00 .00 8.00 44.00
- 57.00 .00 7.00 47.00
- 34.00 .00 7.00 29.00
- 47.00 .00 9.00 66.00
- 57.00 .00 8.00 58.00
- 60.00 1.00 9.00 50.00
- 50.00 .00 9.00 47.00
- 73.00 1.00 9.00 55.00
- 52.00 1.00 8.00 47.00
- 55.00 .00 8.00 53.00
- 47.00 .00 8.00 53.00
- 50.00 .00 8.00 61.00
- 61.00 .00 7.00 44.00
- 52.00 .00 9.00 53.00
- 47.00 .00 7.00 40.00
- 47.00 .00 7.00 50.00
-])
-
-AT_DATA([stringcat.sps], [dnl
-set format=F20.3 /small=0.
-data list notable file='lr-cat2.data' list /read honcomp wiz science *.
-
-string ses(a1).
-recode wiz (7 = "a") (8 = "b") (9 = "c") into ses.
-
-logistic regression honcomp with read science ses
- /categorical = ses.
-
-])
-
-AT_CHECK([pspp -O format=csv stringcat.sps], [0], [dnl
-Table: Dependent Variable Encoding
-Original Value,Internal Value
-.000,.000
-1.000,1.000
-
-Table: Case Processing Summary
-Unweighted Cases,N,Percent
-Included in Analysis,200,100.0%
-Missing Cases,0,.0%
-Total,200,100.0%
-
-note: Estimation terminated at iteration number 5 because parameter estimates changed by less than 0.001
-
-Table: Model Summary
-Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
-1,165.701,.280,.408
-
-Table: Categorical Variables' Codings
-,,Frequency,Parameter coding,
-,,,(1),(2)
-ses,a,47,1,0
-,b,95,0,1
-,c,58,0,0
-
-Table: Classification Table
-,Observed,,Predicted,,
-,,,honcomp,,Percentage Correct
-,,,.000,1.000,
-Step 1,honcomp,.000,132,15,89.8%
-,,1.000,26,27,50.9%
-,Overall Percentage,,,,79.5%
-
-Table: Variables in the Equation
-,,B,S.E.,Wald,df,Sig.,Exp(B)
-Step 1,read,.098,.025,15.199,1,.000,1.103
-,science,.066,.027,5.867,1,.015,1.068
-,ses,,,6.690,2,.035,
-,ses(1),.058,.532,.012,1,.913,1.060
-,ses(2),-1.013,.444,5.212,1,.022,.363
-,Constant,-9.561,1.662,33.113,1,.000,.000
-])
-
-AT_CLEANUP
-
-
-dnl Check that it doesn't crash if a categorical variable
-dnl has only one distinct value
-AT_SETUP([LOGISTIC REGRESSION identical categories])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([crash.sps], [dnl
-data list notable list /y x1 x2*.
-begin data
-0 1 1
-1 2 1
-end data.
-
-logistic regression y with x1 x2
- /categorical = x2.
-])
-
-AT_CHECK([pspp -O format=csv crash.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-dnl Test that missing values on the categorical predictors are treated
-dnl properly.
-AT_SETUP([LOGISTIC REGRESSION missing categoricals])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([data.txt], [dnl
- .00 3.69 .00
- .00 1.16 1.00
- 1.00 -12.99 .00
- .00 2.97 1.00
- .00 20.48 .00
- .00 4.90 .00
- 1.00 -4.38 .00
- .00 -1.69 1.00
- 1.00 -5.71 .00
- 1.00 -14.28 .00
- .00 9.00 .00
- .00 2.89 1.00
- .00 13.51 1.00
- .00 23.32 1.00
- .00 2.31 1.00
- .00 -2.07 1.00
- 1.00 -4.52 1.00
- 1.00 -5.83 .00
- 1.00 -1.91 .00
- 1.00 -11.12 1.00
- .00 -1.51 .00
- .00 6.59 1.00
- .00 19.28 1.00
- .00 5.94 .00
- .00 8.21 1.00
- .00 8.11 1.00
- .00 2.49 .00
- .00 9.62 .00
- 1.00 -20.74 1.00
- .00 -1.41 1.00
- .00 15.15 1.00
- .00 9.39 .00
- 1.00 -15.14 1.00
- 1.00 -5.86 .00
- 1.00 -11.64 1.00
- 1.00 -14.36 .00
- 1.00 -8.95 1.00
- 1.00 -16.42 1.00
- 1.00 -1.04 1.00
- .00 12.89 1.00
- .00 -7.08 1.00
- .00 4.87 1.00
- .00 11.53 1.00
- 1.00 -6.24 1.00
- .00 1.25 1.00
- .00 4.39 1.00
- .00 3.17 .00
- .00 19.39 1.00
- .00 13.03 1.00
- .00 2.43 .00
- 1.00 -14.73 1.00
- .00 8.25 1.00
- 1.00 -13.28 1.00
- .00 5.27 1.00
- 1.00 -3.46 1.00
- .00 13.81 1.00
- .00 1.35 1.00
- 1.00 -3.94 1.00
- .00 20.73 1.00
- 1.00 -15.40 .00
- 1.00 -11.01 1.00
- .00 4.56 .00
- 1.00 -15.35 1.00
- .00 15.21 .00
- .00 5.34 1.00
- 1.00 -21.55 1.00
- .00 10.12 1.00
- .00 -.73 1.00
- .00 15.28 1.00
- .00 11.08 1.00
- 1.00 -8.24 .00
- .00 2.46 .00
- .00 9.60 .00
- .00 11.24 .00
- .00 14.13 1.00
- .00 19.72 1.00
- .00 5.58 .00
- .00 26.23 1.00
- .00 7.25 .00
- 1.00 -.79 .00
- .00 6.24 .00
- 1.00 1.16 .00
- 1.00 -7.89 1.00
- 1.00 -1.86 1.00
- 1.00 -10.80 1.00
- 1.00 -5.51 .00
- .00 7.51 .00
- .00 11.18 .00
- .00 8.73 .00
- 1.00 -11.21 1.00
- 1.00 -13.24 .00
- .00 19.34 .00
- .00 9.32 1.00
- .00 17.97 1.00
- 1.00 -1.56 1.00
- 1.00 -3.13 .00
- .00 3.98 .00
- .00 -1.21 1.00
- .00 2.37 .00
- 1.00 -18.03 1.00
-])
-
-AT_DATA([miss.sps], [dnl
-data list notable file='data.txt' list /y x1 cat0*.
-
-logistic regression y with x1 cat0
- /categorical = cat0.
-])
-
-AT_CHECK([pspp -O format=csv miss.sps > file1], [0], [ignore])
-
-dnl Append a case with a missing categorical.
-AT_CHECK([echo '1 34 .' >> data.txt], [0], [ignore])
-
-AT_CHECK([pspp -O format=csv miss.sps > file2], [0], [ignore])
-
-AT_CHECK([diff file1 file2], [1], [dnl
-8,10c8,10
-< Included in Analysis,100,100.0%
-< Missing Cases,0,.0%
-< Total,100,100.0%
----
-> Included in Analysis,100,99.0%
-> Missing Cases,1,1.0%
-> Total,101,100.0%
-])
-
-AT_CLEANUP
-
-
-dnl Check that the confidence intervals are properly reported.
-dnl Use an example with categoricals, because that was buggy at
-dnl one point. The data in this example comes from:
-dnl http://people.ysu.edu/~gchang/SPSSE/SPSS_lab2Regression.pdf
-AT_SETUP([LOGISTIC REGRESSION confidence interval])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([ci.sps], [dnl
-set FORMAT=F20.3
-data list notable list /disease age sciostat sector savings *.
-begin data.
-0 33 1 1 1
-0 35 1 1 1
-0 6 1 1 0
-0 60 1 1 1
-1 18 3 1 0
-0 26 3 1 0
-0 6 3 1 0
-1 31 2 1 1
-1 26 2 1 0
-0 37 2 1 0
-0 23 1 1 0
-0 23 1 1 0
-0 27 1 1 1
-1 9 1 1 1
-1 37 1 2 1
-1 22 1 2 1
-1 67 1 2 1
-0 8 1 2 1
-1 6 1 2 1
-1 15 1 2 1
-1 21 2 2 1
-1 32 2 2 1
-1 16 1 2 1
-0 11 2 2 0
-0 14 3 2 0
-0 9 2 2 0
-0 18 2 2 0
-0 2 3 1 0
-0 61 3 1 1
-0 20 3 1 0
-0 16 3 1 0
-0 9 2 1 0
-0 35 2 1 1
-0 4 1 1 1
-0 44 3 2 0
-1 11 3 2 0
-0 3 2 2 1
-0 6 3 2 0
-1 17 2 2 0
-0 1 3 2 1
-1 53 2 2 1
-1 13 1 2 0
-0 24 1 2 0
-1 70 1 2 1
-1 16 3 2 1
-0 12 2 2 1
-1 20 3 2 1
-0 65 3 2 1
-1 40 2 2 0
-1 38 2 2 1
-1 68 2 2 1
-1 74 1 2 1
-1 14 1 2 1
-1 27 1 2 1
-0 31 1 2 1
-0 18 1 2 1
-0 39 1 2 0
-0 50 1 2 1
-0 31 1 2 1
-0 61 1 2 1
-0 18 3 1 0
-0 5 3 1 0
-0 2 3 1 1
-0 16 3 1 0
-1 59 3 1 1
-0 22 3 1 0
-0 24 1 1 1
-0 30 1 1 1
-0 46 1 1 1
-0 28 1 1 0
-0 27 1 1 1
-1 27 1 1 0
-0 28 1 1 1
-1 52 1 1 1
-0 11 3 1 1
-0 6 2 1 1
-0 46 3 1 0
-1 20 2 1 1
-0 3 1 1 1
-0 18 2 1 0
-0 25 2 1 0
-0 6 3 1 1
-1 65 3 1 1
-0 51 3 1 1
-0 39 2 1 1
-0 8 1 1 1
-0 8 2 1 0
-0 14 3 1 0
-0 6 3 1 0
-0 6 3 1 1
-0 7 3 1 0
-0 4 3 1 0
-0 8 3 1 0
-0 9 2 1 0
-1 32 3 1 0
-0 19 3 1 0
-0 11 3 1 0
-0 35 3 1 0
-0 16 1 1 0
-0 1 1 1 1
-0 6 1 1 1
-0 27 1 1 1
-0 25 1 1 1
-0 18 1 1 0
-0 37 3 1 0
-1 33 3 1 0
-0 27 2 1 0
-0 2 1 1 0
-0 8 2 1 0
-0 5 1 1 0
-0 1 1 1 1
-0 32 1 1 0
-1 25 1 1 1
-0 15 1 2 0
-0 15 1 2 1
-0 26 1 2 1
-1 42 1 2 1
-0 7 1 2 1
-0 2 1 2 0
-1 65 1 2 1
-0 33 2 2 1
-1 8 2 2 0
-0 30 2 2 0
-0 5 3 2 0
-0 15 3 2 0
-1 60 3 2 1
-1 13 3 2 1
-0 70 3 1 1
-0 5 3 1 0
-0 3 3 1 1
-0 50 2 1 1
-0 6 2 1 0
-0 12 2 1 1
-1 39 3 2 0
-0 15 2 2 1
-1 35 2 2 0
-0 2 2 2 1
-0 17 3 2 0
-1 43 3 2 1
-0 30 2 2 1
-0 11 1 2 1
-1 39 1 2 1
-0 32 1 2 1
-0 17 1 2 1
-0 3 3 2 1
-0 7 3 2 0
-0 2 2 2 0
-1 64 2 2 1
-1 13 1 2 2
-1 15 2 2 1
-0 48 2 2 1
-0 23 1 2 1
-1 48 1 2 0
-0 25 1 2 1
-0 12 1 2 1
-1 46 1 2 1
-0 79 1 2 1
-0 56 1 2 1
-0 8 1 2 1
-1 29 3 1 0
-1 35 3 1 0
-1 11 3 1 0
-0 69 3 1 1
-1 21 3 1 0
-0 13 3 1 0
-0 21 1 1 1
-1 32 1 1 1
-1 24 1 1 0
-0 24 1 1 1
-0 73 1 1 1
-0 42 1 1 1
-1 34 1 1 1
-0 30 2 1 0
-0 7 2 1 0
-1 29 3 1 0
-1 22 3 1 0
-0 38 2 1 1
-0 13 2 1 1
-0 12 2 1 1
-0 42 3 1 0
-1 17 3 1 0
-0 21 3 1 1
-0 34 1 1 1
-0 1 3 1 0
-0 14 2 1 0
-0 16 2 1 0
-0 9 3 1 0
-0 53 3 1 0
-0 27 3 1 0
-0 15 3 1 0
-0 9 3 1 0
-0 4 2 1 1
-0 10 3 1 1
-0 31 3 1 0
-0 85 3 1 1
-0 24 2 1 0
-end data.
-
-logistic regression
- disease WITH age sciostat sector savings
- /categorical = sciostat sector
- /print = ci(95).
-])
-
-AT_CHECK([pspp -O format=csv ci.sps], [0], [dnl
-Table: Dependent Variable Encoding
-Original Value,Internal Value
-.000,.000
-1.000,1.000
-
-Table: Case Processing Summary
-Unweighted Cases,N,Percent
-Included in Analysis,196,100.0%
-Missing Cases,0,.0%
-Total,196,100.0%
-
-note: Estimation terminated at iteration number 4 because parameter estimates changed by less than 0.001
-
-Table: Model Summary
-Step,-2 Log likelihood,Cox & Snell R Square,Nagelkerke R Square
-1,211.195,.120,.172
-
-Table: Categorical Variables' Codings
-,,Frequency,Parameter coding,
-,,,(1),(2)
-sciostat,1.000,77,1,0
-,2.000,49,0,1
-,3.000,70,0,0
-sector,1.000,117,1,
-,2.000,79,0,
-
-Table: Classification Table
-,Observed,,Predicted,,
-,,,disease,,Percentage Correct
-,,,.000,1.000,
-Step 1,disease,.000,131,8,94.2%
-,,1.000,41,16,28.1%
-,Overall Percentage,,,,75.0%
-
-Table: Variables in the Equation
-,,B,S.E.,Wald,df,Sig.,Exp(B),95% CI for Exp(B),
-,,,,,,,,Lower,Upper
-Step 1,age,.027,.009,8.647,1,.003,1.027,1.009,1.045
-,savings,.061,.386,.025,1,.874,1.063,.499,2.264
-,sciostat,,,.440,2,.803,,,
-,sciostat(1),-.278,.434,.409,1,.522,.757,.323,1.775
-,sciostat(2),-.219,.459,.227,1,.634,.803,.327,1.976
-,sector,,,11.974,1,.001,,,
-,sector(1),-1.235,.357,11.974,1,.001,.291,.145,.586
-,Constant,-.814,.452,3.246,1,.072,.443,,
-])
-
-AT_CLEANUP
-
-AT_SETUP([LOGISTIC REGRESSION syntax errors])
-AT_DATA([logistic.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-LOGISTIC REGRESSION **.
-LOGISTIC REGRESSION x **.
-LOGISTIC REGRESSION x WITH **.
-LOGISTIC REGRESSION x WITH y/MISSING=**.
-LOGISTIC REGRESSION x WITH y/CATEGORICAL=**.
-LOGISTIC REGRESSION x WITH y/PRINT=CI **.
-LOGISTIC REGRESSION x WITH y/PRINT=CI(**).
-LOGISTIC REGRESSION x WITH y/PRINT=CI(123 **).
-LOGISTIC REGRESSION x WITH y/PRINT=**.
-LOGISTIC REGRESSION x WITH y/CRITERIA=BCON **.
-LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(**).
-LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(123 **).
-LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE **.
-LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(**).
-LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(123 **).
-LOGISTIC REGRESSION x WITH y/CRITERIA=LCON **.
-LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(**).
-LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(123 **).
-LOGISTIC REGRESSION x WITH y/CRITERIA=EPS **.
-LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(**).
-LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(123 **).
-LOGISTIC REGRESSION x WITH y/CRITERIA=CUT **.
-LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(**).
-LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(0.5 **).
-LOGISTIC REGRESSION x WITH y/CRITERIA=**.
-])
-AT_CHECK([pspp -O format=csv logistic.sps], [1], [dnl
-"logistic.sps:2.21-2.22: error: LOGISTIC REGRESSION: Syntax error expecting variable name.
- 2 | LOGISTIC REGRESSION **.
- | ^~"
-
-"logistic.sps:3.23-3.24: error: LOGISTIC REGRESSION: Syntax error expecting `WITH'.
- 3 | LOGISTIC REGRESSION x **.
- | ^~"
-
-"logistic.sps:4.28-4.29: error: LOGISTIC REGRESSION: Syntax error expecting variable name.
- 4 | LOGISTIC REGRESSION x WITH **.
- | ^~"
-
-"logistic.sps:5.38-5.39: error: LOGISTIC REGRESSION: Syntax error expecting INCLUDE or EXCLUDE.
- 5 | LOGISTIC REGRESSION x WITH y/MISSING=**.
- | ^~"
-
-"logistic.sps:6.42-6.43: error: LOGISTIC REGRESSION: Syntax error expecting variable name.
- 6 | LOGISTIC REGRESSION x WITH y/CATEGORICAL=**.
- | ^~"
-
-"logistic.sps:7.39-7.40: error: LOGISTIC REGRESSION: Syntax error expecting `('.
- 7 | LOGISTIC REGRESSION x WITH y/PRINT=CI **.
- | ^~"
-
-"logistic.sps:8.39-8.40: error: LOGISTIC REGRESSION: Syntax error expecting number.
- 8 | LOGISTIC REGRESSION x WITH y/PRINT=CI(**).
- | ^~"
-
-"logistic.sps:9.43-9.44: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
- 9 | LOGISTIC REGRESSION x WITH y/PRINT=CI(123 **).
- | ^~"
-
-"logistic.sps:10.36-10.37: error: LOGISTIC REGRESSION: Syntax error expecting DEFAULT, SUMMARY, CI, or ALL.
- 10 | LOGISTIC REGRESSION x WITH y/PRINT=**.
- | ^~"
-
-"logistic.sps:11.44-11.45: error: LOGISTIC REGRESSION: Syntax error expecting `('.
- 11 | LOGISTIC REGRESSION x WITH y/CRITERIA=BCON **.
- | ^~"
-
-"logistic.sps:12.44-12.45: error: LOGISTIC REGRESSION: Syntax error expecting number.
- 12 | LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(**).
- | ^~"
-
-"logistic.sps:13.48-13.49: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
- 13 | LOGISTIC REGRESSION x WITH y/CRITERIA=BCON(123 **).
- | ^~"
-
-"logistic.sps:14.47-14.48: error: LOGISTIC REGRESSION: Syntax error expecting `('.
- 14 | LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE **.
- | ^~"
-
-"logistic.sps:15.47-15.48: error: LOGISTIC REGRESSION: Syntax error expecting non-negative integer for ITERATE.
- 15 | LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(**).
- | ^~"
-
-"logistic.sps:16.51-16.52: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
- 16 | LOGISTIC REGRESSION x WITH y/CRITERIA=ITERATE(123 **).
- | ^~"
-
-"logistic.sps:17.44-17.45: error: LOGISTIC REGRESSION: Syntax error expecting `('.
- 17 | LOGISTIC REGRESSION x WITH y/CRITERIA=LCON **.
- | ^~"
-
-"logistic.sps:18.44-18.45: error: LOGISTIC REGRESSION: Syntax error expecting number.
- 18 | LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(**).
- | ^~"
-
-"logistic.sps:19.48-19.49: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
- 19 | LOGISTIC REGRESSION x WITH y/CRITERIA=LCON(123 **).
- | ^~"
-
-"logistic.sps:20.43-20.44: error: LOGISTIC REGRESSION: Syntax error expecting `('.
- 20 | LOGISTIC REGRESSION x WITH y/CRITERIA=EPS **.
- | ^~"
-
-"logistic.sps:21.43-21.44: error: LOGISTIC REGRESSION: Syntax error expecting number.
- 21 | LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(**).
- | ^~"
-
-"logistic.sps:22.47-22.48: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
- 22 | LOGISTIC REGRESSION x WITH y/CRITERIA=EPS(123 **).
- | ^~"
-
-"logistic.sps:23.43-23.44: error: LOGISTIC REGRESSION: Syntax error expecting `('.
- 23 | LOGISTIC REGRESSION x WITH y/CRITERIA=CUT **.
- | ^~"
-
-"logistic.sps:24.43-24.44: error: LOGISTIC REGRESSION: Syntax error expecting number between 0 and 1 for CUT.
- 24 | LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(**).
- | ^~"
-
-"logistic.sps:25.47-25.48: error: LOGISTIC REGRESSION: Syntax error expecting `)'.
- 25 | LOGISTIC REGRESSION x WITH y/CRITERIA=CUT(0.5 **).
- | ^~"
-
-"logistic.sps:26.39-26.40: error: LOGISTIC REGRESSION: Syntax error expecting BCON, ITERATE, LCON, EPS, or CUT.
- 26 | LOGISTIC REGRESSION x WITH y/CRITERIA=**.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-AT_BANNER([MATRIX])
-
-AT_SETUP([MATRIX - empty matrices])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE a={}.
-PRINT a.
-COMPUTE b={a; 1; 2; 3}.
-PRINT b.
-COMPUTE c={a, 1, 2, 3}.
-PRINT c.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-a
-
-b
- 1
- 2
- 3
-
-c
- 1 2 3
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - submatrices as rvalues - all columns or all rows])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(1, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1}, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2}, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2, 3}, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1; 3; 2}, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 3, 3}, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(1:2, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(1:3, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}({}, :).
-
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1}).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2}).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2, 3}).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1; 3; 2}).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 3, 3}).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:2).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:3).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {}).
-
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, :).
-
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(0, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 0).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(4, :).
-PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 4).
-
-PRINT {}(:,{}).
-PRINT {}({},:).
-PRINT {}({},{}).
-
-PRINT {1, 2, 3, 4}({1, 2; 3, 4}, :).
-PRINT {1, 2, 3, 4}(:, {1, 2; 3, 4}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(1, :)
- 1 2 3
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}({1}, :)
- 1 2 3
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2}, :)
- 1 2 3
- 4 5 6
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 2, 3}, :)
- 1 2 3
- 4 5 6
- 7 8 9
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}({1; 3; 2}, :)
- 1 2 3
- 7 8 9
- 4 5 6
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}({1, 3, 3}, :)
- 1 2 3
- 7 8 9
- 7 8 9
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(1:2, :)
- 1 2 3
- 4 5 6
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(1:3, :)
- 1 2 3
- 4 5 6
- 7 8 9
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}({}, :)
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1)
- 1
- 4
- 7
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1})
- 1
- 4
- 7
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2})
- 1 2
- 4 5
- 7 8
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 2, 3})
- 1 2 3
- 4 5 6
- 7 8 9
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1; 3; 2})
- 1 3 2
- 4 6 5
- 7 9 8
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {1, 3, 3})
- 1 3 3
- 4 6 6
- 7 9 9
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:2)
- 1 2
- 4 5
- 7 8
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 1:3)
- 1 2 3
- 4 5 6
- 7 8 9
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, {})
-
-
-
-{1, 2, 3; 4, 5, 6; 7, 8, 9}(:, :)
- 1 2 3
- 4 5 6
- 7 8 9
-
-matrix.sps:24.35: error: MATRIX: 0 is not a valid row index for a 3×3 matrix.
- 24 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(0, :).
- | ^
-
-matrix.sps:25.38: error: MATRIX: 0 is not a valid column index for a 3×3
-matrix.
- 25 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 0).
- | ^
-
-matrix.sps:26.35: error: MATRIX: 4 is not a valid row index for a 3×3 matrix.
- 26 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(4, :).
- | ^
-
-matrix.sps:27.38: error: MATRIX: 4 is not a valid column index for a 3×3
-matrix.
- 27 | PRINT {1, 2, 3; 4, 5, 6; 7, 8, 9}(:, 4).
- | ^
-
-{}(:,{})
-
-{}({},:)
-
-{}({},{})
-
-matrix.sps:33.20-33.31: error: MATRIX: Matrix row index must be scalar or
-vector, not a 2×2 matrix.
- 33 | PRINT {1, 2, 3, 4}({1, 2; 3, 4}, :).
- | ^~~~~~~~~~~~
-
-matrix.sps:34.23-34.34: error: MATRIX: Matrix column index must be scalar or
-vector, not a 2×2 matrix.
- 34 | PRINT {1, 2, 3, 4}(:, {1, 2; 3, 4}).
- | ^~~~~~~~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - COMPUTE submatrices as lvalues])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE y={1, 2, 3; 4, 5, 6; 7, 8, 9}.
-
-COMPUTE x1=y.
-COMPUTE x1(1, :) = {11, 12, 13}.
-PRINT x1.
-
-COMPUTE x2=y.
-COMPUTE x2(2, :) = {14, 15, 16}.
-PRINT x2.
-
-COMPUTE x3=y.
-COMPUTE x3(3, :) = {17, 18, 19}.
-PRINT x3.
-
-COMPUTE x4=y.
-COMPUTE x4(:, 1) = {11; 14; 17}.
-PRINT x4.
-
-COMPUTE x5=y.
-COMPUTE x5(:, 2) = {12; 15; 18}.
-PRINT x5.
-
-COMPUTE x6=y.
-COMPUTE x6(:, 3) = {13; 16; 19}.
-PRINT x6.
-
-COMPUTE x7=y.
-COMPUTE x7(1, 1) = 11.
-PRINT x7.
-
-COMPUTE x8=y.
-COMPUTE x8(1:2, 2:3) = {12, 13; 15, 16}.
-PRINT x8.
-
-COMPUTE x9=y.
-COMPUTE x9({3, 1}, {2; 3}) = {18, 19; 12, 13}.
-PRINT x9.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-x1
- 11 12 13
- 4 5 6
- 7 8 9
-
-x2
- 1 2 3
- 14 15 16
- 7 8 9
-
-x3
- 1 2 3
- 4 5 6
- 17 18 19
-
-x4
- 11 2 3
- 14 5 6
- 17 8 9
-
-x5
- 1 12 3
- 4 15 6
- 7 18 9
-
-x6
- 1 2 13
- 4 5 16
- 7 8 19
-
-x7
- 11 2 3
- 4 5 6
- 7 8 9
-
-x8
- 1 12 13
- 4 15 16
- 7 8 9
-
-x9
- 1 12 13
- 4 5 6
- 7 18 19
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - COMPUTE submatrices as lvalues - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE x={1, 2, 3; 4, 5, 6; 7, 8, 9}.
-COMPUTE x(1, :) = {}.
-COMPUTE x(1, :) = 15.
-COMPUTE x(1, :) = {11, 12}.
-COMPUTE x(1, :) = {11, 12, 13, 14}.
-COMPUTE x(:, 1) = {}.
-COMPUTE x(:, 1) = 15.
-COMPUTE x(:, 1) = {11, 12}.
-COMPUTE x(:, 1) = {11, 12, 13, 14}.
-COMPUTE x(:) = 1.
-COMPUTE x(0, 1) = 1.
-COMPUTE x(1, 0) = 1.
-COMPUTE x({1, 0, 2}, 1) = {1; 2; 3}.
-COMPUTE x(4, 3) = 1.
-COMPUTE x(3, 4) = 1.
-COMPUTE x({1, 2; 3, 4}, 5) = 1.
-COMPUTE x(3, {1, 2; 3, 4}) = 1.
-PRINT x.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:3.9-3.15: error: MATRIX: Numbers of indexes for assigning to x
-differ from the size of the source matrix.
- 3 | COMPUTE x(1, :) = {}.
- | ^~~~~~~
-
-matrix.sps:3.11: note: MATRIX: There is 1 row index.
- 3 | COMPUTE x(1, :) = {}.
- | ^
-
-matrix.sps:3.14: note: MATRIX: Destination matrix x has 3 columns.
- 3 | COMPUTE x(1, :) = {}.
- | ^
-
-matrix.sps:3.19-3.20: note: MATRIX: The source matrix is 0×0.
- 3 | COMPUTE x(1, :) = {}.
- | ^~
-
-matrix.sps:4.9-4.15: error: MATRIX: Number of column indexes for assigning to x
-differs from number of columns in source matrix.
- 4 | COMPUTE x(1, :) = 15.
- | ^~~~~~~
-
-matrix.sps:4.14: note: MATRIX: Destination matrix x has 3 columns.
- 4 | COMPUTE x(1, :) = 15.
- | ^
-
-matrix.sps:4.19-4.20: note: MATRIX: The source matrix is 1×1.
- 4 | COMPUTE x(1, :) = 15.
- | ^~
-
-matrix.sps:5.9-5.15: error: MATRIX: Number of column indexes for assigning to x
-differs from number of columns in source matrix.
- 5 | COMPUTE x(1, :) = {11, 12}.
- | ^~~~~~~
-
-matrix.sps:5.14: note: MATRIX: Destination matrix x has 3 columns.
- 5 | COMPUTE x(1, :) = {11, 12}.
- | ^
-
-matrix.sps:5.19-5.26: note: MATRIX: The source matrix is 1×2.
- 5 | COMPUTE x(1, :) = {11, 12}.
- | ^~~~~~~~
-
-matrix.sps:6.9-6.15: error: MATRIX: Number of column indexes for assigning to x
-differs from number of columns in source matrix.
- 6 | COMPUTE x(1, :) = {11, 12, 13, 14}.
- | ^~~~~~~
-
-matrix.sps:6.14: note: MATRIX: Destination matrix x has 3 columns.
- 6 | COMPUTE x(1, :) = {11, 12, 13, 14}.
- | ^
-
-matrix.sps:6.19-6.34: note: MATRIX: The source matrix is 1×4.
- 6 | COMPUTE x(1, :) = {11, 12, 13, 14}.
- | ^~~~~~~~~~~~~~~~
-
-matrix.sps:7.9-7.15: error: MATRIX: Numbers of indexes for assigning to x
-differ from the size of the source matrix.
- 7 | COMPUTE x(:, 1) = {}.
- | ^~~~~~~
-
-matrix.sps:7.11: note: MATRIX: Destination matrix x has 3 rows.
- 7 | COMPUTE x(:, 1) = {}.
- | ^
-
-matrix.sps:7.14: note: MATRIX: There is 1 column index.
- 7 | COMPUTE x(:, 1) = {}.
- | ^
-
-matrix.sps:7.19-7.20: note: MATRIX: The source matrix is 0×0.
- 7 | COMPUTE x(:, 1) = {}.
- | ^~
-
-matrix.sps:8.9-8.15: error: MATRIX: Number of row indexes for assigning to x
-differs from number of rows in source matrix.
- 8 | COMPUTE x(:, 1) = 15.
- | ^~~~~~~
-
-matrix.sps:8.11: note: MATRIX: Destination matrix x has 3 rows.
- 8 | COMPUTE x(:, 1) = 15.
- | ^
-
-matrix.sps:8.19-8.20: note: MATRIX: The source matrix is 1×1.
- 8 | COMPUTE x(:, 1) = 15.
- | ^~
-
-matrix.sps:9.9-9.15: error: MATRIX: Numbers of indexes for assigning to x
-differ from the size of the source matrix.
- 9 | COMPUTE x(:, 1) = {11, 12}.
- | ^~~~~~~
-
-matrix.sps:9.11: note: MATRIX: Destination matrix x has 3 rows.
- 9 | COMPUTE x(:, 1) = {11, 12}.
- | ^
-
-matrix.sps:9.14: note: MATRIX: There is 1 column index.
- 9 | COMPUTE x(:, 1) = {11, 12}.
- | ^
-
-matrix.sps:9.19-9.26: note: MATRIX: The source matrix is 1×2.
- 9 | COMPUTE x(:, 1) = {11, 12}.
- | ^~~~~~~~
-
-matrix.sps:10.9-10.15: error: MATRIX: Numbers of indexes for assigning to x
-differ from the size of the source matrix.
- 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
- | ^~~~~~~
-
-matrix.sps:10.11: note: MATRIX: Destination matrix x has 3 rows.
- 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
- | ^
-
-matrix.sps:10.14: note: MATRIX: There is 1 column index.
- 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
- | ^
-
-matrix.sps:10.19-10.34: note: MATRIX: The source matrix is 1×4.
- 10 | COMPUTE x(:, 1) = {11, 12, 13, 14}.
- | ^~~~~~~~~~~~~~~~
-
-matrix.sps:11.9-11.12: error: MATRIX: Can't use vector indexing on 3×3 matrix
-x.
- 11 | COMPUTE x(:) = 1.
- | ^~~~
-
-matrix.sps:12.11: error: MATRIX: 0 is not a valid row index for a 3×3 matrix.
- 12 | COMPUTE x(0, 1) = 1.
- | ^
-
-matrix.sps:13.14: error: MATRIX: 0 is not a valid column index for a 3×3
-matrix.
- 13 | COMPUTE x(1, 0) = 1.
- | ^
-
-matrix.sps:14.11-14.19: error: MATRIX: 0 is not a valid row index for a 3×3
-matrix.
- 14 | COMPUTE x({1, 0, 2}, 1) = {1; 2; 3}.
- | ^~~~~~~~~
-
-matrix.sps:15.11: error: MATRIX: 4 is not a valid row index for a 3×3 matrix.
- 15 | COMPUTE x(4, 3) = 1.
- | ^
-
-matrix.sps:16.14: error: MATRIX: 4 is not a valid column index for a 3×3
-matrix.
- 16 | COMPUTE x(3, 4) = 1.
- | ^
-
-matrix.sps:17.11-17.22: error: MATRIX: Matrix row index must be scalar or
-vector, not a 2×2 matrix.
- 17 | COMPUTE x({1, 2; 3, 4}, 5) = 1.
- | ^~~~~~~~~~~~
-
-matrix.sps:18.14-18.25: error: MATRIX: Matrix column index must be scalar or
-vector, not a 2×2 matrix.
- 18 | COMPUTE x(3, {1, 2; 3, 4}) = 1.
- | ^~~~~~~~~~~~
-
-x
- 1 2 3
- 4 5 6
- 7 8 9
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - subvectors as rvalues])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT {10, 20, 30}({}).
-PRINT {10, 20, 30}(2).
-PRINT {10, 20, 30}({2}).
-PRINT {10, 20, 30}({1,3}).
-PRINT {10, 20, 30}({2,3}).
-PRINT {10, 20, 30}({1;3}).
-PRINT {10, 20, 30}({2;3}).
-PRINT {10, 20, 30}(2:3).
-PRINT {10, 20, 30}(:).
-
-PRINT {10; 20; 30}({}).
-PRINT {10; 20; 30}(2).
-PRINT {10; 20; 30}({2}).
-PRINT {10; 20; 30}({1,3}).
-PRINT {10; 20; 30}({2,3}).
-PRINT {10; 20; 30}({1;3}).
-PRINT {10; 20; 30}({2;3}).
-PRINT {10; 20; 30}(2:3).
-PRINT {10; 20; 30}(:).
-
-PRINT {}({}).
-
-PRINT {1, 2; 3, 4}(:).
-PRINT {1, 2, 3, 4}({1, 2; 3, 4}).
-PRINT {1, 2, 3, 4}(0).
-PRINT {1, 2, 3, 4}(5).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-{10, 20, 30}({})
-
-{10, 20, 30}(2)
- 20
-
-{10, 20, 30}({2})
- 20
-
-{10, 20, 30}({1,3})
- 10 30
-
-{10, 20, 30}({2,3})
- 20 30
-
-{10, 20, 30}({1;3})
- 10 30
-
-{10, 20, 30}({2;3})
- 20 30
-
-{10, 20, 30}(2:3)
- 20 30
-
-{10, 20, 30}(:)
- 10 20 30
-
-{10; 20; 30}({})
-
-{10; 20; 30}(2)
- 20
-
-{10; 20; 30}({2})
- 20
-
-{10; 20; 30}({1,3})
- 10
- 30
-
-{10; 20; 30}({2,3})
- 20
- 30
-
-{10; 20; 30}({1;3})
- 10
- 30
-
-{10; 20; 30}({2;3})
- 20
- 30
-
-{10; 20; 30}(2:3)
- 20
- 30
-
-{10; 20; 30}(:)
- 10
- 20
- 30
-
-{}({})
-
-matrix.sps:24.7-24.18: error: MATRIX: Vector index operator may not be applied
-to a 2×2 matrix.
- 24 | PRINT {1, 2; 3, 4}(:).
- | ^~~~~~~~~~~~
-
-matrix.sps:25.20-25.31: error: MATRIX: Vector index must be scalar or vector,
-not a 2×2 matrix.
- 25 | PRINT {1, 2, 3, 4}({1, 2; 3, 4}).
- | ^~~~~~~~~~~~
-
-matrix.sps:26.20: error: MATRIX: Index 0 is out of range for vector with 4
-elements.
- 26 | PRINT {1, 2, 3, 4}(0).
- | ^
-
-matrix.sps:27.20: error: MATRIX: Index 5 is out of range for vector with 4
-elements.
- 27 | PRINT {1, 2, 3, 4}(5).
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - COMPUTE subvectors as lvalues])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE r={1, 2, 3, 4, 5, 6, 7, 8, 9}.
-
-COMPUTE r1=r.
-COMPUTE r1(:) = {11, 12, 13, 14, 15, 16, 17, 18, 19}.
-PRINT r1.
-
-COMPUTE r2=r.
-COMPUTE r2(:) = {11; 12; 13; 14; 15; 16; 17; 18; 19}.
-PRINT r2.
-
-COMPUTE r3=r.
-COMPUTE r3(1) = 11.
-PRINT r3.
-
-COMPUTE r4=r.
-COMPUTE r4(1:2) = {11:12}.
-PRINT r4.
-
-COMPUTE r5=r.
-COMPUTE r5({8;9}) = {18:19}.
-PRINT r5.
-
-COMPUTE c={1, 2, 3, 4, 5, 6, 7, 8, 9}.
-
-COMPUTE c1=c.
-COMPUTE c1(:) = {11, 12, 13, 14, 15, 16, 17, 18, 19}.
-PRINT c1.
-
-COMPUTE c2=c.
-COMPUTE c2(:) = {11; 12; 13; 14; 15; 16; 17; 18; 19}.
-PRINT c2.
-
-COMPUTE c3=c.
-COMPUTE c3(1) = 11.
-PRINT c3.
-
-COMPUTE c4=c.
-COMPUTE c4(1:2) = {11:12}.
-PRINT c4.
-
-COMPUTE c5=c.
-COMPUTE c5(8:9) = {18:19}.
-PRINT c5.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-r1
- 11 12 13 14 15 16 17 18 19
-
-r2
- 11 12 13 14 15 16 17 18 19
-
-r3
- 11 2 3 4 5 6 7 8 9
-
-r4
- 11 12 3 4 5 6 7 8 9
-
-r5
- 1 2 3 4 5 6 7 18 19
-
-c1
- 11 12 13 14 15 16 17 18 19
-
-c2
- 11 12 13 14 15 16 17 18 19
-
-c3
- 11 2 3 4 5 6 7 8 9
-
-c4
- 11 12 3 4 5 6 7 8 9
-
-c5
- 1 2 3 4 5 6 7 18 19
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - COMPUTE subvectors as lvalues - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE r={1, 2, 3, 4, 5, 6, 7, 8, 9}.
-COMPUTE r(1:3) = {1, 2; 3, 4}.
-COMPUTE r(1:3) = {}.
-COMPUTE r(1:3) = {1}.
-COMPUTE r(1:3) = {1, 2}.
-COMPUTE r(1:3) = {1, 2, 3, 4}.
-COMPUTE r(1:3) = {}.
-COMPUTE r(1:3) = {1}.
-COMPUTE r(1:3) = {1; 2}.
-COMPUTE r(1:3) = {1; 2; 3; 4}.
-COMPUTE r(:) = {1; 2; 3; 4}.
-COMPUTE r(0) = 5.
-COMPUTE r(10) = 5.
-COMPUTE r({1, 2; 3, 4}) = 1.
-
-COMPUTE c={1, 2, 3, 4, 5, 6, 7, 8, 9}.
-COMPUTE c(1:3) = {1, 2; 3, 4}.
-COMPUTE c(1:3) = {}.
-COMPUTE c(1:3) = {1}.
-COMPUTE c(1:3) = {1, 2}.
-COMPUTE c(1:3) = {1, 2, 3, 4}.
-COMPUTE c(1:3) = {}.
-COMPUTE c(1:3) = {1}.
-COMPUTE c(1:3) = {1; 2}.
-COMPUTE c(1:3) = {1; 2; 3; 4}.
-COMPUTE c(:) = {1; 2; 3; 4}.
-COMPUTE c(0) = 5.
-COMPUTE c(10) = 5.
-COMPUTE c({1, 2; 3, 4}) = 1.
-
-COMPUTE m = {1, 2; 3, 4}.
-COMPUTE m(5) = 1.
-COMPUTE m(:) = 1.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:3.9-3.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 3 | COMPUTE r(1:3) = {1, 2; 3, 4}.
- | ^~~~~~
-
-matrix.sps:3.18-3.29: error: MATRIX: The source is an 2×2 matrix.
- 3 | COMPUTE r(1:3) = {1, 2; 3, 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:4.9-4.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 4 | COMPUTE r(1:3) = {}.
- | ^~~~~~
-
-matrix.sps:4.18-4.19: error: MATRIX: The source vector has 0 elements.
- 4 | COMPUTE r(1:3) = {}.
- | ^~
-
-matrix.sps:5.9-5.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 5 | COMPUTE r(1:3) = {1}.
- | ^~~~~~
-
-matrix.sps:5.19: error: MATRIX: The source vector has 1 element.
- 5 | COMPUTE r(1:3) = {1}.
- | ^
-
-matrix.sps:6.9-6.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 6 | COMPUTE r(1:3) = {1, 2}.
- | ^~~~~~
-
-matrix.sps:6.18-6.23: error: MATRIX: The source vector has 2 elements.
- 6 | COMPUTE r(1:3) = {1, 2}.
- | ^~~~~~
-
-matrix.sps:7.9-7.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 7 | COMPUTE r(1:3) = {1, 2, 3, 4}.
- | ^~~~~~
-
-matrix.sps:7.18-7.29: error: MATRIX: The source vector has 4 elements.
- 7 | COMPUTE r(1:3) = {1, 2, 3, 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:8.9-8.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 8 | COMPUTE r(1:3) = {}.
- | ^~~~~~
-
-matrix.sps:8.18-8.19: error: MATRIX: The source vector has 0 elements.
- 8 | COMPUTE r(1:3) = {}.
- | ^~
-
-matrix.sps:9.9-9.14: error: MATRIX: Only an 3-element vector may be assigned to
-this 3-element subvector of r.
- 9 | COMPUTE r(1:3) = {1}.
- | ^~~~~~
-
-matrix.sps:9.19: error: MATRIX: The source vector has 1 element.
- 9 | COMPUTE r(1:3) = {1}.
- | ^
-
-matrix.sps:10.9-10.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of r.
- 10 | COMPUTE r(1:3) = {1; 2}.
- | ^~~~~~
-
-matrix.sps:10.18-10.23: error: MATRIX: The source vector has 2 elements.
- 10 | COMPUTE r(1:3) = {1; 2}.
- | ^~~~~~
-
-matrix.sps:11.9-11.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of r.
- 11 | COMPUTE r(1:3) = {1; 2; 3; 4}.
- | ^~~~~~
-
-matrix.sps:11.18-11.29: error: MATRIX: The source vector has 4 elements.
- 11 | COMPUTE r(1:3) = {1; 2; 3; 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:12.9-12.12: error: MATRIX: Only an 9-element vector may be assigned
-to this 9-element subvector of r.
- 12 | COMPUTE r(:) = {1; 2; 3; 4}.
- | ^~~~
-
-matrix.sps:12.16-12.27: error: MATRIX: The source vector has 4 elements.
- 12 | COMPUTE r(:) = {1; 2; 3; 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:13.11: error: MATRIX: Index 0 is out of range for vector with 9
-elements.
- 13 | COMPUTE r(0) = 5.
- | ^
-
-matrix.sps:14.11-14.12: error: MATRIX: Index 10 is out of range for vector with
-9 elements.
- 14 | COMPUTE r(10) = 5.
- | ^~
-
-matrix.sps:15.11-15.22: error: MATRIX: Vector index must be scalar or vector,
-not a 2×2 matrix.
- 15 | COMPUTE r({1, 2; 3, 4}) = 1.
- | ^~~~~~~~~~~~
-
-matrix.sps:18.9-18.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 18 | COMPUTE c(1:3) = {1, 2; 3, 4}.
- | ^~~~~~
-
-matrix.sps:18.18-18.29: error: MATRIX: The source is an 2×2 matrix.
- 18 | COMPUTE c(1:3) = {1, 2; 3, 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:19.9-19.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 19 | COMPUTE c(1:3) = {}.
- | ^~~~~~
-
-matrix.sps:19.18-19.19: error: MATRIX: The source vector has 0 elements.
- 19 | COMPUTE c(1:3) = {}.
- | ^~
-
-matrix.sps:20.9-20.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 20 | COMPUTE c(1:3) = {1}.
- | ^~~~~~
-
-matrix.sps:20.19: error: MATRIX: The source vector has 1 element.
- 20 | COMPUTE c(1:3) = {1}.
- | ^
-
-matrix.sps:21.9-21.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 21 | COMPUTE c(1:3) = {1, 2}.
- | ^~~~~~
-
-matrix.sps:21.18-21.23: error: MATRIX: The source vector has 2 elements.
- 21 | COMPUTE c(1:3) = {1, 2}.
- | ^~~~~~
-
-matrix.sps:22.9-22.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 22 | COMPUTE c(1:3) = {1, 2, 3, 4}.
- | ^~~~~~
-
-matrix.sps:22.18-22.29: error: MATRIX: The source vector has 4 elements.
- 22 | COMPUTE c(1:3) = {1, 2, 3, 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:23.9-23.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 23 | COMPUTE c(1:3) = {}.
- | ^~~~~~
-
-matrix.sps:23.18-23.19: error: MATRIX: The source vector has 0 elements.
- 23 | COMPUTE c(1:3) = {}.
- | ^~
-
-matrix.sps:24.9-24.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 24 | COMPUTE c(1:3) = {1}.
- | ^~~~~~
-
-matrix.sps:24.19: error: MATRIX: The source vector has 1 element.
- 24 | COMPUTE c(1:3) = {1}.
- | ^
-
-matrix.sps:25.9-25.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 25 | COMPUTE c(1:3) = {1; 2}.
- | ^~~~~~
-
-matrix.sps:25.18-25.23: error: MATRIX: The source vector has 2 elements.
- 25 | COMPUTE c(1:3) = {1; 2}.
- | ^~~~~~
-
-matrix.sps:26.9-26.14: error: MATRIX: Only an 3-element vector may be assigned
-to this 3-element subvector of c.
- 26 | COMPUTE c(1:3) = {1; 2; 3; 4}.
- | ^~~~~~
-
-matrix.sps:26.18-26.29: error: MATRIX: The source vector has 4 elements.
- 26 | COMPUTE c(1:3) = {1; 2; 3; 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:27.9-27.12: error: MATRIX: Only an 9-element vector may be assigned
-to this 9-element subvector of c.
- 27 | COMPUTE c(:) = {1; 2; 3; 4}.
- | ^~~~
-
-matrix.sps:27.16-27.27: error: MATRIX: The source vector has 4 elements.
- 27 | COMPUTE c(:) = {1; 2; 3; 4}.
- | ^~~~~~~~~~~~
-
-matrix.sps:28.11: error: MATRIX: Index 0 is out of range for vector with 9
-elements.
- 28 | COMPUTE c(0) = 5.
- | ^
-
-matrix.sps:29.11-29.12: error: MATRIX: Index 10 is out of range for vector with
-9 elements.
- 29 | COMPUTE c(10) = 5.
- | ^~
-
-matrix.sps:30.11-30.22: error: MATRIX: Vector index must be scalar or vector,
-not a 2×2 matrix.
- 30 | COMPUTE c({1, 2; 3, 4}) = 1.
- | ^~~~~~~~~~~~
-
-matrix.sps:33.9-33.12: error: MATRIX: Can't use vector indexing on 2×2 matrix
-m.
- 33 | COMPUTE m(5) = 1.
- | ^~~~
-
-matrix.sps:34.9-34.12: error: MATRIX: Can't use vector indexing on 2×2 matrix
-m.
- 34 | COMPUTE m(:) = 1.
- | ^~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - COMPUTE - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE x.
-COMPUTE x=.
-COMPUTE x(5)=1.
-COMPUTE y(5)=1.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.10: error: COMPUTE: Syntax error expecting `='.
- 2 | COMPUTE x.
- | ^
-
-matrix.sps:3.11: error: COMPUTE: Syntax error expecting matrix expression.
- 3 | COMPUTE x=.
- | ^
-
-matrix.sps:4.9: error: MATRIX: Undefined variable x.
- 4 | COMPUTE x(5)=1.
- | ^
-
-matrix.sps:5.9: error: COMPUTE: Undefined variable y.
- 5 | COMPUTE y(5)=1.
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - elementwise arithmetic operators])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT (-(5)).
-PRINT (-{1,2;3,4}).
-
-PRINT ({1,2;3,4} + {5,6;7,8}).
-PRINT ({1,2;3,4} + 5).
-PRINT (5 + {5,6;7,8}).
-PRINT ({1,2;3,4} + {5,6}).
-
-PRINT ({1,2;3,4} - {5,6;7,8}).
-PRINT ({1,2;3,4} - 5).
-PRINT (5 - {5,6;7,8}).
-PRINT ({1,2;3,4} - {5,6}).
-
-PRINT ({1,2;3,4} * 5).
-PRINT (5 * {5,6;7,8}).
-
-PRINT ({2,4;6,8} / 2).
-PRINT (12 / {1,2;3,4}).
-PRINT ({2,8;18,32} / {1,2;3,4}).
-
-PRINT ({1,2;3,4} &* {5,6;7,8}).
-PRINT ({1,2;3,4} &* 5).
-PRINT (5 &* {5,6;7,8}).
-PRINT ({1,2;3,4} &* {5,6}).
-
-PRINT ({2,4;6,8} &/ 2).
-PRINT (12 &/ {1,2;3,4}).
-PRINT ({2,8;18,32} &/ {1,2;3,4}).
-
-PRINT ({1,2;3,4} &** 2).
-PRINT (2 &** {1,2;3,4}).
-PRINT ({1,2;3,4} &** {2,3;4,5}).
-PRINT ({1,2;3,4} &** {5,6}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-(-(5))
- -5
-
-(-{1,2;3,4})
- -1 -2
- -3 -4
-
-({1,2;3,4} + {5,6;7,8})
- 6 8
- 10 12
-
-({1,2;3,4} + 5)
- 6 7
- 8 9
-
-(5 + {5,6;7,8})
- 10 11
- 12 13
-
-matrix.sps:8.8-8.24: error: MATRIX: The operands of + must have the same
-dimensions or one must be a scalar.
- 8 | PRINT ({1,2;3,4} + {5,6}).
- | ^~~~~~~~~~~~~~~~~
-
-matrix.sps:8.8-8.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
- 8 | PRINT ({1,2;3,4} + {5,6}).
- | ^~~~~~~~~
-
-matrix.sps:8.20-8.24: note: MATRIX: The right-hand operand is a 1×2 matrix.
- 8 | PRINT ({1,2;3,4} + {5,6}).
- | ^~~~~
-
-({1,2;3,4} - {5,6;7,8})
- -4 -4
- -4 -4
-
-({1,2;3,4} - 5)
- -4 -3
- -2 -1
-
-(5 - {5,6;7,8})
- 0 -1
- -2 -3
-
-matrix.sps:13.8-13.24: error: MATRIX: The operands of - must have the same
-dimensions or one must be a scalar.
- 13 | PRINT ({1,2;3,4} - {5,6}).
- | ^~~~~~~~~~~~~~~~~
-
-matrix.sps:13.8-13.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
- 13 | PRINT ({1,2;3,4} - {5,6}).
- | ^~~~~~~~~
-
-matrix.sps:13.20-13.24: note: MATRIX: The right-hand operand is a 1×2 matrix.
- 13 | PRINT ({1,2;3,4} - {5,6}).
- | ^~~~~
-
-({1,2;3,4} * 5)
- 5 10
- 15 20
-
-(5 * {5,6;7,8})
- 25 30
- 35 40
-
-({2,4;6,8} / 2)
- 1 2
- 3 4
-
-(12 / {1,2;3,4})
- 12 6
- 4 3
-
-({2,8;18,32} / {1,2;3,4})
- 2 4
- 6 8
-
-({1,2;3,4} &* {5,6;7,8})
- 5 12
- 21 32
-
-({1,2;3,4} &* 5)
- 5 10
- 15 20
-
-(5 &* {5,6;7,8})
- 25 30
- 35 40
-
-matrix.sps:25.8-25.25: error: MATRIX: The operands of &* must have the same
-dimensions or one must be a scalar.
- 25 | PRINT ({1,2;3,4} &* {5,6}).
- | ^~~~~~~~~~~~~~~~~~
-
-matrix.sps:25.8-25.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
- 25 | PRINT ({1,2;3,4} &* {5,6}).
- | ^~~~~~~~~
-
-matrix.sps:25.21-25.25: note: MATRIX: The right-hand operand is a 1×2 matrix.
- 25 | PRINT ({1,2;3,4} &* {5,6}).
- | ^~~~~
-
-({2,4;6,8} &/ 2)
- 1 2
- 3 4
-
-(12 &/ {1,2;3,4})
- 12 6
- 4 3
-
-({2,8;18,32} &/ {1,2;3,4})
- 2 4
- 6 8
-
-({1,2;3,4} &** 2)
- 1 4
- 9 16
-
-(2 &** {1,2;3,4})
- 2 4
- 8 16
-
-({1,2;3,4} &** {2,3;4,5})
- 1 8
- 81 1024
-
-matrix.sps:34.8-34.26: error: MATRIX: The operands of &** must have the same
-dimensions or one must be a scalar.
- 34 | PRINT ({1,2;3,4} &** {5,6}).
- | ^~~~~~~~~~~~~~~~~~~
-
-matrix.sps:34.8-34.16: note: MATRIX: The left-hand operand is a 2×2 matrix.
- 34 | PRINT ({1,2;3,4} &** {5,6}).
- | ^~~~~~~~~
-
-matrix.sps:34.22-34.26: note: MATRIX: The right-hand operand is a 1×2 matrix.
- 34 | PRINT ({1,2;3,4} &** {5,6}).
- | ^~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - relational operators])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT ({1, 1; 2, 2} > {1, 2; 1, 2}).
-PRINT ({1, 1; 2, 2} > 1).
-PRINT (2 > {1, 2; 1, 2}).
-PRINT ({1, 2} > {1; 2}).
-
-PRINT ({1, 1; 2, 2} < {1, 2; 1, 2}).
-PRINT ({1, 1; 2, 2} < 2).
-PRINT (1 < {1, 2; 1, 2}).
-PRINT ({1, 2} < {1; 2}).
-
-PRINT ({1, 1; 2, 2} <> {1, 2; 1, 2}).
-PRINT ({1, 1; 2, 2} <> 2).
-PRINT (1 <> {1, 2; 1, 2}).
-PRINT ({1, 2} <> {1; 2}).
-
-PRINT ({1, 1; 2, 2} >= {1, 2; 1, 2}).
-PRINT ({1, 1; 2, 2} >= 2).
-PRINT (1 >= {1, 2; 1, 2}).
-PRINT ({1, 2} >= {1; 2}).
-
-PRINT ({1, 1; 2, 2} <= {1, 2; 1, 2}).
-PRINT ({1, 1; 2, 2} <= 2).
-PRINT (1 <= {1, 2; 1, 2}).
-PRINT ({1, 2} <= {1; 2}).
-
-PRINT ({1, 1; 2, 2} = {1, 2; 1, 2}).
-PRINT ({1, 1; 2, 2} = 2).
-PRINT (1 = {1, 2; 1, 2}).
-PRINT ({1, 2} = {1; 2}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-({1, 1; 2, 2} > {1, 2; 1, 2})
- 0 0
- 1 0
-
-({1, 1; 2, 2} > 1)
- 0 0
- 1 1
-
-(2 > {1, 2; 1, 2})
- 1 0
- 1 0
-
-matrix.sps:5.8-5.22: error: MATRIX: The operands of > must have the same
-dimensions or one must be a scalar.
- 5 | PRINT ({1, 2} > {1; 2}).
- | ^~~~~~~~~~~~~~~
-
-matrix.sps:5.8-5.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 5 | PRINT ({1, 2} > {1; 2}).
- | ^~~~~~
-
-matrix.sps:5.17-5.22: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 5 | PRINT ({1, 2} > {1; 2}).
- | ^~~~~~
-
-({1, 1; 2, 2} < {1, 2; 1, 2})
- 0 1
- 0 0
-
-({1, 1; 2, 2} < 2)
- 1 1
- 0 0
-
-(1 < {1, 2; 1, 2})
- 0 1
- 0 1
-
-matrix.sps:10.8-10.22: error: MATRIX: The operands of < must have the same
-dimensions or one must be a scalar.
- 10 | PRINT ({1, 2} < {1; 2}).
- | ^~~~~~~~~~~~~~~
-
-matrix.sps:10.8-10.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 10 | PRINT ({1, 2} < {1; 2}).
- | ^~~~~~
-
-matrix.sps:10.17-10.22: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 10 | PRINT ({1, 2} < {1; 2}).
- | ^~~~~~
-
-({1, 1; 2, 2} <> {1, 2; 1, 2})
- 0 1
- 1 0
-
-({1, 1; 2, 2} <> 2)
- 1 1
- 0 0
-
-(1 <> {1, 2; 1, 2})
- 0 1
- 0 1
-
-matrix.sps:15.8-15.23: error: MATRIX: The operands of <> must have the same
-dimensions or one must be a scalar.
- 15 | PRINT ({1, 2} <> {1; 2}).
- | ^~~~~~~~~~~~~~~~
-
-matrix.sps:15.8-15.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 15 | PRINT ({1, 2} <> {1; 2}).
- | ^~~~~~
-
-matrix.sps:15.18-15.23: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 15 | PRINT ({1, 2} <> {1; 2}).
- | ^~~~~~
-
-({1, 1; 2, 2} >= {1, 2; 1, 2})
- 1 0
- 1 1
-
-({1, 1; 2, 2} >= 2)
- 0 0
- 1 1
-
-(1 >= {1, 2; 1, 2})
- 1 0
- 1 0
-
-matrix.sps:20.8-20.23: error: MATRIX: The operands of >= must have the same
-dimensions or one must be a scalar.
- 20 | PRINT ({1, 2} >= {1; 2}).
- | ^~~~~~~~~~~~~~~~
-
-matrix.sps:20.8-20.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 20 | PRINT ({1, 2} >= {1; 2}).
- | ^~~~~~
-
-matrix.sps:20.18-20.23: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 20 | PRINT ({1, 2} >= {1; 2}).
- | ^~~~~~
-
-({1, 1; 2, 2} <= {1, 2; 1, 2})
- 1 1
- 0 1
-
-({1, 1; 2, 2} <= 2)
- 1 1
- 1 1
-
-(1 <= {1, 2; 1, 2})
- 1 1
- 1 1
-
-matrix.sps:25.8-25.23: error: MATRIX: The operands of <= must have the same
-dimensions or one must be a scalar.
- 25 | PRINT ({1, 2} <= {1; 2}).
- | ^~~~~~~~~~~~~~~~
-
-matrix.sps:25.8-25.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 25 | PRINT ({1, 2} <= {1; 2}).
- | ^~~~~~
-
-matrix.sps:25.18-25.23: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 25 | PRINT ({1, 2} <= {1; 2}).
- | ^~~~~~
-
-({1, 1; 2, 2} = {1, 2; 1, 2})
- 1 0
- 0 1
-
-({1, 1; 2, 2} = 2)
- 0 0
- 1 1
-
-(1 = {1, 2; 1, 2})
- 1 0
- 1 0
-
-matrix.sps:30.8-30.22: error: MATRIX: The operands of = must have the same
-dimensions or one must be a scalar.
- 30 | PRINT ({1, 2} = {1; 2}).
- | ^~~~~~~~~~~~~~~
-
-matrix.sps:30.8-30.13: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 30 | PRINT ({1, 2} = {1; 2}).
- | ^~~~~~
-
-matrix.sps:30.17-30.22: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 30 | PRINT ({1, 2} = {1; 2}).
- | ^~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - logical operators])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT (NOT {-1, 0, 1}).
-
-PRINT ({-1, 0, 1; -1, 0, 1; -1, 0, 1} AND {-1, -1, -1; 0, 0, 0; 1, 1, 1}).
-PRINT ({-1, 0, 1} AND -1).
-PRINT ({-1, 0, 1} AND 0).
-PRINT ({-1, 0, 1} AND 1).
-PRINT ({-1, 0} AND {2; 3}).
-
-PRINT ({-1, 0, 1; -1, 0, 1; -1, 0, 1} OR {-1, -1, -1; 0, 0, 0; 1, 1, 1}).
-PRINT ({-1, 0, 1} OR -1).
-PRINT ({-1, 0, 1} OR 0).
-PRINT ({-1, 0, 1} OR 1).
-PRINT ({-1, 0} OR {2; 3}).
-
-PRINT ({-1, 0, 1; -1, 0, 1; -1, 0, 1} XOR {-1, -1, -1; 0, 0, 0; 1, 1, 1}).
-PRINT ({-1, 0, 1} XOR -1).
-PRINT ({-1, 0, 1} XOR 0).
-PRINT ({-1, 0, 1} XOR 1).
-PRINT ({-1, 0} XOR {2; 3}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-(NOT {-1, 0, 1})
- 1 1 0
-
-({-1, 0, 1; -1, 0, 1; -1, 0, 1} AND {-1, -1, -1; 0, 0, 0; 1, 1, 1})
- 0 0 0
- 0 0 0
- 0 0 1
-
-({-1, 0, 1} AND -1)
- 0 0 0
-
-({-1, 0, 1} AND 0)
- 0 0 0
-
-({-1, 0, 1} AND 1)
- 0 0 1
-
-matrix.sps:8.8-8.25: error: MATRIX: The operands of AND must have the same
-dimensions or one must be a scalar.
- 8 | PRINT ({-1, 0} AND {2; 3}).
- | ^~~~~~~~~~~~~~~~~~
-
-matrix.sps:8.8-8.14: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 8 | PRINT ({-1, 0} AND {2; 3}).
- | ^~~~~~~
-
-matrix.sps:8.20-8.25: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 8 | PRINT ({-1, 0} AND {2; 3}).
- | ^~~~~~
-
-({-1, 0, 1; -1, 0, 1; -1, 0, 1} OR {-1, -1, -1; 0, 0, 0; 1, 1, 1})
- 0 0 1
- 0 0 1
- 1 1 1
-
-({-1, 0, 1} OR -1)
- 0 0 1
-
-({-1, 0, 1} OR 0)
- 0 0 1
-
-({-1, 0, 1} OR 1)
- 1 1 1
-
-matrix.sps:14.8-14.24: error: MATRIX: The operands of OR must have the same
-dimensions or one must be a scalar.
- 14 | PRINT ({-1, 0} OR {2; 3}).
- | ^~~~~~~~~~~~~~~~~
-
-matrix.sps:14.8-14.14: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 14 | PRINT ({-1, 0} OR {2; 3}).
- | ^~~~~~~
-
-matrix.sps:14.19-14.24: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 14 | PRINT ({-1, 0} OR {2; 3}).
- | ^~~~~~
-
-({-1, 0, 1; -1, 0, 1; -1, 0, 1} XOR {-1, -1, -1; 0, 0, 0; 1, 1, 1})
- 0 0 1
- 0 0 1
- 1 1 0
-
-({-1, 0, 1} XOR -1)
- 0 0 1
-
-({-1, 0, 1} XOR 0)
- 0 0 1
-
-({-1, 0, 1} XOR 1)
- 1 1 0
-
-matrix.sps:20.8-20.25: error: MATRIX: The operands of XOR must have the same
-dimensions or one must be a scalar.
- 20 | PRINT ({-1, 0} XOR {2; 3}).
- | ^~~~~~~~~~~~~~~~~~
-
-matrix.sps:20.8-20.14: note: MATRIX: The left-hand operand is a 1×2 matrix.
- 20 | PRINT ({-1, 0} XOR {2; 3}).
- | ^~~~~~~
-
-matrix.sps:20.20-20.25: note: MATRIX: The right-hand operand is a 2×1 matrix.
- 20 | PRINT ({-1, 0} XOR {2; 3}).
- | ^~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - matrix operators])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT ({0, 1; 0, 0} * {0, 0; 1, 0}).
-PRINT ({0, 0; 1, 0} * {0, 1; 0, 0}).
-PRINT ({1, 2, 3; 4, 5, 6} * {7, 8; 9, 10; 11, 12}).
-PRINT ({3, 4, 2} * {13, 9, 7, 15; 8, 7, 4, 6; 6, 4, 0, 3}).
-COMPUTE m = {0, 1, 0, 0; 1, 0, 1, 0; 0, 1, 0, 1; 0, 0, 1, 0}.
-PRINT m**-2.
-PRINT m**-1.
-PRINT m**0.
-PRINT m**1.
-PRINT m**2.
-PRINT m**3.
-PRINT m**5.
-PRINT {3, 3.5; 3.2, 3.6}**-1/FORMAT F6.2.
-
-PRINT ({1, 2, 3} * {1, 2}).
-PRINT {1, 2, 3}**2.
-PRINT m**{1, 2}.
-PRINT m**1.5.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-({0, 1; 0, 0} * {0, 0; 1, 0})
- 1 0
- 0 0
-
-({0, 0; 1, 0} * {0, 1; 0, 0})
- 0 0
- 0 1
-
-({1, 2, 3; 4, 5, 6} * {7, 8; 9, 10; 11, 12})
- 58 64
- 139 154
-
-({3, 4, 2} * {13, 9, 7, 15; 8, 7, 4, 6; 6, 4, 0, 3})
- 83 63 37 75
-
-m**-2
- 2 0 -1 0
- 0 1 0 -1
- -1 0 1 0
- 0 -1 0 2
-
-m**-1
- 0 1 0 -1
- 1 0 0 0
- 0 0 0 1
- -1 0 1 0
-
-m**0
- 1 0 0 0
- 0 1 0 0
- 0 0 1 0
- 0 0 0 1
-
-m**1
- 0 1 0 0
- 1 0 1 0
- 0 1 0 1
- 0 0 1 0
-
-m**2
- 1 0 1 0
- 0 2 0 1
- 1 0 2 0
- 0 1 0 1
-
-m**3
- 0 2 0 1
- 2 0 3 0
- 0 3 0 2
- 1 0 2 0
-
-m**5
- 0 5 0 3
- 5 0 8 0
- 0 8 0 5
- 3 0 5 0
-
-{3, 3.5; 3.2, 3.6}**-1
- -9.00 8.75
- 8.00 -7.50
-
-matrix.sps:16.8-16.25: error: MATRIX: Matrices not conformable for
-multiplication.
- 16 | PRINT ({1, 2, 3} * {1, 2}).
- | ^~~~~~~~~~~~~~~~~~
-
-matrix.sps:16.8-16.16: note: MATRIX: The left-hand operand is a 1×3 matrix.
- 16 | PRINT ({1, 2, 3} * {1, 2}).
- | ^~~~~~~~~
-
-matrix.sps:16.20-16.25: note: MATRIX: The right-hand operand is a 1×2 matrix.
- 16 | PRINT ({1, 2, 3} * {1, 2}).
- | ^~~~~~
-
-matrix.sps:17.7-17.15: error: MATRIX: Matrix exponentation with ** requires a
-square matrix on the left-hand size, not one with dimensions 1×3.
- 17 | PRINT {1, 2, 3}**2.
- | ^~~~~~~~~
-
-matrix.sps:18.10-18.15: error: MATRIX: Matrix exponentiation with ** requires a
-scalar on the right-hand side, not a matrix with dimensions 1×2.
- 18 | PRINT m**{1, 2}.
- | ^~~~~~
-
-matrix.sps:19.10-19.12: error: MATRIX: Exponent 1.5 in matrix exponentiation is
-non-integer or outside the valid range.
- 19 | PRINT m**1.5.
- | ^~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - sequences and construction])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT {1:3:-1}.
-PRINT {1:3}.
-PRINT {1:10:2}.
-PRINT {1:11:2}.
-
-PRINT {-1:-3}.
-PRINT {-1:-3:-1}.
-PRINT {-1:-10:-2}.
-PRINT {-1:-11:-2}.
-
-PRINT {1:1}.
-PRINT {1:1:-1}.
-
-PRINT {1:3:0}.
-PRINT {-1:-3:0}.
-
-PRINT {1, 2; 3}.
-PRINT {{2; 5}, 3}.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-{1:3:-1}
-
-{1:3}
- 1 2 3
-
-{1:10:2}
- 1 3 5 7 9
-
-{1:11:2}
- 1 3 5 7 9 11
-
-{-1:-3}
-
-{-1:-3:-1}
- -1 -2 -3
-
-{-1:-10:-2}
- -1 -3 -5 -7 -9
-
-{-1:-11:-2}
- -1 -3 -5 -7 -9 -11
-
-{1:1}
- 1
-
-{1:1:-1}
- 1
-
-matrix.sps:15.12: error: MATRIX: The increment operand to : must be nonzero.
- 15 | PRINT {1:3:0}.
- | ^
-
-matrix.sps:16.14: error: MATRIX: The increment operand to : must be nonzero.
- 16 | PRINT {-1:-3:0}.
- | ^
-
-matrix.sps:18.7-18.15: error: MATRIX: This expression tries to vertically join
-matrices with differing numbers of columns.
- 18 | PRINT {1, 2; 3}.
- | ^~~~~~~~~
-
-matrix.sps:18.8-18.11: note: MATRIX: This operand is a 1×2 matrix.
- 18 | PRINT {1, 2; 3}.
- | ^~~~
-
-matrix.sps:18.14: note: MATRIX: This operand is a 1×1 matrix.
- 18 | PRINT {1, 2; 3}.
- | ^
-
-matrix.sps:19.7-19.17: error: MATRIX: This expression tries to horizontally
-join matrices with differing numbers of rows.
- 19 | PRINT {{2; 5}, 3}.
- | ^~~~~~~~~~~
-
-matrix.sps:19.8-19.13: note: MATRIX: This operand is a 2×1 matrix.
- 19 | PRINT {{2; 5}, 3}.
- | ^~~~~~
-
-matrix.sps:19.16: note: MATRIX: This operand is a 1×1 matrix.
- 19 | PRINT {{2; 5}, 3}.
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - comments])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-* Comment one.
-PRINT (1+2).
-COMMENT Comment two.
-PRINT (3+4).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-(1+2)
- 3
-
-(3+4)
- 7
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - string matrices])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE m={'This is', 'a string', 'matrix', 'including', 'some', 'long strings'}.
-PRINT m/FORMAT=A8.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-m
- This is a string matrix includin some long str
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - ABS ALL ANY ARSIN ARTAN])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT ABS({-1, 0, 1}).
-
-PRINT ALL({0, 0, 0}).
-PRINT ALL({-1, 1}).
-PRINT ALL({-1, 0, 1}).
-
-PRINT ANY({0, 0, 0}).
-PRINT ANY({-1, 1}).
-PRINT ANY({-1, 0, 1}).
-
-PRINT ARSIN({-1, 0, 1})/FORMAT=F5.2.
-
-PRINT ARTAN({-5, -1, 0, 1, 5})/FORMAT=F5.2.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-ABS({-1, 0, 1})
- 1 0 1
-
-ALL({0, 0, 0})
- 0
-
-ALL({-1, 1})
- 1
-
-ALL({-1, 0, 1})
- 0
-
-ANY({0, 0, 0})
- 0
-
-ANY({-1, 1})
- 1
-
-ANY({-1, 0, 1})
- 1
-
-ARSIN({-1, 0, 1})
- -1.57 .00 1.57
-
-ARTAN({-5, -1, 0, 1, 5})
- -1.37 -.79 .00 .79 1.37
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - BLOCK CHOL CMAX CMIN COS])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT BLOCK({1, 2; 3, 4}, 5, {7; 8; 9}, {10, 11}).
-
-COMPUTE b=CHOL({4, 12, -16; 12, 37, -43; -16, -43, 98}).
-PRINT b.
-PRINT (T(b)*b).
-
-PRINT CMAX({9, 3, 4; 5, 8, 6; 7, 4, 11}).
-
-PRINT CMIN({9, 3, 4; 5, 8, 6; 7, 4, 11}).
-
-PRINT COS({0.785, 1.57; 3.14, 1.57 + 3.14}) /FORMAT=F5.2.
-
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-BLOCK({1, 2; 3, 4}, 5, {7; 8; 9}, {10, 11})
- 1 2 0 0 0 0
- 3 4 0 0 0 0
- 0 0 5 0 0 0
- 0 0 0 7 0 0
- 0 0 0 8 0 0
- 0 0 0 9 0 0
- 0 0 0 0 10 11
-
-b
- 2 6 -8
- 0 1 5
- 0 0 3
-
-(T(b)*b)
- 4 12 -16
- 12 37 -43
- -16 -43 98
-
-CMAX({9, 3, 4; 5, 8, 6; 7, 4, 11})
- 9 8 11
-
-CMIN({9, 3, 4; 5, 8, 6; 7, 4, 11})
- 5 3 4
-
-COS({0.785, 1.57; 3.14, 1.57 + 3.14})
- .71 .00
- -1.00 .00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - CSSQ CSUM DESIGN DET DIAG])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT CSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9}).
-PRINT CSUM({1, 2, 3; 4, 5, 6; 7, 8, 9}).
-PRINT DESIGN({1, 2, 0; 2, 1, 0; 3, 0, 1}).
-PRINT DESIGN({1, 2, 0; 2, 2, 0; 3, 2, 1}).
-PRINT DET({1, 2, 3; 4, 5, 6; 7, 8, 9}) /FORMAT F4.1.
-PRINT DIAG({1, 2, 3, 4; 4, 5, 6, 7; 7, 8, 9, 10}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-CSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9})
- 66 93 126
-
-CSUM({1, 2, 3; 4, 5, 6; 7, 8, 9})
- 12 15 18
-
-DESIGN({1, 2, 0; 2, 1, 0; 3, 0, 1})
- 1 0 0 0 0 1 1 0
- 0 1 0 0 1 0 1 0
- 0 0 1 1 0 0 0 1
-
-warning: Column 2 in DESIGN argument has constant value.
-
-DESIGN({1, 2, 0; 2, 2, 0; 3, 2, 1})
- 1 0 0 1 0
- 0 1 0 1 0
- 0 0 1 0 1
-
-DET({1, 2, 3; 4, 5, 6; 7, 8, 9})
- .0
-
-DIAG({1, 2, 3, 4; 4, 5, 6, 7; 7, 8, 9, 10})
- 1
- 5
- 9
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - EVAL EXP GINV GRADE GSCH])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT EVAL({2, 0, 0; 0, 3, 4; 0, 4, 9})/FORMAT=F5.2.
-
-PRINT EXP({2, 3; 4, 5})/FORMAT F5.2.
-
-PRINT GINV({1, 2})/FORMAT F5.2.
-COMPUTE a={1, 2, 3; 4, 5, 6; 7, 8, 9}.
-COMPUTE g=GINV(a).
-PRINT (a*g*a)/FORMAT F5.2.
-
-PRINT GRADE({1, 0, 3; 3, 1, 2; 3, 0, 5}).
-COMPUTE x={26, 690, 323, 208, 671, 818, 732, 711, 585, 792}.
-COMPUTE asort=x.
-COMPUTE asort(GRADE(asort))=asort.
-PRINT asort.
-COMPUTE dsort=x.
-COMPUTE dsort(GRADE(-dsort))=dsort.
-PRINT dsort.
-
-PRINT (GSCH({3, 2; 1, 2}) * SQRT(10))/FORMAT F5.2.
-PRINT (GSCH({0, 3, 6, 2; 0, 1, 2, 2}) * SQRT(10))/FORMAT F5.2.
-PRINT GSCH({0; 0}).
-PRINT GSCH({0, 0, 0; 0, 0, 0}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-EVAL({2, 0, 0; 0, 3, 4; 0, 4, 9})
- 11.00
- 2.00
- 1.00
-
-EXP({2, 3; 4, 5})
- 7.39 20.09
- 54.60 148.4
-
-GINV({1, 2})
- .20
- .40
-
-(a*g*a)
- 1.00 2.00 3.00
- 4.00 5.00 6.00
- 7.00 8.00 9.00
-
-GRADE({1, 0, 3; 3, 1, 2; 3, 0, 5})
- 3 1 6
- 7 4 5
- 8 2 9
-
-asort
- 26 208 323 585 671 690 711 732 792 818
-
-dsort
- 818 792 732 711 690 671 585 323 208 26
-
-(GSCH({3, 2; 1, 2}) * SQRT(10))
- 3.00 -1.00
- 1.00 3.00
-
-(GSCH({0, 3, 6, 2; 0, 1, 2, 2}) * SQRT(10))
- 3.00 -1.00
- 1.00 3.00
-
-matrix.sps:22.12-22.17: error: MATRIX: GSCH requires its argument to have at
-least as many columns as rows, but it has dimensions 2×1.
- 22 | PRINT GSCH({0; 0}).
- | ^~~~~~
-
-matrix.sps:23.12-23.29: error: MATRIX: 2×3 argument to GSCH contains only 0
-linearly independent columns.
- 23 | PRINT GSCH({0, 0, 0; 0, 0, 0}).
- | ^~~~~~~~~~~~~~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - IDENT INV KRONEKER LG10 LN])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT IDENT(1).
-PRINT IDENT(2).
-PRINT IDENT(3,5).
-PRINT IDENT(5,3).
-
-PRINT INV({3, 3.5; 3.2, 3.6})/FORMAT F8.2.
-PRINT INV({4, 7; 2, 6})/FORMAT F8.2.
-PRINT (INV({4, -2, 1; 5, 0, 3; -1, 2, 6})*52)/FORMAT F8.2.
-
-PRINT KRONEKER({1, 2; 3, 4}, {0, 5; 6, 7}).
-
-PRINT LG10({1, 10, 100, 1000}).
-
-PRINT LN({1, 2; 3, 4})/FORMAT F5.2.
-PRINT LN(0).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-IDENT(1)
- 1
-
-IDENT(2)
- 1 0
- 0 1
-
-IDENT(3,5)
- 1 0 0 0 0
- 0 1 0 0 0
- 0 0 1 0 0
-
-IDENT(5,3)
- 1 0 0
- 0 1 0
- 0 0 1
- 0 0 0
- 0 0 0
-
-INV({3, 3.5; 3.2, 3.6})
- -9.00 8.75
- 8.00 -7.50
-
-INV({4, 7; 2, 6})
- .60 -.70
- -.20 .40
-
-(INV({4, -2, 1; 5, 0, 3; -1, 2, 6})*52)
- -6.00 14.00 -6.00
- -33.00 25.00 -7.00
- 10.00 -6.00 10.00
-
-KRONEKER({1, 2; 3, 4}, {0, 5; 6, 7})
- 0 5 0 10
- 6 7 12 14
- 0 15 0 20
- 18 21 24 28
-
-LG10({1, 10, 100, 1000})
- 0 1 2 3
-
-LN({1, 2; 3, 4})
- .00 .69
- 1.10 1.39
-
-matrix.sps:16.7-16.11: error: MATRIX: Argument 1 to matrix function LN must be
-greater than 0.
- 16 | PRINT LN(0).
- | ^~~~~
-
-matrix.sps:16.10: note: MATRIX: Argument 1 is 0.
- 16 | PRINT LN(0).
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MAGIC])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-
-LOOP n=3 to 10.
- COMPUTE m=MAGIC(n).
- COMPUTE total=n*(n**2 + 1) / 2.
- COMPUTE tb={MSUM(DIAG(T(m))), CSUM(m), MSUM(DIAG(m))} - total.
- COMPUTE lr=RSUM(m) - total.
- PRINT {tb; lr, m, lr; tb}/FORMAT F4.0.
-END LOOP.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-{tb; lr, m, lr; tb}
- 0 0 0 0 0
- 0 8 1 6 0
- 0 3 5 7 0
- 0 4 9 2 0
- 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0
- 0 1 5 12 16 0
- 0 15 11 6 2 0
- 0 14 8 9 3 0
- 0 4 10 7 13 0
- 0 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0 0
- 0 17 24 1 8 15 0
- 0 23 5 7 14 16 0
- 0 4 6 13 20 22 0
- 0 10 12 19 21 3 0
- 0 11 18 25 2 9 0
- 0 0 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0 0 0
- 0 1 5 9 28 32 36 0
- 0 35 30 27 10 7 2 0
- 0 24 14 22 18 17 16 0
- 0 13 23 15 19 20 21 0
- 0 34 31 26 11 6 3 0
- 0 4 8 12 25 29 33 0
- 0 0 0 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0 0 0 0
- 0 30 39 48 1 10 19 28 0
- 0 38 47 7 9 18 27 29 0
- 0 46 6 8 17 26 35 37 0
- 0 5 14 16 25 34 36 45 0
- 0 13 15 24 33 42 44 4 0
- 0 21 23 32 41 43 3 12 0
- 0 22 31 40 49 2 11 20 0
- 0 0 0 0 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0 0 0 0 0
- 0 1 9 17 25 40 48 56 64 0
- 0 63 55 47 39 26 18 10 2 0
- 0 3 11 19 27 38 46 54 62 0
- 0 61 53 45 37 28 20 12 4 0
- 0 60 52 44 32 33 21 13 5 0
- 0 6 14 22 30 35 43 51 59 0
- 0 58 50 42 34 31 23 15 7 0
- 0 8 16 24 36 29 41 49 57 0
- 0 0 0 0 0 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0 0 0 0 0 0
- 0 47 58 69 80 1 12 23 34 45 0
- 0 57 68 79 9 11 22 33 44 46 0
- 0 67 78 8 10 21 32 43 54 56 0
- 0 77 7 18 20 31 42 53 55 66 0
- 0 6 17 19 30 41 52 63 65 76 0
- 0 16 27 29 40 51 62 64 75 5 0
- 0 26 28 39 50 61 72 74 4 15 0
- 0 36 38 49 60 71 73 3 14 25 0
- 0 37 48 59 70 81 2 13 24 35 0
- 0 0 0 0 0 0 0 0 0 0 0
-{tb; lr, m, lr; tb}
- 0 0 0 0 0 0 0 0 0 0 0 0
- 0 1 9 17 25 33 68 76 84 92 100 0
- 0 99 91 83 75 67 34 26 18 10 2 0
- 0 3 11 19 27 35 66 74 82 90 98 0
- 0 97 89 81 72 65 36 29 20 12 4 0
- 0 60 42 58 44 56 50 49 53 47 46 0
- 0 41 59 43 57 45 51 52 48 54 55 0
- 0 96 88 80 73 64 37 28 21 13 5 0
- 0 6 14 22 30 38 63 71 79 87 95 0
- 0 94 86 78 70 62 39 31 23 15 7 0
- 0 8 16 24 32 40 61 69 77 85 93 0
- 0 0 0 0 0 0 0 0 0 0 0 0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MAKE MDIAG MMAX MMIN MOD])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT MAKE(1, 2, 3).
-PRINT MAKE(2, 1, 4).
-PRINT MAKE(2, 3, 5).
-
-PRINT MDIAG({1, 2, 3, 4}).
-PRINT MDIAG({1; 2; 3; 4}).
-PRINT MDIAG({1, 2; 3, 4}).
-
-PRINT MMAX({55, 44; 66, 11}).
-
-PRINT MMIN({55, 44; 66, 11}).
-
-PRINT MOD({5, 4, 3, 2, 1, 0}, 3).
-PRINT MOD({5, 4, 3, 2, 1, 0}, -3).
-PRINT MOD({-5, -4, -3, -2, -1, 0}, 3).
-PRINT MOD({-5, -4, -3, -2, -1, 0}, -3).
-PRINT MOD({5, 4, 3, 2, 1, 0}, 1.5) /FORMAT F5.1.
-PRINT MOD({5, 4, 3, 2, 1, 0}, 0).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-MAKE(1, 2, 3)
- 3 3
-
-MAKE(2, 1, 4)
- 4
- 4
-
-MAKE(2, 3, 5)
- 5 5 5
- 5 5 5
-
-MDIAG({1, 2, 3, 4})
- 1 0 0 0
- 0 2 0 0
- 0 0 3 0
- 0 0 0 4
-
-MDIAG({1; 2; 3; 4})
- 1 0 0 0
- 0 2 0 0
- 0 0 3 0
- 0 0 0 4
-
-matrix.sps:8.13-8.24: error: MATRIX: Function MDIAG argument 1 must be a
-vector, not a 2×2 matrix.
- 8 | PRINT MDIAG({1, 2; 3, 4}).
- | ^~~~~~~~~~~~
-
-MMAX({55, 44; 66, 11})
- 66
-
-MMIN({55, 44; 66, 11})
- 11
-
-MOD({5, 4, 3, 2, 1, 0}, 3)
- 2 1 0 2 1 0
-
-MOD({5, 4, 3, 2, 1, 0}, -3)
- 2 1 0 2 1 0
-
-MOD({-5, -4, -3, -2, -1, 0}, 3)
- -2 -1 0 -2 -1 0
-
-MOD({-5, -4, -3, -2, -1, 0}, -3)
- -2 -1 0 -2 -1 0
-
-MOD({5, 4, 3, 2, 1, 0}, 1.5)
- .5 1.0 .0 .5 1.0 .0
-
-matrix.sps:19.7-19.32: error: MATRIX: Argument 2 to matrix function MOD must
-not be equal to 0.
- 19 | PRINT MOD({5, 4, 3, 2, 1, 0}, 0).
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MSSQ MSUM NCOL NROW RANK])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT MSSQ({1, 0, 1; -2, -3, 1; 3, 3, 0}).
-
-PRINT MSUM({1, 0, 1; -2, -3, 1; 3, 3, 0}).
-
-PRINT NCOL({1, 0; -2, -3; 3, 3}).
-
-PRINT NROW({1, 0; -2, -3; 3, 3}).
-
-PRINT RANK({1, 0, 1; -2, -3, 1; 3, 3, 0}).
-PRINT RANK({1, 1, 0, 2; -1, -1, 0, -2}).
-PRINT RANK({1, -1; 1, -1; 0, 0; 2, -2}).
-PRINT RANK({1, 2, 1; -2, -3, 1; 3, 5, 0}).
-PRINT RANK({1, 0, 2; 2, 1, 0; 3, 2, 1}).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-MSSQ({1, 0, 1; -2, -3, 1; 3, 3, 0})
- 34
-
-MSUM({1, 0, 1; -2, -3, 1; 3, 3, 0})
- 4
-
-NCOL({1, 0; -2, -3; 3, 3})
- 2
-
-NROW({1, 0; -2, -3; 3, 3})
- 3
-
-RANK({1, 0, 1; -2, -3, 1; 3, 3, 0})
- 2
-
-RANK({1, 1, 0, 2; -1, -1, 0, -2})
- 1
-
-RANK({1, -1; 1, -1; 0, 0; 2, -2})
- 1
-
-RANK({1, 2, 1; -2, -3, 1; 3, 5, 0})
- 2
-
-RANK({1, 0, 2; 2, 1, 0; 3, 2, 1})
- 3
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - RESHAPE RMAX RMIN RND RNKORDER])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT RESHAPE(1:12, 1, 12).
-PRINT RESHAPE(1:12, 2, 6).
-PRINT RESHAPE(1:12, 3, 4).
-PRINT RESHAPE(1:12, 4, 3).
-PRINT RESHAPE(1:12, 6, 2).
-PRINT RESHAPE(1:12, 12, 1).
-
-PRINT RMAX({1, 0, 1; -2, -3, 1; 3, 3, 0}).
-
-PRINT RMIN({1, 0, 1; -2, -3, 1; 3, 3, 0}).
-
-PRINT RND({-1.6, -1.5, -1.4;
- -.6, -.5, -.4;
- .4, .5, .6;
- 1.4, 1.5, 1.6})/FORMAT F5.1.
-
-PRINT RNKORDER({1, 0, 3; 3, 1, 2; 3, 0, 5}) /FORMAT F5.1.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-RESHAPE(1:12, 1, 12)
- 1 2 3 4 5 6 7 8 9 10 11 12
-
-RESHAPE(1:12, 2, 6)
- 1 2 3 4 5 6
- 7 8 9 10 11 12
-
-RESHAPE(1:12, 3, 4)
- 1 2 3 4
- 5 6 7 8
- 9 10 11 12
-
-RESHAPE(1:12, 4, 3)
- 1 2 3
- 4 5 6
- 7 8 9
- 10 11 12
-
-RESHAPE(1:12, 6, 2)
- 1 2
- 3 4
- 5 6
- 7 8
- 9 10
- 11 12
-
-RESHAPE(1:12, 12, 1)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
-
-RMAX({1, 0, 1; -2, -3, 1; 3, 3, 0})
- 1
- 1
- 3
-
-RMIN({1, 0, 1; -2, -3, 1; 3, 3, 0})
- 0
- -3
- 0
-
-RND({-1.6, -1.5, -1.4;
- -.6, -.5, -.4;
- .4, .5, .6;
- 1.4, 1.5, 1.6})
- -2.0 -2.0 -1.0
- -1.0 .0 .0
- .0 .0 1.0
- 1.0 2.0 2.0
-
-RNKORDER({1, 0, 3; 3, 1, 2; 3, 0, 5})
- 3.5 1.5 7.0
- 7.0 3.5 5.0
- 7.0 1.5 9.0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - RSSQ RSUM SIN SOLVE SQRT])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT RSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9}).
-PRINT RSUM({1, 2, 3; 4, 5, 6; 7, 8, 9}).
-
-PRINT SIN({0, .78, 1.57, 2.35, 3.14}) /FORMAT F5.2.
-
-PRINT SOLVE({2, 3; 4, 9}, {6, 2; 15, 5}) /FORMAT=F6.2.
-PRINT SOLVE({1, 3, -2; 3, 5, 6; 2, 4, 3}, {5; 7; 8}) /FORMAT=F6.2.
-PRINT SOLVE({2, 1, -1; -3, -1, 2; -2, 1, 2}, {8; -11; -3}) /FORMAT=F6.2.
-PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
-
-PRINT SQRT({0, 1, 2, 3, 4, 9, 81}) /FORMAT=F5.2.
-PRINT SQRT(-1).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-RSSQ({1, 2, 3; 4, 5, 6; 7, 8, 9})
- 14
- 77
- 194
-
-RSUM({1, 2, 3; 4, 5, 6; 7, 8, 9})
- 6
- 15
- 24
-
-SIN({0, .78, 1.57, 2.35, 3.14})
- .00 .70 1.00 .71 .00
-
-SOLVE({2, 3; 4, 9}, {6, 2; 15, 5})
- 1.50 .50
- 1.00 .33
-
-SOLVE({1, 3, -2; 3, 5, 6; 2, 4, 3}, {5; 7; 8})
- -15.00
- 8.00
- 2.00
-
-SOLVE({2, 1, -1; -3, -1, 2; -2, 1, 2}, {8; -11; -3})
- 2.00
- 3.00
- -1.00
-
-matrix.sps:10.7-10.33: error: MATRIX: SOLVE arguments must have the same number
-of rows.
- 10 | PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-matrix.sps:10.13-10.24: note: MATRIX: Argument 1 has dimensions 2×2.
- 10 | PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
- | ^~~~~~~~~~~~
-
-matrix.sps:10.27-10.32: note: MATRIX: Argument 2 has dimensions 1×2.
- 10 | PRINT SOLVE({1, 2; 3, 4}, {1, 2}).
- | ^~~~~~
-
-SQRT({0, 1, 2, 3, 4, 9, 81})
- .00 1.00 1.41 1.73 2.00 3.00 9.00
-
-matrix.sps:13.7-13.14: error: MATRIX: Argument 1 to matrix function SQRT must
-be greater than or equal to 0.
- 13 | PRINT SQRT(-1).
- | ^~~~~~~~
-
-matrix.sps:13.12-13.13: note: MATRIX: Argument 1 is -1.
- 13 | PRINT SQRT(-1).
- | ^~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - SSCP SVAL SWEEP TRACE TRANSPOS TRUNC])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE m={1, 2, 3; 4, 5, 6}
-COMPUTE sscp1=SSCP(m).
-COMPUTE sscp2=T(m)*m.
-PRINT sscp1.
-PRINT (sscp1 <> sscp2).
-
-PRINT SVAL({1, 1; 0, 0})/FORMAT F5.2.
-PRINT SVAL({1, 0, 1; 0, 1, 1; 0, 0, 0})/FORMAT F5.2.
-PRINT SVAL({1, 0, 0, 0, 2; 0, 0, 3, 0, 0; 0, 0, 0, 0, 0; 0, 2, 0, 0, 0})
- /FORMAT F5.2.
-PRINT SVAL({2, 4; 1, 3; 0, 0; 0, 0})/FORMAT F5.2.
-
-COMPUTE s0 = {6, 12, 0, 12; 12, 28, 0, 25; 0, 0, 6, 2; 12, 25, 2, 28}.
-PRINT SWEEP(s0, 1)/FORMAT F5.2.
-PRINT SWEEP(SWEEP(s0, 1), 2)/FORMAT F5.2.
-PRINT SWEEP(SWEEP(SWEEP(s0, 1), 2), 3)/FORMAT F5.2.
-
-COMPUTE s1 = {6, 12, 0, 12; 12, 0, 0, 25; 0, 0, 6, 2; 12, 25, 2, 28}.
-PRINT SWEEP(s1, 2).
-
-COMPUTE s2 = {0, 1, 2; 3, 4, 5; 6, 7, 8}.
-PRINT SWEEP(s2, 1).
-PRINT SWEEP(s2, 2).
-PRINT SWEEP(s2, 3).
-
-PRINT TRACE(s0).
-
-PRINT T(s0).
-PRINT TRANSPOS(s0).
-PRINT ALL(T(T(s0)) = s0).
-
-PRINT TRUNC(SVAL({2, 4; 1, 3; 0, 0; 0, 0})).
-PRINT TRUNC(-SVAL({2, 4; 1, 3; 0, 0; 0, 0})).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-sscp1
- 17 22 27
- 22 29 36
- 27 36 45
-
-(sscp1 <> sscp2)
- 0 0 0
- 0 0 0
- 0 0 0
-
-SVAL({1, 1; 0, 0})
- 1.41
- .00
-
-SVAL({1, 0, 1; 0, 1, 1; 0, 0, 0})
- 1.73
- 1.00
- .00
-
-SVAL({1, 0, 0, 0, 2; 0, 0, 3, 0, 0; 0, 0, 0, 0, 0; 0, 2, 0, 0, 0})
- 3.00
- 2.24
- 2.00
- .00
-
-SVAL({2, 4; 1, 3; 0, 0; 0, 0})
- 5.46
- .37
-
-SWEEP(s0, 1)
- .17 2.00 .00 2.00
- -2.00 4.00 .00 1.00
- .00 .00 6.00 2.00
- -2.00 1.00 2.00 4.00
-
-SWEEP(SWEEP(s0, 1), 2)
- 1.17 -.50 .00 1.50
- -.50 .25 .00 .25
- .00 .00 6.00 2.00
- -1.50 -.25 2.00 3.75
-
-SWEEP(SWEEP(SWEEP(s0, 1), 2), 3)
- 1.17 -.50 .00 1.50
- -.50 .25 .00 .25
- .00 .00 .17 .33
- -1.50 -.25 -.33 3.08
-
-SWEEP(s1, 2)
- 6 0 0 12
- 0 0 0 0
- 0 0 6 2
- 12 0 2 28
-
-SWEEP(s2, 1)
- 0 0 0
- 0 4 5
- 0 7 8
-
-SWEEP(s2, 2)
- -.7500000000 -.2500000000 .7500000000
- .7500000000 .2500000000 1.2500000000
- .7500000000 -1.7500000000 -.7500000000
-
-SWEEP(s2, 3)
- -1.5000000000 -.7500000000 -.2500000000
- -.7500000000 -.3750000000 -.6250000000
- .7500000000 .8750000000 .1250000000
-
-TRACE(s0)
- 68
-
-T(s0)
- 6 12 0 12
- 12 28 0 25
- 0 0 6 2
- 12 25 2 28
-
-TRANSPOS(s0)
- 6 12 0 12
- 12 28 0 25
- 0 0 6 2
- 12 25 2 28
-
-ALL(T(T(s0)) = s0)
- 1
-
-TRUNC(SVAL({2, 4; 1, 3; 0, 0; 0, 0}))
- 5
- 0
-
-TRUNC(-SVAL({2, 4; 1, 3; 0, 0; 0, 0}))
- -5
- 0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - UNIFORM])
-AT_DATA([matrix.sps], [dnl
-SET SEED=10.
-MATRIX.
-PRINT (UNIFORM(4, 5)*10)/FORMAT F5.2.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-(UNIFORM(4, 5)*10)
- 7.71 2.99 .21 4.95 6.34
- 4.43 7.49 8.32 4.99 5.83
- 2.25 .25 1.98 7.09 7.61
- 2.66 1.69 2.64 .88 1.50
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - invalid function arguments])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE x=MOD({1,2,3},{4,5,6}).
-COMPUTE x=MDIAG({1, 2; 3, 4}).
-COMPUTE x=ARSIN(2).
-COMPUTE x=ARSIN({1, 1; -1, 2}).
-COMPUTE x=CDF.UNIFORM(2,1,1).
-COMPUTE x=CDF.UNIFORM(1,2,1).
-COMPUTE x=CDF.UNIFORM({1,2},1,1).
-COMPUTE x=MAGIC(2).
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.23-2.29: error: MATRIX: Function MOD argument 2 must be a scalar,
-not a 1×3 matrix.
- 2 | COMPUTE x=MOD({1,2,3},{4,5,6}).
- | ^~~~~~~
-
-matrix.sps:3.17-3.28: error: MATRIX: Function MDIAG argument 1 must be a
-vector, not a 2×2 matrix.
- 3 | COMPUTE x=MDIAG({1, 2; 3, 4}).
- | ^~~~~~~~~~~~
-
-matrix.sps:4.17: error: MATRIX: Argument 1 to matrix function ARSIN is 2, which
-is outside the valid range [[-1,1]].
- 4 | COMPUTE x=ARSIN(2).
- | ^
-
-matrix.sps:5.17-5.29: error: MATRIX: Row 2, column 2 of argument 1 to matrix
-function ARSIN is 2, which is outside the valid range [[-1,1]].
- 5 | COMPUTE x=ARSIN({1, 1; -1, 2}).
- | ^~~~~~~~~~~~~
-
-error: Argument 1 to matrix function CDF.UNIFORM must be less than or equal to
-argument 3.
-
-matrix.sps:6.23: note: MATRIX: Argument 1 is 2.
- 6 | COMPUTE x=CDF.UNIFORM(2,1,1).
- | ^
-
-matrix.sps:6.27: note: MATRIX: Argument 3 is 1.
- 6 | COMPUTE x=CDF.UNIFORM(2,1,1).
- | ^
-
-error: Argument 2 to matrix function CDF.UNIFORM must be less than or equal to
-argument 3.
-
-matrix.sps:7.25: note: MATRIX: Argument 2 is 2.
- 7 | COMPUTE x=CDF.UNIFORM(1,2,1).
- | ^
-
-matrix.sps:7.27: note: MATRIX: Argument 3 is 1.
- 7 | COMPUTE x=CDF.UNIFORM(1,2,1).
- | ^
-
-error: Argument 1 to matrix function CDF.UNIFORM must be less than or equal to
-argument 3.
-
-matrix.sps:8.23-8.27: note: MATRIX: Row 1, column 2 of argument 1 is 2.
- 8 | COMPUTE x=CDF.UNIFORM({1,2},1,1).
- | ^~~~~
-
-matrix.sps:8.31: note: MATRIX: Argument 3 is 1.
- 8 | COMPUTE x=CDF.UNIFORM({1,2},1,1).
- | ^
-
-matrix.sps:9.11-9.18: error: MATRIX: Argument 1 to matrix function MAGIC must
-be greater than or equal to 3.
- 9 | COMPUTE x=MAGIC(2).
- | ^~~~~~~~
-
-matrix.sps:9.17: note: MATRIX: Argument 1 is 2.
- 9 | COMPUTE x=MAGIC(2).
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - invalid number function arguments])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE x=ABS().
-COMPUTE x=ABS(1,2).
-COMPUTE x=KRONEKER(1,2,3).
-COMPUTE x=IDENT().
-COMPUTE x=IDENT(1,2,3).
-COMPUTE x=BLOCK().
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2: error: COMPUTE: Matrix function ABS requires 1 argument.
-
-matrix.sps:3: error: COMPUTE: Matrix function ABS requires 1 argument.
-
-matrix.sps:4: error: COMPUTE: Matrix function KRONEKER requires 2 arguments.
-
-matrix.sps:5: error: COMPUTE: Matrix function IDENT requires 1 or 2 arguments,
-but 0 were provided.
-
-matrix.sps:6: error: COMPUTE: Matrix function IDENT requires 1 or 2 arguments,
-but 3 were provided.
-
-matrix.sps:7: error: COMPUTE: Matrix function BLOCK requires at least one
-argument.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - CALL SETDIAG])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE x={1, 2, 3; 4, 5, 6; 7, 8, 9}.
-
-COMPUTE x1=x.
-CALL SETDIAG(x1, 10).
-PRINT x1.
-
-COMPUTE x2=x.
-CALL SETDIAG(x2, {10, 11}).
-PRINT x2.
-
-COMPUTE x3=x.
-CALL SETDIAG(x3, {10, 11, 12}).
-PRINT x3.
-
-COMPUTE x4=x.
-CALL SETDIAG(x4, {10, 11, 12, 13}).
-PRINT x4.
-
-COMPUTE x5=x.
-CALL SETDIAG(x5, {10, 11; 12, 13}).
-PRINT x5.
-
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-x1
- 10 2 3
- 4 10 6
- 7 8 10
-
-x2
- 10 2 3
- 4 11 6
- 7 8 9
-
-x3
- 10 2 3
- 4 11 6
- 7 8 12
-
-x4
- 10 2 3
- 4 11 6
- 7 8 12
-
-matrix.sps:21.18-21.33: error: MATRIX: SETDIAG argument 2 must be a scalar or a
-vector, not a 2×2 matrix.
- 21 | CALL SETDIAG(x5, {10, 11; 12, 13}).
- | ^~~~~~~~~~~~~~~~
-
-x5
- 1 2 3
- 4 5 6
- 7 8 9
-])
-AT_CLEANUP
-
-dnl I have some doubts about the correctness of the results below.
-AT_SETUP([MATRIX - CALL EIGEN])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-CALL EIGEN({1, 0; 0, 1}, evec, eval).
-PRINT evec.
-PRINT eval.
-
-CALL EIGEN({3, 2, 4; 2, 0, 2; 4, 2, 3}, evec2, eval2).
-PRINT evec2.
-PRINT eval2.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-evec
- 1 0
- 0 1
-
-eval
- 1
- 1
-
-evec2
- -.6666666667 .0000000000 .7453559925
- -.3333333333 -.8944271910 -.2981423970
- -.6666666667 .4472135955 -.5962847940
-
-eval2
- 8.0000000000
- -1.0000000000
- -1.0000000000
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - CALL SVD])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-CALL SVD({3, 2, 2; 2, 3, -2}, u, s, v).
-PRINT (u * s * T(v))/FORMAT F5.1.
-
-CALL SVD({2, 4; 1, 3; 0, 0; 0, 0}, u, s, v).
-PRINT (u*s*T(v))/FORMAT F5.1.
-
-CALL SVD({-3, 1; 6, -2; 6, -2}, u, s, v).
-PRINT (u*s*T(v))/FORMAT F5.1.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-(u * s * T(v))
- 3.0 2.0 2.0
- 2.0 3.0 -2.0
-
-(u*s*T(v))
- 2.0 4.0
- 1.0 3.0
- .0 .0
- .0 .0
-
-(u*s*T(v))
- -3.0 1.0
- 6.0 -2.0
- 6.0 -2.0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - PRINT])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-PRINT/TITLE="title 1".
-PRINT/SPACE=2/TITLE="title 2".
-
-COMPUTE m={1, 2, 3; 3, 4, 5; 6, 7, 8}.
-PRINT m/RLABELS=123, a b c, long name.
-PRINT m/RNAMES={'123', 'a b c', 'long name'}.
-PRINT m/CLABELS=col1, col2, long column name.
-PRINT m/CNAMES={'col1', 'col2', 'long column name'}.
-PRINT m/RLABELS=123, a b c, long name
- /CLABELS=col1, col2, long column name.
-PRINT m/RNAMES={'123', 'a b c', 'long name'}
- /CNAMES={'col1', 'col2', 'long column name'}.
-PRINT {123e10, 456e10, 500}.
-END MATRIX.
-])
-
-AT_DATA([matrix-tables.sps], [dnl
-SET MDISPLAY=TABLES.
-INCLUDE 'matrix.sps'.
-])
-
-AT_CHECK([pspp matrix.sps], [0], [dnl
-title 1
-
-
-
-title 2
-
-m
-123 1 2 3
-a b c 3 4 5
-long nam 6 7 8
-
-m
-123 1 2 3
-a b c 3 4 5
-long nam 6 7 8
-
-m
- col1 col2 long col
- 1 2 3
- 3 4 5
- 6 7 8
-
-m
- col1 col2 long col
- 1 2 3
- 3 4 5
- 6 7 8
-
-m
- col1 col2 long col
-123 1 2 3
-a b c 3 4 5
-long nam 6 7 8
-
-m
- col1 col2 long col
-123 1 2 3
-a b c 3 4 5
-long nam 6 7 8
-
-{123e10, 456e10, 500}
- 10 ** 12 X
- 1.2300000000 4.5600000000 .0000000005
-])
-
-AT_CHECK([pspp matrix-tables.sps], [0], [dnl
-title 1
-
-
-
-title 2
-
- m
-+---------+-----+
-|123 |1 2 3|
-|a b c |3 4 5|
-|long name|6 7 8|
-+---------+-----+
-
- m
-+--------+-----+
-|123 |1 2 3|
-|a b c |3 4 5|
-|long nam|6 7 8|
-+--------+-----+
-
- m
-+----+----+----------------+
-|col1|col2|long column name|
-+----+----+----------------+
-| 1| 2| 3|
-| 3| 4| 5|
-| 6| 7| 8|
-+----+----+----------------+
-
- m
-+----+----+--------+
-|col1|col2|long col|
-+----+----+--------+
-| 1| 2| 3|
-| 3| 4| 5|
-| 6| 7| 8|
-+----+----+--------+
-
- m
-+---------+----+----+----------------+
-| |col1|col2|long column name|
-+---------+----+----+----------------+
-|123 | 1| 2| 3|
-|a b c | 3| 4| 5|
-|long name| 6| 7| 8|
-+---------+----+----+----------------+
-
- m
-+--------+----+----+--------+
-| |col1|col2|long col|
-+--------+----+----+--------+
-|123 | 1| 2| 3|
-|a b c | 3| 4| 5|
-|long nam| 6| 7| 8|
-+--------+----+----+--------+
-
- {123e10, 456e10, 500}
-+----------------------------------------------+
-|1.2300000000[[a]] 4.5600000000[[a]] .0000000005[[a]]|
-+----------------------------------------------+
-a. × 10**12
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - DO IF])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-DO IF 1.
-PRINT/TITLE '1'.
-END IF.
-
-DO IF 0.
-PRINT/TITLE '2'.
-ELSE IF 1.
-PRINT/TITLE '3'.
-END IF.
-
-DO IF -1.
-PRINT/TITLE '4'.
-ELSE IF 0.
-PRINT/TITLE '5'.
-ELSE.
-PRINT/TITLE '6'.
-END IF.
-
-DO IF {1, 2}.
-END IF.
-
-DO IF 0.
-ELSE IF {}.
-END IF.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-1
-
-3
-
-6
-
-matrix.sps:20.7-20.12: error: MATRIX: Expression for DO IF must evaluate to
-scalar, not a 1×2 matrix.
- 20 | DO IF {1, 2}.
- | ^~~~~~
-
-matrix.sps:24.9-24.10: error: MATRIX: Expression for ELSE IF must evaluate to
-scalar, not a 0×0 matrix.
- 24 | ELSE IF {}.
- | ^~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - unbounded LOOP])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-* Truly unbounded loop.
-COMPUTE x=0.
-COMPUTE y={}.
-LOOP.
-COMPUTE x=x+1.
-COMPUTE y={y, x}.
-END LOOP.
-PRINT x.
-PRINT y.
-
-* Unbounded loop terminates with BREAK.
-COMPUTE x=0.
-COMPUTE y={}.
-LOOP.
-COMPUTE x=x+1.
-COMPUTE y={y, x}.
-DO IF x >= 20.
- BREAK.
-END IF.
-END LOOP.
-PRINT x.
-PRINT y.
-
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-x
- 40
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
-40
-
-x
- 20
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-20
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - indexed or conditional LOOP])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-* Indexed loop terminates based on index.
-COMPUTE y={}.
-LOOP x=1 TO 20.
-COMPUTE y={y, x}.
-END LOOP.
-PRINT x.
-PRINT y.
-
-* Indexed loop terminates based on MXLOOPS.
-COMPUTE y={}.
-LOOP x=1 TO 50.
-COMPUTE y={y, x}.
-END LOOP.
-PRINT x.
-PRINT y.
-
-* Indexed loop terminates with BREAK.
-COMPUTE y={}.
-LOOP x=1 TO 50.
-COMPUTE y={y, x}.
-DO IF x >= 20.
- BREAK.
-END IF.
-END LOOP.
-PRINT x.
-PRINT y.
-
-* Indexed loop terminates with top IF.
-COMPUTE y={}.
-LOOP x=1 TO 50 IF NCOL(y) < 15.
-COMPUTE y={y, x}.
-END LOOP.
-PRINT x.
-PRINT y.
-
-* Indexed loop terminates with bottom IF.
-COMPUTE y={}.
-LOOP x=1 TO 50.
-COMPUTE y={y, x}.
-END LOOP IF NCOL(y) >= 22.
-PRINT x.
-PRINT y.
-
-* Index behavior.
-COMPUTE indexing={
- 1, 10, 1;
- 1, 10, 2;
- 1, 10, 3;
- 1, 10, -1;
- 1, 10, 0;
- 10, 1, -1;
- 10, 1, -2;
- 10, 1, -3;
- 10, 1, 1;
- 10, 1, 0
-}.
-LOOP i=1 TO NROW(indexing).
- COMPUTE y={}.
- LOOP j=indexing(i, 1) TO indexing(i, 2) BY indexing(i, 3).
- COMPUTE y={y, j}.
- END LOOP.
- PRINT {indexing(i, :), y}.
-END LOOP.
-
-LOOP i={} TO 5.
-END LOOP.
-
-LOOP i=5 TO {}.
-END LOOP.
-
-LOOP i=5 TO 8 BY {}.
-END LOOP.
-
-LOOP IF {}.
-END LOOP.
-
-LOOP.
-END LOOP IF {}.
-
-LOOP i=1e100 to 1e200.
-END LOOP.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-x
- 20
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-20
-
-x
- 40
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
-40
-
-x
- 20
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-20
-
-x
- 16
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-
-x
- 22
-
-y
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
-20 21 22
-
-{indexing(i, :), y}
- 1 10 1 1 2 3 4 5 6 7 8 9 10
-{indexing(i, :), y}
- 1 10 2 1 3 5 7 9
-{indexing(i, :), y}
- 1 10 3 1 4 7 10
-{indexing(i, :), y}
- 1 10 -1
-{indexing(i, :), y}
- 1 10 0
-{indexing(i, :), y}
- 10 1 -1 10 9 8 7 6 5 4 3 2 1
-{indexing(i, :), y}
- 10 1 -2 10 8 6 4 2
-{indexing(i, :), y}
- 10 1 -3 10 7 4 1
-{indexing(i, :), y}
- 10 1 1
-{indexing(i, :), y}
- 10 1 0
-
-matrix.sps:66.8-66.9: error: MATRIX: Expression for LOOP must evaluate to
-scalar, not a 0×0 matrix.
- 66 | LOOP i={} TO 5.
- | ^~
-
-matrix.sps:69.13-69.14: error: MATRIX: Expression for TO must evaluate to
-scalar, not a 0×0 matrix.
- 69 | LOOP i=5 TO {}.
- | ^~
-
-matrix.sps:72.18-72.19: error: MATRIX: Expression for BY must evaluate to
-scalar, not a 0×0 matrix.
- 72 | LOOP i=5 TO 8 BY {}.
- | ^~
-
-matrix.sps:75.9-75.10: error: MATRIX: Expression for LOOP IF must evaluate to
-scalar, not a 0×0 matrix.
- 75 | LOOP IF {}.
- | ^~
-
-matrix.sps:79.13-79.14: error: MATRIX: Expression for END LOOP IF must evaluate
-to scalar, not a 0×0 matrix.
- 79 | END LOOP IF {}.
- | ^~
-
-matrix.sps:81.8-81.12: error: MATRIX: Expression for LOOP is outside the
-integer range.
- 81 | LOOP i=1e100 to 1e200.
- | ^~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - BREAK outside LOOP])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-BREAK.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.1-2.5: error: BREAK: BREAK not inside LOOP.
- 2 | BREAK.
- | ^~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - READ])
-AT_DATA([matrix.txt], [dnl
-9
-8
-7
-6
-1 2 3
-4 5 6
-7 8 9
-10 11 12,13
-14, 15 ,16 , 17
-18
-19
-20 21 22 23
- 12 34
-5 6
- 78 89
-10 11
-$1 $2 3
-4 $5 6
-$1 $2 $3 4
- $5$6 $78
-1% 2% 3% 4
- 56% 7%8
-abcdefghijkl
-ABCDEFGHIJKL
-])
-AT_DATA([matrix2.txt], [dnl
-2, 3, 5, 7
-11, 13, 17, 19
-23, 29, 31, 37
-41, 43, 47, 53
-])
-AT_DATA([matrix3.txt], [dnl
-1 5
-3 1 2 3
-5 6 -1 2 5 1
-2 8 9
-3 1 3 2
-])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-READ x/FILE='matrix.txt'/SIZE=4/FIELD=1 TO 1.
-PRINT x.
-READ x/FILE='matrix.txt'/SIZE={3,3}/FIELD=1 TO 80.
-PRINT x.
-READ x/SIZE={2,4}/FIELD=1 TO 80.
-PRINT x.
-READ x(:,2)/FILE='matrix.txt'/FIELD=1 TO 80.
-PRINT x.
-READ x(1,:)/SIZE={1,4}/FIELD=1 TO 80.
-PRINT x.
-
-READ x/SIZE={2,6}/FIELD=1 TO 20 BY 5.
-PRINT x.
-READ x/SIZE={2,3}/FIELD=1 TO 20/FORMAT=DOLLAR.
-PRINT x.
-READ x/SIZE={2,4}/FIELD=1 TO 20/FORMAT=DOLLAR5.1.
-PRINT x.
-READ x/SIZE={2,4}/FIELD=1 TO 12/FORMAT='4PCT'.
-PRINT x.
-READ x/SIZE={2,4}/FIELD=1 TO 12/FORMAT='4A'.
-PRINT x/FORMAT=A3.
-
-COMPUTE y={}.
-LOOP IF NOT EOF('matrix2.txt').
-READ x/FILE='matrix2.txt'/SIZE={1,4}/FIELD=1 TO 80.
-COMPUTE y={y; x}.
-END LOOP.
-PRINT y.
-
-COMPUTE m = MAKE(5, 5, 0).
-LOOP i = 1 TO 5.
-READ count /FILE='matrix3.txt' /FIELD=1 TO 1 /SIZE=1.
-READ m(i, 1:count) /FIELD=3 TO 100 /REREAD.
-END LOOP.
-PRINT m.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [0], [dnl
-x
- 9
- 8
- 7
- 6
-
-x
- 1 2 3
- 4 5 6
- 7 8 9
-
-x
- 10 11 12 13
- 14 15 16 17
-
-x
- 10 18 12 13
- 14 19 16 17
-
-x
- 20 21 22 23
- 14 19 16 17
-
-x
- 1 2 3 4 5 6
- 7 8 8 9 10 11
-
-x
- 1 2 3
- 4 5 6
-
-x
- 1 2 3 4
- 5 6 7 8
-
-x
- 1 2 3 4
- 5 6 7 8
-
-x
- abc def ghi jkl
- ABC DEF GHI JKL
-
-y
- 2 3 5 7
- 11 13 17 19
- 23 29 31 37
- 41 43 47 53
-
-m
- 5 0 0 0 0
- 1 2 3 0 0
- 6 -1 2 5 1
- 8 9 0 0 0
- 1 3 2 0 0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - READ - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-READ !.
-READ x/FILE=!.
-READ x/ENCODING=!.
-READ x/FIELD=!.
-READ x/FIELD=1 !.
-READ x/FIELD=1 TO !.
-READ x/FIELD=1 TO 0.
-READ x/FIELD=1 TO 10 BY !.
-READ x/FIELD=1 TO 10 BY 6.
-READ x/SIZE=!.
-READ x/MODE=!.
-READ x/FORMAT=!.
-READ x/FORMAT=F8.2/FORMAT=F8.2.
-READ x/FORMAT='5XYZZY'.
-READ x/FORMAT=XYZZY.
-READ x/!.
-READ x.
-READ x/FIELD=1 TO 10.
-READ x/FIELD=1 TO 10/SIZE={1,2}.
-READ x/FIELD=1 TO 10/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='15F'.
-READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT=F5.
-READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
-READ x/FIELD=1 TO 10/SIZE={1,2;3,4}/FILE='matrix.txt'.
-READ x/FIELD=1 TO 10/SIZE={1,2,3}/FILE='matrix.txt'.
-READ x/FIELD=1 TO 10/SIZE={-1}/FILE='matrix.txt'.
-COMPUTE x={1,2,3}.
-READ x(:,:)/FIELD=1 TO 10/SIZE={2,2}/FILE='matrix.txt'.
-READ x/FIELD=1 TO 10/SIZE={1,3}/FILE='matrix.txt'/MODE=SYMMETRIC.
-READ x/FIELD=1 TO 10/SIZE=2/FILE='matrix.txt'.
-END MATRIX.
-])
-AT_DATA([matrix.txt], [dnl
-xyzzy
-.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.6: error: READ: Syntax error expecting identifier.
- 2 | READ !.
- | ^
-
-matrix.sps:3.13: error: READ: Syntax error expecting a file name or handle
-name.
- 3 | READ x/FILE=!.
- | ^
-
-matrix.sps:4.17: error: READ: Syntax error expecting string.
- 4 | READ x/ENCODING=!.
- | ^
-
-matrix.sps:5.14: error: READ: Syntax error expecting positive integer for
-FIELD.
- 5 | READ x/FIELD=!.
- | ^
-
-matrix.sps:6.16: error: READ: Syntax error expecting `TO'.
- 6 | READ x/FIELD=1 !.
- | ^
-
-matrix.sps:7.19: error: READ: Syntax error expecting positive integer for TO.
- 7 | READ x/FIELD=1 TO !.
- | ^
-
-matrix.sps:8.19: error: READ: Syntax error expecting positive integer for TO.
- 8 | READ x/FIELD=1 TO 0.
- | ^
-
-matrix.sps:9.25: error: READ: Syntax error expecting integer between 1 and 10
-for BY.
- 9 | READ x/FIELD=1 TO 10 BY !.
- | ^
-
-matrix.sps:10.14-10.25: error: READ: Field width 6 does not evenly divide
-record width 10.
- 10 | READ x/FIELD=1 TO 10 BY 6.
- | ^~~~~~~~~~~~
-
-matrix.sps:10.14-10.20: note: READ: This syntax designates the record width.
- 10 | READ x/FIELD=1 TO 10 BY 6.
- | ^~~~~~~
-
-matrix.sps:10.25: note: READ: This syntax specifies the field width.
- 10 | READ x/FIELD=1 TO 10 BY 6.
- | ^
-
-matrix.sps:11.13: error: READ: Syntax error expecting matrix expression.
- 11 | READ x/SIZE=!.
- | ^
-
-matrix.sps:12.13: error: READ: Syntax error expecting RECTANGULAR or SYMMETRIC.
- 12 | READ x/MODE=!.
- | ^
-
-matrix.sps:13.15: error: READ: Syntax error expecting identifier.
- 13 | READ x/FORMAT=!.
- | ^
-
-matrix.sps:14.20-14.25: error: READ: Subcommand FORMAT may only be specified
-once.
- 14 | READ x/FORMAT=F8.2/FORMAT=F8.2.
- | ^~~~~~
-
-matrix.sps:15.15-15.22: error: READ: Unknown format XYZZY.
- 15 | READ x/FORMAT='5XYZZY'.
- | ^~~~~~~~
-
-matrix.sps:16.15-16.19: error: READ: Unknown format type `XYZZY'.
- 16 | READ x/FORMAT=XYZZY.
- | ^~~~~
-
-matrix.sps:17.8: error: READ: Syntax error expecting FILE, FIELD, MODE, REREAD,
-or FORMAT.
- 17 | READ x/!.
- | ^
-
-matrix.sps:18.1-18.7: error: READ: Required subcommand FIELD was not specified.
- 18 | READ x.
- | ^~~~~~~
-
-matrix.sps:19: error: READ: SIZE is required for reading data into a full
-matrix (as opposed to a submatrix).
-
-matrix.sps:19.6: note: READ: This expression designates a full matrix.
- 19 | READ x/FIELD=1 TO 10.
- | ^
-
-matrix.sps:20.1-20.32: error: READ: Required subcommand FILE was not specified.
- 20 | READ x/FIELD=1 TO 10/SIZE={1,2}.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-matrix.sps:21: error: READ: 15 repetitions cannot fit in record width 10.
-
-matrix.sps:21.57-21.61: note: READ: This syntax designates the number of
-repetitions.
- 21 | READ x/FIELD=1 TO 10/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='15F'.
- | ^~~~~
-
-matrix.sps:21.14-21.20: note: READ: This syntax designates the record width.
- 21 | READ x/FIELD=1 TO 10/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='15F'.
- | ^~~~~~~
-
-matrix.sps:22: error: READ: This command specifies two different field widths.
-
-matrix.sps:22.62-22.63: note: READ: This syntax specifies field width 5.
- 22 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT=F5.
- | ^~
-
-matrix.sps:22.25: note: READ: This syntax specifies field width 2.
- 22 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT=F5.
- | ^
-
-matrix.sps:23: error: READ: This command specifies two different field widths.
-
-matrix.sps:23.62-23.65: note: READ: This syntax specifies 2 repetitions.
- 23 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
- | ^~~~
-
-matrix.sps:23.14-23.20: note: READ: This syntax designates record width 10,
-which divided by 2 repetitions implies field width 5.
- 23 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
- | ^~~~~~~
-
-matrix.sps:23.25: note: READ: This syntax specifies field width 2.
- 23 | READ x/FIELD=1 TO 10 BY 2/SIZE={1,2}/FILE='xyzzy.txt'/FORMAT='2F'.
- | ^
-
-matrix.sps:24.27-24.35: error: MATRIX: SIZE must evaluate to a scalar or a 2-
-element vector, not a 2×2 matrix.
- 24 | READ x/FIELD=1 TO 10/SIZE={1,2;3,4}/FILE='matrix.txt'.
- | ^~~~~~~~~
-
-matrix.sps:25.27-25.33: error: MATRIX: SIZE must evaluate to a scalar or a 2-
-element vector, not a 1×3 matrix.
- 25 | READ x/FIELD=1 TO 10/SIZE={1,2,3}/FILE='matrix.txt'.
- | ^~~~~~~
-
-matrix.sps:26.28-26.29: error: MATRIX: Matrix dimensions -1×1 specified on SIZE
-are outside valid range.
- 26 | READ x/FIELD=1 TO 10/SIZE={-1}/FILE='matrix.txt'.
- | ^~
-
-matrix.sps:28: error: MATRIX: Dimensions specified on SIZE differ from
-dimensions of destination submatrix.
-
-matrix.sps:28.32-28.36: note: MATRIX: SIZE specifies dimensions 2×2.
- 28 | READ x(:,:)/FIELD=1 TO 10/SIZE={2,2}/FILE='matrix.txt'.
- | ^~~~~
-
-matrix.sps:28.6-28.11: note: MATRIX: Destination submatrix has dimensions 1×3.
- 28 | READ x(:,:)/FIELD=1 TO 10/SIZE={2,2}/FILE='matrix.txt'.
- | ^~~~~~
-
-matrix.sps:29: error: MATRIX: Cannot read non-square 1×3 matrix using READ with
-MODE=SYMMETRIC.
-
-matrix.txt:1.1-1.5: warning: Error reading "xyzzy" as format F for matrix row
-1, column 1: Field contents are not numeric.
-
-matrix.txt:2.1: warning: Error reading "." as format F for matrix row 2, column
-1: Matrix data may not contain missing value.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - WRITE])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-WRITE {1.5, 2; 3, 4.12345}/OUTFILE='matrix.txt'/FIELD=1 TO 80.
-WRITE {1.5, 2; 3, 4.12345}/OUTFILE='matrix.txt'/FIELD=1 TO 5.
-WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 80 BY 5.
-WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=F8.2.
-WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=E.
-WRITE {1, 2; 3, 4}/OUTFILE='matrix.txt'/FIELD=1 TO 10 BY 10/FORMAT=E.
-WRITE "abcdefhi"/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=A8.
-WRITE "abcdefhi"/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=A4.
-WRITE "abcdefhi"/OUTFILE='matrix.txt'/FIELD=1 TO 80/FORMAT=AHEX12.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps])
-AT_CHECK([cat matrix.txt], [0], [dnl
- 1.5 2
- 3 4.12345
- 1.5 2
- 3
- 4.12345
- 1 2
- 3 4
- 1.00 2.00
- 3.00 4.00
- 1 2
- 3 4
- 1.E+000
- 2.E+000
- 3.E+000
- 4.E+000
- abcdefhi
- abcd
- 616263646566
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - WRITE - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-WRITE !.
-WRITE 1/OUTFILE=!.
-WRITE 1/ENCODING=!.
-WRITE 1/FIELD=!.
-WRITE 1/FIELD=1 !.
-WRITE 1/FIELD=1 TO 0.
-WRITE 1/FIELD=1 TO 10 BY 20.
-WRITE 1/FIELD=1 TO 10 BY 6.
-WRITE 1/MODE=TRAPEZOIDAL.
-WRITE 1/FORMAT=F5/FORMAT=F5.
-WRITE 1/FORMAT='5ASDF'.
-WRITE 1/FORMAT=ASDF5.
-WRITE 1/!.
-WRITE 1.
-WRITE 1/FIELD=1 TO 10.
-WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT='15F'.
-WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
-WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT=E.
-WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT=A9.
-WRITE {1,2}/FIELD=1 TO 10/OUTFILE='matrix.txt'/MODE=TRIANGULAR.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.7: error: WRITE: Syntax error expecting matrix expression.
- 2 | WRITE !.
- | ^
-
-matrix.sps:3.17: error: WRITE: Syntax error expecting a file name or handle
-name.
- 3 | WRITE 1/OUTFILE=!.
- | ^
-
-matrix.sps:4.18: error: WRITE: Syntax error expecting string.
- 4 | WRITE 1/ENCODING=!.
- | ^
-
-matrix.sps:5.15: error: WRITE: Syntax error expecting positive integer for
-FIELD.
- 5 | WRITE 1/FIELD=!.
- | ^
-
-matrix.sps:6.17: error: WRITE: Syntax error expecting `TO'.
- 6 | WRITE 1/FIELD=1 !.
- | ^
-
-matrix.sps:7.20: error: WRITE: Syntax error expecting positive integer for TO.
- 7 | WRITE 1/FIELD=1 TO 0.
- | ^
-
-matrix.sps:8.26-8.27: error: WRITE: Syntax error expecting integer between 1
-and 10 for BY.
- 8 | WRITE 1/FIELD=1 TO 10 BY 20.
- | ^~
-
-matrix.sps:9.15-9.26: error: WRITE: Field width 6 does not evenly divide record
-width 10.
- 9 | WRITE 1/FIELD=1 TO 10 BY 6.
- | ^~~~~~~~~~~~
-
-matrix.sps:9.15-9.21: note: WRITE: This syntax designates the record width.
- 9 | WRITE 1/FIELD=1 TO 10 BY 6.
- | ^~~~~~~
-
-matrix.sps:9.26: note: WRITE: This syntax specifies the field width.
- 9 | WRITE 1/FIELD=1 TO 10 BY 6.
- | ^
-
-matrix.sps:10.14-10.24: error: WRITE: Syntax error expecting RECTANGULAR or
-TRIANGULAR.
- 10 | WRITE 1/MODE=TRAPEZOIDAL.
- | ^~~~~~~~~~~
-
-matrix.sps:11.19-11.24: error: WRITE: Subcommand FORMAT may only be specified
-once.
- 11 | WRITE 1/FORMAT=F5/FORMAT=F5.
- | ^~~~~~
-
-matrix.sps:12.16-12.22: error: WRITE: Unknown format ASDF.
- 12 | WRITE 1/FORMAT='5ASDF'.
- | ^~~~~~~
-
-matrix.sps:13.16-13.20: error: WRITE: Unknown format type `ASDF'.
- 13 | WRITE 1/FORMAT=ASDF5.
- | ^~~~~
-
-matrix.sps:14.9: error: WRITE: Syntax error expecting OUTFILE, FIELD, MODE,
-HOLD, or FORMAT.
- 14 | WRITE 1/!.
- | ^
-
-matrix.sps:15.1-15.8: error: WRITE: Required subcommand FIELD was not
-specified.
- 15 | WRITE 1.
- | ^~~~~~~~
-
-matrix.sps:16.1-16.22: error: WRITE: Required subcommand OUTFILE was not
-specified.
- 16 | WRITE 1/FIELD=1 TO 10.
- | ^~~~~~~~~~~~~~~~~~~~~~
-
-matrix.sps:17.51-17.55: note: WRITE: This syntax designates the number of
-repetitions.
- 17 | WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT='15F'.
- | ^~~~~
-
-matrix.sps:17.15-17.21: note: WRITE: This syntax designates the record width.
- 17 | WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT='15F'.
- | ^~~~~~~
-
-matrix.sps:18: error: WRITE: This command specifies two different field widths.
-
-matrix.sps:18.56-18.59: note: WRITE: This syntax specifies 5 repetitions.
- 18 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
- | ^~~~
-
-matrix.sps:18.15-18.21: note: WRITE: This syntax designates record width 10,
-which divided by 5 repetitions implies field width 2.
- 18 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
- | ^~~~~~~
-
-matrix.sps:18.26: note: WRITE: This syntax specifies field width 5.
- 18 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT='5F'.
- | ^
-
-matrix.sps:19: error: WRITE: Output format E5.0 specifies width 5, but E
-requires a width between 6 and 40.
-
-matrix.sps:19.56: note: WRITE: This syntax specifies format E.
- 19 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT=E.
- | ^
-
-matrix.sps:19.26: note: WRITE: This syntax specifies field width 5.
- 19 | WRITE 1/FIELD=1 TO 10 BY 5/OUTFILE='matrix.txt'/FORMAT=E.
- | ^
-
-matrix.sps:20.51-20.52: error: WRITE: Format A9 is too wide for 8-byte matrix
-elements.
- 20 | WRITE 1/FIELD=1 TO 10/OUTFILE='matrix.txt'/FORMAT=A9.
- | ^~
-
-matrix.sps:21.7-21.11: error: MATRIX: WRITE with MODE=TRIANGULAR requires a
-square matrix but the matrix to be written has dimensions 1×2.
- 21 | WRITE {1,2}/FIELD=1 TO 10/OUTFILE='matrix.txt'/MODE=TRIANGULAR.
- | ^~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - GET])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /a b c.
-MISSING VALUES a(1) b(5).
-BEGIN DATA.
-0 0 0
-1 2 3
-4 5 6
-7 8 .
-END DATA.
-
-MATRIX.
-GET x0 /NAMES=names0.
-PRINT x0.
-PRINT names0/FORMAT=A8.
-END MATRIX.
-
-MATRIX.
-GET x1 /VARIABLES=a b c /NAMES=names1 /MISSING=OMIT.
-PRINT x1.
-PRINT names1/FORMAT=A8.
-END MATRIX.
-
-MATRIX.
-GET x2 /VARIABLES=a b /NAMES=names2 /MISSING=OMIT.
-PRINT x2.
-PRINT names2/FORMAT=A8.
-END MATRIX.
-
-MATRIX.
-GET x3 /FILE=* /VARIABLES=a b c /NAMES=names3 /MISSING=5.
-PRINT x3.
-PRINT names3/FORMAT=A8.
-END MATRIX.
-
-MATRIX.
-GET x4 /FILE=* /VARIABLES=a b /NAMES=names4 /MISSING=5.
-PRINT x4.
-PRINT names4/FORMAT=A8.
-END MATRIX.
-
-SAVE OUTFILE='matrix.sav'.
-NEW FILE.
-
-MATRIX.
-GET x5 /FILE='matrix.sav' /VARIABLES=a b c /NAMES=names5 /MISSING=ACCEPT.
-PRINT x5.
-PRINT names5/FORMAT=A8.
-END MATRIX.
-
-MATRIX.
-GET x6 /FILE='matrix.sav' /VARIABLES=a b c /NAMES=names6 /MISSING=ACCEPT /SYSMIS=9.
-PRINT x6.
-PRINT names6/FORMAT=A8.
-END MATRIX.
-
-MATRIX.
-GET x7 /FILE='matrix.sav' /VARIABLES=a b c /NAMES=names7 /MISSING=ACCEPT /SYSMIS=OMIT.
-PRINT x7.
-PRINT names7/FORMAT=A8.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:11: error: MATRIX: Variable a in case 2 has user-missing value 1.
-
-matrix.sps:12.7-12.8: error: MATRIX: Uninitialized variable x0 used in
-expression.
- 12 | PRINT x0.
- | ^~
-
-names0
- a
- b
- c
-
-matrix.sps:17: error: MATRIX: Variable c in case 4 is system-missing.
-
-matrix.sps:18.7-18.8: error: MATRIX: Uninitialized variable x1 used in
-expression.
- 18 | PRINT x1.
- | ^~
-
-names1
- a
- b
- c
-
-x2
- 0 0
- 7 8
-
-names2
- a
- b
-
-matrix.sps:29: error: MATRIX: Variable c in case 4 is system-missing.
-
-matrix.sps:30.7-30.8: error: MATRIX: Uninitialized variable x3 used in
-expression.
- 30 | PRINT x3.
- | ^~
-
-names3
- a
- b
- c
-
-x4
- 0 0
- 5 2
- 4 5
- 7 8
-
-names4
- a
- b
-
-matrix.sps:44: error: MATRIX: Variable c in case 4 is system-missing.
-
-matrix.sps:45.7-45.8: error: MATRIX: Uninitialized variable x5 used in
-expression.
- 45 | PRINT x5.
- | ^~
-
-names5
- a
- b
- c
-
-x6
- 0 0 0
- 1 2 3
- 4 5 6
- 7 8 9
-
-names6
- a
- b
- c
-
-x7
- 0 0 0
- 1 2 3
- 4 5 6
-
-names7
- a
- b
- c
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - GET - negative])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /a b c * d(a1).
-MISSING VALUES a(1) b(5).
-BEGIN DATA.
-0 0 0 a
-1 2 3 b
-4 5 6 b
-7 8 . d
-END DATA.
-SAVE OUTFILE='matrix.sav'.
-
-MATRIX.
-GET !.
-GET x/VARIABLES=!.
-GET x/FILE=!.
-GET x/ENCODING=!.
-GET x/NAMES=!.
-GET x/MISSING=!.
-GET x/SYSMIS=!.
-GET x/!.
-GET x/VARIABLES=!.
-GET x/VARIABLES=x TO !.
-GET x/VARIABLES=x.
-GET x/VARIABLES=c TO a.
-GET x/VARIABLES=d.
-GET x.
-END MATRIX.
-
-NEW FILE.
-MATRIX.
-GET x/VARIABLES=a.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:12.5: error: GET: Syntax error expecting identifier.
- 12 | GET !.
- | ^
-
-matrix.sps:13.17: error: GET: Syntax error expecting variable name.
- 13 | GET x/VARIABLES=!.
- | ^
-
-matrix.sps:14.12: error: GET: Syntax error expecting a file name or handle
-name.
- 14 | GET x/FILE=!.
- | ^
-
-matrix.sps:15.16: error: GET: Syntax error expecting string.
- 15 | GET x/ENCODING=!.
- | ^
-
-matrix.sps:16.13: error: GET: Syntax error expecting identifier.
- 16 | GET x/NAMES=!.
- | ^
-
-matrix.sps:17.15: error: GET: Syntax error expecting ACCEPT or OMIT or a number
-for MISSING.
- 17 | GET x/MISSING=!.
- | ^
-
-matrix.sps:18.14: error: GET: Syntax error expecting OMIT or a number for
-SYSMIS.
- 18 | GET x/SYSMIS=!.
- | ^
-
-matrix.sps:19.7: error: GET: Syntax error expecting FILE, VARIABLES, NAMES,
-MISSING, or SYSMIS.
- 19 | GET x/!.
- | ^
-
-matrix.sps:20.17: error: GET: Syntax error expecting variable name.
- 20 | GET x/VARIABLES=!.
- | ^
-
-matrix.sps:21.22: error: GET: Syntax error expecting variable name.
- 21 | GET x/VARIABLES=x TO !.
- | ^
-
-matrix.sps:22.17: error: MATRIX: x is not a variable name.
- 22 | GET x/VARIABLES=x.
- | ^
-
-matrix.sps:23.17-23.22: error: MATRIX: c TO a is not valid syntax since c
-precedes a in the dictionary.
- 23 | GET x/VARIABLES=c TO a.
- | ^~~~~~
-
-matrix.sps:24.17: error: MATRIX: d is not a numeric variable.
- 24 | GET x/VARIABLES=d.
- | ^
-
-matrix.sps:25: error: MATRIX: Variable d is not numeric.
-
-matrix.sps:30: error: MATRIX: The GET command cannot read an empty active file.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - SAVE])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-SAVE {1,2,3; 4,5,6}/OUTFILE='matrix.sav'.
-SAVE {7,8,9}/VARIABLES=a b c d.
-
-SAVE {1,2,3}/OUTFILE='matrix2.sav'/VARIABLES=v01 TO v03.
-SAVE {4,5,6}/NAMES={'x', 'y', 'z', 'w'}.
-
-SAVE {1,'abcd',3}/OUTFILE='matrix3.sav'/NAMES={'a', 'b', 'c'}/STRINGS=b.
-SAVE {4,'xyzw',6}/STRINGS=a, b.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps])
-AT_CHECK([pspp-convert matrix.sav matrix.csv && cat matrix.csv], [0], [dnl
-COL1,COL2,COL3
-1,2,3
-4,5,6
-7,8,9
-])
-AT_CHECK([pspp-convert matrix2.sav matrix2.csv && cat matrix2.csv], [0], [dnl
-v01,v02,v03
-1,2,3
-4,5,6
-])
-AT_CHECK([pspp-convert matrix3.sav matrix3.csv && cat matrix3.csv], [0], [dnl
-a,b,c
-1,abcd,3
-4,xyzw,6
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - SAVE - inline])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-SAVE {1,2,3; 4,5,6}/OUTFILE=*.
-SAVE {7,8,9}/VARIABLES=a b c d.
-END MATRIX.
-LIST.
-
-MATRIX.
-SAVE {1,2,3}/OUTFILE=*/VARIABLES=v01 TO v03.
-SAVE {4,5,6}/NAMES={'x', 'y', 'z', 'w'}.
-END MATRIX.
-LIST.
-
-MATRIX.
-SAVE {1,'abcd',3}/OUTFILE=*/NAMES={'a', 'b', 'c'}/STRINGS=b.
-SAVE {4,'xyzw',6}/STRINGS=a, b.
-END MATRIX.
-LIST.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Data List
-COL1,COL2,COL3
-1.00,2.00,3.00
-4.00,5.00,6.00
-7.00,8.00,9.00
-
-Table: Data List
-v01,v02,v03
-1.00,2.00,3.00
-4.00,5.00,6.00
-
-Table: Data List
-a,b,c
-1.00,abcd,3.00
-4.00,xyzw,6.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - SAVE - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-SAVE !.
-SAVE 1/OUTFILE=!.
-SAVE 1/VARIABLES=!.
-SAVE 1/NAMES=!.
-SAVE 1/!.
-SAVE 1.
-SAVE 1/OUTFILE='matrix.sav'/NAMES={'a'}/VARIABLES=a.
-SAVE 1/OUTFILE='matrix2.sav'.
-SAVE {1,2}/OUTFILE='matrix2.sav'.
-SAVE {1,2}/OUTFILE='matrix3.sav'/NAMES={'a', 'a'}.
-SAVE {1,2}/OUTFILE='matrix4.sav'/STRINGS=a.
-SAVE {1,2}/OUTFILE='matrix5.sav'/STRINGS=a, b.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.6: error: SAVE: Syntax error expecting matrix expression.
- 2 | SAVE !.
- | ^
-
-matrix.sps:3.16: error: SAVE: Syntax error expecting a file name or handle
-name.
- 3 | SAVE 1/OUTFILE=!.
- | ^
-
-matrix.sps:4.18: error: SAVE: Syntax error expecting variable name.
- 4 | SAVE 1/VARIABLES=!.
- | ^
-
-matrix.sps:5.14: error: SAVE: Syntax error expecting matrix expression.
- 5 | SAVE 1/NAMES=!.
- | ^
-
-matrix.sps:6.8: error: SAVE: Syntax error expecting OUTFILE, VARIABLES, NAMES,
-or STRINGS.
- 6 | SAVE 1/!.
- | ^
-
-matrix.sps:7.1-7.7: error: SAVE: Required subcommand OUTFILE was not specified.
- 7 | SAVE 1.
- | ^~~~~~~
-
-matrix.sps:8.35-8.39: warning: SAVE: Ignoring NAMES because VARIABLES was also
-specified.
- 8 | SAVE 1/OUTFILE='matrix.sav'/NAMES={'a'}/VARIABLES=a.
- | ^~~~~
-
-matrix.sps:10: error: MATRIX: Cannot save 1×2 matrix to `matrix2.sav' because
-the first SAVE to `matrix2.sav' in this matrix program wrote a 1-column matrix.
-
-matrix.sps:9: error: MATRIX: This is the location of the first SAVE to
-`matrix2.sav'.
-
-error: Duplicate variable name a in SAVE statement.
-
-error: The SAVE command STRINGS subcommand specifies an unknown variable a.
-
-error: The SAVE command STRINGS subcommand specifies 2 unknown variables,
-including a.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET])
-AT_DATA([matrix.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ var01 TO var08.
-BEGIN DATA.
-MEAN 24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
-SD 5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
-N 92 92 92 92 92 92 92 92
-CORR 1.00
-CORR .18 1.00
-CORR -.22 -.17 1.00
-CORR .36 .31 -.14 1.00
-CORR .27 .16 -.12 .22 1.00
-CORR .33 .15 -.17 .24 .21 1.00
-CORR .50 .29 -.20 .32 .12 .38 1.00
-CORR .17 .29 -.05 .20 .27 .20 .04 1.00
-END DATA.
-
-MATRIX.
-MGET.
-PRINT MN/FORMAT=F5.1.
-PRINT SD/FORMAT=F5.1.
-PRINT NC/FORMAT=F5.0.
-PRINT CR/FORMAT=F5.2.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Matrix Variables Created by MGET
-,Dimensions,
-,Rows,Columns
-MN,1,8
-SD,1,8
-NC,1,8
-CR,8,8
-
-MN
-24.3 5.4 69.7 20.1 13.4 2.7 27.9 3.7
-
-SD
-5.7 1.5 23.5 5.8 2.8 4.5 5.4 1.5
-
-NC
-92 92 92 92 92 92 92 92
-
-CR
-1.00 .18 -.22 .36 .27 .33 .50 .17
-.18 1.00 -.17 .31 .16 .15 .29 .29
--.22 -.17 1.00 -.14 -.12 -.17 -.20 -.05
-.36 .31 -.14 1.00 .22 .24 .32 .20
-.27 .16 -.12 .22 1.00 .21 .12 .27
-.33 .15 -.17 .24 .21 1.00 .38 .20
-.50 .29 -.20 .32 .12 .38 1.00 .04
-.17 .29 -.05 .20 .27 .20 .04 1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET with split variables])
-AT_DATA([matrix.sps], [dnl
-matrix data
- variables = s1 s2 rowtype_ var01 var02 var03
- /split=s1 s2.
-
-begin data
-8 0 mean 21.4 5.0 72.9
-8 0 sd 6.5 1.6 22.8
-8 0 n 106 106 106
-8 0 corr 1
-8 0 corr .41 1
-8 0 corr -.16 -.22 1
-8 1 mean 11.4 1.0 52.9
-8 1 sd 9.5 8.6 12.8
-8 1 n 10 11 12
-8 1 corr 1
-8 1 corr .51 1
-8 1 corr .36 -.41 1
-end data.
-
-MATRIX.
-MGET.
-PRINT MNS1/FORMAT=F5.1.
-PRINT SDS1/FORMAT=F5.1.
-PRINT NCS1/FORMAT=F5.0.
-PRINT CRS1/FORMAT=F5.2.
-PRINT MNS2/FORMAT=F5.1.
-PRINT SDS2/FORMAT=F5.1.
-PRINT NCS2/FORMAT=F5.0.
-PRINT CRS2/FORMAT=F5.2.
-END MATRIX.
-])
-AT_CHECK([pspp -O format=csv matrix.sps], [0], [dnl
-Table: Matrix Variables Created by MGET
-,Split Values,,Dimensions,
-,s1,s2,Rows,Columns
-MNS1,8,0,1,3
-SDS1,8,0,1,3
-NCS1,8,0,1,3
-CRS1,8,0,3,3
-MNS2,8,1,1,3
-SDS2,8,1,1,3
-NCS2,8,1,1,3
-CRS2,8,1,3,3
-
-MNS1
-21.4 5.0 72.9
-
-SDS1
-6.5 1.6 22.8
-
-NCS1
-106 106 106
-
-CRS1
-1.00 .41 -.16
-.41 1.00 -.22
--.16 -.22 1.00
-
-MNS2
-11.4 1.0 52.9
-
-SDS2
-9.5 8.6 12.8
-
-NCS2
-10 11 12
-
-CRS2
-1.00 .51 .36
-.51 1.00 -.41
-.36 -.41 1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET with factor variables])
-AT_DATA([matrix.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ f1 var01 TO var04
- /FACTOR=f1.
-BEGIN DATA.
-MEAN 0 34 35 36 37
-SD 0 22 11 55 66
-N 0 99 98 99 92
-MEAN 1 44 45 34 39
-SD 1 23 15 51 46
-N 1 98 34 87 23
-CORR . 1
-CORR . .9 1
-CORR . .8 .6 1
-CORR . .7 .5 .4 1
-END DATA.
-FORMATS var01 TO var04(F5.1).
-SAVE OUTFILE='matrix.sav'.
-])
-AT_DATA([matrix2.sps], [dnl
-MATRIX.
-MGET FILE='matrix.sav'.
-PRINT MNF1/FORMAT=F2.0.
-PRINT SDF1/FORMAT=F2.0.
-PRINT NCF1/FORMAT=F2.0.
-PRINT MNF2/FORMAT=F2.0.
-PRINT SDF2/FORMAT=F2.0.
-PRINT NCF2/FORMAT=F2.0.
-PRINT CR/FORMAT=F3.1.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps])
-AT_CHECK([pspp -O format=csv matrix2.sps], [0], [dnl
-Table: Matrix Variables Created by MGET
-,Factors,Dimensions,
-,f1,Rows,Columns
-MNF1,0,1,4
-SDF1,0,1,4
-NCF1,0,1,4
-MNF2,1,1,4
-SDF2,1,1,4
-NCF2,1,1,4
-CR,.,4,4
-
-MNF1
-34 35 36 37
-
-SDF1
-22 11 55 66
-
-NCF1
-99 98 99 92
-
-MNF2
-44 45 34 39
-
-SDF2
-23 15 51 46
-
-NCF2
-98 34 87 23
-
-CR
-1.0 .9 .8 .7
-.9 1.0 .6 .5
-.8 .6 1.0 .4
-.7 .5 .4 1.0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET with factor and split variables])
-AT_DATA([matrix.sps], [dnl
-matrix data
- variables = s f rowtype_ var01 var02 var03
- /split=s
- /factor=f.
-
-begin data
-8 0 mean 21.4 5.0 72.9
-8 0 sd 6.5 1.6 22.8
-8 0 n 106 106 106
-8 . corr 1
-8 . corr .41 1
-8 . corr -.16 -.22 1
-9 1 mean 11.4 1.0 52.9
-9 1 sd 9.5 8.6 12.8
-9 1 n 10 11 12
-9 . corr 1
-9 . corr .51 1
-9 . corr .36 -.41 1
-end data.
-
-MATRIX.
-MGET.
-PRINT MNF1S1/FORMAT=F5.1.
-PRINT SDF1S1/FORMAT=F5.1.
-PRINT NCF1S1/FORMAT=F5.0.
-PRINT CRS1/FORMAT=F5.2.
-PRINT MNF1S2/FORMAT=F5.1.
-PRINT SDF1S2/FORMAT=F5.1.
-PRINT NCF1S2/FORMAT=F5.0.
-PRINT CRS2/FORMAT=F5.2.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Matrix Variables Created by MGET
-,Split Values,Factors,Dimensions,
-,s,f,Rows,Columns
-MNF1S1,8,0,1,3
-SDF1S1,8,0,1,3
-NCF1S1,8,0,1,3
-CRS1,8,.,3,3
-MNF1S2,9,1,1,3
-SDF1S2,9,1,1,3
-NCF1S2,9,1,1,3
-CRS2,9,.,3,3
-
-MNF1S1
-21.4 5.0 72.9
-
-SDF1S1
-6.5 1.6 22.8
-
-NCF1S1
-106 106 106
-
-CRS1
-1.00 .41 -.16
-.41 1.00 -.22
--.16 -.22 1.00
-
-MNF1S2
-11.4 1.0 52.9
-
-SDF1S2
-9.5 8.6 12.8
-
-NCF1S2
-10 11 12
-
-CRS2
-1.00 .51 .36
-.51 1.00 -.41
-.36 -.41 1.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET with TYPE])
-AT_DATA([matrix.sps], [dnl
-MATRIX DATA
- VARIABLES=ROWTYPE_ f1 var01 TO var04
- /FACTOR=f1.
-BEGIN DATA.
-MEAN 0 34 35 36 37
-SD 0 22 11 55 66
-N 0 99 98 99 92
-MEAN 1 44 45 34 39
-SD 1 23 15 51 46
-N 1 98 34 87 23
-CORR . 1
-CORR . .9 1
-CORR . .8 .6 1
-CORR . .7 .5 .4 1
-END DATA.
-FORMATS var01 TO var04(F5.1).
-SAVE OUTFILE='matrix.sav'.
-])
-AT_DATA([matrix2.sps], [dnl
-MATRIX.
-MGET/FILE='matrix.sav'/TYPE=CORR.
-PRINT CR/FORMAT=F3.1.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps])
-AT_CHECK([pspp -O format=csv matrix2.sps], [0], [dnl
-Table: Matrix Variables Created by MGET
-,Factors,Dimensions,
-,f1,Rows,Columns
-CR,.,4,4
-
-CR
-1.0 .9 .8 .7
-.9 1.0 .6 .5
-.8 .6 1.0 .4
-.7 .5 .4 1.0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - negative - parsing])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-MGET !.
-MGET FILE=!.
-MGET ENCODING=!.
-MGET TYPE=!.
-MGET TYPE=CORR !.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.6: error: MGET: Syntax error expecting FILE or TYPE.
- 2 | MGET !.
- | ^
-
-matrix.sps:3.11: error: MGET: Syntax error expecting a file name or handle
-name.
- 3 | MGET FILE=!.
- | ^
-
-matrix.sps:4.15: error: MGET: Syntax error expecting string.
- 4 | MGET ENCODING=!.
- | ^
-
-matrix.sps:5.11: error: MGET: Syntax error expecting COV, CORR, MEAN, STDDEV,
-N, or COUNT.
- 5 | MGET TYPE=!.
- | ^
-
-matrix.sps:6.16: error: MGET: Syntax error expecting COV, CORR, MEAN, STDDEV,
-N, or COUNT.
- 6 | MGET TYPE=CORR !.
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - missing VARNAME_ and ROWTYPE_])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-BEGIN DATA.
-1
-END DATA.
-
-MATRIX.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:7: error: MATRIX: Matrix data file lacks ROWTYPE_ variable.
-
-matrix.sps:7: error: MATRIX: Matrix data file lacks VARNAME_ variable.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - wrong format for VARNAME_ and ROWTYPE_])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /VARNAME_ * ROWTYPE_ (A7).
-BEGIN DATA.
-1 asdf
-END DATA.
-
-MATRIX.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:7: error: MATRIX: ROWTYPE_ variable in matrix data file must be 8-
-byte string, but it has width 7.
-
-matrix.sps:7: error: MATRIX: VARNAME_ variable in matrix data file must be 8-
-byte string, but it has width 0.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - wrong order for VARNAME_ and ROWTYPE_])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /VARNAME_ ROWTYPE_ (A8).
-BEGIN DATA.
-asdf jkl;
-END DATA.
-
-MATRIX.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:7: error: MATRIX: ROWTYPE_ must precede VARNAME_ in matrix data
-file.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - no continuous variables])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ (A8).
-BEGIN DATA.
-asdf jkl;
-END DATA.
-
-MATRIX.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:7: error: MATRIX: Matrix data file contains no continuous variables.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - unexpected string variables])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ cvar1 (A8).
-BEGIN DATA.
-asdf jkl; zxcv
-END DATA.
-
-MATRIX.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:7: error: MATRIX: Matrix data file contains unexpected string
-variable cvar1.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - unknown ROWTYPE_])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ (A8) cvar1.
-BEGIN DATA.
-asdf jkl; 1
-END DATA.
-
-MATRIX.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [1], [dnl
-"matrix.sps:7: error: MATRIX: Matrix data file contains unknown ROWTYPE_ ""asdf""."
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - duplicate matrix variable name])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /ROWTYPE_ VARNAME_ (A8) cvar1.
-BEGIN DATA.
-corr jkl; 1
-END DATA.
-
-MATRIX.
-MGET.
-MGET.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Matrix Variables Created by MGET
-,Dimensions,
-,Rows,Columns
-CR,1,1
-
-matrix.sps:8: warning: MATRIX: Matrix data file contains variable with existing name CR.
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MGET - missing values in input])
-AT_DATA([matrix.sps], [dnl
-DATA LIST LIST NOTABLE /s1 * ROWTYPE_ VARNAME_ (A8) cvar1 cvar2.
-BEGIN DATA.
-1 n "" 1 .
-2 n "" . .
-END DATA.
-
-MATRIX.
-MGET.
-PRINT ncs1/FORMAT=F5.
-PRINT ncs2/FORMAT=F5.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [1], [dnl
-"matrix.sps:8: error: MATRIX: Matrix data file variable NCS1 contains a missing value, which was treated as zero."
-
-"matrix.sps:8: error: MATRIX: Matrix data file variable NCS2 contains 2 missing values, which were treated as zero."
-
-Table: Matrix Variables Created by MGET
-,Split Values,Dimensions,
-,s1,Rows,Columns
-NCS1,1.00,1,2
-NCS2,2.00,1,2
-
-ncs1
-1 0
-
-ncs2
-0 0
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MSAVE])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-MSAVE {1, 2; 3, 4}/TYPE=CORR/VARIABLES=X,Y/OUTFILE='matrix.sav'.
-MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV/VARIABLES=X,Y.
-MSAVE {11, 12}/TYPE=MEAN.
-MSAVE {13, 14}/TYPE=STDDEV.
-MSAVE {15, 16}/TYPE=N.
-MSAVE {17, 18}/TYPE=COUNT.
-END MATRIX.
-GET 'matrix.sav'.
-LIST.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Data List
-ROWTYPE_,VARNAME_,X,Y
-CORR,X,1.00,2.00
-CORR,Y,3.00,4.00
-COV,X,5.00,6.00
-COV,Y,7.00,8.00
-COV,,9.00,10.00
-MEAN,,11.00,12.00
-STDDEV,,13.00,14.00
-N,,15.00,16.00
-COUNT,,17.00,18.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MSAVE with factor variables])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-MSAVE {1, 2; 3, 4}/TYPE=CORR/FACTOR={1,1}/FNAMES=X,Y/OUTFILE='matrix.sav'.
-MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV.
-MSAVE {11, 12}/TYPE=MEAN.
-MSAVE {13, 14}/FACTOR={2,1}/TYPE=STDDEV.
-MSAVE {15, 16}/TYPE=N.
-MSAVE {17, 18}/FACTOR={1,2}/TYPE=COUNT.
-END MATRIX.
-GET 'matrix.sav'.
-LIST.
-
-MATRIX.
-MSAVE {1, 2; 3, 4}/TYPE=CORR/FACTOR={5,6,7,8}/OUTFILE='matrix2.sav'.
-END MATRIX.
-GET 'matrix2.sav'.
-LIST.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Data List
-ROWTYPE_,X,Y,VARNAME_,COL1,COL2
-CORR,1.00,1.00,COL1,1.00,2.00
-CORR,1.00,1.00,COL2,3.00,4.00
-COV,1.00,1.00,COL1,5.00,6.00
-COV,1.00,1.00,COL2,7.00,8.00
-COV,1.00,1.00,,9.00,10.00
-MEAN,1.00,1.00,,11.00,12.00
-STDDEV,2.00,1.00,,13.00,14.00
-N,2.00,1.00,,15.00,16.00
-COUNT,1.00,2.00,,17.00,18.00
-
-Table: Data List
-ROWTYPE_,FAC1,FAC2,FAC3,FAC4,VARNAME_,COL1,COL2
-CORR,5.00,6.00,7.00,8.00,COL1,1.00,2.00
-CORR,5.00,6.00,7.00,8.00,COL2,3.00,4.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MSAVE with split variables])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-MSAVE {1, 2; 3, 4}/TYPE=CORR/SPLIT={1,1}/SNAMES=X,Y/OUTFILE='matrix.sav'.
-MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV.
-MSAVE {11, 12}/TYPE=MEAN.
-MSAVE {13, 14}/SPLIT={2,1}/TYPE=STDDEV.
-MSAVE {15, 16}/TYPE=N.
-MSAVE {17, 18}/SPLIT={1,2}/TYPE=COUNT.
-END MATRIX.
-GET 'matrix.sav'.
-LIST.
-
-MATRIX.
-MSAVE {1, 2; 3, 4}/TYPE=CORR/SPLIT={5,6,7,8}/OUTFILE='matrix2.sav'.
-END MATRIX.
-GET 'matrix2.sav'.
-LIST.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Data List
-X,Y,ROWTYPE_,VARNAME_,COL1,COL2
-1.00,1.00,CORR,COL1,1.00,2.00
-1.00,1.00,CORR,COL2,3.00,4.00
-1.00,1.00,COV,COL1,5.00,6.00
-1.00,1.00,COV,COL2,7.00,8.00
-1.00,1.00,COV,,9.00,10.00
-1.00,1.00,MEAN,,11.00,12.00
-2.00,1.00,STDDEV,,13.00,14.00
-2.00,1.00,N,,15.00,16.00
-1.00,2.00,COUNT,,17.00,18.00
-
-Table: Data List
-SPL1,SPL2,SPL3,SPL4,ROWTYPE_,VARNAME_,COL1,COL2
-5.00,6.00,7.00,8.00,CORR,COL1,1.00,2.00
-5.00,6.00,7.00,8.00,CORR,COL2,3.00,4.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MSAVE with factor and split variables])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-MSAVE {1, 2; 3, 4}/TYPE=CORR/SPLIT=1/FACTOR=1/OUTFILE='matrix.sav'.
-MSAVE {5, 6; 7, 8; 9, 10}/TYPE=COV.
-MSAVE {11, 12}/FACTOR=2/TYPE=MEAN.
-MSAVE {13, 14}/FACTOR=1/SPLIT=2/TYPE=STDDEV.
-MSAVE {15, 16}/TYPE=N.
-MSAVE {17, 18}/FACTOR=2/TYPE=COUNT.
-END MATRIX.
-GET 'matrix.sav'.
-LIST.
-])
-AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Data List
-SPL1,ROWTYPE_,FAC1,VARNAME_,COL1,COL2
-1.00,CORR,1.00,COL1,1.00,2.00
-1.00,CORR,1.00,COL2,3.00,4.00
-1.00,COV,1.00,COL1,5.00,6.00
-1.00,COV,1.00,COL2,7.00,8.00
-1.00,COV,1.00,,9.00,10.00
-1.00,MEAN,2.00,,11.00,12.00
-2.00,STDDEV,1.00,,13.00,14.00
-2.00,N,1.00,,15.00,16.00
-2.00,COUNT,2.00,,17.00,18.00
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - MSAVE - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-MSAVE !.
-MSAVE 1/TYPE=!.
-MSAVE 1/OUTFILE=!.
-MSAVE 1/VARIABLES=!.
-MSAVE 1/FNAMES=!.
-MSAVE 1/SNAMES=!.
-MSAVE 1/SPLIT=!.
-MSAVE 1/FACTOR=!.
-MSAVE 1/!.
-MSAVE 1.
-MSAVE 1/TYPE=COV/FNAMES=x.
-MSAVE 1/TYPE=COV/SNAMES=x.
-MSAVE 1/TYPE=COV.
-
-MSAVE 1/TYPE=COV/OUTFILE='matrix.sav'
- /FACTOR=1 /FNAMES=y
- /SPLIT=2 /SNAMES=z
- /VARIABLES=w.
-MSAVE 1/TYPE=COV/OUTFILE='matrix2.sav'.
-MSAVE 1/TYPE=COV/VARIABLES=x.
-MSAVE 1/TYPE=COV/FNAMES=x.
-MSAVE 1/TYPE=COV/SNAMES=x.
-END MATRIX.
-
-MATRIX.
-MSAVE 1/TYPE=COV/VARIABLES=x/OUTFILE='matrix3.sav'/FACTOR=1/SPLIT=2.
-MSAVE {1,2}/TYPE=COV/VARIABLES=x/OUTFILE='matrix3.sav'/FACTOR=1/SPLIT=2.
-MSAVE {1,2;3}/TYPE=COV.
-MSAVE 0/TYPE=COV/FACTOR={1,2}.
-MSAVE 0/TYPE=COV/FACTOR=1/SPLIT={1;2}.
-END MATRIX.
-
-MATRIX.
-MSAVE 1/TYPE=COV/OUTFILE='matrix4.sav'/SNAMES=x,x/SPLIT=1.
-END MATRIX.
-
-MATRIX.
-MSAVE 1/TYPE=COV/OUTFILE='matrix5.sav'/SNAMES=x/FNAMES=x/SPLIT=1/FACTOR=1.
-END MATRIX.
-
-MATRIX.
-MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/FNAMES=x/FACTOR=1.
-END MATRIX.
-
-MATRIX.
-MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/SNAMES=x/SPLIT=1.
-END MATRIX.
-
-MATRIX.
-MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=VARNAME_.
-MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=ROWTYPE_.
-MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=VARNAME_.
-MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=ROWTYPE_.
-MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=VARNAME_.
-MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=ROWTYPE_.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.7: error: MSAVE: Syntax error expecting matrix expression.
- 2 | MSAVE !.
- | ^
-
-matrix.sps:3.14: error: MSAVE: Syntax error expecting COV, CORR, MEAN, STDDEV,
-N, or COUNT.
- 3 | MSAVE 1/TYPE=!.
- | ^
-
-matrix.sps:4.17: error: MSAVE: Syntax error expecting a file name or handle
-name.
- 4 | MSAVE 1/OUTFILE=!.
- | ^
-
-matrix.sps:5.19: error: MSAVE: Syntax error expecting variable name.
- 5 | MSAVE 1/VARIABLES=!.
- | ^
-
-matrix.sps:6.16: error: MSAVE: Syntax error expecting variable name.
- 6 | MSAVE 1/FNAMES=!.
- | ^
-
-matrix.sps:7.16: error: MSAVE: Syntax error expecting variable name.
- 7 | MSAVE 1/SNAMES=!.
- | ^
-
-matrix.sps:8.15: error: MSAVE: Syntax error expecting matrix expression.
- 8 | MSAVE 1/SPLIT=!.
- | ^
-
-matrix.sps:9.16: error: MSAVE: Syntax error expecting matrix expression.
- 9 | MSAVE 1/FACTOR=!.
- | ^
-
-matrix.sps:10.9: error: MSAVE: Syntax error expecting TYPE, OUTFILE, VARIABLES,
-FNAMES, SNAMES, SPLIT, or FACTOR.
- 10 | MSAVE 1/!.
- | ^
-
-matrix.sps:11.1-11.8: error: MSAVE: Required subcommand TYPE was not specified.
- 11 | MSAVE 1.
- | ^~~~~~~~
-
-matrix.sps:12.25: error: MSAVE: FNAMES requires FACTOR.
- 12 | MSAVE 1/TYPE=COV/FNAMES=x.
- | ^
-
-matrix.sps:13.25: error: MSAVE: SNAMES requires SPLIT.
- 13 | MSAVE 1/TYPE=COV/SNAMES=x.
- | ^
-
-matrix.sps:14.1-14.17: error: MSAVE: Required subcommand OUTFILE was not
-specified.
- 14 | MSAVE 1/TYPE=COV.
- | ^~~~~~~~~~~~~~~~~
-
-matrix.sps:20: error: MSAVE: OUTFILE must name the same file on each MSAVE
-within a single MATRIX command.
-
-matrix.sps:16.26-16.37: note: MSAVE: This is the OUTFILE on the first MSAVE
-command.
- 16 | MSAVE 1/TYPE=COV/OUTFILE='matrix.sav'
- | ^~~~~~~~~~~~
-
-matrix.sps:20.26-20.38: note: MSAVE: This is the OUTFILE on a later MSAVE
-command.
- 20 | MSAVE 1/TYPE=COV/OUTFILE='matrix2.sav'.
- | ^~~~~~~~~~~~~
-
-matrix.sps:21: error: MSAVE: VARIABLES must specify the same variables on each
-MSAVE within a given MATRIX.
-
-matrix.sps:19.16: error: MSAVE: This is the specification of VARIABLES on the
-first MSAVE.
- 19 | /VARIABLES=w.
- | ^
-
-matrix.sps:21.28: error: MSAVE: This is a different specification of VARIABLES
-on a later MSAVE.
- 21 | MSAVE 1/TYPE=COV/VARIABLES=x.
- | ^
-
-matrix.sps:22: error: MSAVE: FNAMES must specify the same variables on each
-MSAVE within a given MATRIX.
-
-matrix.sps:17.23: error: MSAVE: This is the specification of FNAMES on the
-first MSAVE.
- 17 | /FACTOR=1 /FNAMES=y
- | ^
-
-matrix.sps:22.25: error: MSAVE: This is a different specification of FNAMES on
-a later MSAVE.
- 22 | MSAVE 1/TYPE=COV/FNAMES=x.
- | ^
-
-matrix.sps:23: error: MSAVE: SNAMES must specify the same variables on each
-MSAVE within a given MATRIX.
-
-matrix.sps:18.22: error: MSAVE: This is the specification of SNAMES on the
-first MSAVE.
- 18 | /SPLIT=2 /SNAMES=z
- | ^
-
-matrix.sps:23.25: error: MSAVE: This is a different specification of SNAMES on
-a later MSAVE.
- 23 | MSAVE 1/TYPE=COV/SNAMES=x.
- | ^
-
-matrix.sps:28.7-28.11: error: MATRIX: Matrix on MSAVE has 2 columns but there
-are 1 variables.
- 28 | MSAVE {1,2}/TYPE=COV/VARIABLES=x/OUTFILE='matrix3.sav'/FACTOR=1/
-SPLIT=2.
- | ^~~~~
-
-matrix.sps:29.7-29.13: error: MATRIX: This expression tries to vertically join
-matrices with differing numbers of columns.
- 29 | MSAVE {1,2;3}/TYPE=COV.
- | ^~~~~~~
-
-matrix.sps:29.8-29.10: note: MATRIX: This operand is a 1×2 matrix.
- 29 | MSAVE {1,2;3}/TYPE=COV.
- | ^~~
-
-matrix.sps:29.12: note: MATRIX: This operand is a 1×1 matrix.
- 29 | MSAVE {1,2;3}/TYPE=COV.
- | ^
-
-matrix.sps:30.25-30.29: error: MATRIX: There are 1 factor variables, but 2
-factor values were supplied.
- 30 | MSAVE 0/TYPE=COV/FACTOR={1,2}.
- | ^~~~~
-
-matrix.sps:31.33-31.37: error: MATRIX: There are 1 split variables, but 2 split
-values were supplied.
- 31 | MSAVE 0/TYPE=COV/FACTOR=1/SPLIT={1;2}.
- | ^~~~~
-
-matrix.sps:35.49: error: MSAVE: Variable x appears twice in variable list.
- 35 | MSAVE 1/TYPE=COV/OUTFILE='matrix4.sav'/SNAMES=x,x/SPLIT=1.
- | ^
-
-matrix.sps:39.56: error: MATRIX: Duplicate or invalid FACTOR variable name x.
- 39 | MSAVE 1/TYPE=COV/OUTFILE='matrix5.sav'/SNAMES=x/FNAMES=x/SPLIT=1/
-FACTOR=1.
- | ^
-
-matrix.sps:43.50: error: MATRIX: Duplicate or invalid variable name x.
- 43 | MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/FNAMES=x/FACTOR=1.
- | ^
-
-matrix.sps:47.50: error: MATRIX: Duplicate or invalid variable name x.
- 47 | MSAVE 1/TYPE=COV/OUTFILE='matrix6.sav'/VARIABLES=x/SNAMES=x/SPLIT=1.
- | ^
-
-matrix.sps:51.47-51.54: error: MSAVE: Variable name VARNAME_ is reserved.
- 51 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=VARNAME_.
- | ^~~~~~~~
-
-matrix.sps:52.47-52.54: error: MSAVE: Variable name ROWTYPE_ is reserved.
- 52 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/SNAMES=ROWTYPE_.
- | ^~~~~~~~
-
-matrix.sps:53.47-53.54: error: MSAVE: Variable name VARNAME_ is reserved.
- 53 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=VARNAME_.
- | ^~~~~~~~
-
-matrix.sps:54.47-54.54: error: MSAVE: Variable name ROWTYPE_ is reserved.
- 54 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/FNAMES=ROWTYPE_.
- | ^~~~~~~~
-
-matrix.sps:55.50-55.57: error: MSAVE: Variable name VARNAME_ is reserved.
- 55 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=VARNAME_.
- | ^~~~~~~~
-
-matrix.sps:56.50-56.57: error: MSAVE: Variable name ROWTYPE_ is reserved.
- 56 | MSAVE 1/TYPE=COV/OUTFILE='matrix7.sav'/VARIABLES=ROWTYPE_.
- | ^~~~~~~~
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - DISPLAY])
-AT_DATA([matrix-template.sps], [dnl
-MATRIX.
-COMPUTE a={1,2,3}.
-COMPUTE b={1;2;3}.
-COMPUTE c={T(b),a}.
-COMPUTE d={T(a),b}.
-command.
-END MATRIX.
-])
-for command in 'DISPLAY' 'DISPLAY DICTIONARY' 'DISPLAY STATUS'; do
- sed "s/command/$command/" < matrix-template.sps > matrix.sps
- AT_CHECK([pspp matrix.sps -O format=csv], [0], [dnl
-Table: Matrix Variables
-,Dimension,,Size (kB)
-,Rows,Columns,
-a,1,3,0
-b,3,1,0
-c,1,6,0
-d,3,2,0
-])
-done
-AT_CLEANUP
-
-AT_SETUP([MATRIX - DISPLAY - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-DISPLAY !.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.9: error: DISPLAY: Syntax error expecting DICTIONARY or STATUS.
- 2 | DISPLAY !.
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - RELEASE])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-COMPUTE x=1.
-PRINT x.
-RELEASE X.
-PRINT x.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-x
- 1
-
-matrix.sps:5.7: error: MATRIX: Uninitialized variable x used in expression.
- 5 | PRINT x.
- | ^
-])
-AT_CLEANUP
-
-AT_SETUP([MATRIX - RELEASE - negative])
-AT_DATA([matrix.sps], [dnl
-MATRIX.
-RELEASE !.
-RELEASE x.
-COMPUTE x=1.
-RELEASE x, !.
-COMPUTE x=1.
-RELEASE x y.
-COMPUTE x=1.
-RELEASE x.
-RELEASE x.
-END MATRIX.
-])
-AT_CHECK([pspp matrix.sps], [1], [dnl
-matrix.sps:2.9: error: RELEASE: Syntax error expecting end of command.
- 2 | RELEASE !.
- | ^
-
-matrix.sps:3.9: error: RELEASE: Syntax error expecting variable name.
- 3 | RELEASE x.
- | ^
-
-matrix.sps:5.12: error: RELEASE: Syntax error expecting end of command.
- 5 | RELEASE x, !.
- | ^
-
-matrix.sps:7.11: error: RELEASE: Syntax error expecting end of command.
- 7 | RELEASE x y.
- | ^
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017, 2019 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([MEANS procedure])
-
-AT_SETUP([MEANS simple])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-simple.sps], [dnl
-data list notable list /hand * score * w *.
-begin data.
-1 17 4
-1 16 5
-2 21 1
-2 22 1
-2 20 8
-end data.
-
-weight by w.
-
-means tables = score by hand
- /cells = mean count.
-])
-
-AT_CHECK([pspp -O format=csv means-simple.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * hand,19,100.0%,0,.0%,19,100.0%
-
-Table: Report
-hand,Mean,N
-1.00,16.44,9
-2.00,20.30,10
-Total,18.47,19
-])
-
-AT_CLEANUP
-
-AT_SETUP([MEANS very simple])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([very-simple.sps], [dnl
-data list notable list /score *.
-begin data.
-17
-17
-17
-16
-17
-16
-16
-16
-16
-21
-22
-20
-20
-20
-20
-20
-20
-20
-20
-end data.
-
-means tables = score
- /cells = mean count.
-])
-
-AT_CHECK([pspp -O format=csv very-simple.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score,19,100.0%,0,.0%,19,100.0%
-
-Table: Report
-Mean,N
-18.47,19
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([MEANS empty factor spec])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-bad.sps], [dnl
-data list list /outcome *.
-begin data.
-1
-2
-3
-end data.
-
-MEANS TABLES = outcome
- BY.
-])
-
-AT_CHECK([pspp -O format=csv means-bad.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([MEANS parser bug])
-AT_KEYWORDS([categorical categoricals])
-
-dnl This bug caused an infinite loop
-AT_DATA([means-bad.sps], [dnl
-DATA LIST notable LIST /a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 fylo *.
-begin data.
-1 2 3 4 5 6 7 8 9 0 11
-end data.
-
-MEANS TABLES = a1 a2 a3 a4 a5 a6 a7 a8 a9 a10a BY fylo.
-])
-
-AT_CHECK([pspp -O format=csv means-bad.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-dnl This example is based upon info from https://libguides.library.kent.edu/SPSS/CompareMeans
-AT_SETUP([MEANS default missing behaviour])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-missing.sps], [dnl
-data list notable list /w * score * a * b *.
-begin data
- 12 . 0 0
- 13 . 0 1
- 11 . 1 0
- 7 . 1 1
- 5 1 0 .
- 91 1 0 0
- 130 1 0 1
- 4 1 1 .
- 90 1 1 0
- 72 1 1 1
-end data.
-
-weight by w.
-
-MEANS tables=score
- /cells = count.
-
-MEANS tables=score by a
- /cells = count.
-
-MEANS tables=score by a by b
- /cells = count.
-])
-
-AT_CHECK([pspp -O format=csv means-missing.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score,392,90.1%,43,9.9%,435,100.0%
-
-Table: Report
-N
-392
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * a,392,90.1%,43,9.9%,435,100.0%
-
-Table: Report
-a,N
-.00,226
-1.00,166
-Total,392
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * a * b,383,88.0%,52,12.0%,435,100.0%
-
-Table: Report
-a,b,N
-.00,.00,91
-,1.00,130
-,Total,221
-1.00,.00,90
-,1.00,72
-,Total,162
-Total,.00,181
-,1.00,202
-,Total,383
-])
-
-AT_CLEANUP
-
-
-dnl This example from https://www.spss-tutorials.com/spss-means-command/
-AT_SETUP([MEANS two way])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-freelancer.sps], [dnl
-data list notable list /income_2010 * gender sector_2010.
-begin data
-6072.40 0 5
-12706.65 1 4
-14912.82 0 2
-16338.36 1 5
-22606.99 0 .
-23544.95 1 1
-24985.21 0 2
-26586.48 0 1
-29076.24 1 3
-31010.18 0 2
-33190.63 1 1
-35570.67 1 4
-36202.60 1 4
-36205.85 1 2
-36262.56 1 .
-38283.56 0 1
-38569.91 1 5
-39057.56 1 4
-39594.68 1 5
-42087.38 0 1
-42370.92 0 2
-42931.32 1 2
-45907.58 0 4
-45911.32 1 .
-47227.09 1 3
-50440.71 1 5
-57440.17 1 3
-58918.86 0 5
-59430.07 1 2
-61135.95 0 4
-64193.85 0 4
-64857.02 0 3
-65903.42 0 4
-66592.38 1 3
-70986.10 0 3
-71229.94 0 4
-74663.05 1 4
-76676.14 1 4
-79260.80 0 4
-80311.71 0 4
-end data.
-
-means income_2010 by gender by sector_2010
- /cells count min mean stddev.
-])
-
-AT_CHECK([pspp -O format=csv means-freelancer.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-income_2010 * gender * sector_2010,37,92.5%,3,7.5%,40,100.0%
-
-Table: Report
-gender,sector_2010,N,Minimum,Mean,Std. Deviation
-.00,1.00,3,26586.48,35652.47,8078.46
-,2.00,4,14912.82,28319.78,11482.43
-,3.00,2,64857.02,67921.56,4333.91
-,4.00,7,45907.58,66849.04,11787.11
-,5.00,2,6072.40,32495.63,37368.09
-,Total,18,6072.40,49389.68,22371.48
-1.00,1.00,2,23544.95,28367.79,6820.53
-,2.00,3,36205.85,46189.08,11949.93
-,3.00,4,29076.24,50083.97,16084.44
-,4.00,6,12706.65,45812.78,24995.16
-,5.00,4,16338.36,36235.92,14311.04
-,Total,19,12706.65,42918.90,17851.64
-Total,1.00,5,23544.95,32738.60,7757.62
-,2.00,7,14912.82,35978.05,14309.27
-,3.00,6,29076.24,56029.83,15615.06
-,4.00,13,12706.65,57139.99,21187.85
-,5.00,6,6072.40,34989.15,20146.69
-,Total,37,6072.40,46066.84,20160.12
-])
-
-AT_CLEANUP
-
-
-dnl Check that rows are suppressed and that things generally work ok
-dnl when there are a 2 way instance contains an unbalanced set of
-dnl categorical values.
-AT_SETUP([MEANS unbalanced])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-unbalanced.sps], [dnl
-data list notable list /b c x *.
-begin data.
-4 1 123
-3 1 123
-5 0 246
-4 0 246
-3 0 246
-end data.
-
-* The data above lack a 5 1 case.
-
-means
- table=x by b by c
- /cells = mean count
- .
-])
-
-AT_CHECK([pspp -O format=csv means-unbalanced.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-x * b * c,5,100.0%,0,.0%,5,100.0%
-
-Table: Report
-b,c,Mean,N
-3.00,.00,246.00,1
-,1.00,123.00,1
-,Total,184.50,2
-4.00,.00,246.00,1
-,1.00,123.00,1
-,Total,184.50,2
-5.00,.00,246.00,1
-,Total,246.00,1
-Total,.00,246.00,3
-,1.00,123.00,2
-,Total,196.80,5
-])
-
-AT_CLEANUP
-
-dnl This example kindly provided by Dana Williams
-AT_SETUP([MEANS three way])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-threeway.sps], [dnl
-data list notable list /score a b c.
-begin data.
-3 0 0 0
-4 0 0 1
-41 0 0 2
-5 0 1 0
-6 0 1 1
-7 1 0 0
-8 1 0 1
-9 1 1 0
-10 1 1 1
-end data.
-
-means score by a by b by c.
-])
-
-AT_CHECK([pspp -O format=csv means-threeway.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * a * b * c,9,100.0%,0,.0%,9,100.0%
-
-Table: Report
-a,b,c,Mean,N,Std. Deviation
-.00,.00,.00,3.00,1,NaN
-,,1.00,4.00,1,NaN
-,,2.00,41.00,1,NaN
-,,Total,16.00,3,21.66
-,1.00,.00,5.00,1,NaN
-,,1.00,6.00,1,NaN
-,,Total,5.50,2,.71
-,Total,.00,4.00,2,1.41
-,,1.00,5.00,2,1.41
-,,2.00,41.00,1,NaN
-,,Total,11.80,5,16.36
-1.00,.00,.00,7.00,1,NaN
-,,1.00,8.00,1,NaN
-,,Total,7.50,2,.71
-,1.00,.00,9.00,1,NaN
-,,1.00,10.00,1,NaN
-,,Total,9.50,2,.71
-,Total,.00,8.00,2,1.41
-,,1.00,9.00,2,1.41
-,,Total,8.50,4,1.29
-Total,.00,.00,5.00,2,2.83
-,,1.00,6.00,2,2.83
-,,2.00,41.00,1,NaN
-,,Total,12.60,5,16.01
-,1.00,.00,7.00,2,2.83
-,,1.00,8.00,2,2.83
-,,Total,7.50,4,2.38
-,Total,.00,6.00,4,2.58
-,,1.00,7.00,4,2.58
-,,2.00,41.00,1,NaN
-,,Total,10.33,9,11.73
-])
-
-AT_CLEANUP
-
-dnl The above example again, but with string variables for
-dnl the control vars.
-AT_SETUP([MEANS three way string])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-threeway-string.sps], [dnl
-data list notable list /score (f22.2) a (a24) b (a16) c (a8).
-begin data.
-3 fooberrycrumblexzaQ fosilationwereqd zero
-4 fooberrycrumblexzaQ fosilationwereqd one
-41 fooberrycrumblexzaQ fosilationwereqd two
-5 fooberrycrumblexzaQ onlyonekonboys zero
-6 fooberrycrumblexzaQ onlyonekonboys one
-7 wontledingbatsXASDF fosilationwereqd zero
-8 wontledingbatsXASDF fosilationwereqd one
-9 wontledingbatsXASDF onlyonekonboys zero
-10 wontledingbatsXASDF onlyonekonboys one
-end data.
-
-means score by a by b by c.
-])
-
-AT_CHECK([pspp -O format=csv means-threeway-string.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * a * b * c,9,100.0%,0,.0%,9,100.0%
-
-Table: Report
-a,b,c,Mean,N,Std. Deviation
-fooberrycrumblexzaQ,fosilationwereqd,one,4.00,1,NaN
-,,two,41.00,1,NaN
-,,zero,3.00,1,NaN
-,,Total,16.00,3,21.66
-,onlyonekonboys,one,6.00,1,NaN
-,,zero,5.00,1,NaN
-,,Total,5.50,2,.71
-,Total,one,5.00,2,1.41
-,,two,41.00,1,NaN
-,,zero,4.00,2,1.41
-,,Total,11.80,5,16.36
-wontledingbatsXASDF,fosilationwereqd,one,8.00,1,NaN
-,,zero,7.00,1,NaN
-,,Total,7.50,2,.71
-,onlyonekonboys,one,10.00,1,NaN
-,,zero,9.00,1,NaN
-,,Total,9.50,2,.71
-,Total,one,9.00,2,1.41
-,,zero,8.00,2,1.41
-,,Total,8.50,4,1.29
-Total,fosilationwereqd,one,6.00,2,2.83
-,,two,41.00,1,NaN
-,,zero,5.00,2,2.83
-,,Total,12.60,5,16.01
-,onlyonekonboys,one,8.00,2,2.83
-,,zero,7.00,2,2.83
-,,Total,7.50,4,2.38
-,Total,one,7.00,4,2.58
-,,two,41.00,1,NaN
-,,zero,6.00,4,2.58
-,,Total,10.33,9,11.73
-])
-
-AT_CLEANUP
-
-
-
-dnl An example with multiple tables
-AT_SETUP([MEANS multiple tables])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-multi-table.sps], [dnl
-data list notable list /a * b * c * x * y *.
-begin data.
-6 3 0 123 456
-6 3 1 123 456
-6 4 0 123 456
-6 4 1 123 456
-6 5 0 123 456
-6 5 1 123 456
-7 3 0 123 456
-7 3 1 123 456
-7 4 0 123 456
-7 4 1 123 456
-7 5 0 123 456
-7 5 1 123 456
-8 3 0 123 456
-8 3 1 123 456
-8 4 0 123 456
-8 4 1 123 456
-8 5 0 123 456
-8 5 1 123 456
-9 3 0 123 456
-9 3 1 123 456
-9 4 0 123 456
-9 4 1 123 456
-9 5 0 123 456
-9 5 1 123 456
-end data.
-
-
-means table = x by b by c
- /x by b
- /y by a by b
- cells = min count .
-])
-
-AT_CHECK([pspp -O format=csv means-multi-table.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-x * b * c,24,100.0%,0,.0%,24,100.0%
-
-Table: Report
-b,c,Minimum,N
-3.00,.00,123.00,4
-,1.00,123.00,4
-,Total,123.00,8
-4.00,.00,123.00,4
-,1.00,123.00,4
-,Total,123.00,8
-5.00,.00,123.00,4
-,1.00,123.00,4
-,Total,123.00,8
-Total,.00,123.00,12
-,1.00,123.00,12
-,Total,123.00,24
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-x * b,24,100.0%,0,.0%,24,100.0%
-
-Table: Report
-b,Minimum,N
-3.00,123.00,8
-4.00,123.00,8
-5.00,123.00,8
-Total,123.00,24
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-y * a * b,24,100.0%,0,.0%,24,100.0%
-
-Table: Report
-a,b,Minimum,N
-6.00,3.00,456.00,2
-,4.00,456.00,2
-,5.00,456.00,2
-,Total,456.00,6
-7.00,3.00,456.00,2
-,4.00,456.00,2
-,5.00,456.00,2
-,Total,456.00,6
-8.00,3.00,456.00,2
-,4.00,456.00,2
-,5.00,456.00,2
-,Total,456.00,6
-9.00,3.00,456.00,2
-,4.00,456.00,2
-,5.00,456.00,2
-,Total,456.00,6
-Total,3.00,456.00,8
-,4.00,456.00,8
-,5.00,456.00,8
-,Total,456.00,24
-])
-
-AT_CLEANUP
-
-
-
-dnl An example with more than one dependent variable.
-dnl This case uses a somewhat different table layout.
-AT_SETUP([MEANS multi variable])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-multi-variable.sps], [dnl
-data list notable list /b c x y.
-begin data.
-5 1 123 55
-5 1 123 55
-5 1 123 55
-5 1 123 55
-4 1 456 44
-4 1 456 44
-4 1 456 44
-4 1 456 44
-3 1 789 55
-3 1 789 55
-3 1 789 55
-3 1 789 55
-5 0 246 99
-5 0 246 99
-5 0 246 99
-5 0 246 .
-4 0 987 99
-4 0 987 99
-4 0 987 99
-4 0 987 99
-3 0 654 11
-3 0 654 11
-3 0 654 11
-3 0 654 11
-end data.
-
-means
- table = x y by b by c
- .
-])
-
-AT_CHECK([pspp -O format=csv means-multi-variable.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-x * b * c,24,100.0%,0,.0%,24,100.0%
-y * b * c,23,95.8%,1,4.2%,24,100.0%
-
-Table: x * y * b * c
-b,c,,x,y
-3.00,.00,Mean,654.00,11.00
-,,N,4,4
-,,Std. Deviation,.00,.00
-,1.00,Mean,789.00,55.00
-,,N,4,4
-,,Std. Deviation,.00,.00
-,Total,Mean,721.50,33.00
-,,N,8,8
-,,Std. Deviation,72.16,23.52
-4.00,.00,Mean,987.00,99.00
-,,N,4,4
-,,Std. Deviation,.00,.00
-,1.00,Mean,456.00,44.00
-,,N,4,4
-,,Std. Deviation,.00,.00
-,Total,Mean,721.50,71.50
-,,N,8,8
-,,Std. Deviation,283.83,29.40
-5.00,.00,Mean,246.00,99.00
-,,N,4,3
-,,Std. Deviation,.00,.00
-,1.00,Mean,123.00,55.00
-,,N,4,4
-,,Std. Deviation,.00,.00
-,Total,Mean,184.50,73.86
-,,N,8,7
-,,Std. Deviation,65.75,23.52
-Total,.00,Mean,629.00,67.00
-,,N,12,11
-,,Std. Deviation,316.50,44.40
-,1.00,Mean,456.00,51.33
-,,N,12,12
-,,Std. Deviation,283.98,5.42
-,Total,Mean,542.50,58.83
-,,N,24,23
-,,Std. Deviation,307.06,31.22
-])
-
-
-AT_CLEANUP
-
-
-dnl This example is based upon one kindly provided by Dana Williams
-dnl It exercises the most complex case where there are multiple
-dnl dependent variables AND multiple control variables in each layer.
-AT_SETUP([MEANS multi combination])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-multi-combination.sps], [dnl
-data list notable list /one (F22.5) two (F22.5) three four five six.
-begin data
-1 1 1 1 1 1
-2 1 1 1 1 1
-1 2 1 1 1 1
-2 2 1 1 1 1
-1 1 2 1 1 1
-2 1 2 1 1 1
-1 2 2 1 1 1
-2 2 2 1 1 1
-1 1 1 2 1 1
-2 1 1 2 1 1
-1 2 1 2 1 1
-2 2 1 2 1 1
-1 1 2 2 1 1
-2 1 2 2 1 1
-1 2 2 2 1 1
-2 2 2 2 1 1
-1 1 1 1 2 1
-2 1 1 1 2 1
-1 2 1 1 2 1
-2 2 1 1 2 1
-1 1 2 1 2 1
-2 1 2 1 2 1
-1 2 2 1 2 1
-2 2 2 1 2 1
-1 1 1 2 2 1
-2 1 1 2 2 1
-1 2 1 2 2 1
-2 2 1 2 2 1
-1 1 2 2 2 1
-2 1 2 2 2 1
-1 2 2 2 2 1
-2 2 2 2 2 1
-1 1 1 1 1 2
-2 1 1 1 1 2
-1 2 1 1 1 2
-2 2 1 1 1 2
-1 1 2 1 1 2
-2 1 2 1 1 2
-1 2 2 1 1 2
-2 2 2 1 1 2
-1 1 1 2 1 2
-2 1 1 2 1 2
-1 2 1 2 1 2
-2 2 1 2 1 2
-1 1 2 2 1 2
-2 1 2 2 1 2
-1 2 2 2 1 2
-2 2 2 2 1 2
-1 1 1 1 2 2
-2 1 1 1 2 2
-1 2 1 1 2 2
-2 2 1 1 2 2
-1 1 2 1 2 2
-2 1 2 1 2 2
-1 2 2 1 2 2
-2 2 2 1 2 2
-1 1 1 2 2 2
-2 1 1 2 2 2
-1 2 1 2 2 2
-2 2 1 2 2 2
-1 1 2 2 2 2
-2 1 2 2 2 2
-1 2 2 2 2 2
-2 2 2 2 2 2
-end data.
-
-recode six (2 = 62) (1 = 61).
-recode five (2 = 52) (1 = 51).
-recode four (2 = 42) (1 = 41).
-recode three (2 = 32) (1 = 31).
-
-means tables = one two BY three four BY five six.
-])
-
-AT_CHECK([pspp -O format=csv means-multi-combination.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-one * three * five,64,100.0%,0,.0%,64,100.0%
-two * three * five,64,100.0%,0,.0%,64,100.0%
-one * three * six,64,100.0%,0,.0%,64,100.0%
-two * three * six,64,100.0%,0,.0%,64,100.0%
-one * four * five,64,100.0%,0,.0%,64,100.0%
-two * four * five,64,100.0%,0,.0%,64,100.0%
-one * four * six,64,100.0%,0,.0%,64,100.0%
-two * four * six,64,100.0%,0,.0%,64,100.0%
-
-Table: one * two * three * five
-three,five,,one,two
-31.00,51.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,52.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-32.00,51.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,52.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-Total,51.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,52.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,Total,Mean,1.50000,1.50000
-,,N,64,64
-,,Std. Deviation,.50395,.50395
-
-Table: one * two * three * six
-three,six,,one,two
-31.00,61.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,62.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-32.00,61.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,62.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-Total,61.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,62.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,Total,Mean,1.50000,1.50000
-,,N,64,64
-,,Std. Deviation,.50395,.50395
-
-Table: one * two * four * five
-four,five,,one,two
-41.00,51.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,52.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-42.00,51.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,52.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-Total,51.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,52.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,Total,Mean,1.50000,1.50000
-,,N,64,64
-,,Std. Deviation,.50395,.50395
-
-Table: one * two * four * six
-four,six,,one,two
-41.00,61.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,62.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-42.00,61.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,62.00,Mean,1.50000,1.50000
-,,N,16,16
-,,Std. Deviation,.51640,.51640
-,Total,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-Total,61.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,62.00,Mean,1.50000,1.50000
-,,N,32,32
-,,Std. Deviation,.50800,.50800
-,Total,Mean,1.50000,1.50000
-,,N,64,64
-,,Std. Deviation,.50395,.50395
-])
-
-AT_CLEANUP
-
-
-dnl This example was observed to cause a crash in the
-dnl destructor. Found by zzuf.
-AT_SETUP([MEANS clean up])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-bad.sps], [dnl
-data list notable list /one two three four five six.
-begin data
-1 1 1 1 1 1
-2 1 1 1 1 !
-1 2 2 2 2 2
-2 2 2 2 2 2
-end data.
-
-means tables = one two BY thsee four BY five six.
-])
-
-AT_CHECK([pspp -O format=csv means-bad.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-dnl Another example which caused a crash.
-dnl Found by zzuf.
-AT_SETUP([MEANS control all missing])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-bad.sps], [dnl
-data list notable list /a * b * y * uu *.
-begin data.
-6 3 . 5
-6 3 . 5
-6 4 . 5
-end data.
-
-means table = b by a by y by uu
- .
-])
-
-AT_CHECK([pspp -O format=csv means-bad.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-b * a * y * uu,0,.0%,3,100.0%,3,100.0%
-
-"warning: The table ""a * y * uu"" has no non-empty control variables. No result for this table will be displayed."
-])
-
-AT_CLEANUP
-
-
-dnl Do some tests on the MISSING keyword.
-AT_SETUP([MEANS missing classes])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-missing-classes.sps], [dnl
-data list notable list /hand * score *.
-begin data.
-1 17
-1 17
-1 17
-1 16
-1 17
-1 16
-1 16
-1 .
-1 99
-2 21
-2 22
-2 20
-2 20
-2 20
-2 20
-2 20
-2 20
-2 20
-2 20
-9 55
-end data.
-
-missing values score (99).
-missing values hand (9).
-
-means tables=score by hand
- /cells = count max
- /missing = dependent
- .
-
-means tables=score by hand
- /cells = count max
- /missing = include
- .
-
-means tables=score by hand
- /cells = count max
- .
-
-])
-
-AT_CHECK([pspp -O format=csv means-missing-classes.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * hand,18,90.0%,2,10.0%,20,100.0%
-
-Table: Report
-hand,N,Maximum
-1.00,7,17.00
-2.00,10,22.00
-9.00,1,55.00
-Total,18,55.00
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * hand,19,95.0%,1,5.0%,20,100.0%
-
-Table: Report
-hand,N,Maximum
-1.00,8,99.00
-2.00,10,22.00
-9.00,1,55.00
-Total,19,99.00
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-score * hand,17,85.0%,3,15.0%,20,100.0%
-
-Table: Report
-hand,N,Maximum
-1.00,7,17.00
-2.00,10,22.00
-Total,17,22.00
-])
-
-AT_CLEANUP
-
-
-dnl Make sure that behaviour with SPLIT is correct.
-AT_SETUP([MEANS split])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-split.sps], [dnl
-data list notable list /b g *.
-begin data
-2 0
-2 0
-4 0
-4 0
-11 1
-11 1
-end data.
-
-split file by g.
-
-means b /cells = count mean.
-])
-
-AT_CHECK([pspp -O format=csv means-split.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-b,4,100.0%,0,.0%,4,100.0%
-
-Table: Report
-N,Mean
-4,3.00
-
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-b,2,100.0%,0,.0%,2,100.0%
-
-Table: Report
-N,Mean
-2,11.00
-])
-
-AT_CLEANUP
-
-
-dnl Test the output with unusual dependent variable formats
-AT_SETUP([MEANS formats])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([means-formats.sps], [dnl
-data list notable list /hours (TIME11.0) rate (DOLLAR8.2).
-begin data
-12:00 4.09
-14:01 5.23
-end data.
-
-means hours rate
- /cells = mean count max range.
-])
-
-AT_CHECK([pspp -O format=csv means-formats.sps], [0], [dnl
-Table: Case Processing Summary
-,Cases,,,,,
-,Included,,Excluded,,Total,
-,N,Percent,N,Percent,N,Percent
-hours,2,100.0%,0,.0%,2,100.0%
-rate,2,100.0%,0,.0%,2,100.0%
-
-Table: hours * rate
-,hours,rate
-Mean,13:00:30,$4.66
-N,2,2
-Maximum,14:01:00,$5.23
-Range,02:01:00,$1.14
-])
-
-AT_CLEANUP
-
-AT_SETUP([MEANS syntax errors])
-AT_DATA([means.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-MEANS TABLES **.
-MEANS x BY **.
-MEANS x/MISSING=**.
-MEANS x/CELLS=**.
-MEANS x/ **.
-])
-AT_CHECK([pspp -O format=csv means.sps], [1], [dnl
-"means.sps:2.14-2.15: error: MEANS: Syntax error expecting `='.
- 2 | MEANS TABLES **.
- | ^~"
-
-"means.sps:3.12-3.13: error: MEANS: Syntax error expecting variable name.
- 3 | MEANS x BY **.
- | ^~"
-
-"means.sps:4.17-4.18: error: MEANS: Syntax error expecting INCLUDE or DEPENDENT.
- 4 | MEANS x/MISSING=**.
- | ^~"
-
-"means.sps:5.15-5.16: error: MEANS: Syntax error expecting one of the following: MEAN, COUNT, STDDEV, SEMEAN, SUM, MIN, MAX, RANGE, VARIANCE, KURT, SEKURT, SKEW, SESKEW, FIRST, LAST, HARMONIC, GEOMETRIC.
- 5 | MEANS x/CELLS=**.
- | ^~"
-
-"means.sps:6.10-6.11: error: MEANS: Syntax error expecting MISSING or CELLS.
- 6 | MEANS x/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017, 2022 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([NPAR TESTS])
-
-AT_SETUP([NPAR TESTS BINOMIAL P < 0.5; N1/N2 < 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x * w *.
-BEGIN DATA.
-1 6
-2 15
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.3) = x
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
-x,Group 1,1.000,6.000,.286,.300,.551
-,Group 2,2.000,15.000,.714,,
-,Total,,21.000,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P < 0.5; N1/N2 > 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 7
-2 6
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.4) = x
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
-x,Group 1,1,7,.538,.400,.229
-,Group 2,2,6,.462,,
-,Total,,13,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P < 0.5; N1/N2 = 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 8
-2 8
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.4) = x
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
-x,Group 1,1,8,.500,.400,.284
-,Group 2,2,8,.500,,
-,Total,,16,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P > 0.5; N1/N2 < 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 11
-2 12
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.6) = x
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
-x,Group 1,1,11,.478,.600,.164
-,Group 2,2,12,.522,,
-,Total,,23,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P > 0.5; N1/N2 > 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 11
-2 9
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.6) = x.
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
-x,Group 1,1,11,.550,.600,.404
-,Group 2,2,9,.450,,
-,Total,,20,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P > 0.5; N1/N2 = 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 11
-2 11
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.6) = x.
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (1-tailed)
-x,Group 1,1,11,.500,.600,.228
-,Group 2,2,11,.500,,
-,Total,,22,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 < 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 8
-2 15
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL = x
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
-x,Group 1,1,8,.348,.500,.210
-,Group 2,2,15,.652,,
-,Total,,23,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 > 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 12
-2 6
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.5) = x.
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
-x,Group 1,1,12,.667,.500,.238
-,Group 2,2,6,.333,,
-,Total,,18,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 = 1])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x (F8.0) w (F8.0).
-BEGIN DATA.
-1 10
-2 10
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.5) = x
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
-x,Group 1,1,10,.500,.500,1.000
-,Group 2,2,10,.500,,
-,Total,,20,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 = 1 Cutpoint])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x * w *.
-BEGIN DATA.
-9 3
-10 7
-11 16
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.5) = x (10)
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
-x,Group 1,<= 10,10.000,.385,.500,.327
-,Group 2,,16.000,.615,,
-,Total,,26.000,1.000,,
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS BINOMIAL P = 0.5; N1/N2 = 1 Named values])
-AT_DATA([npar.sps], [dnl
-SET FORMAT F8.3.
-
-DATA LIST LIST NOTABLE /x * w *.
-BEGIN DATA.
-10 10
-15 45
-20 13
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- /BINOMIAL(0.5) = x (10, 20)
- .
-])
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Binomial Test
-,,Category,N,Observed Prop.,Test Prop.,Exact Sig. (2-tailed)
-x,Group 1,10.000,10.000,.435,.500,.678
-,Group 2,20.000,13.000,.565,,
-,Total,,23.000,1.000,,
-])
-AT_CLEANUP
-
-
-
-dnl Test for a bug which caused binomial to crash.
-AT_SETUP([NPAR TESTS BINOMIAL - crash])
-AT_DATA([nparX.sps], [dnl
-data list list /range *.
-begin data.
-0
-1
-end data.
-
-* This is invalid syntax
-NPAR TEST
- /BINOMIAL(0.5) = Range().
-
-])
-AT_CHECK([pspp -O format=csv nparX.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS CHISQUARE])
-AT_DATA([npar.sps], [dnl
-DATA LIST NOTABLE LIST /x * y * w *.
-BEGIN DATA.
-1 2 1
-2 1 3
-3.1 1 4
-3.2 2 1
-4 2 2
-5 3 1
-1 4 2
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- CHISQUARE=x y
- .
-
-NPAR TESTS
- CHISQUARE=y
- /EXPECTED=3 4 5 4
- .
-
-NPAR TESTS
- CHISQUARE=x y(2, 4)
- /EXPECTED = 6 10 3
- .
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: x
-Value,Observed N,Expected N,Residual
-1.00,3.00,2.33,.67
-2.00,3.00,2.33,.67
-3.10,4.00,2.33,1.67
-3.20,1.00,2.33,-1.33
-4.00,2.00,2.33,-.33
-5.00,1.00,2.33,-1.33
-Total,14.00,,
-
-Table: y
-Value,Observed N,Expected N,Residual
-1.00,7.00,3.50,3.50
-2.00,4.00,3.50,.50
-3.00,1.00,3.50,-2.50
-4.00,2.00,3.50,-1.50
-Total,14.00,,
-
-Table: Test Statistics
-,Chi-square,df,Asymp. Sig.
-x,3.14,5,.678
-y,6.00,3,.112
-
-Table: y
-Value,Observed N,Expected N,Residual
-1.00,7.00,2.63,4.38
-2.00,4.00,3.50,.50
-3.00,1.00,4.38,-3.38
-4.00,2.00,3.50,-1.50
-Total,14.00,,
-
-Table: Test Statistics
-,Chi-square,df,Asymp. Sig.
-y,10.61,3,.014
-
-Table: Frequencies
-,x,,,,y,,,
-,Category,Observed N,Expected N,Residual,Category,Observed N,Expected N,Residual
-1,2.00,3.00,3.16,-.16,2.00,4.00,2.21,1.79
-2,3.00,5.00,5.26,-.26,3.00,1.00,3.68,-2.68
-3,4.00,2.00,1.58,.42,4.00,2.00,1.11,.89
-Total,,10.00,,,,7.00,,
-
-Table: Test Statistics
-,Chi-square,df,Asymp. Sig.
-x,.13,2,.936
-y,4.13,2,.127
-])
-
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS CHISQUARE expected values missing])
-AT_DATA([npar.sps], [dnl
-DATA LIST NOTABLE LIST /x * y * w *.
-BEGIN DATA.
-1 2 1
-2 1 3
-3.1 1 4
-3.2 2 1
-4 2 2
-5 3 1
-1 4 2
-END DATA.
-
-WEIGHT BY w.
-
-NPAR TESTS
- CHISQUARE=y
- /EXPECTED = 3 4 5 4 3 1
- .
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [1], [dnl
-"error: CHISQUARE test specified 6 expected values, but variable y has 4 distinct values."
-
-Table: Test Statistics
-,Chi-square,df,Asymp. Sig.
-y,.00,0,1.000
-])
-
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS CHISQUARE with DESCRIPTIVES])
-AT_DATA([npar.sps], [dnl
-DATA LIST NOTABLE LIST /x * y * w * .
-BEGIN DATA.
-1 2 1
-2 1 3
-3.1 1 4
-3.2 2 1
-4 2 2
-5 3 1
-1 4 2
-. 5 1
-END DATA.
-
-WEIGHT BY w.
-
-MISSING VALUES x (4).
-
-NPAR TESTS
- CHISQUARE=x y(-2,5)
- /MISSING=ANALYSIS
- /STATISTICS=DESCRIPTIVES
- .
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Frequencies
-,x,,,,y,,,
-,Category,Observed N,Expected N,Residual,Category,Observed N,Expected N,Residual
-1,-2.00,.00,1.50,-1.50,-2.00,.00,1.88,-1.88
-2,-1.00,.00,1.50,-1.50,-1.00,.00,1.88,-1.88
-3,.00,.00,1.50,-1.50,.00,.00,1.88,-1.88
-4,1.00,3.00,1.50,1.50,1.00,7.00,1.88,5.13
-5,2.00,3.00,1.50,1.50,2.00,4.00,1.88,2.13
-6,3.00,5.00,1.50,3.50,3.00,1.00,1.88,-.88
-7,4.00,.00,1.50,-1.50,4.00,2.00,1.88,.13
-8,5.00,1.00,1.50,-.50,5.00,1.00,1.88,-.88
-Total,,12.00,,,,15.00,,
-
-Table: Test Statistics
-,Chi-square,df,Asymp. Sig.
-x,17.33,7,.015
-y,22.87,7,.002
-
-Table: Descriptive Statistics
-,N,Mean,Std. Deviation,Minimum,Maximum
-x,12.00,2.47,1.19,1.00,5.00
-y,15.00,2.07,1.33,1.00,5.00
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS CHISQUARE, listwise missing])
-AT_DATA([npar.sps], [dnl
-DATA LIST NOTABLE LIST /x * y * w * .
-BEGIN DATA.
-1 2 1
-2 1 3
-3.1 1 4
-3.2 2 1
-4 2 2
-5 3 1
-1 4 2
-. 5 1
-END DATA.
-
-WEIGHT BY w.
-
-* MISSING VALUES x (4).
-
-NPAR TESTS
- CHISQUARE=x y(-2,5)
- /MISSING=LISTWISE
- /STATISTICS=DESCRIPTIVES
- .
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Frequencies
-,x,,,,y,,,
-,Category,Observed N,Expected N,Residual,Category,Observed N,Expected N,Residual
-1,-2.00,.00,1.75,-1.75,-2.00,.00,1.75,-1.75
-2,-1.00,.00,1.75,-1.75,-1.00,.00,1.75,-1.75
-3,.00,.00,1.75,-1.75,.00,.00,1.75,-1.75
-4,1.00,3.00,1.75,1.25,1.00,7.00,1.75,5.25
-5,2.00,3.00,1.75,1.25,2.00,4.00,1.75,2.25
-6,3.00,5.00,1.75,3.25,3.00,1.00,1.75,-.75
-7,4.00,2.00,1.75,.25,4.00,2.00,1.75,.25
-8,5.00,1.00,1.75,-.75,5.00,.00,1.75,-1.75
-Total,,14.00,,,,14.00,,
-
-Table: Test Statistics
-,Chi-square,df,Asymp. Sig.
-x,13.43,7,.062
-y,26.00,7,.001
-
-Table: Descriptive Statistics
-,N,Mean,Std. Deviation,Minimum,Maximum
-x,14.00,2.69,1.23,1.00,5.00
-y,14.00,1.86,1.10,1.00,4.00
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS WILCOXON])
-AT_DATA([npar.sps], [dnl
-data list notable list /foo * bar * w (f8.0).
-begin data.
-1.00 1.00 1
-1.00 2.00 1
-2.00 1.00 1
-1.00 4.00 1
-2.00 5.00 1
-1.00 19.00 1
-2.00 7.00 1
-4.00 5.00 1
-1.00 12.00 1
-2.00 13.00 1
-2.00 2.00 1
-12.00 .00 2
-12.00 1.00 1
-13.00 1.00 1
-end data
-
-variable labels foo "first" bar "second".
-
-weight by w.
-
-npar test
- /wilcoxon=foo with bar (paired)
- /missing analysis
- /method=exact.
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Ranks
-,,N,Mean Rank,Sum of Ranks
-first - second,Negative Ranks,8,6.00,48.00
-,Positive Ranks,5,8.60,43.00
-,Ties,2,,
-,Total,15,,
-
-Table: Test Statistics
-,first - second
-Z,-.18
-Asymp. Sig. (2-tailed),.861
-Exact Sig. (2-tailed),.893
-Exact Sig. (1-tailed),.446
-])
-
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS WILCOXON with missing values])
-AT_DATA([npar.sps], [dnl
-data list notable list /foo * bar * dummy *.
-begin data.
-1.00 1.00 1
-1.00 2.00 1
-2.00 1.00 1
-1.00 4.00 .
-2.00 5.00 .
-1.00 19.00 .
-2.00 7.00 1
-4.00 5.00 1
-1.00 12.00 1
-2.00 13.00 1
-2.00 2.00 1
-12.00 .00 1
-12.00 .00 1
-34.2 . 1
-12.00 1.00 1
-13.00 1.00 1
-end data
-
-variable labels foo "first" bar "second".
-
-npar test
- /wilcoxon=foo with bar (paired)
- /missing analysis
- /method=exact.
-])
-
-dnl This is the same output as the previous test.
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Ranks
-,,N,Mean Rank,Sum of Ranks
-first - second,Negative Ranks,8,6.00,48.00
-,Positive Ranks,5,8.60,43.00
-,Ties,2,,
-,Total,15,,
-
-Table: Test Statistics
-,first - second
-Z,-.18
-Asymp. Sig. (2-tailed),.861
-Exact Sig. (2-tailed),.893
-Exact Sig. (1-tailed),.446
-])
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS SIGN])
-AT_DATA([npar.sps], [dnl
-set format = F9.3.
-
-data list notable list /age * height rank *.
-begin data.
-10 12 11
-12 13 13
-13 14 12
-12 12 10
-9 9 10
-10.3 10.2 12
-end data.
-
-npar tests
- /sign=age height WITH height rank (PAIRED)
- /MISSING ANALYSIS
- /METHOD=EXACT
- .
-])
-AT_CHECK([pspp -o pspp.csv npar.sps])
-dnl Some machines return .313 instead of .312
-dnl (see bug #31611).
-AT_CHECK([sed -e 's/\.313$/.312/' -e 's/^Exact Sig\. (1-tailed),\.313/Exact Sig. (1-tailed),.312/' pspp.csv], [0], [dnl
-Table: Frequencies
-,,N
-age - height,Negative Differences,3
-,Positive Differences,1
-,Ties,2
-,Total,6
-height - rank,Negative Differences,2
-,Positive Differences,3
-,Ties,1
-,Total,6
-
-Table: Test Statistics
-,age - height,height - rank
-Exact Sig. (2-tailed),.625,1.000
-Exact Sig. (1-tailed),.312,.500
-Point Probability,.250,.312
-])
-AT_CLEANUP
-
-
-AT_SETUP([NPAR Kruskal-Wallis test])
-
-dnl Simple case
-AT_DATA([kw-simple.sps], [dnl
-set format = F9.3.
-
-data list notable list /gv * xscore *.
-begin data
-1 96
-1 128
-1 83
-2 132
-2 135
-2 109
-3 115
-1 61
-1 101
-2 82
-2 124
-3 149
-3 166
-3 147
-end data.
-
-value label /gv
- 1 "timed out"
- 2 "hit wicket"
- 3 "handled the ball".
-
-npar tests
- /kruskal-wallis xscore by gv (1, 3)
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt kw-simple.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Ranks
-,,N,Mean Rank
-xscore,timed out,5,4.400
-,hit wicket,5,7.400
-,handled the ball,4,11.500
-,Total,14,
-
-Table: Test Statistics
-,xscore
-Chi-Square,6.406
-df,2
-Asymp. Sig.,.041
-])
-
-
-dnl Now try a missing value in the group variable
-AT_DATA([kw-missing-group.sps], [dnl
-set format = F9.3.
-
-data list notable list /gv * xscore *.
-begin data
-1 96
-1 128
-1 83
-1 61
-1 101
-2 82
-2 124
-2 132
-2 135
-2 109
-3 115
-3 149
-3 166
-3 147
-2.5 344
-end data.
-
-missing values gv (2.5).
-
-value label /gv
- 1 "timed out"
- 2 "hit wicket"
- 3 "handled the ball".
-
-npar tests
- /kruskal-wallis xscore by gv (1, 3)
- /missing=exclude
- .
-])
-
-AT_CHECK([pspp -o pspp2.csv kw-missing-group.sps])
-
-dnl The result should be the same as before
-AT_CHECK([diff pspp.csv pspp2.csv], [0])
-
-dnl Reverse the order of the group values
-AT_DATA([kw-reverse-group.sps], [dnl
-set format = F9.3.
-
-data list notable list /gv * xscore *.
-begin data
-1 96
-1 128
-1 83
-1 61
-1 101
-2 82
-2 124
-2 132
-2 135
-2 109
-3 115
-3 149
-3 166
-3 147
-end data.
-
-value label /gv
- 1 "timed out"
- 2 "hit wicket"
- 3 "handled the ball".
-
-npar tests
- /kruskal-wallis xscore by gv (3, 1)
- /missing=exclude
- .
-])
-
-AT_CHECK([pspp -o pspp2.csv kw-reverse-group.sps])
-
-dnl The result should be the same as before
-AT_CHECK([diff pspp.csv pspp2.csv], [0])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR Kruskal-Wallis multiple-variables])
-
-AT_DATA([kw-multi.sps], [dnl
-set format = F9.3.
-
-data list notable list /gv * xscore * yscore.
-begin data
-1 96 .
-1 128 .
-1 83 .
-2 132 132
-2 135 135
-2 109 109
-3 115 115
-1 61 .
-1 101 .
-2 82 82
-2 124 124
-3 149 149
-3 166 166
-3 147 147
-4 . 96
-4 . 128
-4 . 83
-4 . 61
-4 . 101
-end data.
-
-value label /gv
- 1 "timed out"
- 2 "hit wicket"
- 3 "handled the ball"
- 4 "bowled"
- 5 "lbw"
- .
-
-npar tests
- /k-w xscore yscore by gv (1, 5)
- .
-
-])
-
-
-AT_CHECK([pspp -o pspp.csv kw-multi.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Ranks
-,,N,Mean Rank
-xscore,timed out,5,4.400
-,hit wicket,5,7.400
-,handled the ball,4,11.500
-,Total,14,
-yscore,hit wicket,5,7.400
-,handled the ball,4,11.500
-,bowled,5,4.400
-,Total,14,
-
-Table: Test Statistics
-,xscore,yscore
-Chi-Square,6.406,6.406
-df,2,2
-Asymp. Sig.,.041,.041
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS Runs])
-AT_DATA([npar-runs.sps], [dnl
-set format F11.4.
-data list notable list /score * w *.
-begin data
-4 6
-. 4
-4 3
-3 20
-2 29
-1 42
-6 18
-5 7
-6 78
-5 10
-6 46
-5 5
-6 17
-5 1
-6 11
-4 2
-3 7
-2 6
-1 10
-4 13
-3 22
-3 11
-2 24
-1 18
-4 4
-3 12
-2 10
-1 25
-4 4
-3 7
-2 3
-1 4
-4 2
-3 3
-2 2
-1 4
-end data.
-
-weight by w.
-
-npar tests
- /runs (MEDIAN) = score
- /runs (MEAN) = score
- /runs (MODE) = score
- .
-])
-
-AT_CHECK([pspp -O format=csv npar-runs.sps], [0],
-[Table: Runs Test
-,score
-Test Value (median),3.0000
-Cases < Test Value,177.0000
-Cases ≥ Test Value,309.0000
-Total Cases,486.0000
-Number of Runs,12
-Z,-20.9931
-Asymp. Sig. (2-tailed),.000
-
-Table: Runs Test
-,score
-Test Value (mean),3.6379
-Cases < Test Value,259.0000
-Cases ≥ Test Value,227.0000
-Total Cases,486.0000
-Number of Runs,12
-Z,-21.0650
-Asymp. Sig. (2-tailed),.000
-
-Table: Runs Test
-,score
-Test Value (mode),6.0000
-Cases < Test Value,316.0000
-Cases ≥ Test Value,170.0000
-Total Cases,486.0000
-Number of Runs,11
-Z,-21.0742
-Asymp. Sig. (2-tailed),.000
-])
-
-AT_CLEANUP
-
-
-dnl Thanks to Douglas Bonett for providing this test case.
-AT_SETUP([NPAR TESTS Runs (2)])
-AT_DATA([npar-runs.sps], [dnl
-data list notable free /y.
-begin data
-1 1 2 1 2 1 1 2 1 1 1 2 1 2
-end data.
-NPAR TEST /RUNS(1.5) = y.
-])
-
-AT_CHECK([pspp -O format=csv npar-runs.sps], [0], [dnl
-Table: Runs Test
-,y
-Test Value,1.50
-Cases < Test Value,9
-Cases ≥ Test Value,5
-Total Cases,14
-Number of Runs,10
-Z,1.26
-Asymp. Sig. (2-tailed),.206
-])
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Friedman])
-AT_DATA([npar-friedman.sps], [dnl
-set format F15.4.
-data list notable list /x * y * z.
-begin data
-9.5 6.5 8.1
-8.0 6.0 6.0
-7.0 6.5 4.2
-9.5 5.0 7.3
-9.0 7.0 6.2
-8.5 6.9 6.5
-7.5 8.0 6.5
-6.0 8.0 3.1
-5.0 6.0 4.9
-7.5 7.5 6.2
-end data.
-
-npar tests
- /friedman = x y z.
-])
-
-AT_CHECK([pspp -O format=csv npar-friedman.sps], [0], [dnl
-Table: Ranks
-,Mean Rank
-x,2.6500
-y,2.1000
-z,1.2500
-
-Table: Test Statistics
-N,10
-Chi-Square,10.4737
-df,2
-Asymp. Sig.,.005
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS Mann-Whitney])
-AT_DATA([npar-mann-whitney.sps], [dnl
-SET FORMAT = F11.4
-
-data list notable list /height * sex (f1.0).
-begin data.
-201 1
-84 1
-83 1
-94 1
-88 0
-99 0
-55 0
-69 0
-86 1
-79 1
-91 0
-201 0
-88 1
-85 1
-82 1
-88 0
-75 0
-99 0
-81 0
-72 1
-89 1
-92 1
-80 0
-82 0
-76 0
-65 0
-85 0
-76 1
-145 1
-24 1
-1 4
--4 5
-34 5
-21 4
-end data.
-
-NPAR TESTS
- /M-W = height BY sex (0,1).
-])
-
-AT_CHECK([pspp -O format=csv npar-mann-whitney.sps], [0], [dnl
-Table: Ranks
-,,N,Mean Rank,Sum of Ranks
-height,0,15,14.5333,218.0000
-,1,15,16.4667,247.0000
-,Total,30,,
-
-Table: Test Statistics
-,Mann-Whitney U,Wilcoxon W,Z,Asymp. Sig. (2-tailed)
-height,98.0000,218.0000,-.6020,.547
-])
-
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Mann-Whitney Multiple])
-dnl Check for a bug where the ranks were inappropriately allocated, when
-dnl multiple variables were tested and MISSING=ANALYSIS chosen.
-
-cp "$abs_srcdir/language/mann-whitney.txt" .
-
-AT_DATA([npar-mann-whitney.sps], [dnl
-SET FORMAT = F11.3
-
-DATA LIST NOTABLE FILE='mann-whitney.txt'
- LIST /I002_01 I002_02 I002_03 I002_04 sum_HL *.
-
-VARIABLE LABELS
- I002_01 'IOS: Familie'
- I002_02 'IOS: Freunde'
- I002_03 'IOS: Partner*in'
- I002_04 'IOS: Bekannte'.
-
-MISSING VALUES I002_01 I002_02 I002_03 I002_04 (-9 -1).
-
-NPAR TESTS
- /MISSING=ANALYSIS
- /M-W=I002_01 I002_02 I002_03 I002_04 BY sum_HL (0 1).
-])
-
-AT_CHECK([pspp -O format=csv npar-mann-whitney.sps], [0], [dnl
-Table: Ranks
-,,N,Mean Rank,Sum of Ranks
-IOS: Familie,.000,114,110.018,12542.000
-,1.000,115,119.939,13793.000
-,Total,229,,
-IOS: Freunde,.000,115,108.339,12459.000
-,1.000,115,122.661,14106.000
-,Total,230,,
-IOS: Partner*in,.000,97,95.351,9249.000
-,1.000,91,93.593,8517.000
-,Total,188,,
-IOS: Bekannte,.000,115,111.065,12772.500
-,1.000,115,119.935,13792.500
-,Total,230,,
-
-Table: Test Statistics
-,Mann-Whitney U,Wilcoxon W,Z,Asymp. Sig. (2-tailed)
-IOS: Familie,5987.000,12542.000,-1.167,.243
-IOS: Freunde,5789.000,12459.000,-1.674,.094
-IOS: Partner*in,4331.000,8517.000,-.245,.807
-IOS: Bekannte,6102.500,12772.500,-1.046,.296
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS Cochran])
-AT_DATA([npar-cochran.sps], [dnl
-set format f11.3.
-
-data list notable list /v1 * v2 * v3 * v4 * v5 * v6 * v7 *.
-begin data.
-2 1 1 2 1 1 2
-2 2 2 2 1 1 1
-1 1 2 2 1 1 2
-2 2 2 2 1 1 2
-2 1 2 1 1 2 1
-1 2 2 1 1 1 1
-1 2 2 2 2 2 2
-2 2 1 2 1 1 1
-1 2 1 2 1 1 2
-end data.
-
-npar tests
- /cochran = v1 to v7 .
-
-])
-
-AT_CHECK([pspp -o pspp.csv npar-cochran.sps])
-
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Frequencies
-,Value,
-,Success (2),Failure (1)
-v1,5,4
-v2,6,3
-v3,6,3
-v4,7,2
-v5,1,8
-v6,2,7
-v7,5,4
-
-Table: Test Statistics
-,Value
-N,9
-Cochran's Q,12.735
-df,6
-Asymp. Sig.,.047
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS Kendall])
-AT_DATA([npar-kendall.sps], [dnl
-SET FORMAT F14.3.
-
-data list notable list /v1 * v2 * v3
-begin data.
- 7 7 2
- 5 6 5
- 8 6 4
- 5 7 4
- 5 4 4
- 8 6 5
- 6 3 5
- 7 6 5
- 8 5 5
- . 2 2
- 5 4 5
- 3 4 4
- 5 1 2
- 5 2 1
- 7 6 5
- 6 3 4
- 6 6 6
- 5 4 5
- 4 3 4
- 9 1 1
- 6 2 1
- 3 7 8
- 6 3 4
- 4 4 4
- 5 4 3
- 6 5 2
- 4 4 8
- 4 6 4
- 6 5 5
- 7 8 6
- 5 3 5
-end data.
-
-npar tests
- /kendall = all
- .
-])
-
-AT_CHECK([pspp -o pspp.csv npar-kendall.sps])
-
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Ranks
-,Mean Rank
-v1,2.500
-v2,1.817
-v3,1.683
-
-Table: Test Statistics
-N,30
-Kendall's W,.233
-Chi-Square,13.960
-df,2
-Asymp. Sig.,.001
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS McNemar])
-
-AT_DATA([mcnemar.sps], [dnl
-set format = F12.3.
-data list notable list /v1 * v2 * junk *.
-begin data.
-0 0 0
-0 0 0
-0 0 0
-0 0 0
-0 1 0
-0 1 0
-0 1 0
-0 1 0
-0 1 1
-0 1 1
-0 1 1
-0 1 1
-0 1 1
-1 0 1
-1 0 1
-1 1 1
-1 1 1
-1 1 0
-1 1 0
-1 1 1
-end data.
-
-npar tests
- /mcnemar = v1 WITH v2 junk.
-])
-
-AT_CHECK([pspp -O format=csv mcnemar.sps], [0], [dnl
-Table: v1 & v2
-v1,v2,
-,.000,1.000
-.000,4,9
-1.000,2,5
-
-Table: v1 & junk
-v1,junk,
-,.000,1.000
-.000,8,5
-1.000,2,5
-
-Table: Test Statistics
-,N,Exact Sig. (2-tailed),Exact Sig. (1-tailed),Point Probability
-v1 & v2,20,.065,.033,.027
-v1 & junk,20,.453,.227,.164
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS McNemar Symetricity])
-
-AT_DATA([mcnemar.sps], [dnl
-data list notable list /var1 var2 w (F2.0).
-begin data
-0 0 9
-0 1 8
-1 0 1
-1 1 5
-end data.
-
-weight by w.
-
-NPAR TEST
- /MCNEMAR var1 WITH var2 (PAIRED).
-
-NPAR TEST
- /MCNEMAR var2 WITH var1 (PAIRED).
-])
-
-AT_CHECK([pspp -O format=csv mcnemar.sps], [0], [dnl
-Table: var1 & var2
-var1,var2,
-,0,1
-0,9,8
-1,1,5
-
-Table: Test Statistics
-,N,Exact Sig. (2-tailed),Exact Sig. (1-tailed),Point Probability
-var1 & var2,23,.039,.020,.02
-
-Table: var2 & var1
-var2,var1,
-,0,1
-0,9,1
-1,8,5
-
-Table: Test Statistics
-,N,Exact Sig. (2-tailed),Exact Sig. (1-tailed),Point Probability
-var2 & var1,23,.039,.020,.02
-])
-
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS Kolmogorov-Smirnov Uniform parameters given])
-
-AT_DATA([ks-uniform.sps], [dnl
-set format F12.3.
-data list notable list /x *.
-begin data
-.554
-.382
-.329
-.480
-.711
-.503
-.203
-.477
-.621
-.581
-end data.
-
-npar tests k-s (uniform 0 1) = x.
-])
-
-AT_CHECK([pspp -O format=csv ks-uniform.sps], [0], [dnl
-Table: One-Sample Kolmogorov-Smirnov Test
-,,x
-N,,10
-Uniform Parameters,Minimum,.000
-,Maximum,1.000
-Most Extreme Differences,Absolute,.289
-,Positive,.289
-,Negative,-.229
-Kolmogorov-Smirnov Z,,.914
-Asymp. Sig. (2-tailed),,.374
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Kolmogorov-Smirnov Normal parameters imputed])
-
-AT_DATA([ks-normal.sps], [dnl
-set format = F12.3.
-
-data list notable list /foo * bar *.
-begin data.
-65 12.5
-59 14.2
-43 12.6
-57
-68
-79
-51
-62
-57
-73
-58
-58
-68
-75
-47
-70
-59
-71
-52
-48 13.0
-58 14.1
-37 15.0
-39 13.1
-58 13.2
-43 14.5
-58 13.5
-86 14.0
-63 12.5
-80 12.8
-70
-63
-53
-53
-48
-49
-51
-47
-81
-66
-78
-65
-69
-70 12.1
-63 12.5
-64 12.4
-39 13.8
-51 13.2
-68 14.0
-76 12.6
-53 12.1
-71 13.5
-47 13.8
-87 14.1
-72 12.9
-48 12.1
-75 12.8
-51 13.4
-63 13.9
-61 12.5
-61 12.4
-66 12.8
-82 12.9
-81 13.6
-46
-52
-71
-73
-58
-57
-46
-58
-52 13.5
-71 13.2
-57 12.8
-78 14.1
-73 12.1
-50 12.6
-71
-51
-51
-68
-84
-64
-66
-65
-52
-56
-70
-68
-66
-78
-65
-71
-53
-81
-53
-57
-64
-61
-43
-56
-37
-74
-66
-81
-67
-80
-68
-76
-70
-80
-42
-74
-80
-70
-60
-39
-72
-69
-63
-72
-63
-49
-53 13.2
-43 13.8
-51 12.5
-63 12.6
-64 12.9
-65 13.0
-64 12.5
-66 12.0
-55
-62
-58
-48
-67
-46
-36
-61
-55
-77
-74
-60
-70
-69
-57
-49
-63
-69
-63
-76
-53
-54
-42
-64
-66
-61
-62
-73
-73
-60
-79
-40
-48
-76
-60
-76
-54
-69
-65
-69
-51
-54
-82
-end data.
-
-npar tests
- /k-s (normal) = foo bar.
-])
-
-AT_CHECK([pspp -O format=csv ks-normal.sps], [0], [dnl
-Table: One-Sample Kolmogorov-Smirnov Test
-,,foo,bar
-N,,174,48
-Normal Parameters,Mean,62.109,13.108
-,Std. Deviation,11.548,.718
-Most Extreme Differences,Absolute,.059,.115
-,Positive,.055,.115
-,Negative,-.059,-.082
-Kolmogorov-Smirnov Z,,.785,.795
-Asymp. Sig. (2-tailed),,.569,.552
-])
-
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Median Test (median imputed)])
-
-AT_DATA([median1.sps], [dnl
-set format F12.3.
-data list notable list /ignore * animal * years * w *.
-begin data
-99 1 10 1
-99 4 1 1
-99 5 11 1
-99 5 10 1
-99 3 7 1
-99 6 10 1
-99 0 7 1
-99 3 14 1
-99 2 3 1
-99 1 1 1
-99 4 7 1
-99 5 12 1
-99 3 6 1
-99 4 1 1
-99 3 5 1
-99 5 7 1
-99 4 6 1
-99 3 14 1
-99 4 8 1
-99 5 13 1
-99 2 0 1
-99 4 7 1
-99 4 7 1
-99 1 0 1
-99 2 8 1
-99 4 10 1
-99 2 3 1
-99 2 0 1
-99 4 8 1
-99 1 8 1
-end data.
-
-
-variable label years 'Years expected'.
-variable label animal 'Animal Genus'.
-
-add value labels animal 1 'Animal 1' 2 'Animal 2' 3 'Animal 3' 4 'Animal 4' 5 'Animal 5'.
-
-npar tests
- /median = years by animal (1, 5)
- .
-])
-
-
-AT_CHECK([pspp -O format=csv median1.sps], [0], [dnl
-Table: Frequencies
-,,Animal Genus,,,,
-,,Animal 1,Animal 2,Animal 3,Animal 4,Animal 5
-Years expected,> Median,2,1,2,3,4
-,≤ Median,2,4,3,6,1
-
-Table: Test Statistics
-,N,Median,Chi-Square,df,Asymp. Sig.
-Years expected,28,7.000,4.317,4,.365
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Median Test (median given)])
-
-AT_DATA([median2.sps], [dnl
-set format F12.3.
-data list notable list /ignore * animal * years * w *.
-begin data
-99 1 10 1
-99 4 1 1
-99 5 11 1
-99 5 10 1
-99 3 7 1
-99 3 14 1
-99 2 3 1
-99 1 1 1
-99 4 7 1
-99 5 12 1
-99 3 6 1
-99 4 1 1
-99 3 5 1
-99 5 7 1
-99 4 6 1
-99 3 14 1
-99 4 8 1
-99 5 13 1
-99 2 0 1
-99 4 7 1
-99 4 7 1
-99 1 0 1
-99 2 8 1
-99 4 10 1
-99 2 3 1
-99 2 0 1
-99 4 8 1
-99 1 8 1
-end data.
-
-
-variable label years 'Years expected'.
-variable label animal 'Animal Genus'.
-
-add value labels animal 1 'Animal 1' 2 'Animal 2' 3 'Animal 3' 4 'Animal 4' 5 'Animal 5'.
-
-npar tests
- /median (7) = years by animal (1, 5)
- .
-])
-
-
-AT_CHECK([pspp -O format=csv median2.sps], [0], [dnl
-Table: Frequencies
-,,Animal Genus,,,,
-,,Animal 1,Animal 2,Animal 3,Animal 4,Animal 5
-Years expected,> Median,2,1,2,3,4
-,≤ Median,2,4,3,6,1
-
-Table: Test Statistics
-,N,Median,Chi-Square,df,Asymp. Sig.
-Years expected,28,7.000,4.317,4,.365
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Median Test (two sample)])
-
-AT_DATA([median3.sps], [dnl
-set format F12.3.
-data list notable list /xx * animal * years * w *.
-begin data
-99 1 10 1
-99 4 1 1
-99 5 11 1
-99 5 10 1
-99 3 7 1
-99 3 14 1
-99 2 3 1
-99 1 1 1
-99 4 7 1
-99 5 12 1
-99 3 6 1
-99 4 1 1
-99 3 5 1
-99 5 7 1
-99 4 6 1
-99 3 14 1
-99 4 8 1
-99 5 13 1
-99 2 0 1
-99 4 7 1
-99 4 7 1
-99 1 0 1
-99 2 8 1
-99 4 10 1
-99 2 3 1
-99 2 0 1
-99 4 8 1
-99 1 8 1
-end data.
-
-
-variable label years 'Years expected'.
-variable label animal 'Animal Genus'.
-
-add value labels animal 1 'Animal 1' 2 'Animal 2' 3 'Animal 3' 4 'Animal 4' 5 'Animal 5'.
-
-npar tests
- /median (7) = xx years by animal (5, 1)
- .
-])
-
-
-AT_CHECK([pspp -O format=csv median3.sps], [0], [dnl
-Table: Frequencies
-,,Animal Genus,
-,,Animal 1,Animal 5
-xx,> Median,4,5
-,≤ Median,0,0
-Years expected,> Median,2,4
-,≤ Median,2,1
-
-Table: Test Statistics
-,N,Median,Chi-Square,df,Asymp. Sig.
-xx,9,7.000,NaN,1,NaN
-Years expected,9,7.000,.900,1,.343
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS Jonckheere-Terpstra])
-
-AT_DATA([jt.sps], [dnl
-set format = F12.3.
-data list notable list /x * g * w *.
-begin data.
-52 2 2
-58 2 1
-60 2 1
-62 2 1
-58 0 1
-44 2 1
-46 2 1
-14 3 1
-32 2 1
-16 3 1
-56 2 1
-26 3 1
-40 3 2
-50 4 1
-6 5 1
-34 2 3
-36 2 2
-40 2 2
-50 2 1
-end data.
-
-weight by w.
-
-npar test /jonckheere-terpstra = x by g (5, 2).
-])
-
-
-AT_CHECK([pspp -O format=csv jt.sps], [0], [dnl
-Table: Jonckheere-Terpstra Test
-,Number of levels in g,N,Observed J-T Statistic,Mean J-T Statistic,Std. Deviation of J-T Statistic,Std. J-T Statistic,Asymp. Sig. (2-tailed)
-x,4,24.000,29.500,65.000,15.902,-2.232,.026
-])
-
-AT_CLEANUP
-
-dnl Checks that (PAIRED) can have lists where the same
-dnl variable appears more than once.
-AT_SETUP([NPAR TESTS (PAIRED)])
-AT_DATA([npar.sps], [dnl
-set format = F12.3.
-data list notable list /a * b * c *.
-begin data.
-1 2 4
-4 5 3
-1 2 2
-4 5 1
-end data.
-
-npar tests /wilcoxon a b with c c (paired).
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [0], [dnl
-Table: Ranks
-,,N,Mean Rank,Sum of Ranks
-a - c,Negative Ranks,2,2.500,5.000
-,Positive Ranks,2,2.500,5.000
-,Ties,0,,
-,Total,4,,
-b - c,Negative Ranks,1,1.500,1.500
-,Positive Ranks,2,2.250,4.500
-,Ties,1,,
-,Total,4,,
-
-Table: Test Statistics
-,a - c,b - c
-Z,.000,-.816
-Asymp. Sig. (2-tailed),1.000,.414
-])
-
-
-AT_CLEANUP
-
-
-
-AT_SETUP([NPAR TESTS CHISQUARE crash])
-dnl This syntax had been observed to crash pspp
-
-AT_DATA([npar.sps], [dnl
-data list list /x *.
-begin data.
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-end data.
-
-* This happens to be invalid syntax. But should not crash.
-NPAR TEST
- /CHISQUARE= x(0.098, 99.098)
- /EXPECTED = 1.2.
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([NPAR TESTS - crash on invalid syntax])
-
-AT_DATA([npar.sps], [dnl
-data list notable list /ev * xscore *.
-begin data.
-2 109
-3 115
-1 61
-1 101
-3 147
-end data.
-
-
-npar tests
- /kruskal-wallis xscore by(gv (1, 3).
-])
-
-AT_CHECK([pspp -O format=csv npar.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-
-AT_SETUP([NPAR TESTS - crash on unterminated string])
-
-AT_DATA([npar.sps], [dnl
-DATA LIST NOTABLE LIST /x * y * w * .
-BEGIN DATA.
-3.1 1 4
-3.2 2 1
-4 2 6
-END DATA.
-
-
-NPAR TESTS
-" CHISQUARE=x y(-2,5)
- /STATISTICS=DESCRIPTIVES
- .
-]) dnl "
-
-AT_CHECK([pspp -O format=csv npar.sps], [1], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([NPAR TESTS - syntax errors])
-AT_DATA([npar.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-NPAR TESTS COCHRAN **.
-NPAR TESTS FRIEDMAN **.
-NPAR TESTS KENDALL **.
-NPAR TESTS RUNS **.
-NPAR TESTS RUNS (**).
-NPAR TESTS RUNS (MEAN **).
-NPAR TESTS RUNS (MEAN)=**.
-NPAR TESTS CHISQUARE **.
-NPAR TESTS CHISQUARE x **.
-NPAR TESTS CHISQUARE x (**).
-NPAR TESTS CHISQUARE x (1 **).
-NPAR TESTS CHISQUARE x (1, -1).
-NPAR TESTS CHISQUARE x (1, 2 **).
-NPAR TESTS CHISQUARE x /EXPECTED **.
-NPAR TESTS CHISQUARE x /EXPECTED=1* **.
-NPAR TESTS CHISQUARE x (1,5)/EXPECTED=2.
-NPAR TESTS BINOMIAL (**).
-NPAR TESTS BINOMIAL (1 **).
-NPAR TESTS BINOMIAL (1)**.
-NPAR TESTS BINOMIAL x(**).
-NPAR TESTS BINOMIAL x(1,**).
-NPAR TESTS BINOMIAL x(1,2**).
-NPAR TESTS BINOMIAL x(1**).
-NPAR TESTS K-S **.
-NPAR TESTS K-S (**).
-NPAR TESTS K-S (NORMAL **).
-NPAR TESTS K-S (NORMAL)=**.
-NPAR TESTS J-T **.
-NPAR TESTS J-T x **.
-NPAR TESTS J-T x BY **.
-NPAR TESTS J-T x BY y **.
-NPAR TESTS J-T x BY y (**).
-NPAR TESTS J-T x BY y (1, **).
-NPAR TESTS J-T x BY y (1, 2 **).
-NPAR TESTS MCNEMAR **.
-NPAR TESTS MCNEMAR x **.
-NPAR TESTS MCNEMAR x WITH **.
-NPAR TESTS MCNEMAR x WITH y (**).
-NPAR TESTS MCNEMAR x WITH y (PAIRED **).
-NPAR TESTS MCNEMAR x WITH y z (PAIRED).
-NPAR TESTS MEDIAN (**).
-NPAR TESTS MEDIAN (1 **).
-NPAR TESTS MISSING/MISSING.
-NPAR TESTS MISSING **.
-NPAR TESTS METHOD/METHOD.
-NPAR TESTS METHOD EXACT TIMER(**).
-NPAR TESTS METHOD EXACT TIMER(5 **).
-NPAR TESTS STATISTICS **.
-NPAR TESTS ALGORITHM **.
-NPAR TESTS **.
-])
-AT_CHECK([pspp -O format=csv npar.sps], [1], [dnl
-"npar.sps:2.20-2.21: error: NPAR TESTS: Syntax error expecting variable name.
- 2 | NPAR TESTS COCHRAN **.
- | ^~"
-
-"npar.sps:3.21-3.22: error: NPAR TESTS: Syntax error expecting variable name.
- 3 | NPAR TESTS FRIEDMAN **.
- | ^~"
-
-"npar.sps:4.20-4.21: error: NPAR TESTS: Syntax error expecting variable name.
- 4 | NPAR TESTS KENDALL **.
- | ^~"
-
-"npar.sps:5.17-5.18: error: NPAR TESTS: Syntax error expecting `@{:@'.
- 5 | NPAR TESTS RUNS **.
- | ^~"
-
-"npar.sps:6.18-6.19: error: NPAR TESTS: Syntax error expecting MEAN, MEDIAN, MODE or a number.
- 6 | NPAR TESTS RUNS (**).
- | ^~"
-
-"npar.sps:7.23-7.24: error: NPAR TESTS: Syntax error expecting `@:}@='.
- 7 | NPAR TESTS RUNS (MEAN **).
- | ^~"
-
-"npar.sps:8.24-8.25: error: NPAR TESTS: Syntax error expecting variable name.
- 8 | NPAR TESTS RUNS (MEAN)=**.
- | ^~"
-
-"npar.sps:9.22-9.23: error: NPAR TESTS: Syntax error expecting variable name.
- 9 | NPAR TESTS CHISQUARE **.
- | ^~"
-
-"npar.sps:10.24-10.25: error: NPAR TESTS: Syntax error expecting `BEGIN DATA'.
- 10 | NPAR TESTS CHISQUARE x **.
- | ^~"
-
-"npar.sps:10.24-10.25: error: NPAR TESTS: Syntax error expecting end of command.
- 10 | NPAR TESTS CHISQUARE x **.
- | ^~"
-
-"npar.sps:11.25-11.26: error: NPAR TESTS: Syntax error expecting number.
- 11 | NPAR TESTS CHISQUARE x (**).
- | ^~"
-
-"npar.sps:12.27-12.28: error: NPAR TESTS: Syntax error expecting `,'.
- 12 | NPAR TESTS CHISQUARE x (1 **).
- | ^~"
-
-"npar.sps:13.28-13.29: error: NPAR TESTS: Syntax error expecting number greater than 1 for HI.
- 13 | NPAR TESTS CHISQUARE x (1, -1).
- | ^~"
-
-"npar.sps:14.30-14.31: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 14 | NPAR TESTS CHISQUARE x (1, 2 **).
- | ^~"
-
-"npar.sps:15.34-15.35: error: NPAR TESTS: Syntax error expecting `='.
- 15 | NPAR TESTS CHISQUARE x /EXPECTED **.
- | ^~"
-
-"npar.sps:16.37-16.38: error: NPAR TESTS: Syntax error expecting number.
- 16 | NPAR TESTS CHISQUARE x /EXPECTED=1* **.
- | ^~"
-
-"npar.sps:17.39: error: NPAR TESTS: 1 expected values were given, but the specified range (1-5) requires exactly 5 values.
- 17 | NPAR TESTS CHISQUARE x (1,5)/EXPECTED=2.
- | ^"
-
-"npar.sps:18.22-18.23: error: NPAR TESTS: Syntax error expecting number.
- 18 | NPAR TESTS BINOMIAL (**).
- | ^~"
-
-"npar.sps:19.24-19.25: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 19 | NPAR TESTS BINOMIAL (1 **).
- | ^~"
-
-"npar.sps:20.24-20.25: error: NPAR TESTS: Syntax error expecting `='.
- 20 | NPAR TESTS BINOMIAL (1)**.
- | ^~"
-
-"npar.sps:21.23-21.24: error: NPAR TESTS: Syntax error expecting number.
- 21 | NPAR TESTS BINOMIAL x(**).
- | ^~"
-
-"npar.sps:22.25-22.26: error: NPAR TESTS: Syntax error expecting number.
- 22 | NPAR TESTS BINOMIAL x(1,**).
- | ^~"
-
-"npar.sps:23.26-23.27: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 23 | NPAR TESTS BINOMIAL x(1,2**).
- | ^~"
-
-"npar.sps:24.24-24.25: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 24 | NPAR TESTS BINOMIAL x(1**).
- | ^~"
-
-"npar.sps:25.16-25.17: error: NPAR TESTS: Syntax error expecting `@{:@'.
- 25 | NPAR TESTS K-S **.
- | ^~"
-
-"npar.sps:26.17-26.18: error: NPAR TESTS: Syntax error expecting NORMAL, POISSON, UNIFORM, or EXPONENTIAL.
- 26 | NPAR TESTS K-S (**).
- | ^~"
-
-"npar.sps:27.24-27.25: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 27 | NPAR TESTS K-S (NORMAL **).
- | ^~"
-
-"npar.sps:28.25-28.26: error: NPAR TESTS: Syntax error expecting variable name.
- 28 | NPAR TESTS K-S (NORMAL)=**.
- | ^~"
-
-"npar.sps:29.16-29.17: error: NPAR TESTS: Syntax error expecting variable name.
- 29 | NPAR TESTS J-T **.
- | ^~"
-
-"npar.sps:30.18-30.19: error: NPAR TESTS: Syntax error expecting `BY'.
- 30 | NPAR TESTS J-T x **.
- | ^~"
-
-"npar.sps:31.21-31.22: error: NPAR TESTS: Syntax error expecting variable name.
- 31 | NPAR TESTS J-T x BY **.
- | ^~"
-
-"npar.sps:32.23-32.24: error: NPAR TESTS: Syntax error expecting `@{:@'.
- 32 | NPAR TESTS J-T x BY y **.
- | ^~"
-
-"npar.sps:33.24-33.25: error: NPAR TESTS: Syntax error expecting number.
- 33 | NPAR TESTS J-T x BY y (**).
- | ^~"
-
-"npar.sps:34.27-34.28: error: NPAR TESTS: Syntax error expecting number.
- 34 | NPAR TESTS J-T x BY y (1, **).
- | ^~"
-
-"npar.sps:35.29-35.30: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 35 | NPAR TESTS J-T x BY y (1, 2 **).
- | ^~"
-
-"npar.sps:36.20-36.21: error: NPAR TESTS: Syntax error expecting variable name.
- 36 | NPAR TESTS MCNEMAR **.
- | ^~"
-
-"npar.sps:37.22-37.23: error: NPAR TESTS: Syntax error expecting end of command.
- 37 | NPAR TESTS MCNEMAR x **.
- | ^~"
-
-"npar.sps:38.27-38.28: error: NPAR TESTS: Syntax error expecting variable name.
- 38 | NPAR TESTS MCNEMAR x WITH **.
- | ^~"
-
-"npar.sps:39.30-39.31: error: NPAR TESTS: Syntax error expecting `PAIRED@:}@'.
- 39 | NPAR TESTS MCNEMAR x WITH y (**).
- | ^~"
-
-"npar.sps:40.30-40.38: error: NPAR TESTS: Syntax error expecting `PAIRED)'.
- 40 | NPAR TESTS MCNEMAR x WITH y (PAIRED **).
- | ^~~~~~~~~"
-
-"npar.sps:41.20-41.29: error: NPAR TESTS: PAIRED was specified, but the number of variables preceding WITH (1) does not match the number following (2).
- 41 | NPAR TESTS MCNEMAR x WITH y z (PAIRED).
- | ^~~~~~~~~~"
-
-"npar.sps:42.20-42.21: error: NPAR TESTS: Syntax error expecting number.
- 42 | NPAR TESTS MEDIAN (**).
- | ^~"
-
-"npar.sps:43.22-43.23: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 43 | NPAR TESTS MEDIAN (1 **).
- | ^~"
-
-"npar.sps:44.20-44.26: error: NPAR TESTS: Subcommand MISSING may only be specified once.
- 44 | NPAR TESTS MISSING/MISSING.
- | ^~~~~~~"
-
-"npar.sps:45.20-45.21: error: NPAR TESTS: Syntax error expecting ANALYSIS, LISTWISE, INCLUDE, or EXCLUDE.
- 45 | NPAR TESTS MISSING **.
- | ^~"
-
-"npar.sps:46.19-46.24: error: NPAR TESTS: Subcommand METHOD may only be specified once.
- 46 | NPAR TESTS METHOD/METHOD.
- | ^~~~~~"
-
-"npar.sps:47.31-47.32: error: NPAR TESTS: Syntax error expecting number.
- 47 | NPAR TESTS METHOD EXACT TIMER(**).
- | ^~"
-
-"npar.sps:48.33-48.34: error: NPAR TESTS: Syntax error expecting `@:}@'.
- 48 | NPAR TESTS METHOD EXACT TIMER(5 **).
- | ^~"
-
-"npar.sps:49.23-49.24: error: NPAR TESTS: Syntax error expecting DESCRIPTIVES, QUARTILES, or ALL.
- 49 | NPAR TESTS STATISTICS **.
- | ^~"
-
-"npar.sps:50.22-50.23: error: NPAR TESTS: Syntax error expecting COMPATIBLE or ENHANCED.
- 50 | NPAR TESTS ALGORITHM **.
- | ^~"
-
-"npar.sps:51.12-51.13: error: NPAR TESTS: Syntax error expecting one of the following: COCHRAN, FRIEDMAN, KENDALL, RUNS, CHISQUARE, BINOMIAL, K-S, J-T, K-W, MCNEMAR, M-W, MEDIAN, WILCOXON, SIGN, MISSING, METHOD, STATISTICS, ALGORITHM.
- 51 | NPAR TESTS **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017, 2020 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([ONEWAY procedure])
-
-AT_SETUP([ONEWAY basic operation])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([oneway.sps],
- [DATA LIST NOTABLE LIST /QUALITY * BRAND * .
-BEGIN DATA
-7 3
-4 3
-3 1
-2 1
-1 1
-4 2
-2 2
-3 2
-5 3
-1 1
-4 1
-5 2
-2 2
-3 3
-6 3
-END DATA
-
-VARIABLE LABELS brand 'Manufacturer'.
-VARIABLE LABELS quality 'Breaking Strain'.
-
-VALUE LABELS /brand 1 'Aspeger' 2 'Bloggs' 3 'Charlies'.
-
-ONEWAY
- quality BY brand
- /STATISTICS descriptives homogeneity
- /CONTRAST = -2 1 1
- /CONTRAST = 0 -1 1
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Descriptives
-,Manufacturer,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
-,,,,,,Lower Bound,Upper Bound,,
-Breaking Strain,Aspeger,5,2.20,1.30,.58,.58,3.82,1.00,4.00
-,Bloggs,5,3.20,1.30,.58,1.58,4.82,2.00,5.00
-,Charlies,5,5.00,1.58,.71,3.04,6.96,3.00,7.00
-,Total,15,3.47,1.77,.46,2.49,4.45,1.00,7.00
-
-Table: Test of Homogeneity of Variances
-,Levene Statistic,df1,df2,Sig.
-Breaking Strain,.09,2,12,.913
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-Breaking Strain,Between Groups,20.13,2,10.07,5.12,.025
-,Within Groups,23.60,12,1.97,,
-,Total,43.73,14,,,
-
-Table: Contrast Coefficients
-Contrast,Manufacturer,,
-,Aspeger,Bloggs,Charlies
-1,-2,1,1
-2,0,-1,1
-
-Table: Contrast Tests
-,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
-Breaking Strain,Assume equal variances,1,3.80,1.54,2.47,12.00,.029
-,,2,1.80,.89,2.03,12.00,.065
-,Does not assume equal variances,1,3.80,1.48,2.56,8.74,.031
-,,2,1.80,.92,1.96,7.72,.086
-])
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY with splits])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([oneway-splits.sps],
-[DATA LIST NOTABLE LIST /QUALITY * BRAND * S *.
-BEGIN DATA
-3 1 1
-2 1 1
-1 1 1
-1 1 1
-4 1 1
-5 2 1
-2 2 1
-4 2 2
-2 2 2
-3 2 2
-7 3 2
-4 3 2
-5 3 2
-3 3 2
-6 3 2
-END DATA
-
-VARIABLE LABELS brand 'Manufacturer'.
-VARIABLE LABELS quality 'Breaking Strain'.
-
-VALUE LABELS /brand 1 'Aspeger' 2 'Bloggs' 3 'Charlies'.
-
-SPLIT FILE by s.
-
-ONEWAY
- quality BY brand
- /STATISTICS descriptives homogeneity
- /CONTRAST = -2 2
- /CONTRAST = -1 1
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-splits.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Split Values
-Variable,Value
-S,1.00
-
-Table: Descriptives
-,Manufacturer,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
-,,,,,,Lower Bound,Upper Bound,,
-Breaking Strain,Aspeger,5,2.20,1.30,.58,.58,3.82,1.00,4.00
-,Bloggs,2,3.50,2.12,1.50,-15.56,22.56,2.00,5.00
-,Total,7,2.57,1.51,.57,1.17,3.97,1.00,5.00
-
-Table: Test of Homogeneity of Variances
-,Levene Statistic,df1,df2,Sig.
-Breaking Strain,1.09,1,5,.345
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-Breaking Strain,Between Groups,2.41,1,2.41,1.07,.349
-,Within Groups,11.30,5,2.26,,
-,Total,13.71,6,,,
-
-Table: Contrast Coefficients
-Contrast,Manufacturer,
-,Aspeger,Bloggs
-1,-2,2
-2,-1,1
-
-Table: Contrast Tests
-,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
-Breaking Strain,Assume equal variances,1,2.60,2.52,1.03,5.00,.349
-,,2,1.30,1.26,1.03,5.00,.349
-,Does not assume equal variances,1,2.60,3.22,.81,1.32,.539
-,,2,1.30,1.61,.81,1.32,.539
-
-Table: Split Values
-Variable,Value
-S,2.00
-
-Table: Descriptives
-,Manufacturer,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
-,,,,,,Lower Bound,Upper Bound,,
-Breaking Strain,Bloggs,3,3.00,1.00,.58,.52,5.48,2.00,4.00
-,Charlies,5,5.00,1.58,.71,3.04,6.96,3.00,7.00
-,Total,8,4.25,1.67,.59,2.85,5.65,2.00,7.00
-
-Table: Test of Homogeneity of Variances
-,Levene Statistic,df1,df2,Sig.
-Breaking Strain,.92,1,6,.374
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-Breaking Strain,Between Groups,7.50,1,7.50,3.75,.101
-,Within Groups,12.00,6,2.00,,
-,Total,19.50,7,,,
-
-Table: Contrast Coefficients
-Contrast,Manufacturer,
-,Bloggs,Charlies
-1,-2,2
-2,-1,1
-
-Table: Contrast Tests
-,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
-Breaking Strain,Assume equal variances,1,4.00,2.07,1.94,6.00,.101
-,,2,2.00,1.03,1.94,6.00,.101
-,Does not assume equal variances,1,4.00,1.83,2.19,5.88,.072
-,,2,2.00,.91,2.19,5.88,.072
-])
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY with missing values])
-AT_KEYWORDS([categorical categoricals])
-dnl Check that missing are treated properly
-AT_DATA([oneway-missing1.sps],
-[DATA LIST NOTABLE LIST /v1 * v2 * dep * vn *.
-BEGIN DATA
-. . 1 4
-3 3 1 2
-2 2 1 2
-1 1 1 2
-1 1 1 2
-4 4 1 2
-5 5 2 2
-2 2 2 2
-4 4 2 2
-2 2 2 2
-3 3 2 2
-7 7 3 2
-4 4 3 2
-5 5 3 2
-3 3 3 2
-6 6 3 2
-END DATA
-
-ONEWAY
- v1 v2 BY dep
- /STATISTICS descriptives homogeneity
- /MISSING ANALYSIS
- .
-])
-
-AT_DATA([oneway-missing2.sps],
-[DATA LIST NOTABLE LIST /v1 * v2 * dep * vn * .
-BEGIN DATA
-4 . 1 2
-3 3 1 2
-2 2 1 2
-1 1 1 2
-1 1 1 2
-4 4 1 2
-5 5 2 2
-2 2 2 2
-4 4 2 2
-2 2 2 2
-3 3 2 2
-7 7 3 2
-4 4 3 2
-5 5 3 2
-3 3 3 2
-6 6 3 2
-END DATA
-
-ONEWAY
- v1 v2 BY dep
- /STATISTICS descriptives homogeneity
- /MISSING LISTWISE
- .
-])
-
-
-
-AT_CHECK([pspp -O format=csv oneway-missing1.sps > first.out], [0])
-
-AT_CHECK([pspp -O format=csv oneway-missing2.sps > second.out], [0])
-
-AT_CHECK([diff first.out second.out], [0], [])
-
-dnl Now a test with missing values in the independent variable
-AT_DATA([oneway-missing3.sps],
-[DATA LIST NOTABLE LIST /v1 * v2 * dep * vn * .
-BEGIN DATA
-4 2 . 2
-3 3 1 2
-2 2 1 2
-1 1 1 2
-1 1 1 2
-4 4 1 2
-5 5 2 2
-2 2 2 2
-4 4 2 2
-2 2 2 2
-3 3 2 2
-7 7 3 2
-4 4 3 2
-5 5 3 4
-3 3 3 2
-6 6 3 2
-END DATA
-
-ONEWAY
- v1 v2 BY dep
- /STATISTICS descriptives homogeneity
- /MISSING ANALYSIS
- .
-])
-
-AT_CHECK([pspp -O format=csv oneway-missing3.sps > third.out], [0])
-
-AT_CHECK([diff first.out third.out], [0], [])
-
-AT_CLEANUP
-
-
-
-
-
-AT_SETUP([ONEWAY descriptives subcommand])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([oneway-descriptives.sps],
- [DATA LIST NOTABLE LIST /QUALITY * BRAND * .
-BEGIN DATA
-13 11
-12 11
-11 11
-11 11
-14 11
-15 25
-12 25
-14 25
-12 25
-13 25
-17 301
-14 301
-15 301
-13 301
-16 301
-END DATA
-
-
-ONEWAY
- quality BY brand
- /STATISTICS descriptives
- .
-])
-
-AT_CHECK([pspp -O format=csv oneway-descriptives.sps], [0],
-[Table: Descriptives
-,BRAND,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
-,,,,,,Lower Bound,Upper Bound,,
-QUALITY,11.00,5,12.20,1.30,.58,10.58,13.82,11.00,14.00
-,25.00,5,13.20,1.30,.58,11.58,14.82,12.00,15.00
-,301.00,5,15.00,1.58,.71,13.04,16.96,13.00,17.00
-,Total,15,13.47,1.77,.46,12.49,14.45,11.00,17.00
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-QUALITY,Between Groups,20.13,2,10.07,5.12,.025
-,Within Groups,23.60,12,1.97,,
-,Total,43.73,14,,,
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([ONEWAY homogeneity subcommand])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([oneway-homogeneity.sps],
- [DATA LIST NOTABLE LIST /QUALITY * BRAND * .
-BEGIN DATA
-13 11
-12 11
-11 11
-11 11
-14 11
-15 25
-12 25
-14 25
-12 25
-13 25
-17 301
-14 301
-15 301
-13 301
-16 301
-END DATA
-
-
-ONEWAY
- quality BY brand
- /STATISTICS homogeneity
- .
-])
-
-AT_CHECK([pspp -O format=csv oneway-homogeneity.sps], [0],
-[Table: Test of Homogeneity of Variances
-,Levene Statistic,df1,df2,Sig.
-QUALITY,.09,2,12,.913
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-QUALITY,Between Groups,20.13,2,10.07,5.12,.025
-,Within Groups,23.60,12,1.97,,
-,Total,43.73,14,,,
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([ONEWAY multiple variables])
-AT_KEYWORDS([categorical categoricals])
-dnl check that everything works ok when several different dependent variables are specified.
-dnl This of course does not mean that we're doing a multivariate analysis. It's just like
-dnl running several tests at once.
-AT_DATA([multivar.sps],
-[DATA LIST notable LIST /x * y * z * g *.
-begin data.
-1 1 0 10
-1 1 9 10
-9 1 2 10
-1 1 3 20
-1 1 8 20
-1 1 1 20
-1 1 2 20
-0 1 3 20
-1 1 4 30
-0 1 5 30
-1 1 6 30
-0 1 7 30
-1 2 8 30
-2 2 9 30
-1 2 1 30
-1 2 0 30
-1 2 2 40
-8 2 3 40
-1 2 4 40
-1 2 9 40
-9 2 8 40
-7 3 7 40
-2 3 6 40
-3 3 5 40
-end data.
-
-ONEWAY x y z by g
- /STATISTICS = DESCRIPTIVES HOMOGENEITY
- /CONTRAST 3 2 0 -5
- /CONTRAST 2 -9 7 0
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt multivar.sps])
-
-dnl Some machines return 3.88 instead of 3.87 below (see bug #31611).
-AT_CHECK([sed -e 's/^,Within Groups,3.88/,Within Groups,3.87/' pspp.csv], [0],
- [Table: Descriptives
-,g,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
-,,,,,,Lower Bound,Upper Bound,,
-x,10.00,3,3.67,4.62,2.67,-7.81,15.14,1.00,9.00
-,20.00,5,.80,.45,.20,.24,1.36,.00,1.00
-,30.00,8,.88,.64,.23,.34,1.41,.00,2.00
-,40.00,8,4.00,3.42,1.21,1.14,6.86,1.00,9.00
-,Total,24,2.25,2.83,.58,1.05,3.45,.00,9.00
-y,10.00,3,1.00,.00,.00,1.00,1.00,1.00,1.00
-,20.00,5,1.00,.00,.00,1.00,1.00,1.00,1.00
-,30.00,8,1.50,.53,.19,1.05,1.95,1.00,2.00
-,40.00,8,2.38,.52,.18,1.94,2.81,2.00,3.00
-,Total,24,1.63,.71,.15,1.32,1.93,1.00,3.00
-z,10.00,3,3.67,4.73,2.73,-8.07,15.41,.00,9.00
-,20.00,5,3.40,2.70,1.21,.05,6.75,1.00,8.00
-,30.00,8,5.00,3.21,1.13,2.32,7.68,.00,9.00
-,40.00,8,5.50,2.45,.87,3.45,7.55,2.00,9.00
-,Total,24,4.67,2.99,.61,3.40,5.93,.00,9.00
-
-Table: Test of Homogeneity of Variances
-,Levene Statistic,df1,df2,Sig.
-x,18.76,3,20,.000
-y,71.41,3,20,.000
-z,.89,3,20,.463
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-x,Between Groups,56.16,3,18.72,2.92,.059
-,Within Groups,128.34,20,6.42,,
-,Total,184.50,23,,,
-y,Between Groups,7.75,3,2.58,13.33,.000
-,Within Groups,3.87,20,.19,,
-,Total,11.63,23,,,
-z,Between Groups,17.47,3,5.82,.62,.610
-,Within Groups,187.87,20,9.39,,
-,Total,205.33,23,,,
-
-Table: Contrast Coefficients
-Contrast,g,,,
-,10.00,20.00,30.00,40.00
-1,3,2,0,-5
-2,2,-9,7,0
-
-Table: Contrast Tests
-,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
-x,Assume equal variances,1,-7.40,6.67,-1.11,20.00,.280
-,,2,6.26,12.32,.51,20.00,.617
-,Does not assume equal variances,1,-7.40,10.04,-.74,4.53,.497
-,,2,6.26,5.85,1.07,2.87,.366
-y,Assume equal variances,1,-6.88,1.16,-5.94,20.00,.000
-,,2,3.50,2.14,1.63,20.00,.118
-,Does not assume equal variances,1,-6.88,.91,-7.51,7.00,.000
-,,2,3.50,1.32,2.65,7.00,.033
-z,Assume equal variances,1,-9.70,8.07,-1.20,20.00,.243
-,,2,11.73,14.91,.79,20.00,.440
-,Does not assume equal variances,1,-9.70,9.57,-1.01,3.64,.373
-,,2,11.73,14.53,.81,9.88,.438
-])
-
-AT_CLEANUP
-
-
-
-dnl Tests that everything treats weights properly
-AT_SETUP([ONEWAY vs. weights])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([oneway-unweighted.sps],
-[DATA LIST NOTABLE LIST /QUALITY * BRAND * W *.
-BEGIN DATA
-3 1 1
-2 1 1
-1 1 1
-1 1 1
-4 1 1
-5 2 1
-2 2 1
-4 2 1
-4 2 1
-4 2 1
-2 2 1
-2 2 1
-3 2 1
-7 3 1
-4 3 1
-5 3 1
-5 3 1
-3 3 1
-6 3 1
-END DATA.
-
-WEIGHT BY W.
-
-ONEWAY
- quality BY brand
- /STATISTICS descriptives homogeneity
- .
-])
-
-AT_CHECK([pspp -o pspp-unweighted.csv oneway-unweighted.sps], [0], [ignore], [ignore])
-
-AT_DATA([oneway-weighted.sps],
-[DATA LIST NOTABLE LIST /QUALITY * BRAND * W *.
-BEGIN DATA
-3 1 1
-2 1 1
-1 1 2
-4 1 1
-5 2 1
-2 2 1
-4 2 3
-2 2 2
-3 2 1
-7 3 1
-4 3 1
-5 3 2
-3 3 1
-6 3 1
-END DATA.
-
-WEIGHT BY W.
-
-ONEWAY
- quality BY brand
- /STATISTICS descriptives homogeneity
- .
-])
-
-AT_CHECK([pspp -o pspp-weighted.csv oneway-weighted.sps], [0], [ignore], [ignore])
-
-AT_CHECK([diff pspp-weighted.csv pspp-unweighted.csv], [0])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([ONEWAY posthoc LSD and BONFERRONI])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([oneway-pig.sps],[dnl
-SET FORMAT F12.3.
-data list notable list /pigmentation * family *.
-begin data.
-36 1
-39 1
-43 1
-38 1
-37 1
-46 2
-47 2
-47 2
-47 2
-43 2
-40 3
-50 3
-44 3
-48 3
-50 3
-45 4
-53 4
-56 4
-52 4
-56 4
-end data.
-
-
-oneway pigmentation by family
- /statistics = descriptives
- /posthoc = lsd bonferroni alpha (0.05)
- .
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-pig.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Descriptives
-,family,N,Mean,Std. Deviation,Std. Error,95% Confidence Interval for Mean,,Minimum,Maximum
-,,,,,,Lower Bound,Upper Bound,,
-pigmentation,1.000,5,38.600,2.702,1.208,35.245,41.955,36.000,43.000
-,2.000,5,46.000,1.732,.775,43.849,48.151,43.000,47.000
-,3.000,5,46.400,4.336,1.939,41.016,51.784,40.000,50.000
-,4.000,5,52.400,4.506,2.015,46.806,57.994,45.000,56.000
-,Total,20,45.850,5.967,1.334,43.057,48.643,36.000,56.000
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-pigmentation,Between Groups,478.950,3,159.650,12.927,.000
-,Within Groups,197.600,16,12.350,,
-,Total,676.550,19,,,
-
-Table: Multiple Comparisons (pigmentation)
-,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
-,,,,,,Lower Bound,Upper Bound
-LSD,1.000,2.000,-7.400,2.223,.004,-12.112,-2.688
-,,3.000,-7.800,2.223,.003,-12.512,-3.088
-,,4.000,-13.800,2.223,.000,-18.512,-9.088
-,2.000,1.000,7.400,2.223,.004,2.688,12.112
-,,3.000,-.400,2.223,.859,-5.112,4.312
-,,4.000,-6.400,2.223,.011,-11.112,-1.688
-,3.000,1.000,7.800,2.223,.003,3.088,12.512
-,,2.000,.400,2.223,.859,-4.312,5.112
-,,4.000,-6.000,2.223,.016,-10.712,-1.288
-,4.000,1.000,13.800,2.223,.000,9.088,18.512
-,,2.000,6.400,2.223,.011,1.688,11.112
-,,3.000,6.000,2.223,.016,1.288,10.712
-Bonferroni,1.000,2.000,-7.400,2.223,.025,-14.086,-.714
-,,3.000,-7.800,2.223,.017,-14.486,-1.114
-,,4.000,-13.800,2.223,.000,-20.486,-7.114
-,2.000,1.000,7.400,2.223,.025,.714,14.086
-,,3.000,-.400,2.223,1.000,-7.086,6.286
-,,4.000,-6.400,2.223,.065,-13.086,.286
-,3.000,1.000,7.800,2.223,.017,1.114,14.486
-,,2.000,.400,2.223,1.000,-6.286,7.086
-,,4.000,-6.000,2.223,.095,-12.686,.686
-,4.000,1.000,13.800,2.223,.000,7.114,20.486
-,,2.000,6.400,2.223,.065,-.286,13.086
-,,3.000,6.000,2.223,.095,-.686,12.686
-])
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY posthoc Tukey HSD and Games-Howell])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([oneway-tukey.sps],[dnl
-set format = f11.3.
-data list notable list /libido * dose *.
-begin data.
-3 0
-2 0
-1 0
-1 0
-4 0
-5 1
-2 1
-4 1
-2 1
-3 1
-7 2
-4 2
-5 2
-3 2
-6 2
-end data.
-
-variable label dose 'Dose of Viagra'.
-
-add value labels dose 0 'Placebo' 1 '1 Dose' 2 '2 Doses'.
-
-oneway libido by dose
- /posthoc tukey gh.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-tukey.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-libido,Between Groups,20.133,2,10.067,5.119,.025
-,Within Groups,23.600,12,1.967,,
-,Total,43.733,14,,,
-
-Table: Multiple Comparisons (libido)
-,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
-,,,,,,Lower Bound,Upper Bound
-Tukey HSD,Placebo,1 Dose,-1.000,.887,.516,-3.366,1.366
-,,2 Doses,-2.800,.887,.021,-5.166,-.434
-,1 Dose,Placebo,1.000,.887,.516,-1.366,3.366
-,,2 Doses,-1.800,.887,.147,-4.166,.566
-,2 Doses,Placebo,2.800,.887,.021,.434,5.166
-,,1 Dose,1.800,.887,.147,-.566,4.166
-Games-Howell,Placebo,1 Dose,-1.000,.887,.479,-3.356,1.356
-,,2 Doses,-2.800,.887,.039,-5.439,-.161
-,1 Dose,Placebo,1.000,.887,.479,-1.356,3.356
-,,2 Doses,-1.800,.887,.185,-4.439,.839
-,2 Doses,Placebo,2.800,.887,.039,.161,5.439
-,,1 Dose,1.800,.887,.185,-.839,4.439
-])
-
-AT_CLEANUP
-
-AT_SETUP([ONEWAY posthoc Sidak])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([oneway-sidak.sps],[dnl
-SET FORMAT F20.4.
-
-DATA LIST notable LIST /program score.
-BEGIN DATA.
-1 9
-1 12
-1 14
-1 11
-1 13
-2 10
-2 6
-2 9
-2 9
-2 10
-3 12
-3 14
-3 11
-3 13
-3 11
-4 9
-4 8
-4 11
-4 7
-4 8
-END DATA.
-
-ONEWAY
- score BY program
- /MISSING ANALYSIS
- /POSTHOC = SIDAK.
-])
-
-AT_CHECK([pspp -O format=csv oneway-sidak.sps], [0],
-[Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-score,Between Groups,54.9500,3,18.3167,7.0449,.003
-,Within Groups,41.6000,16,2.6000,,
-,Total,96.5500,19,,,
-
-Table: Multiple Comparisons (score)
-,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
-,,,,,,Lower Bound,Upper Bound
-Šidák,1.0000,2.0000,3.0000,1.0198,.056,-.0575,6.0575
-,,3.0000,-.4000,1.0198,.999,-3.4575,2.6575
-,,4.0000,3.2000,1.0198,.038,.1425,6.2575
-,2.0000,1.0000,-3.0000,1.0198,.056,-6.0575,.0575
-,,3.0000,-3.4000,1.0198,.025,-6.4575,-.3425
-,,4.0000,.2000,1.0198,1.000,-2.8575,3.2575
-,3.0000,1.0000,.4000,1.0198,.999,-2.6575,3.4575
-,,2.0000,3.4000,1.0198,.025,.3425,6.4575
-,,4.0000,3.6000,1.0198,.017,.5425,6.6575
-,4.0000,1.0000,-3.2000,1.0198,.038,-6.2575,-.1425
-,,2.0000,-.2000,1.0198,1.000,-3.2575,2.8575
-,,3.0000,-3.6000,1.0198,.017,-6.6575,-.5425
-])
-
-AT_CLEANUP
-
-AT_SETUP([ONEWAY posthoc Scheffe])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([oneway-scheffe.sps],[dnl
-set format = f11.3.
-data list notable list /usage * group *.
-begin data.
-21.00 1
-19.00 1
-18.00 1
-25.00 1
-14.00 1
-13.00 1
-24.00 1
-19.00 1
-20.00 1
-21.00 1
-15.00 2
-10.00 2
-13.00 2
-16.00 2
-14.00 2
-24.00 2
-16.00 2
-14.00 2
-18.00 2
-16.00 2
-10.00 3
- 7.00 3
-13.00 3
-20.00 3
- .00 3
- 8.00 3
- 6.00 3
- 1.00 3
-12.00 3
-14.00 3
-18.00 4
-15.00 4
- 3.00 4
-27.00 4
- 6.00 4
-14.00 4
-13.00 4
-11.00 4
- 9.00 4
-18.00 4
-end data.
-
-variable label usage 'Days of Use'.
-
-add value labels group 0 'none' 1 'one' 2 'two' 3 'three' 4 'four'.
-
-oneway usage by group
- /posthoc scheffe.
-])
-
-AT_CHECK([pspp -O format=csv oneway-scheffe.sps], [0],
-[Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-Days of Use,Between Groups,555.275,3,185.092,6.663,.001
-,Within Groups,1000.100,36,27.781,,
-,Total,1555.375,39,,,
-
-Table: Multiple Comparisons (Days of Use)
-,(J) Family,(J) Family,Mean Difference (I - J),Std. Error,Sig.,95% Confidence Interval,
-,,,,,,Lower Bound,Upper Bound
-Scheffé,one,two,3.800,2.357,.467,-3.112,10.712
-,,three,10.300,2.357,.001,3.388,17.212
-,,four,6.000,2.357,.110,-.912,12.912
-,two,one,-3.800,2.357,.467,-10.712,3.112
-,,three,6.500,2.357,.072,-.412,13.412
-,,four,2.200,2.357,.832,-4.712,9.112
-,three,one,-10.300,2.357,.001,-17.212,-3.388
-,,two,-6.500,2.357,.072,-13.412,.412
-,,four,-4.300,2.357,.358,-11.212,2.612
-,four,one,-6.000,2.357,.110,-12.912,.912
-,,two,-2.200,2.357,.832,-9.112,4.712
-,,three,4.300,2.357,.358,-2.612,11.212
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY bad contrast count])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([oneway-bad-contrast.sps],[dnl
-DATA LIST NOTABLE LIST /height * weight * temperature * sex *.
-BEGIN DATA.
-1884 88.6 39.97 0
-1801 90.9 39.03 0
-1801 91.7 38.98 0
-1607 56.3 36.26 1
-1608 46.3 46.26 1
-1607 55.9 37.84 1
-1604 56.6 36.81 1
-1606 56.1 34.56 1
-END DATA.
-
-ONEWAY /VARIABLES= height weight temperature BY sex
- /CONTRAST = -1 1
- /CONTRAST = -3 3
- /CONTRAST = 2 -2 1
- /CONTRAST = -9 9
- .
-])
-
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt oneway-bad-contrast.sps], [0], [dnl
-oneway-bad-contrast.sps:18: warning: ONEWAY: In contrast list 3, the number of coefficients (3) does not equal the number of groups (2). This contrast list will be ignored.
-])
-AT_CHECK([cat pspp.csv], [0], [dnl
-"oneway-bad-contrast.sps:18: warning: ONEWAY: In contrast list 3, the number of coefficients (3) does not equal the number of groups (2). This contrast list will be ignored."
-
-Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-height,Between Groups,92629.63,1,92629.63,120.77,.000
-,Within Groups,4601.87,6,766.98,,
-,Total,97231.50,7,,,
-weight,Between Groups,2451.65,1,2451.65,174.59,.000
-,Within Groups,84.25,6,14.04,,
-,Total,2535.90,7,,,
-temperature,Between Groups,1.80,1,1.80,.13,.733
-,Within Groups,84.55,6,14.09,,
-,Total,86.36,7,,,
-
-Table: Contrast Coefficients
-Contrast,sex,
-,.00,1.00
-1,-1,1
-2,-3,3
-3,-9,9
-
-Table: Contrast Tests
-,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
-height,Assume equal variances,1,-222.27,20.23,-10.99,6.00,.000
-,,2,-666.80,60.68,-10.99,6.00,.000
-,,3,-2000.40,182.03,-10.99,6.00,.000
-,Does not assume equal variances,1,-222.27,27.67,-8.03,2.00,.015
-,,2,-666.80,83.02,-8.03,2.00,.015
-,,3,-2000.40,249.07,-8.03,2.00,.015
-weight,Assume equal variances,1,-36.16,2.74,-13.21,6.00,.000
-,,2,-108.48,8.21,-13.21,6.00,.000
-,,3,-325.44,24.63,-13.21,6.00,.000
-,Does not assume equal variances,1,-36.16,2.19,-16.48,5.42,.000
-,,2,-108.48,6.58,-16.48,5.42,.000
-,,3,-325.44,19.75,-16.48,5.42,.000
-temperature,Assume equal variances,1,-.98,2.74,-.36,6.00,.733
-,,2,-2.94,8.22,-.36,6.00,.733
-,,3,-8.83,24.67,-.36,6.00,.733
-,Does not assume equal variances,1,-.98,2.07,-.47,4.19,.660
-,,2,-2.94,6.22,-.47,4.19,.660
-,,3,-8.83,18.66,-.47,4.19,.660
-])
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY crash on single category independent variable])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([crash.sps],[
-input program.
-loop #i = 1 to 10.
-compute test = #i.
-end case.
-end loop.
-end file.
-end input program.
-
-compute x = 1.
-
-oneway test by x.
-])
-
-AT_CHECK([pspp -O format=csv crash.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([ONEWAY crash on missing dependent variable])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([crash2.sps],[dnl
-data list notable list /dv1 * dv2 * y * .
-begin data.
-2 . 2
-1 . 2
-1 . 1
-2 . 4
-3 . 4
-4 . 4
-5 . 4
-end data.
-
-ONEWAY
- /VARIABLES= dv1 dv2 BY y
- /STATISTICS = DESCRIPTIVES
- /POSTHOC = BONFERRONI LSD SCHEFFE SIDAK TUKEY
- /MISSING = ANALYSIS
- .
-])
-
-AT_CHECK([pspp -O format=csv crash2.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-
-
-AT_SETUP([ONEWAY Games-Howell test with few cases])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([crash3.sps],[dnl
-data list notable list /dv * y * .
-begin data.
-2 2
-1 2
-1 1
-2 4
-3 4
-end data.
-
-ONEWAY
- /VARIABLES= dv BY y
- /POSTHOC = GH
- .
-])
-
-AT_CHECK([pspp -O format=csv crash3.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY Crash on empty data])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([crash4.sps],[dnl
-DATA LIST NOTABLE LIST /height * weight * temperature * sex *.
-BEGIN DATA.
-1801 . . 0
-1606 . . 1
-END DATA.
-
-ONEWAY /VARIABLES= height weight temperature BY sex
- /CONTRAST = -1 1
- /CONTRAST = -3 3
- /CONTRAST = 2 -2 1
- /CONTRAST = -9 9
- .
-])
-
-AT_CHECK([pspp -O format=csv crash4.sps], [0], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([ONEWAY Crash on invalid dependent variable])
-AT_KEYWORDS([categorical categoricals])
-AT_DATA([crash5.sps],[dnl
-data list notable list /a * b *.
-begin data.
-3 0
-2 0
-6 2
-end data.
-
-oneway a"by b.
-
-])
-
-AT_CHECK([pspp -O format=csv crash5.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-
-AT_SETUP([ONEWAY Crash on unterminated string])
-AT_KEYWORDS([categorical categoricals])
-
-AT_DATA([crash6.sps], [dnl
-DATA LIST NOTABLE LIST /height * weight * temperature * sex *.
-BEGIN DATA.
-1801 . . 0
-1606 . 0 . 1
-END DATA.
-
-ONEWAY /VARIABLES= height weight temperature BY sex
- /CONTRAST =" 2 -2 1
- .
-])
-
-AT_CHECK([pspp -O format=csv crash6.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([ONEWAY contrast bug])
-
-AT_KEYWORDS([categorical categoricals])
-
-
-
-dnl this example comes from: https://case.truman.edu/files/2015/06/SPSS-One-Way-ANOVA.pdf
-AT_DATA([contrasts.sps],
-[
-SET FORMAT=F10.3.
-
-DATA LIST notable LIST /relieftime drugs *.
-begin data.
-12 0
-15 0
-18 0
-16 0
-20 0
-20 1
-21 1
-22 1
-19 1
-20 1
-17 2
-16 2
-19 2
-15 2
-19 2
-14 3
-13 3
-12 3
-14 3
-11 3
-end data.
-
-ONEWAY relieftime by drugs
- /CONTRAST 3 -1 -1 -1
- /CONTRAST 0 2 -1 -1
- /CONTRAST 0 0 1 -1
- .
-])
-
-AT_CHECK([pspp -O format=csv contrasts.sps], [0], [Table: ANOVA
-,,Sum of Squares,df,Mean Square,F,Sig.
-relieftime,Between Groups,146.950,3,48.983,12.723,.000
-,Within Groups,61.600,16,3.850,,
-,Total,208.550,19,,,
-
-Table: Contrast Coefficients
-Contrast,drugs,,,
-,.000,1.000,2.000,3.000
-1,3,-1,-1,-1
-2,0,2,-1,-1
-3,0,0,1,-1
-
-Table: Contrast Tests
-,,Contrast,Value of Contrast,Std. Error,t,df,Sig. (2-tailed)
-relieftime,Assume equal variances,1,-1.800,3.040,-.592,16.000,.562
-,,2,10.800,2.149,5.025,16.000,.000
-,,3,4.400,1.241,3.546,16.000,.003
-,Does not assume equal variances,1,-1.800,4.219,-.427,4.611,.689
-,,2,10.800,1.421,7.599,10.158,.000
-,,3,4.400,.990,4.445,7.315,.003
-])
-AT_CLEANUP
-
-AT_SETUP([ONEWAY syntax errors])
-AT_DATA([oneway.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-ONEWAY/ **.
-ONEWAY **.
-ONEWAY x **.
-ONEWAY x BY **.
-ONEWAY x BY y/STATISTICS=**.
-ONEWAY x BY y/POSTHOC=ALPHA **.
-ONEWAY x BY y/POSTHOC=ALPHA(**).
-ONEWAY x BY y/POSTHOC=ALPHA(123 **).
-ONEWAY x BY y/POSTHOC=**.
-ONEWAY x BY y/CONTRAST=**.
-ONEWAY x BY y/MISSING=**.
-ONEWAY x BY y/ **.
-])
-AT_CHECK([pspp -O format=csv oneway.sps], [1], [dnl
-"oneway.sps:2.9-2.10: error: ONEWAY: Syntax error expecting VARIABLES.
- 2 | ONEWAY/ **.
- | ^~"
-
-"oneway.sps:3.8-3.9: error: ONEWAY: Syntax error expecting variable name.
- 3 | ONEWAY **.
- | ^~"
-
-"oneway.sps:4.10-4.11: error: ONEWAY: Syntax error expecting `BY'.
- 4 | ONEWAY x **.
- | ^~"
-
-"oneway.sps:5.13-5.14: error: ONEWAY: Syntax error expecting variable name.
- 5 | ONEWAY x BY **.
- | ^~"
-
-"oneway.sps:6.26-6.27: error: ONEWAY: Syntax error expecting DESCRIPTIVES or HOMOGENEITY.
- 6 | ONEWAY x BY y/STATISTICS=**.
- | ^~"
-
-"oneway.sps:7.29-7.30: error: ONEWAY: Syntax error expecting `('.
- 7 | ONEWAY x BY y/POSTHOC=ALPHA **.
- | ^~"
-
-"oneway.sps:8.29-8.30: error: ONEWAY: Syntax error expecting number.
- 8 | ONEWAY x BY y/POSTHOC=ALPHA(**).
- | ^~"
-
-"oneway.sps:9.33-9.34: error: ONEWAY: Syntax error expecting `)'.
- 9 | ONEWAY x BY y/POSTHOC=ALPHA(123 **).
- | ^~"
-
-"oneway.sps:10.23-10.24: error: ONEWAY: Unknown post hoc analysis method.
- 10 | ONEWAY x BY y/POSTHOC=**.
- | ^~"
-
-"oneway.sps:11.24-11.25: error: ONEWAY: Syntax error expecting number.
- 11 | ONEWAY x BY y/CONTRAST=**.
- | ^~"
-
-"oneway.sps:12.23-12.24: error: ONEWAY: Syntax error expecting INCLUDE, EXCLUDE, LISTWISE, or ANALYSIS.
- 12 | ONEWAY x BY y/MISSING=**.
- | ^~"
-
-"oneway.sps:13.16-13.17: error: ONEWAY: Syntax error expecting STATISTICS, POSTHOC, CONTRAST, or MISSING.
- 13 | ONEWAY x BY y/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([QUICK CLUSTER])
-
-AT_SETUP([QUICK CLUSTER with small data set])
-AT_DATA([quick-cluster.sps], [dnl
-DATA LIST LIST /x y z.
-BEGIN DATA.
-22,2930,4099
-17,3350,4749
-22,2640,3799
-20, 3250,4816
-15,4080,7827
-4,5,4
-5,6,5
-6,7,6
-7,8,7
-8,9,8
-9,10,9
-END DATA.
-QUICK CLUSTER x y z
- /CRITERIA=CLUSTER(2) MXITER(20).
-])
-AT_CHECK([pspp -o pspp.csv quick-cluster.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-y,F8.0
-z,F8.0
-
-Table: Final Cluster Centers
-,Cluster,
-,1,2
-x,6.50,19.20
-y,7.50,3250.00
-z,6.50,5058.00
-
-Table: Number of Cases in each Cluster
-,,Count
-Cluster,1,6
-,2,5
-Valid,,11
-])
-AT_CLEANUP
-
-AT_SETUP([QUICK CLUSTER with large data set])
-AT_KEYWORDS([slow])
-AT_DATA([quick-cluster.sps], [dnl
-input program.
-loop #i = 1 to 50000.
-compute x = 3.
-end case.
-end loop.
-end file.
-end input program.
-QUICK CLUSTER x /CRITERIA = CLUSTER(4) NOINITIAL.
-])
-AT_CHECK([pspp -o pspp.csv quick-cluster.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Final Cluster Centers
-,Cluster,,,
-,1,2,3,4
-x,NaN,NaN,NaN,3.00
-
-Table: Number of Cases in each Cluster
-,,Count
-Cluster,1,0
-,2,0
-,3,0
-,4,50000
-Valid,,50000
-])
-AT_CLEANUP
-
-
-AT_SETUP([QUICK CLUSTER with weights])
-AT_DATA([qc-weighted.sps], [dnl
-input program.
-loop #i = 1 to 400.
- compute x = mod (#i, 4).
- compute w = 5.
- end case.
-end loop.
-loop #i = 1 to 400.
- compute x = mod (#i, 4).
- compute w = 3.
- end case.
-end loop.
-end file.
-end input program.
-
-weight by w.
-
-QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
-])
-
-AT_CHECK([pspp -o pspp-w.csv qc-weighted.sps])
-
-
-AT_DATA([qc-unweighted.sps], [dnl
-input program.
-loop #i = 1 to 3200.
- compute x = mod (#i, 4).
- end case.
-end loop.
-end file.
-end input program.
-
-QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
-])
-
-AT_CHECK([pspp -o pspp-unw.csv qc-unweighted.sps])
-
-AT_CHECK([diff pspp-w.csv pspp-unw.csv], [0])
-
-AT_CLEANUP
-
-AT_SETUP([QUICK CLUSTER with listwise missing])
-AT_DATA([quick-miss.sps], [dnl
-data list notable list /x *.
-begin data.
-1
-1
-2
-3
-4
-.
-2
-end data.
-
-QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
-])
-
-AT_CHECK([pspp -o pspp-m.csv quick-miss.sps])
-
-AT_DATA([quick-nmiss.sps], [dnl
-data list notable list /x *.
-begin data.
-1
-1
-2
-3
-4
-2
-end data.
-
-QUICK CLUSTER x /CRITERIA = CLUSTER(4) MXITER (10).
-])
-
-AT_CHECK([pspp -o pspp-nm.csv quick-nmiss.sps])
-
-AT_CHECK([diff pspp-m.csv pspp-nm.csv], [0])
-
-AT_CLEANUP
-
-
-AT_SETUP([QUICK CLUSTER with pairwise missing])
-
-dnl This test runs two programs, which are identical except that one
-dnl has an extra case with one missing value. Because the syntax uses
-dnl NOINITIAL and NOUPDATE, the results should be identical except for
-dnl the final classification.
-
-AT_DATA([quick-s.sps], [dnl
-data list notable list /x * y *.
-begin data.
-1 2
-1 2.2
-1.1 1.9
-1 9
-1 10
-1.3 9.5
-0.9 8.9
-3.5 2
-3.4 3
-3.5 2.5
-3.1 2.0
-end data.
-
-QUICK CLUSTER x y
- /PRINT = INITIAL
- /CRITERIA = CLUSTER(3) NOINITIAL NOUPDATE
- .
-])
-
-AT_CHECK([pspp -O format=csv quick-s.sps > pspp-s.csv])
-
-AT_DATA([quick-pw.sps], [dnl
-data list notable list /x * y *.
-begin data.
-1 2
-1 2.2
-1.1 1.9
-1 9
-1 10
-1.3 9.5
-0.9 8.9
-3.5 2
-3.4 3
-3.5 2.5
-3.1 2.0
-. 2.3
-end data.
-
-QUICK CLUSTER x y
- /CRITERIA = CLUSTER(3) NOINITIAL NOUPDATE
- /PRINT = INITIAL
- /MISSING = PAIRWISE
- .
-])
-
-AT_CHECK([pspp -O format=csv quick-pw.sps > pspp-pw.csv])
-
-AT_CHECK([head -n 13 pspp-s.csv > top-s.csv])
-AT_CHECK([head -n 13 pspp-pw.csv > top-pw.csv])
-AT_CHECK([diff top-s.csv top-pw.csv])
-
-
-AT_CHECK([grep Valid pspp-s.csv], [0], [Valid,,11
-])
-
-AT_CHECK([grep Valid pspp-pw.csv], [0], [Valid,,12
-])
-
-
-AT_CLEANUP
-
-
-
-AT_SETUP([QUICK CLUSTER crash on bad cluster quantity])
-AT_DATA([badn.sps], [dnl
-data list notable list /x * y *.
-begin data.
-1 2
-1 2.2
-end data.
-
-QUICK CLUSTER x y
- /CRITERIA = CLUSTER(0)
- .
-])
-
-AT_CHECK([pspp -O format=csv badn.sps], [1], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([QUICK CLUSTER infinite loop on bad command name])
-AT_DATA([quick-cluster.sps], [dnl
-data list notable list /x y.
-begin data.
-1 2
-1 2.2
-end data.
-
-QUICK CLUSTER x y /UNSUPPORTED.
-])
-AT_CHECK([pspp -O format=csv quick-cluster.sps], [1], [dnl
-"quick-cluster.sps:7.20-7.30: error: QUICK CLUSTER: Syntax error expecting MISSING, PRINT, SAVE, or CRITERIA.
- 7 | QUICK CLUSTER x y /UNSUPPORTED.
- | ^~~~~~~~~~~"
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([QUICK CLUSTER /PRINT subcommand])
-AT_DATA([quick-cluster.sps], [dnl
-data list notable list /cluster (A8) x y (F8.0).
-begin data.
-A 10.45 9.38
-A 10.67 9.17
-A 10.86 9.63
-A 8.77 8.45
-A 8.04 11.77
-A 10.34 9.83
-A 10.37 10.54
-A 11.49 8.18
-A 10.17 11.10
-A 11.37 9.16
-A 10.25 8.83
-A 8.69 9.92
-A 10.36 10.39
-A 10.89 10.51
-A 9.9 11.39
-A 11.1 10.91
-A 11.77 8.47
-A 9.5 10.46
-B -11.01 -9.21
-B -10.8 -11.76
-B -10.03 -10.29
-B -9.54 -9.17
-B -10.16 -9.82
-B -10.01 -8.63
-B -9.6 -10.22
-B -11.36 -10.93
-B -10.63 -10.97
-B -9.53 -10.78
-B -9.40 -10.26
-B -10.76 -9.76
-B -9.9 -10.11
-B -10.16 -9.75
-B -8.65 -11.31
-B -10.10 -10.90
-B -11.67 -9.89
-B -11.11 -9.23
-B -8.7 -8.43
-B -11.35 -8.68
-C -10.20 9.00
-C -10.12 9.92
-C -10.41 10.16
-C -9.86 10.12
-C -10.31 10.12
-C -9.57 10.16
-C -9.69 9.93
-C -9.14 10.84
-C -9.8 10.19
-C -9.97 10.22
-C -11.65 10.81
-C -9.80 11.39
-C -10.31 10.74
-C -10.26 10.38
-C -11.57 10.02
-C -10.50 9.75
-C -9.06 9.63
-C -10.17 10.82
-C -10.22 9.99
-end data.
-
-QUICK CLUSTER x y
- /CRITERIA=CLUSTERS(3)
- /PRINT=INITIAL CLUSTER.
-])
-
-AT_CHECK([pspp -O format=csv quick-cluster.sps], [0], [dnl
-Table: Initial Cluster Centers
-,Cluster,,
-,1,2,3
-x,-11,-12,11
-y,-12,11,11
-
-Table: Final Cluster Centers
-,Cluster,,
-,1,2,3
-x,-10,-10,10
-y,-10,10,10
-
-Table: Number of Cases in each Cluster
-,,Count
-Cluster,1,20
-,2,19
-,3,18
-Valid,,57
-
-Table: Cluster Membership
-Case Number,Cluster
-1,3
-2,3
-3,3
-4,3
-5,3
-6,3
-7,3
-8,3
-9,3
-10,3
-11,3
-12,3
-13,3
-14,3
-15,3
-16,3
-17,3
-18,3
-19,1
-20,1
-21,1
-22,1
-23,1
-24,1
-25,1
-26,1
-27,1
-28,1
-29,1
-30,1
-31,1
-32,1
-33,1
-34,1
-35,1
-36,1
-37,1
-38,1
-39,2
-40,2
-41,2
-42,2
-43,2
-44,2
-45,2
-46,2
-47,2
-48,2
-49,2
-50,2
-51,2
-52,2
-53,2
-54,2
-55,2
-56,2
-57,2
-])
-
-AT_CLEANUP
-
-
-dnl Test for a crash which happened on bad input syntax
-AT_SETUP([QUICK CLUSTER -- Empty Parentheses])
-
-AT_DATA([empty-parens.sps], [dnl
-data list notable list /x * y *.
-begin data.
-1 2
-1 2.2
-end data.
-
-QUICK CLUSTER x y
- /CRITERIA = CONVERGE()
- .
-])
-
-AT_CHECK([pspp -o pspp.csv empty-parens.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([QUICK CLUSTER with save])
-AT_DATA([quick-cluster.sps], [dnl
-DATA LIST notable LIST /x y z.
-BEGIN DATA.
-22,2930,4099
-17,3350,4749
-22,2640,3799
-20, 3250,4816
-15,4080,7827
-4,5,4
-5,6,5
-6,7,6
-7,8,7
-8,9,8
-9,10,9
-END DATA.
-QUICK CLUSTER x y z
- /CRITERIA=CLUSTER(2) MXITER(20)
- /SAVE = CLUSTER (cluster) DISTANCE (distance).
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv quick-cluster.sps], [0], [dnl
-Table: Final Cluster Centers
-,Cluster,
-,1,2
-x,6.50,19.20
-y,7.50,3250.00
-z,6.50,5058.00
-
-Table: Number of Cases in each Cluster
-,,Count
-Cluster,1,6
-,2,5
-Valid,,11
-
-Table: Data List
-x,y,z,cluster,distance
-22.00,2930.00,4099.00,2.00,1010.98
-17.00,3350.00,4749.00,2.00,324.79
-22.00,2640.00,3799.00,2.00,1399.00
-20.00,3250.00,4816.00,2.00,242.00
-15.00,4080.00,7827.00,2.00,2890.72
-4.00,5.00,4.00,1.00,4.33
-5.00,6.00,5.00,1.00,2.60
-6.00,7.00,6.00,1.00,.87
-7.00,8.00,7.00,1.00,.87
-8.00,9.00,8.00,1.00,2.60
-9.00,10.00,9.00,1.00,4.33
-])
-AT_CLEANUP
-
-
-AT_SETUP([QUICK CLUSTER with single save])
-AT_DATA([quick-cluster.sps], [dnl
-DATA LIST notable LIST /x y z.
-BEGIN DATA.
-22,2930,4099
-17,3350,4749
-22,2640,3799
-20, 3250,4816
-15,4080,7827
-4,5,4
-5,6,5
-6,7,6
-7,8,7
-8,9,8
-9,10,9
-END DATA.
-QUICK CLUSTER x y z
- /CRITERIA=CLUSTER(2) MXITER(20)
- /SAVE = DISTANCE.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv quick-cluster.sps], [0], [dnl
-Table: Final Cluster Centers
-,Cluster,
-,1,2
-x,6.50,19.20
-y,7.50,3250.00
-z,6.50,5058.00
-
-Table: Number of Cases in each Cluster
-,,Count
-Cluster,1,6
-,2,5
-Valid,,11
-
-Table: Data List
-x,y,z,QCL_0
-22.00,2930.00,4099.00,1010.98
-17.00,3350.00,4749.00,324.79
-22.00,2640.00,3799.00,1399.00
-20.00,3250.00,4816.00,242.00
-15.00,4080.00,7827.00,2890.72
-4.00,5.00,4.00,4.33
-5.00,6.00,5.00,2.60
-6.00,7.00,6.00,.87
-7.00,8.00,7.00,.87
-8.00,9.00,8.00,2.60
-9.00,10.00,9.00,4.33
-])
-AT_CLEANUP
-
-
-dnl This one was noticed to crash at one point.
-AT_SETUP([QUICK CLUSTER crash on bizarre input])
-AT_DATA([badn.sps], [dnl
-data list notable list /x.
-begin da\a*
-22
-17
-22
-22
-15
-4,
-5,
-6,
-7,j8,
-9,
-end data.
-
-quick cluster x
-" /criteria=cluster(2) mxiter(20)
- /save = distance
- .
-
-list.
-]) dnl "
-
-AT_CHECK([pspp -O format=csv badn.sps], [1], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([QUICK CLUSTER syntax errors])
-AT_DATA([quick-cluster.sps], [dnl
-DATA LIST LIST NOTABLE /x y.
-QUICK CLUSTER **.
-QUICK CLUSTER x/MISSING=**.
-QUICK CLUSTER x/PRINT=**.
-QUICK CLUSTER x/SAVE=CLUSTER(**).
-QUICK CLUSTER x/SAVE=CLUSTER(x).
-QUICK CLUSTER x/SAVE=CLUSTER(c **).
-QUICK CLUSTER x/SAVE=DISTANCE(**).
-QUICK CLUSTER x/SAVE=DISTANCE(x).
-QUICK CLUSTER x/SAVE=DISTANCE(d **).
-QUICK CLUSTER x/SAVE=**.
-QUICK CLUSTER x/CRITERIA=CLUSTERS **.
-QUICK CLUSTER x/CRITERIA=CLUSTERS(**).
-QUICK CLUSTER x/CRITERIA=CLUSTERS(5 **).
-QUICK CLUSTER x/CRITERIA=CONVERGE **.
-QUICK CLUSTER x/CRITERIA=CONVERGE(**).
-QUICK CLUSTER x/CRITERIA=CONVERGE(5 **).
-QUICK CLUSTER x/CRITERIA=**.
-QUICK CLUSTER x/ **.
-])
-AT_CHECK([pspp -O format=csv quick-cluster.sps], [1], [dnl
-"quick-cluster.sps:2.15-2.16: error: QUICK CLUSTER: Syntax error expecting variable name.
- 2 | QUICK CLUSTER **.
- | ^~"
-
-"quick-cluster.sps:3.25-3.26: error: QUICK CLUSTER: Syntax error expecting LISTWISE, DEFAULT, PAIRWISE, INCLUDE, or EXCLUDE.
- 3 | QUICK CLUSTER x/MISSING=**.
- | ^~"
-
-"quick-cluster.sps:4.23-4.24: error: QUICK CLUSTER: Syntax error expecting CLUSTER or INITIAL.
- 4 | QUICK CLUSTER x/PRINT=**.
- | ^~"
-
-"quick-cluster.sps:5.30-5.31: error: QUICK CLUSTER: Syntax error expecting identifier.
- 5 | QUICK CLUSTER x/SAVE=CLUSTER(**).
- | ^~"
-
-"quick-cluster.sps:6.30: error: QUICK CLUSTER: A variable called `x' already exists.
- 6 | QUICK CLUSTER x/SAVE=CLUSTER(x).
- | ^"
-
-"quick-cluster.sps:7.32-7.33: error: QUICK CLUSTER: Syntax error expecting `@:}@'.
- 7 | QUICK CLUSTER x/SAVE=CLUSTER(c **).
- | ^~"
-
-"quick-cluster.sps:8.31-8.32: error: QUICK CLUSTER: Syntax error expecting identifier.
- 8 | QUICK CLUSTER x/SAVE=DISTANCE(**).
- | ^~"
-
-"quick-cluster.sps:9.31: error: QUICK CLUSTER: A variable called `x' already exists.
- 9 | QUICK CLUSTER x/SAVE=DISTANCE(x).
- | ^"
-
-"quick-cluster.sps:10.33-10.34: error: QUICK CLUSTER: Syntax error expecting `@:}@'.
- 10 | QUICK CLUSTER x/SAVE=DISTANCE(d **).
- | ^~"
-
-"quick-cluster.sps:11.22-11.23: error: QUICK CLUSTER: Syntax error expecting CLUSTER or DISTANCE.
- 11 | QUICK CLUSTER x/SAVE=**.
- | ^~"
-
-"quick-cluster.sps:12.35-12.36: error: QUICK CLUSTER: Syntax error expecting `('.
- 12 | QUICK CLUSTER x/CRITERIA=CLUSTERS **.
- | ^~"
-
-"quick-cluster.sps:13.35-13.36: error: QUICK CLUSTER: Syntax error expecting positive integer for CLUSTERS.
- 13 | QUICK CLUSTER x/CRITERIA=CLUSTERS(**).
- | ^~"
-
-"quick-cluster.sps:14.37-14.38: error: QUICK CLUSTER: Syntax error expecting `)'.
- 14 | QUICK CLUSTER x/CRITERIA=CLUSTERS(5 **).
- | ^~"
-
-"quick-cluster.sps:15.35-15.36: error: QUICK CLUSTER: Syntax error expecting `('.
- 15 | QUICK CLUSTER x/CRITERIA=CONVERGE **.
- | ^~"
-
-"quick-cluster.sps:16.35-16.36: error: QUICK CLUSTER: Syntax error expecting positive number for CONVERGE.
- 16 | QUICK CLUSTER x/CRITERIA=CONVERGE(**).
- | ^~"
-
-"quick-cluster.sps:17.37-17.38: error: QUICK CLUSTER: Syntax error expecting `)'.
- 17 | QUICK CLUSTER x/CRITERIA=CONVERGE(5 **).
- | ^~"
-
-"quick-cluster.sps:18.26-18.27: error: QUICK CLUSTER: Syntax error expecting CLUSTERS, CONVERGE, MXITER, NOINITIAL, or NOUPDATE.
- 18 | QUICK CLUSTER x/CRITERIA=**.
- | ^~"
-
-"quick-cluster.sps:19.18-19.19: error: QUICK CLUSTER: Syntax error expecting MISSING, PRINT, SAVE, or CRITERIA.
- 19 | QUICK CLUSTER x/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([RANK])
-
-AT_SETUP([RANK simple case with defaults])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /x (f8).
-BEGIN DATA.
--1
-0
-1
-2
-2
-4
-5
-END DATA.
-
-RANK x.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,Rx,RANK
-
-Table: Data List
-x,Rx
--1,1.000
-0,2.000
-1,3.000
-2,4.500
-2,4.500
-4,6.000
-5,7.000
-])
-AT_CLEANUP
-
-# This checks for regression against a crash reported as bug #38482
-# that occurred when multiple variables were specified without any
-# rank specifications.
-AT_SETUP([RANK multiple variables with defaults])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /x * y * z *.
-BEGIN DATA.
- 1.00 2.00 3.00
- 4.00 5.00 6.00
-END DATA.
-
-RANK ALL.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,Rx,RANK
-y,Ry,RANK
-z,Rz,RANK
-
-Table: Data List
-x,y,z,Rx,Ry,Rz
-1.00,2.00,3.00,1.000,1.000,1.000
-4.00,5.00,6.00,2.000,2.000,2.000
-])
-AT_CLEANUP
-
-AT_SETUP([RANK with RANK, RFRACTION, N])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /a * b *.
-BEGIN DATA.
-0 24
-1 32
-2 31
-2 32
-4 30
-5 29
-6 1
-7 43
-8 .
-9 45
-END DATA.
-
-RANK a b (D)
- /PRINT=YES
- /RANK
- /TIES=HIGH
- /RFRACTION
- /N INTO count
- .
-
-DISPLAY DICTIONARY.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-a,Ra,RANK
-b,Rb,RANK
-a,RFR001,RFRACTION
-b,RFR002,RFRACTION
-a,count,N
-b,Nb,N
-
-Table: Variables
-Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-a,1,,Nominal,Input,8,Right,F8.2,F8.2
-b,2,,Nominal,Input,8,Right,F8.2,F8.2
-Ra,3,RANK of a,Ordinal,Input,8,Right,F9.3,F9.3
-RFR001,4,RFRACTION of a,Ordinal,Input,8,Right,F6.4,F6.4
-count,5,N of a,Scale,Input,8,Right,F6.0,F6.0
-Rb,6,RANK of b,Ordinal,Input,8,Right,F9.3,F9.3
-RFR002,7,RFRACTION of b,Ordinal,Input,8,Right,F6.4,F6.4
-Nb,8,N of b,Scale,Input,8,Right,F6.0,F6.0
-
-Table: Data List
-a,b,Ra,RFR001,count,Rb,RFR002,Nb
-.00,24.00,10.000,1.0000,10,8.000,.8889,9
-1.00,32.00,9.000,.9000,10,4.000,.4444,9
-2.00,31.00,8.000,.8000,10,5.000,.5556,9
-2.00,32.00,8.000,.8000,10,4.000,.4444,9
-4.00,30.00,6.000,.6000,10,6.000,.6667,9
-5.00,29.00,5.000,.5000,10,7.000,.7778,9
-6.00,1.00,4.000,.4000,10,9.000,1.0000,9
-7.00,43.00,3.000,.3000,10,2.000,.2222,9
-8.00,. ,2.000,.2000,10,. ,. ,.
-9.00,45.00,1.000,.1000,10,1.000,.1111,9
-])
-AT_CLEANUP
-
-AT_SETUP([RANK with SAVAGE, PERCENT, PROPORTION, NTILES])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /a * b *.
-BEGIN DATA.
-0 24
-1 32
-2 31
-2 32
-4 30
-5 29
-6 1
-7 43
-8 8
-9 45
-END DATA.
-
-RANK a
- /PRINT=YES
- /TIES=CONDENSE
- /SAVAGE
- /PERCENT
- /PROPORTION
- /NTILES(4)
- /NORMAL
-.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function,Fraction
-a,Sa,SAVAGE,
-a,Pa,PERCENT,
-a,PRO001,PROPORTION,BLOM
-a,Na,NTILES,
-a,NOR001,NORMAL,BLOM
-
-Table: Data List
-a,b,Sa,Pa,PRO001,Na,NOR001
-.00,24.00,-.9000,10.00,.0610,1,-1.547
-1.00,32.00,-.7889,20.00,.1585,1,-1.000
-2.00,31.00,-.5925,30.00,.2561,2,-.6554
-2.00,32.00,-.5925,30.00,.2561,2,-.6554
-4.00,30.00,-.3544,40.00,.3537,2,-.3755
-5.00,29.00,-.1544,50.00,.4512,2,-.1226
-6.00,1.00,.0956,60.00,.5488,3,.1226
-7.00,43.00,.4290,70.00,.6463,3,.3755
-8.00,8.00,.9290,80.00,.7439,3,.6554
-9.00,45.00,1.9290,90.00,.8415,4,1.0005
-])
-AT_CLEANUP
-
-AT_SETUP([RANK with SPLIT FILE])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /a * g1 g2 *.
-BEGIN DATA.
-2 1 2
-2 1 2
-3 1 2
-4 1 2
-5 1 2
-1 0 2
-2 0 2
-3 0 2
-4 0 2
-5 0 2
-6 0 2
-7 0 2
-8 0 2
-6 1 2
-7 1 2
-7 1 2
-8 1 2
-9 1 1
-END DATA.
-
-RANK a (D) BY g2 g1
- /PRINT=YES
- /TIES=LOW
- /MISSING=INCLUDE
- /FRACTION=RANKIT
- /RANK
- /NORMAL
- .
-
-SPLIT FILE BY g1.
-
-RANK a (D) BY g2
- /PRINT=YES
- /TIES=LOW
- /MISSING=INCLUDE
- /FRACTION=RANKIT
- /RANK
- /NORMAL
- .
-
-SPLIT FILE OFF.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function,Fraction,Grouping Variables
-a,Ra,RANK,,g2 g1
-a,Na,NORMAL,RANKIT,g2 g1
-
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function,Fraction,Grouping Variables
-a,RAN001,RANK,,g2
-a,NOR001,NORMAL,RANKIT,g2
-
-Table: Data List
-a,g1,g2,Ra,Na,RAN001,NOR001
-2.00,1.00,2.00,8.000,.9674,4.000,.5244
-2.00,1.00,2.00,8.000,.9674,4.000,.5244
-3.00,1.00,2.00,7.000,.5895,3.000,.0000
-4.00,1.00,2.00,6.000,.2822,2.000,-.5244
-5.00,1.00,2.00,5.000,.0000,1.000,-1.282
-1.00,.00,2.00,8.000,1.5341,8.000,1.5341
-2.00,.00,2.00,7.000,.8871,7.000,.8871
-3.00,.00,2.00,6.000,.4888,6.000,.4888
-4.00,.00,2.00,5.000,.1573,5.000,.1573
-5.00,.00,2.00,4.000,-.1573,4.000,-.1573
-6.00,.00,2.00,3.000,-.4888,3.000,-.4888
-7.00,.00,2.00,2.000,-.8871,2.000,-.8871
-8.00,.00,2.00,1.000,-1.534,1.000,-1.534
-6.00,1.00,2.00,4.000,-.2822,4.000,1.1503
-7.00,1.00,2.00,2.000,-.9674,2.000,-.3186
-7.00,1.00,2.00,2.000,-.9674,2.000,-.3186
-8.00,1.00,2.00,1.000,-1.593,1.000,-1.150
-9.00,1.00,1.00,1.000,.0000,1.000,.0000
-])
-AT_CLEANUP
-
-# Also tests small ranks for special case of SAVAGE ranks.
-AT_SETUP([RANK with fractional ranks])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /a * w * .
-BEGIN DATA.
-1 1.5
-2 0.2
-3 0.1
-4 1
-5 1
-6 1
-7 1
-8 1
-END DATA.
-
-WEIGHT BY w.
-
-RANK a
- /FRACTION=TUKEY
- /PROPORTION
- /SAVAGE
-.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function,Fraction
-a,Pa,PROPORTION,TUKEY
-a,Sa,SAVAGE,
-
-Table: Data List
-a,w,Pa,Sa
-1.00,1.50,.1285,-.8016
-2.00,.20,.1776,-.6905
-3.00,.10,.1986,-.6905
-4.00,1.00,.3458,-.5305
-5.00,1.00,.4860,-.2905
-6.00,1.00,.6262,.0262
-7.00,1.00,.7664,.4929
-8.00,1.00,.9065,1.3929
-])
-AT_CLEANUP
-
-AT_SETUP([RANK all-ties due to tiny weights])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /x * w *.
-BEGIN DATA.
-1 0.1
-2 0.1
-3 0.1
-4 0.2
-5 0.1
-6 0.1
-7 0.1
-8 0.1
-END DATA.
-
-WEIGHT BY w.
-
-RANK x
- /TIES=low
- /RANK into xl.
-
-
-RANK x
- /TIES=high
- /RANK into xh.
-
-RANK x
- /TIES=condense
- /RANK into xc.
-
-
-* Test VW fraction
-
-RANK x
- /FRACTION=VW
- /NORMAL.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,xl,RANK
-
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,xh,RANK
-
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,xc,RANK
-
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function,Fraction
-x,Nx,NORMAL,VW
-
-Table: Data List
-x,w,xl,xh,xc,Nx
-1.00,.10,.000,.100,1.000,-1.938
-2.00,.10,.100,.200,2.000,-1.412
-3.00,.10,.200,.300,3.000,-1.119
-4.00,.20,.300,.500,4.000,-.8046
-5.00,.10,.500,.600,5.000,-.5549
-6.00,.10,.600,.700,6.000,-.4067
-7.00,.10,.700,.800,7.000,-.2670
-8.00,.10,.800,.900,8.000,-.1323
-])
-AT_CLEANUP
-
-AT_SETUP([RANK and TEMPORARY])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /age (f2) gender (a1).
-BEGIN DATA.
-44 m
-32 f
-43 m
-49 m
-33 f
-35 f
-29 f
-50 m
-42 m
-33 f
-48 m
-END DATA.
-
-TEMPORARY.
-SELECT IF gender = 'm'.
-RANK age /RANK INTO Rm.
-
-TEMPORARY.
-SELECT IF gender = 'f'.
-RANK age /RANK INTO Rf.
-
-LIST.
-])
-AT_CHECK([pspp -O format=csv rank.sps], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-age,Rm,RANK
-
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-age,Rf,RANK
-
-Table: Data List
-age,gender,Rm,Rf
-44,m,3.000,. @&t@
-32,f,. ,2.000
-43,m,2.000,. @&t@
-49,m,5.000,. @&t@
-33,f,. ,3.500
-35,f,. ,5.000
-29,f,. ,1.000
-50,m,6.000,. @&t@
-42,m,1.000,. @&t@
-33,f,. ,3.500
-48,m,4.000,. @&t@
-])
-AT_CLEANUP
-
-AT_SETUP([RANK variable name fallback])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /foo * rfoo * ran003 *.
-BEGIN DATA.
-0 3 2
-1 3 2
-2 3 2
-2 3 2
-4 3 2
-5 3 2
-6 3 2
-7 3 2
-8 3 2
-9 3 2
-END DATA.
-
-RANK foo.
-
-
-DISPLAY DICTIONARY.
-])
-AT_CHECK([pspp -o pspp.csv rank.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-foo,RAN001,RANK
-
-Table: Variables
-Name,Position,Label,Measurement Level,Role,Width,Alignment,Print Format,Write Format
-foo,1,,Nominal,Input,8,Right,F8.2,F8.2
-rfoo,2,,Nominal,Input,8,Right,F8.2,F8.2
-ran003,3,,Nominal,Input,8,Right,F8.2,F8.2
-RAN001,4,RANK of foo,Ordinal,Input,8,Right,F9.3,F9.3
-])
-AT_CLEANUP
-
-AT_SETUP([RANK robust variable name creation])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST notable /x * rx * ran001 TO ran999.
-BEGIN DATA.
-1
-2
-3
-4
-5
-6
-7
-END DATA.
-
-RANK x.
-
-DELETE VAR ran001 TO ran999.
-
-LIST.
-])
-AT_CHECK([pspp -O format=csv rank.sps], [0], [dnl
-"rank.sps:3: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"rank.sps:4: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"rank.sps:5: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"rank.sps:6: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"rank.sps:7: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"rank.sps:8: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-"rank.sps:9: warning: Missing value(s) for all variables from rx onward. These will be filled with the system-missing value or blanks, as appropriate."
-
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,RNKRA01,RANK
-
-Table: Data List
-x,rx,RNKRA01
-1.00,. ,1.000
-2.00,. ,2.000
-3.00,. ,3.000
-4.00,. ,4.000
-5.00,. ,5.000
-6.00,. ,6.000
-7.00,. ,7.000
-])
-AT_CLEANUP
-
-dnl Test for proper behaviour in the face of invalid input.
-AT_SETUP([RANK handling of invalid input])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /x * a (a2).
-BEGIN DATA.
--1 s
-0 s
-1 s
-2 s
-2 s
-4 s
-5 s
-END DATA.
-
-DEBUG XFORM FAIL.
-
-RANK x.
-])
-AT_CHECK([pspp -O format=csv --testing-mode rank.sps], [1], [dnl
-Table: Variables Created by RANK
-Existing Variable,New Variable,Function
-x,Rx,RANK
-
-rank.sps:14: error: RANK: DEBUG XFORM FAIL transformation executed
-])
-AT_CLEANUP
-
-AT_SETUP([RANK syntax errors])
-AT_DATA([rank.sps], [dnl
-DATA LIST LIST NOTABLE /x y z * a b c (a2).
-RANK VARIABLES **.
-RANK **.
-RANK x BY **.
-RANK x/TIES **.
-RANK x/TIES=**.
-RANK x/FRACTION **.
-RANK x/FRACTIO=**.
-RANK x/PRINT **.
-RANK x/PRINT=**.
-RANK x/MISSING **.
-RANK x/MISSING=**.
-RANK x/NTILES **.
-RANK x/NTILES(**).
-RANK x/NTILES(5 **).
-RANK x/ **.
-RANK x/N INTO v w.
-RANK x/N INTO y.
-RANK x y/N INTO v v.
-])
-AT_CHECK([pspp -O format=csv rank.sps], [1], [dnl
-"rank.sps:2.16-2.17: error: RANK: Syntax error expecting `='.
- 2 | RANK VARIABLES **.
- | ^~"
-
-"rank.sps:3.6-3.7: error: RANK: Syntax error expecting variable name.
- 3 | RANK **.
- | ^~"
-
-"rank.sps:4.11-4.12: error: RANK: Syntax error expecting variable name.
- 4 | RANK x BY **.
- | ^~"
-
-"rank.sps:5.13-5.14: error: RANK: Syntax error expecting `='.
- 5 | RANK x/TIES **.
- | ^~"
-
-"rank.sps:6.13-6.14: error: RANK: Syntax error expecting MEAN, LOW, HIGH, or CONDENSE.
- 6 | RANK x/TIES=**.
- | ^~"
-
-"rank.sps:7.17-7.18: error: RANK: Syntax error expecting `='.
- 7 | RANK x/FRACTION **.
- | ^~"
-
-"rank.sps:8.16-8.17: error: RANK: Syntax error expecting BLOM, TUKEY, VW, or RANKIT.
- 8 | RANK x/FRACTIO=**.
- | ^~"
-
-"rank.sps:9.14-9.15: error: RANK: Syntax error expecting `='.
- 9 | RANK x/PRINT **.
- | ^~"
-
-"rank.sps:10.14-10.15: error: RANK: Syntax error expecting YES or NO.
- 10 | RANK x/PRINT=**.
- | ^~"
-
-"rank.sps:11.16-11.17: error: RANK: Syntax error expecting `='.
- 11 | RANK x/MISSING **.
- | ^~"
-
-"rank.sps:12.16-12.17: error: RANK: Syntax error expecting INCLUDE or EXCLUDE.
- 12 | RANK x/MISSING=**.
- | ^~"
-
-"rank.sps:13.15-13.16: error: RANK: Syntax error expecting `('.
- 13 | RANK x/NTILES **.
- | ^~"
-
-"rank.sps:14.15-14.16: error: RANK: Syntax error expecting positive integer for NTILES.
- 14 | RANK x/NTILES(**).
- | ^~"
-
-"rank.sps:15.17-15.18: error: RANK: Syntax error expecting `)'.
- 15 | RANK x/NTILES(5 **).
- | ^~"
-
-"rank.sps:16.9-16.10: error: RANK: Syntax error expecting RANK, NORMAL, RFRACTION, N, SAVAGE, PERCENT, PROPORTION, or NTILES.
- 16 | RANK x/ **.
- | ^~"
-
-"rank.sps:17.15-17.17: error: RANK: Too many variables in INTO clause.
- 17 | RANK x/N INTO v w.
- | ^~~"
-
-"rank.sps:18.15: error: RANK: Variable y already exists.
- 18 | RANK x/N INTO y.
- | ^"
-
-"rank.sps:19.19: error: RANK: Duplicate variable name v.
- 19 | RANK x y/N INTO v v.
- | ^"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([LINEAR REGRESSION])
-
-AT_SETUP([LINEAR REGRESSION - basic])
-AT_DATA([regression.sps], [dnl
-set format = F22.3.
-data list notable list / v0 to v2.
-filter by v0.
-begin data
- 0.65377128 7.735648 -23.97588
--0.13087553 6.142625 -19.63854
- 0.34880368 7.651430 -25.26557
- 0.69249021 6.125125 -16.57090
--0.07368178 8.245789 -25.80001
--0.34404919 6.031540 -17.56743
- 0.75981559 9.832291 -28.35977
--0.46958313 5.343832 -16.79548
--0.06108490 8.838262 -29.25689
- 0.56154863 6.200189 -18.58219
-end data
-regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=pred resid.
-list.
-])
-
-AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
-"regression.sps:16.82-16.96: warning: REGRESSION: REGRESSION with SAVE ignores FILTER. All cases will be processed.
- 16 | regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=pred resid.
- | ^~~~~~~~~~~~~~~"
-
-Table: Model Summary (v2)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.971,.942,.925,1.337
-
-Table: ANOVA (v2)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,202.753,2,101.376,56.754,.000
-Residual,12.504,7,1.786,,
-Total,215.256,9,,,
-
-Table: Coefficients (v2)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),2.191,2.357,.000,.930,.380
-v0,1.813,1.053,.171,1.722,.129
-v1,-3.427,.332,-1.026,-10.334,.000
-
-Table: Data List
-v0,v1,v2,RES1,PRED1
-.654,7.736,-23.976,-.84,-23.13
--.131,6.143,-19.639,-.54,-19.10
-.349,7.651,-25.266,-1.87,-23.40
-.692,6.125,-16.571,.97,-17.54
--.074,8.246,-25.800,.40,-26.20
--.344,6.032,-17.567,1.53,-19.10
-.760,9.832,-28.360,1.77,-30.13
--.470,5.344,-16.795,.18,-16.97
--.061,8.838,-29.257,-1.05,-28.21
-.562,6.200,-18.582,-.54,-18.04
-])
-AT_CLEANUP
-
-
-AT_SETUP([LINEAR REGRESSION - one save])
-AT_DATA([regression.sps], [dnl
-set format = F22.3.
-data list notable list / v0 to v2.
-begin data
- 0.65377128 7.735648 -23.97588
--0.13087553 6.142625 -19.63854
- 0.34880368 7.651430 -25.26557
- 0.69249021 6.125125 -16.57090
--0.07368178 8.245789 -25.80001
--0.34404919 6.031540 -17.56743
- 0.75981559 9.832291 -28.35977
--0.46958313 5.343832 -16.79548
--0.06108490 8.838262 -29.25689
- 0.56154863 6.200189 -18.58219
-end data
-regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=resid.
-regression /variables=v0 v1 v2 /statistics defaults /dependent=v2 /method=enter /save=pred.
-list.
-])
-
-AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
-Table: Model Summary (v2)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.971,.942,.925,1.337
-
-Table: ANOVA (v2)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,202.753,2,101.376,56.754,.000
-Residual,12.504,7,1.786,,
-Total,215.256,9,,,
-
-Table: Coefficients (v2)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),2.191,2.357,.000,.930,.380
-v0,1.813,1.053,.171,1.722,.129
-v1,-3.427,.332,-1.026,-10.334,.000
-
-Table: Model Summary (v2)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.971,.942,.925,1.337
-
-Table: ANOVA (v2)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,202.753,2,101.376,56.754,.000
-Residual,12.504,7,1.786,,
-Total,215.256,9,,,
-
-Table: Coefficients (v2)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),2.191,2.357,.000,.930,.380
-v0,1.813,1.053,.171,1.722,.129
-v1,-3.427,.332,-1.026,-10.334,.000
-
-Table: Data List
-v0,v1,v2,RES1,PRED1
-.654,7.736,-23.976,-.84,-23.13
--.131,6.143,-19.639,-.54,-19.10
-.349,7.651,-25.266,-1.87,-23.40
-.692,6.125,-16.571,.97,-17.54
--.074,8.246,-25.800,.40,-26.20
--.344,6.032,-17.567,1.53,-19.10
-.760,9.832,-28.360,1.77,-30.13
--.470,5.344,-16.795,.18,-16.97
--.061,8.838,-29.257,-1.05,-28.21
-.562,6.200,-18.582,-.54,-18.04
-])
-AT_CLEANUP
-
-
-# Test to ensure that the /SAVE subcommand works properly when SPLIT is active
-AT_SETUP([LINEAR REGRESSION - SAVE vs SPLITS])
-
-# Generate some test data based on a linear model
-AT_DATA([gen-data.sps], [dnl
-set seed = 1.
-input program.
-loop #c = 1 to 20.
- compute x0 = rv.normal (0,1).
- compute x1 = rv.normal (0,2).
- compute err = rv.normal (0,0.1).
- compute y = 4 - 2 * x0 + 3 * x1 + err.
- compute g = (#c > 10).
- end case.
-end loop.
-end file.
-end input program.
-
-print outfile='regdata.txt' /g x0 x1 y err *.
-execute.
-])
-
-AT_CHECK([pspp -O format=csv gen-data.sps], [0], [ignore])
-
-# Use our test data to create a predictor and a residual variable
-# for G == 0
-AT_DATA([regression0.sps], [dnl
-data list notable file='regdata.txt' list /g x0 x1 y err *.
-
-select if (g = 0).
-
-regression
- /variables = x0 x1
- /dependent = y
- /statistics = all
- /save = pred resid.
- .
-
-print outfile='outdata-g0.txt' /g x0 x1 y err res1 pred1 *.
-execute.
-])
-
-
-AT_CHECK([pspp -O format=csv regression0.sps], [0], [ignore])
-
-# Use our test data to create a predictor and a residual variable
-# for G == 1
-AT_DATA([regression1.sps], [dnl
-data list notable file='regdata.txt' list /g x0 x1 y err *.
-
-select if (g = 1).
-
-regression
- /variables = x0 x1
- /dependent = y
- /statistics = all
- /save = pred resid.
- .
-
-print outfile='outdata-g1.txt' /g x0 x1 y err res1 pred1 *.
-execute.
-])
-
-
-AT_CHECK([pspp -O format=csv regression1.sps], [0], [ignore])
-
-# Use our test data to create a predictor and a residual variable
-# The data is split on G
-AT_DATA([regression-split.sps], [dnl
-data list notable file='regdata.txt' list /g x0 x1 y err *.
-
-split file by g.
-
-regression
- /variables = x0 x1
- /dependent = y
- /statistics = all
- /save = pred resid.
- .
-
-print outfile='outdata-split.txt' /g x0 x1 y err res1 pred1 *.
-execute.
-])
-
-AT_CHECK([pspp -O format=csv regression-split.sps], [0], [ignore])
-
-# The concatenation of G==0 and G==1 should be identical to the SPLIT data
-AT_CHECK([cat outdata-g0.txt outdata-g1.txt | diff outdata-split.txt - ], [0], [])
-
-AT_CLEANUP
-
-
-# Test that the procedure behaves sensibly when presented with
-# multiple dependent variables
-AT_SETUP([LINEAR REGRESSION multiple dependent variables])
-AT_DATA([regression.sps], [dnl
-set seed = 2.
-input program.
-loop #c = 1 to 200.
- compute x0 = rv.normal (0, 1).
- compute x1 = rv.normal (0, 2).
- compute err = rv.normal (0, 0.8).
- compute y = 2 - 1.5 * x0 + 8.4 * x1 + err.
- compute ycopy = y.
- end case.
-end loop.
-end file.
-end input program.
-
-regression
- /variables = x0 x1
- /dependent = y ycopy
- /statistics = default.
-])
-
-AT_CHECK([pspp -O format=csv regression.sps > output], [0], [ignore])
-
-AT_CHECK([head -16 output > first], [0], [])
-AT_CHECK([tail -16 output > second], [0], [])
-
-AT_CHECK([sed -e 's/ycopy/y/g' second | diff first -], [0], [])
-
-
-AT_CLEANUP
-
-# Tests the QR decomposition used by the REGRESSION command.
-AT_SETUP([LINEAR REGRESSION test of QR decomposition])
-AT_DATA([regression.sps], [dnl
-data list list / v0 to v1.
-begin data
--12.84099361 0.873270778
- 16.64932538 0.371315664
- -1.88061907 0.505503722
- -6.20952354 0.734698282
- 0.33272576 0.891224610
- -5.54912717 0.052318165
- 6.11832417 0.448853404
- 11.78124974 0.470447593
- 0.75960353 0.565082303
- 6.06432768 0.149316743
- -2.64919436 0.752532411
--10.32250712 0.798263603
- 2.06355038 0.469129797
- -9.71851742 0.927162270
- 4.65582553 0.250629262
- 9.54574474 0.847032310
- 7.35544368 0.197028541
- -2.09609740 0.400584261
- 10.30101161 0.671546480
- -5.24501039 0.929962876
- 1.73412473 0.758161354
- -3.12732732 0.569785505
- 12.66261501 0.630640223
- -2.90956805 0.576067804
- 4.89649177 0.624483995
- 13.64613114 0.591089881
- 14.03198397 0.544587572
- 2.23566810 0.967898139
- 5.37367760 0.916246929
- 9.01346888 0.451702743
- 0.75378683 0.235544137
- -3.47470624 0.742668194
- -1.02063266 0.860311687
- -2.67132813 0.082460702
- 23.67661680 0.932553932
- 7.95061359 0.430161125
- 2.05300558 0.066331375
- -2.01332644 0.163705417
- 20.00663784 0.587292630
- 3.06099417 0.161411889
- -3.46115358 0.216684625
- -6.85287183 0.548714855
- -4.27923809 0.630997663
- -0.94863395 0.880612945
- 4.47481747 0.359885215
--12.80962955 0.886070341
- 9.35753086 0.187176558
- 2.81002235 0.063035095
- 0.01532424 0.964327101
- 0.29867732 0.866408063
- -2.89035649 0.812135868
- 4.17352811 0.608884061
- 18.15502183 0.920568258
- -2.92662792 0.550792959
- -6.08090449 0.965036595
- -1.09135397 0.862548019
- 7.02816784 0.042277017
--21.20245068 0.430673493
- -8.83397584 0.724976162
- -0.89055843 0.017934904
- 7.03871587 0.308829557
- 3.84286316 0.685105924
- 4.50280692 0.447635420
- 11.39207346 0.875177896
- 10.86673874 0.518530912
- 7.09853081 0.588367569
--12.82864915 0.184667098
- 13.74888760 0.610891139
- 0.37379146 0.557720134
- -9.79020267 0.942839981
- 0.71574466 0.564570338
--17.56040637 0.182061777
- 2.52620466 0.306875011
- 5.37718673 0.366807049
- -1.83964300 0.465772898
- 6.04848363 0.644501799
- 4.57402403 0.121419591
- 8.55606848 0.373011464
- -8.46827907 0.491176571
- -1.77989798 0.734722847
- -0.68661121 0.540984182
- 1.55798880 0.822587656
- 5.22810831 0.333747878
- 9.50280477 0.068100934
- -3.74521465 0.248537644
- 1.36045068 0.851827791
- 4.41604088 0.197207162
- -3.72568327 0.726916693
- -5.36123334 0.906513529
- 3.61594583 0.414340595
--10.01952852 0.140372658
- 25.48681482 0.354309660
- -3.34529093 0.090075388
--18.00437582 0.461438059
- -5.29782460 0.004362856
- 2.79608522 0.861294398
- -1.64076209 0.345775481
- 6.82802334 0.137933862
- -0.45416818 0.404379208
- -1.66868582 0.797685201
--10.02820292 0.075876582
- 5.68232031 0.404815042
- 8.25113850 0.769173748
- -2.83544237 0.076583474
- 0.87659945 0.092751009
- 6.60270870 0.530444351
--12.63924989 0.362099960
- -6.24451253 0.641993458
- 3.53339015 0.461991892
- -0.74012232 0.437409755
- 15.37311996 0.974913038
- -8.09464797 0.543308711
- -9.61320222 0.221564578
- 0.21843662 0.856512540
- -1.56958954 0.610709221
- 6.44977372 0.200382138
--13.29136274 0.093222309
- 6.46257214 0.024135196
- -3.82727990 0.601335801
- 0.43081953 0.268230667
- 19.06654416 0.219972815
- 17.02906651 0.996849502
--10.18073139 0.012543080
- 12.72088788 0.910600764
- 10.45328185 0.331285901
- 7.14370922 0.896312020
- -2.81754334 0.048741266
- 6.40217095 0.075796756
- -3.18030478 0.666325307
- 8.64585957 0.120549153
- 1.37952764 0.899991932
--11.81143886 0.601949630
- 0.03899706 0.363808260
--10.63828243 0.031092967
- -6.66940972 0.246204205
- -5.07374962 0.951272057
- 4.82281566 0.063928187
--21.93693564 0.050972680
- -4.54569883 0.225839693
- -0.92422779 0.437796785
- -1.11683029 0.740215139
- 16.77765554 0.851072372
- 9.73614597 0.388180586
- 14.05345168 0.063760129
- 1.20512012 0.665964184
- 8.00307080 0.102447114
- 8.01252623 0.580929209
--13.54924183 0.438420739
- 9.87164361 0.970859344
- 17.63437095 0.250501797
- -3.42503574 0.873290220
- -2.45873197 0.847756049
- 17.29212092 0.411683187
- 1.15496098 0.530658504
- -2.14438907 0.592255367
- -1.79942021 0.517773009
- -1.30677990 0.830860762
- 1.70233874 0.291826660
- -3.05532536 0.801767829
- -4.06732625 0.092294501
- 6.34665476 0.270426235
- 9.46946411 0.196915311
- 14.50919907 0.480357167
- 8.93767237 0.778228613
- 1.90298854 0.903146151
- 18.50500507 0.598561307
- 4.45123027 0.555898218
- 11.37344114 0.616557707
--12.14693218 0.409187285
- 18.27198688 0.141619222
- -5.75939569 0.056989619
- -4.05515382 0.369281201
- 16.69882098 0.946885257
- 6.39050536 0.679704228
- 4.04213339 0.662792380
- 6.89608366 0.419877433
- 1.56496633 0.358227958
- 5.16679947 0.095144366
- -3.06280456 0.883265975
- 2.76279175 0.866571973
- 1.84969249 0.264869828
- 21.79840498 0.702650979
- 1.42450528 0.719308635
- 0.96797046 0.111937435
- 18.26840323 0.075621738
- 13.38288377 0.573399086
- 2.41101500 0.766238677
- 3.83866337 0.499888953
- -1.56577367 0.695244089
- -0.90342790 0.671654151
- 10.83775583 0.026041124
- -9.89767935 0.745297991
- 11.74840150 0.309144074
- 1.73069359 0.814063985
- -5.27966183 0.591005828
- 3.33030043 0.559401806
- 1.31427975 0.520950237
--10.04588558 0.507008362
- 10.41228345 0.425867272
- 1.71961097 0.595783108
--17.54904427 0.328788939
- -2.23545419 0.223377350
- -8.68774333 0.980964240
- -3.48048220 0.008877675
- -3.69635326 0.090236718
- 9.76114237 0.769375983
--10.25662038 0.508137553
- 0.11155446 0.468504431
- -8.06824580 0.414098962
- 3.10031660 0.327130207
- -3.33393146 0.756896774
- -3.96276749 0.530956360
- 14.53610268 0.846474699
- 1.70505918 0.754662464
- -1.93495001 0.656650411
- 5.01974522 0.745337633
- 13.41249973 0.489362476
- 11.49288744 0.335924476
- 12.59019763 0.155560469
--10.17947298 0.677318449
- 0.05556115 0.655090105
- 3.82092860 0.051838719
- 8.23041456 0.918272190
- -0.50314649 0.772015826
- 20.05162157 0.880265258
- 8.98816884 0.666646668
- -6.28312120 0.138534416
- 3.68589909 0.274559458
- 0.59699510 0.253180863
- -2.74783135 0.983525221
- 0.32515065 0.839969577
- -3.60606166 0.330646732
- -0.82037740 0.129591173
- 6.12444860 0.098536516
- 10.95671074 0.033546728
- -2.84911174 0.720288722
- 6.04597572 0.577061422
- -0.60147150 0.674096868
- -5.30458364 0.291468008
- 2.68044943 0.379853840
- 0.85986585 0.984214339
--12.77906359 0.882390290
- 7.21420144 0.550884826
- 2.31817022 0.231021556
- 11.60161950 0.888496654
- -0.19346228 0.242609713
- 5.07478120 0.759161318
- 14.54155003 0.040387654
- 3.81039636 0.874572741
- 2.23233049 0.448317248
- 0.19481869 0.201906051
- 2.81530451 0.132131690
- 12.39893259 0.674693704
- 0.47054642 0.632959494
- 2.16152913 0.734480632
- 0.33398836 0.315024718
- 7.35509037 0.304570986
- -2.92336559 0.539062343
- 5.79622573 0.392393310
- -2.37607425 0.403380474
- 0.04498550 0.756875541
- -1.63674414 0.613789514
- 11.80310547 0.832651469
- 6.30630243 0.850689403
- 1.48394652 0.096243229
- 4.03361865 0.799660045
- 3.54707273 0.408520520
- 2.00327040 0.702944912
- 17.30761707 0.380542812
- 5.72738968 0.105447516
--13.64604891 0.328506659
- 8.35976334 0.702173924
- -7.41197443 0.134396488
--15.95683040 0.618526462
- 8.76889573 0.950243069
- -1.13482624 0.113477080
- -0.60311407 0.090444247
- 4.95508365 0.612511543
- 5.36934491 0.979213258
- -0.03554882 0.807185690
--11.58131144 0.183341373
- 4.46809041 0.796330582
- 12.49741067 0.346860912
- 8.63824488 0.073684997
- 0.49990913 0.732519306
- 12.82688360 0.109400213
- 13.20375065 0.850369092
- -8.41110869 0.177717087
- 16.31959963 0.727704840
- 17.59203613 0.235311681
- 0.32148420 0.842195936
- 5.43148331 0.670904647
- 7.14649727 0.028190029
- 0.25410683 0.421535783
--12.41047826 0.086404379
--10.64180909 0.229659236
- -6.40185653 0.876365242
- 15.63063324 0.667672536
- 1.94280423 0.799266628
- -5.76507450 0.367344192
- 8.60895533 0.154109357
- 9.38306751 0.788742770
- 3.43573528 0.284535277
- 4.81848966 0.872283177
- 11.65839314 0.234109111
- -5.57884822 0.030363060
- -3.94238060 0.325320686
- 9.38133340 0.201141788
- -7.65003459 0.647734396
- 11.23091019 0.084927159
- -6.07705432 0.037273791
- 7.46380750 0.506897136
- 7.42034855 0.869351148
- -4.43031973 0.231191152
- -1.07351537 0.480234836
- -1.40653281 0.690620421
- -3.82710168 0.990191328
- 5.04583490 0.543427375
--11.54265099 0.270542185
- 0.49059479 0.991447248
- -1.40871469 0.555998766
- 3.64241437 0.743840673
--18.30031589 0.357478210
- 4.27487959 0.770619738
- 1.28805821 0.654787106
- -3.19542768 0.218110139
- 12.53375654 0.011857644
- 11.78889419 0.054127726
- -5.38392310 0.839309080
- 16.38024181 0.228801038
- -0.59622631 0.134381782
- -0.74107258 0.258146632
--12.31429450 0.020524447
- -0.79785028 0.968028764
- 6.39899711 0.038162566
- 7.42024044 0.716163692
- -3.62470664 0.018201813
- -2.55049724 0.162446610
--10.79888854 0.683070478
- 10.18490144 0.546461234
- -2.76979044 0.198830067
- 4.85164813 0.094100357
- 0.96477200 0.381801756
- 8.13344336 0.639730450
- 9.04684412 0.786084368
- 10.41746272 0.828304181
- 0.94334368 0.798419831
- 10.13116556 0.191715972
- -4.12728628 0.575178239
- -9.59222379 0.876405375
- 1.64680258 0.391003085
- -4.58897613 0.039176486
- 0.38394379 0.511577564
- -4.80428215 0.222785463
- 0.35363661 0.681658725
- -9.63685708 0.183035382
- 3.54363414 0.766127414
- 6.89610808 0.967514568
- -2.03781105 0.464416752
- 8.67956196 0.421424078
- -1.09959038 0.061231448
- 7.12587456 0.028601318
- -6.93064672 0.402561175
- 8.57989199 0.925089270
- -9.55071810 0.454993099
- -8.11914736 0.509644286
- -5.41909698 0.077813151
--17.03336572 0.875713545
- -1.27438609 0.602163625
- 3.09834374 0.105599007
- -1.59865741 0.439939102
- 11.82272089 0.754984309
- 4.30969696 0.483834579
--10.76886192 0.222486992
- 7.05419803 0.903020271
- 7.36096847 0.440357053
- -2.05864869 0.581170147
- -9.08366913 0.318677911
- 8.57119930 0.605668919
- 7.87702340 0.570206991
- 5.22035786 0.542344385
- 2.37238850 0.595969470
- -4.29809941 0.634313781
- 4.51647479 0.796663089
- -0.62478780 0.562099444
- 8.50866078 0.490014249
- 3.46694991 0.122890089
- -7.31956453 0.885170890
- 2.20259268 0.167180856
- -1.81003626 0.702563515
- 8.44526939 0.973495019
- 8.19767069 0.881261264
- -5.92422578 0.686557351
- -0.11826129 0.712798344
- 5.66132869 0.922826429
- -5.40845018 0.642183516
- 6.67839036 0.680978989
- 11.88962825 0.487904896
- 3.32266332 0.931709581
- 0.24234019 0.405641313
--12.79023339 0.361005489
--13.57875491 0.266289733
- 1.81304596 0.775093821
- 0.36755600 0.400225605
- -9.15574205 0.518040748
- -3.90436548 0.396869908
- 9.24764042 0.669374848
- 0.74869385 0.609881390
- -3.62958907 0.928867495
- -0.02527232 0.557679930
- 0.04433418 0.152565816
- 11.76152632 0.865663501
- -4.62181124 0.007000650
- 5.82271403 0.389678502
- 0.33289002 0.532940826
- -7.65647076 0.681574524
- 11.81023732 0.107165912
- 11.42121999 0.989580324
- -5.47120641 0.762285550
- 3.82311561 0.388755074
- 16.91059711 0.461236022
- 4.14012105 0.802420151
- -1.35278659 0.036646959
--12.81733350 0.179096148
- -0.94395134 0.450959878
- -5.39002376 0.264783062
- 4.16227017 0.780743762
- 7.26179625 0.821382454
- 15.10062276 0.469253936
- 1.45877225 0.685434405
- -9.87966760 0.767201511
- 7.03156071 0.195142483
--11.71327419 0.774014869
- -4.55518706 0.973103604
- -1.75221406 0.175172193
- 10.35631400 0.080670414
- 4.97650495 0.597478189
- 2.25703939 0.585949751
- 10.72500409 0.339720931
- -5.02901029 0.997874377
- 6.24398637 0.655067479
- -5.83880059 0.184948259
- 2.17256077 0.746741866
- -5.59809380 0.277523381
- 8.19384177 0.334565607
- 3.35250431 0.952057263
- 16.20874892 0.901400446
- 1.63205839 0.235388475
- -1.07921163 0.608376332
- 0.24315118 0.862639830
- 15.61923078 0.050955422
- 1.99639207 0.358905687
- 8.14825538 0.190069662
- 4.55210835 0.784025901
- 13.51582298 0.973572910
- 15.42415796 0.969080992
- 2.23978124 0.551857514
- 1.00858991 0.919566804
- -2.77293574 0.906998180
- 7.10750420 0.934792213
- -8.01377290 0.682306063
- 9.67873875 0.239576806
- 7.54867950 0.065860266
- 13.85701962 0.733823443
- 8.48212853 0.285731085
- 3.55278843 0.998255904
- 21.94592206 0.205463912
- -2.07957143 0.948665109
- 1.54169997 0.200109744
--11.36934275 0.447122472
- 3.07094572 0.815147945
- 6.45818709 0.007849948
- 1.85594578 0.818796540
- -2.43799564 0.962013689
--17.96539549 0.654190963
- -0.93433746 0.454930236
--11.06904368 0.898560975
- 14.89733742 0.479152492
- -5.72390675 0.136197255
- 9.46781102 0.669006610
- 5.35954546 0.259381138
- 3.78388994 0.933778797
- 1.95373423 0.517555994
- 10.96772341 0.666138826
- 9.40585102 0.779906833
- 0.75347502 0.142656741
- 7.64803672 0.734297119
- -0.40051164 0.362230232
- 10.00747057 0.660820381
--12.86024975 0.072988046
- 1.43515528 0.229672223
- -6.75981709 0.658534537
- -5.61355474 0.795897133
- -4.40596595 0.038787666
- -1.37033650 0.371835229
- 6.66666573 0.560963737
- 8.18430044 0.284787698
- -0.55742330 0.622783662
- -0.39757686 0.673551753
--12.68628005 0.373038561
- 4.06416215 0.760546238
- 4.65859855 0.516761886
- 3.55304076 0.266856843
- -7.35294817 0.615783196
- 1.01222898 0.158266779
- 9.91052610 0.285619547
- -6.42966726 0.573689954
- 10.97425098 0.985095061
- 5.79394599 0.404333309
- 10.09106608 0.441037857
- -1.47295537 0.577661077
- -2.07959719 0.547176133
- -8.76910940 0.498979558
--11.15658312 0.135862745
- -0.88456783 0.326480064
- 9.71607440 0.998076370
- -8.76072622 0.386244511
--19.26823092 0.461833959
- -0.11280313 0.064155908
- 0.64625887 0.172078148
- -5.35323428 0.331153163
- -1.71034509 0.330955888
- 4.27104744 0.590544244
- 7.33843789 0.263171531
- -5.38121637 0.539675802
- -6.87566548 0.127313096
- -2.50161298 0.269417630
- 10.99076986 0.097362729
- 6.34017269 0.318528587
- -4.63672382 0.451038055
--11.55122495 0.987073278
- 4.78618612 0.297342215
- 2.97547390 0.197312152
- -5.54495280 0.499701114
- 17.67606173 0.810316588
- 16.01578815 0.643667608
- -0.16258467 0.228284761
- 7.92123340 0.784289369
- -2.26303900 0.270764770
- 5.84136933 0.437763291
- 4.96955217 0.389720490
- -8.09516710 0.829068548
- 14.59207207 0.513593803
- 2.80954688 0.650799867
- 4.53653552 0.672326278
- 4.49116737 0.807447691
- 18.87549709 0.647303378
- 9.80118464 0.932576117
--13.02124969 0.038651904
- -6.15189291 0.697593318
- 15.81920283 0.249825051
- 10.81503188 0.152372300
--23.58738366 0.593560367
- 8.15716338 0.411680007
- 3.45349379 0.351061414
- -6.39345334 0.374926213
- 8.72924585 0.165759028
- 22.17948804 0.003736780
- -4.73053410 0.582425257
- 16.88289626 0.484899167
- -1.78826142 0.663273340
- -0.78106025 0.337039969
- -2.92461669 0.810174719
--13.89224399 0.177428986
- 4.56809819 0.025010350
- -1.07452825 0.649632933
- 0.58148751 0.829606422
- 12.13329525 0.354819526
- 17.35605568 0.284862590
--12.43678107 0.827661083
- -1.89492796 0.574929572
- 2.18520382 0.846299917
- 18.11449649 0.603173531
- 4.34508582 0.484049042
- 17.49394569 0.094811656
- 10.67752350 0.166176400
- 17.13374502 0.547208197
- 4.42138123 0.768691494
- 5.38445574 0.788597361
- 0.79946671 0.851883720
- -4.67547904 0.995621191
- -5.61496422 0.523793593
--20.52093126 0.881207308
- -8.95996814 0.851078124
- -7.63483710 0.739657373
- 11.02131097 0.678060014
--10.56228517 0.202393048
- 6.48841788 0.143946271
- 3.44853632 0.913249620
- -0.02080024 0.070765134
- 2.08654297 0.032468089
- 8.13415912 0.439470874
- 11.19028936 0.944954026
- 0.26670866 0.492724593
- -9.33692734 0.982611921
- 17.23967092 0.313428994
- 0.36906670 0.660669528
- 7.89735684 0.977628886
- -4.00171487 0.379327632
- 5.01615432 0.735627296
- 0.42214214 0.092461754
- 13.60634772 0.218359635
- 6.57431413 0.067653525
- -1.77668341 0.717799276
- 5.16227422 0.325502093
--15.29091550 0.332815338
- 3.33602480 0.594168551
- 13.80131443 0.817724470
- 5.92111679 0.947854666
- 3.59747624 0.330860216
- -6.79722403 0.093518715
- -1.86606213 0.824179728
- 17.05226458 0.466573672
- 10.39712467 0.409067778
- -4.78536386 0.891470739
- 11.92963128 0.719633060
- -1.44230992 0.232628002
--12.31860616 0.834134222
- 2.93439660 0.957842480
- 14.27963295 0.546264646
- 2.17488820 0.701170328
- 10.78772417 0.612332448
- -0.47049341 0.378564293
- -0.35140634 0.034337429
- 5.04887868 0.211697132
- -3.51562580 0.663243607
- -0.82013387 0.602497174
- 2.78954743 0.325294790
- 8.67905777 0.820296625
--12.70343389 0.315467361
- -2.59373236 0.015571904
- -4.60369241 0.293737716
- 1.58669084 0.671091860
--10.44245103 0.501340276
- 4.85215578 0.141572007
- 10.46303284 0.801814632
- 1.27898298 0.236929983
- -1.72225479 0.608500539
- 20.18685735 0.827124630
- 3.27308817 0.542065179
- 1.01596956 0.254672115
- -8.88872881 0.460876757
--11.31397349 0.636168639
- 0.85294367 0.816417328
- 3.54262337 0.944147626
--10.53603202 0.675775741
- 4.34832198 0.121988381
- 11.56451662 0.283063133
- -7.36454369 0.500596540
- -8.23701113 0.379483261
- -8.36081323 0.219730782
- -6.39158860 0.739171315
- -1.40518544 0.478709398
- -4.01314821 0.460476388
- -7.34814047 0.406242873
- -7.80836711 0.730648091
- -0.57729135 0.152336258
- 4.98352832 0.026424939
- -3.78181635 0.453598432
- 20.16821827 0.845273124
- 5.20758271 0.573569671
- -3.05534245 0.286828574
- -5.31306254 0.961990401
- 1.09307567 0.006478724
- -3.75412572 0.598277695
- -2.38444245 0.777900122
- 2.46837742 0.280363751
- 9.72195519 0.041094463
- 3.96271247 0.604775284
- 2.14105354 0.400315328
- 7.88645912 0.404573389
- -4.03565076 0.798377309
- 10.80180959 0.932152434
--10.89359212 0.446813857
- 1.43144578 0.310194540
- 4.79825196 0.504826858
- 10.73201365 0.384306369
- -4.07526187 0.893893643
- -2.84330198 0.390202663
- 5.81825057 0.830581384
- -2.77842745 0.382966910
- -7.70333673 0.157692966
- -3.25753058 0.726303603
- 8.50032387 0.556524444
- 2.35027236 0.857076526
- -1.70740565 0.194760923
- -3.40693880 0.696420946
- -8.03983352 0.514393263
- -1.85105344 0.609459979
- -9.01148029 0.526019631
- 18.37344635 0.690793045
- 16.46079416 0.811535334
- 4.10224315 0.043403618
- 7.06657672 0.831274577
- 15.31421824 0.434558881
--12.36760970 0.004215634
- 1.95473415 0.277788662
- -0.93207006 0.368433415
- 15.39919341 0.843189783
- 5.23452387 0.626226925
- 11.40805770 0.002417288
- -1.30282837 0.072493756
- 3.92130690 0.675355182
- 2.53148399 0.027222295
- 4.92705318 0.934429364
- 5.54978818 0.042268708
- -2.19608977 0.246743834
- -0.62565550 0.858214200
- -8.98329365 0.646827226
- 12.78468146 0.533966352
- 2.01061290 0.418710227
- 1.03689579 0.019241741
- 8.01166696 0.992268130
- -4.49786437 0.694127903
- -8.15387184 0.066275002
- 2.22256207 0.083301613
--12.27145086 0.535369809
- 9.95709112 0.227692557
- 14.58198717 0.667298058
- 5.98046083 0.922503625
- 1.25640725 0.632933575
- 9.77623752 0.136171032
- 5.57068426 0.374916651
--10.07048336 0.470411379
- 3.69267954 0.897278365
- 2.22185354 0.212539549
- 7.96155623 0.720525208
- -6.21272358 0.771491819
- 2.63054735 0.474989115
- -2.81488890 0.675381020
- 4.52747191 0.118615879
- -3.22975936 0.783991133
- 11.42834761 0.423344824
- 0.26512464 0.617515445
- -5.84322807 0.210915613
- 9.61073028 0.988117333
- -6.11878012 0.492318959
- 5.30581443 0.339379499
- -6.40132703 0.903540026
- 1.22921808 0.122161655
- 8.08547837 0.197296588
- -0.77943801 0.935963718
- 11.43194858 0.828270943
- -5.41689395 0.556863468
- 15.14667847 0.565186375
- -5.15327419 0.542802437
- -3.95903082 0.643379366
- 5.78847793 0.391369361
- 11.54430873 0.158789330
- 1.90340148 0.841316129
- 14.69680285 0.532022770
- 0.68552840 0.367073827
- -8.72287967 0.250127491
- 9.35401445 0.836083158
- 5.32139524 0.996712598
--14.53387897 0.825434481
- -2.93925146 0.513153861
- 12.54386493 0.713306793
- 2.04842442 0.993893406
- 2.87461954 0.049843312
- 4.89765230 0.376710062
- -6.23945314 0.321108142
- -3.45840168 0.854710947
- 9.05807160 0.199992188
- 3.33815006 0.787302467
- 4.22244242 0.351841910
- 15.75879160 0.268699469
- 2.78549859 0.920299974
- -4.46643118 0.727283862
- 0.48021298 0.428672083
- 2.55814938 0.130915212
- 5.00692968 0.062266047
- 4.78801127 0.325124688
- 6.39524485 0.693406744
--10.46792584 0.458128441
- 10.14111908 0.353412759
--10.56424183 0.821588957
- 7.60967746 0.267669137
- -2.34956688 0.434855697
- 23.82269027 0.802311880
- 8.37170447 0.445185000
- 10.05024769 0.778687843
- -9.15753018 0.957292819
- 12.17438228 0.774769426
- 1.57960028 0.783591989
- 0.06719501 0.849073924
- 16.21114558 0.243444943
- -3.79808298 0.842994720
- 8.98927715 0.020537113
- 7.72362992 0.984168340
- 11.25158442 0.152385348
--21.23936903 0.909204114
- 7.34995949 0.987249305
- -7.99435203 0.335456401
- -2.78218185 0.768517548
- 11.59547596 0.466617637
- 15.90870706 0.071892573
- 5.58160897 0.554485703
- 16.05253351 0.815206562
- -3.23103465 0.280495460
- -4.61108636 0.035757819
- 5.41596511 0.746146856
- 2.92445613 0.136743821
- 11.23628254 0.681316365
--12.93714705 0.838791576
- 9.94668264 0.084457395
- -4.56061529 0.983605894
- -4.24795688 0.601732731
- -2.83740044 0.375102341
- -0.43078317 0.403870303
- 15.19689584 0.114826374
--10.29920266 0.731582141
- 6.01686515 0.641655876
- 6.69431335 0.496723697
- 4.62223602 0.328118236
- -1.74309026 0.072604771
- 14.31971261 0.827101483
- -1.86629155 0.613346722
- 8.30971428 0.274948560
- 8.50080711 0.059822908
- -7.94061422 0.121069240
- -2.72096492 0.710791774
- 3.33259421 0.398621625
- 1.73248470 0.488581205
- 9.56008489 0.011104565
- 12.71499762 0.038568985
- 4.11512127 0.219846314
- -0.96707584 0.822646857
- 4.98621667 0.633779997
- 4.69384821 0.295708955
- 10.16008645 0.778287787
- -7.72973800 0.097096969
- 2.87264210 0.796538177
- 4.56095440 0.862952770
- 5.02621658 0.934628629
- 3.18138681 0.805600816
- -1.02245780 0.317640678
- 18.16001652 0.992503640
- 4.13729026 0.941910149
- 1.61211303 0.377271914
- 1.71520009 0.735196094
- 3.26325421 0.514432564
- 12.94663819 0.591190711
- 10.53239931 0.005877708
- 8.06705056 0.340779884
- -5.09007267 0.332516161
- 12.31973355 0.323119296
- -2.69957650 0.633232996
- 12.51207803 0.377641090
- 8.02081444 0.859293157
- -0.13098726 0.099370804
- -0.97757546 0.852873609
- 16.73605399 0.595854575
- 3.63219184 0.329310613
- -4.79105630 0.247760146
- -4.77209495 0.708235587
- 0.92107647 0.924567254
- 12.12724271 0.433550712
- -5.07731478 0.200109463
- 9.16019579 0.897456586
- 18.33260560 0.649877409
- 1.93596773 0.584401505
- 8.51254631 0.283154523
- 11.41092928 0.698703314
- 10.85035748 0.351078210
- 12.62749979 0.570101319
- -2.32028296 0.313842122
- -2.45778301 0.007943144
- 6.93102526 0.108737491
- -0.67304654 0.245399613
- 9.27294774 0.204010286
- 14.29292826 0.396294626
- 14.05843185 0.864613328
- -3.73515954 0.305862948
- 0.36606339 0.116802407
- -5.79235478 0.457308058
- 8.70346900 0.858244380
- -8.91321043 0.077001581
- 0.58499566 0.503209780
- 0.39160153 0.324883353
- 7.46715326 0.343451039
- 12.36256009 0.679483638
- 8.84283689 0.687359177
- -6.39396909 0.113065562
- -3.67844896 0.667335667
- 9.36904962 0.009815419
- -3.25244888 0.213105120
- 19.09389976 0.593130536
- 7.28826611 0.829483570
- -5.44565944 0.956490203
- 7.96993416 0.770961635
- 0.20683778 0.006497153
- -3.73273760 0.037042812
--10.64745846 0.813594448
- 5.70578906 0.157678242
- 4.05282218 0.224663656
- 14.77711159 0.577586777
- 0.89685942 0.297213941
- 3.92600687 0.672347849
--12.29347477 0.367072171
- -9.33603480 0.456544225
- -0.86683190 0.088696811
- 4.65685745 0.779783359
- 1.24438030 0.712958633
- 11.43533814 0.920345548
--10.18380242 0.044456697
- -1.20684029 0.992051648
- -9.78059038 0.611477837
- 3.05588762 0.581933667
- 3.47419279 0.769325101
- 0.87528245 0.455214184
- -3.13185655 0.805887381
- -0.82283965 0.707668384
- -1.86717272 0.984060013
- 16.56357048 0.217369677
- -2.11052646 0.474156371
- -1.39795364 0.958554209
- 4.87468692 0.328779186
- 2.69163553 0.401633221
- 6.08640626 0.599963560
- 7.41420081 0.240202007
- 5.73729928 0.696034193
- 7.31747120 0.569520861
- -6.20465547 0.214005920
- 17.52477873 0.667125450
- 12.97855692 0.796977778
- -3.35883428 0.379721403
- -2.90306972 0.552454626
- 5.31617371 0.401625473
- -3.86414389 0.830986352
--14.94107832 0.702705123
- -5.74060402 0.833328045
- -8.10116203 0.078855027
- 23.48247017 0.568666620
- 20.22005082 0.357069809
- -2.53387193 0.637455425
- 15.72048831 0.845354124
- -4.41494567 0.934471473
- -8.02254420 0.378467959
- -0.13398716 0.489382793
- 0.95967155 0.813667919
- 0.14835664 0.215786848
- 14.31875579 0.675145039
- -6.36589196 0.822037848
- 8.25942906 0.156787526
- -7.33597529 0.051076292
- 12.58936771 0.666507807
- 2.34653798 0.626196518
- -0.69351398 0.050664564
- 7.08738260 0.808776877
- 5.19653521 0.779008623
- 3.20900427 0.197212774
- 7.81171331 0.744975548
- 6.49008186 0.991318119
- 7.27471854 0.839642650
- -7.68367290 0.880500743
- 12.04846713 0.797754890
- 14.93435279 0.190527791
- -3.83641079 0.075995951
- 2.15090497 0.426560973
- -3.61166623 0.777188818
- 8.49333248 0.891445999
- -7.46936100 0.148607446
- 13.85406193 0.983656455
- 12.20477754 0.499345090
- 10.09415710 0.638127733
- 5.37134772 0.110929011
- 8.17660840 0.879411588
- -4.38804367 0.608933700
--11.78145902 0.265134740
- 6.18940186 0.970982743
--16.24831477 0.844983635
- 9.52790402 0.578152651
- 16.44372225 0.264144422
- -2.48286428 0.893865621
- 5.33297280 0.512990215
- -2.68912507 0.851636020
- 9.94607707 0.644483197
- -1.93526852 0.550759844
- 2.34310539 0.787853650
- 11.79131608 0.983668283
- 3.16689104 0.605394987
- 10.47759320 0.919442774
- 2.86973133 0.557835916
- 10.30674302 0.442504870
- 10.92820575 0.976183635
- 7.98050212 0.139334994
- -0.64719705 0.981199028
- -2.63625596 0.341524563
- 11.38799583 0.858987904
- 1.37321916 0.202373294
- 12.66698520 0.142127091
- 5.83599540 0.864497670
- 4.88659560 0.472598564
- 13.00108599 0.961629827
- 5.79514791 0.408377170
- -1.47807631 0.536772872
- -3.38142805 0.288956265
- 6.25154986 0.828695103
- 2.40919373 0.478123848
- 3.72990486 0.056539500
- -9.90915815 0.603356617
- 0.21737084 0.737251896
- 5.36929388 0.026920178
- -1.05027354 0.034992509
- -4.97887624 0.506301429
- -6.40058435 0.014061876
- -0.14610837 0.619699963
- 3.78483619 0.653952701
- 3.84143365 0.162122572
- 2.66030676 0.196542503
--10.56809462 0.386200215
- -5.01140125 0.711703654
- -3.09809005 0.118120179
- -2.76110171 0.118809515
- 2.85825107 0.129646974
- 2.75993661 0.171779333
- 11.55931169 0.372165133
- 9.21211486 0.969079819
- 6.02207148 0.498965865
- -3.52883224 0.954619249
- -2.60190803 0.069405278
- 1.34183694 0.569402487
--11.35155228 0.766344735
- 1.04661568 0.023673810
- -1.90461932 0.179728300
- 13.72465582 0.467775796
- 19.14882438 0.476924297
- -1.07480326 0.944407858
- -8.44289331 0.059804028
- 1.89732882 0.743225795
- -7.87832463 0.672539050
--12.24163608 0.916803014
--12.77212790 0.648129714
- 6.39197262 0.622954436
- 5.26261666 0.494421400
--10.65239640 0.695527931
- 4.63841458 0.499519163
- -2.94276544 0.429201572
- 4.68788953 0.639613685
- -1.03031400 0.349342009
- -2.69946354 0.221796918
--15.32237714 0.631289988
- -8.31962698 0.925363812
- -5.80897714 0.833536878
- 7.16070989 0.832098478
--10.99679727 0.794048223
- 0.84514458 0.748014415
- 2.23308495 0.111176288
- 3.56351018 0.599805508
- 0.88336430 0.746908710
--14.63461670 0.314391808
- 4.39039715 0.079604833
- -7.07001439 0.633705345
- 2.11252583 0.461468123
- 7.60219364 0.497389476
- -4.87713428 0.039952736
- 2.17515292 0.421830084
- 0.64302362 0.267982804
--22.29371533 0.646257366
- 0.31652779 0.060548371
- 7.93445046 0.343570449
- 0.28292029 0.651909785
- 2.77775640 0.637679287
- 6.22941586 0.291132945
- 23.68567532 0.708513840
- 9.49503014 0.645200206
- 0.87405420 0.063154289
- -4.04931224 0.110797498
- 8.91607239 0.732917195
- -5.77728018 0.635435595
--16.37296319 0.343727613
- 9.87409940 0.774177478
- -8.11360210 0.377765616
- 14.54242540 0.204343527
- 0.36239636 0.115528352
- 19.51009176 0.181365423
- 1.23592729 0.011676577
--15.81877035 0.767155028
- -0.05911251 0.737944231
- -6.55395965 0.214062137
- -7.85591487 0.539865054
- -9.73010882 0.730924287
- 11.79433862 0.267116856
- -8.84308360 0.088069165
- -5.56689174 0.405987947
- 7.59010135 0.655631611
- 10.07629305 0.031106157
- -6.19331485 0.052350502
- -4.58626710 0.326901540
- -5.19431549 0.125740555
- -2.08129025 0.034657174
--10.48798034 0.153632237
- 13.04657686 0.317295703
- 1.94142067 0.731437668
- -1.62470735 0.701070475
--12.27046912 0.505781742
- -2.96095228 0.122808075
- -2.91847765 0.372668438
--14.83230131 0.100749725
- 16.57350659 0.707854947
- 10.05473238 0.244046174
- 5.50858969 0.070691273
- 7.65309196 0.245393047
- 7.16359996 0.056261015
- 4.33026356 0.318855549
- -4.65721575 0.271249938
- 2.85909691 0.309377566
- 3.02736080 0.553944209
- 6.22796768 0.763945813
- -4.47036396 0.197721195
- 2.78901176 0.441166128
- -9.94574794 0.964660659
- 1.86451969 0.704635530
--10.38926659 0.772304221
- -5.36565800 0.029527218
- 1.99230152 0.578448308
- 13.65547415 0.936050102
- -2.05229879 0.851521142
- 0.99504588 0.974891334
- -1.46027404 0.320227281
- -8.45614275 0.727910071
- -0.95201934 0.199101032
- -2.46642929 0.462252060
- -6.44060430 0.703637604
- -2.58115910 0.084948525
- 0.76248197 0.125769097
- 12.00603845 0.675927328
- 1.97538215 0.782502470
- 2.23331320 0.870228155
- -3.10226060 0.485056198
- 12.59337170 0.584729095
- -2.42247402 0.387588168
- 9.41981063 0.374604221
- 6.26806243 0.727453335
- -5.30630356 0.427294265
- 13.81542647 0.394246994
- 1.05647858 0.646684666
--12.25005208 0.010531726
- -4.58162076 0.077133994
- 0.58094190 0.400275636
- 5.79443858 0.641731247
- 8.87635216 0.913593476
- 9.71048520 0.955285711
- 13.10563373 0.908471848
- 4.99194220 0.967014095
- 15.88178853 0.041518216
- 9.35962068 0.864770023
- -7.53095731 0.300106124
- 12.18427585 0.248876997
- 9.22034502 0.450149366
- -1.02861237 0.684246939
- -2.98140404 0.326901490
- -4.64316598 0.425381055
- 15.35233259 0.630774937
- -1.85655250 0.226889991
- 15.43748330 0.584219351
- 10.39060893 0.387854461
- 2.80705696 0.564024865
- 3.48201221 0.787103673
- 7.03787977 0.112019552
- 8.41853061 0.798376796
- 15.63925527 0.873984550
- -4.05742183 0.699131238
- 6.56954685 0.720018710
- 2.44007265 0.232697343
- 3.75597926 0.975133449
- 2.92362149 0.290975435
- -4.74372257 0.003738451
- 1.28365940 0.987536495
- 15.65288265 0.179629701
--11.76385004 0.850614822
- 1.56331228 0.592017435
- -9.64774741 0.024951969
- -9.44879860 0.993960270
- 29.33340056 0.913358233
- 7.97233120 0.021820585
--12.10837953 0.401535846
- -1.20729618 0.984977268
- 3.63219301 0.491142613
- 2.79853507 0.663823888
- 3.19584583 0.612511282
- -0.81790885 0.908769330
- -1.67795944 0.611690031
- 2.55137163 0.109447998
- 4.36572889 0.382049700
- -6.35667866 0.162787163
- -0.76239101 0.892383562
- -3.99558996 0.466572017
- -0.47513018 0.457760464
--10.69568261 0.544872910
- 4.30943512 0.982456072
- 2.91825703 0.823403368
- 0.10753188 0.945676881
- -8.38623073 0.923085521
- 4.95690232 0.188128654
- 5.39956649 0.331692462
- -1.47421789 0.327711090
- -1.81689665 0.713285385
- 5.15137860 0.414906436
- 5.68897151 0.110799415
- 0.78825159 0.396824099
- -1.78376652 0.929264595
- 0.76991060 0.950124414
- 15.81469073 0.951245195
- -4.33820920 0.009896093
- 1.67174323 0.821983745
- 0.38997945 0.928857784
- 1.97848484 0.175680230
- -5.81067801 0.772580245
--10.45208478 0.418845035
- 13.34024524 0.905645046
- -8.79585122 0.906516178
- 2.89093397 0.113010960
- 2.22324289 0.799940482
- 8.95497981 0.984663669
- 0.93288527 0.277914575
--17.35306978 0.455587022
- -3.26914604 0.406757639
- 8.75871227 0.067059659
- 1.79914932 0.784879863
- -0.67305388 0.006393497
- 1.66805704 0.039614073
- 9.03868439 0.601066847
- 4.29458670 0.015772820
- -8.15564320 0.939633197
- 12.50538902 0.766347628
- -0.45547258 0.464314122
- -9.47180656 0.640114882
--13.25567198 0.125841930
- 2.87660101 0.381931128
- 7.37834152 0.648958712
- -0.45874073 0.303139498
- 4.87941450 0.500090729
- 4.50344891 0.311329309
- -6.14257896 0.061368838
- 5.98243271 0.873804882
- -2.64694079 0.080493398
--17.79727796 0.188420116
--13.52552336 0.798403568
- 2.29086373 0.518700767
- 5.21493652 0.788828533
- -8.09641615 0.775041349
- 5.87005782 0.079447757
- 10.74795720 0.955691540
- -8.01115709 0.004508053
- -7.53735064 0.054195934
- -6.79130165 0.877193354
- -1.26419539 0.837772170
- 8.31082852 0.967509866
- 21.83090247 0.261529880
- 11.20453234 0.913858875
- 7.19128396 0.541942489
- -2.93623595 0.860095891
- -3.61446403 0.022418065
- -6.59997709 0.532998307
- 3.71486934 0.522669434
- 18.03420874 0.295064126
- -8.75452291 0.390175021
- -7.83680812 0.263760724
- -1.10263921 0.501819826
- 2.05633484 0.338684642
- 5.25636848 0.558667384
- -7.33260497 0.457327559
- 3.86721296 0.612182242
--15.94331373 0.478329365
- 3.71501899 0.264241588
- 18.26175822 0.023212661
- -5.21093378 0.184378036
- -2.44074986 0.297114134
--11.88339919 0.875956945
- 7.52127093 0.927322099
- 5.31597834 0.416344968
- 11.42012314 0.952491078
- 13.64950955 0.794183413
- 12.50861255 0.390723282
- -3.48142207 0.538708662
- 14.32910902 0.085221990
- 5.76196699 0.313860477
- 2.63751452 0.917424732
- 2.99975231 0.208662214
- 7.09852825 0.798246964
- -1.95742636 0.352166210
- 7.80534904 0.386523123
- -4.47229047 0.290188493
- -4.35535158 0.761527294
- 1.47083860 0.447897289
- 3.09504296 0.048513534
- 3.50446804 0.925072429
- 12.00487617 0.574499971
- 10.35171466 0.934193962
- -5.63256003 0.968833982
- 7.15625220 0.467160468
- -7.81378393 0.790220187
- 4.52101003 0.014459969
- 12.90773453 0.990835752
- 7.70737851 0.785329264
- -3.37196794 0.066025357
- -5.12793918 0.347459322
- -7.96083724 0.216608294
- 12.81247279 0.287880308
- 4.63872463 0.426881173
- -3.85439309 0.336532356
- 4.55633320 0.108001536
- -2.40824634 0.135247519
- 1.65932541 0.005108006
- 3.26129578 0.093163961
- -3.52114597 0.544041275
- -9.08479260 0.111212700
- 7.75150456 0.942726234
- 7.44829768 0.396996218
- 14.44430576 0.525470762
- -2.13457508 0.207577358
- 9.74871681 0.537177845
- -4.53338693 0.625854028
- 16.15962824 0.947933141
- -0.17711664 0.480940902
- -7.21470818 0.006952612
- -6.27644212 0.737909602
- -0.81648165 0.230003567
- -2.10429152 0.209671398
- 7.69291241 0.987903443
- -0.32284504 0.183904658
- -0.90833921 0.782169770
- 10.35542238 0.201758865
- 10.40788689 0.540802365
- 10.80011849 0.298263948
- 7.39943598 0.785716539
--12.71674257 0.154135834
- 16.67139866 0.116794235
- 12.47832985 0.998179468
- 3.24041348 0.653080096
- -5.50381593 0.995396942
--10.41952811 0.576472768
- 4.33514092 0.146434686
- 4.41294276 0.507165968
- -0.14746982 0.698144836
- -1.33323051 0.466481571
- -7.01201350 0.797150114
- 6.58669848 0.942287809
- 7.19444974 0.053569397
- -5.66046997 0.435728340
- -5.07828702 0.497727572
- 13.72045272 0.324222944
- 9.99111984 0.355713969
- -4.42363728 0.071790181
- -2.34300923 0.618434528
- 4.98594041 0.605667438
- -2.45307721 0.894546647
- -3.52276424 0.760086779
- -3.69489441 0.960758209
- 13.04792817 0.511273320
- 21.61433486 0.236270637
- -9.57303968 0.964235539
- 11.70400744 0.391045695
- 4.25170422 0.411577090
- 7.87516537 0.952858161
- 9.89202673 0.971106687
- 7.51554467 0.505791978
- 2.17944879 0.893835908
- 0.82420351 0.213912155
- 2.47121932 0.688019842
--14.88503628 0.640950883
- 8.16032283 0.277742858
- -4.65776244 0.129415853
- 7.48274838 0.074213153
- 7.70537066 0.476778957
- 5.88202944 0.351838898
- 1.95618325 0.106331699
- 7.22064623 0.511434587
--18.64632081 0.009314188
- -6.16794611 0.526204245
- 2.14042033 0.675800465
- -1.89535048 0.916845690
- -7.77156605 0.069742819
- -6.84078801 0.865082345
- 8.17539904 0.095895629
- 10.75170309 0.821383078
- 14.31498805 0.117893208
- -2.82264467 0.809086411
- 0.11117380 0.400587471
- 6.43898314 0.333163663
- 9.48110784 0.465173316
- 5.39395511 0.695273081
- -2.05636570 0.508060862
- 0.68666117 0.647109494
- -6.41880322 0.267530762
- -0.12096589 0.986901165
- 6.46062643 0.588580914
- -4.20926136 0.550783675
- 4.07354300 0.907963701
- 10.84045143 0.900920521
- 2.64504664 0.767700269
- 10.34578229 0.197810342
- -0.19222273 0.281932395
- 3.47400952 0.977555902
- 11.04549389 0.694010579
- 6.79729856 0.056652433
- 9.28300628 0.598930531
- -3.53453282 0.183412212
- 8.04028248 0.250746943
- 5.75964045 0.424692336
- -4.98252741 0.867446071
- 12.00352175 0.289615080
- 7.53497791 0.350915526
- 2.54579776 0.655113837
- -9.29572208 0.900136667
- 6.41659249 0.100570650
- 7.37095646 0.907179211
--12.78417775 0.262214556
- 0.87962107 0.624657444
- -5.96939907 0.296725805
- -2.56857339 0.604065931
- 4.27131811 0.962952479
- 21.72603838 0.442485270
- 6.10056565 0.418383130
- 10.48099521 0.333593221
- 19.28363092 0.382408442
- 2.12080726 0.601206970
- -6.82450704 0.740158518
- 11.32395692 0.627015570
- 5.00040701 0.476274658
--11.64750733 0.105099095
- 5.77442654 0.576560214
- 0.31340364 0.516479036
- -2.09881449 0.146089191
- 5.12411327 0.368130477
- 1.70530391 0.621828438
--12.95649749 0.355726301
- 8.43735652 0.275383759
--15.56161079 0.413160084
- 5.28942694 0.069125495
- 5.96040877 0.438716686
- -2.59318107 0.571116303
- 6.95988992 0.650760909
- 14.00074797 0.623645969
- 1.66101456 0.558763985
- -2.57968349 0.648185379
- -5.47584253 0.716901151
- 6.37222581 0.060563130
- 2.83664864 0.842419730
- 1.48926558 0.620280308
- 0.33471689 0.170312461
- 5.21648412 0.317639631
- 0.51733642 0.843867329
- 9.86005834 0.306036746
- -5.81145791 0.975655452
- -5.43219061 0.303385368
- 5.87157118 0.677369776
- 2.08889926 0.310200439
- -2.53433085 0.194730908
- 7.01359575 0.674259533
- -2.00936260 0.682056466
- -2.98240739 0.787899917
- -7.43289210 0.357483044
--12.58905988 0.981387385
- 5.78095517 0.533526274
- -1.23065889 0.687266774
- -6.82309960 0.293249774
- 8.47000829 0.842056399
- -5.81624772 0.303700280
--14.83571031 0.311387926
- 4.66808472 0.091222946
- -2.90144463 0.438301785
- 10.62458662 0.828335698
- 7.88002491 0.990156110
- 10.27680283 0.251087079
- -9.42498970 0.292462244
- 6.73027640 0.213065205
- 1.28169895 0.353152789
--14.29203733 0.264563048
- 20.35772711 0.265208837
- 3.55095071 0.242905653
--17.97067670 0.373951756
- 10.53141139 0.247520698
- 0.05293205 0.579940423
- 12.79674707 0.288031751
- -5.44235185 0.075899079
- 14.29464811 0.960707538
- -1.36753291 0.124265178
- -4.25946974 0.521720352
--12.46519252 0.385503339
- -6.65343143 0.540942219
- 5.55949184 0.143194404
- -1.20480594 0.515905644
- -4.13839908 0.164461445
- -2.21345425 0.812969725
- 3.94223380 0.229238952
--10.78661097 0.395049514
- 3.06997341 0.791234255
- 24.82205477 0.110859039
- 6.28791249 0.867125744
- -2.80296119 0.703583849
- 13.24274039 0.425951975
- -0.19577471 0.361568727
- -2.34894781 0.954814545
- 19.76339577 0.635462177
- -1.87591480 0.149121567
- -7.70962391 0.711708342
- -2.46291902 0.390902746
-end data
-regression /variables=v0 v1 /statistics defaults /dependent=v0 /method=enter.
-])
-
-AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-v0,F8.0
-v1,F8.0
-
-Table: Model Summary (v0)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.05,.00,.00,8.11
-
-Table: ANOVA (v0)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,235.23,1,235.23,3.58,.059
-Residual,98438.40,1498,65.71,,
-Total,98673.63,1499,,,
-
-Table: Coefficients (v0)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),1.24,.42,.00,2.95,.003
-v1,1.37,.72,.05,1.89,.059
-])
-
-AT_CLEANUP
-
-AT_SETUP([LINEAR REGRESSION no crash on all missing])
-AT_DATA([regcrash.sps], [dnl
-data list list /x * y.
-begin data.
- . .
- . .
- . .
- . .
- . .
- . .
- . .
- . .
- . .
- . .
-end data.
-
-
-regression /variables=x y /dependent=y.
-])
-
-AT_CHECK([pspp -o pspp.csv regcrash.sps], [1], [ignore], [ignore])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([LINEAR REGRESSION missing dependent variable])
-
-dnl Test for a bug where missing values in the dependent variable were not being
-dnl ignored like they should have been.
-AT_DATA([reg-mdv-ref.sps], [dnl
-data list notable list / v0 to v2.
-begin data
- 0.65377128 7.735648 -23.97588
--0.13087553 6.142625 -19.63854
- 0.34880368 7.651430 -25.26557
- 0.69249021 6.125125 -16.57090
--0.07368178 8.245789 -25.80001
--0.34404919 6.031540 -17.56743
- 0.75981559 9.832291 -28.35977
--0.46958313 5.343832 -16.79548
--0.06108490 8.838262 -29.25689
- 0.56154863 6.200189 -18.58219
-end data
-regression /variables=v0 v1
- /statistics defaults
- /dependent=v2
- /method=enter.
-])
-
-AT_CHECK([pspp -o pspp-ref.csv reg-mdv-ref.sps])
-
-AT_DATA([reg-mdv.sps], [dnl
-data list notable list / v0 to v2.
-begin data
- 0.65377128 7.735648 -23.97588
--0.13087553 6.142625 -19.63854
- 0.34880368 7.651430 -25.26557
- 0.69249021 6.125125 -16.57090
--0.07368178 8.245789 -25.80001
--0.34404919 6.031540 -17.56743
- 0.75981559 9.832291 -28.35977
--0.46958313 5.343832 -16.79548
--0.06108490 8.838262 -29.25689
- 0.56154863 6.200189 -18.58219
- 0.5 8 9
-end data
-
-missing values v2 (9).
-
-regression /variables=v0 v1
- /statistics defaults
- /dependent=v2
- /method=enter.
-])
-
-AT_CHECK([pspp -o pspp.csv reg-mdv.sps])
-
-AT_CHECK([diff pspp.csv pspp-ref.csv])
-
-
-AT_CLEANUP
-
-AT_SETUP([LINEAR REGRESSION with invalid syntax (and empty dataset)])
-
-AT_DATA([ss.sps], [dnl
-data list notable list / v0 to v2.
-begin data
-end data.
-
-regression /variables=v0 v1
- /statistics r coeff anova
- /dependent=v2
- /method=enter v2.
-])
-
-AT_CHECK([pspp ss.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-dnl The following example comes from
-dnl http://www.ats.ucla.edu/stat/spss/output/reg_spss%28long%29.htm
-AT_SETUP([LINEAR REGRESSION coefficient confidence interval])
-
-AT_DATA([conf.sps], [dnl
-set format = F22.3.
-
-data list notable list /math female socst read science *
-begin data.
- 41.00 .00 57.00 57.00 47.00
- 53.00 1.00 61.00 68.00 63.00
- 54.00 .00 31.00 44.00 58.00
- 47.00 .00 56.00 63.00 53.00
- 57.00 .00 61.00 47.00 53.00
- 51.00 .00 61.00 44.00 63.00
- 42.00 .00 61.00 50.00 53.00
- 45.00 .00 36.00 34.00 39.00
- 54.00 .00 51.00 63.00 58.00
- 52.00 .00 51.00 57.00 50.00
- 51.00 .00 61.00 60.00 53.00
- 51.00 .00 61.00 57.00 63.00
- 71.00 .00 71.00 73.00 61.00
- 57.00 .00 46.00 54.00 55.00
- 50.00 .00 56.00 45.00 31.00
- 43.00 .00 56.00 42.00 50.00
- 51.00 .00 56.00 47.00 50.00
- 60.00 .00 56.00 57.00 58.00
- 62.00 .00 61.00 68.00 55.00
- 57.00 .00 46.00 55.00 53.00
- 35.00 .00 41.00 63.00 66.00
- 75.00 .00 66.00 63.00 72.00
- 45.00 .00 56.00 50.00 55.00
- 57.00 .00 61.00 60.00 61.00
- 45.00 .00 46.00 37.00 39.00
- 46.00 .00 31.00 34.00 39.00
- 66.00 .00 66.00 65.00 61.00
- 57.00 .00 46.00 47.00 58.00
- 49.00 .00 46.00 44.00 39.00
- 49.00 .00 41.00 52.00 55.00
- 57.00 .00 51.00 42.00 47.00
- 64.00 .00 61.00 76.00 64.00
- 63.00 .00 71.00 65.00 66.00
- 57.00 .00 31.00 42.00 72.00
- 50.00 .00 61.00 52.00 61.00
- 58.00 .00 66.00 60.00 61.00
- 75.00 .00 66.00 68.00 66.00
- 68.00 .00 66.00 65.00 66.00
- 44.00 .00 36.00 47.00 36.00
- 40.00 .00 51.00 39.00 39.00
- 41.00 .00 51.00 47.00 42.00
- 62.00 .00 51.00 55.00 58.00
- 57.00 .00 51.00 52.00 55.00
- 43.00 .00 41.00 42.00 50.00
- 48.00 .00 66.00 65.00 63.00
- 63.00 .00 46.00 55.00 69.00
- 39.00 .00 47.00 50.00 49.00
- 70.00 .00 51.00 65.00 63.00
- 63.00 .00 46.00 47.00 53.00
- 59.00 .00 51.00 57.00 47.00
- 61.00 .00 56.00 53.00 57.00
- 38.00 .00 41.00 39.00 47.00
- 61.00 .00 46.00 44.00 50.00
- 49.00 .00 71.00 63.00 55.00
- 73.00 .00 66.00 73.00 69.00
- 44.00 .00 42.00 39.00 26.00
- 42.00 .00 32.00 37.00 33.00
- 39.00 .00 46.00 42.00 56.00
- 55.00 .00 41.00 63.00 58.00
- 52.00 .00 51.00 48.00 44.00
- 45.00 .00 61.00 50.00 58.00
- 61.00 .00 66.00 47.00 69.00
- 39.00 .00 46.00 44.00 34.00
- 41.00 .00 36.00 34.00 36.00
- 50.00 .00 61.00 50.00 36.00
- 40.00 .00 26.00 44.00 50.00
- 60.00 .00 66.00 60.00 55.00
- 47.00 .00 26.00 47.00 42.00
- 59.00 .00 44.00 63.00 65.00
- 49.00 .00 36.00 50.00 44.00
- 46.00 .00 51.00 44.00 39.00
- 58.00 .00 61.00 60.00 58.00
- 71.00 .00 66.00 73.00 63.00
- 58.00 .00 66.00 68.00 74.00
- 46.00 .00 51.00 55.00 58.00
- 43.00 .00 31.00 47.00 45.00
- 54.00 .00 61.00 55.00 49.00
- 56.00 .00 66.00 68.00 63.00
- 46.00 .00 46.00 31.00 39.00
- 54.00 .00 56.00 47.00 42.00
- 57.00 .00 56.00 63.00 55.00
- 54.00 .00 36.00 36.00 61.00
- 71.00 .00 56.00 68.00 66.00
- 48.00 .00 56.00 63.00 63.00
- 40.00 .00 41.00 55.00 44.00
- 64.00 .00 66.00 55.00 63.00
- 51.00 .00 56.00 52.00 53.00
- 39.00 .00 56.00 34.00 42.00
- 40.00 .00 31.00 50.00 34.00
- 61.00 .00 56.00 55.00 61.00
- 66.00 .00 46.00 52.00 47.00
- 49.00 .00 46.00 63.00 66.00
- 65.00 1.00 61.00 68.00 69.00
- 52.00 1.00 48.00 39.00 44.00
- 46.00 1.00 51.00 44.00 47.00
- 61.00 1.00 51.00 50.00 63.00
- 72.00 1.00 56.00 71.00 66.00
- 71.00 1.00 71.00 63.00 69.00
- 40.00 1.00 41.00 34.00 39.00
- 69.00 1.00 61.00 63.00 61.00
- 64.00 1.00 66.00 68.00 69.00
- 56.00 1.00 61.00 47.00 66.00
- 49.00 1.00 41.00 47.00 33.00
- 54.00 1.00 51.00 63.00 50.00
- 53.00 1.00 51.00 52.00 61.00
- 66.00 1.00 56.00 55.00 42.00
- 67.00 1.00 56.00 60.00 50.00
- 40.00 1.00 33.00 35.00 51.00
- 46.00 1.00 56.00 47.00 50.00
- 69.00 1.00 71.00 71.00 58.00
- 40.00 1.00 56.00 57.00 61.00
- 41.00 1.00 51.00 44.00 39.00
- 57.00 1.00 66.00 65.00 46.00
- 58.00 1.00 56.00 68.00 59.00
- 57.00 1.00 66.00 73.00 55.00
- 37.00 1.00 41.00 36.00 42.00
- 55.00 1.00 46.00 43.00 55.00
- 62.00 1.00 66.00 73.00 58.00
- 64.00 1.00 56.00 52.00 58.00
- 40.00 1.00 51.00 41.00 39.00
- 50.00 1.00 51.00 60.00 50.00
- 46.00 1.00 56.00 50.00 50.00
- 53.00 1.00 56.00 50.00 39.00
- 52.00 1.00 46.00 47.00 48.00
- 45.00 1.00 46.00 47.00 34.00
- 56.00 1.00 61.00 55.00 58.00
- 45.00 1.00 56.00 50.00 44.00
- 54.00 1.00 41.00 39.00 50.00
- 56.00 1.00 46.00 50.00 47.00
- 41.00 1.00 26.00 34.00 29.00
- 54.00 1.00 56.00 57.00 50.00
- 72.00 1.00 56.00 57.00 54.00
- 56.00 1.00 51.00 68.00 50.00
- 47.00 1.00 46.00 42.00 47.00
- 49.00 1.00 66.00 61.00 44.00
- 60.00 1.00 66.00 76.00 67.00
- 54.00 1.00 46.00 47.00 58.00
- 55.00 1.00 56.00 46.00 44.00
- 33.00 1.00 41.00 39.00 42.00
- 49.00 1.00 61.00 52.00 44.00
- 43.00 1.00 51.00 28.00 44.00
- 50.00 1.00 52.00 42.00 50.00
- 52.00 1.00 51.00 47.00 39.00
- 48.00 1.00 41.00 47.00 44.00
- 58.00 1.00 66.00 52.00 53.00
- 43.00 1.00 61.00 47.00 48.00
- 41.00 1.00 31.00 50.00 55.00
- 43.00 1.00 51.00 44.00 44.00
- 46.00 1.00 41.00 47.00 40.00
- 44.00 1.00 41.00 45.00 34.00
- 43.00 1.00 46.00 47.00 42.00
- 61.00 1.00 56.00 65.00 58.00
- 40.00 1.00 51.00 43.00 50.00
- 49.00 1.00 61.00 47.00 53.00
- 56.00 1.00 66.00 57.00 58.00
- 61.00 1.00 71.00 68.00 55.00
- 50.00 1.00 61.00 52.00 54.00
- 51.00 1.00 61.00 42.00 47.00
- 42.00 1.00 41.00 42.00 42.00
- 67.00 1.00 66.00 66.00 61.00
- 53.00 1.00 61.00 47.00 53.00
- 50.00 1.00 58.00 57.00 51.00
- 51.00 1.00 31.00 47.00 63.00
- 72.00 1.00 61.00 57.00 61.00
- 48.00 1.00 61.00 52.00 55.00
- 40.00 1.00 31.00 44.00 40.00
- 53.00 1.00 61.00 50.00 61.00
- 39.00 1.00 36.00 39.00 47.00
- 63.00 1.00 41.00 57.00 55.00
- 51.00 1.00 37.00 57.00 53.00
- 45.00 1.00 43.00 42.00 50.00
- 39.00 1.00 61.00 47.00 47.00
- 42.00 1.00 39.00 42.00 31.00
- 62.00 1.00 51.00 60.00 61.00
- 44.00 1.00 51.00 44.00 35.00
- 65.00 1.00 66.00 63.00 54.00
- 63.00 1.00 71.00 65.00 55.00
- 54.00 1.00 41.00 39.00 53.00
- 45.00 1.00 36.00 50.00 58.00
- 60.00 1.00 51.00 52.00 56.00
- 49.00 1.00 51.00 60.00 50.00
- 48.00 1.00 51.00 44.00 39.00
- 57.00 1.00 61.00 52.00 63.00
- 55.00 1.00 61.00 55.00 50.00
- 66.00 1.00 56.00 50.00 66.00
- 64.00 1.00 71.00 65.00 58.00
- 55.00 1.00 51.00 52.00 53.00
- 42.00 1.00 36.00 47.00 42.00
- 56.00 1.00 61.00 63.00 55.00
- 53.00 1.00 66.00 50.00 53.00
- 41.00 1.00 41.00 42.00 42.00
- 42.00 1.00 41.00 36.00 50.00
- 53.00 1.00 56.00 50.00 55.00
- 42.00 1.00 51.00 41.00 34.00
- 60.00 1.00 56.00 47.00 50.00
- 52.00 1.00 56.00 55.00 42.00
- 38.00 1.00 46.00 42.00 36.00
- 57.00 1.00 52.00 57.00 55.00
- 58.00 1.00 61.00 55.00 58.00
- 65.00 1.00 61.00 63.00 53.00
-end data.
-
-regression
- /variables = math female socst read
- /statistics = coeff r anova ci (95)
- /dependent = science
- /method = enter
-])
-
-AT_CHECK([pspp -O format=csv conf.sps], [0], [dnl
-Table: Model Summary (science)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.699,.489,.479,7.148
-
-Table: ANOVA (science)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,9543.721,4,2385.930,46.695,.000
-Residual,9963.779,195,51.096,,
-Total,19507.500,199,,,
-
-Table: Coefficients (science)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.,95% Confidence Interval for B,
-,B,Std. Error,Beta,,,Lower Bound,Upper Bound
-(Constant),12.325,3.194,.000,3.859,.000,6.027,18.624
-math,.389,.074,.368,5.252,.000,.243,.535
-female,-2.010,1.023,-.101,-1.965,.051,-4.027,.007
-socst,.050,.062,.054,.801,.424,-.073,.173
-read,.335,.073,.347,4.607,.000,.192,.479
-])
-
-
-AT_CLEANUP
-
-
-dnl Checks for regression against bug #44877.
-AT_SETUP([LINEAR REGRESSION crash with long string variables])
-AT_DATA([regression.sps], [dnl
-SET DECIMAL=DOT.
-
-DATA LIST notable LIST /text (A24) Y * X1 *
-BEGIN DATA.
-V00276601 0.00 90.00
-V00292909 10.00 30.00
-V00291204 20.00 20.00
-V00300070 0.00 90.00
-END DATA.
-
-REGRESSION
-/VARIABLES= Y
-/DEPENDENT= X1
-/METHOD=ENTER
-/STATISTICS=COEFF R ANOVA
-/SAVE= RESID.
-
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv regression.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Model Summary (X1)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.95,.89,.84,15.08
-
-Table: ANOVA (X1)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,3820.45,1,3820.45,16.81,.055
-Residual,454.55,2,227.27,,
-Total,4275.00,3,,,
-
-Table: Coefficients (X1)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),85.45,10.16,.00,8.41,.004
-Y,-3.73,.91,-.95,-4.10,.055
-
-Table: Data List
-text,Y,X1,RES1
-V00276601,.00,90.00,4.55
-V00292909,10.00,30.00,-18.18
-V00291204,20.00,20.00,9.09
-V00300070,.00,90.00,4.55
-])
-AT_CLEANUP
-
-
-dnl Test for a crash which happened on bad input syntax
-AT_SETUP([LINEAR REGRESSION -- Empty Parentheses])
-
-AT_DATA([empty-parens.sps], [dnl
-set format = F22.3.
-
-data list notable list /math female socst read science *
-begin data.
- 58.00 1.00 61.00 55.00 58.00
- 65.00 1.00 61.00 63.00 53.00
-end data.
-
-regression
- /variables = math female socst read
- /statistics = coeff r anova ci ()
- /dependent = science
- /method = enter
-])
-
-AT_CHECK([pspp -o pspp.csv empty-parens.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-
-
-AT_SETUP([LINEAR REGRESSION varibles on ENTER subcommand])
-AT_DATA([regression.sps], [dnl
-SET FORMAT=F10.3.
-
-DATA LIST notable LIST /number * value *.
-BEGIN DATA
- 16 7.25
- 0 .00
- 1 .10
- 9 27.9
- 0 .00
- 7 3.65
- 14 16.8
- 24 9.15
- 0 .00
- 24 19.0
- 7 4.05
- 12 7.90
- 6 .75
- 11 1.40
- 0 .00
- 3 2.30
- 12 7.60
- 11 6.80
- 16 8.65
-END DATA.
-
-REGRESSION
- /STATISTICS COEFF R ANOVA
- /DEPENDENT value
- /METHOD=ENTER number.
-])
-
-
-AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
-Table: Model Summary (value)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.612,.374,.338,6.176
-
-Table: ANOVA (value)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,388.065,1,388.065,10.173,.005
-Residual,648.498,17,38.147,,
-Total,1036.563,18,,,
-
-Table: Coefficients (value)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),.927,2.247,.000,.413,.685
-number,.611,.192,.612,3.189,.005
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([LINEAR REGRESSION /ORIGIN])
-AT_DATA([regression-origin.sps], [dnl
-SET FORMAT=F10.3.
-
-DATA LIST notable LIST /number * value *.
-BEGIN DATA
- 16 7.25
- 0 .00
- 1 .10
- 9 27.9
- 0 .00
- 7 3.65
- 14 16.8
- 24 9.15
- 0 .00
- 24 19.0
- 7 4.05
- 12 7.90
- 6 .75
- 11 1.40
- 0 .00
- 3 2.30
- 12 7.60
- 11 6.80
- 16 8.65
-END DATA.
-
-REGRESSION
- /STATISTICS COEFF R ANOVA
- /DEPENDENT value
- /ORIGIN
- /METHOD=ENTER number.
-])
-
-
-AT_CHECK([pspp -O format=csv regression-origin.sps], [0], [dnl
-Table: Model Summary (value)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.802,.643,.622,6.032
-
-Table: ANOVA (value)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,1181.726,1,1181.726,32.475,.000
-Residual,654.989,18,36.388,,
-Total,1836.715,19,,,
-
-Table: Coefficients (value)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-number,.672,.118,.802,5.699,.000
-])
-
-AT_CLEANUP
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([REGRESSION tutorial example])
-cp $top_srcdir/examples/repairs.sav .
-AT_DATA([regression.sps], [dnl
-GET FILE='repairs.sav'.
-REGRESSION /VARIABLES=mtbf duty_cycle /DEPENDENT=mttr.
-REGRESSION /VARIABLES=mtbf /DEPENDENT=mttr.
-])
-
-AT_CHECK([pspp -O format=csv regression.sps], [0], [dnl
-Table: Model Summary (Mean time to repair (hours) )
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.94,.89,.88,6.54
-
-Table: ANOVA (Mean time to repair (hours) )
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,9576.26,2,4788.13,111.94,.000
-Residual,1154.94,27,42.78,,
-Total,10731.20,29,,,
-
-Table: Coefficients (Mean time to repair (hours) )
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),10.59,3.11,.00,3.40,.002
-Mean time between failures (months) ,3.02,.20,.95,14.88,.000
-Ratio of working to non-working time,-1.12,3.69,-.02,-.30,.763
-
-Table: Model Summary (Mean time to repair (hours) )
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.94,.89,.89,6.43
-
-Table: ANOVA (Mean time to repair (hours) )
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,9572.30,1,9572.30,231.28,.000
-Residual,1158.90,28,41.39,,
-Total,10731.20,29,,,
-
-Table: Coefficients (Mean time to repair (hours) )
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.
-,B,Std. Error,Beta,,
-(Constant),9.90,2.10,.00,4.71,.000
-Mean time between failures (months) ,3.01,.20,.94,15.21,.000
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([LINEAR REGRESSION vif])
-AT_DATA([regression-vif.sps], [dnl
-SET FORMAT=F10.3.
-
-data list notable list /competence_x1 motivation_x2 performance_y.
-begin data
-32 34 36
-35 37 39
-38 45 49
-31 41 41
-36 40 38
-32 38 36
-33 39 37
-31 40 41
-30 37 40
-35 37 43
-31 34 36
-34 32 35
-31 42 34
-25 36 40
-35 42 40
-36 41 44
-30 38 32
-34 41 41
-34 41 44
-22 27 26
-27 26 33
-30 30 35
-30 35 37
-37 39 44
-29 35 36
-31 35 29
-31 45 41
-29 30 32
-29 35 36
-31 37 37
-36 45 42
-32 44 39
-27 26 31
-33 39 35
-20 25 28
-30 36 39
-27 37 39
-25 39 36
-32 38 38
-32 38 35
-end data.
-
-regression /variables=competence_x1 motivation_x2
- /statistics=defaults tol
- /dependent=performance_y
- .
-])
-
-
-AT_CHECK([pspp -O format=csv regression-vif.sps], [0], [dnl
-Table: Model Summary (performance_y)
-R,R Square,Adjusted R Square,Std. Error of the Estimate
-.785,.616,.595,2.980
-
-Table: ANOVA (performance_y)
-,Sum of Squares,df,Mean Square,F,Sig.
-Regression,526.494,2,263.247,29.641,.000
-Residual,328.606,37,8.881,,
-Total,855.100,39,,,
-
-Table: Coefficients (performance_y)
-,Unstandardized Coefficients,,Standardized Coefficients,t,Sig.,Collinearity Statistics,
-,B,Std. Error,Beta,,,Tolerance,VIF
-(Constant),7.220,4.020,.000,1.796,.080,,
-competence_x1,.432,.166,.358,2.609,.013,.552,1.812
-motivation_x2,.453,.125,.499,3.636,.001,.552,1.812
-])
-
-AT_CLEANUP
-
-AT_SETUP([REGRESSION syntax errors])
-AT_DATA([regression.sps], [dnl
-DATA LIST LIST NOTABLE /x y z.
-REGRESSION VARIABLES=**.
-REGRESSION METHOD=ENTER x/VARIABLES.
-REGRESSION DEPENDENT=x/VARIABLES.
-REGRESSION DEPENDENT=**.
-REGRESSION METHOD=**.
-REGRESSION METHOD=ENTER **.
-REGRESSION STATISTICS=**.
-REGRESSION STATISTICS=CI(**).
-REGRESSION STATISTICS=CI(1 **).
-REGRESSION SAVE=**.
-REGRESSION **.
-])
-AT_CHECK([pspp -O format=csv regression.sps], [1], [dnl
-"regression.sps:2.22-2.23: error: REGRESSION: Syntax error expecting variable name.
- 2 | REGRESSION VARIABLES=**.
- | ^~"
-
-"regression.sps:3.27-3.35: error: REGRESSION: VARIABLES may not appear after METHOD.
- 3 | REGRESSION METHOD=ENTER x/VARIABLES.
- | ^~~~~~~~~"
-
-"regression.sps:4.24-4.32: error: REGRESSION: VARIABLES may not appear after DEPENDENT.
- 4 | REGRESSION DEPENDENT=x/VARIABLES.
- | ^~~~~~~~~"
-
-"regression.sps:5.22-5.23: error: REGRESSION: Syntax error expecting variable name.
- 5 | REGRESSION DEPENDENT=**.
- | ^~"
-
-"regression.sps:6.19-6.20: error: REGRESSION: Syntax error expecting ENTER.
- 6 | REGRESSION METHOD=**.
- | ^~"
-
-"regression.sps:7.25-7.26: error: REGRESSION: Syntax error expecting variable name.
- 7 | REGRESSION METHOD=ENTER **.
- | ^~"
-
-"regression.sps:8.23-8.24: error: REGRESSION: Syntax error expecting ALL, DEFAULTS, R, COEFF, ANOVA, BCOV, TOL, or CI.
- 8 | REGRESSION STATISTICS=**.
- | ^~"
-
-"regression.sps:9.26-9.27: error: REGRESSION: Syntax error expecting number.
- 9 | REGRESSION STATISTICS=CI(**).
- | ^~"
-
-"regression.sps:10.28-10.29: error: REGRESSION: Syntax error expecting `@:}@'.
- 10 | REGRESSION STATISTICS=CI(1 **).
- | ^~"
-
-"regression.sps:11.17-11.18: error: REGRESSION: Syntax error expecting PRED or RESID.
- 11 | REGRESSION SAVE=**.
- | ^~"
-
-"regression.sps:12.12-12.13: error: REGRESSION: Syntax error expecting VARIABLES, DEPENDENT, ORIGIN, NOORIGIN, METHOD, STATISTICS, or SAVE.
- 12 | REGRESSION **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([RELIABILITY])
-
-AT_SETUP([RELIABILITY])
-AT_DATA([reliability.sps], [dnl
-data list notable list /var1 *
- var2 *
- var6 *
- var7 *
- var8 *
- var9 *
- var11 *
- var12 *
- var15 *
- var16 *
- var17 *
- var19 *
- .
-
-begin data.
-6 7 7 5 7 7 7 7 7 7 6 6
-6 7 7 6 7 6 7 5 6 5 7 7
-6 6 7 6 5 3 6 4 5 6 4 5
-4 6 5 6 6 5 4 3 5 6 5 6
-5 6 5 5 6 5 4 4 6 6 5 5
-6 6 7 6 6 5 6 5 6 6 5 6
-5 6 6 5 6 5 5 4 6 5 5 5
-5 7 7 7 7 7 6 5 7 7 7 7
-6 6 6 5 5 7 6 5 6 6 5 6
-. . . . . . . . . . . .
-6 6 5 5 5 6 6 4 6 5 5 5
-7 7 7 6 7 6 7 6 6 6 7 6
-4 7 6 6 6 5 5 4 4 5 5 6
-5 6 3 5 4 1 4 6 2 3 3 2
-3 6 6 5 6 2 4 2 2 4 4 5
-6 6 7 5 6 5 7 6 5 6 6 5
-6 5 6 6 5 6 6 6 6 4 5 5
-5 7 7 . 6 6 6 5 6 6 6 6
-5 7 5 5 4 6 7 6 5 4 6 5
-7 7 7 6 7 7 7 6 7 7 7 6
-3 6 5 6 5 7 7 3 4 7 5 7
-6 7 7 6 5 6 5 5 6 6 6 6
-5 5 6 5 5 5 5 4 5 5 5 6
-6 6 7 4 5 6 6 6 6 5 5 6
-6 5 6 6 4 4 5 4 5 6 4 5
-5 6 7 6 6 7 7 5 6 6 6 5
-5 6 5 7 4 6 6 5 7 7 5 6
-. . . . . . . . . . . .
-7 6 6 5 6 6 7 6 6 5 5 6
-6 6 7 7 7 7 7 6 7 6 6 7
-7 5 5 . 5 . 7 3 5 4 5 3
-7 6 7 5 4 5 7 5 7 5 5 6
-6 5 6 6 6 5 5 5 5 6 5 6
-7 7 7 7 7 7 7 7 5 6 7 7
-. . . . . . . . . . . .
-5 5 6 7 5 6 6 4 6 6 6 5
-6 6 5 7 5 6 7 5 6 5 4 6
-7 6 7 6 7 5 6 7 7 6 6 6
-5 6 5 6 5 6 7 2 5 7 3 7
-6 6 5 6 5 6 6 6 6 6 5 6
-7 6 7 6 6 6 6 6 6 7 6 7
-7 7 6 5 6 6 7 7 7 4 6 5
-3 7 7 6 6 7 7 7 6 6 6 4
-3 5 3 4 3 3 3 3 3 3 3 5
-5 7 7 7 5 7 6 2 6 7 6 7
-7 7 7 7 7 7 7 6 7 7 7 6
-6 5 7 4 4 4 5 6 5 5 4 5
-4 7 7 4 4 3 6 3 5 3 4 5
-7 7 7 7 7 7 7 7 7 7 7 5
-3 6 5 5 4 5 4 4 5 5 3 5
-6 7 6 6 6 7 7 6 6 6 7 6
-2 5 4 6 3 2 2 2 2 7 2 2
-4 6 6 5 5 5 6 5 5 6 6 5
-5 7 4 5 6 6 6 5 6 6 5 6
-5 7 7 5 6 5 6 5 5 4 5 4
-4 5 6 5 6 4 5 5 5 4 5 5
-7 6 6 5 5 6 7 5 6 5 7 6
-5 6 6 5 4 5 5 3 4 5 5 5
-5 7 6 4 4 5 6 5 6 4 4 6
-6 6 6 6 5 7 7 6 5 5 6 6
-6 6 7 6 7 6 6 5 6 7 6 5
-7 6 7 6 7 6 7 7 5 5 6 6
-5 6 6 5 5 5 6 5 6 7 7 5
-5 6 6 5 6 5 6 6 6 6 6 6
-5 5 5 5 6 4 5 3 4 7 6 5
-5 7 7 6 6 6 6 5 6 7 6 7
-6 6 7 7 7 5 6 5 5 5 5 4
-2 7 5 4 6 5 5 2 5 6 4 6
-6 7 7 5 6 6 7 6 6 7 5 7
-5 6 7 6 6 3 5 7 6 6 5 6
-6 6 6 3 5 5 5 6 6 6 4 5
-4 7 7 4 7 4 5 5 5 7 4 4
-. . . . . . . . . . . .
-6 6 7 6 7 6 7 7 6 7 7 6
-. . . . . . . . . . . .
-5 6 5 7 6 5 6 6 5 6 4 6
-5 5 5 5 4 5 5 5 7 5 5 5
-6 6 6 4 5 4 6 6 6 4 5 4
-6 5 7 4 6 4 6 5 6 6 6 3
-5 7 6 5 5 5 5 5 6 7 6 6
-5 5 7 7 5 5 6 6 5 5 5 7
-5 6 7 6 7 5 6 4 6 7 6 7
-4 5 5 5 6 5 6 5 6 6 5 6
-6 5 5 5 6 3 4 5 5 4 5 3
-6 6 6 5 5 5 4 3 4 5 5 5
-6 7 7 6 2 3 6 6 6 5 7 7
-6 7 5 5 6 6 6 5 6 6 6 6
-6 7 7 6 7 7 7 5 5 6 6 6
-6 6 6 6 7 6 6 7 6 6 6 6
-5 6 6 6 3 5 6 6 5 5 4 6
-4 6 5 6 6 5 6 5 6 6 5 5
-6 4 6 5 4 6 7 4 5 6 5 5
-6 7 6 4 6 5 7 6 7 7 6 5
-6 7 7 6 7 6 7 7 7 6 6 6
-6 6 6 4 5 6 7 7 5 6 4 4
-3 3 5 3 3 1 5 6 3 2 3 3
-7 7 5 6 6 7 7 6 7 7 7 7
-5 6 6 6 7 5 4 5 4 7 6 7
-3 6 5 4 3 3 3 5 5 6 3 4
-5 7 6 4 6 5 5 6 6 7 5 6
-5 7 6 6 6 6 6 5 6 7 7 6
-7 7 5 6 7 7 7 7 6 5 7 7
-6 7 6 6 5 6 7 7 6 5 6 6
-6 7 7 7 7 6 6 7 6 7 7 7
-4 6 4 7 3 6 5 5 4 3 5 6
-5 5 7 5 4 6 7 5 4 6 6 5
-5 5 6 4 6 5 7 6 5 5 5 6
-. . . . . . . . . . . .
-. . . . . . . . . . . .
-5 7 7 5 6 6 7 7 6 6 6 7
-6 7 7 1 2 1 7 7 5 5 5 2
-. . . . . . . . . . . .
-3 7 4 6 4 7 4 6 4 7 4 7
-5 7 3 5 5 6 7 5 4 7 7 4
-4 7 7 5 4 6 7 7 6 5 4 4
-6 6 2 2 6 4 6 5 5 1 5 2
-5 5 6 4 5 4 6 5 5 6 5 5
-. . . . . . . . . . . .
-5 7 6 6 6 6 6 6 5 6 6 6
-6 6 6 5 6 6 6 6 7 5 6 7
-3 6 3 3 5 3 3 5 3 5 7 4
-4 4 6 3 3 3 4 3 4 2 3 6
-5 7 7 6 5 4 7 5 7 7 3 7
-4 5 4 4 4 4 3 3 3 4 3 3
-6 7 7 5 6 6 7 5 4 5 5 5
-3 5 3 3 1 3 4 3 4 7 6 7
-4 5 4 4 4 3 4 5 6 6 4 5
-5 6 3 4 5 3 5 3 4 5 6 4
-5 5 5 6 6 6 6 4 5 6 6 5
-6 7 7 2 2 6 7 7 7 7 5 7
-5 7 7 4 6 5 7 5 5 5 6 6
-6 6 7 7 5 5 5 7 6 7 7 7
-6 5 7 3 6 5 6 5 5 6 5 4
-5 7 6 5 6 6 6 5 6 5 5 6
-4 5 5 5 6 3 5 3 3 6 5 5
-. . . . . . . . . . . .
-5 6 6 4 4 4 5 3 5 5 2 6
-5 6 7 5 5 6 6 5 5 6 6 6
-6 7 7 6 4 7 7 6 7 5 6 7
-6 6 5 4 5 2 7 6 6 5 6 6
-2 2 2 2 2 2 3 2 3 1 1 2
-end data.
-
-RELIABILITY
- /VARIABLES=var2 var8 var15 var17 var6
- /SCALE('Everything') var6 var8 var15 var17
- /MODEL=ALPHA.
-
-RELIABILITY
- /VARIABLES=var6 var8 var15 var17
- /SCALE('Nothing') ALL
- /MODEL=SPLIT(2)
- .
-
-RELIABILITY
- /VARIABLES=var2 var6 var8 var15 var17 var19
- /SCALE('Totals') var6 var8 var15 var17
- /SUMMARY = total
- /STATISTICS = DESCRIPTIVES COVARIANCES
- .
-
-
-RELIABILITY
- /VARIABLES=var6 var8 var15 var17
- .
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt reliability.sps], [0], [dnl
-reliability.sps:174.4-174.40: warning: RELIABILITY: The STATISTICS subcommand is not yet implemented. No statistics will be produced.
- 174 | /STATISTICS = DESCRIPTIVES COVARIANCES
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Scale: Everything
-
-Table: Case Processing Summary
-Cases,N,Percent
-Valid,131,92.9%
-Excluded,10,7.1%
-Total,141,100.0%
-
-Table: Reliability Statistics
-Cronbach's Alpha,N of Items
-.75,4
-
-Scale: Nothing
-
-Table: Case Processing Summary
-Cases,N,Percent
-Valid,131,92.9%
-Excluded,10,7.1%
-Total,141,100.0%
-
-Table: Reliability Statistics
-Cronbach's Alpha,Part 1,Value,.55
-,,N of Items,2
-,Part 2,Value,.63
-,,N of Items,2
-,Total N of Items,,4
-Correlation Between Forms,,,.61
-Spearman-Brown Coefficient,Equal Length,,.75
-,Unequal Length,,.75
-Guttman Split-Half Coefficient,,,.75
-
-"reliability.sps:174.4-174.40: warning: RELIABILITY: The STATISTICS subcommand is not yet implemented. No statistics will be produced.
- 174 | /STATISTICS = DESCRIPTIVES COVARIANCES
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-
-Scale: Totals
-
-Table: Case Processing Summary
-Cases,N,Percent
-Valid,131,92.9%
-Excluded,10,7.1%
-Total,141,100.0%
-
-Table: Reliability Statistics
-Cronbach's Alpha,N of Items
-.75,4
-
-Table: Item-Total Statistics
-,Scale Mean if Item Deleted,Scale Variance if Item Deleted,Corrected Item-Total Correlation,Cronbach's Alpha if Item Deleted
-var6,15.97,8.43,.51,.71
-var8,16.56,7.86,.53,.70
-var15,16.47,8.45,.56,.68
-var17,16.60,8.00,.57,.67
-
-Scale: ANY
-
-Table: Case Processing Summary
-Cases,N,Percent
-Valid,131,92.9%
-Excluded,10,7.1%
-Total,141,100.0%
-
-Table: Reliability Statistics
-Cronbach's Alpha,N of Items
-.75,4
-])
-AT_CLEANUP
-
-
-dnl This was causing a AT.
-AT_SETUP([RELIABILITY bad syntax])
-AT_DATA([bad-syntax.sps], [dnl
-data list notable list /x * y *.
-begin data.
-1 10
-2 20
-3 30
-4 50
-5 50
-end data.
-
-* This syntax is incorrect
-reliability x y.
-])
-
-AT_CHECK([pspp -O format=csv bad-syntax.sps], [1], [ignore])
-
-AT_CLEANUP
-
-dnl Checks for a crash when bad syntax followed scale specification.
-AT_SETUP([RELIABILITY bad syntax 2])
-AT_DATA([bad-syntax.sps], [dnl
-new file.
-data list notable list /f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 *.
-begin data.
-end data.
-
-* This syntax is incorrect
-reliability variables=f01 to f10/asdfj.
-])
-AT_CHECK([pspp -O format=csv bad-syntax.sps], [1], [ignore])
-AT_CLEANUP
-
-
-dnl Checks for a crash when the active file was empty. Bug #38660.
-AT_SETUP([RELIABILITY crash with no data])
-AT_DATA([reliability.sps], [dnl
-new file.
-data list notable list /f01 f02 f03 f04 f05 f06 f07 f08 f09 f10 *.
-begin data.
-end data.
-
-reliability variables=f01 to f10.
-])
-AT_CHECK([pspp -O format=csv reliability.sps], [0], [])
-AT_CLEANUP
-
-
-
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([RELIABILITY tutorial example])
-AT_DATA([tut-example.sps], [dnl
-get file='hotel.sav'.
-
-compute v3 = 6 - v3.
-compute v5 = 6 - v5.
-
-reliability variables = v1 v3 v4.
-])
-
-AT_CHECK([ln -s $top_srcdir/examples/hotel.sav .], [0])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt tut-example.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Scale: ANY
-
-Table: Case Processing Summary
-Cases,N,Percent
-Valid,17,100.0%
-Excluded,0,.0%
-Total,17,100.0%
-
-Table: Reliability Statistics
-Cronbach's Alpha,N of Items
-.81,3
-])
-AT_CLEANUP
-
-AT_SETUP([RELIABILITY syntax errors])
-AT_DATA([reliability.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-RELIABILITY **.
-RELIABILITY VARIABLES=**.
-RELIABILITY VARIABLES=x/ **.
-RELIABILITY VARIABLES=x y/SCALE **.
-RELIABILITY VARIABLES=x y/SCALE(**).
-RELIABILITY VARIABLES=x y/SCALE('a' **).
-RELIABILITY VARIABLES=x y/SCALE('a')=**.
-RELIABILITY VARIABLES=x y/MODEL SPLIT(**).
-RELIABILITY VARIABLES=x y/MODEL SPLIT(1 **).
-RELIABILITY VARIABLES=x y/MODEL **.
-RELIABILITY VARIABLES=x y/SUMMARY **.
-RELIABILITY VARIABLES=x y/MISSING=**.
-RELIABILITY VARIABLES=x y/STATISTICS=x y zz y/ **.
-RELIABILITY VARIABLES=x y/MODEL SPLIT(5).
-])
-AT_CHECK([pspp -O format=csv reliability.sps], [1], [dnl
-"reliability.sps:2.13-2.14: error: RELIABILITY: Syntax error expecting VARIABLES.
- 2 | RELIABILITY **.
- | ^~"
-
-"reliability.sps:3.23-3.24: error: RELIABILITY: Syntax error expecting variable name.
- 3 | RELIABILITY VARIABLES=**.
- | ^~"
-
-"reliability.sps:4.23: warning: RELIABILITY: Reliability on a single variable is not useful.
- 4 | RELIABILITY VARIABLES=x/ **.
- | ^"
-
-"reliability.sps:4.26-4.27: error: RELIABILITY: Syntax error expecting SCALE, MODEL, SUMMARY, MISSING, or STATISTICS.
- 4 | RELIABILITY VARIABLES=x/ **.
- | ^~"
-
-"reliability.sps:5.33-5.34: error: RELIABILITY: Syntax error expecting `('.
- 5 | RELIABILITY VARIABLES=x y/SCALE **.
- | ^~"
-
-"reliability.sps:6.33-6.34: error: RELIABILITY: Syntax error expecting string.
- 6 | RELIABILITY VARIABLES=x y/SCALE(**).
- | ^~"
-
-"reliability.sps:7.37-7.38: error: RELIABILITY: Syntax error expecting `)'.
- 7 | RELIABILITY VARIABLES=x y/SCALE('a' **).
- | ^~"
-
-"reliability.sps:8.38-8.39: error: RELIABILITY: Syntax error expecting variable name.
- 8 | RELIABILITY VARIABLES=x y/SCALE('a')=**.
- | ^~"
-
-"reliability.sps:9.39-9.40: error: RELIABILITY: Syntax error expecting number.
- 9 | RELIABILITY VARIABLES=x y/MODEL SPLIT(**).
- | ^~"
-
-"reliability.sps:10.41-10.42: error: RELIABILITY: Syntax error expecting `@:}@'.
- 10 | RELIABILITY VARIABLES=x y/MODEL SPLIT(1 **).
- | ^~"
-
-"reliability.sps:11.33-11.34: error: RELIABILITY: Syntax error expecting ALPHA or SPLIT.
- 11 | RELIABILITY VARIABLES=x y/MODEL **.
- | ^~"
-
-"reliability.sps:12.35-12.36: error: RELIABILITY: Syntax error expecting TOTAL or ALL.
- 12 | RELIABILITY VARIABLES=x y/SUMMARY **.
- | ^~"
-
-"reliability.sps:13.35-13.36: error: RELIABILITY: Syntax error expecting INCLUDE or EXCLUDE.
- 13 | RELIABILITY VARIABLES=x y/MISSING=**.
- | ^~"
-
-"reliability.sps:14.27-14.45: warning: RELIABILITY: The STATISTICS subcommand is not yet implemented. No statistics will be produced.
- 14 | RELIABILITY VARIABLES=x y/STATISTICS=x y zz y/ **.
- | ^~~~~~~~~~~~~~~~~~~"
-
-"reliability.sps:14.48-14.49: error: RELIABILITY: Syntax error expecting SCALE, MODEL, SUMMARY, MISSING, or STATISTICS.
- 14 | RELIABILITY VARIABLES=x y/STATISTICS=x y zz y/ **.
- | ^~"
-
-"reliability.sps:15.39: error: RELIABILITY: The split point must be less than the number of variables.
- 15 | RELIABILITY VARIABLES=x y/MODEL SPLIT(5).
- | ^"
-
-"reliability.sps:15.23-15.25: note: RELIABILITY: There are 2 variables.
- 15 | RELIABILITY VARIABLES=x y/MODEL SPLIT(5).
- | ^~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([ROC])
-
-AT_SETUP([ROC free distribution])
-AT_DATA([roc.sps], [dnl
-set format F10.3.
-data list notable list /x * y * w * a *.
-begin data.
-1 1 2 1
-1 2 28 0
-2 3 4 1
-2 4 14 0
-3 5 10 1
-. . 1 0
-3 1 5 0
-4 2 14 1
-4 3 2 0
-5 4 20 1
-5 4 20 .
-5 5 1 0
-end data.
-
-weight by w.
-
-roc x by a (1)
- /plot = none
- /print = se coordinates
- /criteria = testpos(large) distribution(free) ci(99)
- /missing = exclude .
-])
-AT_CHECK([pspp -o pspp.csv roc.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Case Summary
-a,Valid N (listwise),
-,Unweighted,Weighted
-Positive,5,50.000
-Negative,5,50.000
-
-Table: Area Under the Curve
-Variable under test,Area,Std. Error,Asymptotic Sig.,Asymp. 99% Confidence Interval,
-,,,,Lower Bound,Upper Bound
-x,.910,.030,.000,.839,.981
-
-Table: Coordinates of the Curve
-Test variable,Positive if greater than or equal to,Sensitivity,1 - Specificity
-x,.000,1.000,1.000
-,1.500,.960,.440
-,2.500,.880,.160
-,3.500,.680,.060
-,4.500,.400,.020
-,6.000,.000,.000
-])
-AT_CLEANUP
-
-AT_SETUP([ROC negative exponential distribution])
-AT_DATA([roc.sps], [dnl
-set format F10.3.
-data list notable list /x * y * w * a *.
-begin data.
-1 1 2 1
-1 2 28 0
-2 3 4 1
-2 4 14 0
-3 5 10 1
-. . 1 0
-3 1 5 0
-4 2 14 1
-4 3 2 0
-5 4 20 1
-5 4 20 .
-5 5 1 0
-end data.
-
-weight by w.
-
-roc x y by a (1)
- /plot = curve(reference)
- /print = se coordinates
- /criteria = testpos(large) distribution(negexpo) ci(95)
- /missing = exclude .
-])
-AT_CHECK([pspp -o pspp.csv roc.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Case Summary
-a,Valid N (listwise),
-,Unweighted,Weighted
-Positive,5,50.000
-Negative,5,50.000
-
-Table: Area Under the Curve
-Variable under test,Area,Std. Error,Asymptotic Sig.,Asymp. 95% Confidence Interval,
-,,,,Lower Bound,Upper Bound
-x,.910,.030,.000,.860,.960
-y,.697,.052,.001,.611,.783
-
-Table: Coordinates of the Curve
-Test variable,Positive if greater than or equal to,Sensitivity,1 - Specificity
-x,.000,1.000,1.000
-,1.500,.960,.440
-,2.500,.880,.160
-,3.500,.680,.060
-,4.500,.400,.020
-,6.000,.000,.000
-y,.000,1.000,1.000
-,1.500,.960,.900
-,2.500,.680,.340
-,3.000,.600,.340
-,3.500,.600,.300
-,4.500,.200,.020
-,6.000,.000,.000
-])
-AT_CLEANUP
-
-AT_SETUP([ROC with anomaly])
-AT_DATA([roc.sps], [dnl
-set format F10.3.
-data list notable list /x * a * comment (a20).
-begin data.
-0 1 ""
-0 0 ""
-1 1 ""
-1 0 ""
-2 1 ""
-2 0 ""
-5 1 ""
-5 0 ""
-10 1 ""
-10 0 ""
-15 1 ""
-15 0 ""
-20 1 ""
-20 1 ""
-22 0 "here and"
-22 0 "here is the anomaly"
-25 1 ""
-25 0 ""
-30 1 ""
-30 0 ""
-35 1 ""
-35 0 ""
-38 1 ""
-38 0 ""
-39 1 ""
-39 0 ""
-40 1 ""
-40 0 ""
-end data.
-
-roc x by a (1)
- /plot = none
- print = se
- .
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt roc.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Case Summary
-a,Valid N (listwise),
-,Unweighted,Weighted
-Positive,14,14.000
-Negative,14,14.000
-
-Table: Area Under the Curve
-Variable under test,Area,Std. Error,Asymptotic Sig.,Asymp. 95% Confidence Interval,
-,,,,Lower Bound,Upper Bound
-x,.490,.111,.927,.307,.673
-])
-AT_CLEANUP
-
-
-
-
-AT_SETUP([ROC crash on no state variable])
-AT_DATA([roc.sps], [dnl
-data list notable list /x * y * w * a *.
-begin data.
-5 5 1 0
-end data.
-
-
-roc x y By(a (1)
- .
-])
-
-AT_CHECK([pspp -o pspp.csv roc.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([ROC crash on invalid syntax])
-AT_DATA([roc.sps], [dnl
-data list notable list /x * y * a *.
-bggin data.
-1 1 2
-1 2 28
-end data.
-
-
-roc x y by a (1)
- /criteria = ci(y5)
-])
-
-AT_CHECK([pspp -O format=csv roc.sps], [1], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([ROC syntax errors])
-AT_DATA([roc.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-ROC **.
-ROC x **.
-ROC x BY **.
-ROC x BY y **.
-ROC x BY y(**).
-ROC x BY y(5 **).
-ROC x BY y(5)/MISSING=**.
-ROC x BY y(5)/PLOT=CURVE(**).
-ROC x BY y(5)/PLOT=CURVE(REFERENCE **).
-ROC x BY y(5)/PLOT=**.
-ROC x BY y(5)/PRINT=**.
-ROC x BY y(5)/CRITERIA=CUTOFF **.
-ROC x BY y(5)/CRITERIA=CUTOFF(**).
-ROC x BY y(5)/CRITERIA=CUTOFF(INCLUDE **).
-ROC x BY y(5)/CRITERIA=TESTPOS **.
-ROC x BY y(5)/CRITERIA=TESTPOS(**).
-ROC x BY y(5)/CRITERIA=TESTPOS(LARGE **).
-ROC x BY y(5)/CRITERIA=CI **.
-ROC x BY y(5)/CRITERIA=CI(**).
-ROC x BY y(5)/CRITERIA=CI(5 **).
-ROC x BY y(5)/CRITERIA=DISTRIBUTION **.
-ROC x BY y(5)/CRITERIA=DISTRIBUTION(**).
-ROC x BY y(5)/CRITERIA=DISTRIBUTION(FREE **).
-ROC x BY y(5)/CRITERIA=**.
-ROC x BY y(5)/ **.
-])
-AT_CHECK([pspp -O format=csv roc.sps], [1], [dnl
-"roc.sps:2.5-2.6: error: ROC: Syntax error expecting variable name.
- 2 | ROC **.
- | ^~"
-
-"roc.sps:3.7-3.8: error: ROC: Syntax error expecting `BY'.
- 3 | ROC x **.
- | ^~"
-
-"roc.sps:4.10-4.11: error: ROC: Syntax error expecting variable name.
- 4 | ROC x BY **.
- | ^~"
-
-"roc.sps:5.12-5.13: error: ROC: Syntax error expecting `('.
- 5 | ROC x BY y **.
- | ^~"
-
-"roc.sps:6.12-6.13: error: ROC: Syntax error expecting number.
- 6 | ROC x BY y(**).
- | ^~"
-
-"roc.sps:7.14-7.15: error: ROC: Syntax error expecting `)'.
- 7 | ROC x BY y(5 **).
- | ^~"
-
-"roc.sps:8.23-8.24: error: ROC: Syntax error expecting INCLUDE or EXCLUDE.
- 8 | ROC x BY y(5)/MISSING=**.
- | ^~"
-
-"roc.sps:9.26-9.27: error: ROC: Syntax error expecting REFERENCE.
- 9 | ROC x BY y(5)/PLOT=CURVE(**).
- | ^~"
-
-"roc.sps:10.36-10.37: error: ROC: Syntax error expecting `@:}@'.
- 10 | ROC x BY y(5)/PLOT=CURVE(REFERENCE **).
- | ^~"
-
-"roc.sps:11.20-11.21: error: ROC: Syntax error expecting CURVE or NONE.
- 11 | ROC x BY y(5)/PLOT=**.
- | ^~"
-
-"roc.sps:12.21-12.22: error: ROC: Syntax error expecting SE or COORDINATES.
- 12 | ROC x BY y(5)/PRINT=**.
- | ^~"
-
-"roc.sps:13.31-13.32: error: ROC: Syntax error expecting `('.
- 13 | ROC x BY y(5)/CRITERIA=CUTOFF **.
- | ^~"
-
-"roc.sps:14.31-14.32: error: ROC: Syntax error expecting INCLUDE or EXCLUDE.
- 14 | ROC x BY y(5)/CRITERIA=CUTOFF(**).
- | ^~"
-
-"roc.sps:15.39-15.40: error: ROC: Syntax error expecting `)'.
- 15 | ROC x BY y(5)/CRITERIA=CUTOFF(INCLUDE **).
- | ^~"
-
-"roc.sps:16.32-16.33: error: ROC: Syntax error expecting `('.
- 16 | ROC x BY y(5)/CRITERIA=TESTPOS **.
- | ^~"
-
-"roc.sps:17.32-17.33: error: ROC: Syntax error expecting LARGE or SMALL.
- 17 | ROC x BY y(5)/CRITERIA=TESTPOS(**).
- | ^~"
-
-"roc.sps:18.38-18.39: error: ROC: Syntax error expecting `)'.
- 18 | ROC x BY y(5)/CRITERIA=TESTPOS(LARGE **).
- | ^~"
-
-"roc.sps:19.27-19.28: error: ROC: Syntax error expecting `('.
- 19 | ROC x BY y(5)/CRITERIA=CI **.
- | ^~"
-
-"roc.sps:20.27-20.28: error: ROC: Syntax error expecting number.
- 20 | ROC x BY y(5)/CRITERIA=CI(**).
- | ^~"
-
-"roc.sps:21.29-21.30: error: ROC: Syntax error expecting `)'.
- 21 | ROC x BY y(5)/CRITERIA=CI(5 **).
- | ^~"
-
-"roc.sps:22.37-22.38: error: ROC: Syntax error expecting `('.
- 22 | ROC x BY y(5)/CRITERIA=DISTRIBUTION **.
- | ^~"
-
-"roc.sps:23.37-23.38: error: ROC: Syntax error expecting FREE or NEGEXPO.
- 23 | ROC x BY y(5)/CRITERIA=DISTRIBUTION(**).
- | ^~"
-
-"roc.sps:24.42-24.43: error: ROC: Syntax error expecting `)'.
- 24 | ROC x BY y(5)/CRITERIA=DISTRIBUTION(FREE **).
- | ^~"
-
-"roc.sps:25.24-25.25: error: ROC: Syntax error expecting CUTOFF, TESTPOS, CI, or DISTRIBUTION.
- 25 | ROC x BY y(5)/CRITERIA=**.
- | ^~"
-
-"roc.sps:26.16-26.17: error: ROC: Syntax error expecting MISSING, PLOT, PRINT, or CRITERIA.
- 26 | ROC x BY y(5)/ **.
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SORT CASES])
-
-m4_divert_push([PREPARE_TESTS])
-[sort_cases_gen_data () {
- cat > gen-data.py <<'EOF'
-#! /usr/bin/python3
-
-import random
-import sys
-
-data = []
-for i in range(int(sys.argv[1])):
- data += [i] * int(sys.argv[2])
-random.shuffle(data)
-
-data_txt = open('data.txt', 'w')
-for i, item in enumerate(data):
- data_txt.write('%s %s\n' % (item, i))
-data_txt.close()
-
-shuffled = ((item, i) for i, item in enumerate(data))
-expout = open('expout', 'w')
-for item, i in sorted(shuffled):
- expout.write(' %8d %8d \n' % (item, i))
-expout.close()
-EOF
- $PYTHON3 gen-data.py "$@"]
-}
-m4_divert_pop([PREPARE_TESTS])
-
-m4_define([SORT_CASES_TEST],
- [AT_SETUP([sort m4_eval([$1 * $2]) cases[]m4_if([$2], [1], [], [ ($1 unique)])[]m4_if([$3], [], [], [ with $3 buffers])])
- AT_KEYWORDS([SORT CASES $4])
- AT_CHECK([sort_cases_gen_data $1 $2 $3])
- AT_CAPTURE_FILE([data.txt])
- AT_CAPTURE_FILE([output.txt])
- AT_CAPTURE_FILE([sort-cases.sps])
- AT_DATA([sort-cases.sps], [dnl
-DATA LIST LIST NOTABLE FILE='data.txt'/x y (F8).
-SORT CASES BY x[]m4_if([$3], [], [], [/BUFFERS=$3]).
-PRINT OUTFILE='output.txt'/x y.
-EXECUTE.
-])
- AT_CHECK([pspp --testing-mode -o pspp.csv sort-cases.sps])
- AT_CHECK([cat output.txt], [0], [expout])
- AT_CLEANUP])
-
-SORT_CASES_TEST(100, 5, 2)
-SORT_CASES_TEST(100, 5, 3)
-SORT_CASES_TEST(100, 5, 4)
-SORT_CASES_TEST(100, 5, 5)
-SORT_CASES_TEST(100, 5, 10)
-SORT_CASES_TEST(100, 5, 50)
-SORT_CASES_TEST(100, 5, 100)
-SORT_CASES_TEST(100, 5)
-
-SORT_CASES_TEST(100, 10, 2)
-SORT_CASES_TEST(100, 10, 3)
-SORT_CASES_TEST(100, 10, 5)
-SORT_CASES_TEST(100, 10)
-
-SORT_CASES_TEST(1000, 5, 5, slow)
-SORT_CASES_TEST(1000, 5, 50, slow)
-SORT_CASES_TEST(1000, 5, [], slow)
-
-SORT_CASES_TEST(100, 100, 3, slow)
-SORT_CASES_TEST(100, 100, 5, slow)
-SORT_CASES_TEST(100, 100, [], slow)
-
-SORT_CASES_TEST(10000, 5, 500, slow)
-
-SORT_CASES_TEST(50000, 1, [], slow)
-
-dnl Bug #33089 caused SORT CASES to delete filtered cases permanently.
-AT_SETUP([SORT CASES preserves filtered cases])
-AT_DATA([sort-cases.sps], [dnl
-DATA LIST FREE /x.
-BEGIN DATA.
-5 4 3 2 1 0
-END DATA.
-COMPUTE mod2 = MOD(x, 2).
-LIST.
-FILTER BY mod2.
-LIST.
-SORT CASES BY x.
-LIST.
-FILTER OFF.
-LIST.
-])
-AT_CHECK([pspp -O format=csv sort-cases.sps], [0], [dnl
-Table: Data List
-x,mod2
-5.00,1.00
-4.00,.00
-3.00,1.00
-2.00,.00
-1.00,1.00
-.00,.00
-
-Table: Data List
-x,mod2
-5.00,1.00
-3.00,1.00
-1.00,1.00
-
-Table: Data List
-x,mod2
-1.00,1.00
-3.00,1.00
-5.00,1.00
-
-Table: Data List
-x,mod2
-.00,.00
-1.00,1.00
-2.00,.00
-3.00,1.00
-4.00,.00
-5.00,1.00
-])
-AT_CLEANUP
-
-AT_SETUP([SORT CASES syntax errors])
-AT_DATA([sort-cases.sps], [dnl
-DATA LIST LIST NOTABLE/x y z.
-SORT CASES BY **.
-SORT CASES BY x(**).
-SORT CASES BY x(D**).
-SORT CASES BY x(A) x(D) y(**).
-])
-AT_DATA([insert.sps], [dnl
-INSERT FILE='sort-cases.sps' ERROR=IGNORE.
-])
-AT_CHECK([pspp --testing-mode -O format=csv insert.sps], [1], [dnl
-"sort-cases.sps:2.15-2.16: error: SORT CASES: Syntax error expecting variable name.
- 2 | SORT CASES BY **.
- | ^~"
-
-"sort-cases.sps:3.17-3.18: error: SORT CASES: Syntax error expecting A or D.
- 3 | SORT CASES BY x(**).
- | ^~"
-
-"sort-cases.sps:4.18-4.19: error: SORT CASES: Syntax error expecting `)'.
- 4 | SORT CASES BY x(D**).
- | ^~"
-
-"sort-cases.sps:5.15-5.23: warning: SORT CASES: Variable x specified twice in sort criteria.
- 5 | SORT CASES BY x(A) x(D) y(**).
- | ^~~~~~~~~"
-
-"sort-cases.sps:5.27-5.28: error: SORT CASES: Syntax error expecting A or D.
- 5 | SORT CASES BY x(A) x(D) y(**).
- | ^~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([T-TEST])
-
-AT_SETUP([T-TEST /PAIRS])
-AT_DATA([t-test.sps], [dnl
-data list list /ID * A * B *.
-begin data.
-1 2.0 3.0
-2 1.0 2.0
-3 2.0 4.5
-4 2.0 4.5
-5 3.0 6.0
-end data.
-
-t-test /PAIRS a with b (PAIRED).
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-ID,F8.0
-A,F8.0
-B,F8.0
-
-Table: Paired Sample Statistics
-,,N,Mean,Std. Deviation,S.E. Mean
-Pair 1,A,5,2.00,.71,.32
-,B,5,4.00,1.54,.69
-
-Table: Paired Samples Correlations
-,,N,Correlation,Sig.
-Pair 1,A & B,5,.918,.028
-
-Table: Paired Samples Test
-,,Paired Differences,,,,,t,df,Sig. (2-tailed)
-,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
-,,,,,Lower,Upper,,,
-Pair 1,A - B,-2.00,.94,.42,-3.16,-.84,-4.78,4,.009
-])
-AT_CLEANUP
-
-
-AT_SETUP([T-TEST /PAIRS with per-analysis missing values])
-
-AT_DATA([ref.sps], [dnl
-data list list /id * a * b * c * d *.
-begin data.
-1 2.0 3.0 4.0 4.0
-2 1.0 2.0 5.1 3.9
-3 2.0 4.5 5.2 3.8
-4 2.0 4.5 5.3 3.7
-56 3.0 6.0 5.9 3.6
-end data.
-
-t-test /PAIRS a c with b d (PAIRED).
-])
-
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-id,F8.0
-a,F8.0
-b,F8.0
-c,F8.0
-d,F8.0
-
-Table: Paired Sample Statistics
-,,N,Mean,Std. Deviation,S.E. Mean
-Pair 1,a,5,2.00,.71,.32
-,b,5,4.00,1.54,.69
-Pair 2,c,5,5.10,.69,.31
-,d,5,3.80,.16,.07
-
-Table: Paired Samples Correlations
-,,N,Correlation,Sig.
-Pair 1,a & b,5,.918,.028
-Pair 2,c & d,5,-.918,.028
-
-Table: Paired Samples Test
-,,Paired Differences,,,,,t,df,Sig. (2-tailed)
-,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
-,,,,,Lower,Upper,,,
-Pair 1,a - b,-2.00,.94,.42,-3.16,-.84,-4.78,4,.009
-Pair 2,c - d,1.30,.84,.37,.26,2.34,3.47,4,.025
-])
-
-AT_CHECK([pspp -o ref.csv ref.sps])
-AT_CHECK([cat ref.csv], [0], [expout])
-AT_DATA([missing.sps], [dnl
-data list list /id * a * b * c * d *.
-begin data.
-1 2.0 3.0 4.0 4.0
-2 1.0 2.0 5.1 3.9
-3 2.0 4.5 5.2 3.8
-4 2.0 4.5 5.3 3.7
-5 3.0 6.0 5.9 .
-6 3.0 . 5.9 3.6
-end data.
-
-
-t-test /MISSING=analysis /PAIRS a c with b d (PAIRED) /CRITERIA=CI(0.95).
-])
-
-AT_CHECK([pspp -o missing.csv missing.sps])
-AT_CHECK([cat missing.csv], [0], [expout])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /PAIRS with listwise missing values])
-AT_DATA([ref.sps], [dnl
-data list list /id * a * b * c * d *.
-begin data.
-1 2.0 3.0 4.0 4.0
-2 1.0 2.0 5.1 3.9
-3 2.0 4.5 5.2 3.8
-4 2.0 4.5 5.3 3.7
-5 3.0 6.0 5.9 3.6
-end data.
-
-t-test /PAIRS a b with c d (PAIRED).
-])
-
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-id,F8.0
-a,F8.0
-b,F8.0
-c,F8.0
-d,F8.0
-
-Table: Paired Sample Statistics
-,,N,Mean,Std. Deviation,S.E. Mean
-Pair 1,a,5,2.00,.71,.32
-,c,5,5.10,.69,.31
-Pair 2,b,5,4.00,1.54,.69
-,d,5,3.80,.16,.07
-
-Table: Paired Samples Correlations
-,,N,Correlation,Sig.
-Pair 1,a & c,5,.410,.493
-Pair 2,b & d,5,-.872,.054
-
-Table: Paired Samples Test
-,,Paired Differences,,,,,t,df,Sig. (2-tailed)
-,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
-,,,,,Lower,Upper,,,
-Pair 1,a - c,-3.10,.76,.34,-4.04,-2.16,-9.14,4,.001
-Pair 2,b - d,.20,1.68,.75,-1.89,2.29,.27,4,.803
-])
-
-AT_CHECK([pspp -o ref.csv ref.sps])
-
-AT_CHECK([cat ref.csv], [0], [expout])
-
-AT_DATA([missing.sps], [dnl
-data list list /id * a * b * c * d *.
-begin data.
-1 2.0 3.0 4.0 4.0
-2 1.0 2.0 5.1 3.9
-3 2.0 4.5 5.2 3.8
-4 2.0 4.5 5.3 3.7
-5 3.0 6.0 5.9 3.6
-6 3.0 6.0 5.9 .
-end data.
-
-
-t-test /MISSING=listwise /PAIRS a b with c d (PAIRED).
-])
-AT_CHECK([pspp -o missing.csv missing.sps])
-AT_CHECK([cat missing.csv], [0], [expout])
-AT_CLEANUP
-
-
-dnl Tests for a bug in the paired samples T test when weighted
-dnl Thanks to Douglas Bonett for reporting this.
-AT_SETUP([T-TEST weighted paired bug])
-AT_DATA([t-test.sps], [dnl
-DATA LIST notable LIST /x y w *.
-BEGIN DATA.
-1 1 255
-1 2 43
-1 3 216
-2 1 3
-2 2 1
-2 3 12
-END DATA.
-
-WEIGHT BY w.
-
-T-TEST
- PAIRS = y WITH x (PAIRED)
- /MISSING=ANALYSIS
- /CRITERIA=CI(0.95).
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Paired Sample Statistics
-,,N,Mean,Std. Deviation,S.E. Mean
-Pair 1,y,530.00,1.94,.96,.04
-,x,530.00,1.03,.17,.01
-
-Table: Paired Samples Correlations
-,,N,Correlation,Sig.
-Pair 1,y & x,530.00,.114,.008
-
-Table: Paired Samples Test
-,,Paired Differences,,,,,t,df,Sig. (2-tailed)
-,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
-,,,,,Lower,Upper,,,
-Pair 1,y - x,.91,.95,.04,.83,.99,22.07,529.00,.000
-])
-AT_CLEANUP
-
-
-dnl Tests for a bug in the paired samples T test.
-dnl Thanks to Mike Griffiths for reporting this problem.
-AT_SETUP([T-TEST /PAIRS bug])
-AT_DATA([t-test.sps], [dnl
-set format f8.3.
-data list list /A * B *.
-begin data.
-11 2
-1 1
-1 1
-end data.
-
-t-test pairs = a with b (paired).
-])
-AT_CHECK([pspp -o pspp.csv t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-A,F8.0
-B,F8.0
-
-Table: Paired Sample Statistics
-,,N,Mean,Std. Deviation,S.E. Mean
-Pair 1,A,3,4.333,5.774,3.333
-,B,3,1.333,.577,.333
-
-Table: Paired Samples Correlations
-,,N,Correlation,Sig.
-Pair 1,A & B,3,1.000,.000
-
-Table: Paired Samples Test
-,,Paired Differences,,,,,t,df,Sig. (2-tailed)
-,,Mean,Std. Deviation,S.E. Mean,95% Confidence Interval of the Difference,,,,
-,,,,,Lower,Upper,,,
-Pair 1,A - B,3.000,5.196,3.000,-9.908,15.908,1.000,2,.423
-])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /GROUPS])
-AT_DATA([t-test.sps], [dnl
-data list list /ID * INDEP * DEP1 * DEP2 *.
-begin data.
-1 1.1 1 3
-2 1.1 2 4
-3 1.1 2 4
-4 1.1 2 4
-5 1.1 3 5
-6 2.1 3 1
-7 2.1 4 2
-8 2.1 4 2
-9 2.1 4 2
-10 2.1 5 3
-11 3.1 2 2
-end data.
-
-* Note that the last case should be IGNORED since it doesn't have a
- dependent variable of either 1.1 or 2.1.
-
-t-test /GROUPS=indep(1.1,2.1) /var=dep1 dep2.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-ID,F8.0
-INDEP,F8.0
-DEP1,F8.0
-DEP2,F8.0
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-DEP1,1.10,5,2.00,.71,.32
-,2.10,5,4.00,.71,.32
-DEP2,1.10,5,4.00,.71,.32
-,2.10,5,2.00,.71,.32
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-DEP1,Equal variances assumed,.00,1.000,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
-,Equal variances not assumed,,,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
-DEP2,Equal variances assumed,.00,1.000,4.47,8.00,.002,2.00,.45,.97,3.03
-,Equal variances not assumed,,,4.47,8.00,.002,2.00,.45,.97,3.03
-])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /GROUPS with one value for independent variable])
-AT_DATA([t-test.sps], [dnl
-data list list /INDEP * DEP *.
-begin data.
- 1 6
- 1 6
- 1 7
- 1 6
- 1 13
- 1 4
- 1 7
- 1 9
- 1 7
- 1 12
- 1 11
- 2 11
- 2 9
- 2 8
- 2 4
- 2 16
- 2 9
- 2 9
- 2 5
- 2 4
- 2 10
- 2 14
-end data.
-t-test /groups=indep(1.514) /var=dep.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-INDEP,F8.0
-DEP,F8.0
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-DEP,≥ 1.51,11,9.00,3.82,1.15
-,< 1.51,11,8.00,2.86,.86
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-DEP,Equal variances assumed,.17,.683,.69,20.00,.495,1.00,1.44,-2.00,4.00
-,Equal variances not assumed,,,.69,18.54,.496,1.00,1.44,-2.02,4.02
-])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /GROUPS with per-analysis missing values])
-AT_DATA([ref.sps], [dnl
-data list list /id * indep * dep1 * dep2 *.
-begin data.
-1 1.0 3.5 6
-2 1.0 2.0 5
-3 1.0 2.0 4
-4 2.0 3.5 3
-56 2.0 3.0 1
-end data.
-
-t-test /group=indep /var=dep1, dep2.
-])
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-id,F8.0
-indep,F8.0
-dep1,F8.0
-dep2,F8.0
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-dep1,1.00,3,2.50,.87,.50
-,2.00,2,3.25,.35,.25
-dep2,1.00,3,5.00,1.00,.58
-,2.00,2,2.00,1.41,1.00
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-dep1,Equal variances assumed,3.75,.148,-1.12,3.00,.346,-.75,.67,-2.89,1.39
-,Equal variances not assumed,,,-1.34,2.78,.279,-.75,.56,-2.61,1.11
-dep2,Equal variances assumed,.60,.495,2.85,3.00,.065,3.00,1.05,-.35,6.35
-,Equal variances not assumed,,,2.60,1.68,.144,3.00,1.15,-2.98,8.98
-])
-AT_CHECK([pspp -o ref.csv ref.sps])
-AT_CHECK([cat ref.csv], [0], [expout])
-AT_DATA([missing.sps], [dnl
-data list list /id * indep * dep1 * dep2.
-begin data.
-1 1.0 3.5 6
-2 1.0 2.0 5
-3 1.0 2.0 4
-4 2.0 3.5 3
-5 2.0 3.0 .
-6 2.0 . 1
-7 . 3.1 5
-end data.
-
-* Note that if the independent variable is missing, then it's implicitly
-* listwise missing.
-
-t-test /missing=analysis /group=indep /var=dep1 dep2.
-])
-AT_CHECK([pspp -o missing.csv missing.sps])
-AT_CHECK([cat missing.csv], [0], [expout])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /GROUPS with listwise missing values])
-AT_DATA([ref.sps], [dnl
-data list list /id * indep * dep1 * dep2.
-begin data.
-1 1.0 3.5 6
-2 1.0 2.0 5
-3 1.0 2.0 4
-4 2.0 3.5 3
-5 2.0 3.0 2
-6 2.0 4.0 1
-end data.
-
-t-test /group=indep /var=dep1 dep2.
-])
-
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-id,F8.0
-indep,F8.0
-dep1,F8.0
-dep2,F8.0
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-dep1,1.00,3,2.50,.87,.50
-,2.00,3,3.50,.50,.29
-dep2,1.00,3,5.00,1.00,.58
-,2.00,3,2.00,1.00,.58
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-dep1,Equal variances assumed,2.00,.230,-1.73,4.00,.158,-1.00,.58,-2.60,.60
-,Equal variances not assumed,,,-1.73,3.20,.176,-1.00,.58,-2.77,.77
-dep2,Equal variances assumed,.00,1.000,3.67,4.00,.021,3.00,.82,.73,5.27
-,Equal variances not assumed,,,3.67,4.00,.021,3.00,.82,.73,5.27
-])
-
-AT_CHECK([pspp -o ref.csv ref.sps])
-AT_CHECK([cat ref.csv], [0], [expout])
-AT_DATA([missing.sps], [dnl
-data list list /id * indep * dep1 * dep2 *.
-begin data.
-1 1.0 3.5 6
-2 1.0 2.0 5
-3 1.0 2.0 4
-4 2.0 3.5 3
-5 2.0 3.0 2
-6 2.0 4.0 1
-7 2.0 . 0
-end data.
-
-t-test /missing=listwise,exclude /group=indep /var=dep1, dep2.
-])
-AT_CHECK([pspp -o missing.csv missing.sps])
-AT_CHECK([cat missing.csv], [0], [expout])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /TESTVAL])
-AT_DATA([t-test.sps], [dnl
-data list list /ID * ABC *.
-begin data.
-1 3.5
-2 2.0
-3 2.0
-4 3.5
-5 3.0
-6 4.0
-end data.
-
-t-test /testval=2.0 /var=abc.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-ID,F8.0
-ABC,F8.0
-
-Table: One-Sample Statistics
-,N,Mean,Std. Deviation,S.E. Mean
-ABC,6,3.00,.84,.34
-
-Table: One-Sample Test
-,Test Value = 2,,,,,
-,t,df,Sig. (2-tailed),Mean Difference,95% Confidence Interval of the Difference,
-,,,,,Lower,Upper
-ABC,2.93,5,.033,1.00,.12,1.88
-])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /TESTVAL with per-analysis missing values])
-AT_DATA([ref.sps], [dnl
-data list list /id * x1 * x2.
-begin data.
-1 3.5 34
-2 2.0 10
-3 2.0 23
-4 3.5 98
-5 3.0 23
-67 4.0 8
-end data.
-
-t-test /testval=3.0 /var=x1 x2.
-])
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-id,F8.0
-x1,F8.0
-x2,F8.0
-
-Table: One-Sample Statistics
-,N,Mean,Std. Deviation,S.E. Mean
-x1,6,3.00,.84,.34
-x2,6,32.67,33.40,13.64
-
-Table: One-Sample Test
-,Test Value = 3,,,,,
-,t,df,Sig. (2-tailed),Mean Difference,95% Confidence Interval of the Difference,
-,,,,,Lower,Upper
-x1,.00,5,1.000,.00,-.88,.88
-x2,2.18,5,.082,29.67,-5.39,64.72
-])
-AT_CHECK([pspp -o ref.csv ref.sps])
-AT_CHECK([cat ref.csv], [0], [expout])
-AT_DATA([missing.sps], [dnl
-data list list /id * x1 * x2.
-begin data.
-1 3.5 34
-2 2.0 10
-3 2.0 23
-4 3.5 98
-5 3.0 23
-6 4.0 .
-7 . 8
-end data.
-
-t-test /missing=analysis /testval=3.0 /var=x1 x2.
-])
-AT_CHECK([pspp -o missing.csv missing.sps])
-AT_CHECK([cat missing.csv], [0], [expout])
-AT_CLEANUP
-
-AT_SETUP([T-TEST /TESTVAL with listwise missing values])
-AT_DATA([ref.sps], [dnl
-data list list /id * x1 * x2.
-begin data.
-1 3.5 34
-2 2.0 10
-3 2.0 23
-4 3.5 98
-5 3.0 23
-end data.
-
-t-test /testval=3.0 /var=x1 x2.
-])
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-id,F8.0
-x1,F8.0
-x2,F8.0
-
-Table: One-Sample Statistics
-,N,Mean,Std. Deviation,S.E. Mean
-x1,5,2.80,.76,.34
-x2,5,37.60,34.82,15.57
-
-Table: One-Sample Test
-,Test Value = 3,,,,,
-,t,df,Sig. (2-tailed),Mean Difference,95% Confidence Interval of the Difference,
-,,,,,Lower,Upper
-x1,-.59,4,.587,-.20,-1.14,.74
-x2,2.22,4,.090,34.60,-8.63,77.83
-])
-AT_CHECK([pspp -o ref.csv ref.sps])
-AT_CHECK([cat ref.csv], [0], [expout])
-AT_DATA([missing.sps], [dnl
-data list list /id * x1 * x2.
-begin data.
-1 3.5 34
-2 2.0 10
-3 2.0 23
-4 3.5 98
-5 3.0 23
-6 4.0 99
-end data.
-
-MISSING VALUES x2(99).
-
-t-test /missing=listwise /testval=3.0 /var=x1 x2.
-])
-AT_CHECK([pspp -o missing.csv missing.sps])
-AT_CHECK([cat missing.csv], [0], [expout])
-AT_CLEANUP
-
-AT_SETUP([T-TEST wih TEMPORARY transformation])
-AT_DATA([ref.sps], [dnl
-data list list /ind * x * .
-begin data.
-1 3.5
-1 2.0
-1 2.0
-2 3.5
-2 3.0
-2 4.0
-end data.
-
-t-test /groups=ind(1,2) /var x.
-])
-AT_DATA([expout], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-ind,F8.0
-x,F8.0
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-x,1.00,3,2.50,.87,.50
-,2.00,3,3.50,.50,.29
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-x,Equal variances assumed,2.00,.230,-1.73,4.00,.158,-1.00,.58,-2.60,.60
-,Equal variances not assumed,,,-1.73,3.20,.176,-1.00,.58,-2.77,.77
-])
-AT_CHECK([pspp -o ref.csv ref.sps])
-AT_CHECK([cat ref.csv], [0], [expout])
-AT_DATA([temporary.sps], [dnl
-data list list /ind * x * .
-begin data.
-1 3.5
-1 2.0
-1 2.0
-2 3.5
-2 3.0
-2 4.0
-2 9.0
-end data.
-
-TEMPORARY.
-SELECT IF x < 7.
-
-t-test /groups=ind(1 2) /var x.
-])
-AT_CHECK([pspp -o temporary.csv temporary.sps])
-AT_CHECK([cat temporary.csv], [0], [expout])
-AT_CLEANUP
-
-dnl This is an example from doc/tutorial.texi
-dnl So if the results of this have to be changed in any way,
-dnl make sure to update that file.
-AT_SETUP([T-TEST tutorial example])
-cp $top_srcdir/examples/physiology.sav .
-AT_DATA([t-test.sps], [dnl
-GET FILE='physiology.sav'.
-RECODE height (179 = SYSMIS).
-T-TEST GROUP=sex(0,1) /VARIABLES=height temperature.
-])
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-Height in millimeters ,Male,22,1796.49,49.71,10.60
-,Female,17,1610.77,25.43,6.17
-Internal body temperature in degrees Celcius,Male,22,36.68,1.95,.42
-,Female,18,37.43,1.61,.38
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-Height in millimeters ,Equal variances assumed,.97,.331,14.02,37.00,.000,185.72,13.24,158.88,212.55
-,Equal variances not assumed,,,15.15,32.71,.000,185.72,12.26,160.76,210.67
-Internal body temperature in degrees Celcius,Equal variances assumed,.31,.581,-1.31,38.00,.198,-.75,.57,-1.91,.41
-,Equal variances not assumed,,,-1.33,37.99,.190,-.75,.56,-1.89,.39
-])
-AT_CLEANUP
-
-dnl Tests for a bug which caused T-TEST to crash when given invalid syntax.
-AT_SETUP([T-TEST invalid syntax])
-AT_DATA([t-test.sps], [dnl
-DATA LIST LIST NOTABLE /id * a * .
-BEGIN DATA.
-1 3.5
-2 2.0
-3 2.0
-4 3.5
-5 3.0
-6 4.0
-END DATA.
-
-T-TEST /testval=2.0 .
-T-TEST /groups=id(3) .
-])
-AT_CHECK([pspp -O format=csv t-test.sps], [1], [dnl
-"t-test.sps:11.1-11.21: error: T-TEST: Required subcommand VARIABLES was not specified.
- 11 | T-TEST /testval=2.0 .
- | ^~~~~~~~~~~~~~~~~~~~~"
-
-"t-test.sps:12.1-12.22: error: T-TEST: Required subcommand VARIABLES was not specified.
- 12 | T-TEST /groups=id(3) .
- | ^~~~~~~~~~~~~~~~~~~~~~"
-])
-AT_CLEANUP
-
-dnl Tests for bug #11227, exhibited when the independent variable is a string.
-AT_SETUP([T-TEST string variable])
-AT_DATA([t-test.sps], [dnl
-data list list /ID * INDEP (a1) DEP1 * DEP2 *.
-begin data.
-1 'a' 1 3
-2 'a' 2 4
-3 'a' 2 4
-4 'a' 2 4
-5 'a' 3 5
-6 'b' 3 1
-7 'b' 4 2
-8 'b' 4 2
-9 'b' 4 2
-10 'b' 5 3
-11 'c' 2 2
-end data.
-
-
-t-test /GROUPS=indep('a','b') /var=dep1 dep2.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-ID,F8.0
-INDEP,A1
-DEP1,F8.0
-DEP2,F8.0
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-DEP1,a,5,2.00,.71,.32
-,b,5,4.00,.71,.32
-DEP2,a,5,4.00,.71,.32
-,b,5,2.00,.71,.32
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-DEP1,Equal variances assumed,.00,1.000,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
-,Equal variances not assumed,,,-4.47,8.00,.002,-2.00,.45,-3.03,-.97
-DEP2,Equal variances assumed,.00,1.000,4.47,8.00,.002,2.00,.45,.97,3.03
-,Equal variances not assumed,,,4.47,8.00,.002,2.00,.45,.97,3.03
-])
-AT_CLEANUP
-
-AT_SETUP([T-TEST string variable, only one value])
-AT_DATA([t-test.sps], [dnl
-data list list notable /id * indep (a1) dep1 * dep2 *.
-begin data.
-1 'a' 1 3
-2 'a' 2 4
-3 'a' 2 4
-4 'a' 2 4
-5 'a' 3 5
-6 'b' 3 1
-7 'b' 4 2
-8 'b' 4 2
-9 'b' 4 2
-10 'b' 5 3
-11 'c' 2 2
-end data.
-
-
-t-test /GROUPS=indep('a') /var=dep1 dep2.
-])
-AT_CHECK([pspp -O format=csv t-test.sps], [1], [dnl
-"t-test.sps:17.16-17.25: error: T-TEST: When applying GROUPS to a string variable, two values must be specified.
- 17 | t-test /GROUPS=indep('a') /var=dep1 dep2.
- | ^~~~~~~~~~"
-])
-AT_CLEANUP
-
-dnl Tests for a bug which didn't properly compare string values.
-AT_SETUP([T-TEST string variable comparison bug])
-AT_DATA([t-test.sps], [dnl
-data list list /x * gv (a8).
-begin data.
-3 One
-2 One
-3 One
-2 One
-3 One
-4 Two
-3.5 Two
-3.0 Two
-end data.
-
-t-test group=gv('One', 'Two')
- /variables = x.
-])
-
-AT_CHECK([pspp -o pspp.csv -o pspp.txt t-test.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-gv,A8
-
-Table: Group Statistics
-,Group,N,Mean,Std. Deviation,S.E. Mean
-x,One,5,2.60,.55,.24
-,Two,3,3.50,.50,.29
-
-Table: Independent Samples Test
-,,Levene's Test for Equality of Variances,,T-Test for Equality of Means,,,,,,
-,,F,Sig.,t,df,Sig. (2-tailed),Mean Difference,Std. Error Difference,95% Confidence Interval of the Difference,
-,,,,,,,,,Lower,Upper
-x,Equal variances assumed,1.13,.329,-2.32,6.00,.060,-.90,.39,-1.85,.05
-,Equal variances not assumed,,,-2.38,4.70,.067,-.90,.38,-1.89,.09
-])
-AT_CLEANUP
-
-
-
-dnl Tests for a bug assert failed when the group variables were not of either class
-AT_SETUP([T-TEST wrong group])
-AT_DATA([t-test-crs.sps], [dnl
-data list list /x * g *.
-begin data.
-1 2
-2 2
-3 2
-4 2
-5 2
-end data.
-
-t-test /variables = x group=g(1,3).
-])
-
-AT_CHECK([pspp t-test-crs.sps], [0],[ignore], [ignore])
-
-AT_CLEANUP
-
-
-
-dnl Tests for a bug assert failed when a non-number was passes as the p value
-AT_SETUP([T-TEST non number p value])
-AT_DATA([t.sps], [dnl
-data list list /age d_frage_1 weight height *.
-begin data.
-1 2 3 1
-4 5 6 2
-end data.
-
-T-TEST /VARIABLES=age weight height
- /GROUPS=d_frage_1(1,0) /MISSING=ANALYSIS /CRITERIA=CIN(p.95).
-])
-
-AT_CHECK([pspp t.sps], [1],[ignore], [ignore])
-
-AT_CLEANUP
-
-
-
-dnl Another crash on invalid input
-AT_SETUP([T-TEST unterminated string - paired])
-AT_DATA([t.sps], [dnl
-data list list /id * a * b * c * d *.
-begin data.
-5 2.0 3.0 4.0 4.0
-3 1.0 2.0 5.1 3.9
-3 2.0 4.5 5.2(3.8
-4 2.0 4.5 5n3 3.7
-5 3.0 6.0 5.9 3.6
-6 3.4 6.0 5.9 .
-end data.
-
-
-t-test /MISSING=listwise /PAIRS a"b with c d (PA RED).
-]) dnl "
-
-AT_CHECK([pspp t.sps],[1],[ignore],[ignore])
-
-AT_CLEANUP
-
-AT_SETUP([T-TEST syntax errors])
-AT_DATA([t-test.sps], [dnl
-DATA LIST LIST NOTABLE/x y z * s(a10).
-T-TEST TESTVAL=**.
-T-TEST GROUPS=**.
-T-TEST GROUPS=x(**).
-T-TEST GROUPS=x(1,**).
-T-TEST GROUPS=x(1,2 **).
-T-TEST GROUPS=s('a').
-T-TEST VARIABLES=x y PAIRS.
-T-TEST PAIRS=**.
-T-TEST PAIRS=x WITH **.
-T-TEST PAIRS=x WITH y z (PAIRED).
-T-TEST PAIRS=x WITH y/VARIABLES.
-T-TEST VARIABLES=**.
-T-TEST MISSING=**.
-T-TEST CRITERIA=**.
-T-TEST CRITERIA=CIN**.
-T-TEST CRITERIA=CIN(**).
-T-TEST CRITERIA=CIN(1 **).
-T-TEST **.
-T-TEST MISSING=INCLUDE.
-T-TEST MISSING=INCLUDE/TESTVAL=1.
-])
-AT_CHECK([pspp -O format=csv t-test.sps], [1], [dnl
-"t-test.sps:2.16-2.17: error: T-TEST: Syntax error expecting number.
- 2 | T-TEST TESTVAL=**.
- | ^~"
-
-"t-test.sps:3.15-3.16: error: T-TEST: Syntax error expecting variable name.
- 3 | T-TEST GROUPS=**.
- | ^~"
-
-"t-test.sps:4.17-4.18: error: T-TEST: Syntax error expecting number.
- 4 | T-TEST GROUPS=x(**).
- | ^~"
-
-"t-test.sps:5.19-5.20: error: T-TEST: Syntax error expecting number.
- 5 | T-TEST GROUPS=x(1,**).
- | ^~"
-
-"t-test.sps:6.21-6.22: error: T-TEST: Syntax error expecting `)'.
- 6 | T-TEST GROUPS=x(1,2 **).
- | ^~"
-
-"t-test.sps:7.15-7.20: error: T-TEST: When applying GROUPS to a string variable, two values must be specified.
- 7 | T-TEST GROUPS=s('a').
- | ^~~~~~"
-
-"t-test.sps:8.22-8.26: error: T-TEST: VARIABLES subcommand may not be used with PAIRS.
- 8 | T-TEST VARIABLES=x y PAIRS.
- | ^~~~~"
-
-"t-test.sps:9.14-9.15: error: T-TEST: Syntax error expecting variable name.
- 9 | T-TEST PAIRS=**.
- | ^~"
-
-"t-test.sps:10.21-10.22: error: T-TEST: Syntax error expecting variable name.
- 10 | T-TEST PAIRS=x WITH **.
- | ^~"
-
-"t-test.sps:11.14-11.23: error: T-TEST: PAIRED was specified, but the number of variables preceding WITH (1) does not match the number following (2).
- 11 | T-TEST PAIRS=x WITH y z (PAIRED).
- | ^~~~~~~~~~"
-
-"t-test.sps:12.23-12.31: error: T-TEST: VARIABLES subcommand may not be used with PAIRS.
- 12 | T-TEST PAIRS=x WITH y/VARIABLES.
- | ^~~~~~~~~"
-
-"t-test.sps:13.18-13.19: error: T-TEST: Syntax error expecting variable name.
- 13 | T-TEST VARIABLES=**.
- | ^~"
-
-"t-test.sps:14.16-14.17: error: T-TEST: Syntax error expecting INCLUDE, EXCLUDE, LISTWISE, or ANALYSIS.
- 14 | T-TEST MISSING=**.
- | ^~"
-
-"t-test.sps:15.17-15.18: error: T-TEST: Syntax error expecting CIN or CI.
- 15 | T-TEST CRITERIA=**.
- | ^~"
-
-"t-test.sps:16.20-16.21: error: T-TEST: Syntax error expecting `('.
- 16 | T-TEST CRITERIA=CIN**.
- | ^~"
-
-"t-test.sps:17.21-17.22: error: T-TEST: Syntax error expecting number.
- 17 | T-TEST CRITERIA=CIN(**).
- | ^~"
-
-"t-test.sps:18.23-18.24: error: T-TEST: Syntax error expecting `)'.
- 18 | T-TEST CRITERIA=CIN(1 **).
- | ^~"
-
-"t-test.sps:19.8-19.9: error: T-TEST: Syntax error expecting TESTVAL, GROUPS, PAIRS, VARIABLES, MISSING, or CRITERIA.
- 19 | T-TEST **.
- | ^~"
-
-"t-test.sps:20: error: T-TEST: Exactly one of TESTVAL, GROUPS and PAIRS subcommands must be specified."
-
-"t-test.sps:21.1-21.33: error: T-TEST: Required subcommand VARIABLES was not specified.
- 21 | T-TEST MISSING=INCLUDE/TESTVAL=1.
- | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([CACHE])
-
-AT_SETUP([CACHE])
-AT_DATA([cache.sps], [dnl
-CACHE.
-])
-AT_CHECK([pspp -O format=csv cache.sps])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([CD])
-
-AT_SETUP([CD])
-mkdir subdir
-AT_DATA([cd.sps], [dnl
-cd 'subdir'.
-host command=[['pwd > mydir']].
-])
-AT_CHECK([pspp -O format=csv cd.sps])
-AT_CAPTURE_FILE([subdir/mydir])
-AT_CHECK([sed 's,.*/,,' subdir/mydir], [0], [subdir
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([USE])
-
-AT_SETUP([USE ALL])
-AT_DATA([use.sps], [dnl
-data list notable /X 1-2.
-begin data.
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-end data.
-use all.
-list.
-])
-AT_CHECK([pspp -o pspp.csv use.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-X
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([HOST - portable tests])
-
-AT_SETUP([HOST - one command])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['echo "hi there" > file']].
-])
-AT_CHECK([pspp -O format=csv host.sps])
-AT_CHECK([cat file], [0], [hi there
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - two commands])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['echo a > a' 'echo b > b']].
-])
-AT_CHECK([pspp -O format=csv host.sps])
-AT_CHECK([cat a], [0], [a
-])
-AT_CHECK([cat b], [0], [b
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - time limit])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['sleep 10']] TIMELIMIT=0.1.
-])
-if $MINGW; then
- AT_CHECK([pspp -O format=csv host.sps], [1], [dnl
-host.sps:1: error: HOST: Time limit not supported on this platform.
-])
-else
- AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
-"host.sps:1: warning: HOST: Command ""sleep 10"" timed out."
-])
-fi
-AT_CLEANUP
-
-AT_BANNER([HOST - Unix-like OSes only])
-
-AT_SETUP([HOST - command failure])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['exit 1' 'echo "Not reached"']].
-])
-AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
-"host.sps:1: warning: HOST: Command ""exit 1"" exited with status 1."
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - nonexistent shell])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['echo hi']].
-])
-AT_CHECK([SHELL=/nonexistent pspp -O format=csv host.sps], [0], [dnl
-"host.sps:1: warning: HOST: Command ""echo hi"" exited with status 127 (Command or shell not found)."
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - nonexistent command])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['/nonexistent']].
-])
-AT_CHECK([pspp -O format=csv host.sps | head -1], [0], [dnl
-"host.sps:1: warning: HOST: Command ""/nonexistent"" exited with status 127 (Command or shell not found)."
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - output to stdout])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['echo hi']].
-])
-AT_CHECK([pspp -O format=csv host.sps], [0], [hi
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - output to stderr])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['echo hi 2>&1']].
-])
-AT_CHECK([pspp -O format=csv host.sps], [0], [hi
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - input from stdin])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['cat && echo ok || echo fail']] TIMELIMIT=5.
-])
-AT_CHECK([pspp -O format=csv host.sps], [0], [ok
-])
-AT_CLEANUP
-
-dnl This is a special case inside run_command().
-AT_SETUP([HOST - zero time limit])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['sleep 10']] TIMELIMIT=0.
-])
-AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
-"host.sps:1: warning: HOST: Command ""sleep 10"" timed out."
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - signal termination])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-HOST COMMAND=[['kill -ABRT $$' 'echo "Not reached"']].
-])
-AT_CHECK([pspp -O format=csv host.sps], [0], [dnl
-"host.sps:1: warning: HOST: Command ""kill -ABRT $$"" terminated by signal 6."
-])
-AT_CLEANUP
-
-AT_SETUP([HOST - SAFER])
-AT_SKIP_IF([$MINGW])
-AT_DATA([host.sps], [dnl
-SET SAFER=ON.
-HOST COMMAND=[['sleep 10']] TIMELIMIT=0.1.
-])
-AT_CHECK([pspp -O format=csv host.sps], [1], [dnl
-"host.sps:2.1-2.4: error: HOST: This command not allowed when the SAFER option is set.
- 2 | HOST COMMAND=[['sleep 10']] TIMELIMIT=0.1.
- | ^~~~"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([INSERT])
-
-dnl Create a file "batch.sps" that is valid syntax only in batch mode.
-m4_define([CREATE_BATCH_SPS],
- [AT_DATA([batch.sps], [dnl
-input program
-loop #i = 1 to 5
-+ compute z = #i
-+ end case
-end loop
-end file
-end input program
-])])
-
-AT_SETUP([INSERT SYNTAX=INTERACTIVE])
-CREATE_BATCH_SPS
-AT_DATA([insert.sps], [dnl
-INSERT
- FILE='batch.sps'
- SYNTAX=interactive.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
-batch.sps:2.1-2.4: error: INPUT PROGRAM: Syntax error expecting end of command.
- 2 | loop #i = 1 to 5
- | ^~~~
-batch.sps:3.4-3.10: error: COMPUTE: COMPUTE is allowed only after the active dataset has been defined or inside INPUT PROGRAM.
- 3 | + compute z = #i
- | ^~~~~~~
-batch.sps:4.4-4.11: error: END CASE: END CASE is allowed only inside INPUT PROGRAM.
- 4 | + end case
- | ^~~~~~~~
-insert.sps:4.1-4.4: error: LIST: LIST is allowed only after the active dataset has been defined.
- 4 | LIST.
- | ^~~~
-])
-AT_CLEANUP
-
-AT_SETUP([INSERT SYNTAX=BATCH])
-CREATE_BATCH_SPS
-AT_DATA([insert.sps], [dnl
-INSERT
- FILE='batch.sps'
- SYNTAX=BATCH.
-LIST.
-])
-AT_CHECK([pspp -o pspp.csv insert.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-z
-1.00
-2.00
-3.00
-4.00
-5.00
-])
-AT_CLEANUP
-
-AT_SETUP([INSERT CD=NO])
-AT_DATA([insert.sps], [INSERT FILE='Dir1/foo.sps'.
-LIST.
-])
-mkdir Dir1
-AT_DATA([Dir1/foo.sps], [INSERT FILE='bar.sps' CD=NO.
-])
-AT_DATA([Dir1/bar.sps],
- [DATA LIST LIST /x *.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-])
-AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
-Dir1/foo.sps:1: error: INSERT: Can't find `bar.sps' in include file search path.
-insert.sps:2.1-2.4: error: LIST: LIST is allowed only after the active dataset has been defined.
- 2 | LIST.
- | ^~~~
-])
-AT_CLEANUP
-
-AT_SETUP([INSERT CD=YES])
-AT_DATA([insert.sps], [INSERT FILE='Dir1/foo.sps' CD=YES.
-LIST.
-])
-mkdir Dir1
-AT_DATA([Dir1/foo.sps], [INSERT FILE='bar.sps'.
-])
-AT_DATA([Dir1/bar.sps],
- [DATA LIST LIST /x *.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-])
-AT_CHECK([pspp -o pspp.csv insert.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-
-Table: Data List
-x
-1.00
-2.00
-3.00
-])
-AT_CLEANUP
-
-m4_define([CREATE_ERROR_SPS],
- [AT_DATA([error.sps], [dnl
-DATA LIST NOTABLE LIST /x *.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-
-* The following line is erroneous
-
-DISPLAY AKSDJ.
-LIST.
-])])
-
-AT_SETUP([INSERT ERROR=STOP])
-CREATE_ERROR_SPS
-AT_DATA([insert.sps], [INSERT FILE='error.sps' ERROR=STOP.
-])
-AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
-error.sps:10.9-10.13: error: DISPLAY: AKSDJ is not a variable name.
- 10 | DISPLAY AKSDJ.
- | ^~~~~
-warning: Error encountered while ERROR=STOP is effective.
-])
-AT_CLEANUP
-
-AT_SETUP([INSERT ERROR=CONTINUE])
-CREATE_ERROR_SPS
-AT_DATA([insert.sps], [INSERT FILE='error.sps' ERROR=CONTINUE.
-])
-AT_CHECK([pspp -o pspp.csv insert.sps], [1], [dnl
-error.sps:10.9-10.13: error: DISPLAY: AKSDJ is not a variable name.
- 10 | DISPLAY AKSDJ.
- | ^~~~~
-])
-AT_CHECK([cat pspp.csv], [0], [dnl
-"error.sps:10.9-10.13: error: DISPLAY: AKSDJ is not a variable name.
- 10 | DISPLAY AKSDJ.
- | ^~~~~"
-
-Table: Data List
-x
-1.00
-2.00
-3.00
-])
-AT_CLEANUP
-
-dnl Test for regression against bug #24569 in which PSPP crashed
-dnl upon attempt to insert a nonexistent file.
-AT_SETUP([INSERT nonexistent file])
-AT_DATA([insert.sps], [dnl
-INSERT
- FILE='nonexistent'
- ERROR=CONTINUE.
- .
-
-LIST.
-])
-AT_CHECK([pspp -O format=csv insert.sps], [1], [dnl
-insert.sps:2: error: INSERT: Can't find `nonexistent' in include file search path.
-
-"insert.sps:6.1-6.4: error: LIST: LIST is allowed only after the active dataset has been defined.
- 6 | LIST.
- | ^~~~"
-])
-AT_CLEANUP
-
-
-dnl A test to check the INCLUDE command complete with the
-dnl syntax and function of the ENCODING subcommand.
-AT_SETUP([INCLUDE full check])
-AT_DATA([two-utf8.sps], [dnl
-echo 'Äpfelfölfaß'.
-])
-
-AT_DATA([include.sps], [dnl
-echo 'ONE'.
-
-include FILE='two-latin1.sps' ENCODING='ISO_8859-1'.
-])
-
-AT_CHECK([iconv -f UTF-8 -t iso-8859-1 two-utf8.sps > two-latin1.sps], [0], [])
-
-AT_CHECK([pspp -O format=csv include.sps], [0], [dnl
-ONE
-
-Äpfelfölfaß
-])
-AT_CLEANUP
-
-
-
-
-dnl Test for a bug where insert crashed on an unterminated string input
-AT_SETUP([INSERT unterminated string])
-
-AT_DATA([insert.sps], [INSERT FILE=7bar.sps' CD=NO.
-])
-
-AT_CHECK([pspp -O format=csv insert.sps], [1], [ignore])
-
-AT_CLEANUP
-
-AT_SETUP([INSERT syntax errors])
-AT_KEYWORDS([INCLUDE])
-: >inner.sps
-AT_DATA([insert.sps], [dnl
-INSERT **.
-INSERT 'nonexistent.sps'.
-INSERT 'inner.sps' ENCODING=**.
-INSERT 'inner.sps' SYNTAX=**.
-INSERT 'inner.sps' CD=**.
-INSERT 'inner.sps' ERROR=**.
-INSERT 'inner.sps' **.
-INCLUDE 'inner.sps' **.
-])
-AT_CHECK([pspp -O format=csv insert.sps], [1], [dnl
-"insert.sps:1.8-1.9: error: INSERT: Syntax error expecting string.
- 1 | INSERT **.
- | ^~"
-
-insert.sps:2: error: INSERT: Can't find `nonexistent.sps' in include file search path.
-
-"insert.sps:3.29-3.30: error: INSERT: Syntax error expecting string.
- 3 | INSERT 'inner.sps' ENCODING=**.
- | ^~"
-
-"insert.sps:4.27-4.28: error: INSERT: Syntax error expecting BATCH, INTERACTIVE, or AUTO.
- 4 | INSERT 'inner.sps' SYNTAX=**.
- | ^~"
-
-"insert.sps:5.23-5.24: error: INSERT: Syntax error expecting YES or NO.
- 5 | INSERT 'inner.sps' CD=**.
- | ^~"
-
-"insert.sps:6.26-6.27: error: INSERT: Syntax error expecting CONTINUE or STOP.
- 6 | INSERT 'inner.sps' ERROR=**.
- | ^~"
-
-"insert.sps:7.20-7.21: error: INSERT: Syntax error expecting ENCODING, SYNTAX, CD, or ERROR.
- 7 | INSERT 'inner.sps' **.
- | ^~"
-
-"insert.sps:8.21-8.22: error: INCLUDE: Syntax error expecting ENCODING.
- 8 | INCLUDE 'inner.sps' **.
- | ^~"
-])
-AT_CLEANUP
+++ /dev/null
-AT_BANNER([OUTPUT MODIFY])
-
-AT_SETUP([OUTPUT MODIFY syntax errors])
-AT_DATA([output.sps], [dnl
-OUTPUT MODIFY/SELECT **.
-OUTPUT MODIFY/TABLECELLS SELECT **.
-OUTPUT MODIFY/TABLECELLS SELECT=**.
-OUTPUT MODIFY/TABLECELLS SELECT=[[**]].
-OUTPUT MODIFY/TABLECELLS FORMAT **.
-OUTPUT MODIFY/TABLECELLS FORMAT=ASDF5.
-OUTPUT MODIFY/TABLECELLS **.
-OUTPUT MODIFY/TABLECELLS SELECT="xyzzy" FORMAT=F8.2.
-])
-AT_CHECK([pspp -O format=csv output.sps], [1], [dnl
-"output.sps:1.22-1.23: error: OUTPUT MODIFY: Syntax error expecting TABLES.
- 1 | OUTPUT MODIFY/SELECT **.
- | ^~"
-
-"output.sps:2.33-2.34: error: OUTPUT MODIFY: Syntax error expecting `='.
- 2 | OUTPUT MODIFY/TABLECELLS SELECT **.
- | ^~"
-
-"output.sps:3.33-3.34: error: OUTPUT MODIFY: Syntax error expecting `[['.
- 3 | OUTPUT MODIFY/TABLECELLS SELECT=**.
- | ^~"
-
-"output.sps:4.34-4.35: error: OUTPUT MODIFY: Syntax error expecting `]]'.
- 4 | OUTPUT MODIFY/TABLECELLS SELECT=[[**]].
- | ^~"
-
-"output.sps:5.33-5.34: error: OUTPUT MODIFY: Syntax error expecting `='.
- 5 | OUTPUT MODIFY/TABLECELLS FORMAT **.
- | ^~"
-
-"output.sps:6.38: error: OUTPUT MODIFY: Unknown format type `ASDF'.
- 6 | OUTPUT MODIFY/TABLECELLS FORMAT=ASDF5.
- | ^"
-
-"output.sps:7.26-7.27: error: OUTPUT MODIFY: Syntax error expecting SELECT or FORMAT.
- 7 | OUTPUT MODIFY/TABLECELLS **.
- | ^~"
-
-"output.sps:8.33-8.39: error: OUTPUT MODIFY: Syntax error expecting `@<:@'.
- 8 | OUTPUT MODIFY/TABLECELLS SELECT=""xyzzy"" FORMAT=F8.2.
- | ^~~~~~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([PERMISSIONS])
-
-AT_SETUP([PERMISSIONS])
-AT_DATA([foobar], [Hello
-])
-
-chmod 666 foobar
-AT_CHECK([ls -l foobar], [0], [stdout])
-AT_CHECK([sed 's/^\(..........\).*/\1/' stdout], [0], [-rw-rw-rw-
-])
-
-AT_DATA([permissions.sps], [PERMISSIONS /FILE='foobar' PERMISSIONS=READONLY.
-])
-AT_CHECK([pspp -O format=csv permissions.sps])
-AT_CHECK([ls -l foobar], [0], [stdout])
-AT_CHECK([sed 's/^\(..........\).*/\1/' stdout], [0], [-r--r--r--
-])
-
-AT_DATA([permissions.sps], [PERMISSIONS /FILE='foobar' PERMISSIONS=WRITEABLE.
-])
-AT_CHECK([pspp -O format=csv permissions.sps])
-AT_CHECK([ls -l foobar], [0], [stdout])
-AT_CHECK([sed 's/^\(..........\).*/\1/' stdout], [0], [-rw-r--r--
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([PERMISSIONS - bad syntax])
-AT_DATA([pe.sps], [[PERMI|SIONS /FILE='foobar' PERMISSIONS=WRITEABLE.
-]])
-
-AT_CHECK([pspp -O format=csv pe.sps], [1], [dnl
-"pe.sps:1.6: error: PERMISSIONS: Syntax error expecting STRING.
- 1 | PERMI|SIONS /FILE='foobar' PERMISSIONS=WRITEABLE.
- | ^"
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SET])
-
-# This crashed older versions of PSPP (bug #30682).
-AT_SETUP([SET FORMAT to invalid output format])
-AT_DATA([set.pspp], [dnl
-DATA LIST LIST NOTABLE /x.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-SET FORMAT F41.
-DESCRIPTIVES /x.
-])
-AT_CHECK([pspp -O format=csv set.pspp], [1], [dnl
-"set.pspp:7.12-7.14: error: SET: Output format F41.0 specifies width 41, but F requires a width between 1 and 40.
- 7 | SET FORMAT F41.
- | ^~~"
-
-Table: Descriptive Statistics
-,N,Mean,Std Dev,Minimum,Maximum
-x,3,2.00,1.00,1.00,3.00
-Valid N (listwise),3,,,,
-Missing N (listwise),0,,,,
-])
-AT_CLEANUP
-
-
-dnl This scenario was observed to erroneously free things twice
-AT_SETUP([SET crash on invalid cc])
-AT_DATA([set.pspp], [dnl
-SET CCA='xxxx'.SHGW CCA.
-])
-
-AT_CHECK([pspp -O format=csv set.pspp], [1], [dnl
-"set.pspp:1.9-1.14: error: SET: Custom currency string `CCA' for xxxx does not contain exactly three periods or commas (or it contains both).
- 1 | SET CCA='xxxx'.SHGW CCA.
- | ^~~~~~"
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([SET MXWARNS])
-dnl Make sure that syntax processing stops and that
-dnl a warning is issued when the MXWARNS figure is
-dnl exceeded.
-AT_DATA([set.pspp], [dnl
-set mxwarns=2.
-data list notable list /x (f8.2) y (f8.2).
-begin data
-1 2
-3 r
-5 x
-q 8
-9 9
-3 x
-w w
-end data.
-
-comment The following line should not be executed.
-list.
-])
-
-AT_CHECK([pspp -O format=csv set.pspp], [0], [dnl
-set.pspp:5.3: warning: Data for variable y is not valid as format F: Field contents are not numeric.
-
-set.pspp:6.3: warning: Data for variable y is not valid as format F: Field contents are not numeric.
-
-set.pspp:7.1: warning: Data for variable x is not valid as format F: Field contents are not numeric.
-
-note: Warnings (3) exceed limit (2). Syntax processing will be halted.
-])
-
-AT_CLEANUP
-
-
-
-
-AT_SETUP([SET MXWARNS special case zero])
-dnl Make sure that MXWARNS interprets zero as infinity.
-AT_DATA([mxwarns.pspp], [dnl
-set mxwarns=0.
-data list notable list /x (f8.2) y (f8.2) z *.
-begin data
-1 2 3
-3 r 3
-5 x 3
-q 8 4
-9 9 4
-3 x 4
-w w 4
-end data.
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv mxwarns.pspp], [0],
-[warning: MXWARNS set to zero. No further warnings will be given even when potentially problematic situations are encountered.
-
-Table: Data List
-x,y,z
-1.00,2.00,3.00
-3.00,. ,3.00
-5.00,. ,3.00
-. ,8.00,4.00
-9.00,9.00,4.00
-3.00,. ,4.00
-. ,. ,4.00
-])
-
-AT_CLEANUP
-
-AT_SETUP([SET macro - MEXPAND MPRINT MITERATE MNEST])
-AT_DATA([set-macro.sps], [dnl
-show mexpand mprint miterate mnest.
-preserve.
-set mexpand=off mprint=on miterate=10 mnest=11.
-show mexpand mprint miterate mnest.
-restore.
-show mexpand mprint miterate mnest.
-])
-AT_CHECK([pspp -O format=csv set-macro.sps], [0], [dnl
-Table: Settings
-MEXPAND,ON
-MPRINT,OFF
-MITERATE,1000
-MNEST,50
-
-Table: Settings
-MEXPAND,OFF
-MPRINT,ON
-MITERATE,10
-MNEST,11
-
-Table: Settings
-MEXPAND,ON
-MPRINT,OFF
-MITERATE,1000
-MNEST,50
-])
-AT_CLEANUP
-
-AT_SETUP([SET syntax errors])
-AT_DATA([set.sps], [dnl
-SET **.
-SET BASETEXTDIRECTION=**.
-SET BLANKS=**.
-SET BOX=**.
-SET CACHE=**.
-SET CCA=**.
-SET CELLSBREAK=**.
-SET CMPTRANS=**.
-SET COMPRESSION=**.
-SET CTEMPLATE=**.
-SET DECIMAL=**.
-SET EPOCH=**.
-SET EPOCH=1234.
-SET ERRORS=**.
-SET FORMAT=**.
-SET FORMAT=A8.
-SET FORMAT=F1.2.
-SET FUZZBITS=40.
-SET HEADER=**.
-SET INCLUDE=**.
-SET JOURNAL=**.
-SET LEADZERO=**.
-SET LENGTH=**.
-SET LOCALE='Neverland'.
-SET LOCALE=**.
-SET MDISPLAY=**.
-SET MESSAGES=**.
-SET MEXPAND=**.
-SET MITERATE=0.
-SET MNEST=0.
-SET MPRINT=**.
-SET MXERRS=0.
-SET MXLOOPS=0.
-SET MXWARNS=-1.
-SET PRINTBACK=**.
-SET RESULTS=**.
-SET RIB=**.
-SET RRB=**.
-SET SAFER=**.
-SET SCOMPRESSION=**.
-SET SEED=**.
-SET SMALL=**.
-SET SUBTITLE=**.
-SET TNUMBERS=**.
-SET TVARS=**.
-SET TLOOK='nonexistent.xml'.
-SET UNDEFINED=**.
-SET WIB=**.
-SET WRB=**.
-SET WIDTH=**.
-SET WORKSPACE=**.
-])
-AT_CHECK([pspp -O format=csv set.sps], [1], [dnl
-"set.sps:1.5-1.6: error: SET: Syntax error expecting the name of a setting.
- 1 | SET **.
- | ^~"
-
-"set.sps:2.5-2.24: warning: SET: BASETEXTDIRECTION is not yet implemented.
- 2 | SET BASETEXTDIRECTION=**.
- | ^~~~~~~~~~~~~~~~~~~~"
-
-"set.sps:3.12-3.13: error: SET: Syntax error expecting number.
- 3 | SET BLANKS=**.
- | ^~"
-
-"set.sps:4.5-4.10: warning: SET: BOX is not yet implemented.
- 4 | SET BOX=**.
- | ^~~~~~"
-
-"set.sps:5.5-5.12: warning: SET: CACHE is not yet implemented.
- 5 | SET CACHE=**.
- | ^~~~~~~~"
-
-"set.sps:6.9-6.10: error: SET: Syntax error expecting string.
- 6 | SET CCA=**.
- | ^~"
-
-"set.sps:7.5-7.17: warning: SET: CELLSBREAK is not yet implemented.
- 7 | SET CELLSBREAK=**.
- | ^~~~~~~~~~~~~"
-
-"set.sps:8.5-8.15: warning: SET: CMPTRANS is not yet implemented.
- 8 | SET CMPTRANS=**.
- | ^~~~~~~~~~~"
-
-"set.sps:9.5-9.18: warning: SET: COMPRESSION is not yet implemented.
- 9 | SET COMPRESSION=**.
- | ^~~~~~~~~~~~~~"
-
-"set.sps:10.5-10.16: warning: SET: CTEMPLATE is not yet implemented.
- 10 | SET CTEMPLATE=**.
- | ^~~~~~~~~~~~"
-
-"set.sps:11.13-11.14: error: SET: Syntax error expecting DOT or COMMA.
- 11 | SET DECIMAL=**.
- | ^~"
-
-"set.sps:12.11-12.12: error: SET: Syntax error expecting AUTOMATIC or year.
- 12 | SET EPOCH=**.
- | ^~"
-
-"set.sps:13.11-13.14: error: SET: Syntax error expecting integer 1500 or greater for EPOCH.
- 13 | SET EPOCH=1234.
- | ^~~~"
-
-"set.sps:14.12-14.13: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
- 14 | SET ERRORS=**.
- | ^~"
-
-"set.sps:15.12-15.13: error: SET: Syntax error expecting valid format specifier.
- 15 | SET FORMAT=**.
- | ^~"
-
-"set.sps:16.5-16.13: error: SET: FORMAT requires numeric output format as an argument. Specified format A8 is of type string.
- 16 | SET FORMAT=A8.
- | ^~~~~~~~~"
-
-"set.sps:17.12-17.15: error: SET: Output format F1.2 specifies 2 decimal places, but width 1 does not allow for any decimals.
- 17 | SET FORMAT=F1.2.
- | ^~~~"
-
-"set.sps:18.14-18.15: error: SET: Syntax error expecting integer between 0 and 20 for FUZZBITS.
- 18 | SET FUZZBITS=40.
- | ^~"
-
-"set.sps:19.5-19.13: warning: SET: HEADER is not yet implemented.
- 19 | SET HEADER=**.
- | ^~~~~~~~~"
-
-"set.sps:20.13-20.14: error: SET: Syntax error expecting ON, YES, OFF, or NO.
- 20 | SET INCLUDE=**.
- | ^~"
-
-"set.sps:21.13-21.14: error: SET: Syntax error expecting ON or OFF or a file name.
- 21 | SET JOURNAL=**.
- | ^~"
-
-"set.sps:22.14-22.15: error: SET: Syntax error expecting ON, YES, OFF, or NO.
- 22 | SET LEADZERO=**.
- | ^~"
-
-"set.sps:23.12-23.13: error: SET: Syntax error expecting positive integer for LENGTH.
- 23 | SET LENGTH=**.
- | ^~"
-
-"set.sps:24.12-24.22: error: SET: Neverland is not a recognized encoding or locale name.
- 24 | SET LOCALE='Neverland'.
- | ^~~~~~~~~~~"
-
-"set.sps:25.12-25.13: error: SET: Syntax error expecting string.
- 25 | SET LOCALE=**.
- | ^~"
-
-"set.sps:26.14-26.15: error: SET: Syntax error expecting TEXT or TABLES.
- 26 | SET MDISPLAY=**.
- | ^~"
-
-"set.sps:27.14-27.15: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
- 27 | SET MESSAGES=**.
- | ^~"
-
-"set.sps:28.13-28.14: error: SET: Syntax error expecting ON, YES, OFF, or NO.
- 28 | SET MEXPAND=**.
- | ^~"
-
-"set.sps:29.14: error: SET: Syntax error expecting positive integer for MITERATE.
- 29 | SET MITERATE=0.
- | ^"
-
-"set.sps:30.11: error: SET: Syntax error expecting positive integer for MNEST.
- 30 | SET MNEST=0.
- | ^"
-
-"set.sps:31.12-31.13: error: SET: Syntax error expecting ON, YES, OFF, or NO.
- 31 | SET MPRINT=**.
- | ^~"
-
-"set.sps:32.12: error: SET: Syntax error expecting positive integer for MXERRS.
- 32 | SET MXERRS=0.
- | ^"
-
-"set.sps:33.13: error: SET: Syntax error expecting positive integer for MXLOOPS.
- 33 | SET MXLOOPS=0.
- | ^"
-
-"set.sps:34.13-34.14: error: SET: Syntax error expecting non-negative integer for MXWARNS.
- 34 | SET MXWARNS=-1.
- | ^~"
-
-"set.sps:35.15-35.16: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
- 35 | SET PRINTBACK=**.
- | ^~"
-
-"set.sps:36.13-36.14: error: SET: Syntax error expecting ON, BOTH, TERMINAL, LISTING, OFF, or NONE.
- 36 | SET RESULTS=**.
- | ^~"
-
-"set.sps:37.9-37.10: error: SET: Syntax error expecting MSBFIRST, LSBFIRST, VAX, or NATIVE.
- 37 | SET RIB=**.
- | ^~"
-
-"set.sps:38.9-38.10: error: SET: Syntax error expecting one of the following: NATIVE, ISL, ISB, IDL, IDB, VF, VD, VG, ZS.
- 38 | SET RRB=**.
- | ^~"
-
-"set.sps:39.11-39.12: error: SET: Syntax error expecting ON or YES.
- 39 | SET SAFER=**.
- | ^~"
-
-"set.sps:40.18-40.19: error: SET: Syntax error expecting ON, YES, OFF, or NO.
- 40 | SET SCOMPRESSION=**.
- | ^~"
-
-"set.sps:41.10-41.11: error: SET: Syntax error expecting number.
- 41 | SET SEED=**.
- | ^~"
-
-"set.sps:42.11-42.12: error: SET: Syntax error expecting number.
- 42 | SET SMALL=**.
- | ^~"
-
-"set.sps:43.5-43.12: error: SET: Syntax error expecting the name of a setting.
- 43 | SET SUBTITLE=**.
- | ^~~~~~~~"
-
-"set.sps:44.14-44.15: error: SET: Syntax error expecting LABELS, VALUES, or BOTH.
- 44 | SET TNUMBERS=**.
- | ^~"
-
-"set.sps:45.11-45.12: error: SET: Syntax error expecting LABELS, NAMES, or BOTH.
- 45 | SET TVARS=**.
- | ^~"
-
-set.sps:46: error: SET: nonexistent.xml: not found
-
-"set.sps:47.15-47.16: error: SET: Syntax error expecting WARN or NOWARN.
- 47 | SET UNDEFINED=**.
- | ^~"
-
-"set.sps:48.9-48.10: error: SET: Syntax error expecting MSBFIRST, LSBFIRST, VAX, or NATIVE.
- 48 | SET WIB=**.
- | ^~"
-
-"set.sps:49.9-49.10: error: SET: Syntax error expecting one of the following: NATIVE, ISL, ISB, IDL, IDB, VF, VD, VG, ZS.
- 49 | SET WRB=**.
- | ^~"
-
-"set.sps:50.11-50.12: error: SET: Syntax error expecting integer 40 or greater for WIDTH.
- 50 | SET WIDTH=**.
- | ^~"
-
-"set.sps:51.15-51.16: error: SET: Syntax error expecting integer 1024 or greater for WORKSPACE.
- 51 | SET WORKSPACE=**.
- | ^~"
-])
-AT_CLEANUP
-\f
-AT_BANNER([PRESERVE and RESTORE])
-
-AT_SETUP([PRESERVE of SET FORMAT])
-AT_DATA([set.pspp], [dnl
-SHOW FORMAT.
-PRESERVE.
-SET FORMAT F10.0.
-SHOW FORMAT
-RESTORE.
-SHOW FORMAT.
-])
-AT_CHECK([pspp -O format=csv set.pspp], [0], [dnl
-Table: Settings
-FORMAT,F8.2
-
-Table: Settings
-FORMAT,F10.0
-
-Table: Settings
-FORMAT,F8.2
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SHOW])
-
-AT_SETUP([SHOW N])
-
-AT_DATA([show.sps], [dnl
-DATA LIST LIST NOTABLE /x.
-BEGIN DATA.
-1
-2
-3
-END DATA.
-
-SHOW N.
-])
-
-AT_CHECK([pspp -O format=csv show.sps], [0], [dnl
-Table: Settings
-N,3
-])
-
-AT_CLEANUP
-
-
-AT_SETUP([SHOW N empty])
-
-AT_DATA([shown-empty.sps], [dnl
-SHOW N.
-])
-
-AT_CHECK([pspp -O format=csv shown-empty.sps], [0], [dnl
-Table: Settings
-N,Unknown
-])
-
-AT_CLEANUP
-
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([TITLE and related commands])
-
-AT_SETUP([FILE LABEL and (ADD) DOCUMENT])
-AT_DATA([file-label.sps], [dnl
-/* Set up a dummy active dataset in memory.
-data list /X 1 Y 2.
-begin data.
-16
-27
-38
-49
-50
-end data.
-
-/* Add value labels for some further testing of value labels.
-value labels x y 1 'first label' 2 'second label' 3 'third label'.
-add value labels x 1 'first label mark two'.
-
-/* Add a file label and a few documents.
-file label This is a test file label.
-document First line of a document
-Second line of a document
-The last line should end with a period: .
-
-
-/* Display the documents.
-display documents.
-display file label.
-
-ADD DOCUMENT 'Line one' 'Line two'.
-
-/* Save the active dataset then get it and display the documents again.
-save /OUTFILE='foo.save'.
-get /FILE='foo.save'.
-display documents.
-display file label.
-
-/* There is an interesting interaction that occurs if the 'execute'
-/* command below. What happens is that an error message is output
-/* at the next 'save' command that 'foo.save' is already open for
-/* input. This is because the 'get' hasn't been executed yet and
-/* therefore PSPP would be reading from and writing to the same
-/* file at once, which is obviously a Bad Thing. But 'execute'
-/* here clears up that potential problem.
-execute.
-
-/* Add another (shorter) document and try again.
-document There should be another document now.
-display documents.
-
-/* Save and get.
-save /OUTFILE='foo.save'.
-get /FILE='foo.save'.
-display documents.
-display file label.
-
-/* Done.
-])
-AT_CHECK([pspp -o pspp.csv file-label.sps])
-dnl Filter out the dates/times
-AT_CHECK([[sed 's/(Entered [^)]*)/(Entered <date>)/' pspp.csv]], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-X,1,1-1,F1.0
-Y,1,2-2,F1.0
-
-Table: Documents
-"document First line of a document
-Second line of a document
-The last line should end with a period: .
- (Entered <date>)"
-
-Table: File Label
-Label,This is a test file label
-
-Table: Documents
-"document First line of a document
-Second line of a document
-The last line should end with a period: .
- (Entered <date>)
-Line one
-Line two
- (Entered <date>)"
-
-Table: File Label
-Label,This is a test file label
-
-Table: Documents
-"document First line of a document
-Second line of a document
-The last line should end with a period: .
- (Entered <date>)
-Line one
-Line two
- (Entered <date>)
-document There should be another document now.
- (Entered <date>)"
-
-Table: Documents
-"document First line of a document
-Second line of a document
-The last line should end with a period: .
- (Entered <date>)
-Line one
-Line two
- (Entered <date>)
-document There should be another document now.
- (Entered <date>)"
-
-Table: File Label
-Label,This is a test file label
-])
-AT_CLEANUP
-
-AT_SETUP([TITLE and SUBTITLE])
-for command in TITLE SUBTITLE; do
- cat >title.sps <<EOF
-$command foo bar.
-SHOW $command.
-
-$command 'foo bar baz quux'.
-SHOW $command.
-EOF
- cat >expout <<EOF
-Table: Settings
-$command,foo bar
-
-Table: Settings
-$command,foo bar baz quux
-EOF
- AT_CHECK([pspp -O format=csv title.sps], [0], [expout])
-done
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([COMPUTE transformation])
-
-AT_SETUP([COMPUTE crash with SAVE])
-AT_DATA([compute.sps],
- [INPUT PROGRAM.
-COMPUTE num = 3.
-END FILE.
-END INPUT PROGRAM.
-EXECUTE.
-
-SAVE outfile='temp.sav'.
-])
-AT_CHECK([pspp -O format=csv compute.sps])
-AT_DATA([list.sps],
- [GET FILE='temp.sav'.
-LIST.
-])
-AT_CHECK([pspp -O format=csv list.sps], [0],
- [])
-AT_CLEANUP
-
-AT_SETUP([COMPUTE bug in long string UPCASE])
-AT_DATA([compute.sps],
- [DATA LIST LIST
- /A (A161)
- B (A3).
-
-BEGIN DATA
-abc def
-ghi jkl
-END DATA.
-
-COMPUTE A=upcase(A).
-EXECUTE.
-LIST.
-])
-AT_CHECK([pspp -O format=csv compute.sps], [0],
- [Table: Reading free-form data from INLINE.
-Variable,Format
-A,A161
-B,A3
-
-Table: Data List
-A,B
-ABC,def
-GHI,jkl
-])
-AT_CLEANUP
-
-AT_SETUP([COMPUTE bug with long variable names])
-AT_DATA([compute.sps],
- [DATA LIST LIST /longVariablename * x *.
-BEGIN DATA.
-1 2
-3 4
-END DATA.
-
-
-COMPUTE longvariableName=100-longvariablename.
-
-LIST.
-])
-AT_CHECK([pspp -O format=csv compute.sps], [0],
- [Table: Reading free-form data from INLINE.
-Variable,Format
-longVariablename,F8.0
-x,F8.0
-
-Table: Data List
-longVariablename,x
-99.00,2.00
-97.00,4.00
-])
-AT_CLEANUP
-
-AT_SETUP([COMPUTE self-reference to new variable])
-AT_DATA([compute.sps],
- [DATA LIST /ITEM 1-3.
-COMPUTE SUM=SUM+ITEM.
-PRINT OUTFILE='compute-sum.out' /ITEM SUM.
-LEAVE SUM
-BEGIN DATA.
-123
-404
-555
-999
-END DATA.
-])
-AT_CHECK([pspp -O format=csv compute.sps], [0],
- [Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-ITEM,1,1-3,F3.0
-])
-AT_CHECK([cat compute-sum.out], [0],
- [ 123 123.00 @&t@
- 404 527.00 @&t@
- 555 1082.00 @&t@
- 999 2081.00 @&t@
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([COUNT])
-
-AT_SETUP([COUNT -- numeric data])
-AT_DATA([count.sps], [dnl
-DATA LIST LIST /x y.
-BEGIN DATA.
-1 2
-2 3
-4 5
-2 2
-5 6
-7 2
-. 2
-END DATA.
-
-MISSING VALUES x(7)/y(3).
-
-COUNT c=x y (2)/d=x y(7)/e=x y(missing)/f=x y(sysmis).
-
-FORMATS ALL(F1).
-
-LIST.
-])
-AT_CHECK([pspp -O format=csv count.sps], [0], [dnl
-Table: Reading free-form data from INLINE.
-Variable,Format
-x,F8.0
-y,F8.0
-
-Table: Data List
-x,y,c,d,e,f
-1,2,1,0,0,0
-2,3,1,0,1,0
-4,5,0,0,0,0
-2,2,2,0,0,0
-5,6,0,0,0,0
-7,2,1,1,1,0
-.,2,1,0,1,1
-])
-AT_CLEANUP
-
-AT_SETUP([COUNT -- string data])
-AT_DATA([count.sps], [dnl
-TITLE 'Test COUNT transformation'.
-
-DATA LIST /v1 to v2 1-4(a).
-BEGIN DATA.
-1234
-321
-2 13
-4121
-1104
-03 4
-0193
-END DATA.
-COUNT c=v1 to v2('2',' 4','1').
-LIST.
-])
-AT_CHECK([pspp -O format=csv count.sps], [0], [dnl
-Table: Reading 1 record from INLINE.
-Variable,Record,Columns,Format
-v1,1,1-2,A2
-v2,1,3-4,A2
-
-Table: Data List
-v1,v2,c
-12,34,.00
-32,1,1.00
-2,13,1.00
-41,21,.00
-11,04,.00
-03,4,1.00
-01,93,.00
-])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([RECODE transformation])
-
-m4_define([RECODE_SAMPLE_DATA],
- [DATA LIST LIST NOTABLE/x (f1) s (a4) t (a10).
-MISSING VALUES x(9)/s('xxx').
-BEGIN DATA.
-0, '', ''
-1, a, a
-2, ab, ab
-3, abc, abc
-4, abcd, abcd
-5, 123, 123
-6, ' 123', ' 123'
-7, +1, +1
-8, 1x, 1x
-9, abcd, abcdefghi
-, xxx, abcdefghij
-END DATA.
-])
-
-AT_SETUP([RECODE numeric to numeric, without INTO])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-NUMERIC x0 TO x8 (F3).
-MISSING VALUES x0 to x8 (9).
-COMPUTE x0=value(x).
-RECODE x0 (1=9).
-COMPUTE x1=value(x).
-RECODE x1 (1=9)(3=8)(5=7).
-COMPUTE x2=value(x).
-RECODE x2 (1=8)(2,3,4,5,6,8=9)(9=1).
-COMPUTE x3=value(x).
-RECODE x3 (1 THRU 9=10)(MISSING=11).
-COMPUTE x4=value(x).
-RECODE x4 (MISSING=11)(1 THRU 9=10).
-COMPUTE x5=value(x).
-RECODE x5 (LOWEST THRU 5=1).
-COMPUTE x6=value(x).
-RECODE x6 (4 THRU HIGHEST=2).
-COMPUTE x7=value(x).
-RECODE x7 (LO THRU HI=3).
-COMPUTE x8=value(x).
-RECODE x8 (SYSMIS=4).
-LIST x x0 TO x8.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-x,x0,x1,x2,x3,x4,x5,x6,x7,x8
-0,0,0,0,0,0,1,0,3,0
-1,9,9,8,10,10,1,1,3,1
-2,2,2,9,10,10,1,2,3,2
-3,3,8,9,10,10,1,3,3,3
-4,4,4,9,10,10,1,2,3,4
-5,5,7,9,10,10,1,2,3,5
-6,6,6,9,10,10,6,2,3,6
-7,7,7,7,10,10,7,2,3,7
-8,8,8,9,10,10,8,2,3,8
-9,9,9,1,10,11,9,2,3,9
-.,.,.,.,11,11,.,.,.,4
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE numeric to numeric, with INTO, without COPY])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-NUMERIC ix0 TO ix8 (F3).
-RECODE x (1=9) INTO ix0.
-RECODE x (1=9)(3=8)(5=7) INTO ix1.
-RECODE x (1=8)(2,3,4,5,6,8=9)(9=1) INTO ix2.
-RECODE x (1 THRU 9=10)(MISSING=11) INTO ix3.
-RECODE x (MISSING=11)(1 THRU 9=10) INTO ix4.
-RECODE x (LOWEST THRU 5=1) INTO ix5.
-RECODE x (4 THRU HIGHEST=2) INTO ix6.
-RECODE x (LO THRU HI=3) INTO ix7.
-RECODE x (SYSMIS=4) INTO ix8.
-LIST x ix0 TO ix8.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-x,ix0,ix1,ix2,ix3,ix4,ix5,ix6,ix7,ix8
-0,.,.,.,.,.,1,.,3,.
-1,9,9,8,10,10,1,.,3,.
-2,.,.,9,10,10,1,.,3,.
-3,.,8,9,10,10,1,.,3,.
-4,.,.,9,10,10,1,2,3,.
-5,.,7,9,10,10,1,2,3,.
-6,.,.,9,10,10,.,2,3,.
-7,.,.,.,10,10,.,2,3,.
-8,.,.,9,10,10,.,2,3,.
-9,.,.,1,10,11,.,2,3,.
-.,.,.,.,11,11,.,.,.,4
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE numeric to numeric, with INTO, with COPY])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-NUMERIC cx0 TO cx8 (F3).
-RECODE x (1=9)(ELSE=COPY) INTO cx0.
-RECODE x (1=9)(3=8)(5=7)(ELSE=COPY) INTO cx1.
-RECODE x (1=8)(2,3,4,5,6,8=9)(9=1)(ELSE=COPY) INTO cx2.
-RECODE x (1 THRU 9=10)(MISSING=11)(ELSE=COPY) INTO cx3.
-RECODE x (MISSING=11)(1 THRU 9=10)(ELSE=COPY) INTO cx4.
-RECODE x (LOWEST THRU 5=1)(ELSE=COPY) INTO cx5.
-RECODE x (4 THRU HIGHEST=2)(ELSE=COPY) INTO cx6.
-RECODE x (LO THRU HI=3)(ELSE=COPY) INTO cx7.
-RECODE x (SYSMIS=4)(ELSE=COPY) INTO cx8.
-RECODE x (5=COPY)(ELSE=22) INTO cx9.
-LIST x cx0 TO cx9.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-x,cx0,cx1,cx2,cx3,cx4,cx5,cx6,cx7,cx8,cx9
-0,0,0,0,0,0,1,0,3,0,22.00
-1,9,9,8,10,10,1,1,3,1,22.00
-2,2,2,9,10,10,1,2,3,2,22.00
-3,3,8,9,10,10,1,3,3,3,22.00
-4,4,4,9,10,10,1,2,3,4,22.00
-5,5,7,9,10,10,1,2,3,5,5.00
-6,6,6,9,10,10,6,2,3,6,22.00
-7,7,7,7,10,10,7,2,3,7,22.00
-8,8,8,9,10,10,8,2,3,8,22.00
-9,9,9,1,10,11,9,2,3,9,22.00
-.,.,.,.,11,11,.,.,.,4,22.00
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE string to string, with INTO, without COPY])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-STRING s0 TO s3 (A4)/t0 TO t3 (A10).
-RECODE s t ('a'='b')('ab'='bc') INTO s0 t0.
-RECODE s t ('abcd'='xyzw') INTO s1 t1.
-RECODE s t ('abc'='def')(ELSE='xyz') INTO s2 t2.
-RECODE t ('a'='b')('abcdefghi'='xyz')('abcdefghij'='jklmnopqr') INTO t3.
-RECODE s (MISSING='gone') INTO s3.
-LIST s t s0 TO s3 t0 TO t3.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-s,t,s0,s1,s2,s3,t0,t1,t2,t3
-,,,,xyz,,,,xyz,
-a,a,b,,xyz,,b,,xyz,b
-ab,ab,bc,,xyz,,bc,,xyz,
-abc,abc,,,def,,,,def,
-abcd,abcd,,xyzw,xyz,,,xyzw,xyz,
-123,123,,,xyz,,,,xyz,
-123,123,,,xyz,,,,xyz,
-+1,+1,,,xyz,,,,xyz,
-1x,1x,,,xyz,,,,xyz,
-abcd,abcdefghi,,xyzw,xyz,,,,xyz,xyz
-xxx,abcdefghij,,,xyz,gone,,,xyz,jklmnopqr
-])
-AT_CLEANUP
-
-AT_SETUP(RECODE string to string, with INTO, with COPY])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-STRING cs0 TO cs2 (A4)/ct0 TO ct3 (A10).
-RECODE s t ('a'='b')('ab'='bc')(ELSE=COPY) INTO cs0 ct0.
-RECODE s t ('abcd'='xyzw')(ELSE=COPY) INTO cs1 ct1.
-RECODE s t ('abc'='def')(ELSE='xyz')(ELSE=COPY) INTO cs2 ct2.
-RECODE t ('a'='b')('abcdefghi'='xyz')('abcdefghij'='jklmnopqr')(ELSE=COPY)
- INTO ct3.
-LIST s t cs0 TO cs2 ct0 TO ct3.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-s,t,cs0,cs1,cs2,ct0,ct1,ct2,ct3
-,,,,xyz,,,xyz,
-a,a,b,a,xyz,b,a,xyz,b
-ab,ab,bc,ab,xyz,bc,ab,xyz,ab
-abc,abc,abc,abc,def,abc,abc,def,abc
-abcd,abcd,abcd,xyzw,xyz,abcd,xyzw,xyz,abcd
-123,123,123,123,xyz,123,123,xyz,123
-123,123,123,123,xyz,123,123,xyz,123
-+1,+1,+1,+1,xyz,+1,+1,xyz,+1
-1x,1x,1x,1x,xyz,1x,1x,xyz,1x
-abcd,abcdefghi,abcd,xyzw,xyz,abcdefghi,abcdefghi,xyz,xyz
-xxx,abcdefghij,xxx,xxx,xyz,abcdefghij,abcdefghij,xyz,jklmnopqr
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE string to numeric])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-NUMERIC ns0 TO ns2 (F3)/nt0 TO nt2 (F3).
-RECODE s t (CONVERT)(' '=0)('abcd'=1) INTO ns0 nt0.
-RECODE s t (' '=0)(CONVERT)('abcd'=1) INTO ns1 nt1.
-RECODE s t ('1x'=1)('abcd'=2)(ELSE=3) INTO ns2 nt2.
-LIST s t ns0 TO ns2 nt0 TO nt2.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-s,t,ns0,ns1,ns2,nt0,nt1,nt2
-,,.,0,3,.,0,3
-a,a,.,.,3,.,.,3
-ab,ab,.,.,3,.,.,3
-abc,abc,.,.,3,.,.,3
-abcd,abcd,1,1,2,1,1,2
-123,123,123,123,3,123,123,3
-123,123,123,123,3,123,123,3
-+1,+1,1,1,3,1,1,3
-1x,1x,.,.,1,.,.,1
-abcd,abcdefghi,1,1,2,.,.,3
-xxx,abcdefghij,.,.,3,.,.,3
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE numeric to string])
-AT_DATA([recode.sps],
- [RECODE_SAMPLE_DATA
-STRING sx0 TO sx2 (a10).
-RECODE x (1 THRU 9='abcdefghij') INTO sx0.
-RECODE x (0,1,3,5,7,MISSING='xxx') INTO sx1.
-RECODE x (2 THRU 6,SYSMIS='xyz')(ELSE='foobar') INTO sx2.
-LIST x sx0 TO sx2.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Data List
-x,sx0,sx1,sx2
-0,,xxx,foobar
-1,abcdefghij,xxx,foobar
-2,abcdefghij,,xyz
-3,abcdefghij,xxx,xyz
-4,abcdefghij,,xyz
-5,abcdefghij,xxx,xyz
-6,abcdefghij,,xyz
-7,abcdefghij,xxx,foobar
-8,abcdefghij,,foobar
-9,abcdefghij,xxx,foobar
-.,,xxx,xyz
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE bug in COPY])
-AT_DATA([recode.sps],
- [DATA LIST LIST
- /A (A1)
- B (A1).
-
-BEGIN DATA
-1 2
-2 3
-3 4
-END DATA.
-
-** Clearly, the else=copy is superfluous here
-RECODE A ("1"="3") ("3"="1") (ELSE=COPY).
-EXECUTE.
-LIST.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Reading free-form data from INLINE.
-Variable,Format
-A,A1
-B,A1
-
-Table: Data List
-A,B
-3,2
-2,3
-1,4
-])
-AT_CLEANUP
-
-AT_SETUP([RECODE bug in COPY with INTO])
-AT_DATA([recode.sps],
- [DATA LIST LIST
- /A (A1)
- B (A1).
-
-BEGIN DATA
-1 2
-2 3
-3 4
-END DATA.
-
-STRING A1 (A1).
-RECODE A ("1"="3") ("3"="1") (ELSE=COPY) INTO a1.
-EXECUTE.
-LIST.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [0],
- [Table: Reading free-form data from INLINE.
-Variable,Format
-A,A1
-B,A1
-
-Table: Data List
-A,B,A1
-1,2,3
-2,3,2
-3,4,1
-])
-AT_CLEANUP
-
-
-
-AT_SETUP([RECODE increased string widths])
-
-AT_DATA([recode.sps],[dnl
-data list notable list /x (a1) y (a8) z *.
-begin data.
-a a 2
-a two 2
-b three 2
-c b 2
-end data.
-
-recode x y ("a" = "first") .
-
-list.
-])
-
-AT_CHECK([pspp -O format=csv recode.sps], [1], [dnl
-recode.sps:9: error: RECODE: At least one target variable is too narrow for the output values.
-
-"recode.sps:9.19-9.25: note: RECODE: This recoding output value has width 5.
- 9 | recode x y (""a"" = ""first"") .
- | ^~~~~~~"
-
-"recode.sps:9.8-9.10: note: RECODE: Target variable x only has width 1.
- 9 | recode x y (""a"" = ""first"") .
- | ^~~"
-
-Table: Data List
-x,y,z
-a,a,2.00
-a,two,2.00
-b,three,2.00
-c,b,2.00
-])
-
-AT_CLEANUP
-
-
-
-AT_SETUP([RECODE crash on invalid dest variable])
-
-AT_DATA([recode.sps],[dnl
-DATA LIST LIST NOTABLE/x (f1) s (a4) t (a10).
-MISSING VALUES x(9)/s('xxx').
-BEGIN DATA.
-0, '', ''
-1, a, a
-2, ab, ab
-3, abc, abc
-END DATA.
-
-RECODE x (1=9) INTO ".
-EXECUTE.
-dnl "
-])
-
-AT_CHECK([pspp -O format=csv recode.sps], [1], [ignore])
-
-AT_CLEANUP
-
-
-AT_SETUP([RECODE syntax errors])
-AT_DATA([recode.sps], [dnl
-DATA LIST LIST NOTABLE/n1 to n4 (F8.0) s1 (A1) s2 (A2) s3 (A3) s4 (A4).
-RECODE **.
-RECODE n1 **.
-RECODE n1(**).
-RECODE s1(**).
-RECODE s1('x' THRU 'y').
-RECODE n1(1=**).
-RECODE n1(CONVERT).
-RECODE n1(1=2)(1='x').
-RECODE n1(1='x')(1=2).
-RECODE s1(CONVERT)('1'='1').
-RECODE n1(1=2 **).
-RECODE n1(CONVERT).
-RECODE s1(CONVERT) INTO n1 n2.
-RECODE n1(1='1') INTO xyzzy.
-RECODE n1(1='1').
-RECODE s1('1'=1).
-RECODE n1(1='1') INTO n2.
-RECODE s1(CONVERT) INTO s2.
-RECODE n1 TO n4(1='123456') INTO s1 TO s4.
-])
-AT_CHECK([pspp -O format=csv recode.sps], [1], [dnl
-"recode.sps:2.8-2.9: error: RECODE: Syntax error expecting variable name.
- 2 | RECODE **.
- | ^~"
-
-"recode.sps:3.11-3.12: error: RECODE: Syntax error expecting `('.
- 3 | RECODE n1 **.
- | ^~"
-
-"recode.sps:4.11-4.12: error: RECODE: Syntax error expecting number.
- 4 | RECODE n1(**).
- | ^~"
-
-"recode.sps:5.11-5.12: error: RECODE: Syntax error expecting string.
- 5 | RECODE s1(**).
- | ^~"
-
-"recode.sps:6.15-6.18: error: RECODE: THRU is not allowed with string variables.
- 6 | RECODE s1('x' THRU 'y').
- | ^~~~"
-
-"recode.sps:7.13-7.14: error: RECODE: Syntax error expecting output value.
- 7 | RECODE n1(1=**).
- | ^~"
-
-"recode.sps:8.11-8.17: error: RECODE: CONVERT requires string input values.
- 8 | RECODE n1(CONVERT).
- | ^~~~~~~"
-
-recode.sps:9: error: RECODE: Output values must be all numeric or all string.
-
-"recode.sps:9.13: note: RECODE: This output value is numeric.
- 9 | RECODE n1(1=2)(1='x').
- | ^"
-
-"recode.sps:9.18-9.20: note: RECODE: This output value is string.
- 9 | RECODE n1(1=2)(1='x').
- | ^~~"
-
-recode.sps:10: error: RECODE: Output values must be all numeric or all string.
-
-"recode.sps:10.20: note: RECODE: This output value is numeric.
- 10 | RECODE n1(1='x')(1=2).
- | ^"
-
-"recode.sps:10.13-10.15: note: RECODE: This output value is string.
- 10 | RECODE n1(1='x')(1=2).
- | ^~~"
-
-recode.sps:11: error: RECODE: Output values must be all numeric or all string.
-
-"recode.sps:11.11-11.17: note: RECODE: This output value is numeric.
- 11 | RECODE s1(CONVERT)('1'='1').
- | ^~~~~~~"
-
-"recode.sps:11.24-11.26: note: RECODE: This output value is string.
- 11 | RECODE s1(CONVERT)('1'='1').
- | ^~~"
-
-"recode.sps:12.15-12.16: error: RECODE: Syntax error expecting `)'.
- 12 | RECODE n1(1=2 **).
- | ^~"
-
-"recode.sps:13.11-13.17: error: RECODE: CONVERT requires string input values.
- 13 | RECODE n1(CONVERT).
- | ^~~~~~~"
-
-recode.sps:14: error: RECODE: Source and target variable counts must match.
-
-"recode.sps:14.8-14.9: note: RECODE: There is 1 source variable.
- 14 | RECODE s1(CONVERT) INTO n1 n2.
- | ^~"
-
-"recode.sps:14.25-14.29: note: RECODE: There are 2 target variables.
- 14 | RECODE s1(CONVERT) INTO n1 n2.
- | ^~~~~"
-
-recode.sps:15: error: RECODE: All string variables specified on INTO must already exist. (Use the STRING command to create a string variable.)
-
-"recode.sps:15.23-15.27: note: RECODE: There is no variable named xyzzy.
- 15 | RECODE n1(1='1') INTO xyzzy.
- | ^~~~~"
-
-"recode.sps:16.10-16.16: error: RECODE: INTO is required with numeric input values and string output values.
- 16 | RECODE n1(1='1').
- | ^~~~~~~"
-
-"recode.sps:17.10-17.16: error: RECODE: INTO is required with string input values and numeric output values.
- 17 | RECODE s1('1'=1).
- | ^~~~~~~"
-
-"recode.sps:18.23-18.24: error: RECODE: Type mismatch: cannot store string data in numeric variable n2.
- 18 | RECODE n1(1='1') INTO n2.
- | ^~"
-
-"recode.sps:19.25-19.26: error: RECODE: Type mismatch: cannot store numeric data in string variable s2.
- 19 | RECODE s1(CONVERT) INTO s2.
- | ^~"
-
-recode.sps:20: error: RECODE: At least one target variable is too narrow for the output values.
-
-"recode.sps:20.19-20.26: note: RECODE: This recoding output value has width 6.
- 20 | RECODE n1 TO n4(1='123456') INTO s1 TO s4.
- | ^~~~~~~~"
-
-"recode.sps:20.29-20.41: note: RECODE: Target variable s1 only has width 1.
- 20 | RECODE n1 TO n4(1='123456') INTO s1 TO s4.
- | ^~~~~~~~~~~~~"
-])
-AT_CLEANUP
\ No newline at end of file
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([SAMPLE])
-
-AT_SETUP([SAMPLE])
-AT_DATA([sample.sps], [dnl
-set seed=3
-
-data list notable /A 1-2.
-begin data.
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-end data.
-sample .5.
-list.
-])
-AT_CHECK([pspp -o pspp.csv sample.sps])
-AT_CAPTURE_FILE([pspp.csv])
-AT_CHECK(
- [n=0
- while read line; do
- n=`expr $n + 1`
- line=$(echo "$line" | tr -d '\r')
- case $line in # (
- "Table: Data List" | A | [[0-9]] | 10) ;; # (
- *) echo $line; exit 1;
- esac
- done < pspp.csv
- if test $n -ge 11; then exit 1; fi
- ])
-AT_CLEANUP
+++ /dev/null
-dnl PSPP - a program for statistical analysis.
-dnl Copyright (C) 2017 Free Software Foundation, Inc.
-dnl
-dnl This program is free software: you can redistribute it and/or modify
-dnl it under the terms of the GNU General Public License as published by
-dnl the Free Software Foundation, either version 3 of the License, or
-dnl (at your option) any later version.
-dnl
-dnl This program is distributed in the hope that it will be useful,
-dnl but WITHOUT ANY WARRANTY; without even the implied warranty of
-dnl MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-dnl GNU General Public License for more details.
-dnl
-dnl You should have received a copy of the GNU General Public License
-dnl along with this program. If not, see <http://www.gnu.org/licenses/>.
-dnl
-AT_BANNER([FILTER])
-
-AT_SETUP([FILTER])
-AT_DATA([filter.sps], [dnl
-data list notable /X 1-2.
-begin data.
-1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-end data.
-compute FILTER_$ = mod(x,2).
-
-filter by filter_$.
-list.
-filter off.
-list.
-compute filter_$ = 1 - filter_$.
-filter by filter_$.
-list.
-])
-AT_CHECK([pspp -o pspp.csv filter.sps])
-AT_CHECK([cat pspp.csv], [0], [dnl
-Table: Data List
-X,FILTER_$
-1,1.00
-3,1.00
-5,1.00
-7,1.00
-9,1.00
-
-Table: Data List
-X,FILTER_$
-1,1.00
-2,.00
-3,1.00
-4,.00
-5,1.00
-6,.00
-7,1.00
-8,.00
-9,1.00
-10,.00
-
-Table: Data List
-X,FILTER_$
-2,1.00
-4,1.00
-6,1.00
-8,1.00
-10,1.00
-])
-AT_CLEANUP
projects / pspp / commitdiff