From: Ben Pfaff <blp@gnu.org>
Date: Thu, 4 May 2006 06:19:02 +0000 (+0000)
Subject: Continue reforming procedure execution.  In this phase, get rid of
X-Git-Tag: v0.6.0~905
X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?p=pspp-builds.git;a=commitdiff_plain;h=b0bf9b1b0f727fafac4296a048e3f45db5936f81

Continue reforming procedure execution.  In this phase, get rid of
many global variables, consolidating procedure execution in
procedure.c.  Encapsulate transformations in new "struct trns_chain".
Also, change implementation of N OF CASES, FILTER, and PROCESS IF from
special cases to transformations.
---

diff --git a/TODO b/TODO
index 8d1c99f8..13f839f0 100644
--- a/TODO
+++ b/TODO
@@ -1,4 +1,18 @@
-Time-stamp: <2006-04-26 15:15:36 blp>
+Time-stamp: <2006-05-03 22:28:57 blp>
+
+Procedure processing:
+
+* Should not need temporary casefile in the common case.
+
+* All of the procedure_*() variants can (and should) be implemented in terms of
+  a variant that provides "proc_func" plus an "end_func" called after all
+  processing is complete.
+
+* The "split" variants should not dump the splits to the output file
+  automatically.  There is no need for the procedure code to talk to the output
+  manager.
+
+* LAG need not be as much of a special case.
 
 Get rid of need for GNU diff in `make check'.
 
diff --git a/src/ChangeLog b/src/ChangeLog
index 604fb254..0fded64c 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,70 @@
+Wed May  3 22:24:34 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* procedure.c: (global var vfm_source) Make static.  Changed
+	external references to use in_input_program(), proc_set_source(),
+	or proc_capture_output() instead.
+	(global var vfm_sink) Make static.  Changed external references to
+	use proc_set_sink() instead.
+	(global var default_dict) Move here from data/dictionary.c.
+	(static var permanent_trns_chain) New var.
+	(static var temp_dict) Renamed permanent_dict, updated references.
+	(static var temporary_trns_chain) New var.
+	(static var cur_trns_chain) New var.
+	(static var in_procedure) Removed.
+	(global var t_trns) Removed.
+	(global var n_trns) Removed.
+	(global var m_trns) Removed.
+	(global var f_trns) Removed.
+	(procedure) Even if there's "nothing to do" we need to clear
+	PROCESS IF, N OF CASES, vector state.  (This should be
+	abstracted.)
+	(multipass_callback) New function.
+	(multipass_procedure) New function.
+	(open_active_file) Add N OF CASES, FILTER, PROCESS IF
+	transformations.  Finalize transformations.  No need to call
+	ctl_stack_clear() anymore because finalizers will do that.
+	(write_case) Simplify and rewrite.
+	(execute_transformations) Removed.
+	(filter_case) Removed.
+	(close_active_file) Use proc_cancel_temporary_transformations().
+	No need to clear PROCESS IF, N OF CASES here anymore because
+	helpers do that.
+	(multipass_procedure_with_splits) Keep track of success.
+	(multipass_split_callback) Ditto.
+	(multipass_split_output) Ditto.
+	(discard_variables) No need to call ctl_stack_clear() anymore
+	because finalizers will do that.
+	(proc_capture_transformations) New function.
+	(add_transformation) Rewrite in terms of trns_chain_append().
+	(add_transformation_with_finalizer) New function.
+	(next_transformation) Rewrite in terms of trns_chain_next().
+	(proc_in_temporary_transformations) New function.
+	(proc_start_temporary_transformations) New function.
+	(proc_make_temporary_transformations_permanent) New function.
+	(proc_cancel_temporary_transformations) New function.
+	(cancel_transformations) Rename proc_cancel_all_transformations(),
+	rewrite in terms of trns_chain_destroy().
+	(proc_init) New function.
+	(proc_done) New function.
+	(proc_set_sink) New function.
+	(proc_set_source) New function.
+	(proc_has_source) New function.
+	(proc_capture_output) New function.
+	(add_case_limit_trns) New function.
+	(case_limit_trns_proc) New function.
+	(case_limit_trns_free) New function.
+	(add_filter_trns) New function.
+	(filter_trns_proc) New function.
+	(add_process_if_trns) New function.
+	(process_if_trns_proc) New function.
+	(process_if_trns_free) New function.
+
 Wed Apr 26 20:00:00 2006  Ben Pfaff  <blp@gnu.org>
 
 	* procedure.c (create_trns_case): Fix inverted decision on whether
diff --git a/src/data/ChangeLog b/src/data/ChangeLog
index 63bdd402..49c1af84 100644
--- a/src/data/ChangeLog
+++ b/src/data/ChangeLog
@@ -1,3 +1,20 @@
+Wed May  3 22:42:12 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* automake.mk: (src_data_libdata_a_SOURCES) Add transformations.c,
+	transformations.h.
+
+	* dictionary.c: (global variable default_dict) Move to
+	src/procedure.c.
+
+	* variable.h: (TRNS_*) Move to transformations.h.
+	(struct transformation) Move to transformations.c.
+
 Thu May  4 13:47:06 WST 2006 John Darrington <john@darrington.wattle.id.au>
 
 	* sys-file-reader.c: Fixed invalid read problems.
diff --git a/src/data/automake.mk b/src/data/automake.mk
index 459ea2fa..2084e4f6 100644
--- a/src/data/automake.mk
+++ b/src/data/automake.mk
@@ -57,6 +57,8 @@ src_data_libdata_a_SOURCES = \
 	src/data/sys-file-reader.h \
 	src/data/sys-file-writer.c \
 	src/data/sys-file-writer.h \
+	src/data/transformations.c \
+	src/data/transformations.h \
 	src/data/value.h \
 	src/data/value-labels.c \
 	src/data/value-labels.h \
diff --git a/src/data/dictionary.c b/src/data/dictionary.c
index c8840f30..e9a5a014 100644
--- a/src/data/dictionary.c
+++ b/src/data/dictionary.c
@@ -59,9 +59,6 @@ struct dictionary
     size_t vector_cnt;          /* Number of vectors. */
   };
 
-/* Active file dictionary. */
-struct dictionary *default_dict;
-
 /* Creates and returns a new dictionary. */
 struct dictionary *
 dict_create (void) 
diff --git a/src/data/transformations.c b/src/data/transformations.c
new file mode 100644
index 00000000..e5542b88
--- /dev/null
+++ b/src/data/transformations.c
@@ -0,0 +1,211 @@
+/* PSPP - computes sample statistics.
+   Copyright (C) 1997-9, 2000, 2006 Free Software Foundation, Inc.
+   Written by Ben Pfaff <blp@gnu.org>.
+
+   This program is free software; you can redistribute it and/or
+   modify it under the terms of the GNU General Public License as
+   published by the Free Software Foundation; either version 2 of the
+   License, or (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software
+   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+   02110-1301, USA. */
+
+#include <config.h>
+
+#include <data/transformations.h>
+
+#include <assert.h>
+#include <stdlib.h>
+
+#include <libpspp/str.h>
+#include <procedure.h>
+
+#include "xalloc.h"
+
+/* A single transformation. */
+struct transformation
+  {
+    /* Offset to add to EXECUTE's return value, if it returns a
+       transformation index.  Normally 0 but set to the starting
+       index of a spliced chain after splicing. */
+    int idx_ofs;                        
+    trns_finalize_func *finalize;       /* Finalize proc. */
+    trns_proc_func *execute;            /* Executes the transformation. */
+    trns_free_func *free;               /* Garbage collector proc. */
+    void *aux;                          /* Auxiliary data. */
+  };
+
+/* A chain of transformations. */
+struct trns_chain 
+  {
+    struct transformation *trns;        /* Array of transformations. */
+    size_t trns_cnt;                    /* Number of transformations. */
+    size_t trns_cap;                    /* Allocated capacity. */
+    bool finalized;                     /* Finalize functions called? */
+  };
+
+/* Allocates and returns a new transformation chain. */
+struct trns_chain *
+trns_chain_create (void) 
+{
+  struct trns_chain *chain = xmalloc (sizeof *chain);
+  chain->trns = NULL;
+  chain->trns_cnt = 0;
+  chain->trns_cap = 0;
+  chain->finalized = false;
+  return chain;
+}
+
+/* Finalizes all the transformations in CHAIN.
+   A chain is only finalized once; afterward, calling this
+   function is a no-op.
+   Finalizers may add transformations to CHAIN, but after
+   finalization the chain's contents are fixed, so that no more
+   transformations may be added afterward. */
+void
+trns_chain_finalize (struct trns_chain *chain) 
+{
+  if (!chain->finalized) 
+    {
+      size_t i;
+
+      for (i = 0; i < chain->trns_cnt; i++) 
+        {
+          struct transformation *trns = &chain->trns[i];
+          if (trns->finalize != NULL)
+            trns->finalize (trns->aux); 
+        }
+      chain->finalized = true;
+    }
+}
+
+/* Destroys CHAIN, finalizing it in the process if it has not
+   already been finalized. */
+bool
+trns_chain_destroy (struct trns_chain *chain) 
+{
+  bool ok = true;
+
+  if (chain != NULL) 
+    {
+      size_t i;
+      
+      /* Needed to ensure that the control stack gets cleared. */
+      trns_chain_finalize (chain);
+      
+      for (i = 0; i < chain->trns_cnt; i++) 
+        {
+          struct transformation *trns = &chain->trns[i];
+          if (trns->free != NULL) 
+            ok = trns->free (trns->aux) && ok;
+        }
+      free (chain);
+    }
+  
+  return ok;
+}
+
+/* Returns true if CHAIN contains any transformations,
+   false otherwise. */
+bool
+trns_chain_is_empty (const struct trns_chain *chain) 
+{
+  return chain->trns_cnt == 0;
+}
+
+/* Adds a transformation to CHAIN with finalize function
+   FINALIZE, execute function EXECUTE, free function FREE, and
+   auxiliary data AUX. */
+void
+trns_chain_append (struct trns_chain *chain, trns_finalize_func *finalize,
+                   trns_proc_func *execute, trns_free_func *free,
+                   void *aux) 
+{
+  struct transformation *trns;
+
+  assert (!chain->finalized);
+
+  if (chain->trns_cnt == chain->trns_cap)
+    chain->trns = x2nrealloc (chain->trns, &chain->trns_cap,
+                              sizeof *chain->trns);
+
+  trns = &chain->trns[chain->trns_cnt++];
+  trns->idx_ofs = 0;
+  trns->finalize = finalize;
+  trns->execute = execute;
+  trns->free = free;
+  trns->aux = aux;
+}
+
+/* Appends the transformations in SRC to those in DST,
+   and destroys SRC.
+   Both DST and SRC must already be finalized. */
+void
+trns_chain_splice (struct trns_chain *dst, struct trns_chain *src) 
+{
+  size_t i;
+  
+  assert (dst->finalized);
+  assert (src->finalized);
+
+  if (dst->trns_cnt + src->trns_cnt > dst->trns_cap) 
+    {
+      dst->trns_cap = dst->trns_cnt + src->trns_cnt;
+      dst->trns = xnrealloc (dst->trns, dst->trns_cap, sizeof *dst->trns);
+    }
+
+  for (i = 0; i < src->trns_cnt; i++) 
+    {
+      struct transformation *d = &dst->trns[i + dst->trns_cnt];
+      const struct transformation *s = &src->trns[i];
+      *d = *s;
+      d->idx_ofs += src->trns_cnt; 
+    }
+  dst->trns_cnt += src->trns_cnt;
+
+  trns_chain_destroy (src);
+}
+
+/* Returns the index that a transformation execution function may
+   return to "jump" to the next transformation to be added. */
+size_t
+trns_chain_next (struct trns_chain *chain) 
+{
+  return chain->trns_cnt;
+}
+
+/* Executes the given CHAIN of transformations on C,
+   passing *CASE_NR as the case number.
+   If a transformation modifies *CASE_NR, it will affect the case
+   number passed to following transformations.
+   Returns the result code that caused the transformations to
+   terminate, or TRNS_CONTINUE if the transformations finished
+   due to "falling off the end" of the set of transformations. */
+enum trns_result
+trns_chain_execute (struct trns_chain *chain, struct ccase *c,
+                    const size_t *case_nr) 
+{
+  size_t i;
+
+  assert (chain->finalized);
+  for (i = 0; i < chain->trns_cnt; )
+    {
+      struct transformation *trns = &chain->trns[i];
+      int retval = trns->execute (trns->aux, c, *case_nr);
+      if (retval == TRNS_CONTINUE)
+        i++;
+      else if (retval >= 0)
+        i = retval + trns->idx_ofs;
+      else
+        return retval; 
+    }
+
+  return TRNS_CONTINUE;
+}
diff --git a/src/data/transformations.h b/src/data/transformations.h
new file mode 100644
index 00000000..196f65dc
--- /dev/null
+++ b/src/data/transformations.h
@@ -0,0 +1,57 @@
+/* PSPP - computes sample statistics.
+   Copyright (C) 1997-9, 2000, 2006 Free Software Foundation, Inc.
+   Written by Ben Pfaff <blp@gnu.org>.
+
+   This program is free software; you can redistribute it and/or
+   modify it under the terms of the GNU General Public License as
+   published by the Free Software Foundation; either version 2 of the
+   License, or (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, write to the Free Software
+   Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+   02110-1301, USA. */
+
+#ifndef TRANSFORMATIONS_H
+#define TRANSFORMATIONS_H 1
+
+#include <stdbool.h>
+#include <stddef.h>
+
+/* trns_proc_func return values. */
+enum trns_result 
+  {
+    TRNS_CONTINUE = -1,         /* Continue to next transformation. */
+    TRNS_DROP_CASE = -2,        /* Drop this case. */
+    TRNS_ERROR = -3,            /* A serious error, so stop the procedure. */
+    TRNS_NEXT_CASE = -4,        /* Skip to next case.  INPUT PROGRAM only. */
+    TRNS_END_FILE = -5          /* End of input.  INPUT PROGRAM only. */
+  };
+
+struct ccase;
+typedef void trns_finalize_func (void *);
+typedef int trns_proc_func (void *, struct ccase *, int);
+typedef bool trns_free_func (void *);
+
+/* Transformation chains. */
+
+struct trns_chain *trns_chain_create (void);
+void trns_chain_finalize (struct trns_chain *);
+bool trns_chain_destroy (struct trns_chain *);
+
+bool trns_chain_is_empty (const struct trns_chain *);
+
+void trns_chain_append (struct trns_chain *, trns_finalize_func *,
+                        trns_proc_func *, trns_free_func *, void *);
+size_t trns_chain_next (struct trns_chain *);
+enum trns_result trns_chain_execute (struct trns_chain *, struct ccase *,
+                                     const size_t *case_nr);
+
+void trns_chain_splice (struct trns_chain *, struct trns_chain *);
+
+#endif /* transformations.h */
diff --git a/src/data/variable.h b/src/data/variable.h
index 2dfe5648..0f5bb26d 100644
--- a/src/data/variable.h
+++ b/src/data/variable.h
@@ -28,8 +28,6 @@
 #include "format.h"
 #include "missing-values.h"
 
-/* Script variables. */
-
 /* Variable type. */
 enum var_type
   {
@@ -128,71 +126,15 @@ struct vector
     int cnt;			/* Number of variables. */
   };
 
-void discard_variables (void);
-
-/* This is the active file dictionary. */
-extern struct dictionary *default_dict;
-
-/* Transformation state. */
-
 /* PROCESS IF expression. */
 extern struct expression *process_if_expr;
 
-/* TEMPORARY support. */
-
-/* 1=TEMPORARY has been executed at some point. */
-extern int temporary;
-
-/* If temporary!=0, the saved dictionary. */
-extern struct dictionary *temp_dict;
-
-/* If temporary!=0, index into t_trns[] (declared far below) that
-   gives the point at which data should be written out.  -1 means that
-   the data shouldn't be changed since all transformations are
-   temporary. */
-extern size_t temp_trns;
-
-void cancel_temporary (void);
-
 struct ccase;
 void dump_split_vars (const struct ccase *);
 
-/* Transformations. */
-
-/* trns_proc_func return values. */
-#define TRNS_CONTINUE   -1 /* Continue to next transformation. */
-#define TRNS_DROP_CASE  -2 /* Drop this case. */
-#define TRNS_ERROR      -3 /* A serious error, so stop the procedure. */
-#define TRNS_NEXT_CASE  -4 /* Skip to next case.  INPUT PROGRAM only. */
-#define TRNS_END_FILE   -5 /* End of input.  INPUT PROGRAM only. */
-
-typedef int trns_proc_func (void *, struct ccase *, int);
-typedef bool trns_free_func (void *);
-
-/* A transformation. */
-struct transformation
-  {
-    trns_proc_func *proc;       /* Transformation proc. */
-    trns_free_func *free;       /* Garbage collector proc. */
-    void *private;              /* Private data. */
-  };
-
-/* Array of transformations */
-extern struct transformation *t_trns;
-
-/* Number of transformations, maximum number in array currently. */
-extern size_t n_trns, m_trns;
-
-/* Index of first transformation that is really a transformation.  Any
-   transformations before this belong to INPUT PROGRAM. */
-extern size_t f_trns;
-
-void add_transformation (trns_proc_func *, trns_free_func *, void *);
-size_t next_transformation (void);
-bool cancel_transformations (void);
-
 struct var_set;
 
+struct dictionary;
 struct var_set *var_set_create_from_dict (const struct dictionary *d);
 struct var_set *var_set_create_from_array (struct variable *const *var,
                                            size_t);
diff --git a/src/language/control/ChangeLog b/src/language/control/ChangeLog
index 4a96aeb9..82404832 100644
--- a/src/language/control/ChangeLog
+++ b/src/language/control/ChangeLog
@@ -1,3 +1,30 @@
+Wed May  3 22:45:41 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* do-if.c: (cmd_do_if) Use finalizer to ensure control stack gets
+	cleared.
+	(do_if_finalize_func) New function.
+
+	* loop.c: (create_loop_trns) Use finalizer to ensure control stack gets
+	cleared.
+	(loop_trns_finalize) New function.
+
+	* temporary.c: (global var temporary) Removed.  Changed references
+	to use proc_make_temporary_transformations_permanent() or
+	proc_in_temporary_transformations().
+	(global var temp_dict) Removed.
+	(global var temp_trns) Removed.
+	(cmd_temporary) Reimplement in terms of
+	proc_in_temporary_transformations() and
+	proc_start_temporary_transformations().
+	(cancel_temporary) Moved to procedure.c, renamed
+	proc_cancel_temporary_transformations().
+		
 Thu Mar  2 08:40:33 WST 2006 John Darrington <john@darrington.wattle.id.au>
 	
 	* Moved files from src directory
diff --git a/src/language/control/do-if.c b/src/language/control/do-if.c
index c16fca92..0c0ead94 100644
--- a/src/language/control/do-if.c
+++ b/src/language/control/do-if.c
@@ -22,15 +22,17 @@
 #include <stdlib.h>
 
 #include "control-stack.h"
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/compiler.h>
-#include <libpspp/message.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
@@ -89,6 +91,7 @@ static bool has_else (struct do_if_trns *);
 static bool must_not_have_else (struct do_if_trns *);
 static void close_do_if (void *do_if);
 
+static trns_finalize_func do_if_finalize_func;
 static trns_proc_func do_if_trns_proc, break_trns_proc;
 static trns_free_func do_if_trns_free;
 
@@ -101,7 +104,8 @@ cmd_do_if (void)
   do_if->clause_cnt = 0;
 
   ctl_stack_push (&do_if_class, do_if);
-  add_transformation (do_if_trns_proc, do_if_trns_free, do_if);
+  add_transformation_with_finalizer (do_if_finalize_func,
+                                     do_if_trns_proc, do_if_trns_free, do_if);
 
   return parse_clause (do_if);
 }
@@ -219,6 +223,17 @@ add_clause (struct do_if_trns *do_if,
   clause->target_index = target_index;
 }
 
+/* Finalizes DO IF by clearing the control stack, thus ensuring
+   that all open DO IFs are closed. */ 
+static void
+do_if_finalize_func (void *do_if_ UNUSED) 
+{
+  /* This will be called multiple times if multiple DO IFs were
+     executed, which is slightly unclean, but at least it's
+     idempotent. */
+  ctl_stack_clear ();
+}
+
 /* DO IF transformation procedure.
    Checks each clause and jumps to the appropriate
    transformation. */
diff --git a/src/language/control/loop.c b/src/language/control/loop.c
index 80a4d214..abbc4467 100644
--- a/src/language/control/loop.c
+++ b/src/language/control/loop.c
@@ -18,21 +18,23 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
+
+#include "control-stack.h"
 #include <data/case.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
 #include <data/dictionary.h>
-#include "control-stack.h"
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/settings.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/pool.h>
-#include <data/settings.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
@@ -79,6 +81,7 @@ struct loop_trns
 
 static struct ctl_class loop_class;
 
+static trns_finalize_func loop_trns_finalize;
 static trns_proc_func loop_trns_proc, end_loop_trns_proc, break_trns_proc;
 static trns_free_func loop_trns_free;
 
@@ -248,7 +251,8 @@ create_loop_trns (void)
   loop->first_expr = loop->by_expr = loop->last_expr = NULL;
   loop->loop_condition = loop->end_loop_condition = NULL;
 
-  add_transformation (loop_trns_proc, loop_trns_free, loop);
+  add_transformation_with_finalizer (loop_trns_finalize,
+                                     loop_trns_proc, loop_trns_free, loop);
   loop->past_LOOP_index = next_transformation ();
 
   ctl_stack_push (&loop_class, loop);
@@ -256,6 +260,17 @@ create_loop_trns (void)
   return loop;
 }
 
+/* Finalizes LOOP by clearing the control stack, thus ensuring
+   that all open LOOPs are closed. */ 
+static void
+loop_trns_finalize (void *do_if_ UNUSED) 
+{
+  /* This will be called multiple times if multiple LOOPs were
+     executed, which is slightly unclean, but at least it's
+     idempotent. */
+  ctl_stack_clear ();
+}
+
 /* Sets up LOOP for the first pass. */
 static int
 loop_trns_proc (void *loop_, struct ccase *c, int case_num)
diff --git a/src/language/control/repeat.c b/src/language/control/repeat.c
index 933c9134..ec37382b 100644
--- a/src/language/control/repeat.c
+++ b/src/language/control/repeat.c
@@ -18,24 +18,29 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include "repeat.h"
-#include <libpspp/message.h>
+
 #include <ctype.h>
 #include <math.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include "intprops.h"
-#include <libpspp/message.h>
-#include <language/line-buffer.h>
+#include <procedure.h>
+#include <data/settings.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <language/line-buffer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/pool.h>
-#include <data/settings.h>
 #include <libpspp/str.h>
 #include <data/variable.h>
 
+#include "intprops.h"
+
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
 
diff --git a/src/language/control/temporary.c b/src/language/control/temporary.c
index c4b2532d..2424c255 100644
--- a/src/language/control/temporary.c
+++ b/src/language/control/temporary.c
@@ -18,66 +18,35 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stddef.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
-#include <data/dictionary.h>
+
 #include "control-stack.h"
-#include <libpspp/message.h>
-#include <libpspp/hash.h>
-#include <language/lexer/lexer.h>
-#include <libpspp/str.h>
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/transformations.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
+#include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
+#include <libpspp/str.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
 
-int temporary;
-struct dictionary *temp_dict;
-size_t temp_trns;
-
 /* Parses the TEMPORARY command. */
 int
 cmd_temporary (void)
 {
-  /* TEMPORARY is not allowed inside DO IF or LOOP. */
-  if (!ctl_stack_is_empty ())
-    {
-      msg (SE, _("This command is not valid inside DO IF or LOOP."));
-      return CMD_FAILURE;
-    }
-
-  /* TEMPORARY can only appear once! */
-  if (temporary)
-    {
-      msg (SE, _("This command may only appear once between "
-	   "procedures and procedure-like commands."));
-      return CMD_FAILURE;
-    }
-
-  /* Make a copy of the current dictionary. */
-  temporary = 1;
-  temp_dict = dict_clone (default_dict);
-  temp_trns = n_trns;
-
+  if (!proc_in_temporary_transformations ())
+    proc_start_temporary_transformations ();
+  else
+    msg (SE, _("This command may only appear once between "
+               "procedures and procedure-like commands."));
   return lex_end_of_command ();
 }
-
-/* Cancels the temporary transformation, if any. */
-void
-cancel_temporary (void)
-{
-  if (temporary)
-    {
-      if (temp_dict) 
-        {
-          dict_destroy (temp_dict);
-          temp_dict = NULL; 
-        }
-      temporary = 0;
-      temp_trns = 0;
-    }
-}
diff --git a/src/language/data-io/ChangeLog b/src/language/data-io/ChangeLog
index ee9949b2..ba70c3f3 100644
--- a/src/language/data-io/ChangeLog
+++ b/src/language/data-io/ChangeLog
@@ -1,3 +1,23 @@
+Wed May  3 23:00:17 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* data-list.c: (data_list_trns_proc) Return TRNS_END_FILE at end
+	of file.  (Why didn't we do this before?)
+	(cmd_match_files) Direct procedure output to null sink.
+	Use discard_variables() instead of indirect version.
+
+	* inpt-pgm.c: Use transformation chain.
+	(struct input_program_pgm) Add trns_chain member.
+	(cmd_input_program) Initialize trns_chain member and capture
+	transformations with proc_capture_transformations().
+	(input_program_source_read) Use trns_chain_execute().
+	(destroy_input_program) Destroy input chain.
+
 Tue May  2 10:39:56 WST 2006 John Darrington <john@darrington.wattle.id.au>
 
 	* list.q Changed from using fixed length char buffers to struct 
diff --git a/src/language/data-io/data-list.c b/src/language/data-io/data-list.c
index 16d29c52..a778b5d8 100644
--- a/src/language/data-io/data-list.c
+++ b/src/language/data-io/data-list.c
@@ -26,10 +26,12 @@
 
 #include <data/case-source.h>
 #include <data/case.h>
+#include <data/case-source.h>
 #include <data/data-in.h>
 #include <data/dictionary.h>
 #include <data/format.h>
 #include <data/settings.h>
+#include <data/transformations.h>
 #include <data/variable.h>
 #include <language/command.h>
 #include <language/data-io/data-reader.h>
@@ -101,6 +103,7 @@ static void dump_fixed_table (const struct dls_var_spec *,
 static void dump_free_table (const struct data_list_pgm *,
                              const struct file_handle *);
 static void destroy_dls_var_spec (struct dls_var_spec *);
+
 static trns_free_func data_list_trns_free;
 static trns_proc_func data_list_trns_proc;
 
@@ -255,10 +258,10 @@ cmd_data_list (void)
   if (dls->reader == NULL)
     goto error;
 
-  if (vfm_source != NULL)
+  if (in_input_program ())
     add_transformation (data_list_trns_proc, data_list_trns_free, dls);
   else 
-    vfm_source = create_case_source (&data_list_source_class, dls);
+    proc_set_source (create_case_source (&data_list_source_class, dls));
 
   return CMD_SUCCESS;
 
@@ -534,7 +537,7 @@ fixed_parse_compatible (struct fixed_parsing_state *fx,
       else
 	{
 	  v = dict_lookup_var_assert (default_dict, fx->name[i]);
-	  if (vfm_source == NULL)
+	  if (!in_input_program ())
 	    {
 	      msg (SE, _("%s is a duplicate variable name."), fx->name[i]);
 	      return 0;
@@ -1228,7 +1231,7 @@ data_list_trns_proc (void *dls_, struct ccase *c, int case_num UNUSED)
       retval = TRNS_ERROR;
     }
   else
-    retval = TRNS_DROP_CASE;
+    retval = TRNS_END_FILE;
   
   /* If there was an END subcommand handle it. */
   if (dls->end != NULL) 
@@ -1237,7 +1240,7 @@ data_list_trns_proc (void *dls_, struct ccase *c, int case_num UNUSED)
       if (retval == TRNS_DROP_CASE)
         {
           *end = 1.0;
-          retval = TRNS_CONTINUE;
+          retval = TRNS_END_FILE;
         }
       else
         *end = 0.0;
diff --git a/src/language/data-io/get.c b/src/language/data-io/get.c
index b73a2bda..b9634b19 100644
--- a/src/language/data-io/get.c
+++ b/src/language/data-io/get.c
@@ -31,6 +31,7 @@
 #include <data/settings.h>
 #include <data/storage-stream.h>
 #include <data/sys-file-writer.h>
+#include <data/transformations.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
 #include <language/command.h>
@@ -147,7 +148,7 @@ parse_read_command (enum reader_command type)
   dict_destroy (default_dict);
   default_dict = dict;
 
-  vfm_source = create_case_source (&case_reader_source_class, pgm);
+  proc_set_source (create_case_source (&case_reader_source_class, pgm));
 
   return CMD_SUCCESS;
 
@@ -887,21 +888,18 @@ cmd_match_files (void)
             }
           used_active_file = true;
 
-          if (vfm_source == NULL)
+          if (!proc_has_source ())
             {
               msg (SE, _("Cannot specify the active file since no active "
                          "file has been defined."));
               goto error;
             }
 
-          if (temporary != 0)
-            {
-              msg (SE,
-                   _("MATCH FILES may not be used after TEMPORARY when "
-                     "the active file is an input source.  "
-                     "Temporary transformations will be made permanent."));
-              cancel_temporary (); 
-            }
+          if (proc_make_temporary_transformations_permanent ())
+            msg (SE,
+                 _("MATCH FILES may not be used after TEMPORARY when "
+                   "the active file is an input source.  "
+                   "Temporary transformations will be made permanent."));
 
           file->dict = default_dict;
         }
@@ -1134,17 +1132,18 @@ cmd_match_files (void)
     goto error;
 
   if (used_active_file) 
-    ok = procedure (mtf_processing, &mtf) && mtf_processing_finish (&mtf);
+    {
+      proc_set_sink (create_case_sink (&null_sink_class, default_dict, NULL));
+      ok = procedure (mtf_processing, &mtf) && mtf_processing_finish (&mtf); 
+    }
   else
     ok = mtf_processing_finish (&mtf);
 
-  free_case_source (vfm_source);
-  vfm_source = NULL;
+  discard_variables ();
 
-  dict_destroy (default_dict);
   default_dict = mtf.dict;
   mtf.dict = NULL;
-  vfm_source = mtf.sink->class->make_source (mtf.sink);
+  proc_set_source (mtf.sink->class->make_source (mtf.sink));
   free_case_sink (mtf.sink);
   
   if (!mtf_free (&mtf))
diff --git a/src/language/data-io/inpt-pgm.c b/src/language/data-io/inpt-pgm.c
index 0581249a..fbf1d342 100644
--- a/src/language/data-io/inpt-pgm.c
+++ b/src/language/data-io/inpt-pgm.c
@@ -28,6 +28,7 @@
 #include <data/case.h>
 #include <data/case-source.h>
 #include <data/dictionary.h>
+#include <data/transformations.h>
 #include <data/variable.h>
 #include <language/command.h>
 #include <language/data-io/data-reader.h>
@@ -64,6 +65,8 @@ enum value_init_type
 
 struct input_program_pgm 
   {
+    struct trns_chain *trns_chain;
+
     size_t case_nr;             /* Incremented by END CASE transformation. */
     write_case_func *write_case;/* Called by END CASE. */
     write_case_data wc_data;    /* Aux data used by END CASE. */
@@ -110,6 +113,7 @@ cmd_input_program (void)
     return lex_end_of_command ();
 
   inp = xmalloc (sizeof *inp);
+  inp->trns_chain = NULL;
   inp->init = NULL;
   
   inside_input_program = true;
@@ -147,9 +151,8 @@ cmd_input_program (void)
       return CMD_FAILURE;
     }
   
-  /* Mark the boundary between INPUT PROGRAM transformations and
-     ordinary transformations. */
-  f_trns = n_trns;
+  inp->trns_chain = proc_capture_transformations ();
+  trns_chain_finalize (inp->trns_chain);
 
   /* Figure out how to initialize each input case. */
   inp->init_cnt = dict_get_next_value_idx (default_dict);
@@ -172,8 +175,7 @@ cmd_input_program (void)
     assert (inp->init[i] != -1);
   inp->case_size = dict_get_case_size (default_dict);
 
-  /* Create vfm_source. */
-  vfm_source = create_case_source (&input_program_source_class, inp);
+  proc_set_source (create_case_source (&input_program_source_class, inp));
 
   return CMD_SUCCESS;
 }
@@ -248,38 +250,12 @@ input_program_source_read (struct case_source *source,
   inp->wc_data = wc_data;
   for (init_case (inp, c); ; clear_case (inp, c))
     {
-      int i;
-      
-      /* Perform transformations on `blank' case. */
-      for (i = 0; i < f_trns; )
-	{
-          int code;
-
-	  code = t_trns[i].proc (t_trns[i].private, c, inp->case_nr);
-	  switch (code)
-	    {
-	    case TRNS_CONTINUE:
-	      i++;
-	      break;
-
-            case TRNS_DROP_CASE:
-              break;
-
-            case TRNS_ERROR:
-              return false;
-
-	    case TRNS_NEXT_CASE:
-	      goto next_case;
-
-	    case TRNS_END_FILE:
-              return true;
-
-	    default:
-	      i = code;
-	      break;
-	    }
-	}
-    next_case: ;
+      enum trns_result result = trns_chain_execute (inp->trns_chain, c,
+                                                    &inp->case_nr);
+      if (result == TRNS_ERROR)
+        return false;
+      else if (result == TRNS_END_FILE)
+        return true;
     }
 }
 
@@ -288,6 +264,7 @@ destroy_input_program (struct input_program_pgm *pgm)
 {
   if (pgm != NULL) 
     {
+      trns_chain_destroy (pgm->trns_chain);
       free (pgm->init);
       free (pgm);
     }
diff --git a/src/language/data-io/matrix-data.c b/src/language/data-io/matrix-data.c
index eb2d226f..ff008caa 100644
--- a/src/language/data-io/matrix-data.c
+++ b/src/language/data-io/matrix-data.c
@@ -972,7 +972,8 @@ read_matrices_without_rowtype (struct matrix_data_pgm *mx)
   nr.split_values = xnmalloc (dict_get_split_cnt (default_dict),
                               sizeof *nr.split_values);
 
-  vfm_source = create_case_source (&matrix_data_without_rowtype_source_class, &nr);
+  proc_set_source (create_case_source (
+                     &matrix_data_without_rowtype_source_class, &nr));
   
   ok = procedure (NULL, NULL);
 
@@ -1512,8 +1513,8 @@ read_matrices_with_rowtype (struct matrix_data_pgm *mx)
   wr.current = NULL;
   mx->cells = 0;
 
-  vfm_source = create_case_source (&matrix_data_with_rowtype_source_class,
-                                   &wr);
+  proc_set_source (create_case_source (&matrix_data_with_rowtype_source_class,
+                                       &wr));
   ok = procedure (NULL, NULL);
 
   free (wr.split_values);
diff --git a/src/language/data-io/print.c b/src/language/data-io/print.c
index 2479b7c1..f6633652 100644
--- a/src/language/data-io/print.c
+++ b/src/language/data-io/print.c
@@ -20,21 +20,25 @@
 /* FIXME: seems like a lot of code duplication with data-list.c. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/compiler.h>
 #include <language/data-io/data-writer.h>
-#include <libpspp/message.h>
-#include <language/expressions/public.h>
 #include <language/data-io/file-handle.h>
+#include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <output/manager.h>
 #include <output/table.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/apply-dictionary.c b/src/language/dictionary/apply-dictionary.c
index 217f0f89..87b71eee 100644
--- a/src/language/dictionary/apply-dictionary.c
+++ b/src/language/dictionary/apply-dictionary.c
@@ -18,18 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
+
 #include <data/any-reader.h>
-#include <language/command.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
-#include <language/data-io/file-handle.h>
 #include <data/file-handle-def.h>
-#include <libpspp/hash.h>
-#include <language/lexer/lexer.h>
-#include <libpspp/str.h>
+#include <procedure.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
+#include <language/command.h>
+#include <language/data-io/file-handle.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
+#include <libpspp/str.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/formats.c b/src/language/dictionary/formats.c
index c325b408..2d5e1c35 100644
--- a/src/language/dictionary/formats.c
+++ b/src/language/dictionary/formats.c
@@ -18,15 +18,18 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <limits.h>
 #include <stdio.h>
 #include <stdlib.h>
+
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/missing-values.c b/src/language/dictionary/missing-values.c
index 6b25cb07..0757fb1d 100644
--- a/src/language/dictionary/missing-values.c
+++ b/src/language/dictionary/missing-values.c
@@ -18,16 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <language/command.h>
+
 #include <data/data-in.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/magic.h>
 #include <language/lexer/range-parser.h>
+#include <libpspp/magic.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/modify-variables.c b/src/language/dictionary/modify-variables.c
index 7b8d5681..ad6a48e4 100644
--- a/src/language/dictionary/modify-variables.c
+++ b/src/language/dictionary/modify-variables.c
@@ -88,12 +88,9 @@ cmd_modify_vars (void)
 
   size_t i;
 
-  if (temporary != 0)
-    {
-      msg (SE, _("MODIFY VARS may not be used after TEMPORARY.  "
-                 "Temporary transformations will be made permanent."));
-      cancel_temporary (); 
-    }
+  if (proc_make_temporary_transformations_permanent ())
+    msg (SE, _("MODIFY VARS may not be used after TEMPORARY.  "
+               "Temporary transformations will be made permanent."));
 
   vm.reorder_vars = NULL;
   vm.reorder_cnt = 0;
diff --git a/src/language/dictionary/numeric.c b/src/language/dictionary/numeric.c
index 9d94cf3f..f946e74a 100644
--- a/src/language/dictionary/numeric.c
+++ b/src/language/dictionary/numeric.c
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/rename-variables.c b/src/language/dictionary/rename-variables.c
index 9ba66302..eedc84d4 100644
--- a/src/language/dictionary/rename-variables.c
+++ b/src/language/dictionary/rename-variables.c
@@ -18,16 +18,18 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
-#include <libpspp/hash.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
@@ -44,12 +46,9 @@ cmd_rename_variables (void)
 
   int status = CMD_CASCADING_FAILURE;
 
-  if (temporary != 0)
-    {
-      msg (SE, _("RENAME VARS may not be used after TEMPORARY.  "
-                 "Temporary transformations will be made permanent."));
-      cancel_temporary (); 
-    }
+  if (proc_make_temporary_transformations_permanent ())
+    msg (SE, _("RENAME VARS may not be used after TEMPORARY.  "
+               "Temporary transformations will be made permanent."));
 
   do
     {
diff --git a/src/language/dictionary/split-file.c b/src/language/dictionary/split-file.c
index 4239ac33..149523d3 100644
--- a/src/language/dictionary/split-file.c
+++ b/src/language/dictionary/split-file.c
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 int
 cmd_split_file (void)
diff --git a/src/language/dictionary/sys-file-info.c b/src/language/dictionary/sys-file-info.c
index d3414cb2..97bf3df5 100644
--- a/src/language/dictionary/sys-file-info.c
+++ b/src/language/dictionary/sys-file-info.c
@@ -18,26 +18,29 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <ctype.h>
 #include <stdlib.h>
-#include <libpspp/array.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
 #include <data/file-handle-def.h>
+#include <procedure.h>
+#include <data/sys-file-reader.h>
+#include <data/value-labels.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/data-io/file-handle.h>
-#include <libpspp/hash.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/array.h>
+#include <libpspp/hash.h>
 #include <libpspp/magic.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
-#include <output/output.h>
-#include <data/sys-file-reader.h>
 #include <output/manager.h>
+#include <output/output.h>
 #include <output/table.h>
-#include <data/value-labels.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/value-labels.c b/src/language/dictionary/value-labels.c
index 8479fe7f..a9d3d875 100644
--- a/src/language/dictionary/value-labels.c
+++ b/src/language/dictionary/value-labels.c
@@ -18,16 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdio.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
+#include <procedure.h>
+#include <data/value-labels.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
-#include <libpspp/hash.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/value-labels.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/variable-display.c b/src/language/dictionary/variable-display.c
index 2632ce9e..ef43c531 100644
--- a/src/language/dictionary/variable-display.c
+++ b/src/language/dictionary/variable-display.c
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdio.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 /* Set variables' alignment
    This is the alignment for GUI display only.
@@ -42,7 +45,6 @@ cmd_variable_alignment (void)
       size_t i;
       enum alignment align;
 
-
       if (!parse_variables (default_dict, &v, &nv, PV_NONE))
         return CMD_FAILURE;
 
diff --git a/src/language/dictionary/variable-label.c b/src/language/dictionary/variable-label.c
index a663dc9d..3692a6c4 100644
--- a/src/language/dictionary/variable-label.c
+++ b/src/language/dictionary/variable-label.c
@@ -18,14 +18,17 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdio.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/vector.c b/src/language/dictionary/vector.c
index 7553ba2c..90f245c8 100644
--- a/src/language/dictionary/vector.c
+++ b/src/language/dictionary/vector.c
@@ -18,16 +18,18 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
+#include <procedure.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/dictionary/weight.c b/src/language/dictionary/weight.c
index 6528141f..121728b6 100644
--- a/src/language/dictionary/weight.c
+++ b/src/language/dictionary/weight.c
@@ -18,14 +18,16 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdio.h>
-#include <language/command.h>
+
+#include <procedure.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/expressions/parse.c b/src/language/expressions/parse.c
index fdd46082..b6c8076c 100644
--- a/src/language/expressions/parse.c
+++ b/src/language/expressions/parse.c
@@ -1222,7 +1222,7 @@ parse_function (struct expression *e)
       msg (SE, _("%s is not yet implemented."), f->prototype);
       goto fail;
     }
-  if ((f->flags & OPF_PERM_ONLY) && temporary != 0) 
+  if ((f->flags & OPF_PERM_ONLY) && proc_in_temporary_transformations ()) 
     {
       msg (SE, _("%s may not appear after TEMPORARY."), f->prototype);
       goto fail;
diff --git a/src/language/lexer/variable-parser.c b/src/language/lexer/variable-parser.c
index 7037ccbf..6475caf4 100644
--- a/src/language/lexer/variable-parser.c
+++ b/src/language/lexer/variable-parser.c
@@ -18,19 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <data/variable.h>
+
 #include <ctype.h>
 #include <stdbool.h>
 #include <stdlib.h>
+
+#include "lexer.h"
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <libpspp/alloc.h>
 #include <libpspp/bit-vector.h>
-#include <data/dictionary.h>
-#include <libpspp/message.h>
 #include <libpspp/hash.h>
-#include "lexer.h"
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/pool.h>
-#include "size_max.h"
 #include <libpspp/str.h>
 
 #include "gettext.h"
diff --git a/src/language/stats/ChangeLog b/src/language/stats/ChangeLog
index 6f7c271f..46c21651 100644
--- a/src/language/stats/ChangeLog
+++ b/src/language/stats/ChangeLog
@@ -1,3 +1,14 @@
+Wed May  3 23:05:31 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* aggregate.c (cmd_aggregate) Use discard_variables().
+	
+
 2006-04-28  Jason Stover  <jhs@math.gcsu.edu>
 
 	* regression.q (regression_trns_resid_proc): Pass only the
diff --git a/src/language/stats/aggregate.c b/src/language/stats/aggregate.c
index a0887e3b..e51a5a6a 100644
--- a/src/language/stats/aggregate.c
+++ b/src/language/stats/aggregate.c
@@ -27,6 +27,7 @@
 #include <data/casefile.h>
 #include <data/dictionary.h>
 #include <data/file-handle-def.h>
+#include <procedure.h>
 #include <data/settings.h>
 #include <data/storage-stream.h>
 #include <data/sys-file-writer.h>
@@ -260,7 +261,7 @@ cmd_aggregate (void)
     {
       /* The active file will be replaced by the aggregated data,
          so TEMPORARY is moot. */
-      cancel_temporary ();
+      proc_cancel_temporary_transformations ();
 
       if (agr.sort != NULL && !presorted) 
         {
@@ -271,7 +272,7 @@ cmd_aggregate (void)
       agr.sink = create_case_sink (&storage_sink_class, agr.dict, NULL);
       if (agr.sink->class->open != NULL)
         agr.sink->class->open (agr.sink);
-      vfm_sink = create_case_sink (&null_sink_class, default_dict, NULL);
+      proc_set_sink (create_case_sink (&null_sink_class, default_dict, NULL));
       if (!procedure (agr_to_active_file, &agr))
         goto error;
       if (agr.case_cnt > 0) 
@@ -280,10 +281,10 @@ cmd_aggregate (void)
           if (!agr.sink->class->write (agr.sink, &agr.agr_case))
             goto error;
         }
-      dict_destroy (default_dict);
+      discard_variables ();
       default_dict = agr.dict;
       agr.dict = NULL;
-      vfm_source = agr.sink->class->make_source (agr.sink);
+      proc_set_source (agr.sink->class->make_source (agr.sink));
       free_case_sink (agr.sink);
     }
   else
diff --git a/src/language/stats/autorecode.c b/src/language/stats/autorecode.c
index 8b7d9aaa..a4177be8 100644
--- a/src/language/stats/autorecode.c
+++ b/src/language/stats/autorecode.c
@@ -18,19 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
+#include <data/dictionary.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
 #include <libpspp/compiler.h>
-#include <data/dictionary.h>
-#include <libpspp/message.h>
 #include <libpspp/hash.h>
-#include <language/lexer/lexer.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/pool.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 #include <procedure.h>
 
 #include "gettext.h"
diff --git a/src/language/stats/correlations.q b/src/language/stats/correlations.q
index cf230067..68d55635 100644
--- a/src/language/stats/correlations.q
+++ b/src/language/stats/correlations.q
@@ -18,15 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <libpspp/compiler.h>
+
 #include <data/dictionary.h>
 #include <data/file-handle-def.h>
-#include <language/data-io/file-handle.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/data-io/file-handle.h>
 #include <language/lexer/lexer.h>
-#include <data/variable.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+
 /* (headers) */
 
 struct cor_set
diff --git a/src/language/stats/descriptives.c b/src/language/stats/descriptives.c
index 62c0ede0..b7237148 100644
--- a/src/language/stats/descriptives.c
+++ b/src/language/stats/descriptives.c
@@ -20,24 +20,26 @@
 /* FIXME: Many possible optimizations. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <limits.h>
 #include <math.h>
 #include <stdlib.h>
-#include <libpspp/array.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
 #include <data/casefile.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
 #include <data/dictionary.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/message.h>
+#include <libpspp/alloc.h>
+#include <libpspp/array.h>
+#include <libpspp/compiler.h>
 #include <libpspp/magic.h>
+#include <libpspp/message.h>
 #include <math/moments.h>
 #include <output/manager.h>
 #include <output/table.h>
-#include <data/variable.h>
 #include <procedure.h>
 
 #include "gettext.h"
diff --git a/src/language/stats/flip.c b/src/language/stats/flip.c
index f3286107..02c764b5 100644
--- a/src/language/stats/flip.c
+++ b/src/language/stats/flip.c
@@ -32,6 +32,7 @@
 #include <data/case-source.h>
 #include <data/case.h>
 #include <data/dictionary.h>
+#include <procedure.h>
 #include <data/settings.h>
 #include <data/value.h>
 #include <data/variable.h>
@@ -91,14 +92,12 @@ int
 cmd_flip (void)
 {
   struct flip_pgm *flip;
+  struct case_sink *sink;
   bool ok;
 
-  if (temporary != 0)
-    {
-      msg (SW, _("FLIP ignores TEMPORARY.  "
-                 "Temporary transformations will be made permanent."));
-      cancel_temporary (); 
-    }
+  if (proc_make_temporary_transformations_permanent ())
+    msg (SW, _("FLIP ignores TEMPORARY.  "
+               "Temporary transformations will be made permanent."));
 
   flip = pool_create_container (struct flip_pgm, pool);
   flip->var = NULL;
@@ -150,10 +149,11 @@ cmd_flip (void)
 
   /* Read the active file into a flip_sink. */
   flip->case_cnt = 0;
-  temp_trns = temporary = 0;
-  vfm_sink = flip_sink_create (flip);
-  if (vfm_sink == NULL)
+  proc_make_temporary_transformations_permanent ();
+  sink = flip_sink_create (flip);
+  if (sink == NULL)
     goto error;
+  proc_set_sink (sink);
   flip->new_names_tail = NULL;
   ok = procedure (NULL, NULL);
 
@@ -174,7 +174,7 @@ cmd_flip (void)
   flip->case_size = dict_get_case_size (default_dict);
 
   /* Set up flipped data for reading. */
-  vfm_source = flip_source_create (flip);
+  proc_set_source (flip_source_create (flip));
 
   return ok ? lex_end_of_command () : CMD_CASCADING_FAILURE;
 
diff --git a/src/language/stats/means.q b/src/language/stats/means.q
index 96c1d703..7fc176f7 100644
--- a/src/language/stats/means.q
+++ b/src/language/stats/means.q
@@ -18,17 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
 #include <stdio.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
-#include <libpspp/alloc.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/hash.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/message.h>
+#include <libpspp/alloc.h>
+#include <libpspp/hash.h>
 #include <libpspp/magic.h>
-#include <data/variable.h>
+#include <libpspp/message.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/stats/rank.q b/src/language/stats/rank.q
index dcd7dae6..d3ac8edd 100644
--- a/src/language/stats/rank.q
+++ b/src/language/stats/rank.q
@@ -19,14 +19,15 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
 02110-1301, USA. */
 
 #include <config.h>
+
+#include "sort-criteria.h"
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/variable.h>
 #include <language/command.h>
 #include <language/stats/sort-criteria.h>
 #include <libpspp/compiler.h>
-#include <data/dictionary.h>
 #include <math/sort.h>
-#include <data/variable.h>
-
-#include "sort-criteria.h"
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/stats/regression.q b/src/language/stats/regression.q
index 793d8325..b5dabb15 100644
--- a/src/language/stats/regression.q
+++ b/src/language/stats/regression.q
@@ -18,33 +18,37 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <stdlib.h>
+
 #include <gsl/gsl_cdf.h>
-#include <gsl/gsl_vector.h>
 #include <gsl/gsl_matrix.h>
+#include <gsl/gsl_vector.h>
 #include <math.h>
-#include <libpspp/alloc.h>
+#include <stdlib.h>
+
+#include "regression-export.h"
 #include <data/case.h>
 #include <data/casefile.h>
-#include <data/category.h>
 #include <data/cat-routines.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
-#include <math/design-matrix.h>
+#include <data/category.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/missing-values.h>
+#include <data/transformations.h>
+#include <data/value-labels.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/data-io/file-handle.h>
-#include "gettext.h"
 #include <language/lexer/lexer.h>
-#include <math/linreg/linreg.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/message.h>
+#include <math/design-matrix.h>
 #include <math/linreg/coefficient.h>
-#include <data/missing-values.h>
-#include "regression-export.h"
+#include <math/linreg/linreg.h>
 #include <output/table.h>
-#include <data/value-labels.h>
-#include <data/variable.h>
 #include <procedure.h>
 
+#include "gettext.h"
+
 #define REG_LARGE_DATA 1000
 
 /* (headers) */
diff --git a/src/language/stats/sort-cases.c b/src/language/stats/sort-cases.c
index dd038741..156c5b42 100644
--- a/src/language/stats/sort-cases.c
+++ b/src/language/stats/sort-cases.c
@@ -18,18 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <sys/types.h>
+
 #include <assert.h>
 #include <stdlib.h>
 #include <limits.h>
-#include <libpspp/alloc.h>
+
+#include "sort-criteria.h"
+#include <procedure.h>
+#include <data/settings.h>
+#include <data/variable.h>
 #include <language/command.h>
-#include <libpspp/message.h>
 #include <language/lexer/lexer.h>
-#include <data/settings.h>
-#include "sort-criteria.h"
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <math/sort.h>
-#include <data/variable.h>
+#include <sys/types.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/utilities/set.q b/src/language/utilities/set.q
index 55913dc0..2ad52b77 100644
--- a/src/language/utilities/set.q
+++ b/src/language/utilities/set.q
@@ -18,27 +18,27 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <data/settings.h>
-#include <libpspp/message.h>
+
 #include <stdio.h>
 #include <errno.h>
 #include <stdlib.h>
 #include <time.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
+
 #include <data/dictionary.h>
-#include <language/line-buffer.h>
+#include <data/format.h>
+#include <procedure.h>
+#include <data/settings.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
-#include <libpspp/message.h>
+#include <language/line-buffer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
+#include <libpspp/copyleft.h>
 #include <libpspp/magic.h>
-#include <output/output.h>
+#include <libpspp/message.h>
 #include <math/random.h>
-#include <data/variable.h>
-#include <data/format.h>
-#include <libpspp/copyleft.h>
-#include <data/variable.h>
-
+#include <output/output.h>
 
 #if HAVE_LIBTERMCAP
 #if HAVE_TERMCAP_H
diff --git a/src/language/xforms/compute.c b/src/language/xforms/compute.c
index e6a17553..0a399285 100644
--- a/src/language/xforms/compute.c
+++ b/src/language/xforms/compute.c
@@ -18,18 +18,22 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
-#include <language/command.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/misc.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/xforms/count.c b/src/language/xforms/count.c
index e3f8f358..a8a208b3 100644
--- a/src/language/xforms/count.c
+++ b/src/language/xforms/count.c
@@ -18,19 +18,23 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
+#include <data/dictionary.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <language/lexer/range-parser.h>
+#include <libpspp/alloc.h>
 #include <libpspp/compiler.h>
-#include <data/dictionary.h>
 #include <libpspp/message.h>
-#include <language/lexer/lexer.h>
+#include <libpspp/message.h>
 #include <libpspp/pool.h>
-#include <language/lexer/range-parser.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/xforms/recode.c b/src/language/xforms/recode.c
index b89071a3..c9cb2039 100644
--- a/src/language/xforms/recode.c
+++ b/src/language/xforms/recode.c
@@ -18,23 +18,27 @@
    02110-1301, USA. */
 
 #include <config.h>
-#include <libpspp/message.h>
+
 #include <ctype.h>
 #include <math.h>
 #include <stdlib.h>
-#include <libpspp/alloc.h>
+
 #include <data/case.h>
-#include <language/command.h>
-#include <libpspp/compiler.h>
 #include <data/data-in.h>
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <procedure.h>
+#include <data/transformations.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/lexer/lexer.h>
+#include <language/lexer/range-parser.h>
+#include <libpspp/alloc.h>
+#include <libpspp/compiler.h>
 #include <libpspp/magic.h>
+#include <libpspp/message.h>
+#include <libpspp/message.h>
 #include <libpspp/pool.h>
-#include <language/lexer/range-parser.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/xforms/sample.c b/src/language/xforms/sample.c
index fd5bd5ec..830c981e 100644
--- a/src/language/xforms/sample.c
+++ b/src/language/xforms/sample.c
@@ -18,18 +18,21 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <gsl/gsl_rng.h>
 #include <limits.h>
 #include <stdio.h>
 #include <math.h>
-#include <libpspp/alloc.h>
+
+#include <data/variable.h>
 #include <language/command.h>
+#include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
 #include <libpspp/compiler.h>
 #include <libpspp/message.h>
-#include <language/lexer/lexer.h>
-#include <math/random.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
+#include <math/random.h>
+#include <procedure.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/language/xforms/select-if.c b/src/language/xforms/select-if.c
index e83abcdb..f6be43c2 100644
--- a/src/language/xforms/select-if.c
+++ b/src/language/xforms/select-if.c
@@ -18,15 +18,19 @@
    02110-1301, USA. */
 
 #include <config.h>
+
 #include <stdlib.h>
-#include <libpspp/alloc.h>
-#include <language/command.h>
+
 #include <data/dictionary.h>
-#include <libpspp/message.h>
+#include <data/transformations.h>
+#include <procedure.h>
+#include <data/variable.h>
+#include <language/command.h>
 #include <language/expressions/public.h>
 #include <language/lexer/lexer.h>
+#include <libpspp/alloc.h>
+#include <libpspp/message.h>
 #include <libpspp/str.h>
-#include <data/variable.h>
 
 #include "gettext.h"
 #define _(msgid) gettext (msgid)
diff --git a/src/math/ChangeLog b/src/math/ChangeLog
index 0c834cb0..c78f51a8 100644
--- a/src/math/ChangeLog
+++ b/src/math/ChangeLog
@@ -1,3 +1,21 @@
+Wed May  3 23:06:43 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* sort.c: (prepare_to_sort_active_file) Don't run a procedure
+	here.
+	(sort_active_file_in_place) Rewrite to run a procedure, capture
+	the output, sort the output, and set that as the source for the
+	next procedure.
+	(struct sort_to_casefile_cb_data) New structure.
+	(sort_to_casefile_callback) New function.
+	(sort_active_file_to_casefile) Rewrite to use
+	multipass_procedure().
+
 Sat Apr 29 11:09:33 WST 2006 John Darrington <john@darrington.wattle.id.au>
 
 	* removed unused variable.
diff --git a/src/math/sort.c b/src/math/sort.c
index 9725da80..8ce95538 100644
--- a/src/math/sort.c
+++ b/src/math/sort.c
@@ -57,24 +57,13 @@ static struct casefile *do_internal_sort (struct casereader *,
 static struct casefile *do_external_sort (struct casereader *,
                                           const struct sort_criteria *);
 
-/* Gets ready to sort the active file, either in-place or to a
-   separate casefile. */
-static bool
+/* Get ready to sort the active file. */
+static void
 prepare_to_sort_active_file (void) 
 {
-  bool ok;
-  
-  /* Cancel temporary transformations and PROCESS IF. */
-  if (temporary != 0)
-    cancel_temporary (); 
+  proc_cancel_temporary_transformations (); 
   expr_free (process_if_expr);
   process_if_expr = NULL;
-
-  /* Make sure source cases are in a storage source. */
-  ok = procedure (NULL, NULL);
-  assert (case_source_is_class (vfm_source, &storage_source_class));
-
-  return ok;
 }
 
 /* Sorts the active file in-place according to CRITERIA.
@@ -82,21 +71,35 @@ prepare_to_sort_active_file (void)
 int
 sort_active_file_in_place (const struct sort_criteria *criteria) 
 {
-  struct casefile *src, *dst;
+  struct casefile *in, *out;
+
+  prepare_to_sort_active_file ();
+  if (!procedure (NULL, NULL))
+    return 0;
   
-  if (!prepare_to_sort_active_file ())
+  in = proc_capture_output ();
+  out = sort_execute (casefile_get_destructive_reader (in), criteria);
+  if (out == NULL) 
     return 0;
 
-  src = storage_source_get_casefile (vfm_source);
-  dst = sort_execute (casefile_get_destructive_reader (src), criteria);
-  free_case_source (vfm_source);
-  vfm_source = NULL;
+  proc_set_source (storage_source_create (out));
+  return 1;
+}
 
-  if (dst == NULL) 
-    return 0;
+/* Data passed to sort_to_casefile_callback(). */
+struct sort_to_casefile_cb_data 
+  {
+    const struct sort_criteria *criteria;
+    struct casefile *output;
+  };
 
-  vfm_source = storage_source_create (dst);
-  return 1;
+/* Sorts casefile CF according to the criteria in CB_DATA. */
+static bool
+sort_to_casefile_callback (const struct casefile *cf, void *cb_data_) 
+{
+  struct sort_to_casefile_cb_data *cb_data = cb_data_;
+  cb_data->output = sort_execute (casefile_get_reader (cf), cb_data->criteria);
+  return cb_data->output != NULL;
 }
 
 /* Sorts the active file to a separate casefile.  If successful,
@@ -105,13 +108,15 @@ sort_active_file_in_place (const struct sort_criteria *criteria)
 struct casefile *
 sort_active_file_to_casefile (const struct sort_criteria *criteria) 
 {
-  struct casefile *src;
+  struct sort_to_casefile_cb_data cb_data;
   
-  if (!prepare_to_sort_active_file ())
-    return NULL;
+  prepare_to_sort_active_file ();
+
+  cb_data.criteria = criteria;
+  cb_data.output = NULL;
+  multipass_procedure (sort_to_casefile_callback, &cb_data);
 
-  src = storage_source_get_casefile (vfm_source);
-  return sort_execute (casefile_get_reader (src), criteria);
+  return cb_data.output;
 }
 
 
diff --git a/src/procedure.c b/src/procedure.c
index 67f22c5c..9b94da01 100644
--- a/src/procedure.c
+++ b/src/procedure.c
@@ -35,9 +35,9 @@
 #include <data/file-handle-def.h>
 #include <data/settings.h>
 #include <data/storage-stream.h>
+#include <data/transformations.h>
 #include <data/value-labels.h>
 #include <data/variable.h>
-#include <language/control/control-stack.h>
 #include <libpspp/alloc.h>
 #include <libpspp/message.h>
 #include <libpspp/misc.h>
@@ -71,32 +71,40 @@ struct write_case_data
     size_t cases_analyzed;      /* Cases passed to procedure so far. */
   };
 
-/* The current active file, from which cases are read. */
-struct case_source *vfm_source;
-
-/* The replacement active file, to which cases are written. */
-struct case_sink *vfm_sink;
-
-/* The compactor used to compact a compact, if necessary;
+/* Cases are read from vfm_source,
+   pass through permanent_trns_chain (which transforms them into
+   the format described by permanent_dict),
+   are written to vfm_sink,
+   pass through temporary_trns_chain (which transforms them into
+   the format described by default_dict),
+   and are finally passed to the procedure. */
+static struct case_source *vfm_source;
+static struct trns_chain *permanent_trns_chain;
+static struct dictionary *permanent_dict;
+static struct case_sink *vfm_sink;
+static struct trns_chain *temporary_trns_chain;
+struct dictionary *default_dict;
+
+/* The transformation chain that the next transformation will be
+   added to. */
+static struct trns_chain *cur_trns_chain;
+
+/* The compactor used to compact a case, if necessary;
    otherwise a null pointer. */
 static struct dict_compactor *compactor;
 
 /* Time at which vfm was last invoked. */
 static time_t last_vfm_invocation;
 
-/* Whether we're inside a procedure.
-   For debugging purposes only. */
-static bool in_procedure;
-
 /* Lag queue. */
 int n_lag;			/* Number of cases to lag. */
 static int lag_count;		/* Number of cases in lag_queue so far. */
 static int lag_head;		/* Index where next case will be added. */
 static struct ccase *lag_queue; /* Array of n_lag ccase * elements. */
 
-/* Active transformations. */
-struct transformation *t_trns;
-size_t n_trns, m_trns, f_trns;
+static void add_case_limit_trns (void);
+static void add_filter_trns (void);
+static void add_process_if_trns (void);
 
 static bool internal_procedure (bool (*proc_func) (struct ccase *, void *),
                                 void *aux);
@@ -104,11 +112,6 @@ static void update_last_vfm_invocation (void);
 static void create_trns_case (struct ccase *, struct dictionary *);
 static void open_active_file (void);
 static bool write_case (struct write_case_data *wc_data);
-static int execute_transformations (struct ccase *c,
-                                    struct transformation *trns,
-                                    int first_idx, int last_idx,
-                                    int case_num);
-static int filter_case (const struct ccase *c, int case_num);
 static void lag_case (const struct ccase *c);
 static void clear_case (struct ccase *c);
 static bool close_active_file (void);
@@ -126,26 +129,17 @@ time_of_last_procedure (void)
 
 /* Reads the data from the input program and writes it to a new
    active file.  For each case we read from the input program, we
-   do the following
+   do the following:
 
    1. Execute permanent transformations.  If these drop the case,
       start the next case from step 1.
 
-   2. N OF CASES.  If we have already written N cases, start the
-      next case from step 1.
-   
-   3. Write case to replacement active file.
+   2. Write case to replacement active file.
    
-   4. Execute temporary transformations.  If these drop the case,
+   3. Execute temporary transformations.  If these drop the case,
       start the next case from step 1.
       
-   5. FILTER, PROCESS IF.  If these drop the case, start the next
-      case from step 1.
-   
-   6. Post-TEMPORARY N OF CASES.  If we have already analyzed N
-      cases, start the next case from step 1.
-      
-   7. Pass case to PROC_FUNC, passing AUX as auxiliary data.
+   4. Pass case to PROC_FUNC, passing AUX as auxiliary data.
 
    Returns true if successful, false if an I/O error occurred. */
 bool
@@ -154,10 +148,14 @@ procedure (bool (*proc_func) (struct ccase *, void *), void *aux)
   if (proc_func == NULL
       && case_source_is_class (vfm_source, &storage_source_class)
       && vfm_sink == NULL
-      && !temporary
-      && n_trns == 0)
+      && temporary_trns_chain == NULL
+      && trns_chain_is_empty (permanent_trns_chain))
     {
-      /* Nothing to do. */
+      expr_free (process_if_expr);
+      process_if_expr = NULL;
+      dict_set_case_limit (default_dict, 0);
+      dict_clear_vectors (default_dict);
+
       update_last_vfm_invocation ();
       return true;
     }
@@ -167,8 +165,57 @@ procedure (bool (*proc_func) (struct ccase *, void *), void *aux)
       
       open_active_file ();
       ok = internal_procedure (proc_func, aux);
-      if (!close_active_file ())
-        ok = false;
+      ok = close_active_file () && ok;
+
+      return ok;
+    }
+}
+
+/* Callback function for multipass_procedure(). */
+static bool
+multipass_callback (struct ccase *c, void *cf_) 
+{
+  struct casefile *cf = cf_;
+  return casefile_append (cf, c);
+}
+
+/* Procedure that allows multiple passes over the input data.
+   The entire active file is passed to PROC_FUNC, with the given
+   AUX as auxiliary data, as a unit. */
+bool
+multipass_procedure (bool (*proc_func) (const struct casefile *, void *aux),
+                     void *aux) 
+{
+  if (case_source_is_class (vfm_source, &storage_source_class)
+      && vfm_sink == NULL
+      && temporary_trns_chain == NULL
+      && trns_chain_is_empty (permanent_trns_chain))
+    {
+      proc_func (storage_source_get_casefile (vfm_source), aux);
+
+      expr_free (process_if_expr);
+      process_if_expr = NULL;
+      dict_set_case_limit (default_dict, 0);
+      dict_clear_vectors (default_dict);
+
+      update_last_vfm_invocation ();
+      return true;
+    }
+  else 
+    {
+      struct casefile *cf;
+      bool ok;
+
+      assert (proc_func != NULL);
+
+      cf = casefile_create (dict_get_next_value_idx (default_dict));
+
+      open_active_file ();
+      ok = internal_procedure (multipass_callback, cf);
+      ok = proc_func (cf, aux) && ok;
+      ok = close_active_file () && ok;
+
+      casefile_destroy (cf);
 
       return ok;
     }
@@ -237,25 +284,26 @@ create_trns_case (struct ccase *trns_case, struct dictionary *dict)
 static void
 open_active_file (void)
 {
-  assert (!in_procedure);
-  in_procedure = true;
+  add_case_limit_trns ();
+  add_filter_trns ();
+  add_process_if_trns ();
 
-  /* Make temp_dict refer to the dictionary right before data
-     reaches the sink */
-  if (!temporary)
-    {
-      temp_trns = n_trns;
-      temp_dict = default_dict;
-    }
+  /* Finalize transformations. */
+  trns_chain_finalize (cur_trns_chain);
+
+  /* Make permanent_dict refer to the dictionary right before
+     data reaches the sink. */
+  if (permanent_dict == NULL)
+    permanent_dict = default_dict;
 
   /* Figure out compaction. */
-  compactor = (dict_needs_compaction (temp_dict)
-               ? dict_make_compactor (temp_dict)
+  compactor = (dict_needs_compaction (permanent_dict)
+               ? dict_make_compactor (permanent_dict)
                : NULL);
 
   /* Prepare sink. */
   if (vfm_sink == NULL)
-    vfm_sink = create_case_sink (&storage_sink_class, temp_dict, NULL);
+    vfm_sink = create_case_sink (&storage_sink_class, permanent_dict, NULL);
   if (vfm_sink->class->open != NULL)
     vfm_sink->class->open (vfm_sink);
 
@@ -270,9 +318,6 @@ open_active_file (void)
       for (i = 0; i < n_lag; i++)
         case_nullify (&lag_queue[i]);
     }
-
-  /* Close any unclosed DO IF or LOOP constructs. */
-  ctl_stack_clear ();
 }
 
 /* Transforms trns_case and writes it to the replacement active
@@ -282,25 +327,22 @@ open_active_file (void)
 static bool
 write_case (struct write_case_data *wc_data)
 {
-  int retval;
+  enum trns_result retval;
+  size_t case_nr;
   
   /* Execute permanent transformations.  */
-  retval = execute_transformations (&wc_data->trns_case, t_trns, f_trns,
-                                    temp_trns, wc_data->cases_written + 1);
-  if (retval != 1)
+  case_nr = wc_data->cases_written + 1;
+  retval = trns_chain_execute (permanent_trns_chain,
+                               &wc_data->trns_case, &case_nr);
+  if (retval != TRNS_CONTINUE)
     goto done;
 
-  /* N OF CASES. */
-  if (dict_get_case_limit (default_dict)
-      && wc_data->cases_written >= dict_get_case_limit (default_dict))
-    goto done;
-  wc_data->cases_written++;
-
   /* Write case to LAG queue. */
   if (n_lag)
     lag_case (&wc_data->trns_case);
 
   /* Write case to replacement active file. */
+  wc_data->cases_written++;
   if (vfm_sink->class->write != NULL) 
     {
       if (compactor != NULL) 
@@ -314,94 +356,24 @@ write_case (struct write_case_data *wc_data)
     }
   
   /* Execute temporary transformations. */
-  retval = execute_transformations (&wc_data->trns_case, t_trns, temp_trns,
-                                    n_trns, wc_data->cases_written);
-  if (retval != 1)
-    goto done;
-  
-  /* FILTER, PROCESS IF, post-TEMPORARY N OF CASES. */
-  if (filter_case (&wc_data->trns_case, wc_data->cases_written)
-      || (dict_get_case_limit (temp_dict)
-          && wc_data->cases_analyzed >= dict_get_case_limit (temp_dict)))
-    goto done;
-  wc_data->cases_analyzed++;
+  if (temporary_trns_chain != NULL) 
+    {
+      retval = trns_chain_execute (temporary_trns_chain,
+                                   &wc_data->trns_case,
+                                   &wc_data->cases_written);
+      if (retval != TRNS_CONTINUE)
+        goto done;
+    }
 
   /* Pass case to procedure. */
+  wc_data->cases_analyzed++;
   if (wc_data->proc_func != NULL)
     if (!wc_data->proc_func (&wc_data->trns_case, wc_data->aux))
-      retval = -1;
+      retval = TRNS_ERROR;
 
  done:
   clear_case (&wc_data->trns_case);
-  return retval != -1;
-}
-
-/* Transforms case C using the transformations in TRNS[] with
-   indexes FIRST_IDX through LAST_IDX, exclusive.  Case C will
-   become case CASE_NUM (1-based) in the output file.  Returns 1
-   if the case was successfully transformed, 0 if it was filtered
-   out by one of the transformations, or -1 if the procedure
-   should be abandoned due to a fatal error. */
-static int
-execute_transformations (struct ccase *c,
-                         struct transformation *trns,
-                         int first_idx, int last_idx,
-                         int case_num) 
-{
-  int idx;
-
-  for (idx = first_idx; idx != last_idx; )
-    {
-      struct transformation *t = &trns[idx];
-      int retval = t->proc (t->private, c, case_num);
-      switch (retval)
-        {
-        case TRNS_CONTINUE:
-          idx++;
-          break;
-          
-        case TRNS_DROP_CASE:
-          return 0;
-
-        case TRNS_ERROR:
-          return -1;
-
-        case TRNS_NEXT_CASE:
-          abort ();
-
-        case TRNS_END_FILE:
-          abort ();
-          
-        default:
-          idx = retval;
-          break;
-        }
-    }
-
-  return 1;
-}
-
-/* Returns nonzero if case C with case number CASE_NUM should be
-   excluded as specified on FILTER or PROCESS IF, otherwise
-   zero. */
-static int
-filter_case (const struct ccase *c, int case_idx)
-{
-  /* FILTER. */
-  struct variable *filter_var = dict_get_filter (default_dict);
-  if (filter_var != NULL) 
-    {
-      double f = case_num (c, filter_var->fv);
-      if (f == 0.0 || mv_is_num_missing (&filter_var->miss, f))
-        return 1;
-    }
-
-  /* PROCESS IF. */
-  if (process_if_expr != NULL
-      && expr_evaluate_num (process_if_expr, c, case_idx) != 1.0)
-    return 1;
-
-  return 0;
+  return retval != TRNS_ERROR;
 }
 
 /* Add C to the lag queue. */
@@ -452,13 +424,8 @@ close_active_file (void)
       n_lag = 0;
     }
   
-  /* Dictionary from before TEMPORARY becomes permanent.. */
-  if (temporary)
-    {
-      dict_destroy (default_dict);
-      default_dict = temp_dict;
-      temp_dict = NULL;
-    }
+  /* Dictionary from before TEMPORARY becomes permanent. */
+  proc_cancel_temporary_transformations ();
 
   /* Finish compaction. */
   if (compactor != NULL) 
@@ -479,16 +446,9 @@ close_active_file (void)
 
   /* Cancel TEMPORARY, PROCESS IF, FILTER, N OF CASES, vectors,
      and get rid of all the transformations. */
-  cancel_temporary ();
-  expr_free (process_if_expr);
-  process_if_expr = NULL;
-  dict_set_case_limit (default_dict, 0);
   dict_clear_vectors (default_dict);
-
-  assert (in_procedure);
-  in_procedure = false;
-
-  return cancel_transformations ();
+  permanent_dict = NULL;
+  return proc_cancel_all_transformations ();
 }
 
 /* Returns a pointer to the lagged case from N_BEFORE cases before the
@@ -509,56 +469,6 @@ lagged_case (int n_before)
   else
     return NULL;
 }
-   
-/* Appends TRNS to t_trns[], the list of all transformations to be
-   performed on data as it is read from the active file. */
-void
-add_transformation (trns_proc_func *proc, trns_free_func *free, void *private)
-{
-  struct transformation *trns;
-
-  assert (!in_procedure);
-
-  if (n_trns >= m_trns)
-    t_trns = x2nrealloc (t_trns, &m_trns, sizeof *t_trns);
-  trns = &t_trns[n_trns++];
-  trns->proc = proc;
-  trns->free = free;
-  trns->private = private;
-}
-
-/* Returns the index number that the next transformation added by
-   add_transformation() will receive.  A trns_proc_func that
-   returns this index causes control flow to jump to it. */
-size_t
-next_transformation (void) 
-{
-  return n_trns;
-}
-
-/* Cancels all active transformations, including any transformations
-   created by the input program.
-   Returns true if successful, false if an I/O error occurred. */
-bool
-cancel_transformations (void)
-{
-  bool ok = true;
-  size_t i;
-  for (i = 0; i < n_trns; i++)
-    {
-      struct transformation *t = &t_trns[i];
-      if (t->free != NULL) 
-        {
-          if (!t->free (t->private))
-            ok = false; 
-        }
-    }
-  n_trns = f_trns = 0;
-  free (t_trns);
-  t_trns = NULL;
-  m_trns = 0;
-  return ok;
-}
 
 /* Represents auxiliary data for handling SPLIT FILE. */
 struct split_aux_data 
@@ -714,7 +624,7 @@ struct multipass_split_aux_data
   };
 
 static bool multipass_split_callback (struct ccase *c, void *aux_);
-static void multipass_split_output (struct multipass_split_aux_data *);
+static bool multipass_split_output (struct multipass_split_aux_data *);
 
 /* Returns true if successful, false if an I/O error occurred. */
 bool
@@ -736,7 +646,7 @@ multipass_procedure_with_splits (bool (*split_func) (const struct casefile *,
 
   ok = internal_procedure (multipass_split_callback, &aux);
   if (aux.casefile != NULL)
-    multipass_split_output (&aux);
+    ok = multipass_split_output (&aux) && ok;
   case_destroy (&aux.prev_case);
 
   if (!close_active_file ())
@@ -750,13 +660,14 @@ static bool
 multipass_split_callback (struct ccase *c, void *aux_)
 {
   struct multipass_split_aux_data *aux = aux_;
+  bool ok = true;
 
   /* Start a new series if needed. */
   if (aux->casefile == NULL || !equal_splits (c, &aux->prev_case))
     {
       /* Pass any cases to split_func. */
       if (aux->casefile != NULL)
-        multipass_split_output (aux);
+        ok = multipass_split_output (aux);
 
       /* Start a new casefile. */
       aux->casefile = casefile_create (dict_get_next_value_idx (default_dict));
@@ -767,18 +678,21 @@ multipass_split_callback (struct ccase *c, void *aux_)
       case_clone (&aux->prev_case, c);
     }
 
-  return casefile_append (aux->casefile, c);
+  return casefile_append (aux->casefile, c) && ok;
 }
 
-static void
+static bool
 multipass_split_output (struct multipass_split_aux_data *aux)
 {
+  bool ok;
+  
   assert (aux->casefile != NULL);
-  aux->split_func (aux->casefile, aux->func_aux);
+  ok = aux->split_func (aux->casefile, aux->func_aux);
   casefile_destroy (aux->casefile);
   aux->casefile = NULL;
-}
 
+  return ok;
+}
 
 /* Discards all the current state in preparation for a data-input
    command like DATA LIST or GET. */
@@ -796,12 +710,307 @@ discard_variables (void)
       vfm_source = NULL;
     }
 
-  cancel_transformations ();
-
-  ctl_stack_clear ();
+  proc_cancel_all_transformations ();
 
   expr_free (process_if_expr);
   process_if_expr = NULL;
 
-  cancel_temporary ();
+  proc_cancel_temporary_transformations ();
+}
+
+/* Returns the current set of permanent transformations,
+   and clears the permanent transformations.
+   For use by INPUT PROGRAM. */
+struct trns_chain *
+proc_capture_transformations (void) 
+{
+  struct trns_chain *chain;
+  
+  assert (temporary_trns_chain == NULL);
+  chain = permanent_trns_chain;
+  cur_trns_chain = permanent_trns_chain = trns_chain_create ();
+  return chain;
+}
+
+/* Adds a transformation that processes a case with PROC and
+   frees itself with FREE to the current set of transformations.
+   The functions are passed AUX as auxiliary data. */
+void
+add_transformation (trns_proc_func *proc, trns_free_func *free, void *aux)
+{
+  trns_chain_append (cur_trns_chain, NULL, proc, free, aux);
+}
+
+/* Adds a transformation that processes a case with PROC and
+   frees itself with FREE to the current set of transformations.
+   When parsing of the block of transformations is complete,
+   FINALIZE will be called.
+   The functions are passed AUX as auxiliary data. */
+void
+add_transformation_with_finalizer (trns_finalize_func *finalize,
+                                   trns_proc_func *proc,
+                                   trns_free_func *free, void *aux)
+{
+  trns_chain_append (cur_trns_chain, finalize, proc, free, aux);
+}
+
+/* Returns the index of the next transformation.
+   This value can be returned by a transformation procedure
+   function to indicate a "jump" to that transformation. */
+size_t
+next_transformation (void) 
+{
+  return trns_chain_next (cur_trns_chain);
+}
+
+/* Returns true if the next call to add_transformation() will add
+   a temporary transformation, false if it will add a permanent
+   transformation. */
+bool
+proc_in_temporary_transformations (void) 
+{
+  return temporary_trns_chain != NULL;
+}
+
+/* Marks the start of temporary transformations.
+   Further calls to add_transformation() will add temporary
+   transformations. */
+void
+proc_start_temporary_transformations (void) 
+{
+  if (!proc_in_temporary_transformations ())
+    {
+      add_case_limit_trns ();
+
+      permanent_dict = dict_clone (default_dict);
+      trns_chain_finalize (permanent_trns_chain);
+      temporary_trns_chain = cur_trns_chain = trns_chain_create ();
+    }
+}
+
+/* Converts all the temporary transformations, if any, to
+   permanent transformations.  Further transformations will be
+   permanent.
+   Returns true if anything changed, false otherwise. */
+bool
+proc_make_temporary_transformations_permanent (void) 
+{
+  if (proc_in_temporary_transformations ()) 
+    {
+      trns_chain_finalize (temporary_trns_chain);
+      trns_chain_splice (permanent_trns_chain, temporary_trns_chain);
+      temporary_trns_chain = NULL;
+
+      dict_destroy (permanent_dict);
+      permanent_dict = NULL;
+
+      return true;
+    }
+  else
+    return false;
+}
+
+/* Cancels all temporary transformations, if any.  Further
+   transformations will be permanent.
+   Returns true if anything changed, false otherwise. */
+bool
+proc_cancel_temporary_transformations (void) 
+{
+  if (proc_in_temporary_transformations ()) 
+    {
+      dict_destroy (default_dict);
+      default_dict = permanent_dict;
+      permanent_dict = NULL;
+
+      trns_chain_destroy (temporary_trns_chain);
+      temporary_trns_chain = NULL;
+
+      return true;
+    }
+  else
+    return false;
+}
+
+/* Cancels all transformations, if any.
+   Returns true if successful, false on I/O error. */
+bool
+proc_cancel_all_transformations (void)
+{
+  bool ok;
+  ok = trns_chain_destroy (permanent_trns_chain);
+  ok = trns_chain_destroy (temporary_trns_chain) && ok;
+  permanent_trns_chain = cur_trns_chain = trns_chain_create ();
+  temporary_trns_chain = NULL;
+  return ok;
+}
+
+/* Initializes procedure handling. */
+void
+proc_init (void) 
+{
+  default_dict = dict_create ();
+  proc_cancel_all_transformations ();
+}
+
+/* Finishes up procedure handling. */
+void
+proc_done (void)
+{
+  discard_variables ();
+}
+
+/* Sets SINK as the destination for procedure output from the
+   next procedure. */
+void
+proc_set_sink (struct case_sink *sink) 
+{
+  assert (vfm_sink == NULL);
+  vfm_sink = sink;
+}
+
+/* Sets SOURCE as the source for procedure input for the next
+   procedure. */
+void
+proc_set_source (struct case_source *source) 
+{
+  assert (vfm_source == NULL);
+  vfm_source = source;
+}
+
+/* Returns true if a source for the next procedure has been
+   configured, false otherwise. */
+bool
+proc_has_source (void) 
+{
+  return vfm_source != NULL;
+}
+
+/* Returns the output from the previous procedure.
+   For use only immediately after executing a procedure.
+   The returned casefile is owned by the caller; it will not be
+   automatically used for the next procedure's input. */
+struct casefile *
+proc_capture_output (void) 
+{
+  struct casefile *casefile;
+
+  /* Try to make sure that this function is called immediately
+     after procedure() or a similar function. */
+  assert (vfm_source != NULL);
+  assert (case_source_is_class (vfm_source, &storage_source_class));
+  assert (trns_chain_is_empty (permanent_trns_chain));
+  assert (!proc_in_temporary_transformations ());
+
+  casefile = storage_source_decapsulate (vfm_source);
+  vfm_source = NULL;
+
+  return casefile;
+}
+
+static trns_proc_func case_limit_trns_proc;
+static trns_free_func case_limit_trns_free;
+
+/* Adds a transformation that limits the number of cases that may
+   pass through, if default_dict has a case limit. */
+static void
+add_case_limit_trns (void) 
+{
+  size_t case_limit = dict_get_case_limit (default_dict);
+  if (case_limit != 0)
+    {
+      size_t *cases_remaining = xmalloc (sizeof *cases_remaining);
+      *cases_remaining = case_limit;
+      add_transformation (case_limit_trns_proc, case_limit_trns_free,
+                          cases_remaining);
+      dict_set_case_limit (default_dict, 0);
+    }
+}
+
+/* Limits the maximum number of cases processed to
+   *CASES_REMAINING. */
+static int
+case_limit_trns_proc (void *cases_remaining_,
+                      struct ccase *c UNUSED, int case_nr UNUSED) 
+{
+  size_t *cases_remaining = cases_remaining_;
+  if (*cases_remaining > 0) 
+    {
+      *cases_remaining--;
+      return TRNS_CONTINUE;
+    }
+  else
+    return TRNS_DROP_CASE;
+}
+
+/* Frees the data associated with a case limit transformation. */
+static bool
+case_limit_trns_free (void *cases_remaining_) 
+{
+  size_t *cases_remaining = cases_remaining_;
+  free (cases_remaining);
+  return true;
+}
+
+static trns_proc_func filter_trns_proc;
+
+/* Adds a temporary transformation to filter data according to
+   the variable specified on FILTER, if any. */
+static void
+add_filter_trns (void) 
+{
+  struct variable *filter_var = dict_get_filter (default_dict);
+  if (filter_var != NULL) 
+    {
+      proc_start_temporary_transformations ();
+      add_transformation (filter_trns_proc, NULL, filter_var);
+    }
+}
+
+/* FILTER transformation. */
+static int
+filter_trns_proc (void *filter_var_,
+                  struct ccase *c UNUSED, int case_nr UNUSED) 
+  
+{
+  struct variable *filter_var = filter_var_;
+  double f = case_num (c, filter_var->fv);
+  return (f != 0.0 && !mv_is_num_missing (&filter_var->miss, f)
+          ? TRNS_CONTINUE : TRNS_DROP_CASE);
+}
+
+static trns_proc_func process_if_trns_proc;
+static trns_free_func process_if_trns_free;
+
+/* Adds a temporary transformation to filter data according to
+   the expression specified on PROCESS IF, if any. */
+static void
+add_process_if_trns (void) 
+{
+  if (process_if_expr != NULL) 
+    {
+      proc_start_temporary_transformations ();
+      add_transformation (process_if_trns_proc, process_if_trns_free,
+                          process_if_expr);
+      process_if_expr = NULL;
+    }
+}
+
+/* PROCESS IF transformation. */
+static int
+process_if_trns_proc (void *expression_,
+                      struct ccase *c UNUSED, int case_nr UNUSED) 
+  
+{
+  struct expression *expression = expression_;
+  return (expr_evaluate_num (expression, c, case_nr) == 1.0
+          ? TRNS_CONTINUE : TRNS_DROP_CASE);
+}
+
+/* Frees a PROCESS IF transformation. */
+static bool
+process_if_trns_free (void *expression_) 
+{
+  struct expression *expression = expression_;
+  expr_free (expression);
+  return true;
 }
diff --git a/src/procedure.h b/src/procedure.h
index f9f43f49..5a196878 100644
--- a/src/procedure.h
+++ b/src/procedure.h
@@ -22,26 +22,59 @@
 
 #include <time.h>
 #include <stdbool.h>
+#include <data/transformations.h>
 
 struct ccase;
 struct casefile;
+struct case_sink;
+struct case_source;
 
-/* The current active file, from which cases are read. */
-extern struct case_source *vfm_source;
+/* Dictionary produced by permanent and temporary transformations
+   on data from the source. */
+extern struct dictionary *default_dict;
+
+/* Transformations. */
 
-/* The replacement active file, to which cases are written. */
-extern struct case_sink *vfm_sink;
+void add_transformation (trns_proc_func *, trns_free_func *, void *);
+void add_transformation_with_finalizer (trns_finalize_func *,
+                                        trns_proc_func *,
+                                        trns_free_func *, void *);
+size_t next_transformation (void);
+
+void discard_variables (void);
+
+bool proc_cancel_all_transformations (void);
+struct trns_chain *proc_capture_transformations (void);
+
+void proc_start_temporary_transformations (void);
+bool proc_in_temporary_transformations (void);
+bool proc_make_temporary_transformations_permanent (void);
+bool proc_cancel_temporary_transformations (void);
+
+/* Procedures. */
+
+void proc_init (void);
+void proc_done (void);
+
+void proc_set_source (struct case_source *);
+bool proc_has_source (void);
+
+void proc_set_sink (struct case_sink *);
+struct casefile *proc_capture_output (void);
 
 bool procedure (bool (*proc_func) (struct ccase *, void *aux), void *aux);
 bool procedure_with_splits (void (*begin_func) (void *aux),
                             bool (*proc_func) (struct ccase *, void *aux),
                             void (*end_func) (void *aux),
                             void *aux);
+bool multipass_procedure (bool (*proc_func) (const struct casefile *,
+                                             void *aux),
+                          void *aux);
 bool multipass_procedure_with_splits (bool (*) (const struct casefile *,
                                                 void *),
                                       void *aux);
 time_t time_of_last_procedure (void);
-
+
 /* Number of cases to lag. */
 extern int n_lag;
 
diff --git a/src/ui/terminal/ChangeLog b/src/ui/terminal/ChangeLog
index 1d081010..330d94ab 100644
--- a/src/ui/terminal/ChangeLog
+++ b/src/ui/terminal/ChangeLog
@@ -1,3 +1,14 @@
+Wed May  3 23:09:37 2006  Ben Pfaff  <blp@gnu.org>
+
+	Continue reforming procedure execution.  In this phase, get rid of
+	many global variables, consolidating procedure execution in
+	procedure.c.  Encapsulate transformations in new "struct
+	trns_chain".  Also, change implementation of N OF CASES, FILTER,
+	and PROCESS IF from special cases to transformations.
+	 
+	* main.c: (main) Use proc_init().
+	(terminate) Use proc_done().
+
 Wed Apr 26 13:34:54 2006  Ben Pfaff  <blp@gnu.org>
 
 	Improve command name completion in readline.
diff --git a/src/ui/terminal/main.c b/src/ui/terminal/main.c
index 2600a048..7560dcfe 100644
--- a/src/ui/terminal/main.c
+++ b/src/ui/terminal/main.c
@@ -19,6 +19,9 @@
 
 #include <config.h>
 
+#include <signal.h>
+#include <stdio.h>
+
 #include "command-line.h"
 #include "msg-ui.h"
 #include "progname.h"
@@ -39,9 +42,7 @@
 #include <libpspp/version.h>
 #include <math/random.h>
 #include <output/output.h>
-#include <signal.h>
-#include <stdio.h>
-
+#include <procedure.h>
 
 #if HAVE_FPU_CONTROL_H
 #include <fpu_control.h>
@@ -93,8 +94,7 @@ main (int argc, char **argv)
   readln_initialize ();
   settings_init ();
   random_init ();
-
-  default_dict = dict_create ();
+  proc_init ();
 
   if (parse_command_line (argc, argv)) 
     {
@@ -135,7 +135,8 @@ execute_command (void)
      Any lines read after the first token must be continuation
      lines. */
   getl_set_prompt_style (GETL_PROMPT_LATER);
-  return cmd_parse (vfm_source != NULL ? CMD_STATE_DATA : CMD_STATE_INITIAL);
+  return cmd_parse (proc_has_source ()
+                    ? CMD_STATE_DATA : CMD_STATE_INITIAL);
 }
 
 static void
@@ -197,8 +198,7 @@ terminate (bool success)
     {
       terminating = true;
 
-      cancel_transformations ();
-      dict_destroy (default_dict);
+      proc_done ();
 
       random_done ();
       settings_done ();