lexer: Change the functions for retrieving token strings.
authorBen Pfaff <blp@cs.stanford.edu>
Sat, 20 Nov 2010 05:05:04 +0000 (21:05 -0800)
committerBen Pfaff <blp@cs.stanford.edu>
Sat, 20 Nov 2010 05:05:04 +0000 (21:05 -0800)
commit55e6e7ba37a30570f5a31e2d78c22dfa7b61a36f
tree2adc4c52da679e3ad419cf10245270bd55fc32eb
parent740218f508466f16a79939cd97027b99f589d682
lexer: Change the functions for retrieving token strings.

Until now, lex_tokid() has been for T_ID tokens only and lex_tokstr() has
been for T_ID and T_STRING tokens.  For T_ID tokens, lex_tokid() and
lex_tokstr() had slightly different semantics.

This doesn't entirely make sense, and these particular functions are not
the ones wanted most by clients, so this commit removes these functions
in favor of lex_tokcstr() and lex_tokss(), which are both applicable
to both T_ID and T_STRING tokens, with the same semantics in each case.

These functions are also easier for the upcoming reimplementation of the
lexer.
48 files changed:
src/language/command.c
src/language/control/loop.c
src/language/control/repeat.c
src/language/data-io/combine-files.c
src/language/data-io/data-list.c
src/language/data-io/file-handle.q
src/language/data-io/get-data.c
src/language/data-io/print.c
src/language/data-io/save-translate.c
src/language/data-io/trim.c
src/language/dictionary/attributes.c
src/language/dictionary/missing-values.c
src/language/dictionary/modify-variables.c
src/language/dictionary/mrsets.c
src/language/dictionary/value-labels.c
src/language/dictionary/variable-label.c
src/language/dictionary/vector.c
src/language/expressions/evaluate.c
src/language/expressions/parse.c
src/language/lexer/format-parser.c
src/language/lexer/lexer.c
src/language/lexer/lexer.h
src/language/lexer/q2c.c
src/language/lexer/value-parser.c
src/language/lexer/variable-parser.c
src/language/stats/aggregate.c
src/language/stats/crosstabs.q
src/language/stats/descriptives.c
src/language/stats/examine.q
src/language/stats/frequencies.q
src/language/stats/npar.c
src/language/stats/rank.q
src/language/stats/regression.q
src/language/stats/reliability.c
src/language/stats/sort-criteria.c
src/language/tests/float-format.c
src/language/tests/format-guesser-test.c
src/language/tests/paper-size.c
src/language/utilities/cd.c
src/language/utilities/echo.c
src/language/utilities/host.c
src/language/utilities/include.c
src/language/utilities/permissions.c
src/language/utilities/set.q
src/language/utilities/title.c
src/language/xforms/compute.c
src/language/xforms/count.c
src/language/xforms/recode.c