lexer: Change the functions for retrieving token strings. 111/pspp 112/pspp 113/pspp 114/pspp 115/pspp 116/pspp 120/pspp 121/pspp 122/pspp 20101122030502/pspp 20101123030502/pspp 20101124030502/pspp 20101125030504/pspp 20101126030502/pspp 20101127030503/pspp 20101202030501/pspp 20101203030504/pspp 20101204030504/pspp
authorBen Pfaff <blp@cs.stanford.edu>
Sat, 20 Nov 2010 05:05:04 +0000 (21:05 -0800)
committerBen Pfaff <blp@cs.stanford.edu>
Sat, 20 Nov 2010 05:05:04 +0000 (21:05 -0800)
commit55e6e7ba37a30570f5a31e2d78c22dfa7b61a36f
tree2adc4c52da679e3ad419cf10245270bd55fc32eb
parent740218f508466f16a79939cd97027b99f589d682
lexer: Change the functions for retrieving token strings.

Until now, lex_tokid() has been for T_ID tokens only and lex_tokstr() has
been for T_ID and T_STRING tokens.  For T_ID tokens, lex_tokid() and
lex_tokstr() had slightly different semantics.

This doesn't entirely make sense, and these particular functions are not
the ones wanted most by clients, so this commit removes these functions
in favor of lex_tokcstr() and lex_tokss(), which are both applicable
to both T_ID and T_STRING tokens, with the same semantics in each case.

These functions are also easier for the upcoming reimplementation of the
lexer.
48 files changed:
src/language/command.c
src/language/control/loop.c
src/language/control/repeat.c
src/language/data-io/combine-files.c
src/language/data-io/data-list.c
src/language/data-io/file-handle.q
src/language/data-io/get-data.c
src/language/data-io/print.c
src/language/data-io/save-translate.c
src/language/data-io/trim.c
src/language/dictionary/attributes.c
src/language/dictionary/missing-values.c
src/language/dictionary/modify-variables.c
src/language/dictionary/mrsets.c
src/language/dictionary/value-labels.c
src/language/dictionary/variable-label.c
src/language/dictionary/vector.c
src/language/expressions/evaluate.c
src/language/expressions/parse.c
src/language/lexer/format-parser.c
src/language/lexer/lexer.c
src/language/lexer/lexer.h
src/language/lexer/q2c.c
src/language/lexer/value-parser.c
src/language/lexer/variable-parser.c
src/language/stats/aggregate.c
src/language/stats/crosstabs.q
src/language/stats/descriptives.c
src/language/stats/examine.q
src/language/stats/frequencies.q
src/language/stats/npar.c
src/language/stats/rank.q
src/language/stats/regression.q
src/language/stats/reliability.c
src/language/stats/sort-criteria.c
src/language/tests/float-format.c
src/language/tests/format-guesser-test.c
src/language/tests/paper-size.c
src/language/utilities/cd.c
src/language/utilities/echo.c
src/language/utilities/host.c
src/language/utilities/include.c
src/language/utilities/permissions.c
src/language/utilities/set.q
src/language/utilities/title.c
src/language/xforms/compute.c
src/language/xforms/count.c
src/language/xforms/recode.c